From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 01:29:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A59F360; Sun, 23 Feb 2014 01:29:18 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD6B11D5; Sun, 23 Feb 2014 01:29:17 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id ec20so3869892lab.23 for ; Sat, 22 Feb 2014 17:29:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GpPINeEJICwqpKXiJHuSWJm5oY3yMaQDzz8N0yTTzsA=; b=USWs2ZeWhRkObeGBTLSHPNi6fcN6y7GWJgGHHfTiWpGEGypZaYvLhbfUTqSmzO7VNb zeQ38wDCBIQEjBUuttKu6RkbZuUnvfTbsJ+IX/E9731fY54RuOmGli3II2tKTsrZ9Sym BA0e6jWawPYgXdft/xutrCYEP68xEPds21pArsDdEGh7WLzKYFriTFaoGFLXuQyjD/jI UsD1sAPExgBXAzZMwadgPlJSqCaAhBfca7fJcEgpGeaGxAtJ7jL0KhNNFDIhLOrR+K9U l49CXUVLKQ2GR26/7jdJT3JzD/S4e7AEYglUI3u8izvogCUcMjdK23lC8gWCzpEzu5BK RNfg== MIME-Version: 1.0 X-Received: by 10.113.3.43 with SMTP id bt11mr7797498lbd.92.1393118954810; Sat, 22 Feb 2014 17:29:14 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sat, 22 Feb 2014 17:29:14 -0800 (PST) In-Reply-To: <524AB5CB.4070909@fastmail.net> References: <521C5EC2.1060901@yandex.ru> <524AB5CB.4070909@fastmail.net> Date: Sat, 22 Feb 2014 17:29:14 -0800 X-Google-Sender-Auth: NKeA_tssP81paP8Bh086KmoKeL0 Message-ID: Subject: Re: devel/jenkins port not starting. Kernel panic in IPv6 multicast code From: Craig Rodrigues To: Bruce Simpson Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, "Andrey V. Elsukov" , Rui Paulo , bms@freebsd.org, Li-Wen Hsu X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 01:29:18 -0000 On Tue, Oct 1, 2013 at 4:45 AM, Bruce Simpson wrote: > Hello, > > > On 29/09/2013 20:42, Rui Paulo wrote: > >> On 29 Aug 2013, at 13:47, Craig Rodrigues wrote: >> >>> Thanks for the analysis. It looks like the mailing list is dropping my >>> attachments, so I uploaded my log files here: >>> >>> http://people.freebsd.org/~rodrigc/jenkins-problem/ >>> >>> I don't understand where to go about fixing the problem. >>> Should we fix the FreeBSD kernel to not panic in this case, by changing >>> the >>> KASSERT >>> to an error? >>> >> Definitely fix FreeBSD... >> > > Somebody (can't remember, too many things on my plate right now) sent a > one liner for the overly paranoid KASSERT involved, so go ahead and > commit... isn't a release about to be branched? > OK, it took a while, but I committed this: http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081233.html -- Craig From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 04:45:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4398734D; Sun, 23 Feb 2014 04:45:25 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A1801693; Sun, 23 Feb 2014 04:45:24 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id hr17so4038894lab.20 for ; Sat, 22 Feb 2014 20:45:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=MY5P7Ign9QqLjAcldl0ptJd2xaivolpOez2/KtHPEmI=; b=o1Q1/qmo8QQNaVGKZDIwOro7ttXhcu4C3HEAIQTAKR/nJVoxHVVh+XTYunSvfySGo4 fdhn09Ozccc2+nFgotTTDFOAXiquCMeLptSUFU9NNaL1xBqc18R1KidCeUp44w3lWajs Bx4mqPw+JOzDwVnb8qPUxh7OXTFm5Dt6kNJqdGyeSuJzGgf9AxGYC9hmOlD0DewZDOFg sxqgnTFNP7FNWdPDTIA+WUqBvm3gdNqokxp7w4bfW1tsrwgjabDXaIgYjYozBOhM6uyS Cqr0NZUrx8ShxCP1lIyD064jMvsO0HeT7TAxsfolWRDKZzXzaleIiivFKIf0J2s6K0rn peJA== MIME-Version: 1.0 X-Received: by 10.152.9.65 with SMTP id x1mr8444750laa.6.1393130722393; Sat, 22 Feb 2014 20:45:22 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sat, 22 Feb 2014 20:45:22 -0800 (PST) Date: Sat, 22 Feb 2014 20:45:22 -0800 X-Google-Sender-Auth: QgatWMUF81WFwF37daHqJKbRKPs Message-ID: Subject: java.net.PlainDatagramSocketImpl.join(Native Method), invalid argument From: Craig Rodrigues To: freebsd-net@freebsd.org, "freebsd-java@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: jenkins-admin@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 04:45:25 -0000 Hi, I can reproduce the following problem pretty easily on FreeBSD 9, 10, CURRENT. (1) Install the devel/jenkins port (2) Run: service jenkins onestart (3) In /var/log/jenkins.log, I see a traceback: WARNING: UDP handling problem java.net.SocketException: Invalid argument at java.net.PlainDatagramSocketImpl.join(Native Method) at java.net.AbstractPlainDatagramSocketImpl.join(AbstractPlainDatagramSocketImpl.java:168) at java.net.MulticastSocket.joinGroup(MulticastSocket.java:300) at hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:76) Feb 22, 2014 5:21:00 PM hudson.WebAppMain$3 run I reported this bug against Jenkins: https://issues.jenkins-ci.org/browse/JENKINS-21727 but now I suspect that this is a FreeBSD bug or implementation with respect to multicast. Can someone help me debug this and isolate the problem? It's been a while since I've debugged Java code. Thanks. -- Craig From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 06:13:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 168F0A8E for ; Sun, 23 Feb 2014 06:13:08 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C58801C4B for ; Sun, 23 Feb 2014 06:13:07 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id i17so7896393qcy.39 for ; Sat, 22 Feb 2014 22:13:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PwiV1J5XFmfeXEbvUT7rgL33B7F9DQ/lA0Iw9hYOihM=; b=rGvHJaGFOq6Vfb0QUwrrP3P0NamRhtGdIWniRvSJjApqjn/mv/VNWKI1Nwb7g1Qtpw jZli++5oD3jg082OjXq9A5iv28zTvhv/fnvAR/b5SclHjDGHdgAeOqcxmBIKEHjtH2kw d17E53pXEgER5bHm2IHnYNz2jzpiXlys1CiN682yXMWAtO3D2cDiq+4+f/fN7vCzhvHG duqNZeJ5XPCsTJHrNZA9Slebp/RP1bu4kMMldkbIUxKJjrHdGKEPzmghjj/T/b7IEVuB z5lmDMSpqRsp0vUKC2T7fTX7ck3XMZW+2qBRj0iF2AtLLIva07rfwJ9oN7fXYU8m4n40 O5fA== MIME-Version: 1.0 X-Received: by 10.140.96.180 with SMTP id k49mr20237754qge.4.1393135987023; Sat, 22 Feb 2014 22:13:07 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Sat, 22 Feb 2014 22:13:06 -0800 (PST) In-Reply-To: <1393109280.11968.3.camel@fr-wks3.corp.novso.com> References: <1391725273.22934.16.camel@fr-wks3.corp.novso.com> <1392417340.10711.5.camel@fr-wks3.corp.novso.com> <52FEA4F5.5010702@yandex.ru> <52FEA5A2.3050106@yandex.ru> <1392641917.21759.1.camel@srv31.corp.novso.com> <53020F0F.9050809@yandex.ru> <1392648453.22762.2.camel@srv31.corp.novso.com> <5304DA80.6010403@yandex.ru> <1393109280.11968.3.camel@fr-wks3.corp.novso.com> Date: Sat, 22 Feb 2014 22:13:06 -0800 X-Google-Sender-Auth: C_pmwbo_poVPlWiC7ZCgssCHQCc Message-ID: Subject: Re: IPsec filtertunnel broken on FreeBSD 10 From: Adrian Chadd To: Nicolas DEFFAYET Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , "Andrey V. Elsukov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 06:13:08 -0000 Hi, Please make sure you file a PR and loop andre@freebsd.org into it too. thanks, -a On 22 February 2014 14:48, Nicolas DEFFAYET wrote: > On Wed, 2014-02-19 at 20:23 +0400, Andrey V. Elsukov wrote: > Hello, > > After very long testing, i have discovered the route cause. > Testing is not easy as there is a dependency between kernel and > userland, it's not like on a Linux where the kernel is not depend of > userland. > Broken userland can generate false-positive result. > > The revision 254519 break the firewall with IPsec. > http://svnweb.freebsd.org/base?view=revision&revision=254519 > > "Move the global M_SKIP_FIREWALL mbuf flags to a protocol layer specific > flag instead. The flag is only used within the IP and IPv6 layer 3 > protocols. > > Because some firewall packages treat IPv4 and IPv6 packets the same the > flag should have the same value for both." > > It seem that some code doesn't have been updated for allow firewall to > work with IPsec. > > > FreeBSD 10.0-CURRENT #0 r254518 (10-userland) > ------------------------------- > # ipfw -a list > 00010 6 326 allow log logamount 10000 tcp from 192.168.0.1 to > 192.168.0.2 dst-port 22 via re0 in > 00020 0 0 allow log logamount 10000 tcp from 192.168.0.2 to > 192.168.0.1 dst-port 22 via re0 out > 65535 5149 411660 allow ip from any to any > => ipfw counters are ok for inbound > > Feb 22 22:16:44 host2 kernel: ipfw: 10 Accept TCP 192.168.0.1:51691 > 192.168.0.2:22 in via re0 > => ipfw see the connection > > 22:18:16.127112 IP 192.168.0.1 > 224.0.0.5: OSPFv2, Hello, length 44 > 22:18:21.042325 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc15a), length 92 > 22:18:21.042548 IP 192.168.0.2 > 192.168.0.1: > ESP(spi=0x00003039,seq=0x6), length 92 > 22:18:21.042785 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc15b), length 84 > 22:18:21.074818 IP 192.168.0.2 > 192.168.0.1: > ESP(spi=0x00003039,seq=0x7), length 116 > 22:18:21.174651 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc15c), length 84 > => traffic is encrypted by IPsec > > > FreeBSD 10.0-CURRENT #0 r254519 (10-userland) > ------------------------------- > # ipfw -a list > 00010 0 0 allow log logamount 10000 tcp from 192.168.0.1 to > 192.168.0.2 dst-port 22 via re0 in > 00020 0 0 allow log logamount 10000 tcp from 192.168.0.2 to > 192.168.0.1 dst-port 22 via re0 out > 65535 3132 249778 allow ip from any to any > => ipfw counters are not increased! > > nothing in log > => ipfw DO NOT see the connection! > > 22:22:05.890386 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc165), length 92 > 22:22:05.890565 IP 192.168.0.2 > 192.168.0.1: > ESP(spi=0x00003039,seq=0x6), length 92 > 22:22:05.890793 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc166), length 84 > 22:22:05.922544 IP 192.168.0.2 > 192.168.0.1: > ESP(spi=0x00003039,seq=0x7), length 116 > 22:22:06.022056 IP 192.168.0.1 > 192.168.0.2: > ESP(spi=0x00003039,seq=0xc167), length 84 > => traffic is encrypted by IPsec > > > Many thanks > > -- > Nicolas DEFFAYET > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 14:53:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E8CDDED for ; Sun, 23 Feb 2014 14:53:15 +0000 (UTC) Received: from nm19-vm1.bullet.mail.ne1.yahoo.com (nm19-vm1.bullet.mail.ne1.yahoo.com [98.138.91.56]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3085E1385 for ; Sun, 23 Feb 2014 14:53:15 +0000 (UTC) Received: from [98.138.100.117] by nm19.bullet.mail.ne1.yahoo.com with NNFMP; 23 Feb 2014 14:53:08 -0000 Received: from [98.138.226.30] by tm108.bullet.mail.ne1.yahoo.com with NNFMP; 23 Feb 2014 14:53:08 -0000 Received: from [127.0.0.1] by smtp201.mail.ne1.yahoo.com with NNFMP; 23 Feb 2014 14:53:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1393167188; bh=OmtGSapt49ZI1Z/eXugm7R8vJPz6n/f77O3vknco2x0=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=zzb/Bmky7FquB2thoRyxGn2S4Gto7j//d9ITKc3+Syz98RUH1BUoITAP2NkMBpCHxXGdMNtLWGYzvybtRF3i3DbXFoI4uoxrnPwzNx61dXFgvp4wdchHYXCDV8HTUxaPwVItWK3OrMbiw28wkSGZIFJ/VlZ2XOie+sXu3uZVDDo= X-Yahoo-Newman-Id: 28095.20872.bm@smtp201.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: p4C0dQkVM1k0q8L.D9aUC3a6Kw6BsmoEr3.joB_4WXG9NuP 6nFsBsZ_v_L4JZR92AZvJIaLSAEC1iDW0QP4buAGIGyrrkXeEYUd_0W8haL6 u7xlZqTZe6zpOV3lSz.40wXQzrNz.Id6IhTaKJnekqSVMQKPuAMw5I7wAA2T U5j6vo3HLoqQmXWLqk2lJqwe0wISTNpMUmQmIYhT5XzEjrZEPzuVJpWZy55i M6Wp0Wf6nOLa_tu2fZb5tDRNKwt_jlhzjRLXfqWQ3QpSRgTd39noGuoqDWv. Ui.oVJO7_MVUsE6D3NbiRCZJ86XVYkFGnqFpbw0yTFyiQHaSy7ZeNQ43LvsN OBL3t.B.VaYBaFDozrxCX3b74_2_tM.HAWmhnHIK41qtycF0wgWxYWS1TMv5 v9DdLS6OrTUY1TVQyiZBuTdZPVQwSfcwKK1dlPNnH6ABN3R2PW3oFZVR3qgf 9Bp9lnqnUIMoGM8yCvaushFjZiUvahdK1fJ.6.uG541UWNc2SE1EUCSdUodA RWrJFcgt8flURwG7kbvFZ5Mgg19fSiLhCHc_6OARr1.ecLn4JkMJ4C2SYuhg PL.vZRYtaWaiU3LeuuPd3qaw7c5sZy5uZphBo8WhfNTrXKgrCbrj.xslM7t4 Y07MkF6NXo6guikwutHvo_gKq5ZH6fgPER6c- X-Yahoo-SMTP: z6dQwTeswBDDR3zJap0nVIllEEY- X-Rocket-Received: from [192.168.1.13] (dddaley@173.106.254.70 with plain [63.250.193.228]) by smtp201.mail.ne1.yahoo.com with SMTP; 23 Feb 2014 06:53:07 -0800 PST Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: java.net.PlainDatagramSocketImpl.join(Native Method), invalid argument From: Dan Daley In-Reply-To: Date: Sun, 23 Feb 2014 08:53:02 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <43F3B96E-0BC7-4BEA-BAAF-CD43E69B1152@yahoo.com> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1499) Cc: freebsd-net@freebsd.org, jenkins-admin@freebsd.org, "freebsd-java@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 14:53:15 -0000 I ran into this issue about a year and a half ago. If this is the same = issue, it had something to do with the way an IPv4 address is bound to = IPv6. =20 I also ran into the issue on some older flavors of linux, though it = seems that the issue didn't happen on newer flavors. Sorry, I don't remember more details now. But, maybe someone else can = shed more light on the IPv4/IPv6 issue with UDP in Java. On Feb 22, 2014, at 10:45 PM, Craig Rodrigues = wrote: > Hi, >=20 > I can reproduce the following problem pretty easily > on FreeBSD 9, 10, CURRENT. >=20 > (1) Install the devel/jenkins port >=20 > (2) Run: >=20 > service jenkins onestart >=20 > (3) In /var/log/jenkins.log, I see a traceback: >=20 > WARNING: UDP handling problem > java.net.SocketException: Invalid argument > at java.net.PlainDatagramSocketImpl.join(Native Method) > at > = java.net.AbstractPlainDatagramSocketImpl.join(AbstractPlainDatagramSocketI= mpl.java:168) > at java.net.MulticastSocket.joinGroup(MulticastSocket.java:300) > at hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:76) > Feb 22, 2014 5:21:00 PM hudson.WebAppMain$3 run >=20 >=20 >=20 > I reported this bug against Jenkins: >=20 > https://issues.jenkins-ci.org/browse/JENKINS-21727 >=20 >=20 > but now I suspect that this is a FreeBSD bug or > implementation with respect to multicast. >=20 >=20 > Can someone help me debug this and isolate the problem? > It's been a while since I've debugged Java code. >=20 > Thanks. > -- > Craig > _______________________________________________ > freebsd-java@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-java > To unsubscribe, send any mail to = "freebsd-java-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 14:59:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 394E5F31 for ; Sun, 23 Feb 2014 14:59:23 +0000 (UTC) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2001:470:1f09:14c0::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D536C13B1 for ; Sun, 23 Feb 2014 14:59:22 +0000 (UTC) Received: from [192.168.248.37] ([192.168.248.37]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id s1NExHMA081848 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 23 Feb 2014 20:59:17 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <530A0CC4.6060807@norma.perm.ru> Date: Sun, 23 Feb 2014 20:59:16 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: bge0: watchdog timeout Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [192.168.3.10]); Sun, 23 Feb 2014 20:59:17 +0600 (YEKT) X-Spam-Status: No hits=-101.0 bayes=0.5 testhits ALL_TRUSTED=-1, USER_IN_WHITELIST=-100 autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 14:59:23 -0000 Hi. Recently I've upgrade two of my FreeBSD servers from 8.x (8.2-STABLE and 8.3-PRERELEASE). On one server, running with hw.bge.allow_asf: 1 dev.bge.0.msi: 1 I got a bunch of # dmesg -a | grep watch Starting watchdogd. bge0: watchdog timeout -- resetting bge0: watchdog timeout -- resetting bge0: watchdog timeout -- resetting bge0: watchdog timeout -- resetting This server has the following chip: bge0@pci0:32:0:0: class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme BCM5723 Gigabit Ethernet PCIe' class = network subclass = ethernet Is this known behaviour ? Are there still any issues with bge(4) ? Another one is running some older version of RC1, r259449 - I've checked the svnweb and saw that last update was at r258830, so it must be a combination of ASF + MSI that does this. Is there anything I can do about this, aside from turning off the MSI/ASF stuff ? Thanks. Eugene. From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 15:09:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9EFE61F6 for ; Sun, 23 Feb 2014 15:09:42 +0000 (UTC) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2001:470:1f09:14c0::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F2D9314E3 for ; Sun, 23 Feb 2014 15:09:41 +0000 (UTC) Received: from [192.168.248.37] ([192.168.248.37]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id s1NF9dvG082112 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 23 Feb 2014 21:09:39 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <530A0F32.3020804@norma.perm.ru> Date: Sun, 23 Feb 2014 21:09:38 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: bge0: watchdog timeout References: <530A0CC4.6060807@norma.perm.ru> In-Reply-To: <530A0CC4.6060807@norma.perm.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [192.168.3.10]); Sun, 23 Feb 2014 21:09:39 +0600 (YEKT) X-Spam-Status: No hits=-101.0 bayes=0.5 testhits ALL_TRUSTED=-1, USER_IN_WHITELIST=-100 autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 15:09:42 -0000 Hi. On 23.02.2014 20:59, Eugene M. Zheganin wrote: > > Recently I've upgrade two of my FreeBSD servers from 8.x (8.2-STABLE > and 8.3-PRERELEASE). > [...] > > Is this known behaviour ? Are there still any issues with bge(4) ? > Another one is running some older version of RC1, r259449 - I've > checked the svnweb and saw that last update was at r258830, so it must > be a combination of ASF + MSI that does this. Is there anything I can > do about this, aside from turning off the MSI/ASF stuff ? Right, I forgot to mention the FreeBSD versions. The one that got bge watchdog timeouts is running FreeBSD 10.0-STABLE r262238 amd64. Another one, that got no timeouts (but is running with AFS/MSI off) is running FreeBSD 10.0-RC2 r259449 amd64. Thanks. Eugene. From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 15:25:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBB9BDCB; Sun, 23 Feb 2014 15:25:45 +0000 (UTC) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5B99B1659; Sun, 23 Feb 2014 15:25:44 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.8/8.14.8/NETPLEX) with ESMTP id s1NFPhKW046631; Sun, 23 Feb 2014 10:25:43 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Sun, 23 Feb 2014 10:25:43 -0500 (EST) Date: Sun, 23 Feb 2014 10:25:43 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Dan Daley Subject: Re: java.net.PlainDatagramSocketImpl.join(Native Method), invalid argument In-Reply-To: <43F3B96E-0BC7-4BEA-BAAF-CD43E69B1152@yahoo.com> Message-ID: References: <43F3B96E-0BC7-4BEA-BAAF-CD43E69B1152@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Craig Rodrigues , freebsd-net@freebsd.org, jenkins-admin@freebsd.org, "freebsd-java@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Daniel Eischen List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 15:25:45 -0000 On Sun, 23 Feb 2014, Dan Daley wrote: > > I ran into this issue about a year and a half ago. If this is the > same issue, it had something to do with the way an IPv4 address is > bound to IPv6. I also ran into the issue on some older flavors of > linux, though it seems that the issue didn't happen on newer flavors. > > Sorry, I don't remember more details now. But, maybe someone else can > shed more light on the IPv4/IPv6 issue with UDP in Java. Sorry, I missed this thread. Craig, did you try forcing IPv4 by setting java.net.preferIPv4Stack=true? You can do it on the run line with -Djava.net.preferIPv4Stack=true or setting it in the code with System.setProperty ("java.net.preferIPv4Stack", "true"). I have to do this for our own applications that use multicast. I think the JRE recognizes that the OS has IPv6 enabled and tries to use that instead of IPv4. > > On Feb 22, 2014, at 10:45 PM, Craig Rodrigues wrote: > >> Hi, >> >> I can reproduce the following problem pretty easily >> on FreeBSD 9, 10, CURRENT. >> >> (1) Install the devel/jenkins port >> >> (2) Run: >> >> service jenkins onestart >> >> (3) In /var/log/jenkins.log, I see a traceback: >> >> WARNING: UDP handling problem >> java.net.SocketException: Invalid argument >> at java.net.PlainDatagramSocketImpl.join(Native Method) >> at >> java.net.AbstractPlainDatagramSocketImpl.join(AbstractPlainDatagramSocketImpl.java:168) >> at java.net.MulticastSocket.joinGroup(MulticastSocket.java:300) >> at hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:76) >> Feb 22, 2014 5:21:00 PM hudson.WebAppMain$3 run >> >> >> >> I reported this bug against Jenkins: >> >> https://issues.jenkins-ci.org/browse/JENKINS-21727 >> >> >> but now I suspect that this is a FreeBSD bug or >> implementation with respect to multicast. >> >> >> Can someone help me debug this and isolate the problem? >> It's been a while since I've debugged Java code. >> >> Thanks. >> -- >> Craig -- DE From owner-freebsd-net@FreeBSD.ORG Sun Feb 23 19:36:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE8DFE6B for ; Sun, 23 Feb 2014 19:36:15 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 38D341D17 for ; Sun, 23 Feb 2014 19:36:15 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id c6so4544477lan.38 for ; Sun, 23 Feb 2014 11:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=FTBC5ln1Vacd/tfpib9ci7V4JHiJLlunwuSCjSZVtmU=; b=YRem3tSO+GYAHneYuIwXFPv5OHn6uzX6rsGyOA3LeREBkI7/qOFibO7t11w3/9tiMb lyTQPlfY3H4XvZJaFf3eUbvHjS7Jh46NbEITPWvG0z0P6Q3VOz76Vmony04/efo87Sm1 upUY6neu2LA+dX3/sK+LbG6Ra/HmQCaHWJInbRgqsX7MjWsXOwjaVgfGVZ+65sacBfKL kPjL5ZkGS3RkB6n5p9nBvj/b5HKspa6VSH2otW0GsszWJy5OouwqBG9Objr2wSLLDWrl hmXVqaQ9k7ibDiUB1FB+74a4twfgPyJHYEVirwAXM5lH5Gq5uhHIXpDSnWjmHGjTqGCz 233g== MIME-Version: 1.0 X-Received: by 10.112.151.146 with SMTP id uq18mr9551546lbb.38.1393184173187; Sun, 23 Feb 2014 11:36:13 -0800 (PST) Received: by 10.112.219.233 with HTTP; Sun, 23 Feb 2014 11:36:13 -0800 (PST) Date: Sun, 23 Feb 2014 19:36:13 +0000 Message-ID: Subject: netmap on Linux - system crash with iwlwifi interface From: Huw Rogers To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 19:36:15 -0000 Hi, I get a hard crash of my system when experimenting with netmap, pkt-gen and iwlwifi (Intel WiFi driver). Linux 3.13, latest netmap from git. insmod netmap_lin.ko pkt-gen -f rx -i wlp1s0 ... kaboom (corrupted display, hard crash, no oops even) pkt-gen on a wired interface works fine. I tried to debug this myself, but couldn't find anything. Perhaps someone else with Intel WiFi could attempt the same? -Huw From owner-freebsd-net@FreeBSD.ORG Mon Feb 24 06:56:00 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 410DCB4; Mon, 24 Feb 2014 06:56:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1354C1A34; Mon, 24 Feb 2014 06:56:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1O6txpZ035273; Mon, 24 Feb 2014 06:55:59 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1O6txYD035272; Mon, 24 Feb 2014 06:55:59 GMT (envelope-from linimon) Date: Mon, 24 Feb 2014 06:55:59 GMT Message-Id: <201402240655.s1O6txYD035272@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/186872: [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [regression] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 06:56:00 -0000 Old Synopsis: [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout New Synopsis: [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [regression] Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Mon Feb 24 06:55:38 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=186872 From owner-freebsd-net@FreeBSD.ORG Mon Feb 24 11:06:53 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81269AB2 for ; Mon, 24 Feb 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6C4421625 for ; Mon, 24 Feb 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1OB6r0h027611 for ; Mon, 24 Feb 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1OB6qUm027608 for freebsd-net@FreeBSD.org; Mon, 24 Feb 2014 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Feb 2014 11:06:52 GMT Message-Id: <201402241106.s1OB6qUm027608@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183390 net [ixgbe] 10gigabit networking problems o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) o kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis p kern/165903 net mbuf leak o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 465 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Feb 24 18:20:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31F0AA46 for ; Mon, 24 Feb 2014 18:20:39 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AE8AE1594 for ; Mon, 24 Feb 2014 18:20:38 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id ec20so1893712lab.29 for ; Mon, 24 Feb 2014 10:20:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=nhmntI0D76r9QqkJtWqSzsktrWCWq/QXar3QDXmWg24=; b=Q/LH1lzOeDf+EGbczTf+kYjtukBeomMyrJu6dZkA1CZGfjqEMI3oUutpD7XAfefxA7 soseqeYakB8fyBtqiYnzEVDZz5j1hp6oIEgciEahndbwV6lbjqs7sundVGZscV9KIa2e RU9M8zN8f2W0hpqU/FyZJIn2dS3a8+Oh8/dFQTqMehVqTekLvvPf6QHNbN+roVUrZgUz QjIc8fMD0FqUQtcp/aRqF27RbJhKag70rhfzaNJaFnXlrAgTZZUT6DCocM/NCAJfY9Q7 mD0ufjOCn8O1rVZcY9tlNw7i5i6q1rm+cM5j5n6Cc+e6Q+iS42BI95+pPDywYZ8L3osD paRw== X-Received: by 10.112.139.232 with SMTP id rb8mr12416702lbb.53.1393266035508; Mon, 24 Feb 2014 10:20:35 -0800 (PST) Received: from ahti.pakhom.spb.ru (pakhomg.static.corbina.ru. [89.179.122.17]) by mx.google.com with ESMTPSA id jt7sm14617678lbc.15.2014.02.24.10.20.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 24 Feb 2014 10:20:34 -0800 (PST) Message-ID: <530B8D79.2000506@gmail.com> Date: Mon, 24 Feb 2014 22:20:41 +0400 From: Pakhom Golynga User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: freebsd10 session/packets close/lost on igb Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 18:20:39 -0000 Hello, After updated two hosts to freebsd 10 I have problem with network (driver igb). When uptime of hosts > ~1-1.5 days host's unexpected drop/lost some tcp connections/packets. I try to disable -tso, -rxcsum and -txcsum but problem still the same. I collect tcpdump on client and server side but not yet understood what was happening today. What are recommendation for debug this issue? Thanks From owner-freebsd-net@FreeBSD.ORG Mon Feb 24 19:11:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E54E19A0; Mon, 24 Feb 2014 19:11:48 +0000 (UTC) Received: from mail-ee0-x22e.google.com (mail-ee0-x22e.google.com [IPv6:2a00:1450:4013:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2CA9B1B37; Mon, 24 Feb 2014 19:11:48 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id d49so976269eek.33 for ; Mon, 24 Feb 2014 11:11:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=r9B7GDwGtGTqzyliUCKfYpYtRn3QqpLUe8/LVUllvAg=; b=KlN4UJq4AdcwVM490gt4WdVHyZjdPx0EMbi9KgjWDEePSCKRHHZi2PV1HTSq4ii6rP +hxEcelSrvZpw2DzhEOG6VJjAovJO7xUxQ38GsXVD0RikoDKak90Cq6pQpT6Fnt6Faeu lLkRnAl9VOMfi2og9RBlJapZ6qhretH6aJs/EoyCp4frYddYI4d21laVDxdsY0bJXU62 O0PhQFuNkt7JMmYq2XLuxnfxSCjxdblJbi/TFmDTUDQpJ+8xjUIizQpa382we4vpl25m qP52/RlpoEIxQh+LecVmXIMLrKpVIM311nHvH0lHkn27TZ30qdcOPophHw6ccGZiZQ1V kkCQ== X-Received: by 10.14.211.71 with SMTP id v47mr26563214eeo.37.1393269106384; Mon, 24 Feb 2014 11:11:46 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id q44sm67075615eez.1.2014.02.24.11.11.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 24 Feb 2014 11:11:45 -0800 (PST) Sender: Alexander Motin Message-ID: <530B996F.4060100@FreeBSD.org> Date: Mon, 24 Feb 2014 21:11:43 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: rpcbind & TCP wrappers Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 19:11:49 -0000 Hi. I've made benchmark to test rpcbind performance and discovered very interesting numbers: on my test machine our present rpcbind is able to handle only 12K RPCs per second, but building it without TCP wrappers (libwrap) improves performance to 116K RPCs/sec. Obviously hosts.allow parsing for each RPC is too expensive. Since rpcbind output is often cached by the clients it may be not so huge problem, but still 10x difference IMO worth some decision to be made there. I've talked to several people and they agree that it is not very useful to protect rpcbind since it is any way effectively read-only for other hosts in default configuration. Since I expect some people may still want it I've implemented patch disabling TCP wrappers in rpcbind by default, but introducing new command line option -t to easily restore functionality when needed: http://people.freebsd.org/~mav/libwrap.patch Any comments or objections? -- Alexander Motin From owner-freebsd-net@FreeBSD.ORG Mon Feb 24 20:14:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23415F5D; Mon, 24 Feb 2014 20:14:19 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E45C61250; Mon, 24 Feb 2014 20:14:18 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 5D90A14065; Mon, 24 Feb 2014 12:14:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1393272858; bh=1iWoCerLsoFGdorZOtcs+6Ow7vf/RmpkC+T9PCC2RXA=; h=Date:From:Reply-To:To:Subject:References:In-Reply-To; b=iAYOG/TlAkRrhEptFIVWOMwcb04RhacJoeJ9EUNRINH4vDPA10t8I5eJ6mesGNmzw yAiduSHCFrpH/p3g1TcPc4TYKAtQ168YNnnhyy2pjjbARBo2L771jcWbMPl1CUa/5W epG7c2J36eLkz98R/FEZrp34qcPZfdbKoCsXMX5Q= Message-ID: <530BA819.1080400@delphij.net> Date: Mon, 24 Feb 2014 12:14:17 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Alexander Motin , freebsd-net@freebsd.org Subject: Re: rpcbind & TCP wrappers References: <530B996F.4060100@FreeBSD.org> In-Reply-To: <530B996F.4060100@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------030602000904060405050301" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 20:14:19 -0000 This is a multi-part message in MIME format. --------------030602000904060405050301 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 02/24/14 11:11, Alexander Motin wrote: > Hi. > > I've made benchmark to test rpcbind performance and discovered > very interesting numbers: on my test machine our present rpcbind is > able to handle only 12K RPCs per second, but building it without > TCP wrappers (libwrap) improves performance to 116K RPCs/sec. > Obviously hosts.allow parsing for each RPC is too expensive. Since > rpcbind output is often cached by the clients it may be not so huge > problem, but still 10x difference IMO worth some decision to be > made there. > > I've talked to several people and they agree that it is not very > useful to protect rpcbind since it is any way effectively read-only > for other hosts in default configuration. Since I expect some > people may still want it I've implemented patch disabling TCP > wrappers in rpcbind by default, but introducing new command line > option -t to easily restore functionality when needed: > http://people.freebsd.org/~mav/libwrap.patch > > Any comments or objections? I think the new 't' option should be wrapped like 'WSOP' (see attachment for a revised version). By the way we need to be careful when changing the defaults, or it creates astonishment (tcpwrap are supposed to work without restarting the service) but I think this is probably a pain we have to face if we can't make TCP wrappers to work faster. If we are going to proceed with the proposed patch, we'd better document it in both UPDATING and release notes. I do agree that it makes little difference protecting or not protecting rpcbind, though. Actually, I always force it to listen only on trusted LAN interface if I have to run it and that's cheaper than TCP wrappers or firewalling the port. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCgAGBQJTC6gZAAoJEJW2GBstM+ns/noP/A2YNG/x0MJBH1QnKEV48WQB oUwYiUjf7wMHFzTJtzX8A8hMZWqunVSMA/0NC0B/vpai6F73BprZHMSxRVu/FmZ/ 4jzXLG5UMwtO9/nEwiMV3ecRBLUKM8ltZEczy1wS1bOqBU8a+6HShhwxUMDotbaW xkiuWrIgrFVstAZP+BMEs/Z4P8xOtMGQXH2eBodZF2Myn5INtmkpXkH3FTxPLoJU NkdcPEGFDesecjm8mwHHEieDlk4cveUki+S1rx9NdE3CqQw74/4ST8QvTWqOkuKG iiGS/lH/76JtojAqQrDw2QzXYNfZuFYqBSN7X5K9qqKsgb0JqLvd6JiwsGEO+Qez X6NHjs/kNBPNu1IZTzM7IKBzrhbqVzj+WO8rZtRzlTshC8XEocdGG+T6QaOrubVP Hn+g1qOZ754ipAOfSX1zSthnVF0psZ+vD3Z/tAhrjnh4e1kLbc91dbcr8wp+Hkxj 32XWZ9Fhhb9o9EkuLVNWgh55xfUqb55qp23QwVE766Gg10OpG9khBJI0tucg9a3+ kdHBaH1b86rB2q7RI3itwMAgPAVU0eK/ASO+QE49pMysw3mw8EhnEEyQrGDO3B4P VSeegOLbIvcLkoiPqhSY0NL5TOeqp97WypYnY01k44iulN7g0XQMS87WnK+YcWUs U8pqHxLwrJ5OzD/gYLht =K3zm -----END PGP SIGNATURE----- --------------030602000904060405050301 Content-Type: text/plain; charset=UTF-8; name="rpcbind.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="rpcbind.diff" Index: etc/hosts.allow =================================================================== --- etc/hosts.allow (revision 262456) +++ etc/hosts.allow (working copy) @@ -60,6 +60,7 @@ exim : localhost : allow exim : ALL : allow # Rpcbind is used for all RPC services; protect your NFS! +# Rpcbind should be running with -t option to support this. # (IP addresses rather than hostnames *MUST* be used here) #rpcbind : 192.0.2.32/255.255.255.224 : allow #rpcbind : 192.0.2.96/255.255.255.224 : allow Index: usr.sbin/rpcbind/rpcbind.8 =================================================================== --- usr.sbin/rpcbind/rpcbind.8 (revision 262456) +++ usr.sbin/rpcbind/rpcbind.8 (working copy) @@ -2,7 +2,7 @@ .\" Copyright 1989 AT&T .\" Copyright 1991 Sun Microsystems, Inc. .\" $FreeBSD$ -.Dd April 23, 2007 +.Dd February 24, 2014 .Dt RPCBIND 8 .Os .Sh NAME @@ -133,6 +133,8 @@ to use non-privileged ports for outgoing connectio clients from using .Nm to connect to services from a privileged port. +.It Fl t +Enable TCP Wrappers support. .El .Sh NOTES All RPC servers must be restarted if Index: usr.sbin/rpcbind/rpcbind.c =================================================================== --- usr.sbin/rpcbind/rpcbind.c (revision 262456) +++ usr.sbin/rpcbind/rpcbind.c (working copy) @@ -88,6 +88,9 @@ rpcblist_ptr list_rbl; /* A list of version 3/4 rp int runasdaemon = 0; int insecure = 0; int oldstyle_local = 0; +#ifdef LIBWRAP +int libwrap = 0; +#endif int verboselog = 0; char **hosts = NULL; @@ -785,7 +788,12 @@ parseargs(int argc, char *argv[]) #else #define WSOP "" #endif - while ((c = getopt(argc, argv, "6adh:iLls" WSOP)) != -1) { +#ifdef LIBWRAP +#define WRAPOP "t" +#else +#define WRAPOP "" +#endif + while ((c = getopt(argc, argv, "6adh:iLls" WRAPOP WSOP)) != -1) { switch (c) { case '6': ipv6_only = 1; @@ -818,6 +826,11 @@ parseargs(int argc, char *argv[]) case 's': runasdaemon = 1; break; +#ifdef LIBWRAP + case 't': + libwrap = 1; + break; +#endif #ifdef WARMSTART case 'w': warmstart = 1; @@ -825,8 +838,8 @@ parseargs(int argc, char *argv[]) #endif default: /* error */ fprintf(stderr, - "usage: rpcbind [-6adiLls%s] [-h bindip]\n", - WSOP); + "usage: rpcbind [-6adiLls%s%s] [-h bindip]\n", + WRAPOP, WSOP); exit (1); } } Index: usr.sbin/rpcbind/rpcbind.h =================================================================== --- usr.sbin/rpcbind/rpcbind.h (revision 262456) +++ usr.sbin/rpcbind/rpcbind.h (working copy) @@ -66,6 +66,9 @@ struct r_rmtcall_args { extern int debugging; extern int doabort; +#ifdef LIBWRAP +extern int libwrap; +#endif extern int verboselog; extern int insecure; extern int oldstyle_local; Index: usr.sbin/rpcbind/security.c =================================================================== --- usr.sbin/rpcbind/security.c (revision 262456) +++ usr.sbin/rpcbind/security.c (working copy) @@ -108,13 +108,15 @@ check_access(SVCXPRT *xprt, rpcproc_t proc, void * } #ifdef LIBWRAP - if (addr->sa_family == AF_LOCAL) - return 1; - request_init(&req, RQ_DAEMON, "rpcbind", RQ_CLIENT_SIN, addr, 0); - sock_methods(&req); - if(!hosts_access(&req)) { - logit(deny_severity, addr, proc, prog, ": request from unauthorized host"); - return 0; + if (libwrap && addr->sa_family != AF_LOCAL) { + request_init(&req, RQ_DAEMON, "rpcbind", RQ_CLIENT_SIN, addr, + 0); + sock_methods(&req); + if(!hosts_access(&req)) { + logit(deny_severity, addr, proc, prog, + ": request from unauthorized host"); + return 0; + } } #endif if (verboselog) --------------030602000904060405050301-- From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 01:00:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61F634AE for ; Tue, 25 Feb 2014 01:00:04 +0000 (UTC) Received: from relay2-bcrtfl2.verio.net (relay2-bcrtfl2.verio.net [131.103.218.177]) by mx1.freebsd.org (Postfix) with ESMTP id 0C6E81B32 for ; Tue, 25 Feb 2014 01:00:03 +0000 (UTC) Received: from iad-wprd-xchw02.corp.verio.net (iad-wprd-xchw02.corp.verio.net [198.87.7.165]) by relay2-bcrtfl2.verio.net (Postfix) with ESMTP id D8C611FF006A; Mon, 24 Feb 2014 19:59:56 -0500 (EST) Received: from IAD-WPRD-XCHB01.corp.verio.net ([198.87.7.137]) by iad-wprd-xchw02.corp.verio.net with Microsoft SMTPSVC(6.0.3790.4675); Mon, 24 Feb 2014 19:59:56 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4913 Content-Class: urn:content-classes:message Importance: normal Priority: normal MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Subject: RE: FreeBSD behind a firewall Date: Mon, 24 Feb 2014 19:59:55 -0500 Message-ID: In-Reply-To: <5308133F.7050504@natserv.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: FreeBSD behind a firewall thread-index: Ac8vepmrgtZ9VCzRSUuItUCxTdtk5QCSGJ8w References: <5308133F.7050504@natserv.net> From: "David DeSimone" To: "Francisco Reyes" X-OriginalArrivalTime: 25 Feb 2014 00:59:56.0582 (UTC) FILETIME=[E665DC60:01CF31C4] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:00:04 -0000 This is a classic routing policy problem. Unless the Vyatta firewall = applies some sort of Source NAT to incoming connections, the replies to = those connections will follow your default route, leave via an interface = that does not pass them back through the firewall. One method of inserting policy-based routing is to use pf, like so: FW_IF =3D xn2 # Firewall-connected interface FW_IP =3D 192.168.3.1 # Firewall's IP pass in on $FW_IF reply-to ( $FW_IF $FW_IP ) proto tcp from any = to any port { http, https } This will build state entries that force replies to go back through the = interface they came in. You might need to add extra logic to match traffic that comes in via xn2 = but didn't actually arrive from the firewall, if that's a possible = traffic pattern for you. -----Original Message----- From: owner-freebsd-net@freebsd.org = [mailto:owner-freebsd-net@freebsd.org] On Behalf Of Francisco Reyes Sent: Friday, February 21, 2014 9:02 PM To: freebsd-net@freebsd.org Subject: FreeBSD behind a firewall Setup Internet --> Vyatta firewall --> FreeBSD Trying to have the FreeBSD machine listen on http and https on local=20 network and have the Vyatta firewall forward the traffic from the=20 external connections. I have the Vyatta already configured to send to FreeBSD, but it seems=20 the packets at the FreeBSD machine are not going back to the firewall.. The FreeBSD machine has 3 interfaces xn0 public - will have ssh open xn1 internal - visible in entire data center (Rackspace VM) xn2 internal - private net on 192.168.3.0 I have the Vyatta firewall sending traffic to xn2 and I am able to see=20 it with TCPdump I tried setting a static route for all of 192.168.3.0 to go through the=20 Vyatta firewall, but that did not seem to help. Output of netstat -r Internet: Destination Gateway Flags Refs Use Netif = Expire default 162.209.99.1 UGS 0 3542 xn0 10.176.0.0/18 link#5 U 0 0 xn1 =3D> 10.176.0.0/12 10.176.0.1 UGS 0 0 xn1 testvm link#5 UHS 0 0 lo0 localhost link#3 UH 0 0 lo0 162.209.99.0 link#4 U 0 0 xn0 testvm link#4 UHS 0 0 lo0 192.168.3.0 link#6 U 0 0 xn2 192.168.3.1 link#6 UHS 0 0 lo0 The FreeBSD machine is 192.168.3.1, the Vyatta firewall is 192.168.3.2 Relevant parts of /etc/rc.conf defaultrouter=3D"162.209.99.1" static_routes=3D"lan0 lan1 lan2" route_lan0=3D"-net 10.176.0.0 -netmask 255.240.0.0 10.176.0.1" route_lan1=3D"-net 10.208.0.0 -netmask 255.240.0.0 10.176.0.1" route_lan1=3D"-net 192.168.3.0 -netmask 255.255.255.0 192.168.3.2" Any pointers on how I can get the traffic to go back to the Vyatta = firewall? Does the firewall needs to be the gateway for the VM? The ideal would be to keep ssh outside as to not depend on the firewall=20 and http and https to go throught he firewall. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" This email message is intended for the use of the person to whom it has = been sent, and may contain information that is confidential or legally = protected. If you are not the intended recipient or have received this = message in error, you are not authorized to copy, distribute, or = otherwise use this message or its attachments. Please notify the sender = immediately by return e-mail and permanently delete this message and any = attachments. Verio Inc. makes no warranty that this email is error or = virus free. Thank you. From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 03:18:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 303FEEC1; Tue, 25 Feb 2014 03:18:52 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 550DE1848; Tue, 25 Feb 2014 03:18:51 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s1P3IYSc066339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 25 Feb 2014 04:18:35 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-net@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s1P3IQB0037321; Tue, 25 Feb 2014 10:18:27 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <530C0B82.8070303@grosbein.net> Date: Tue, 25 Feb 2014 10:18:26 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: d@delphij.net Subject: Re: rpcbind & TCP wrappers References: <530B996F.4060100@FreeBSD.org> <530BA819.1080400@delphij.net> In-Reply-To: <530BA819.1080400@delphij.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: freebsd-net@freebsd.org, Alexander Motin , Xin Li X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 03:18:52 -0000 On 25.02.2014 03:14, Xin Li wrote: > By the way we need to be careful when changing the defaults, or it > creates astonishment (tcpwrap are supposed to work without restarting > the service) but I think this is probably a pain we have to face if we > can't make TCP wrappers to work faster. We can't? What if we make libwrap cache and check hosts.allow/hosts.deny modification times early and just skip if it was not modified since last check? Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 07:24:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA80C40A for ; Tue, 25 Feb 2014 07:24:43 +0000 (UTC) Received: from mail-pd0-f178.google.com (mail-pd0-f178.google.com [209.85.192.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B359114ED for ; Tue, 25 Feb 2014 07:24:43 +0000 (UTC) Received: by mail-pd0-f178.google.com with SMTP id fp1so7400965pdb.23 for ; Mon, 24 Feb 2014 23:24:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=9AWWlYydHS+IEJdqsvKhZKaN8nFVTaEw7Q+AcdC7CuQ=; b=NPyCtbMbVMoE8WyRBeMeG90Il5HlZVQksJFhycIJhLgc/tMYVqUyUizNPgVCbb5QZw V4ESOGmVf2FMlmSauLGN8ySf7T9Xs9lfl0rM86EYHuP6AK131DHh3pj8anCEFSy97jf/ cQEBzayPn0QvK+dJTOkwrNWXbOt5CgEjOwkeEIJpaLCDYphEEFXYoQXO9Uh6H0AkNQ3V 0BmGIfJ2/nRQuk/p9cfvKu7f+eoAzneqt2Ggc5DfTb6m+5gHukBmbH1FoWhJ0F4/sHEe d9Jduu9E0i2adUwO6zSBK81d0KK70nyinlSESZ9m/716BlwW5ZkKCEGELTrqa+rE1O2S QWeQ== X-Gm-Message-State: ALoCoQm56vP/IJHgQFnrVuT1CdiixTuJL5UlXyNM2VC3QE7bUZ9Fj3Ew7flr00O88g8pwDHBGK0F MIME-Version: 1.0 X-Received: by 10.68.163.3 with SMTP id ye3mr4899871pbb.78.1393312742067; Mon, 24 Feb 2014 23:19:02 -0800 (PST) Received: by 10.68.111.37 with HTTP; Mon, 24 Feb 2014 23:19:01 -0800 (PST) Date: Tue, 25 Feb 2014 08:19:01 +0100 Message-ID: Subject: Network loss From: Johan Kooijman To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 07:24:44 -0000 Hi all, I have a weird situation here where I can't get my head around. One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in a while the Linux clients loose their NFS connection: Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, timed out Not all boxes, just one out of the cluster. The weird part is that when I try to ping a Linux client from the FreeBSD box, I have between 10 and 30% packetloss - all day long, no specific timeframe. If I ping the Linux clients - no loss. If I ping back from the Linux clients to FBSD box - no loss. The errors I get when pinging a Linux client is this one: ping: sendto: File too large What we already did: - switched interfaces in the FreeBSD box: switched from ixgbe to a gbe interface. Both have the same issue. - After that I configured the ixgbe interface with a different IP & subnet and I'm able to ping without any issue to any client. - Verified MTU's - all default; - Cheched switch config - is empty now, it's as much configured as a fridge :) Any hints on where to search from here? -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 08:27:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 271D8BA4 for ; Tue, 25 Feb 2014 08:27:12 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0AE681AE7 for ; Tue, 25 Feb 2014 08:27:11 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WIDM1-00032Q-L3 for freebsd-net@freebsd.org; Tue, 25 Feb 2014 00:27:05 -0800 Date: Tue, 25 Feb 2014 00:27:05 -0800 (PST) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <1393316825645-5888870.post@n5.nabble.com> Subject: Unbound's dnssec-keygen in FreeBSD MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 08:27:12 -0000 Unless I'm wrong, Unbound seems to natively support "dnssec-keygen" (from this document) http://www.nlnetlabs.nl/publications/dnssec_howto/#x1-620009.2.1 I have Unbound built and running, but there is no dnssec-keygen on the system. Was this not ported to FreeBSD's version of Unbound? I suppose I have to install dns/bind99 if I need dnssec-keygen? ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/Unbound-s-dnssec-keygen-in-FreeBSD-tp5888870.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 08:49:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE240500 for ; Tue, 25 Feb 2014 08:49:26 +0000 (UTC) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 644231CD7 for ; Tue, 25 Feb 2014 08:49:26 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id l4so3039366lbv.19 for ; Tue, 25 Feb 2014 00:49:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=0hECQJpghXt3gHnN6dZ2Y6J9gtNLzu1mvKZq70eH9zg=; b=CcL7cC7SujUPe4Qwppr9eM1B0EveKgRoap4+XITj0c/Bee6bWfhKeJXflkb/JS0/I7 WFQHBpmP3aZCsKxfbeusETAi2I7LwvmGG09czU4E0k8enoAZ2b6jA0crXmJa2+oIdEvZ 7i8WoV5dui6VfTKaU+diqrhGu3NYPcvhf3sriuqKWA/fsayKkTgTnL5jpFUI3RzQRpGo aMxPo7HVjYAXP/bbcZEhP7gEc/oIoBABJz/C/9Ju1+oIK16b0LMpXxk865DAiyk2M6mR s0viqRZbb7a6wp9VtlNq7S0GqXPXcwxPlWj6IJab3B+S09s6Jov4do/4gOWWlxXBgFsi v8ww== MIME-Version: 1.0 X-Received: by 10.152.164.199 with SMTP id ys7mr178289lab.31.1393318164414; Tue, 25 Feb 2014 00:49:24 -0800 (PST) Received: by 10.152.10.74 with HTTP; Tue, 25 Feb 2014 00:49:24 -0800 (PST) Date: Tue, 25 Feb 2014 16:49:24 +0800 Message-ID: Subject: netmap: bridge example running results in sendmsg: No buffer space available From: Jianyong Chen To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 08:49:26 -0000 Hi all, Today I compile and run netmap in Linux and in my virtual machine. The net adaptor driver is e1000. After insmod netmap_lin.ko and e1000.ko. I run bridge in the example. At the same time, I make a 'ping' in the VM. After a while, the ping throws out the error message "sendmsg: No buffer space available". Could anyone help to interpret and resolve it? Thank you. Regards Jianyong From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 08:59:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D63AEDD9 for ; Tue, 25 Feb 2014 08:59:58 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5E97E1DE9 for ; Tue, 25 Feb 2014 08:59:58 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id c6so6886023lan.38 for ; Tue, 25 Feb 2014 00:59:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=vle3g1DyybiK1w+XfdCHIxLbugJ8nLDws5l62kHKxUU=; b=v4pOdWtRUYh+AGVWL2wK49RU5Y7B6nEoXLGQ40ZjsvCIJdNNrHCsxCdxK8R+WAuHwx 3TEUzih0dcoOFoYqZpUBS5axNdAHf/c71+Jkhh9viK6G2ZmRI4DP6acKpE8vpeV6N3K0 xz9g7p8fTutzn7/H09YAqN36wnO6QMaKnDJF+ixwD1o2wX76IXyZW/H6G4eK395OEICL XU3JD4sgLd96LXZ5YjaNY0yvt5Bl3xeHTmjK0005CqVpCbhyDUYi4vHxjN1/O/tcHiH1 SkringCU70Ogm9MF4gKaAXy4IPD6NtTxFd7maVTbblAd+R+0HnliBvQfapvpcRiTsEvG 2biA== MIME-Version: 1.0 X-Received: by 10.152.44.225 with SMTP id h1mr168144lam.71.1393318796278; Tue, 25 Feb 2014 00:59:56 -0800 (PST) Received: by 10.152.10.74 with HTTP; Tue, 25 Feb 2014 00:59:56 -0800 (PST) In-Reply-To: References: Date: Tue, 25 Feb 2014 16:59:56 +0800 Message-ID: Subject: Re: netmap: bridge example running results in sendmsg: No buffer space available From: Jianyong Chen To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 08:59:59 -0000 The running cmd is 'bridge -i netmap:eth0 -i netmap:eth0' and I want the bridge to link the NIC and SW ring. At first, it's OK and I can see the ping success message. By the ping output, it's should less than 100 pings when it fails. Jianyong On Tue, Feb 25, 2014 at 4:49 PM, Jianyong Chen wrote: > Hi all, > > Today I compile and run netmap in Linux and in my virtual machine. The net > adaptor driver is e1000. > After insmod netmap_lin.ko and e1000.ko. I run bridge in the example. At > the same time, I make a 'ping' in the VM. > After a while, the ping throws out the error message "sendmsg: No buffer > space available". Could anyone help to interpret and resolve it? > > Thank you. > > Regards > Jianyong > From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 10:29:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C8A61E4 for ; Tue, 25 Feb 2014 10:29:41 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E41F9183F for ; Tue, 25 Feb 2014 10:29:40 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id a1so174365wgh.22 for ; Tue, 25 Feb 2014 02:29:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Ohtj/+AMDrM25pIhtrFXQLVdjr5K6xh+KL1fJF42gcU=; b=1GHVyn2kPD185x6K0kSNXVhu3z+Lu8BxQB+S25aho/gWinKeMhWk3m09uSE0UDXVzR YoQPDcWUXj9VZDPcS/Tx2FoviwgigVAi4LgSrIg1nUs4qxpc98s8rg9bYm2Ljzxz2UqF VerR9lJL5MlTojF4JfkFdUwzUjspy3hIGFVkpuXkjX+F0EDuCno8/8tp0mnqDSfkNxwL hzK1gG8afB3GAiXk8si/hBglrHHYQhRxuaBU64+IX2etzs2t4UEkXZLIw5U1yfSwS7Xk bjxcPNzTV+WQfYsJcB+xxq7S/zWejQshL/nt/rgmT/bR0WP4Brrf5WzOMgsK3GTZVyO+ mCcg== X-Received: by 10.194.62.243 with SMTP id b19mr939395wjs.63.1393324176467; Tue, 25 Feb 2014 02:29:36 -0800 (PST) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id m8sm64968881eef.14.2014.02.25.02.29.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Feb 2014 02:29:35 -0800 (PST) Sender: Alexander Motin Message-ID: <530C708C.9060107@FreeBSD.org> Date: Tue, 25 Feb 2014 12:29:32 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Eugene Grosbein , d@delphij.net Subject: Re: rpcbind & TCP wrappers References: <530B996F.4060100@FreeBSD.org> <530BA819.1080400@delphij.net> <530C0B82.8070303@grosbein.net> In-Reply-To: <530C0B82.8070303@grosbein.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Xin Li X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 10:29:41 -0000 On 25.02.2014 05:18, Eugene Grosbein wrote: > On 25.02.2014 03:14, Xin Li wrote: > >> By the way we need to be careful when changing the defaults, or it >> creates astonishment (tcpwrap are supposed to work without restarting >> the service) but I think this is probably a pain we have to face if we >> can't make TCP wrappers to work faster. > > We can't? > > What if we make libwrap cache and check hosts.allow/hosts.deny modification times early > and just skip if it was not modified since last check? Skip what? Configuration can be not trivial, and we can't know what exactly you can or can not cache. Even if we skip just file read, we still have to process it all, but that requires time too. Do we really want/need another firewall there? -- Alexander Motin From owner-freebsd-net@FreeBSD.ORG Tue Feb 25 10:48:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A1EF676; Tue, 25 Feb 2014 10:48:18 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5700619E1; Tue, 25 Feb 2014 10:48:17 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s1PAm2Wr067905 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 25 Feb 2014 11:48:03 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-net@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s1PAlwPU076008; Tue, 25 Feb 2014 17:47:58 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <530C74DE.70203@grosbein.net> Date: Tue, 25 Feb 2014 17:47:58 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Alexander Motin Subject: Re: rpcbind & TCP wrappers References: <530B996F.4060100@FreeBSD.org> <530BA819.1080400@delphij.net> <530C0B82.8070303@grosbein.net> <530C708C.9060107@FreeBSD.org> In-Reply-To: <530C708C.9060107@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=3.2 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * 2.9 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-Spam-Level: *** Cc: Xin Li , d@delphij.net, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 10:48:18 -0000 On 25.02.2014 17:29, Alexander Motin wrote: >> We can't? >> >> What if we make libwrap cache and check hosts.allow/hosts.deny modification times early >> and just skip if it was not modified since last check? > > Skip what? Skip full file parsing. > Configuration can be not trivial, and we can't know what > exactly you can or can not cache. How can result be not cacheable for rpcbind? > Even if we skip just file read, we still have to process it all, > but that requires time too. Do we really > want/need another firewall there? No need in another firewall. Just make small hash containing result of previous check for the client and get result from hash if modification time of file has not changed. With fallback to full file processing when hash overflows. From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 16:33:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC130546 for ; Wed, 26 Feb 2014 16:33:45 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD71E1C32 for ; Wed, 26 Feb 2014 16:33:45 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 93DB9B9D0; Wed, 26 Feb 2014 11:33:44 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Network loss Date: Wed, 26 Feb 2014 11:32:09 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402261132.09203.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 26 Feb 2014 11:33:44 -0500 (EST) Cc: Johan Kooijman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 16:33:45 -0000 On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > Hi all, > > I have a weird situation here where I can't get my head around. > > One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in a while > the Linux clients loose their NFS connection: > > Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, timed out > > Not all boxes, just one out of the cluster. The weird part is that when I > try to ping a Linux client from the FreeBSD box, I have between 10 and 30% > packetloss - all day long, no specific timeframe. If I ping the Linux > clients - no loss. If I ping back from the Linux clients to FBSD box - no > loss. > > The errors I get when pinging a Linux client is this one: > ping: sendto: File too large EFBIG is sometimes used for drivers when a packet takes too many scatter/gather entries. Since you mentioned NFS, one thing you can try is to disable TSO on the intertface you are using for NFS to see if that "fixes" it. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 17:37:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD4E0413 for ; Wed, 26 Feb 2014 17:37:44 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B67DC1390 for ; Wed, 26 Feb 2014 17:37:44 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so1279795pad.0 for ; Wed, 26 Feb 2014 09:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ZFD1CSLKCeJE0T9GW8p74xkDI0fh6XF9FWdeIgVLSks=; b=tUZjoNeaRpoRfwfNOy9KE5fK+t3B+pilRm3LyEFTYbfg4tRsCD2hK+q6pIGDjq15vH cypjoQdt3Q+3Fiaa9BenQb/DjTIipR5jAPcUOSiFLIHdxmTakPfIOcImIOBS8iWMNiMG KyVkALiFWchvEyBbIxtYjLkLwu0AKqQ4JNpu3nLA6b3yloVJzIIc/+/qbNBXuBKVVUlZ bxYum3wtytBYEncECPUXcrlsUHZIyuEz2Mixx7vP51kr4h10QTOqVLZ2nzVr+Yje+ibf +SpdnYhy9X7lHtbYps62qT7PH3R9EidhCTNvQAjkhprYwGeR3K6RJMSAvMVHH+aBh+Z+ MgfA== MIME-Version: 1.0 X-Received: by 10.66.254.3 with SMTP id ae3mr10087247pad.107.1393436264352; Wed, 26 Feb 2014 09:37:44 -0800 (PST) Received: by 10.70.127.142 with HTTP; Wed, 26 Feb 2014 09:37:44 -0800 (PST) Received: by 10.70.127.142 with HTTP; Wed, 26 Feb 2014 09:37:44 -0800 (PST) Date: Wed, 26 Feb 2014 19:37:44 +0200 Message-ID: Subject: TSO From: Sami Halabi To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 17:37:44 -0000 Hi, I'm reading (almost) all mailing emails in mailig list... Almost every / many problem in network performancr / packets loss ended up suggesting disabling TSO. I wonder why.. Is it a bug in the implementation? Or bybdesign? What are the usecases that TSO is needed? Myabe it should be disabled bt default? Thanks in advance, Sami From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 17:56:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 563B0984 for ; Wed, 26 Feb 2014 17:56:24 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 13E0E1595 for ; Wed, 26 Feb 2014 17:56:24 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id x3so1826487qcv.4 for ; Wed, 26 Feb 2014 09:56:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=4lAB3bqPvfj+sjfdq1oeIFBGbhzEyAc6i8kbhDbYphE=; b=zlw8JDD3QkA0ShkUk4VXx1KZs2qwOGKiPEdr4WgpEsVq+eeXE55g1MXucAuGNpg8gP 6OBIGHGKVoYr+84JLmXC9CpttC9RgisXBxis3SuivO6CmEM6DjfBO9H3FEQbflOEj+ya OGaMSXup0+HSxQbn+Bk3EhcIiiAM6Y2WBOdnUFYCoVTBQuEazPZEwXYrkQ4qemE+epJQ kOE/j0HocD0PfEMovlr8Dma7O0LA9zvzOs5VeH6aElsL3a8OaAWNJFkuSwjtTDNeC9BC WBUYvZUjwVSOZ5RNqfdLHXegdrTDaXS3VCbf6dSSK2iDm87VQI/ZdGjvpCAgXYVZ9yfY DqXQ== MIME-Version: 1.0 X-Received: by 10.140.42.54 with SMTP id b51mr894996qga.87.1393437383221; Wed, 26 Feb 2014 09:56:23 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Wed, 26 Feb 2014 09:56:23 -0800 (PST) In-Reply-To: References: Date: Wed, 26 Feb 2014 09:56:23 -0800 X-Google-Sender-Auth: kqW27yvTZb4GLpWUMeNBKPV_bA0 Message-ID: Subject: Re: TSO From: Adrian Chadd To: Sami Halabi Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 17:56:24 -0000 Hi, TSO is needed for high throughput TCP work. The cost of output packet handling (lookups, pcb locks) and the CPU use in tcp_output() is quite high. It'd be great to fix that though. :) -a On 26 February 2014 09:37, Sami Halabi wrote: > Hi, > I'm reading (almost) all mailing emails in mailig list... > > Almost every / many problem in network performancr / packets loss ended up > suggesting disabling TSO. > > I wonder why.. Is it a bug in the implementation? Or bybdesign? > What are the usecases that TSO is needed? Myabe it should be disabled bt > default? > > Thanks in advance, > Sami > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 18:07:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D15E1F39 for ; Wed, 26 Feb 2014 18:07:37 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD55716C8 for ; Wed, 26 Feb 2014 18:07:37 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1QI7aFI038325 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 26 Feb 2014 10:07:37 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1QI7aK4038324; Wed, 26 Feb 2014 10:07:36 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Feb 2014 10:07:36 -0800 From: John-Mark Gurney To: Sami Halabi Subject: Re: TSO Message-ID: <20140226180736.GV92037@funkthat.com> Mail-Followup-To: Sami Halabi , freebsd-net@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Feb 2014 10:07:37 -0800 (PST) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 18:07:37 -0000 Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: > I'm reading (almost) all mailing emails in mailig list... > > Almost every / many problem in network performancr / packets loss ended up > suggesting disabling TSO. > > I wonder why.. Is it a bug in the implementation? Or bybdesign? > What are the usecases that TSO is needed? Myabe it should be disabled bt > default? It looks like most of the problems are in drivers that don't handle packets with a large number of segments properly... The problem is that some drivers limit to how segments a packet can be broken into, and then if they receive such a packet, instead of doing their darnest to deliver it, they drop it... There are some patches that help address the issue... Drivers should complain more loudly when a packet gets dropped by the driver, since it is likely that the OS may retry the same packet, just to have it fail, though sometimes it'll try a different set, and it might go through, so all the user may notice is a slight lag if they notice anything at all... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 18:28:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37FF86C8 for ; Wed, 26 Feb 2014 18:28:00 +0000 (UTC) Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E99C119B3 for ; Wed, 26 Feb 2014 18:27:59 +0000 (UTC) Received: by mail-ve0-f180.google.com with SMTP id jz11so2525618veb.25 for ; Wed, 26 Feb 2014 10:27:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nb/rSiZdgM95+2wPRuMxDCaqCAMyVFgm8ZTHRRoLIYM=; b=Aix5uFNxusA+vrKzLstZwWu+TVfuQd3HdBuCJ46jAoC361PgurdwQ49DTh7ACOayzh X7iw/xvrLpmeeUBoMpaHdQcN1tDEEhPGhdDk1Y1g4xPwc/Ivi/ibwOsBrz2AtDw9AGgN SsWoz7KqNFWHxdvk7M6efKFeLlQHO2VqBCNynb2jhOne8CKVBFvqrcAGyhsZFaA/Iu83 M2oWVQ4XL61iCy9OdWpxS+2ov1iQtPBiWVl0rMKoivzN+h1HI+GF5YOtNiyN1Bi8oVfI 5aPAnxmklVspZRJfafxvXYKi5wupJ/nYjZNBcwKqDJ8XbEuLsu5OkFPKt9a3uLFbeMC9 LPiQ== MIME-Version: 1.0 X-Received: by 10.52.242.167 with SMTP id wr7mr5895088vdc.32.1393439279112; Wed, 26 Feb 2014 10:27:59 -0800 (PST) Received: by 10.221.11.135 with HTTP; Wed, 26 Feb 2014 10:27:59 -0800 (PST) In-Reply-To: <20140226180736.GV92037@funkthat.com> References: <20140226180736.GV92037@funkthat.com> Date: Wed, 26 Feb 2014 10:27:59 -0800 Message-ID: Subject: Re: TSO From: Jack Vogel To: Sami Halabi , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 18:28:00 -0000 Drivers have to work with whatever the requirements/limitations of the hardware, if you have a 5 lb sack you shouldn't be surprised if some drops when you shove 6 lbs at it :) Why not have this limit in the interface so the stack can avoid exceeding it? Jack On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney wrote: > Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: > > I'm reading (almost) all mailing emails in mailig list... > > > > Almost every / many problem in network performancr / packets loss ended > up > > suggesting disabling TSO. > > > > I wonder why.. Is it a bug in the implementation? Or bybdesign? > > What are the usecases that TSO is needed? Myabe it should be disabled bt > > default? > > It looks like most of the problems are in drivers that don't handle > packets with a large number of segments properly... The problem is > that some drivers limit to how segments a packet can be broken into, and > then if they receive such a packet, instead of doing their darnest to > deliver it, they drop it... > > There are some patches that help address the issue... > > Drivers should complain more loudly when a packet gets dropped by the > driver, since it is likely that the OS may retry the same packet, > just to have it fail, though sometimes it'll try a different set, and > it might go through, so all the user may notice is a slight lag if > they notice anything at all... > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 18:28:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F04B2761 for ; Wed, 26 Feb 2014 18:28:39 +0000 (UTC) Received: from aussmtpmrkps320.us.dell.com (aussmtpmrkps320.us.dell.com [143.166.224.254]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B42319C1 for ; Wed, 26 Feb 2014 18:28:38 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.97,549,1389765600"; d="scan'208";a="102402937" Message-ID: <530E3211.6020805@vangyzen.net> Date: Wed, 26 Feb 2014 12:27:29 -0600 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Sami Halabi , Subject: Re: TSO References: In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 18:28:40 -0000 On 02/26/2014 11:37, Sami Halabi wrote: > Hi, > I'm reading (almost) all mailing emails in mailig list... > > Almost every / many problem in network performancr / packets loss ended up > suggesting disabling TSO. > > I wonder why.. Is it a bug in the implementation? Or bybdesign? > What are the usecases that TSO is needed? Myabe it should be disabled bt > default? In some cases, I have disabled TSO to [successfully] work around a bug in a particular NIC's firmware or hardware, usually a low-end "desktop" gigabit NIC. Eric From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 21:13:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97EB1B56 for ; Wed, 26 Feb 2014 21:13:48 +0000 (UTC) Received: from nm18-vm4.bullet.mail.ne1.yahoo.com (nm18-vm4.bullet.mail.ne1.yahoo.com [98.138.91.178]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1F5581C98 for ; Wed, 26 Feb 2014 21:13:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=gcom1024; t=1393449221; bh=7lpCma4jYsxeRIWPEgxRjMbJCUPiGqQfwxw7+5iJHZY=; h=Received:Received:Received:DKIM-Signature:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=XeYplA72BwjUdBOleYSPQcPgvy/140qElgUb5MfwLHcf4ORLqjYAl/rahRTYA6jUgbWaRWrSTVGGF/IZ+0omOdo4gQWnucpWt/RrJCbq947yKWvwxJGYm03jKf2voz78BQXHLqa8KeEcqkAA6tkQ15KnTrcqUg2py6DFxcdGOZk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=gcom1024; d=yahoo.com; b=A4ViExLBcDFB/oR5KJtpx+7ChsXc0cDWV0lPgx9HWt8lI3Sm9kiJL9jfDSQ5o781x27mNdxTwIaCY2iBQYCUdTyX798oEIZ+MycXp2kfdoGLCSxSjOqkabsFN7M9sSvi14o8i0buigNlsLG6cFMonSnI1AMIaHmLn+jhU5q/BmQ=; Received: from [98.138.100.117] by nm18.bullet.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 21:13:41 -0000 Received: from [98.138.226.126] by tm108.bullet.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 21:13:40 -0000 Received: from [127.0.0.1] by smtp205.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 21:13:40 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1393449220; bh=7lpCma4jYsxeRIWPEgxRjMbJCUPiGqQfwxw7+5iJHZY=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=sAa4SptRDNFzJ4oCdl/2NXzeAlw0aqtimXeU/T7UfnbIoUQ0Yz1b7ZtKyvXpb2cWyJVmBpNKpA4XL58yfn4t0ZdKpts+5vwegrds1pdtiaY4J0kZKLFBnFfT2FWtffvjw9GqtTroFHcNAvzsbLsSssygqLBgUDOG3zM2KtTCYgU= X-Yahoo-Newman-Id: 922453.9063.bm@smtp205.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: AWoEOqUVM1k.5N8Z5hAbDR6humzhSxzLJxwQSGRCGr1G_5Q 0eLJmqMMh_wmKxRwsokDdwdlUAFwIKo9olLs9TTP6Lchknm1fSm.d2g4o7ht XmFbmT9.05OQAFkOKYXagtvoSzGF4678oXCaBqKbu8N_KyTvevDOkkU7Wfqk 83Vvu8siqctHADzAZICe15JmGT33IVGKT3NYGpaXkhZ.On.HuDZ1T.B0xcFH I5fdPMUcgGK.oikg56zDG1uEy5rtrf8_CxeUwPk_Yj8IrZHJBHebXh9clHCc PGBN3M9iO2s1vMhi_PoZOhRQSBwmVVxV7NzI_U25JSEa.mnYnU2.OX8gtzTQ HlKi_HfVDmEi9JnWgPHiucSOzsTaoFMl8Lk9sqGZVSLLwbL3CYFYP8V90x.o KRgH.FlulPltWG.giOvUcokND3cZpF1NahTQ16sYgvpEjL4E._tQUwi_n5Hw emQ7e4Pk82DDiKRsQOXZFWMKT_BVhNQxMgCjerFgtiFN86Ga.HFCOqu4LcVe 5rWJmiXZk8.OxGUxwI_cM_ExFhSNYvWy6UorznaJRmro_lrwQYoBF1KOhlVE GVbAXTK4y X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from lgmac-bvermilion.corp.netflix.com (scott4long@69.53.236.251 with plain [98.139.211.125]) by smtp205.mail.ne1.yahoo.com with SMTP; 26 Feb 2014 13:13:40 -0800 PST Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: TSO From: Scott Long In-Reply-To: Date: Wed, 26 Feb 2014 13:13:36 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140226180736.GV92037@funkthat.com> To: Jack Vogel X-Mailer: Apple Mail (2.1827) Cc: FreeBSD Net , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 21:13:48 -0000 Are you proposing that the network stack track the physical memory = segment details of the mbufs as they are formed and chained together? Scott On Feb 26, 2014, at 10:27 AM, Jack Vogel wrote: > Drivers have to work with whatever the requirements/limitations of the > hardware, > if you have a 5 lb sack you shouldn't be surprised if some drops when = you > shove > 6 lbs at it :) >=20 > Why not have this limit in the interface so the stack can avoid = exceeding > it? >=20 > Jack >=20 >=20 >=20 >=20 > On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney = wrote: >=20 >> Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: >>> I'm reading (almost) all mailing emails in mailig list... >>>=20 >>> Almost every / many problem in network performancr / packets loss = ended >> up >>> suggesting disabling TSO. >>>=20 >>> I wonder why.. Is it a bug in the implementation? Or bybdesign? >>> What are the usecases that TSO is needed? Myabe it should be = disabled bt >>> default? >>=20 >> It looks like most of the problems are in drivers that don't handle >> packets with a large number of segments properly... The problem is >> that some drivers limit to how segments a packet can be broken into, = and >> then if they receive such a packet, instead of doing their darnest to >> deliver it, they drop it... >>=20 >> There are some patches that help address the issue... >>=20 >> Drivers should complain more loudly when a packet gets dropped by the >> driver, since it is likely that the OS may retry the same packet, >> just to have it fail, though sometimes it'll try a different set, and >> it might go through, so all the user may notice is a slight lag if >> they notice anything at all... >>=20 >> -- >> John-Mark Gurney Voice: +1 415 225 5579 >>=20 >> "All that I will do, has been done, All that I have, has not." >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 21:24:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1A20148 for ; Wed, 26 Feb 2014 21:24:13 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 897F91D73 for ; Wed, 26 Feb 2014 21:24:13 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1QLOCL8041069 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 26 Feb 2014 13:24:13 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1QLOCNY041068; Wed, 26 Feb 2014 13:24:12 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Feb 2014 13:24:12 -0800 From: John-Mark Gurney To: Jack Vogel Subject: Re: TSO Message-ID: <20140226212412.GZ92037@funkthat.com> Mail-Followup-To: Jack Vogel , Sami Halabi , FreeBSD Net References: <20140226180736.GV92037@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Feb 2014 13:24:13 -0800 (PST) Cc: FreeBSD Net , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 21:24:13 -0000 Jack Vogel wrote this message on Wed, Feb 26, 2014 at 10:27 -0800: > Drivers have to work with whatever the requirements/limitations of the > hardware, > if you have a 5 lb sack you shouldn't be surprised if some drops when you > shove > 6 lbs at it :) But right now, when that happens, the nic just drops it instead of telling the kernel to stop giving 6 lbs sacks.. :) It's only after a large amount of work by various people did we even find out that this is what was happening... > Why not have this limit in the interface so the stack can avoid exceeding > it? One of the patches proposed does that, though I hope that ALL drivers will be properly updated when the patch hits the tree... > On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney wrote: > > > Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: > > > I'm reading (almost) all mailing emails in mailig list... > > > > > > Almost every / many problem in network performancr / packets loss ended > > up > > > suggesting disabling TSO. > > > > > > I wonder why.. Is it a bug in the implementation? Or bybdesign? > > > What are the usecases that TSO is needed? Myabe it should be disabled bt > > > default? > > > > It looks like most of the problems are in drivers that don't handle > > packets with a large number of segments properly... The problem is > > that some drivers limit to how segments a packet can be broken into, and > > then if they receive such a packet, instead of doing their darnest to > > deliver it, they drop it... > > > > There are some patches that help address the issue... > > > > Drivers should complain more loudly when a packet gets dropped by the > > driver, since it is likely that the OS may retry the same packet, > > just to have it fail, though sometimes it'll try a different set, and > > it might go through, so all the user may notice is a slight lag if > > they notice anything at all... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 21:28:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE4F1451 for ; Wed, 26 Feb 2014 21:28:21 +0000 (UTC) Received: from mail-vc0-x236.google.com (mail-vc0-x236.google.com [IPv6:2607:f8b0:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 791141DAE for ; Wed, 26 Feb 2014 21:28:21 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id id10so1579064vcb.13 for ; Wed, 26 Feb 2014 13:28:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pTd68SAq+azO3ZQZiW2x2sZApI+lMnSF9S6dctMpy0Y=; b=qLR9MylIFGt4TKR7dywVlDGTaAukghtUGdVgbPp7TW2D1vOjmgga8y7VqoddscWQJD HezRF0LiA9PQIQbrfYGgY6tQRefQ2ucW+5Pamqt+EGUxnF4XS/FN9dCKqIp8ENEGqEjl 7mRR/+xCOYrNVIgFWeX+VQOU7ePhWWRdvXmrUD4QofpgNoepxXr0MNi0xb5+bQGM9Dem jt1q8nuZPdswvYKvpn+5a62jJ4b5/QVriGoNJJL1o6wmiY7lHrbihE3FdyoMKtk8hvcu PCdvIYBiNs0lFoGd7x59xIJK6aFZE+pU1lfsbnpGh3dVfL6fQcU/eRGn1slTZRDvc4sp hSbQ== MIME-Version: 1.0 X-Received: by 10.220.109.1 with SMTP id h1mr7591850vcp.20.1393450100568; Wed, 26 Feb 2014 13:28:20 -0800 (PST) Received: by 10.221.11.135 with HTTP; Wed, 26 Feb 2014 13:28:20 -0800 (PST) In-Reply-To: References: <20140226180736.GV92037@funkthat.com> Date: Wed, 26 Feb 2014 13:28:20 -0800 Message-ID: Subject: Re: TSO From: Jack Vogel To: Scott Long Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 21:28:21 -0000 Nah that wouldn't be very practical would it :) I was thinking the max segment value could be kept in the interface struct, but as I think about it I guess that wouldn't really help. So, you have other ideas Scott?? Jack On Wed, Feb 26, 2014 at 1:13 PM, Scott Long wrote: > Are you proposing that the network stack track the physical memory segment > details of the mbufs as they are formed and chained together? > > Scott > > On Feb 26, 2014, at 10:27 AM, Jack Vogel wrote: > > > Drivers have to work with whatever the requirements/limitations of the > > hardware, > > if you have a 5 lb sack you shouldn't be surprised if some drops when you > > shove > > 6 lbs at it :) > > > > Why not have this limit in the interface so the stack can avoid exceeding > > it? > > > > Jack > > > > > > > > > > On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney > wrote: > > > >> Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: > >>> I'm reading (almost) all mailing emails in mailig list... > >>> > >>> Almost every / many problem in network performancr / packets loss ended > >> up > >>> suggesting disabling TSO. > >>> > >>> I wonder why.. Is it a bug in the implementation? Or bybdesign? > >>> What are the usecases that TSO is needed? Myabe it should be disabled > bt > >>> default? > >> > >> It looks like most of the problems are in drivers that don't handle > >> packets with a large number of segments properly... The problem is > >> that some drivers limit to how segments a packet can be broken into, and > >> then if they receive such a packet, instead of doing their darnest to > >> deliver it, they drop it... > >> > >> There are some patches that help address the issue... > >> > >> Drivers should complain more loudly when a packet gets dropped by the > >> driver, since it is likely that the OS may retry the same packet, > >> just to have it fail, though sometimes it'll try a different set, and > >> it might go through, so all the user may notice is a slight lag if > >> they notice anything at all... > >> > >> -- > >> John-Mark Gurney Voice: +1 415 225 5579 > >> > >> "All that I will do, has been done, All that I have, has not." > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 22:47:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49E6CFC6 for ; Wed, 26 Feb 2014 22:47:35 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 220FE159B for ; Wed, 26 Feb 2014 22:47:34 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.97,550,1389772800"; d="scan'208,217";a="105149062" Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx11-out.netapp.com with ESMTP; 26 Feb 2014 14:47:17 -0800 Received: from SACEXCMBX03-PRD.hq.netapp.com ([169.254.5.58]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Wed, 26 Feb 2014 14:47:17 -0800 From: "Son, Sonny" To: "freebsd-net (freebsd-net@freebsd.org)" Subject: socket lock Thread-Topic: socket lock Thread-Index: Ac8zQCCXLObL7+kFTXuTI0rB9CfzMw== Date: Wed, 26 Feb 2014 22:47:16 +0000 Message-ID: <3967D1AB1B4E6F4CA4E2329EEB83ED3E30D8C3A7@SACEXCMBX03-PRD.hq.netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:47:35 -0000 Hi all, Can somebody explain me how socket data structure-i.e. 'struct socket'-is p= rotected in FreeBSD? It seems that socket is accessed and modified without = lock in some places. As an instance, the following code reads and/or modifi= es various socket fields including so_error without socket lock held: int sosend_dgram(struct socket *so, struct sockaddr *addr, struct uio *uio, struct mbuf *top, struct mbuf *control, int flags, struct thread *td) { long space, resid; int clen =3D 0, error, dontroute; #ifdef ZERO_COPY_SOCKETS int atomic =3D sosendallatonce(so) || top; #endif KASSERT(so->so_type =3D=3D SOCK_DGRAM, ("sodgram_send: !SOCK_DGRAM"= )); KASSERT(so->so_proto->pr_flags & PR_ATOMIC, ("sodgram_send: !PR_ATOMIC")); if (uio !=3D NULL) resid =3D uio->uio_resid; else resid =3D top->m_pkthdr.len; /* * In theory resid should be unsigned. However, space must be * signed, as it might be less than 0 if we over-committed, and we * must use a signed comparison of space and resid. On the other * hand, a negative resid causes us to loop sending 0-length * segments to the protocol. */ if (resid < 0) { error =3D EINVAL; goto out; } dontroute =3D (flags & MSG_DONTROUTE) && (so->so_options & SO_DONTROUTE) =3D= =3D 0; if (td !=3D NULL) td->td_ru.ru_msgsnd++; if (control !=3D NULL) clen =3D control->m_len; SOCKBUF_LOCK(&so->so_snd); if (so->so_snd.sb_state & SBS_CANTSENDMORE) { SOCKBUF_UNLOCK(&so->so_snd); error =3D EPIPE; goto out; } if (so->so_error) { error =3D so->so_error; so->so_error =3D 0; <=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D we do have socket's send buffer lock but not socket lock (, which is so= cket recv buffer lock) SOCKBUF_UNLOCK(&so->so_snd); goto out; } I am sorry if this was already discussed before... Thank you! From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 22:53:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D504C212 for ; Wed, 26 Feb 2014 22:53:38 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 756E31641 for ; Wed, 26 Feb 2014 22:53:38 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1QMrauJ042402 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 26 Feb 2014 14:53:37 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1QMrab3042401; Wed, 26 Feb 2014 14:53:36 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Feb 2014 14:53:36 -0800 From: John-Mark Gurney To: "Son, Sonny" Subject: Re: socket lock Message-ID: <20140226225336.GG92037@funkthat.com> Mail-Followup-To: "Son, Sonny" , "freebsd-net (freebsd-net@freebsd.org)" References: <3967D1AB1B4E6F4CA4E2329EEB83ED3E30D8C3A7@SACEXCMBX03-PRD.hq.netapp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3967D1AB1B4E6F4CA4E2329EEB83ED3E30D8C3A7@SACEXCMBX03-PRD.hq.netapp.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Feb 2014 14:53:37 -0800 (PST) Cc: "freebsd-net \(freebsd-net@freebsd.org\)" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:53:38 -0000 Son, Sonny wrote this message on Wed, Feb 26, 2014 at 22:47 +0000: > Can somebody explain me how socket data structure-i.e. 'struct socket'-is protected in FreeBSD? It seems that socket is accessed and modified without lock in some places. As an instance, the following code reads and/or modifies various socket fields including so_error without socket lock held: Have you seen this in sys/socketvar.h? /*- * Locking key to struct socket: * (a) constant after allocation, no locking required. * (b) locked by SOCK_LOCK(so). * (c) locked by SOCKBUF_LOCK(&so->so_rcv). * (d) locked by SOCKBUF_LOCK(&so->so_snd). * (e) locked by ACCEPT_LOCK(). * (f) not locked since integer reads/writes are atomic. * (g) used only as a sleep/wakeup address, no value. * (h) locked by global mutex so_global_mtx. */ > int > sosend_dgram(struct socket *so, struct sockaddr *addr, struct uio *uio, > struct mbuf *top, struct mbuf *control, int flags, struct thread *td) > { [...] > if (so->so_error) { > error = so->so_error; > so->so_error = 0; <=========== we do have socket's send buffer lock but not socket lock (, which is socket recv buffer lock) > SOCKBUF_UNLOCK(&so->so_snd); > goto out; > } So, so_error is: u_short so_error; /* (f) error affecting connection */ and f is: * (f) not locked since integer reads/writes are atomic. Though, I'm not so sure u_short counts as an integer.. it maybe should be something like u_register_t, but not sure about this... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 23:28:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27D8996D for ; Wed, 26 Feb 2014 23:28:51 +0000 (UTC) Received: from nm10-vm5.bullet.mail.ne1.yahoo.com (nm10-vm5.bullet.mail.ne1.yahoo.com [98.138.91.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9FE9218A1 for ; Wed, 26 Feb 2014 23:28:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=gcom1024; t=1393457323; bh=YzZXBD/2V3W2yucHIvgfrY3sx1p6JjZR2JmJfilXWw8=; h=Received:Received:Received:DKIM-Signature:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=RyWW6aUXx8UWtlyhZl3+WmIUza5OO3HM/CdfJ2hIPEaJDWWZIuZpAbjSrm9P2GKnanS6LTT5p05qqJ/CvzXXneVuE/uvSBOxCi3X23oNCPtQVxMTngh6mr4qrE5a2Shz/CAsoLYFouRvN+2ozCTVZClKz+ZuyaAXjEC54dpyQ8o= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=gcom1024; d=yahoo.com; b=VTm31CkrnCE8EpsINHsT8fIQUGxuGPt66w+Zm9rKlMh1qV6nvnz5EbLHdMGEgzXD3Z9rZip3TBcNVubGJBbHRr1EDEzpYscxBsvet2J+8h6HDmxp0nQXCMLzQTuiAzmASXigCSMcyWRDDL8laM0Me2lzON4m5xc9FWpFnKvklkE=; Received: from [98.138.101.132] by nm10.bullet.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 23:28:43 -0000 Received: from [98.138.84.38] by tm20.bullet.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 23:28:43 -0000 Received: from [127.0.0.1] by smtp106.mail.ne1.yahoo.com with NNFMP; 26 Feb 2014 23:28:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1393457323; bh=YzZXBD/2V3W2yucHIvgfrY3sx1p6JjZR2JmJfilXWw8=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=peSUt+m66yaoRM7+PpV8DKazi6gBeqnEUA3WsvidLUGM3kxBIfkzRprjfuikYRgv0TkcCbgfXqcI2MlyOu1Kfuv6yiPmzQJBYnqdQ3R7evEFsZZNbVeKVPZMWVJmcYupwQTyQWrt4z35sXP3kUey11J9XqSscGgIsPjq4eCUkPM= X-Yahoo-Newman-Id: 352002.30008.bm@smtp106.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: kGJrw4cVM1lOx7T.eMASgxOLqC9z6dhZWC67QInJQYoEOyp KMa3wfU4C8pR9VGIRzOyOnxFbXpXsSRudDXsVkhaYRnnMQWdfPfsUVCZiR.Z _2YBa6XghGIh4TFewj2NnOOuW3szazqBFCKtIaoMHmXGLCiwla5zWT1mk2ak AWGGdmpJ9UoiOitiVsZlXjgXHbeCKkoO76wzLvR3SNN3Sy0i14x1uEWYtFJr CZ7Ljn4X5mMLxZH6yn6YMcd0DDVxUQ25PY_NXcxfM0NMymMKwdd9KUk6PqHL VcfPZhNWEcqCzuzBeWPDvZNQoZKGb_95XbfUqmqt0RIF0xCZmQciyWF.RoKM 8blAbYEArZJeFzqVDmnKQLXZ8lrib.F.hgze3mzitJ6njln98O.G2RLHXQzP QKJjitFlYhrAGP.SVhoMm0oqEG8AxgasCb_fRtx9BTM79iL3Iv2ANvZXxnvP k0F7Zw8rbQGiFhnQCdrNAlzntnZtYFvGUTZRXw.9bwukL3Rcgo4DIps8JB2S rNvHLifjhdIF9X1y2YZFMTWZf1ulMUJNBRNnfwOkZy4LrEyb76o89exIZVVW YG5DtYKZ9DEDMAg-- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from lgmac-bvermilion.corp.netflix.com (scott4long@69.53.236.251 with plain [98.139.211.125]) by smtp106.mail.ne1.yahoo.com with SMTP; 26 Feb 2014 15:28:43 -0800 PST Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: TSO From: Scott Long In-Reply-To: <530E3211.6020805@vangyzen.net> Date: Wed, 26 Feb 2014 15:28:37 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <3F5216B4-2434-45BF-B83C-1D4E1643EAF6@yahoo.com> References: <530E3211.6020805@vangyzen.net> To: Eric van Gyzen X-Mailer: Apple Mail (2.1827) Cc: FreeBSD Net , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 23:28:51 -0000 On Feb 26, 2014, at 10:27 AM, Eric van Gyzen wrote: > On 02/26/2014 11:37, Sami Halabi wrote: >> Hi, >> I'm reading (almost) all mailing emails in mailig list... >>=20 >> Almost every / many problem in network performancr / packets loss = ended up >> suggesting disabling TSO. >>=20 >> I wonder why.. Is it a bug in the implementation? Or bybdesign? >> What are the usecases that TSO is needed? Myabe it should be = disabled bt >> default? >=20 > In some cases, I have disabled TSO to [successfully] work around a bug > in a particular NIC's firmware or hardware, usually a low-end = "desktop" > gigabit NIC. >=20 The same thing happened 10-15 years ago with TCP Checksum offload. = Hardware support appeared, software support appeared, problems cropped up in = both, and it became an iterative process of identifying, disabling, and fixing. = It=92s now a solid feature that works without question (though there was a paper recently = claiming that hardware CSUM offload doesn=92t matter anymore for performance; = that=92s a different topic of discussion). Scott From owner-freebsd-net@FreeBSD.ORG Wed Feb 26 23:53:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5B88EB8 for ; Wed, 26 Feb 2014 23:53:04 +0000 (UTC) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8BACE1AE8 for ; Wed, 26 Feb 2014 23:53:04 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.97,551,1389772800"; d="scan'208";a="73327679" Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx2-out.netapp.com with ESMTP; 26 Feb 2014 15:52:49 -0800 Received: from SACEXCMBX03-PRD.hq.netapp.com ([169.254.5.58]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Wed, 26 Feb 2014 15:52:50 -0800 From: "Son, Sonny" To: John-Mark Gurney Subject: RE: socket lock Thread-Topic: socket lock Thread-Index: Ac8zQCCXLObL7+kFTXuTI0rB9CfzMwASIJkAABCsxeA= Date: Wed, 26 Feb 2014 23:52:48 +0000 Message-ID: <3967D1AB1B4E6F4CA4E2329EEB83ED3E30D8C43F@SACEXCMBX03-PRD.hq.netapp.com> References: <3967D1AB1B4E6F4CA4E2329EEB83ED3E30D8C3A7@SACEXCMBX03-PRD.hq.netapp.com> <20140226225336.GG92037@funkthat.com> In-Reply-To: <20140226225336.GG92037@funkthat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net \(freebsd-net@freebsd.org\)" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 23:53:04 -0000 Thanks John-Mark. That explains. I will do more research and bug you guys a= gain if I need help. Regards... -----Original Message----- From: John-Mark Gurney [mailto:jmg@funkthat.com]=20 Sent: Wednesday, February 26, 2014 2:54 PM To: Son, Sonny Cc: freebsd-net (freebsd-net@freebsd.org) Subject: Re: socket lock Son, Sonny wrote this message on Wed, Feb 26, 2014 at 22:47 +0000: > Can somebody explain me how socket data structure-i.e. 'struct socket'-is= protected in FreeBSD? It seems that socket is accessed and modified withou= t lock in some places. As an instance, the following code reads and/or modi= fies various socket fields including so_error without socket lock held: Have you seen this in sys/socketvar.h? /*- * Locking key to struct socket: * (a) constant after allocation, no locking required. * (b) locked by SOCK_LOCK(so). * (c) locked by SOCKBUF_LOCK(&so->so_rcv). * (d) locked by SOCKBUF_LOCK(&so->so_snd). * (e) locked by ACCEPT_LOCK(). * (f) not locked since integer reads/writes are atomic. * (g) used only as a sleep/wakeup address, no value. * (h) locked by global mutex so_global_mtx. */ > int > sosend_dgram(struct socket *so, struct sockaddr *addr, struct uio *uio, > struct mbuf *top, struct mbuf *control, int flags, struct thread=20 > *td) { [...] > if (so->so_error) { > error =3D so->so_error; > so->so_error =3D 0; <=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D we do have socket's send buffer lock but not socket lock (, which is so= cket recv buffer lock) > SOCKBUF_UNLOCK(&so->so_snd); > goto out; > } So, so_error is: u_short so_error; /* (f) error affecting connection *= / and f is: * (f) not locked since integer reads/writes are atomic. Though, I'm not so sure u_short counts as an integer.. it maybe should be s= omething like u_register_t, but not sure about this... --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 00:49:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6268F842 for ; Thu, 27 Feb 2014 00:49:52 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 20D5D101E for ; Thu, 27 Feb 2014 00:49:51 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEABGLDlODaFve/2dsb2JhbABag0FXgwOnPZVwT4EtdIInAQEBAwEBAQEgKyALGw4KAgINGQIjBgEJFBIGCAcEARwEh0QDCQgNqQaZKA2HMxeBKYsVgTQLBQIBGzQHgm6BSQSFa4NejBdpBoMZiy+FR4NLHjF8QQ X-IronPort-AV: E=Sophos;i="4.97,551,1389762000"; d="scan'208";a="100707786" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 26 Feb 2014 19:49:44 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D8DC3B4035; Wed, 26 Feb 2014 19:49:44 -0500 (EST) Date: Wed, 26 Feb 2014 19:49:44 -0500 (EST) From: Rick Macklem To: Jack Vogel Message-ID: <1445354566.13933652.1393462184879.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: TSO MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - IE8 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Sami Halabi , Scott Long X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 00:49:52 -0000 Jack Vogel wrote: > Nah that wouldn't be very practical would it :) I was thinking the > max > segment value could be > kept in the interface struct, but as I think about it I guess that > wouldn't > really help. So, you > have other ideas Scott?? > > Jack > I won't pretend to be Scott, but I have an untested patch that allows the device to specify its maximum # of segments for transmit (something like if_hw_maxtsoseg to go along with if_hw_maxtso) and then tcp_output() uses that along with if_hw_maxtso to limit the segment size handed to the device. (I don't have any hardware that does TSO, so I can't do anything more with it. If anyone wants to look at it, email and I'll send you a copy.) John-Mark Gurney looked at it and mentioned a concern w.r.t. "fixing the drivers that could handle long mbuf chains". I think this means that the default would need to be large instead of 30. rick > > > On Wed, Feb 26, 2014 at 1:13 PM, Scott Long > wrote: > > > Are you proposing that the network stack track the physical memory > > segment > > details of the mbufs as they are formed and chained together? > > > > Scott > > > > On Feb 26, 2014, at 10:27 AM, Jack Vogel wrote: > > > > > Drivers have to work with whatever the requirements/limitations > > > of the > > > hardware, > > > if you have a 5 lb sack you shouldn't be surprised if some drops > > > when you > > > shove > > > 6 lbs at it :) > > > > > > Why not have this limit in the interface so the stack can avoid > > > exceeding > > > it? > > > > > > Jack > > > > > > > > > > > > > > > On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney > > > > > wrote: > > > > > >> Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 > > >> +0200: > > >>> I'm reading (almost) all mailing emails in mailig list... > > >>> > > >>> Almost every / many problem in network performancr / packets > > >>> loss ended > > >> up > > >>> suggesting disabling TSO. > > >>> > > >>> I wonder why.. Is it a bug in the implementation? Or bybdesign? > > >>> What are the usecases that TSO is needed? Myabe it should be > > >>> disabled > > bt > > >>> default? > > >> > > >> It looks like most of the problems are in drivers that don't > > >> handle > > >> packets with a large number of segments properly... The problem > > >> is > > >> that some drivers limit to how segments a packet can be broken > > >> into, and > > >> then if they receive such a packet, instead of doing their > > >> darnest to > > >> deliver it, they drop it... > > >> > > >> There are some patches that help address the issue... > > >> > > >> Drivers should complain more loudly when a packet gets dropped > > >> by the > > >> driver, since it is likely that the OS may retry the same > > >> packet, > > >> just to have it fail, though sometimes it'll try a different > > >> set, and > > >> it might go through, so all the user may notice is a slight lag > > >> if > > >> they notice anything at all... > > >> > > >> -- > > >> John-Mark Gurney Voice: +1 415 225 > > >> 5579 > > >> > > >> "All that I will do, has been done, All that I have, has > > >> not." > > >> _______________________________________________ > > >> freebsd-net@freebsd.org mailing list > > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > > >> To unsubscribe, send any mail to > > >> "freebsd-net-unsubscribe@freebsd.org" > > >> > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > "freebsd-net-unsubscribe@freebsd.org" > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 01:01:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FFEAFEF; Thu, 27 Feb 2014 01:01:41 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 290CF1196; Thu, 27 Feb 2014 01:01:40 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAJKNDlODaFve/2dsb2JhbABag0FXgwO9LU+BLXSCJwEBAQQBAQEgKyALGw4KAgINGQIpAQkmBggHBAEcBIdYDakNoGgXgSmMSRACARs0B4JugUkEiUmMF4QIkHaDSx4xewc7 X-IronPort-AV: E=Sophos;i="4.97,551,1389762000"; d="scan'208";a="100163922" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 26 Feb 2014 20:00:31 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id DAADBB3F12; Wed, 26 Feb 2014 20:00:31 -0500 (EST) Date: Wed, 26 Feb 2014 20:00:31 -0500 (EST) From: Rick Macklem To: John Baldwin Message-ID: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> In-Reply-To: <201402261132.09203.jhb@freebsd.org> Subject: Re: Network loss MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - IE8 (Win)/7.2.1_GA_2790) Cc: Johan Kooijman , freebsd-net@freebsd.org, Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 01:01:41 -0000 John Baldwin wrote: > On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > > Hi all, > > > > I have a weird situation here where I can't get my head around. > > > > One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in > > a while > > the Linux clients loose their NFS connection: > > > > Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > > timed out > > > > Not all boxes, just one out of the cluster. The weird part is that > > when I > > try to ping a Linux client from the FreeBSD box, I have between 10 > > and 30% > > packetloss - all day long, no specific timeframe. If I ping the > > Linux > > clients - no loss. If I ping back from the Linux clients to FBSD > > box - no > > loss. > > > > The errors I get when pinging a Linux client is this one: > > ping: sendto: File too large > > EFBIG is sometimes used for drivers when a packet takes too many > scatter/gather entries. Since you mentioned NFS, one thing you can > try is to > disable TSO on the intertface you are using for NFS to see if that > "fixes" it. > And please email if you try it and let us know if it helps. I've think I've figured out how 64K NFS read replies can do this, but I'll admit "ping" is a mystery? (Doesn't it just send a single packet that would be in a single mbuf?) I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I don't know if it can happen for an mbuf chain with < 32 entries? I've cc'd Jack in case he knows, rick ps: The other thing to try is setting rsize=32768,wsize=32768 options on the Linux client mounts. Either change should "fix" the long mbuf chain problem, but if one of these fixes the problem and the other one doesn't...?? > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 10:33:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05C97BD1 for ; Thu, 27 Feb 2014 10:33:09 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 17BB4171D for ; Thu, 27 Feb 2014 10:33:07 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WIyH3-0004fl-Cr for freebsd-net@freebsd.org; Thu, 27 Feb 2014 11:33:05 +0100 Received: from col74-1-88-183-113-97.fbx.proxad.net ([88.183.113.97]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 27 Feb 2014 11:33:05 +0100 Received: from jcharbon by col74-1-88-183-113-97.fbx.proxad.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 27 Feb 2014 11:33:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Thu, 27 Feb 2014 11:32:47 +0100 Lines: 872 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------080207040208060206070203" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: col74-1-88-183-113-97.fbx.proxad.net User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 10:33:09 -0000 This is a multi-part message in MIME format. --------------080207040208060206070203 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, On 07/11/13 14:55, Julien Charbon wrote: > On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon > wrote: >> just a follow-up of vBSDCon discussions about FreeBSD TCP >> performances with short-lived connections. In summary: >> >> I have put technical and how-to-repeat details in below PR: >> >> kern/183659: TCP stack lock contention with short-lived connections >> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >> >> We are currently working on this performance improvement effort; it >> will impact only the TCP locking strategy not the TCP stack logic >> itself. We will share on freebsd-net the patches we made for >> reviewing and improvement propositions; anyway this change might also >> require enough eyeballs to avoid tricky race conditions introduction >> in TCP stack. Just a follow-up on this TCP performance improvements task: 1. Our first related patch has been applied in HEAD: Decrease lock contention within the TCP accept case by removing the INP_INFO lock from tcp_usr_accept. http://svnweb.freebsd.org/base?view=revision&revision=261242 Thanks to reviewers. 2. We studied an another lock contention related to INP_INFO when TCP connections in TIME_WAIT state are cleaned-up. The context: This lock contention was found when checking why our Intel 10G was dropping packets in reception even with plenty of free bandwidth. To study this issue we computed the distribution of TCP connection time lengths (i.e. time between the first SYN and the last ACK for a given TCP session) at the maximum rate of TCP connections per second _without_ a single packet drop using FreeBSD 10.0-RELEASE (see joined conntime-bsd10.0-release.pdf) In this graph, in X you have time in second where the SYN was received, and in Y you have TCP session duration in millisecond (i.e. the difference between first SYN received time and last ACK send time). As you can see at some point every 500ms the TCP connection time raises in spikes. This periodicity led us to tcp_slowtimo() that calls tcp_tw_2msl_scan() with the INP_INFO lock taken. Our theory is: Every 500ms there is a competition for the INP_INFO lock between tcp_slowtimo() and tcp_input(), and tcp_input() is indeed directly called by the NIC RX interruption handler. Then, during the whole duration of tcp_tw_2msl_scan() call, packets are no more dequeued from NIC RX rings, and once all RX rings are full the NIC starts to drop packets in reception. The calculus of time needed to filled-up all available RX rings descriptors follows this theory: - 40k TCP connections per second (with 5 packets received per TCP connection) = 200k packets per second - We use 4 RX queues of 2048 descriptors each = 8192 RX descriptors overall Time to filled-up all available NIC descriptors at 200k packet per second: 8192/200000 = 40.96 milliseconds Which is coherent with what we see on the conntime-bsd10.0-release.pdf graph. To confirm this theory, we introduced a new lock (see joined patch tw-lock.patch) to protect the TIME_WAIT global list instead of using INP_INFO, and now the TIME_WAIT states are cleanup one by one in order to prioritize the NIC interruption handler against tcp_tw_2msl_scan(). See joined conntime-bsd10.0-patched.pdf: No more spikes and the maximum TCP connection rate without dropping a single packet becomes: - FreeBSD 10.0: 40k - FreeBSD 10.0 + patch: 53k Obviously, to mitigate this lock contention there are various solutions: - Introduce a new time-wait lock as proposed in joined patch - Call tcp_tw_2msl_scan() more often in case of high workload - Use INP_INFO_TRY_WLOCK() in tcp_tw_2msl_scan() to clean-up time-wait objects only when nobody else handles INP_INFO lock - Etc. The strategy being to prioritize packet reception over time-wait objects cleaned-up as: - we hate dropping packet in reception when the bandwidth is far from being full - the maximum of used time-wait objects is configurable (net.inet.tcp.maxtcptw) - in case of time-wait objects memory exhaustion, the current behavior is already optimal: The oldest time-wait object is recycled and directly reused. We picked the time-wait lock way because it suits well our long-term strategy to completely mitigate the INP_INFO lock contention everywhere in TCP stack. Any thoughts on this particular behavior? -- Julien --------------080207040208060206070203 Content-Type: application/pdf; name="conntime-bsd10.0-release.pdf" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="conntime-bsd10.0-release.pdf" JVBERi0xLjMgCjEgMCBvYmoKPDwKL1BhZ2VzIDIgMCBSCi9UeXBlIC9DYXRhbG9nCj4+CmVu ZG9iagoyIDAgb2JqCjw8Ci9UeXBlIC9QYWdlcwovS2lkcyBbIDMgMCBSIF0KL0NvdW50IDEK Pj4KZW5kb2JqCjMgMCBvYmoKPDwKL1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovUmVzb3Vy Y2VzIDw8Ci9YT2JqZWN0IDw8IC9JbTAgOCAwIFIgPj4KL1Byb2NTZXQgNiAwIFIgPj4KL01l ZGlhQm94IFswIDAgNjQwIDQ4MF0KL0Nyb3BCb3ggWzAgMCA2NDAgNDgwXQovQ29udGVudHMg NCAwIFIKL1RodW1iIDExIDAgUgo+PgplbmRvYmoKNCAwIG9iago8PAovTGVuZ3RoIDUgMCBS Cj4+CnN0cmVhbQpxCjY0MCAwIDAgNDgwIDAgMCBjbQovSW0wIERvClEKZW5kc3RyZWFtCmVu ZG9iago1IDAgb2JqCjMxCmVuZG9iago2IDAgb2JqClsgL1BERiAvVGV4dCAvSW1hZ2VJIF0K ZW5kb2JqCjcgMCBvYmoKPDwKPj4KZW5kb2JqCjggMCBvYmoKPDwKL1R5cGUgL1hPYmplY3QK L1N1YnR5cGUgL0ltYWdlCi9OYW1lIC9JbTAKL0ZpbHRlciBbIC9GbGF0ZURlY29kZSBdCi9X aWR0aCA2NDAKL0hlaWdodCA0ODAKL0NvbG9yU3BhY2UgMTAgMCBSCi9CaXRzUGVyQ29tcG9u ZW50IDgKL0xlbmd0aCA5IDAgUgo+PgpzdHJlYW0KeNrtneu6qyqQRWH7qCDK6e73/9uAcjOJ WRCuyRzfOdkmLhRhWkCBJSEAAAAAAAAAAAAAAAAAAADgQFLNrrYYp6vonR3wYzAqFFJvbGKn snd+wG+x8XOD7/YDgGbsp+IkZSRQIwBN4CunXElPUN33Y7R3fsBvocQnNmX7hOn6MXQAQVP0 0IPsHPoD/VCye9r+Gt/Mv39qCx/4MB+FlSfZqb+n4w/0BkFdhJHdyp/7X6A/UJlVjT923fY+ 8z9Df6AycrfTboxTfpl/g/5AT6A/0BPoD/Skjf4YPWB3u/l2fNtUL2GXDz+fyMOLGa3kuXw7 zqUHWva8PHB7+t8UK7XpLmc99x+n3Fb1dWXBGYI8RA5VatH7VSqT6Ol1JCDXsM++stzjjEgr /TFxLsJ5vZuZNWJmuKSqTtqfN74Gfyo5fVjJcxlXnSfb1OHseUWkP/ubFrN1B1zPeuw/E/BN 5W6nVgZxHiL9mfMyk3hV+9UfbP7ycgUYnUF+V4vVSn/yD7s3nZnDS04O62V+FjS0PKbuY0/S xa9kT6YOF6rucUsbve0ogMezBqc8dSP5aRL3KA+Pf39ejPsnvLyPS4+tmUcZk7b6U5aAcnkM xA+3uNk4d7NAf4z5VPvqj7PpH2NP+tWv7k8m3+mPb0fiZ2cl9uibagCV7dN/a5QoeJQH8yWw 0Wf6836Qu/SXJ4MycEWw6Wb8zKpr2M0uwQ/zrFtvu032ze+IDsKnbJdb648Lpi3Bppop4TaO 3eIoU93aXVPZ48ijMuOZxOu84pFMCqWJN/rTM+LroZzHs57sXKtL9QLUWXajcc6iPOiZJR70 0M7TU3H5xdq/owyiItAG1evP7pJ0Pxpt1aL/Z7cJj3ZE5TijAJuOP5TxMmVkClJbtXODhT1+ dXNTVcAkMmThoeRlJcV1XYUdQBjLc574ye7tyIJt5IOzuv0aNWwwsnNdvW2N86D+W/klgzZT 0S+nRTzKIC6CNdSf3WUUflpkty048TsuB8ke4HSk6fhD2ro51WU3/vcYaNCzC6ftAWUf6I/p g23+vEJedx9jIbpJKa3Q/FnDsZK2Y+YvdPt+tLviqr81sjzP9OeewLH7H4rA68/9ogdHgly2 t/0YNYnHcpyS1u3vUTcH0m7870MH3fT1g4EEOX0a5Lb9Pf7oSLZRRt60v+zB0RKe1R78rF29 ilxnZN0veVDtKQ/XdFzaXzuiltH+oAi8GbUHPncRod0+p4fHbhvvy/nlcpC3LE3qOo0++jss jNsIO+hnf4wFng39k3OI3I0/jj+yAwAq3+hvXY+Ts6dnPRC2drX7Treg1sEXjD+ECF0rl/FH 1Pa7/UEROBvm9Me8+0cNMPzARW/bQ+kvl4PIGY1gD/0RU3VsdxthB307DMfmLZF4bN7+4n8R dL/X3zn0JXrs+nhW4v7I/J02p8alcrgWVaWH/pedXnt71v+yU/FMf74IjFt6P62sSWR3Me4U Ke227v75Hb4cTf9vRs9MF/0dw7XdbZyOZlOKkmu/7eZ7YrH/mbDI/ywP63HxP59uGz2w9v7n TRP078Tm/k4+ntX5n5XCVr6pvuG+u/sgysPhf9kfrjX0Pz/qzxUBMxu6h2cG2dLv0r8wffHK vP53bu+miOyOSznC/r3iOpLQ7qrNb0QTVHq5zjGddvx8GdZ5F5l4+BafTFUTc63lubX68S39 v1PXpvW8ntV0wEwOqZCrGkuveq+Qj3k4xSiup9fXZuffHvXni0BP4+lbQRx+wGCX7uaZKcGd 0v+O7eMy3Y6oHGd0v2D9wVtWJ+3Yz1KS330oB/p7y67sk4gmgIsD/YHXiH3l1DrgqgD9AdAD 6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+ QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0 BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+gP9AT6A/0BPoDPYH+QE+g P9AT6A/0BPoDPYH+QE+gP9AT6K8JS+8MjAr0BxrAqNSfnK4i3gH9gfpIrvXH6CZ2I0QP9Afq sxv98Z2cHx7oD1RHcN3+SsrU9sajXdAfqA5nWn+C6r4fixUH/TXhp8e/20oO/ZkxSNwBhP5A ZaSye9Af6MW6E/K6/dX8+6e28IEP81Faf/QE4w/QA6HYqBDwv4BeMPifQUfc/BvH/BsYCOiv CT/t/7sD+gM9gf5AT6A/UAIpZVY66A98DFuNg3ll6UmhP/AhelXzxhjbVsqTFQj9gc9YA7PH +JqYGvoDn8Fuvr0H+gM9gf5AIbIGwNAf+JyNEabGvxkKhP6a8N3zbztlhHN2WVr1J6A/8DGc EaEkyHh6UugPfAwVygQSpcGMpL3zDuaH74KvRK6pzj8C/YECCEqVCeRUpCeF/voz/+BECr28 PscBA/01YH6B3SAD0lNDf99IS8HTgIzUDXMKvhG98oWu6nOnWH8FDI0b/PVwPO8Y/4IenANf +P9AF/hm/tkw/wF6sNFNSLGh/wf6sOnBb/riewL9NeKrPYAaITImPwj0B/oC/bVhydw3CZj/ GJwv0NhrBMf8B+jHytlBelLorwnLV9u/nIVXNmnvvIP5yfK8HEB//ZneNgrOMP4YmukldgvW X43Od+uPMYw/QF/ywv9BfwMwv3FkHPO/oBvMrH9esf4FdOEMvLFj/d+ozN/E3oH1z6Pz3fMf WP8MemLXP2/pSaG//sxvG8365xz5QX+gCFj/PDRL9s4pYMr0iTVHgdBfEypIbCDVMroSIuH/ m5SBlJQHxr/9mV5EHwD/H+gJ7N/wfPXzb/D/DU8NjY2jW/j/QF/g/xubcWxVHTJfPw39DcD8 2tQPoIs94/VH0N/EDKNbRndGBcP447cYRn/a/0IF/C/j8t3r/7T/Wf0P//O4fLX/BfZveIbR Sg1Uz09QxrD+YE7m1yb8z7/ISLqF/3lovnz9qSHLAw39gc/ZmOoD0hXxr6Zkevun3/zGOeMZ EyDQX3+m1x9nRI1/CYP/5acYRrdUKBOI9c8jM4xWasB3wVci1xfvvxQrPVdIM04vT8lBf/3J 1eYwmhaUKhPIX4Qhl0pzR99Q+6l3Go1SoL/+ZOtoGAFKoUTFXgx/zbQc00IzIoxHKdBfG2o8 /zGM/O6R2iwyZfakmaCLZ4mhv/5MoqN77t8BIrTVE/RQYpSsd77BF7S/5I3+KOVC60+30Czq AEJ/oAi3+hPm5dTQ35h8v/0jeonM8/ZX8++f2sIHPsxHaWUeK2OU2cP4A5RHurf/vngG8whL rj/hf+nG98bfUI3q/fu3tP9Z6CUK8D/343t9fEy+e/+WWOk57ca4GQgHQH8TM7443wL9teF7 7Z8MSE8N/fVnfI3dQQMyUvfO/o/wvfaPBaSnhv76M/fYGO3vzzKEONH+zs4QMsoG7e8MVFn/ N3fDbYD+wGe8nX+7BfprQ66pGj9wwtv5t/vUvbMP6mizGe/n3+6A/vrzBb5BtL+D01wrLU+o w4+j/R2agWxVeThH+zsxVdbftxT8u7X3d0kbZhM8p0r/r6X+1gzDdwL9jc0UzyYJzjD+mJcq zSjGH+CPzO1/Jhh/TEDz939g/PFr5Nf47C+nwfjja8m2cRh//BpVmsMp+oZYfzACzfU3jP3D +oMRyHeV3KQc3/8ibr69B/orxvhSqQLffbdP7qmvYID+ilGnqcy1je1EvdF1E0IKtq3pryCE /gYgU2OjPP8ht2P+g2/pA2Dob2wmadSlEALvHxyYH45/fwv0V4wq/b/sNnYScUJ/xahS45O0 v9lAf8X41fHvR0B/A3CnsS/vHEJ/xfjp9jdn7YEG+ivGWPpril4ALfaM159Df4347vUHdGdU sPTZD+ivIL+7/lS/XZqKy5tl/gb0NzajzLHdotffq/8F1v/9FqPoD/ZvBNq3v6PYRv1eI8oY xfrTnozlDm56ws2sf8kYfkB/jcid453F/ydE3jOY0N8AzK8/xP8bnOYxNhB/A/yVUXSUC+Jv jEAdpcygP8TfGIH2U2Wj2E3E3xiB9v6XUfSH+BtzM7v/GeOPERir/9d0/QHGHzOT7X8eZWyC 8ccIjKKGMzcNs4PxxwiMNf5oeTtg/DE1VWL8If7fr9H8+csPslMWxP8bgXxrVGP9QTPw/t8p aN0BbDb+wPt/Zyf/GY8RrCP9D+3v3FSJMdRMmh84/wj0NwJ1Apc3bH8/Sd0olz9O9vij1inL QZnE+GN0qsTGr5MwFRqQkbpRLn+A+2a0dR+vnf42hvHHCLS3cUMMQND/G4R7MTRf4wL9gT9S xY0M/f0Ydca4w+vvM6C/RmTauK+YON445SY4B+N0jc0l9FeMfPtXRUbjCHCnTGw6OoyOU7TT yEsI/RWjiv6+IAC5NHGxNqpfVUjODwf014ga9m+K4JREmhdkMipPIUZRAqG/YtRpKm80NozC yB/i3++cCDNUZpHkoL9ifENXLZP38e91cHJBT0MY/A79FSN/FV9u2zzK2sD38e8ZXQn0V5cq 9i//oZKG+nsb/3nT8nve/mr+/VNb+Kj6sdzsXfIOutQ4KPmXrr938e83appmjD86UmGObZnE /m22YYb/pRdL+6UqTft/d/HvJV2FMPGh4X+uSfP1L+MMm2/j32/n4lRp5t845t+GY3b7RxD/ fgCq2L+JHs3MegMr9FeMDiGGRmmcN6b6dnTF80fjkj3GHaeX95JdjTw4ZzzjBcDQXzEGW//S EM6IGv+q0UV6UuivGGPN/7bULRXKBBK8f7UvdZ5/m2GJFd8FX4lc1/Sk0F8xsse/U2jsDkGp MoE850kk6K8Y7fU3TL9RCu1cznHAQH+NqKCjZQrjeA/014gqz58P03Aj/tAAVAkVVOPZzNLg /UcjkL0cOX9xdOYJS7Pi/UcDUKXHn7/Gvq3/Lztpu1z+MkuN9afvTtkMjvcfjUBrU5W/qLAw eP/R3FR5jrxl+4v4pyNw28RWMFXDLE1A/NPxqRHjYBT9abJWn0J/JWk+jBhGnEw7ALNGIdBf G6r04gZxv+joBqrxXSna357UWX+afcaW/pdj4fOO9ac96fD+1UEM4Ol/xvrTrnwwHK3hnG4c /4CQm/gvr4H+ilEnGmTuGr+W/b+NbkKK7Sb+1Uugv2I0j1Q/jP/vPv7BLdBfOZq/x3KU8QdB /IMRaP8A3AjvPz+B/7k3H7yqo0Y/rqUAxaqa3zXHAkJ/I5BrxwbpAFr/M55/60qNANAftOnt xGn9z3j+tyv56++zFwBmZqYw8D+PQKVXTA80xngF/M9D0Pw10nUeXEpH0N34n1n6Emjorxzz x9HNhAakJu2d9y8ivzs2+fpTFpCYFPorR/tXTA+lTsTfHZnsh0Oavzg4A8TfnZnZH39D/N0x aP/8xyAB2BB/dwjar38ZpP1F/N0hGOoFmC2DAyL+7hA0D1U6iv8F8XeHoMpzREO5WF6B+Lsj UMP/PMwagzfkxB7SQH/lGOsFIC3RAVDFnuF+gf4KUif+bvPHmtJhdGdUMDz/1pf2MTYG6Ry+ ef/5HdBfOar0/7Iz0zj+gfof/r++VAmVMZSoXwD7NwTZywiGWUeQier5CcoY4l/1pU78jQnm 3xD/YAiqPP82w/iDIP7BCNRRwwTRYU7fs8D8b1fau+qGsH87pVp4ckf8+760fv/bGM9f7nTT K08ZpTvWP3dlLB210qZ2vgi6UZ7VAYT+ylHlBdR1gvMWRDufZd7gl0B/RWltxobwv5hVf7ny g/4KUqfKR38ByKG/3DdgQn/lqDL+HT7+PfQ3CvfNYY0YVyM8/2GivlCG9192J787ljv/cbuv lf7yg78Q6K8kzc3REP6/D4K/EOivFUvrIAcjSPMPQH/laP4c5Sjr/z4A+ivIUO/4mEJ+0F9J mg9HR/D//QVJzdCY8YeXNEB/bWgfm2MgJDf608ukdxo7aaC/glTwv+THeB5icGxg9NCfic92 CdIG/ZXjXg2tY2wMJL+Naf1J83zI5SEl6K8czYcRc3T/pCRGf8JM0rFYcdDfANQJHDOMAInV n7SbHuhvAOrEeIb+fo0q77icPTab5qb91fz7p7bw8emHMmMv9xoTl3Xku4NWuY5a+sP4ozY1 3Cjto6pWgMH/0oD87lhujIO59Af/cz86xKYcTn96/o1j/q0ag61/mQLorxx13uP7BSK7Afor SPsXIN3tm0K40F9BWk+yTTP+eA30V5Aas2FDPGNeD+ivINnzGN8wxs0D+itI85dsTTMB9xLo ryCt11F9wbgZ+ivIWDqaQoDQX0GGWv8yB9BfG6rMDX/B4Bj6K0iV5z8yTziJAKG/0Zkg/v0H QH8Fyfa/NI4NMxDQX0FaV3n+otZhgP7a8MH6v/mdfDdAfwVp/RzR9OqD/opS5VHJ9pPKLYH+ ClJn/d9NcN7eF/w50F8rWo9xYf9+jt98AdxHQH8FyZ3/yD1opYQtgf4K0jzI1RQSuwX6K0i+ qaowjp1Dm9BfQfLtX/PF0aMA/RVkrDhW0N+v0TyO7vTmD/orSX4v7mcXwEB/jfjuZQTZQH8F GcvFN4Woob+CNH/JzPxzI9BfQepI5asfAIb+CpK//v4bgoxnAf0VZCgX8xzShP4K0l4q03cA ob+C1HmPUe4JZ5Af9FeS5sOIOTR2B/RXkDpqyLaNM4gT+itIlXd1zKCifKC/FEbTAtpf0JM5 3n9+B/SXwvLB7hqLA6eQ2C3QXwrv9FdjIUF+p7JRoXwE9FeQ9usPpnfOQH+NqON/nkFit0B/ BRlr/cEU4oT+UnhXo1VegNQ4XVugvxQ+GH9UWmNQ44Qtgf5S+MD+VTnp/AFQob+SNH9X5fSL U6G/FGbq/80B9JdCF/9z9glnUC7014gq72aYH+ivFVWCs00P9DcENWKsTaFb6C+FD6q0ijcE 659BSJWlKtOv8bsD+kvhE/9z65ejT2H+oL8kPml/c/dWWdQ6DNBfCp/4/2o4YND/+y2W+x9q Tv8uN/v+mtsB6aW/BhVXPdePu2u8AO435j+W82L0x+K/L/7uC+35crxQxfx/veLFX/ji/l+i 8ljOz+Uhhf7xWmzLs8RuzxKf6knJHxld3LY993JJuIQJgvwEJRBk7yiChzJYwosLD3y9rPDr 4g7rckniUiRRCYY5jS8hLHx3yWfFDilHepTpYgt3WYKMLv4Cl6g0Flf8JCo6X63XUrb1viyu as7Ki2vgeSEdfxyVoc3xQwp3J8WVZGtgcfpylxCnJbE0Xd0t/hqOcoryFGRs8SqIhPkkg9Hl 2IOeRw515OuHPLte4pS3xPdOkJkh1af1F1k4J8Ez77a2FrIsj+UVVKk3M7ZeIsNjC9dV5eI0 74qOhEmtJKMMLMTtXlxJL67cXZ4CaxfoJahZa9qdsGw9+1vjYlidqYvuHuIvy5WKO+VTox+W tM8i8cW7+MO76wtL9omWfTmHpj76Oqj1U/rzNbZcqyqwcK7yfaF6kxXezaFZsgL1Rw0qOTBq 7rZ10nM1Y08UCphE6byso4oLTZO/V9wluRx7e+H14m4ar48lrMvgMK7kgrsyKKrg+Eusvwf7 H2TQCt3dF+5rbNjdPRSUUqA0b6avbcdA0Khmn5aANylBZVz+KFBfWI9Bqx4IJ5YcCbRkizks XhJWEXFHDQRNSFTtrooC/bkOmc91bFp9ZVu7H1Wuveq46xF0V4KS8SeOO3mh/lwmSJD7oMkJ bxN/D/vCvOgvyPK1/+Tv+TH1Fxj34FqcQsLWLe7bxpYsaHjcQMbXVnDD+3RBVcV9K1/hQX2F lRLeLpHkw0bRt6Bh/yk21aFRjw1rcAWx9XP6iW4CX+FxtyO8f0ItLsFVhLpzN3tYukGnxlr5 QNP+pvWGOdSfa5/H1J/TgavooCcc2ItQhiSu6KAnF7Q6S6BLp51AfUGCqLMXtGhepRfLHCqA +Ep77HSSILthzywwaFeFLMH1hM2mPye5KiU0LmGBBTdUWEihSQvN94XAjPtrjvUX2YzgRgku OmomOsM4XcVVf75Ig75e1FeKfn9oDbwifLVeyydqxOPOvNe4s3LeFvna9bkKrVFgqn1deevi O4VRDQdmKjbYYdfMaemhW7yE5/HnCnR0yW3QNYluVG+TlleQSz/Y5eBB1D7rTw86ggIZ3cRO Zay/BZThTkdvJPZGf6UO2lt/fLcf0F95aumv3EE7y09Spj43Dv1VoYf+0vb9TSbVZmQF1X0/ Fh//qf7Iq5+ifk/QByPP0z10kx43SXToZMjjl2u/Jxi8pJyFPPviu6jhZti5Ct0yl4q33+iL XfEg1u82/9LHLD1oLKyjZwLsrj9p9Bd1AKl3P1x8mw5XSG64SS8jDD8yC37z7tKjpNyZwpGl 7537ky8P2Qt9OWGf/prRy7/RkWOBhGc7av5FsS9R1sKisomiIelNrtx3GhTqc008+dXN0/tb 6aGaroNj+/Xv2gvPVJwX+ssAiZAoneftLwAxtfT3dPwBQCue+V8AaMUz/zMAzWCccvH5YQAA AAAAAAAAAAAAAACAe548EPInZPIsysYp3xLTiDU9zXFZqdmTZuI9dWYyJ39Zc/xyzyg9XeSJ lXtWa64qksmdkJM8NdFOmdhoWhFKVQosZ7o6PXuMCkViIkH35Gsi+jwi9aJWrksvMdGuKndL Koiz3NpN02YuSGA0tYLPpTdpt71ZqcMylgPtyfrLWhS0rvpca3rCjadl71g4l1h60twZe0Ll 2mpttkwlc0GWuj9SGzi5P1n6+jaRSE+jETw5UUo1OUzpZSBTE9qFm0kXZVcb/zmBrdZ2y/Se Lkh9j5Q5qiDaLqVnMeM+5Cw5e3xVHax0VexZ/dP0quWrVCWRZmmt0fxzSdhqzVRFBk8X5P+J LP2x1L6S7q2nr9fZ1vTsUdPBShOgUOJL7/8p0jWrOmaUpjb0XBWdSOuJnPrLVUUqbfXHkktQ 99ZTVaGaD5GePTP0SDTP4uhgJRszkV52a8bozYh2TRuANNdfrqXNyN2WIb8jXdrfr3tu4WV2 sFJPla7YrNEbOVvU5BJo1/7m9zTTSz3ZfaCNX86pcp6ikSzjTDKzntKb3zyls3Sb3nj8kT/S TtZfTj/pKLtUc6H9axsVSd1GYYp8TTyTKbh0a5Y+bM6zf1rnMq3YWVv/S76nMdn/QlfjeE1P tOd4OZJvD+3g3WniUIcZB29q9kTqaUie917plYk1zdPIGvufsx8ISa3g7WwUE/2uK82bCEqf f9tzppxYutMma/hxzF4mn2lXpZfYZLv5NzwmBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AF+Pe4/Uu7XnGWvMw4X9LO95UvDlmOffmPjvf9783fbpc39rZvwX8O385fmj9BCG1wMzvEkP PEXL5Phfh4gU5oNcnlfbdPMp1C/GDLpd9kmz8191BH4+RabDOepwFi5NdgAs8OV4/XEmODcf 5hl4/WT6+TfH49m7MFGR3K7tfKZ3M8/ACnsEQs5fqHRpzgCAAFzx+jMmjB2/HPGCTs2Y6Bnm WXCmdrtdZmPdzn+5T3z+QqVLkxORBfwEXn+SuI84iqP5JlVLKkgQ4NEGJ3AxeNwR3J/YNKRN jCgwIS/0F7pljjAEOsKCDpNnd4lor0oU6M/GADrTEOgPvOCF/lgQg8aFYVEjD+J2Hb9KcWP/ bBoC/YEXPNXfMVxlp9fPxJRi/Lrr6OWtQf/PHsGETduVRbRp0P8DL3iuv2Nsa73Ox/hXv+Fh JX6XHQgfo10WHcGOf880GP+CFzzXX/wOnMP/p/pyJjS/2/Xg/3PWTr+Vxfb/TBr4/0A+WfMf EZj/AB+QM/8bkR6RDwDPhzEWsf4FAAAAAAAAAAAAABTk/wHxWWDbCmVuZHN0cmVhbQplbmRv YmoKOSAwIG9iago3NDA3CmVuZG9iagoxMCAwIG9iagpbIC9JbmRleGVkIC9EZXZpY2VSR0Ig MTA5IDEzIDAgUiBdCmVuZG9iagoxMSAwIG9iago8PAovRmlsdGVyIFsgL0ZsYXRlRGVjb2Rl IF0KL1dpZHRoIDEwNgovSGVpZ2h0IDgwCi9Db2xvclNwYWNlIDEwIDAgUgovQml0c1BlckNv bXBvbmVudCA4Ci9MZW5ndGggMTIgMCBSCj4+CnN0cmVhbQp42u2ci08c1RfH/xT/BpPGaIIa tb/Y2pYaU1OjVaPWGGNqrEl9x1TtQ4sY0lbo01KgUKmEYm0phZbn8lweC+zyfrZQCrsLC7PA sizz+8wOzFx2Z/ntsvXXjHJCpzPnnnvune+cc8+5d+6OLG/Q30Kjo6NtbW12u93r9T7qvpiP 2tvbMzIyrl69Ojw8zOX9+/enpqZ8Pp/f719cXFxYWPCt0GKQfAJxiZh2ibAqozFVJaKecM7i CmkcTU+IZrGtkCqRuheuxLCh8O4FAoFo0BsbG7NYLHV1dZOTk0tLSw0NDTMzM6oSNKh9VikQ JL9AKscZJDqgcgB/YmICJS6Xa3x83O12w5ybm/N4POpz4aiKPQgSwvPz81zSE/qAC1CF6lzO BokTBNCAzPT0NK0gc+/ePcSoSyu0rjatPTuI/qi9RQPnVNE6rMmo92jIAYoozW8pSJxQt7W1 lWP0pkvFa9euXb58ubq6+tatWyUlJWlpaefOneMcZl5e3u3bt8GwpaWlqKiI0u7u7vz8fDj1 9fXnz58/c+YMl52dnTdu3LgWpNra2uvXr5eWlmZmZoItVX7++WfqFhQUXLp0CWGeLw/ijz/+ QDNjDlWKi4sdDgcCmEFZWRmNcllVVfXXX3/Rq1OnTjU1NTU2NlZWVtbU1MChM6vA4TxqrNag daDHY8Ju6TNwJSUlJScnHz58+MKFC3AAJzs7mxvv7e3t7+8Hh8LCQnrO3YEhTKqcPn36zz// BAHQABlw464R4yQ3N5dhhAdx6NAhHgQQgR7MmzdvYktWq7UpSAij3Gaz0RbtZmVlffPNN6hK T08/cuQIJ8eOHaMij+O3337LyclBVUVFxSrHXFiQ/f5Hgl6wcWXQwLk8QcKb8Ck8BUfDrTgi o7obpfgRRx696s5YF36HbyLJUZIkBKjOCfaJWkDGwBiTaQW3RQknVJ8PErWQ5xJJihCAQxXV xzkyAjAacIIYR87hUBQ/Vg8LvQ1SaQO9eGgDvXhoA714aAO9eGgDvXhoA71YiWRJzc3UaY75 0CPXfXQdJrm9ePEiuf3IyAhIkr2ThaoTPTWn1Sh6jsYMuYySE645Ui3+C8zOBny+h969KOe5 TLdBj7nS6OgoVcrLy9VpuzrrV6fnKqnzaI3DicrRBDQOmb/KUVctxFqqWpGjrTmE6EFS46h6 NLWannk4MzNzXu/cStMPq3uRHJAZUEdHhzZbATFmMUxhzOq5TFH/jx222+1Xr15lZh1etIHe /6S6urrjx48PDQ2FF5kSvbm5VejNz8vRLGwyrK1rXaW4uDgzM7Ovry+8yJToEXNFuDiPZsBH hoqxE4MeYWJsbCy8yJTouVyy06lfRml766Xa2trU1NR/DnpANzmpX+KPD2OhOBKBT3Z2tvoO KIRMiR7QeTz65ezs32p7jY2NZ86cGRgYCC8yB3ohQRa4CBwakYlF6v/DcOqenp6CgoIHDx6E F5kDPbonggAmkqRfroEe/LjfXzAX++mnn5hiGKk3A3ohhO2J7ylAMlL/Z2aUEBMfNTc3f/fd d1q+p73NVE/Mhx5uK3ruyIhijYYEP+79EuBWUVExuRKnXC4XGWBRUVFnZycAmg89n28VJliF CKZIWKkYX2InZrWgV1paKq2MFaOjo+pb6YaGBkrNh17IuEcCs640OBrCQ3Nyco4ePTo4OKhy ZmZmwPPu3bvgaUrbAzq3W79cI2qE7BmIfQsByOTn5x86dEhDL6TUfOjhtsEX7svU3x/Rc0Mk EYvdSvHQjIwME2csIRTiuePjETEJl4x9GCRkpKWl3b9/P7zIHOiRs4muGhILKCKOGFIIenfv riOBId/LyspioAsvMgd6gCM6TggmjEjkdRqJ6XGIJPYjOnJ05PV6gW7ByLxNg97o6KpLzE8j ANHyPYICZqmFBmYo4q2RGYo2HB0xzz127JiJVwmAS1wgCsmBcUbRwNaIqtieuDgTHdXX16ek pJh4hQqvmZjQL0NmaliF5o9Ax3kkANcVNWpqampra2dFa18hc6CH7Ykhj8RDRA9/FG1vjWUB HkGMG/nIh2/cuIHtmXiVgO6J+ICe6ICdnbpFMdCBs7YgA5KiseHjYnyJgggZubm5165dmxCM P559y4+A8FycTiNGftESbDa9FNwIwRp6nIgeh+FFyqsjEDM1cFM3r6ocp9N5+/btkpISArE5 Zmrcsri0S8gQLxn3tEgKXF1duvNyX0bjVUzU1dWF82pzDdC7fPnyuXPnGAzNsZfA45kbHNT2 Esy73QuDg8t7CQKBeZttbmhoLriPWuE4ncpOg6BSn9O5dO+eP/gLDi7nXC7f9LQ/uDoX/V4C h8Nx/vz5EUbXIE1OTg4NDXV0dIAntmeCfSyzs4Hh4eVL7GtkZKm1NbAy+gQGBwMeTyAoyb+l kZFA8IcbigCuOja2FBypFAGXa2lmZvky6n0s+CaWZuK5BgmeuMThdCruqRFF2vwLz7Va9Ykb J2J8IWTEOO7JSlDqxMBMnLEwjjHQabZBRieC2damz+NAr79fn18QcHt7dcl1bV0jWz558qS5 VwlE9EgexH0RxF/NwJChVIu5GG3cK1R2u/3ChQvauGc+9LjrlhYdPTAR9xKAnnaJjMOhQ4Sk uKjCZaTVmMhEgMjPzzfxLiDQwD01i2IIEv0IuLQpPOj19OjohSxt4cixz9Tw3IMHDzY0NJgV PXDDc0X0xDl7e7uePCPDpYZeyASZpDr29T08Nz09fVRc5DEXenQPTLRO4n0iCNyXdmug19Sk oxeytIWDiy4fHVmt1qSkJBOvUNE9HEebQZDFifdCBNEcGfSQ1AY3qohRg+AizviiIBLCmzdv Jicnm/i9BpjU1emYkLaJMRenFj03JGqIM2LQi9H2mIz8/vvvubm5M0bLC6ZBr7RUn7ECAiFY I0nSYwFRg1RQu52QCTJFRluh1mw5wCyjr69Py5a9Xu/g4KD6VQdzrBKASXOzjh5OZLHopYAp TiiwLnGEFG2P+Buj54YTaXNeXl5hYeFwcPJosVjcbnf0X3VY92cTQpSs/Q0HlbOshPm+xeKf nlY4fv/C6Ohifb1PbWhpydfd7evr8wXrKpzmZl/wYwiKkvn5xeCPepb1eDwLbvdicPfOur/q QJWuri7mbpOTk+rP5GP9qoN2ucZHEsI5IUrC9YhNr+IsLPgtlsXgzgflPqemAg7Hsh4iw8CA 8hfUpnBsNn/wQxCKktnZwNjYsh7+QHJiQl1PiOerDkFvMM/qqKxk/PqbIGJuTY0+9cAZxaSO jEVbiMZzxRlWyBaOuMk06JGTaDEXEDo79aKQTeDgrKHHfVGk4UwQiXFlfm0yDXokeNpLW076 +3VMMCcxeSZj0UIw+R7ZsjhBFt+PxE2mQQ9j0zyX4Ev6pw3yeK74MyjSEi06k/iJeTUuL/p4 3GQa9Hp7dUywH3HcY84rOrK4MQNn7+nRJeH/S9ALeS3b3a2jhwO2temYMCSKi6V37+pWykBX X69LosFowvVPQ4/7JQEWUwjuWpy9iiCEvGLDSjXbw0oLC3UfR2eMcw1Tosf9MtqL6au4MV4d 9zRsiRHiuMcUWAsNjHtdXaskceR/PHrcL6OZaHvgICYe4lIzAVe0KBxZe/WDj7e3r/Jco9cT caHX0kIGvjyDJn6RXpIjcVQNoLVVYdKfqanlnQyU0geqwKcKkvxxyf3iXNwmDoJnIUxv4fOn JlrqCY6GczH9VBsiJqqboOBziQAWQmlVlaIQSbJftWkGNJIT4iYCNttyQ+ovMugk2oaGFAGG RBDjHEnUVlQoAkiq1dGG+dH/OH7axkRj+asOstyambn48cfygQPy1q1yYqL84YfK+QcfyCUl 8i+/yK++Kh8+LJ84IR8/LmdlyRkZ8sGD8r598v798rffyp9/Ln//vZyWJn/xhcIsKJCzs+XM TPmrr+RPPlGYu3crYklJcnKy/Omn8tmz8pUr8pEj8mefyTSKQpgXLy5rTkmRc3PlS5cUnW+/ Laemynv3ygkJCpOHS5Wvv5bz8uTqarmoSK6tVTTTBI2+9568Z48iic5Tp+Rdu+SPPlLaInPO yZHff1/pPBwwp/NPPKFUjMPd3G53YWGh8lUHWW48enT62WelZ56RNm+Wnn9ePXq3bJlOSHBu 3ep94w1p2zbpueekN9+Udu2SXnvNu23bVEKCa9Mm786dStGLL0qvvipt3y4lJHhfecW1devU Cy940fbUU9KTT0pPPy0lJip/u3crMi+9JL3+urRvn8L5z3+kHTu8O3ZMb9/u3LnT+8470ltv Se++K738slK6ebN3+/apzZtdjz/u3b9fKiuT9uxRmkPPl196f/xxLDHRuWWLlw6npytMNG/a RGe8x4+P7N7tSUxU+nDggHTokNLDHTskepuZ6eH42GPS3r3S5KTk9UpB8nq9TPkNd0mtgZ66 Vt/T1tZ+5469rMze0GBvbLTX19stFs7bSkqsxcUKh8vSUoVfVYWYo6rKVlbWVFLiqKlRiior 7ZzwV1vrqKtrqqiwUYRkRYVSRC1VZ12dcsKxulo5oS2qoLampq2szEoHkIHJHzopslod5eW2 ysomWrRa7a2t9qamZQ0U2WyNZWUt5eWO2lq7zWZvaVGOaEDSZmuwWNoQQycdaG5W+OXldA+x ZjicU7qaGPwNdwus7bnK+cMaQ/99NDs729PTY/jxW5idnZ2G6zZ9fX08rPCihYUFh8PhjLD6 3dbWZlg0NjZGLcOGurq6rFar4crbxMREj1H64fP5bDaby+jdGXfU29sbvqcCPh3o7u7mGBN6 Ho+Htgw/jUg3BgYGDHteX19vEVd3BfSqqqqaiGtGPU9JSQHz8CLuKCMjw220dnTr1q3MzMx5 ox/x0W3D3+/QUKSX1wxx6rdqw++UB3Hnzp3i4uIo10hVmpubwyTmIuyNUb9RGc5vaWnpFfeH rJDf70eb4esn9PAsDCFiDAFVn9FbflR1dHQY3tH4+LghqghjsVNGvyxQb9awFnfKszDEfIM2 6F9ODMuEG1ye4/DwMOMP8YLAROLEJQkD+SdHzuFXVlZSRA4mPdT1TPNSQ0MDw/Kvv/564sSJ 1NTUuiBlZ2efPn36hx9+yMrKunLlytmzZ69fv65+OfnkyZMEwb/pY7+mI8Z5TIshHaMi7mBg mBkAEheam5sZz8GKI2JYaXV1dWNjI4a6YXuRSN0/vIZA8IXtI0jn/wsJ9FgHCmVuZHN0cmVh bQplbmRvYmoKMTIgMCBvYmoKMzg5OAplbmRvYmoKMTMgMCBvYmoKPDwKL0xlbmd0aCAxNCAw IFIKPj4Kc3RyZWFtCv///wAAAKCgoP8AAADAAACA/8AA/wDu7sBAAO7uACAgwP/AIACAQKCA /4BAAP+A/wDAYADAwABggMBggACAAED/gDBggIBgAEBAQECAAAAAgIBgEIBgYIBggAAAwAAA /wBgAOOwwEDAgGCgwGDAAGDAoIAAAIAAgGAggGBgYCAgICBAQCBAgGCAIGCAYGCAgICAQCCA IICAgKCgoKDQ4MAgIACAgMBgAIDA4MBgwMCAAMCAYP9AAP9AQIDA//+AYP+AgMCgAMDAwMD/ wP8AAP8A//+AoMDAoP9gYAD/AP+AAP+gAIDg4KDg4KD/IMAAAMAAwKAgIKAg/4AgAIAgIIBA IIBAgIBgwIBg/4CAAMDAAP+AQP+gQP+gYP+gcP/AwP//AP//gP//wJ+fnx8fHz8/P7+/v39/ f19fX9/f36enp4+Pj4uLixsbGwplbmRzdHJlYW0KZW5kb2JqCjE0IDAgb2JqCjMzMAplbmRv YmoKMTUgMCBvYmoKPDwKPj4KZW5kb2JqCjE2IDAgb2JqCjMzMAplbmRvYmoKMTcgMCBvYmoK PDwKL1RpdGxlIChjb25udGltZS5wZGYpCi9DcmVhdGlvbkRhdGUgKEQ6MjAxNDAyMjcxMDUw MDApCi9Nb2REYXRlIChEOjIwMTQwMjI3MTA1MDAwKQovUHJvZHVjZXIgKEltYWdlTWFnaWNr IDYuNS40LTcgMjAxNC0wMi0xMCBRMTYgT3Blbk1QIGh0dHA6Ly93d3cuaW1hZ2VtYWdpY2su b3JnKQo+PgplbmRvYmoKeHJlZgowIDE4CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAx MCAwMDAwMCBuIAowMDAwMDAwMDU5IDAwMDAwIG4gCjAwMDAwMDAxMTggMDAwMDAgbiAKMDAw MDAwMDMwMCAwMDAwMCBuIAowMDAwMDAwMzgzIDAwMDAwIG4gCjAwMDAwMDA0MDEgMDAwMDAg biAKMDAwMDAwMDQzOSAwMDAwMCBuIAowMDAwMDAwNDYwIDAwMDAwIG4gCjAwMDAwMDgwNDkg MDAwMDAgbiAKMDAwMDAwODA2OSAwMDAwMCBuIAowMDAwMDA4MTIwIDAwMDAwIG4gCjAwMDAw MTIxNTkgMDAwMDAgbiAKMDAwMDAxMjE4MCAwMDAwMCBuIAowMDAwMDEyNTY1IDAwMDAwIG4g CjAwMDAwMTI1ODUgMDAwMDAgbiAKMDAwMDAxMjYwNyAwMDAwMCBuIAowMDAwMDEyNjI3IDAw MDAwIG4gCnRyYWlsZXIKPDwKL1NpemUgMTgKL0luZm8gMTcgMCBSCi9Sb290IDEgMCBSCj4+ CnN0YXJ0eHJlZgoxMjgxMwolJUVPRgo= --------------080207040208060206070203 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="tw-lock.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tw-lock.patch" diff --git a/sys/netinet/tcp_timer.c b/sys/netinet/tcp_timer.c index bde7503..b45a9ea 100644 --- a/sys/netinet/tcp_timer.c +++ b/sys/netinet/tcp_timer.c @@ -144,9 +144,7 @@ tcp_slowtimo(void) VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); - INP_INFO_WLOCK(&V_tcbinfo); - (void) tcp_tw_2msl_scan(0); - INP_INFO_WUNLOCK(&V_tcbinfo); + tcp_tw_2msl_scan(); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); diff --git a/sys/netinet/tcp_timer.h b/sys/netinet/tcp_timer.h index 3115fb3..c04723a 100644 --- a/sys/netinet/tcp_timer.h +++ b/sys/netinet/tcp_timer.h @@ -178,7 +178,8 @@ extern int tcp_fast_finwait2_recycle; void tcp_timer_init(void); void tcp_timer_2msl(void *xtp); struct tcptw * - tcp_tw_2msl_scan(int _reuse); /* XXX temporary */ + tcp_tw_2msl_reuse(void); /* XXX temporary? */ +void tcp_tw_2msl_scan(void); void tcp_timer_keep(void *xtp); void tcp_timer_persist(void *xtp); void tcp_timer_rexmt(void *xtp); diff --git a/sys/netinet/tcp_timewait.c b/sys/netinet/tcp_timewait.c index 7e6128b..0230b88 100644 --- a/sys/netinet/tcp_timewait.c +++ b/sys/netinet/tcp_timewait.c @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include @@ -98,13 +99,59 @@ static int maxtcptw; * The timed wait queue contains references to each of the TCP sessions * currently in the TIME_WAIT state. The queue pointers, including the * queue pointers in each tcptw structure, are protected using the global - * tcbinfo lock, which must be held over queue iteration and modification. + * timewait lock, which must be held over queue iteration and modification. */ static VNET_DEFINE(TAILQ_HEAD(, tcptw), twq_2msl); #define V_twq_2msl VNET(twq_2msl) -static void tcp_tw_2msl_reset(struct tcptw *, int); -static void tcp_tw_2msl_stop(struct tcptw *); +/* Global timewait lock */ +static VNET_DEFINE(struct rwlock, tw_lock); +#define V_tw_lock VNET(tw_lock) + +#define TW_LOCK_INIT(tw, d) rw_init_flags(&(tw), (d), 0) +#define TW_LOCK_DESTROY(tw) rw_destroy(&(tw)) +#define TW_RLOCK(tw) rw_rlock(&(tw)) +#define TW_WLOCK(tw) rw_wlock(&(tw)) +#define TW_RUNLOCK(tw) rw_runlock(&(tw)) +#define TW_WUNLOCK(tw) rw_wunlock(&(tw)) +#define TW_LOCK_ASSERT(tw) rw_assert(&(tw), RA_LOCKED) +#define TW_RLOCK_ASSERT(tw) rw_assert(&(tw), RA_RLOCKED) +#define TW_WLOCK_ASSERT(tw) rw_assert(&(tw), RA_WLOCKED) +#define TW_UNLOCK_ASSERT(tw) rw_assert(&(tw), RA_UNLOCKED) + +/* + * tw_pcbref() bumps the reference count on an tw in order to maintain + * stability of an tw pointer despite the tw lock being released. + */ +static void +tw_pcbref(struct tcptw *tw) +{ + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); + refcount_acquire(&tw->tw_refcount); +} + +/* + * Drop a refcount on an tw elevated using tw_pcbref(). If it is + * valid, we return with the tw lock held. + */ +static int +tw_pcbrele(struct tcptw *tw) +{ + TW_WLOCK_ASSERT(V_tw_lock); + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); + + if (!refcount_release(&tw->tw_refcount)) { + TW_WUNLOCK(V_tw_lock); + return (0); + } + + uma_zfree(V_tcptw_zone, tw); + TW_WUNLOCK(V_tw_lock); + return (1); +} + +static void tcp_tw_2msl_reset(struct tcptw *, int ream); +static void tcp_tw_2msl_stop(struct tcptw *, int reuse); static int tcptw_auto_size(void) @@ -171,6 +218,7 @@ tcp_tw_init(void) else uma_zone_set_max(V_tcptw_zone, maxtcptw); TAILQ_INIT(&V_twq_2msl); + TW_LOCK_INIT(V_tw_lock, "tcptw"); } #ifdef VIMAGE @@ -179,11 +227,14 @@ tcp_tw_destroy(void) { struct tcptw *tw; - INP_INFO_WLOCK(&V_tcbinfo); - while((tw = TAILQ_FIRST(&V_twq_2msl)) != NULL) - tcp_twclose(tw, 0); - INP_INFO_WUNLOCK(&V_tcbinfo); + TW_WLOCK(V_tw_lock); + while((tw = TAILQ_FIRST(&V_twq_2msl)) != NULL) { + tcp_twclose(tw, 0, 1); + TW_WLOCK(V_tw_lock); + } + TW_WUNLOCK(V_tw_lock); + TW_LOCK_DESTROY(V_tw_lock); uma_zdestroy(V_tcptw_zone); } #endif @@ -204,7 +255,7 @@ tcp_twstart(struct tcpcb *tp) int isipv6 = inp->inp_inc.inc_flags & INC_ISIPV6; #endif - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* tcp_tw_2msl_reset(). */ + INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); if (V_nolocaltimewait) { @@ -229,7 +280,7 @@ tcp_twstart(struct tcpcb *tp) tw = uma_zalloc(V_tcptw_zone, M_NOWAIT); if (tw == NULL) { - tw = tcp_tw_2msl_scan(1); + tw = tcp_tw_2msl_reuse(); if (tw == NULL) { tp = tcp_close(tp); if (tp != NULL) @@ -238,6 +289,7 @@ tcp_twstart(struct tcpcb *tp) } } tw->tw_inpcb = inp; + refcount_init(&tw->tw_refcount, 1); /* * Recover last window size sent. @@ -356,7 +408,6 @@ tcp_twcheck(struct inpcb *inp, struct tcpopt *to, struct tcphdr *th, int thflags; tcp_seq seq; - /* tcbinfo lock required for tcp_twclose(), tcp_tw_2msl_reset(). */ INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); @@ -458,11 +509,11 @@ tcp_twclose(struct tcptw *tw, int reuse) inp = tw->tw_inpcb; KASSERT((inp->inp_flags & INP_TIMEWAIT), ("tcp_twclose: !timewait")); KASSERT(intotw(inp) == tw, ("tcp_twclose: inp_ppcb != tw")); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* tcp_tw_2msl_stop(). */ + INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); tw->tw_inpcb = NULL; - tcp_tw_2msl_stop(tw); + tcp_tw_2msl_stop(tw, reuse); inp->inp_ppcb = NULL; in_pcbdrop(inp); @@ -494,11 +545,6 @@ tcp_twclose(struct tcptw *tw, int reuse) } else in_pcbfree(inp); TCPSTAT_INC(tcps_closed); - crfree(tw->tw_cred); - tw->tw_cred = NULL; - if (reuse) - return; - uma_zfree(V_tcptw_zone, tw); } int @@ -616,34 +662,83 @@ tcp_tw_2msl_reset(struct tcptw *tw, int rearm) INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tw->tw_inpcb); + + TW_WLOCK(V_tw_lock); if (rearm) TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); tw->tw_time = ticks + 2 * tcp_msl; TAILQ_INSERT_TAIL(&V_twq_2msl, tw, tw_2msl); + TW_WUNLOCK(V_tw_lock); } static void -tcp_tw_2msl_stop(struct tcptw *tw) +tcp_tw_2msl_stop(struct tcptw *tw, int reuse) { INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + + TW_WLOCK(V_tw_lock); TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); + crfree(tw->tw_cred); + tw->tw_cred = NULL; + + if (!reuse) { + tw_pcbrele(tw); + return; + } + + TW_WUNLOCK(V_tw_lock); } struct tcptw * -tcp_tw_2msl_scan(int reuse) +tcp_tw_2msl_reuse(void) { - struct tcptw *tw; INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + + struct tcptw *tw; + + TW_WLOCK(V_tw_lock); + tw = TAILQ_FIRST(&V_twq_2msl); + if (tw == NULL) { + TW_WUNLOCK(V_tw_lock); + return NULL; + } + TW_WUNLOCK(V_tw_lock); + + INP_WLOCK(tw->tw_inpcb); + tcp_twclose(tw, 1); + + return (tw); +} + +void +tcp_tw_2msl_scan(void) +{ + + struct tcptw *tw; for (;;) { + TW_RLOCK(V_tw_lock); tw = TAILQ_FIRST(&V_twq_2msl); - if (tw == NULL || (!reuse && (tw->tw_time - ticks) > 0)) + if (tw == NULL || ((tw->tw_time - ticks) > 0)) { + TW_RUNLOCK(V_tw_lock); break; + } + tw_pcbref(tw); + TW_RUNLOCK(V_tw_lock); + + /* Close timewait state */ + INP_INFO_WLOCK(&V_tcbinfo); + + TW_WLOCK(V_tw_lock); + if(tw_pcbrele(tw)) + continue; + + KASSERT(tw->tw_inpcb != NULL, + ("%s: tw->tw_inpcb == NULL", __func__)); + INP_WLOCK(tw->tw_inpcb); - tcp_twclose(tw, reuse); - if (reuse) - return (tw); + tcp_twclose(tw, 0); + INP_INFO_WUNLOCK(&V_tcbinfo); } - return (NULL); } diff --git a/sys/netinet/tcp_var.h b/sys/netinet/tcp_var.h index e3197e5..b44672d 100644 --- a/sys/netinet/tcp_var.h +++ b/sys/netinet/tcp_var.h @@ -355,6 +355,7 @@ struct tcptw { TAILQ_ENTRY(tcptw) tw_2msl; void *tw_pspare; /* TCP_SIGNATURE */ u_int *tw_spare; /* TCP_SIGNATURE */ + u_int tw_refcount; /* refcount */ }; #define intotcpcb(ip) ((struct tcpcb *)(ip)->inp_ppcb) --------------080207040208060206070203 Content-Type: application/pdf; name="conntime-bsd10.0-patched.pdf" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="conntime-bsd10.0-patched.pdf" JVBERi0xLjMgCjEgMCBvYmoKPDwKL1BhZ2VzIDIgMCBSCi9UeXBlIC9DYXRhbG9nCj4+CmVu ZG9iagoyIDAgb2JqCjw8Ci9UeXBlIC9QYWdlcwovS2lkcyBbIDMgMCBSIF0KL0NvdW50IDEK Pj4KZW5kb2JqCjMgMCBvYmoKPDwKL1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovUmVzb3Vy Y2VzIDw8Ci9YT2JqZWN0IDw8IC9JbTAgOCAwIFIgPj4KL1Byb2NTZXQgNiAwIFIgPj4KL01l ZGlhQm94IFswIDAgNjQwIDQ4MF0KL0Nyb3BCb3ggWzAgMCA2NDAgNDgwXQovQ29udGVudHMg NCAwIFIKL1RodW1iIDExIDAgUgo+PgplbmRvYmoKNCAwIG9iago8PAovTGVuZ3RoIDUgMCBS Cj4+CnN0cmVhbQpxCjY0MCAwIDAgNDgwIDAgMCBjbQovSW0wIERvClEKZW5kc3RyZWFtCmVu ZG9iago1IDAgb2JqCjMxCmVuZG9iago2IDAgb2JqClsgL1BERiAvVGV4dCAvSW1hZ2VJIF0K ZW5kb2JqCjcgMCBvYmoKPDwKPj4KZW5kb2JqCjggMCBvYmoKPDwKL1R5cGUgL1hPYmplY3QK L1N1YnR5cGUgL0ltYWdlCi9OYW1lIC9JbTAKL0ZpbHRlciBbIC9GbGF0ZURlY29kZSBdCi9X aWR0aCA2NDAKL0hlaWdodCA0ODAKL0NvbG9yU3BhY2UgMTAgMCBSCi9CaXRzUGVyQ29tcG9u ZW50IDgKL0xlbmd0aCA5IDAgUgo+PgpzdHJlYW0KeNrtne2aoyoaAGG8VBBld+//9wKKgpqk wSTv6TlVz0zaxKCIJV8SVAoAAAAAAAAAAAAAAAAWvI7MYclYPTrp6MC/DKNdwMeFyc3aS8cH /l1Mdl2wc34B+BrzapzXRhU2AnwFO1ptg3pOx7qf0dLxgX8XQT43hbzPpaqfoQIIXyU2PdRs 8Q/kCNpdlr+pb+bPn7DECy/p5c3mebP6d9n+oDYIn8Ul7UZ73f+Cf/BhxtD+mGPZe9X/jH/w Yfycb7sZq+3h/hv+gST4B5LgH0jyNf+MXjDPVttpeTeFisLsTx+v+KUjsxrMU7+zl6N8/Oj3 3V33gsbPzXgMN40hDqNRdeh1l9WmdCauD6FSoMuDaKCId2Q0vRv6B/JF/4xbx+E8Xm3SMLHU Ygpnz+ePJ1sq4a0+Deap303zg338yL/jCTZBnRC1WWcN6ghUm4qRTVEOWwnrwxem/dh6Bawj 6/+qIuuL/vkfrJ5ifJaO8tRTtH7sdJn5pNNfdyZV7/wzuZ7HJn1ubP3R6o236+dzFYHjprYj 2f6Ux3Y/6U7Z86/m6/6FzEBbv7TFl57xtLCuNoV/xuyh5nHfzhQ/rDvT63fTmL7n8h5CaZ5K ylQuTuvuYunp9/0vXxsXa8pKgg/6+TF8105+MdHZKgLpTZFBr1FeLwY/+/3YfJEA1W6d9ov5 uWBPq+IBxK2keOc3ap725XIj9lcWywL+WWdiZjCFksptC8tqtyRrLPCOofJ2/HI+65uJ9btU 0wqVrmnp9pxjaR7zrlgyLrszJg7KSPlZFZFZp/2NRRYz22hXqAKE78xJcGuqCMQ7S7aooS1R 9todD33N/5YEqHdrS//yKq/npdCO8d7ehLptsaJKxN8o4LfbHyHzSsmUkjLmauvCunrJwcL1 rUMaq9I/X27KHwZTVO/8akZUOZq0tgEWb9PuxiUW6ft1RFKlsyopg8xJu62qF3LXKgLh32gP scsxqj5Zc8QlAU673f3Lq5LhuRDY3ji7Lx820t3AEeTb7Q+fT89qV17439LQ0GsVLmYJ2vT5 lze5Z4l+ac1su8v1vCIb2yJyGCMUZU5nNxbuS7nrjv6NVc5z5d/2C5y8/rzbc4x8KGRdDrG9 CW2rvHzYyK9EoPxdTs+Czwv/O9XRU3U/h4qfr90a6lX5W8qYvhqrf5t/RSyW7WwRqUOpvOX1 7MZR5DEW43yIQChPrT0d6Fb+5kZ83fQ57/YcI+Vit0/u4clvYm6+Lh828pLhW+e6ATH/lt6Y baGso6/Vr1xkRuJH6ft5xZP2xyH/C7Ulv21sWeddcba3iFzmfy6f3VipjCVo7uAr2h/OlV0r h/ZHWHVovy7+HXfrTjFKX7a6SLr4plg+bMT/xkxQyL+liWnmbaGso09L3jHZokg9lXCP+18W M5bK+rhWo1L1zW31v7GobeWIpFbLfKz/hY2l7cXafWocu1RNCCe97H+Z9bG2l/tf5rW4PibE abduaUcUMTJ2M9Jvb1xMlLxiT8T1WH8fUv4tLbZ5W1g7mlNCehu7bqel/nfuf86F59rj7JcM pOh/rtu/Xo8utTDDeZxc2dwuWptz3sSp/RsMG+0Uws/zdhFUEVj6X+bTgZb9z2f/qt1OOtb2 UiO7iFH8xMQjj/HOb+ZJ7SsOiUj+94RjSyL2WE37QnWPKo7YWW6nLR8fWnZ7L5k7vUtbXPr/ 8i05k27mRXmiXtPe3biakCMSv19kTnlj2vkxFOFj3Jjz5wism3LHA126HY269q/e7bRUU0dX xShW89JdyBjv9U06yG1FlYi/sfvlrxx/4He5uqjvf4yb13U/yzv59/4o52/0L93/vXFKDznJ HPInV90Afjv493dh3Y1TerrB6uYxFKVr/9tHwD8ACfAPJME/kAT/QBL8A0nwDyTBP5AE/0AS /ANJ8A8kwT+QBP9AEvwDSfAPJME/kAT/QBL8A0nwDyTBP5AE/0AS/ANJ8A8kwT+QBP9AEvwD SfAPJME/kAT/QBL8A0nwDyTBP5AE/0AS/ANJ8A8kwT+QBP9AEvwDSfAPJME/kAT/QBL8A0nw DyTBP5AE/0AS/ANJ8A8kwT+QBP9AEvwDSfAPJME/+AJG+/hq9ejqFfgHn8fb6J/Rk5uTiDv4 B59nTv7ZWa0vO/gHH8fZWP56bcLyZKtV+Acfx5ron9Ox7mdq4/APPs00qsW/1AapK4D4Bx/G h3wP/0CKcVbqcfkb+fMnLPHCS3p5t396hfYHSOACk3aO/heQwtD/DIJs998s99/gHwT+gST4 B5LgH0iCf/AOvPdd4fAPbmPG1ME8mvag+Ac3iaOaJ2PMNGrbbCD+wT3GItszdmwMjX9wD/Pk 3WvwDyTBP3gTXQ1g/IP7TEaZ0P7tMBD/4DazNspacxha9SPwD25jjXJBQWPbg+If3Ea7kAWq 4GBHUOm4w+/Hzs6Oyo+tnX8K/+ANOK1DFmi1aw+Kf3Af7+Lw+p4OGPyDe/iC9tD4B/fQBR2h paMPv5w48kWP4XXWjL8CCcal43mm/QsSrA1f+v9ABDulPxP3P0CCSU/Ou4n6H8gwxcZv++B7 hX/wHpzruPmh8A9kwT94A9z/ADmc5f4HyDFas9AeFP/gNj0Dr3JQ6bjD76er52UB/+A2zhra HyAG469AEmNof4AsfdP/4R+8AWO5/wtimDT+eWT8C4iwTrwxM/4PJGD8M0jC+GeQJI9/ntqD 4h/cJ41/7tEP/+AtMP4Z5DAh63Njj4H4B7cxelTK0/8HMtD+BUno/wNJyP9AEvr/QBT6/0AW +v9Aks7HT+MfvIH4A3Q3dzz+CP/gPkbPRjtD+wNEiP0v2tH/AjLE/ufwn/5nEIH8DyQJNT+n jWH8AchA/zPIQv8zCNPVA41/cJ/JhDqgHpn/CiSIT36z1tiOGyD4B7exRoX2rzL0v4AE2oUs kPHPIISdnR2VHx88/9KNeh0hbaw+/EoO/+A2TuuQBdoH05D74NxSN4z91LOuWin4B/fxLkhl HjR/0205E0VLEtatFPyDD+NjtmhCtufTDbr6LjH+wVt4/gwQF3M9pxcTq2DS8Ya/g6f+aW1d 9C+W0KaqAOIfvIWn/rn0cGr8g4/x6hlck74ufyN//oQlXnhJL+82cxkZE7I92h/wfvz29N8H v8FcpiWPr/S/wNsJherz52/F/mcXhyjQ/wzvx/hXz99yo15vuxmbGsIF+AeS4B/cwxe0h8Y/ uIcu6AgtHX345ZiC9tD4B/eg/AVJKH9BEspf+L3gH9zj5f23p+Af3OPl/bfnoaWjD7+c1/ff noF/8AYof0GOOP045S9IYS3lL8jxauz9s6DScYffz9iR8a3gH9zGWUP7A8Sg/QGS0P4ASWh/ gCS0P0AS2h8gCeMPQBLGH4AY7sm71+Af3MPOe7XPz62PYMA/uMmkx8k578w0tj+CEP/gLn5a 7n/Yqb0BjH/wBrxzjucPwq8D/0AS/ANJ8A8kwT+QBP/gHfSMPYjgH9wnDoB2c8fjz/EP7mP0 bLQz7Xc/8A/eQHy6tHaHJ8v8DPyD28Tx9+G/Y/wfSED+B5LE5xppYzTjT0GEKY1/6Wh+4B+8 Bef6foOJf/AGmP8P5GD+DZCE+TdAEubfAEmYfwMkYf4NkIT2B0hC+wMkof0BktD+AElof4Ak zP8HkjD/H4jB839BEp7/C5Lo/1D+ghw3Ov8U/sFd8A8k0cbT/gAxdEFHaOnowy9HT4b2B4hB /Q8kwT+QBP/g94J/8HEmq22anMNYPdbZJf7Bp5m1cVOcHSbOUzTrqpcQ/+DD+DQv1qTjowrV +rKBf/BhfHpAptF+FbGaJRD/4B28vPc2W+VSU9lUyuEf3Of1/PdxcnKn14yw+Bz/4Dav5783 elT4B5/h5fzPU9TvuvyN/PkTlnjhJb20+/dq/vtJp6KZ9gd8hBf535QLZvpf4BM8n//e69G5 ND80/c/wEZ7Ofz+tg1N9uv9muf8G74f570GYriew4h/cZzKhbqdHfn8EEsyh5WGtsR0PAMY/ uI01KrR/Q+uiPSj+wW20C1mg4vmrIIOdnR2VH8f2oPgHt3FahyzQ9vwSCf/gPt7FzuWeDhj8 A0nwD94A8w+BHDz/CCQZef4RCMLzj0ASy/OPQBCefwSSMP8pSML8pyBN1+hT/IN3YGIHYFcr BP/gNkaPofAdNeUvSLAOfJ4ZfwoSrP3PjD8FEZbJdZ/M//IY/IPbTHpy3k1P5r96CP7BfZ7O f/AU/IN3wPwHIAr9zyCGG0PxO/bkgPgHt8n9z/z+DSTI/c/8/hckoP8ZJKH/GSRxek79z6Z9 CDT+wW10QWtQ6bjD78cUNAbFP3gTzL8LQjD/LgjC/LsgCfPvgiTMvwuSMP8uSML8uyAK8++C LD1zD0XwD+4TJ0B1c0f3C/7BfYyejXaG37+BCC+ef/4M/IPbxIZv+E//H4hA/geShJqf08Yw /xXIwPwHIAvzH4AYa9+z4/4vfJ9Z6yien5n/Hr7PrKc48tRoPTP+Gb5O7HxxetK2qwKIf3CP 2Pns+xq/Cv/gLmnUX69++Ac3WfzrfQIm/sE98A8kSbO+aMPzL0GEG5O/KPyDu9yY/EXhH8iC fyAJ/oEk+Aefx+vUNDb29JAG/IOP423yLw6TnnXdSYN/8GmMXvxL87MdJmnDP/gwIdsz0T+f fh9y+JES/sGH8V4l/1y6SWdq4/APPs/qn8+LO/gHnwf/QJIn5W/kz5+wxAsv6eVT/tH+ACEM /S8giKH/GQQx2/03y/03+AeBfyAJ/oEk+AeS4B9Ign8gCf6BJPgHkuAfSIJ/IAn+gST4B5Lg H0iCfyAJ/oEk+AeS4B9Ign8gCf6BJPgHkuAfSIJ/IAn+gST4B5LgH0iCfyAJ/oEk+AeS4B9I gn8gCf6BJPgHkuAfSIJ/IAn+gST4B5LgH0iCfyAJ/oEk+AeS4B9Ign8gCf6BJPgHkuAfSIJ/ IAn+gST4B5LgH0iCfyAJ/oEk+AeS4B9Ign8gCf6BJPgHkuAfSIJ/IAn+gST4B5LgH0iCfyAJ /l0ybC/wUX6zf0PDp02b6NzYnaP4t7r+z/bv6VkZhudBhstv3z/Rw5OtDC9D/vS7P9n40LSV f2Se/hH/hvYPHunybPtDFWh9V3oWloeHeyjD19IOj4P88Mj7z/Ie420bh6OoU+h17jk8X5G3 IiTmz/2rkna4WjcUSTdcHlPpxsVGh30Le+qW2xmWk7GsHYZiY9Wu960P6/eLbQ2l3EMV723n +VsPr4lLz4ZhO53DvnAOsEWuCFR+b6j2VUajPLrDhVYk0VXcqr0P1SZEKwD6uOsXV/Crg8vZ zlAmcFYkn9Q9HYZ9RZmmqzeLQMVZXRNsWE3NaT3UJ3vdx/q1Ycjv1+2uL5vKw763wtv6rJZh huowNrmHC4/3DRUx3I9I5cjs6bR+YUucYUuhIp22WKxpUR36IQWLYyvSWRWXenk9PC12PuLf o0p8lUfsmdEwFB/VEc5KbOdi2E7QlmvtVpQpvq7b2XKFKrcbNh2HMsSWWWVn9lOy67fpMJTG 5fjssao/LwzNkucYqPJddR3th1ftKce8OvyDQZst+/W3pef27U0/tYu1X+bFR+Xh7y5vEd8i syfshzFWj+7oXyVQkXNtCl4WXNvFXBo7lAe2+1QaWK/aLumhPBtbipYfDMWHpRl7prQHVKdt HDTbPy0vmf0klfHcbVf1MRXXTnHNDNWprzZSJEIRkzKnVsdEPLIfa+HhLnld76h3VCb6nnj7 lvdi5WP66cnN2h/825OmTMiikCsymCqj2jKwfKi1hfmoC2VUuenyDFydzKHeaw6/Xxt1rMsz WZ3YvUwr03kvgCuvtiuryJlqb4sMtv74tLxFodh9UbgWKyoNywB7rl9mnaXU9ekY6u0eolRf 5VU6V/v+FHbOLwf/Dil5GflH764uVnV4f7TxEYdi+PztgwH1mTtmr6reiLrayqngP5TQxYXw 3LOrDOuUXBfGKlV/eE46VUfquHjM3OvjOBc5daiLSH8Mr014nezBv8cpdvbguny9Svdz2j/7 9vONHQQ5xUGdvpaztrNhVwXs1UmoI3fKUC6vsMM5V9WOzvZeC3tQXe3V2cNZ2Ld6Vdg/itQx HSvtU5p9rEfY6Vj3M/X29WVsy/S4WnPIYa5SVFXpf/ze8Uou022raoZlXW3vUdCLYmqoKmRD mdEctlJlFYcibf+CUueDPi1UllSl61G/vaF0TJgHyp7T8To/raN1rDLVn5UF+ZZ+n/TPJ/+q CqCuI3x9aVYN1CdXfnlC9yJyXamr071fb6cEKz/WZ11qUa6SXleF7nXZeIr6fiGWfTMPKgB5 UR+PojyE56l0dWGfFg+Xxt5EvCgWzsHTyT3s5vEB5YT9un8vKmXPz9vllXXOSirTDq2v9WI8 Ka6eqKPKvOL6iiiawXUlUA1PwhUnWOUsU1Uhy0PR24pCu9xLVYYcii9d1U+KvpJT8lbF7rke sP05diKcj1IVCa2UOlcs40dfLn8Baj7l32X7A+BbXPW/AHyLq/5ngK9hrLbu/mYAAAAAAAAA AAAAAJ5z8YOQH+Gb76JMVtupMYwb28Msh9UaPZ9uvLfemeyJX9c9fj93pF5M8saTu57WXiua 6b0h521roFkbN+m2JPQhFUzP7er26BntAo2BnJ6bj0nF/bjWgxptTL3GQHM4uVNTQqzp9r3b tJ0DEoxuPcHr0Ju2yz6N1DEdw4HmZv+6BgWNY9zX2B5wsm3RWwbONaaeT1fG3HBy82n92jCV zgFZ4fpoLeD8fDH09WUg1x4m4mxzoJbTtJFSrwPfGjAP3Gw6qDza+McB8mn93jC9ywGpr/G+ xwoV86X2KHZch9Y0R8+OoYLVbsXcVT9tP7V29CEl2nLanGn+OCXyae20ooPLAfk/oss/01pX irX19vE609gePZ0qWG0CuiBfe/0v0O5sqJhp3VrQ25B0rq0msvrXa0Ur3/XPNKdgrK23WhGK D9cevdT0aMye3VLBas7MXHvajR2ttyTt2NYA+bp/vTltR+ymDv2WcG3fH+fexOusYLXuqt3Y rtabWkvU5hT4XvnbX9NsT/Xm7oOY+fXsqudXNN507Ml3nqf24rfPdNOep3+5/dHf0m72r6ee tKRda3YR+9cm7ZqqjS4l+di4p5Rw7blZe7O5L/+Lnvu2ZDff7X/p72ls7n/RY+p4bQ809/Ry NF8esYN31o1NHZM6eFuj51p3o/p674Ovxo1tPY3my/3P3T8IaT3B01ooNva7jrrvRlD7/be5 55aTae+06Wp+LHcvm/c0h9RrLLK3+2/8TAgAAAAAAAAAAAAAAAAAAAAAAAAAAADgr2d7jtSr secdY8zLgf2m7/ek8JeTfv9m3H/+++J7093f/Y2d87/A385Pfn/UPoXhccOGJ+nBJVGT5X+c ItKlF3X4vdoUi08XPknZ4LYq/9Js/Ru2YNdfkcXpHON0FluY7gmw4C9n988aZ216Sb+Bj79M X7+z/Dx7dmlWpG3VtP6md0q/gXV5C0qtn2i/hVknAAQ4svuXsjCzfLLMF7Q6k2bPSL8FN2H1 tiotjNP61+6B10+038L0zMgC/wp2/7zaXupZHNM7H0pSp4oJHvPkBNscPNsWtq/kMOo7c0TB L+SBf2W3zDINQZxhIU6Tl1e5am0IVPiX5wBawyj8gwc88M8Uc9Bs07CElofaVi2fevck/8th FP7BAy79W5qrZu31S3NKGXtctdTyxqL+l7eQpk2bQ46Yw1D/gwdc+7e0bXOv89L+jU94GNW+ KjeEl9auqbaQ279rGNq/8IBr/+pn4Cz9f6Eul6bm31ad+v+23C4+lSXX/1IY+v+gn677HxXc /4Ab9Nz/rWifkQ9g5+Yci4x/AQAAAAAAAACAN/J/AwdZLAplbmRzdHJlYW0KZW5kb2JqCjkg MCBvYmoKNTg5MAplbmRvYmoKMTAgMCBvYmoKWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDEwOCAx MyAwIFIgXQplbmRvYmoKMTEgMCBvYmoKPDwKL0ZpbHRlciBbIC9GbGF0ZURlY29kZSBdCi9X aWR0aCAxMDYKL0hlaWdodCA4MAovQ29sb3JTcGFjZSAxMCAwIFIKL0JpdHNQZXJDb21wb25l bnQgOAovTGVuZ3RoIDEyIDAgUgo+PgpzdHJlYW0KeNrtXGtPG0cX/jFVVSWt1CZtmkvbJE3U Vq2qSr0oX9pP/R35QBQUlAsRSQgUMGBsY4yBGMrFgLmbi40BY8jaEEMwN3MxtgGvARvIvM8y vCuXXWxjB1VV5tFqNXPOmTNnn52ZnVmvhxCGU4HX6x0fH+c4LhwO/9ux/Pfw8uVLlUpVW1s7 NzeH7NLS0vr6eiQS2d3d3dvbi0ajkf9j7wCRGCALMySoGc7UhgpFJ7F+RMkRt7ES6kd0Iis5 4va48KRuY0tJJWJ4+/v7ybC3vLzc19dntVqDweCbN29sNlsoFKJO4IHGTLF/gN0YUMnW1hYa MJouLYJIVlZW4MTv9+OMNI1qY2MDEmghhPH2AWj8NAEhztQzz/M0DeeIcGdnBza4rZubmxDC cmFhIRAI+Hw++FxbW6PBiPcOQHEaLTzQImLAog0NWFYCKpJsfm8OgATKjo2N4Xyi1vv69WuF QtHY2NjZ2Tk0NFRRUfHkyZP29naNRjMyMmKxWOATt0ar1er1ehj39/f39vaCcLvdPjw8bDKZ CgoKUBz2sGxra2tpaYETjCe4HGgfPXoESV1dXXl5ucFgwP3FjaiurkYVsKmvr4eN0+mEAZpB V1dXTU0NsqiioaEBdel0OrPZ3Nra2t3dPTAwAMno6Og/yEE6aa7iIDX2PB5PR0cHoiorK8MF 3r59Ozs7G6FWVVVBDvYGBwfRPDA4gBO32w3Lnp4esIcELionJycrKwsFcdWo/fHjx/n5+Wq1 GvcCzaaoqCgzMxMMgCLYgP/m5ma0K9ymkQMYjUbcEYfDgbtTWlqKW4AAQLJSqbx37x5NlJSU 4KbgjADgCrX/o2NGo2R3999iTxz60K3QQdCVVldX0dHQ3pBF54IB7jWoQJuhvRJpaNElcYY9 7YN0xIAN/ECIfgrnU1NTaGAYk0X/SMDbzgFgBlfIIgCoaEEUoX0cZ/hEArXADFkkqCR9rt4W ewwUjL10wNhLB4y9dMDYSweMvXTA2DspMFnCTAlzIbrMYeydCJimYo2AhdLi4iKYxOwds1C6 0AP2Y5C8RBQeySYpkXo+rtTphZfkOhfLbbCH1SKWTiiCFRb4BIF0kU6X5xR0HS1KkKAS0UCU YOZPJfT9QGwp8T2AKBHfORzxA0tRQv2IbkU/pxfecR0QyyiXyyWuVugaCksY1nOTAcdxWLBj ZS1VMfYSwmq1Pn36dHZ2Vqpi7CWEyWRSq9XT09NSFWMvITDo4TGxvLwsVTH2EsJiseTl5TH2 UgP40Wq19DegI2DsJcTw8HBhYeHMzIxUxdhLCLfbXVdXt7KyIlUx9hICa7H79+9jiSFVMfYS wm6337lzR5zvib9m0gRjLz7AW09PD5ZmNOv3+zEDbGlpmZiYAIGMvTjAqhbsdXZ28jxPJV6v t7i4GM8Rm80GLWMvDtBDKyoqsrKyPB4PlYRCIfA5Pz8PPlnbiw8wYzAYMjMzRfaOaBl78YEe qlKp2IwlNeCRkZ+fv7S0JFUx9hIC8z2NRoOBTqpi7CVEOBwGddFoVKpi7CUE1rkPHjxgbwlS w+DgYE5ODntDlRoGBgYsFsvW1pZUxdiLD8yHm5qa0PbYW4IUgEeGXq+vr6/3+XyiMM3vlt8d YKUG3ujHq1SytrbW3t7e1taGBzFbqSXE5OQkOq+41gB7Op1OoVBgMGTfEiT8lsDpdBYXFy8u LtJsMBicnZ11uVzgE22PfccS/zsW9E20NLbWSA0TExNoYGzGkhowW87NzWVvCVIDx3GlpaXi uBcLxl5C4AFhMBjYV0CpAT03IyPDZrNJVYy9hEDPVSqVXq9XqmLsJcTQ0NDDhw/ZG6oUgAlh c3NzdnY2+10jBWAxUllZqdfrQ6GQVMvYiw+0PawypqenxdlyOBz2eDx0Vwf2luCkwLS5pqbG aDRiJMTirq+vLxAIJL+rQ8rbJqSwq0NsMKcd3n5yuzqgyOTkJNZuwWAQRaxW60l3dRCzcTZJ kEqOOJH6ia06GcnbDS/5XR3IwQtSwsa99MDYSweMvXTA2EsHjL10wNhLB4y9dMDYSweMvXTw 9tnDeid20r6zQ+gKCOdo9HDzMRxijUiI9pEI2d4+VB3ZpuyIW1GLs+x+XKLxcdudpbENGhYa h7s67O+PdXXtBQLE5yNzc2R2lgSDBFmvl3g8gnBtTThcLtLeTurqSHU16esjVivp7SW1tcRk EtJdXaSpiRiNRK8XDtjYbKS+nqhURKkkDQ2ko4OYzaS5mbx4QfLzBRu4gtZgELQVFUJZh4PU 1JDnz0l/v5D9+29BOzIiVIeyqAsJ+FGrhapRL2ppbBRqgTeLRVChFCSIBD7hqrubVFYKchhw HLHbydgYaW0VtNQV/KAWSGA5MCDcuOQQCASMRqOwqwMhwxkZm7/+yl+9yn/5JX/jBv/99/wf f/A//MD/+CP/009C4uef+W+/5S9f5t97j//ss/CFC5sXLqxfvhw+f16QfPGFYINS778fvn7d d+lS4Px52AiSa9cE7ZUr/Icf8pcu8V99xf/+O//nn/xvv/GffMKfPct//XX46tXQzZvBM2fC sL91i79+nf/mG/7mTf7ixfCVK+s3bvjPng3DEuFBiOPzzwVX587xH3wgRIuYP/4YUcGV4B8V oTocn34q1H7xolAEZriE775DOghXZ87wH30kqH75RdCeOxe+di1469YCuD0Je/RdvdtsfllQ wP31F4dzXp5wFBUJWRyFhYIQByRlZYeHUjleVjYOVWUlp9VyKhWnVnM6HadQOHU6h0YzXlp6 WLC4mCsvF7Q0W1LCVVRwBoNgDBvIcVYoUPsYDGBcXc21tQnaA4fOykqHQjFSVOTU61GpUBcM 4BCW9NBohDPcIkHTMKNu4QSBwRhBolRNjaDSaBxwggSqg08UefFCyJaWjul083L/gozfc4V0 ap2fgZCtrS232y27+S2EExMTsu9tpqen8ayRqqLRqNPpXMM4KYfx8XFZ1fLyMkrJVjQ5OTk0 NCT75s3n8yFyqTwSiTgcDr/fL3tFU1NT0m8qIEcAr169wvlE7G1sbKAu2a0REcbMzIxs5IOD g30YeCUAe729vSMY6uUiz8nJGZMbW3BFKpUKg4lU1draqlard/DglgBhy/5/BxUd9+M1z/N0 r1rpleJGdHR0mEymJN+RUmxvb6NJbB/zuKF7VErlo6OjuGSpfHd3F95kf36CH9wLWYowhoBV XIJUBVcul0v2ilZXV2VZhTFaLN2EU/ZiZUvhSnEvZDlnYHjHgWEZjxt0eZzn5uboFrh4MGHi hOz8/DzmnzgjDbnZbIZqYWFB/GPsOw6bzYZh+fnz58+ePcvLy7MeQKvVFhQU3L17V6PRVFVV FRUVNTY20p2Tc3Nz8RA8pc1+/3PAOI+mhSEdjQrPHTQwNDMQiOeC3W7HeA6ucIYZWml/f//w 8DAaKmt7x4F+PxzHAPP5E/1Q+LbwP7IJmz4KZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoy NTg4CmVuZG9iagoxMyAwIG9iago8PAovTGVuZ3RoIDE0IDAgUgo+PgpzdHJlYW0K////AAAA oKCg/wAAAMAAAID/wAD/AO7uwEAA7u4AICDA/8AgAIBAoID/gEAA/4D/AMBgAMDAAGCAwGCA AIAAQP+AMGCAgGAAQEBAQIAAAACAgGAQgGBggGCAAADAAAD/AGAA47DAQMCAYKDAYMAAYMCg gAAAgACAYCCAYGBgICAgIEBAIECAYIAgYIBgYICAgIBAIIAggICAoKCgoNDgwCAgAICAwGAA gMDgwGDAwIAAwIBg/0AA/0BAgMD//4Bg/4CAwKAAwMDAwP/A/wAA/wD//4CgwMCg/2BgAP8A /4AA/6AAgODgoODgoP8gwAAAwADAoCAgoCD/gCAAgCAggEAggECAgGDAgGD/gIAAwMAA/4BA /6BA/6Bg/6Bw/8DA//8A//+A///An5+fHx8fPz8/v7+/f39/X19f39/fp6enj4+Pi4uLCmVu ZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKMzI3CmVuZG9iagoxNSAwIG9iago8PAo+PgplbmRv YmoKMTYgMCBvYmoKMzI3CmVuZG9iagoxNyAwIG9iago8PAovVGl0bGUgKGNvbm50aW1lLnBk ZikKL0NyZWF0aW9uRGF0ZSAoRDoyMDE0MDIyNzEwNDgyNSkKL01vZERhdGUgKEQ6MjAxNDAy MjcxMDQ4MjUpCi9Qcm9kdWNlciAoSW1hZ2VNYWdpY2sgNi41LjQtNyAyMDE0LTAyLTEwIFEx NiBPcGVuTVAgaHR0cDovL3d3dy5pbWFnZW1hZ2ljay5vcmcpCj4+CmVuZG9iagp4cmVmCjAg MTgKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDEwIDAwMDAwIG4gCjAwMDAwMDAwNTkg MDAwMDAgbiAKMDAwMDAwMDExOCAwMDAwMCBuIAowMDAwMDAwMzAwIDAwMDAwIG4gCjAwMDAw MDAzODMgMDAwMDAgbiAKMDAwMDAwMDQwMSAwMDAwMCBuIAowMDAwMDAwNDM5IDAwMDAwIG4g CjAwMDAwMDA0NjAgMDAwMDAgbiAKMDAwMDAwNjUzMiAwMDAwMCBuIAowMDAwMDA2NTUyIDAw MDAwIG4gCjAwMDAwMDY2MDMgMDAwMDAgbiAKMDAwMDAwOTMzMiAwMDAwMCBuIAowMDAwMDA5 MzUzIDAwMDAwIG4gCjAwMDAwMDk3MzUgMDAwMDAgbiAKMDAwMDAwOTc1NSAwMDAwMCBuIAow MDAwMDA5Nzc3IDAwMDAwIG4gCjAwMDAwMDk3OTcgMDAwMDAgbiAKdHJhaWxlcgo8PAovU2l6 ZSAxOAovSW5mbyAxNyAwIFIKL1Jvb3QgMSAwIFIKPj4Kc3RhcnR4cmVmCjk5ODMKJSVFT0YK --------------080207040208060206070203-- From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 16:01:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 572FE8E1; Thu, 27 Feb 2014 16:01:19 +0000 (UTC) Received: from mail-ob0-x22f.google.com (mail-ob0-x22f.google.com [IPv6:2607:f8b0:4003:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 14C541527; Thu, 27 Feb 2014 16:01:19 +0000 (UTC) Received: by mail-ob0-f175.google.com with SMTP id va2so2515483obc.34 for ; Thu, 27 Feb 2014 08:01:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0wkhok8e7TuG0loisp8IUDdUbrB5lx/jKsgcmbKa698=; b=xoE61ztfZzgGe5Ighm37/ZZkHrMAe4PQlSSNqT+dmjidVVh+iXbj36pMIWGyAyTBW0 3CmqhZqHHJX2Scj0tK2uOHbMnoxiDuQa4MDGQj702nkYlMGENEfnwpZj+HRWNS/zij3A 0N8RhCdIS0oznQLqVjY7UDJOGAbpsH1+yJGc4SM1RPIFOvj0d/nmyd42XgrT7N11mY/0 hXL9x2ts6nM0OrDDuB9vcc3ZTlCpxSKha0yY0lm6jYibVDLHio5PwYGnAqSrf3lHlHMU bxEBAOKfHIYIIuDjNw57fXrvG4rnTIC5poVMb/9QKi4gbF31xZBYI2HeIsXNCE1PcUEg fiiQ== MIME-Version: 1.0 X-Received: by 10.182.29.33 with SMTP id g1mr7494224obh.59.1393516878517; Thu, 27 Feb 2014 08:01:18 -0800 (PST) Received: by 10.76.130.196 with HTTP; Thu, 27 Feb 2014 08:01:18 -0800 (PST) In-Reply-To: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> References: <201402261132.09203.jhb@freebsd.org> <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> Date: Thu, 27 Feb 2014 11:01:18 -0500 Message-ID: Subject: Re: Network loss From: Ryan Stone To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Cc: Johan Kooijman , freebsd-net , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 16:01:19 -0000 On Wed, Feb 26, 2014 at 8:00 PM, Rick Macklem wrote: > And please email if you try it and let us know if it helps. > > I've think I've figured out how 64K NFS read replies can do this, > but I'll admit "ping" is a mystery? (Doesn't it just send a single > packet that would be in a single mbuf?) ixgbe currently returns an error from its if_transmit function if it gets an error sending *any* packet that is queued for transmit. So if there is a 64K NFS request queued then ping could see an EFBIG error. I can't explain the packet loss, unless ping is counting errors from send() as lost packets (which would be completely reasonable). From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 16:05:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61BEBA40; Thu, 27 Feb 2014 16:05:35 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1F3BD1586; Thu, 27 Feb 2014 16:05:34 +0000 (UTC) Received: from [77.109.131.203] (port=61117 helo=[192.168.3.123]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WJ3Sm-0006A0-G2; Thu, 27 Feb 2014 17:05:32 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Network loss From: Markus Gebert In-Reply-To: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> Date: Thu, 27 Feb 2014 17:05:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> To: Rick Macklem , John Baldwin X-Mailer: Apple Mail (2.1827) Cc: Johan Kooijman , freebsd-net@freebsd.org, Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 16:05:35 -0000 On 27.02.2014, at 02:00, Rick Macklem wrote: > John Baldwin wrote: >> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>> Hi all, >>>=20 >>> I have a weird situation here where I can't get my head around. >>>=20 >>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in >>> a while >>> the Linux clients loose their NFS connection: >>>=20 >>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>> timed out >>>=20 >>> Not all boxes, just one out of the cluster. The weird part is that >>> when I >>> try to ping a Linux client from the FreeBSD box, I have between 10 >>> and 30% >>> packetloss - all day long, no specific timeframe. If I ping the >>> Linux >>> clients - no loss. If I ping back from the Linux clients to FBSD >>> box - no >>> loss. >>>=20 >>> The errors I get when pinging a Linux client is this one: >>> ping: sendto: File too large We were facing similar problems when upgrading to 9.2 and have stayed = with 9.1 on affected systems for now. We=92ve seen this on HP G8 blades = with 82599EB controllers: ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 = rev=3D0x01 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' class =3D network subclass =3D ethernet We didn=92t find a way to trigger the problem reliably. But when it = occurs, it usually affects only one interface. Symptoms include: - socket functions return the 'File too large' error mentioned by Johan - socket functions return 'No buffer space=92 available - heavy to full packet loss on the affected interface - =93stuck=94 TCP connection, i.e. ESTABLISHED TCP connections that = should have timed out stick around forever (socket on the other side = could have been closed ours ago) - userland programs using the corresponding sockets usually got stuck = too (can=92t find kernel traces right now, but always in network related = syscalls) Network is only lightly loaded on the affected systems (usually 5-20 = mbit, capped at 200 mbit, per server), and netstat never showed any = indication of ressource shortage (like mbufs). What made the problem go away temporariliy was to ifconfig down/up the = affected interface. We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really = stable. Also, we tested a few revisions between 9.1 and 9.2 to find out = when the problem started. Unfortunately, the ixgbe driver turned out to = be mostly unstable on our systems between these releases, worse than on = 9.2. The instability was introduced shortly after to 9.1 and fixed only = very shortly before 9.2 release. So no luck there. We ended up using 9.1 = with backports of 9.2 features we really need. What we can=92t tell is wether it=92s the 9.2 kernel or the 9.2 ixgbe = driver or a combination of both that causes these problems. = Unfortunately we ran out of time (and ideas). >> EFBIG is sometimes used for drivers when a packet takes too many >> scatter/gather entries. Since you mentioned NFS, one thing you can >> try is to >> disable TSO on the intertface you are using for NFS to see if that >> "fixes" it. >>=20 > And please email if you try it and let us know if it helps. >=20 > I've think I've figured out how 64K NFS read replies can do this, > but I'll admit "ping" is a mystery? (Doesn't it just send a single > packet that would be in a single mbuf?) >=20 > I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > don't know if it can happen for an mbuf chain with < 32 entries? We don=92t use the nfs server on our systems, but they=92re = (new)nfsclients. So I don=92t think our problem is nfs related, unless = the default rsize/wsize for client mounts is not 8K, which I thought it = was. Can you confirm this, Rick? IIRC, disabling TSO did not make any difference in our case. Markus From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 17:02:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A853808; Thu, 27 Feb 2014 17:02:38 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 30D4A1C44; Thu, 27 Feb 2014 17:02:38 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id ks9so2801023vcb.11 for ; Thu, 27 Feb 2014 09:02:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=d09wWvro5YhnbnrYU0NqmAJUrGrmqyn/gtvnDb7ybDk=; b=u4WwclndzUeVoWbcbmTo9GsXPC181DnPR1HxJnxvnL16JAU/FeLaMtavZtYZ20Hzz1 /HJ2lsrCwpTCglHmkaqt0R5n/iDAp1DhbekLrP9qNEGSNmwS5IrNZ38a99fAqskiqFRE rAlq+PKp5Z5+NLBNkR6rvJ6tjyi9uew1SJ1RsAMlSvxLcq/oePYJBVcr89uOFY96DiE9 5qrquKaELmDhmrD7Y2RYf/iu7tFVpvO8MA5mUgTkEFts0WIFy5FbNzkO7qUUEwOlPxan V2xEGktWo5JUy1dKZ6J5pHblH0k/dL6ZGbVKpbZWk7VhKZ9+AOblj84eF3te5DeHhCKK i4xQ== MIME-Version: 1.0 X-Received: by 10.220.69.133 with SMTP id z5mr741219vci.49.1393520556452; Thu, 27 Feb 2014 09:02:36 -0800 (PST) Received: by 10.221.11.135 with HTTP; Thu, 27 Feb 2014 09:02:36 -0800 (PST) In-Reply-To: <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> Date: Thu, 27 Feb 2014 09:02:36 -0800 Message-ID: Subject: Re: Network loss From: Jack Vogel To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 17:02:38 -0000 I would make SURE that you have enough mbuf resources of whatever size pool that you are using (2, 4, 9K), and I would try the code in HEAD if you had not. Jack On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert wrote: > > On 27.02.2014, at 02:00, Rick Macklem wrote: > > > John Baldwin wrote: > >> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>> Hi all, > >>> > >>> I have a weird situation here where I can't get my head around. > >>> > >>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in > >>> a while > >>> the Linux clients loose their NFS connection: > >>> > >>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>> timed out > >>> > >>> Not all boxes, just one out of the cluster. The weird part is that > >>> when I > >>> try to ping a Linux client from the FreeBSD box, I have between 10 > >>> and 30% > >>> packetloss - all day long, no specific timeframe. If I ping the > >>> Linux > >>> clients - no loss. If I ping back from the Linux clients to FBSD > >>> box - no > >>> loss. > >>> > >>> The errors I get when pinging a Linux client is this one: > >>> ping: sendto: File too large > > We were facing similar problems when upgrading to 9.2 and have stayed with > 9.1 on affected systems for now. We've seen this on HP G8 blades with > 82599EB controllers: > > ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 rev=0x01 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82599EB 10 Gigabit Dual Port Backplane Connection' > class = network > subclass = ethernet > > We didn't find a way to trigger the problem reliably. But when it occurs, > it usually affects only one interface. Symptoms include: > > - socket functions return the 'File too large' error mentioned by Johan > - socket functions return 'No buffer space' available > - heavy to full packet loss on the affected interface > - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should > have timed out stick around forever (socket on the other side could have > been closed ours ago) > - userland programs using the corresponding sockets usually got stuck too > (can't find kernel traces right now, but always in network related syscalls) > > Network is only lightly loaded on the affected systems (usually 5-20 mbit, > capped at 200 mbit, per server), and netstat never showed any indication of > ressource shortage (like mbufs). > > What made the problem go away temporariliy was to ifconfig down/up the > affected interface. > > We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really > stable. Also, we tested a few revisions between 9.1 and 9.2 to find out > when the problem started. Unfortunately, the ixgbe driver turned out to be > mostly unstable on our systems between these releases, worse than on 9.2. > The instability was introduced shortly after to 9.1 and fixed only very > shortly before 9.2 release. So no luck there. We ended up using 9.1 with > backports of 9.2 features we really need. > > What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe driver > or a combination of both that causes these problems. Unfortunately we ran > out of time (and ideas). > > > >> EFBIG is sometimes used for drivers when a packet takes too many > >> scatter/gather entries. Since you mentioned NFS, one thing you can > >> try is to > >> disable TSO on the intertface you are using for NFS to see if that > >> "fixes" it. > >> > > And please email if you try it and let us know if it helps. > > > > I've think I've figured out how 64K NFS read replies can do this, > > but I'll admit "ping" is a mystery? (Doesn't it just send a single > > packet that would be in a single mbuf?) > > > > I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > > don't know if it can happen for an mbuf chain with < 32 entries? > > We don't use the nfs server on our systems, but they're (new)nfsclients. > So I don't think our problem is nfs related, unless the default rsize/wsize > for client mounts is not 8K, which I thought it was. Can you confirm this, > Rick? > > IIRC, disabling TSO did not make any difference in our case. > > > Markus > > From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 21:55:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C221DBE for ; Thu, 27 Feb 2014 21:55:03 +0000 (UTC) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2A41A36 for ; Thu, 27 Feb 2014 21:55:02 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id uo5so3099391pbc.13 for ; Thu, 27 Feb 2014 13:54:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=La9Yg4ge7LsxMAU606UYPSqpl6n+aSLOpw02t8ihD28=; b=O/Zxn68jmaz53MRE04wyezDMSt3p4+nKrovKBMWoei7m30QY8jKJx/CTTNX+2M0q4C u6IpeHbtxfpfVY94LcT243doWG5nNt6dz27WWDNVKY3LKYXwox2onGvPiNTpQxWXYIgv iNan5rFwv6PQ4hnZ5RMOo1O16BXBJf5GOL//eZ+mFSC1tpL0n/iLg8YTP7XjpoF+7Qqc Kirq1Bezz6wXfx10z2Endf3fTVBGItshS2AXi3GmHYEJCe8j/aL0cFhcDXwfgiJd0gzh yqx/6hXnBvCgdx5+ibW8VxzPCtyFI2viTi+BoTpHgpgaXvmnXTa80qO+rMFuoPQdat30 xHEg== X-Gm-Message-State: ALoCoQklleZzgPCsaN/ILS+C6wmpe3AFhUtaJkBsP1h5yJuBxEI3Cxx0MXKN5yTmZojR/ego9F1w MIME-Version: 1.0 X-Received: by 10.66.180.200 with SMTP id dq8mr17708425pac.104.1393538096541; Thu, 27 Feb 2014 13:54:56 -0800 (PST) Received: by 10.68.111.37 with HTTP; Thu, 27 Feb 2014 13:54:56 -0800 (PST) In-Reply-To: <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> Date: Thu, 27 Feb 2014 22:54:56 +0100 Message-ID: Subject: Re: Network loss From: Johan Kooijman To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 21:55:03 -0000 Ok, so 9.1 is 100% OK then? Do you have any idea about 10.0 ? On Thu, Feb 27, 2014 at 5:05 PM, Markus Gebert wrote: > > On 27.02.2014, at 02:00, Rick Macklem wrote: > > > John Baldwin wrote: > >> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>> Hi all, > >>> > >>> I have a weird situation here where I can't get my head around. > >>> > >>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in > >>> a while > >>> the Linux clients loose their NFS connection: > >>> > >>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>> timed out > >>> > >>> Not all boxes, just one out of the cluster. The weird part is that > >>> when I > >>> try to ping a Linux client from the FreeBSD box, I have between 10 > >>> and 30% > >>> packetloss - all day long, no specific timeframe. If I ping the > >>> Linux > >>> clients - no loss. If I ping back from the Linux clients to FBSD > >>> box - no > >>> loss. > >>> > >>> The errors I get when pinging a Linux client is this one: > >>> ping: sendto: File too large > > We were facing similar problems when upgrading to 9.2 and have stayed with > 9.1 on affected systems for now. We've seen this on HP G8 blades with > 82599EB controllers: > > ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 rev=0x01 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82599EB 10 Gigabit Dual Port Backplane Connection' > class = network > subclass = ethernet > > We didn't find a way to trigger the problem reliably. But when it occurs, > it usually affects only one interface. Symptoms include: > > - socket functions return the 'File too large' error mentioned by Johan > - socket functions return 'No buffer space' available > - heavy to full packet loss on the affected interface > - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should > have timed out stick around forever (socket on the other side could have > been closed ours ago) > - userland programs using the corresponding sockets usually got stuck too > (can't find kernel traces right now, but always in network related syscalls) > > Network is only lightly loaded on the affected systems (usually 5-20 mbit, > capped at 200 mbit, per server), and netstat never showed any indication of > ressource shortage (like mbufs). > > What made the problem go away temporariliy was to ifconfig down/up the > affected interface. > > We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really > stable. Also, we tested a few revisions between 9.1 and 9.2 to find out > when the problem started. Unfortunately, the ixgbe driver turned out to be > mostly unstable on our systems between these releases, worse than on 9.2. > The instability was introduced shortly after to 9.1 and fixed only very > shortly before 9.2 release. So no luck there. We ended up using 9.1 with > backports of 9.2 features we really need. > > What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe driver > or a combination of both that causes these problems. Unfortunately we ran > out of time (and ideas). > > > >> EFBIG is sometimes used for drivers when a packet takes too many > >> scatter/gather entries. Since you mentioned NFS, one thing you can > >> try is to > >> disable TSO on the intertface you are using for NFS to see if that > >> "fixes" it. > >> > > And please email if you try it and let us know if it helps. > > > > I've think I've figured out how 64K NFS read replies can do this, > > but I'll admit "ping" is a mystery? (Doesn't it just send a single > > packet that would be in a single mbuf?) > > > > I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > > don't know if it can happen for an mbuf chain with < 32 entries? > > We don't use the nfs server on our systems, but they're (new)nfsclients. > So I don't think our problem is nfs related, unless the default rsize/wsize > for client mounts is not 8K, which I thought it was. Can you confirm this, > Rick? > > IIRC, disabling TSO did not make any difference in our case. > > > Markus > > -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 23:13:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27C7ACE8; Thu, 27 Feb 2014 23:13:04 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CC73A1200; Thu, 27 Feb 2014 23:13:03 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEABLGD1ODaFve/2dsb2JhbABag0FXgwO9Pk+BMXSCJQEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASHUAgNqlKgcBeBKYxKEAIBDQ4BMweCLg8xgUkEiF1tjBiECJB4g0seMXsCBRkEHg X-IronPort-AV: E=Sophos;i="4.97,557,1389762000"; d="scan'208";a="100542019" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 27 Feb 2014 18:13:01 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B9670B4054; Thu, 27 Feb 2014 18:13:01 -0500 (EST) Date: Thu, 27 Feb 2014 18:13:01 -0500 (EST) From: Rick Macklem To: Markus Gebert Message-ID: <1673358278.14528789.1393542781747.JavaMail.root@uoguelph.ca> In-Reply-To: <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> Subject: Re: Network loss MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - IE8 (Win)/7.2.1_GA_2790) Cc: Johan Kooijman , freebsd-net@freebsd.org, Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 23:13:04 -0000 Markus Gebert wrote: >=20 > On 27.02.2014, at 02:00, Rick Macklem wrote: >=20 > > John Baldwin wrote: > >> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>> Hi all, > >>>=20 > >>> I have a weird situation here where I can't get my head around. > >>>=20 > >>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once > >>> in > >>> a while > >>> the Linux clients loose their NFS connection: > >>>=20 > >>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>> timed out > >>>=20 > >>> Not all boxes, just one out of the cluster. The weird part is > >>> that > >>> when I > >>> try to ping a Linux client from the FreeBSD box, I have between > >>> 10 > >>> and 30% > >>> packetloss - all day long, no specific timeframe. If I ping the > >>> Linux > >>> clients - no loss. If I ping back from the Linux clients to FBSD > >>> box - no > >>> loss. > >>>=20 > >>> The errors I get when pinging a Linux client is this one: > >>> ping: sendto: File too large >=20 > We were facing similar problems when upgrading to 9.2 and have stayed > with 9.1 on affected systems for now. We=E2=80=99ve seen this on HP G8 > blades with 82599EB controllers: >=20 > ix0@pci0:4:0:0:=09class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' > class =3D network > subclass =3D ethernet >=20 > We didn=E2=80=99t find a way to trigger the problem reliably. But when it > occurs, it usually affects only one interface. Symptoms include: >=20 > - socket functions return the 'File too large' error mentioned by > Johan > - socket functions return 'No buffer space=E2=80=99 available > - heavy to full packet loss on the affected interface > - =E2=80=9Cstuck=E2=80=9D TCP connection, i.e. ESTABLISHED TCP connection= s that > should have timed out stick around forever (socket on the other side > could have been closed ours ago) > - userland programs using the corresponding sockets usually got stuck > too (can=E2=80=99t find kernel traces right now, but always in network > related syscalls) >=20 > Network is only lightly loaded on the affected systems (usually 5-20 > mbit, capped at 200 mbit, per server), and netstat never showed any > indication of ressource shortage (like mbufs). >=20 > What made the problem go away temporariliy was to ifconfig down/up > the affected interface. >=20 > We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not > really stable. Also, we tested a few revisions between 9.1 and 9.2 > to find out when the problem started. Unfortunately, the ixgbe > driver turned out to be mostly unstable on our systems between these > releases, worse than on 9.2. The instability was introduced shortly > after to 9.1 and fixed only very shortly before 9.2 release. So no > luck there. We ended up using 9.1 with backports of 9.2 features we > really need. >=20 > What we can=E2=80=99t tell is wether it=E2=80=99s the 9.2 kernel or the 9= .2 ixgbe > driver or a combination of both that causes these problems. > Unfortunately we ran out of time (and ideas). >=20 >=20 > >> EFBIG is sometimes used for drivers when a packet takes too many > >> scatter/gather entries. Since you mentioned NFS, one thing you > >> can > >> try is to > >> disable TSO on the intertface you are using for NFS to see if that > >> "fixes" it. > >>=20 > > And please email if you try it and let us know if it helps. > >=20 > > I've think I've figured out how 64K NFS read replies can do this, > > but I'll admit "ping" is a mystery? (Doesn't it just send a single > > packet that would be in a single mbuf?) > >=20 > > I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > > don't know if it can happen for an mbuf chain with < 32 entries? >=20 > We don=E2=80=99t use the nfs server on our systems, but they=E2=80=99re > (new)nfsclients. So I don=E2=80=99t think our problem is nfs related, unl= ess > the default rsize/wsize for client mounts is not 8K, which I thought > it was. Can you confirm this, Rick? >=20 Well, if you don't specify any mount options, it will be min(64K, what-the-server-specifies). "nfsstat -m" should show you what it actually is using, for 9.2 or later. 8K would be used if you specified "udp". For the client, it would be write requests that could be 64K. You could try "wsize=3D32768,rsize=3D32768" (it is actually the wsize that matters for this case, but you might as well set rsize at the same time). With these options specified, you know what the maximum value is (it will still be reduced for udp or if the server wants it smaller). rick > IIRC, disabling TSO did not make any difference in our case. >=20 >=20 > Markus >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Thu Feb 27 23:16:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6A57F6C; Thu, 27 Feb 2014 23:16:22 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 781491229; Thu, 27 Feb 2014 23:16:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEABLGD1ODaFve/2dsb2JhbABag0FXgwO9Pk+BMXSCJQEBAQQBAQEgKyALBRYOCgICDRkCKQEJJgYIBwQBHASHWA2qUqBwF4EpjEoQAgEbNAeCboFJBIlKjBiECJB4g0seMXsHOw X-IronPort-AV: E=Sophos;i="4.97,557,1389762000"; d="scan'208";a="100543303" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 27 Feb 2014 18:16:21 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5E80CB406D; Thu, 27 Feb 2014 18:16:21 -0500 (EST) Date: Thu, 27 Feb 2014 18:16:21 -0500 (EST) From: Rick Macklem To: Ryan Stone Message-ID: <917817504.14529709.1393542981376.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Network loss MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - IE8 (Win)/7.2.1_GA_2790) Cc: Johan Kooijman , freebsd-net , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 23:16:22 -0000 Ryan Stone wrote: > On Wed, Feb 26, 2014 at 8:00 PM, Rick Macklem > wrote: > > And please email if you try it and let us know if it helps. > > > > I've think I've figured out how 64K NFS read replies can do this, > > but I'll admit "ping" is a mystery? (Doesn't it just send a single > > packet that would be in a single mbuf?) > > ixgbe currently returns an error from its if_transmit function if it > gets an error sending *any* packet that is queued for transmit. So > if > there is a 64K NFS request queued then ping could see an EFBIG error. > I can't explain the packet loss, unless ping is counting errors from > send() as lost packets (which would be completely reasonable). Yep, I just noticed that when I took another glance at the driver. (Things still look a bit weird, since if m_defrag() returns NULL, the second attempt fails with ENOBUFS in ixgbe_xmit(). It seems that m_defrag() returns a new mbuf list, but it is still > 32 mbufs in length?) rick > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 01:21:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1C9C48F for ; Fri, 28 Feb 2014 01:21:40 +0000 (UTC) Received: from mail-lb0-x244.google.com (mail-lb0-x244.google.com [IPv6:2a00:1450:4010:c04::244]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4611B1BC6 for ; Fri, 28 Feb 2014 01:21:40 +0000 (UTC) Received: by mail-lb0-f196.google.com with SMTP id 10so793081lbg.7 for ; Thu, 27 Feb 2014 17:21:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Yr/aM/YYV431gwi/K4AnE9LfYl7VKwAJMSH5ofsgmtk=; b=egN4wtkh1i5SSImNEFxhmFO/NKZIuFmln0PYA5yUQaCeROeRPTN5N7in3HTFhbrjVS bTbXfOYAnWmgTcla279F3oNLbWS/Dvo3xIuZHF/8NozeQ6M7rEViwaPZZo5/K/Rq2GKi uqft602byPYSDB/0lwCe0JPjSMfeA3v+EWutxd8x9MHuRhcPixsBaDyFkaUONDH6FXIH JByV948zkjPQCsXkT+5p8RupOlzAyiHcn/SUcH18CztqaNv5OSaHXUB8bqqlwSnURnu5 ZfO1Byh9BWiDLM9guQwhj60BIlBqO4hIJPX2PDcWKMBATjdJtM0DvWAd+m+FyCK6vlgS gF7A== MIME-Version: 1.0 X-Received: by 10.112.43.70 with SMTP id u6mr8359334lbl.30.1393550498197; Thu, 27 Feb 2014 17:21:38 -0800 (PST) Received: by 10.114.4.82 with HTTP; Thu, 27 Feb 2014 17:21:37 -0800 (PST) Date: Thu, 27 Feb 2014 17:21:37 -0800 Message-ID: Subject: Panic when downing interface From: olivier To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 01:21:40 -0000 Hi all, The following kernel panic often happens for me after downing an Infiniband interface (i.e. calling ifconfig ib0 down). It happens both under 9-STABLE and 10-STABLE. The stack trace doesn't look Infiniband-specific so I'm not sending to the Infiniband mailing list for now. Details for 9-STABLE are as follows: panic: sbdrop cpuid = 25 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame 0xffffffbff8b97480 kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffbff8b97540 panic() at panic+0x1ce/frame 0xffffffbff8b97640 sbdrop_internal() at sbdrop_internal+0x323/frame 0xffffffbff8b97680 sbflush_internal() at sbflush_internal+0x3b/frame 0xffffffbff8b976a0 sbflush() at sbflush+0x3b/frame 0xffffffbff8b976c0 tcp_disconnect() at tcp_disconnect+0x46/frame 0xffffffbff8b976f0 tcp_usr_disconnect() at tcp_usr_disconnect+0x10f/frame 0xffffffbff8b97710 soclose() at soclose+0x53/frame 0xffffffbff8b97740 _fdrop() at _fdrop+0x23/frame 0xffffffbff8b97760 closef() at closef+0x4c/frame 0xffffffbff8b97810 fdfree() at fdfree+0x23c/frame 0xffffffbff8b978d0 exit1() at exit1+0x330/frame 0xffffffbff8b97940 sys_sys_exit() at sys_sys_exit+0xe/frame 0xffffffbff8b97950 amd64_syscall() at amd64_syscall+0x5ea/frame 0xffffffbff8b97a70 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffffbff8b97a70 (kgdb) list *(sbdrop_internal+0x323) 0xffffffff8099b823 is at /usr/src/sys/kern/uipc_sockbuf.c:855. 850 851 next = (m = sb->sb_mb) ? m->m_nextpkt : 0; 852 while (len > 0) { 853 if (m == 0) { 854 if (next == 0) 855 panic("sbdrop"); 856 m = next; 857 next = m->m_nextpkt; 858 continue; 859 } FreeBSD 9.2 r255606 compiled with options OFED # Infiniband protocol device mlx4ib # ConnectX Infiniband support device mlxen # ConnectX Ethernet support device mthca # Infinihost cards device ipoib # IP over IB devices Mellanox ConnectX HCA Ethernet driver v1.5.2 (July 2010) Mellanox ConnectX InfiniBand driver v1.0-ofed1.5.2 (August 4, 2010) output of ifconfig ib0: ib0: flags=8043 metric 0 mtu 2044 options=8009b lladdr 0.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.3.0.4f.ae.f1 inet 192.168.192.2 netmask 0xffffff00 broadcast 192.168.192.255 inet6 fe80::225:90ff:fe59:b78%ib0 prefixlen 64 scopeid 0xb nd6 options=29 Happy to provide more detail if that helps anyone look into this problem. Thanks Olivier From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 02:02:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A012ECEF for ; Fri, 28 Feb 2014 02:02:10 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A1621093 for ; Fri, 28 Feb 2014 02:02:09 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WJCm5-0001up-BZ for freebsd-net@freebsd.org; Fri, 28 Feb 2014 03:02:05 +0100 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Feb 2014 03:02:05 +0100 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Feb 2014 03:02:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: Re: FreeBSD 10 network flapping, ix driver unreliable? Date: Thu, 27 Feb 2014 19:01:51 -0700 Lines: 63 Message-ID: <530FEE0F.5010306@kev009.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Thunderbird/27.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 02:02:10 -0000 On 2/15/2014 1:14 PM, Kevin Bowling wrote: > Hi, > > I have FreeBSD 10.0-RELEASE installed on two Dell C6100 nodes. Each > node has an Intel X520-DA2 dual port 10gig card. One of the ports on > each go to a switch using direct attach coaxial cables. The other port > is directly connected between the two nodes (think crossover in twisted > pair terminology) again using direct attach coaxial cables. > > On both machines, and on both ports (including the "crossover"), the > links flap several times per day. > > I've pasted the output of lspci -vv and dmesg here: > https://gist.github.com/kev009/9024442 > > There's nothing outstanding about the setup otherwise. I suspected some > interaction with the switch initially but the "crossover" has eliminated > that suspicion. > > It seems the ix driver is not very reliable under common conditions, > i.e. https://forums.freebsd.org/viewtopic.php?f=7&t=44570 and a search > of this list. Any recommendations or tests? > > Regards, > Kevin Bowling > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > After descending a rather dark rabbit hole, I'm pleased to have found a simple solution! On some of these cards, there are known firmware problems. The driver sometimes tries to compensate, but these code paths probably receive far less testing and look pretty fragile. It seems some version of NIC firmware are particularly flaky with DA cables. Some spam in the Linux dmesg led me to this, which does not appear to be in the FreeBSD ixgbe driver, although I did not stay in Linux long enough to see if it fully fixed the problem: http://markmail.org/message/ivsjxoyfbvzv7mvo Instead, I found a way to update the microcode. My card and server are Dell and I was able to use this live cd to do the firmware upgrade: http://linux.dell.com/files/openmanage-contributions/om-firmware-live/ which applies a NIC firmware package like http://www.dell.com/support/drivers/us/en/19/driverdetails?driverid=HKK1W Both of my systems appear to be stable; I'll comment if there are issues over the next few days. Intel is less than forthcoming about these microcode updates; I'm not sure if the preboot code (https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&ProdId=3591&DwnldID=19186&ProductFamily=Network+Connectivity&ProductLine=Intel%C2%AE+Server+Adapters&ProductProduct=Intel%C2%AE+Ethernet+Converged+Network+Adapter+X520+Series&lang=eng) contains the NIC microcode or if you must obtain it from a vendor. Regards, Kevin Bowling From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 02:05:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B29B9A3 for ; Fri, 28 Feb 2014 02:05:06 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 70A1C10BB for ; Fri, 28 Feb 2014 02:05:06 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WJCox-0007pn-Um for freebsd-net@freebsd.org; Fri, 28 Feb 2014 03:05:03 +0100 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Feb 2014 03:05:03 +0100 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Feb 2014 03:05:03 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: Re: FreeBSD 10 network flapping, ix driver unreliable? Date: Thu, 27 Feb 2014 19:02:01 -0700 Lines: 63 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Thunderbird/27.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 02:05:06 -0000 On 2/15/2014 1:14 PM, Kevin Bowling wrote: > Hi, > > I have FreeBSD 10.0-RELEASE installed on two Dell C6100 nodes. Each > node has an Intel X520-DA2 dual port 10gig card. One of the ports on > each go to a switch using direct attach coaxial cables. The other port > is directly connected between the two nodes (think crossover in twisted > pair terminology) again using direct attach coaxial cables. > > On both machines, and on both ports (including the "crossover"), the > links flap several times per day. > > I've pasted the output of lspci -vv and dmesg here: > https://gist.github.com/kev009/9024442 > > There's nothing outstanding about the setup otherwise. I suspected some > interaction with the switch initially but the "crossover" has eliminated > that suspicion. > > It seems the ix driver is not very reliable under common conditions, > i.e. https://forums.freebsd.org/viewtopic.php?f=7&t=44570 and a search > of this list. Any recommendations or tests? > > Regards, > Kevin Bowling > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > After descending a rather dark rabbit hole, I'm pleased to have found a simple solution! On some of these cards, there are known firmware problems. The driver sometimes tries to compensate, but these code paths probably receive far less testing and look pretty fragile. It seems some version of NIC firmware are particularly flaky with DA cables. Some spam in the Linux dmesg led me to this, which does not appear to be in the FreeBSD ixgbe driver, although I did not stay in Linux long enough to see if it fully fixed the problem: http://markmail.org/message/ivsjxoyfbvzv7mvo Instead, I found a way to update the microcode. My card and server are Dell and I was able to use this live cd to do the firmware upgrade: http://linux.dell.com/files/openmanage-contributions/om-firmware-live/ which applies a NIC firmware package like http://www.dell.com/support/drivers/us/en/19/driverdetails?driverid=HKK1W Both of my systems appear to be stable; I'll comment if there are issues over the next few days. Intel is less than forthcoming about these microcode updates; I'm not sure if the preboot code (https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&ProdId=3591&DwnldID=19186&ProductFamily=Network+Connectivity&ProductLine=Intel%C2%AE+Server+Adapters&ProductProduct=Intel%C2%AE+Ethernet+Converged+Network+Adapter+X520+Series&lang=eng) contains the NIC microcode or if you must obtain it from a vendor. Regards, Kevin Bowling From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 05:25:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A473B604 for ; Fri, 28 Feb 2014 05:25:04 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5C5AC10DA for ; Fri, 28 Feb 2014 05:25:04 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s1S5OxWQ062737 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 27 Feb 2014 21:25:02 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <53101DA4.1060507@freebsd.org> Date: Fri, 28 Feb 2014 13:24:52 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Jack Vogel , Sami Halabi , FreeBSD Net Subject: Re: TSO References: <20140226180736.GV92037@funkthat.com> <20140226212412.GZ92037@funkthat.com> In-Reply-To: <20140226212412.GZ92037@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 05:25:04 -0000 On 2/27/14, 5:24 AM, John-Mark Gurney wrote: > Jack Vogel wrote this message on Wed, Feb 26, 2014 at 10:27 -0800: >> Drivers have to work with whatever the requirements/limitations of the >> hardware, >> if you have a 5 lb sack you shouldn't be surprised if some drops when you >> shove >> 6 lbs at it :) > But right now, when that happens, the nic just drops it instead of > telling the kernel to stop giving 6 lbs sacks.. :) It's only after a > large amount of work by various people did we even find out that this > is what was happening... so why not look at what would happen if it were people doing this.. person1 would hand person2 a 4 pound sack. all ok.. person 1 then hands person2 a 6 pound sack and person 2 replies "Ouch", that's too much!.. person 1 now knows not to do that.. until he forgets... part of the trick is knowing while assembling the packet, what interface is going to be used.. which from memory is not 100% guaranteed because routes can change... >> Why not have this limit in the interface so the stack can avoid exceeding >> it? > One of the patches proposed does that, though I hope that ALL drivers > will be properly updated when the patch hits the tree... > >> On Wed, Feb 26, 2014 at 10:07 AM, John-Mark Gurney wrote: >> >>> Sami Halabi wrote this message on Wed, Feb 26, 2014 at 19:37 +0200: >>>> I'm reading (almost) all mailing emails in mailig list... >>>> >>>> Almost every / many problem in network performancr / packets loss ended >>> up >>>> suggesting disabling TSO. >>>> >>>> I wonder why.. Is it a bug in the implementation? Or bybdesign? >>>> What are the usecases that TSO is needed? Myabe it should be disabled bt >>>> default? >>> It looks like most of the problems are in drivers that don't handle >>> packets with a large number of segments properly... The problem is >>> that some drivers limit to how segments a packet can be broken into, and >>> then if they receive such a packet, instead of doing their darnest to >>> deliver it, they drop it... >>> >>> There are some patches that help address the issue... >>> >>> Drivers should complain more loudly when a packet gets dropped by the >>> driver, since it is likely that the OS may retry the same packet, >>> just to have it fail, though sometimes it'll try a different set, and >>> it might go through, so all the user may notice is a slight lag if >>> they notice anything at all... From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 06:42:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6D72FD4; Fri, 28 Feb 2014 06:42:04 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 766871690; Fri, 28 Feb 2014 06:42:04 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1S6g38s068514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Feb 2014 22:42:03 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1S6g3NH068513; Thu, 27 Feb 2014 22:42:03 -0800 (PST) (envelope-from jmg) Date: Thu, 27 Feb 2014 22:42:02 -0800 From: John-Mark Gurney To: Julian Elischer Subject: Re: TSO Message-ID: <20140228064202.GN47921@funkthat.com> Mail-Followup-To: Julian Elischer , Jack Vogel , Sami Halabi , FreeBSD Net References: <20140226180736.GV92037@funkthat.com> <20140226212412.GZ92037@funkthat.com> <53101DA4.1060507@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53101DA4.1060507@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 27 Feb 2014 22:42:03 -0800 (PST) Cc: FreeBSD Net , Sami Halabi , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 06:42:04 -0000 Julian Elischer wrote this message on Fri, Feb 28, 2014 at 13:24 +0800: > On 2/27/14, 5:24 AM, John-Mark Gurney wrote: > >Jack Vogel wrote this message on Wed, Feb 26, 2014 at 10:27 -0800: > >>Drivers have to work with whatever the requirements/limitations of the > >>hardware, > >>if you have a 5 lb sack you shouldn't be surprised if some drops when you > >>shove > >>6 lbs at it :) > >But right now, when that happens, the nic just drops it instead of > >telling the kernel to stop giving 6 lbs sacks.. :) It's only after a > >large amount of work by various people did we even find out that this > >is what was happening... > so why not look at what would happen if it were people doing this.. > > person1 would hand person2 a 4 pound sack. > all ok.. > person 1 then hands person2 a 6 pound sack and person 2 replies > "Ouch", that's too much!.. > person 1 now knows not to do that.. until he forgets... > part of the trick is knowing while assembling the packet, what > interface is going to be used.. > which from memory is not 100% guaranteed because routes can change... Umm... TSO depends upon knowlege that the interface supports it... if it didn't we couldn't do it.. The default is that the person can only ever accept a 1lb bag, but the TSO flag says, they can take more.. If they forget, then they'd go back to the default of 1lb sacks at a time... Plus, we already have something similar for the max size of the TSO, so the code is mostly there already, see t_tsomax... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 17:07:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75F944CB; Fri, 28 Feb 2014 17:07:56 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 026CB10C0; Fri, 28 Feb 2014 17:07:56 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:60829 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1WJQud-0004Zp-Bv; Fri, 28 Feb 2014 18:07:51 +0100 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network loss From: Markus Gebert In-Reply-To: Date: Fri, 28 Feb 2014 18:07:46 +0100 Message-Id: <76CCBE2B-D89E-40AE-9A58-8F022D70913B@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> To: Jack Vogel X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 17:07:56 -0000 On 27.02.2014, at 18:02, Jack Vogel wrote: > I would make SURE that you have enough mbuf resources of whatever size = pool > that you are > using (2, 4, 9K), and I would try the code in HEAD if you had not. >=20 > Jack Thanks for the suggestion, but I do not think it has anything to do with = resource problems. We checked netstat -m among other things all the time = when the problem was occurring, and never saw any indication of an mbuf = shortage or anything similar. Looking at the symptoms we experienced, = especially the TCP connections that never timed out, could that even be = explained by mbuf shortage? At the time we checked, 9.2 included the most recent driver AFAIR. But = this was 3 months ago, I=92ll check if something was commited in the = meantime that could help us. What I remember now is that we did see some error counter sysctl within = dev.ix rise during the network problem. It suggested that something went = wrong on the MAC layer when sending packages. I disabled flow control, = but that did not help. Rick was so kind to point me to this other thread here: = http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D279182+0+current/freebsd-ne= t It=92s about FreeBSD 10 and his links are flapping, which they didn=92t = in our case, so I did not read that thread carefully at first. But now = the OP has found out that his issues were caused by bad firmware, I=92m = not sure anymore, if our problems could be related to his (instead of = the nfs/mbuf problem). What do you think Jack? Is there a way to tell = what firmware we have, and what=92s has been fixed in a more recent = firmware release? Markus From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 17:09:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6878A57B; Fri, 28 Feb 2014 17:09:03 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EDE6A10E1; Fri, 28 Feb 2014 17:09:02 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:50131 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1WJQvk-0004bq-Ay; Fri, 28 Feb 2014 18:09:00 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network loss From: Markus Gebert In-Reply-To: <1673358278.14528789.1393542781747.JavaMail.root@uoguelph.ca> Date: Fri, 28 Feb 2014 18:08:59 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2FDC6123-5891-4DDA-AC41-FE4B639C0042@hostpoint.ch> References: <1673358278.14528789.1393542781747.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1874) Cc: Johan Kooijman , freebsd-net@freebsd.org, Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 17:09:03 -0000 On 28.02.2014, at 00:13, Rick Macklem wrote: > Markus Gebert wrote: >>=20 >> On 27.02.2014, at 02:00, Rick Macklem wrote: >>=20 >>> John Baldwin wrote: >>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>>> Hi all, >>>>>=20 >>>>> I have a weird situation here where I can't get my head around. >>>>>=20 >>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once >>>>> in >>>>> a while >>>>> the Linux clients loose their NFS connection: >>>>>=20 >>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>>>> timed out >>>>>=20 >>>>> Not all boxes, just one out of the cluster. The weird part is >>>>> that >>>>> when I >>>>> try to ping a Linux client from the FreeBSD box, I have between >>>>> 10 >>>>> and 30% >>>>> packetloss - all day long, no specific timeframe. If I ping the >>>>> Linux >>>>> clients - no loss. If I ping back from the Linux clients to FBSD >>>>> box - no >>>>> loss. >>>>>=20 >>>>> The errors I get when pinging a Linux client is this one: >>>>> ping: sendto: File too large >>=20 >> We were facing similar problems when upgrading to 9.2 and have stayed >> with 9.1 on affected systems for now. We=92ve seen this on HP G8 >> blades with 82599EB controllers: >>=20 >> ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c = chip=3D0x10f88086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' >> class =3D network >> subclass =3D ethernet >>=20 >> We didn=92t find a way to trigger the problem reliably. But when it >> occurs, it usually affects only one interface. Symptoms include: >>=20 >> - socket functions return the 'File too large' error mentioned by >> Johan >> - socket functions return 'No buffer space=92 available >> - heavy to full packet loss on the affected interface >> - =93stuck=94 TCP connection, i.e. ESTABLISHED TCP connections that >> should have timed out stick around forever (socket on the other side >> could have been closed ours ago) >> - userland programs using the corresponding sockets usually got stuck >> too (can=92t find kernel traces right now, but always in network >> related syscalls) >>=20 >> Network is only lightly loaded on the affected systems (usually 5-20 >> mbit, capped at 200 mbit, per server), and netstat never showed any >> indication of ressource shortage (like mbufs). >>=20 >> What made the problem go away temporariliy was to ifconfig down/up >> the affected interface. >>=20 >> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not >> really stable. Also, we tested a few revisions between 9.1 and 9.2 >> to find out when the problem started. Unfortunately, the ixgbe >> driver turned out to be mostly unstable on our systems between these >> releases, worse than on 9.2. The instability was introduced shortly >> after to 9.1 and fixed only very shortly before 9.2 release. So no >> luck there. We ended up using 9.1 with backports of 9.2 features we >> really need. >>=20 >> What we can=92t tell is wether it=92s the 9.2 kernel or the 9.2 ixgbe >> driver or a combination of both that causes these problems. >> Unfortunately we ran out of time (and ideas). >>=20 >>=20 >>>> EFBIG is sometimes used for drivers when a packet takes too many >>>> scatter/gather entries. Since you mentioned NFS, one thing you >>>> can >>>> try is to >>>> disable TSO on the intertface you are using for NFS to see if that >>>> "fixes" it. >>>>=20 >>> And please email if you try it and let us know if it helps. >>>=20 >>> I've think I've figured out how 64K NFS read replies can do this, >>> but I'll admit "ping" is a mystery? (Doesn't it just send a single >>> packet that would be in a single mbuf?) >>>=20 >>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >>> don't know if it can happen for an mbuf chain with < 32 entries? >>=20 >> We don=92t use the nfs server on our systems, but they=92re >> (new)nfsclients. So I don=92t think our problem is nfs related, = unless >> the default rsize/wsize for client mounts is not 8K, which I thought >> it was. Can you confirm this, Rick? >>=20 > Well, if you don't specify any mount options, it will be > min(64K, what-the-server-specifies). >=20 > "nfsstat -m" should show you what it actually is using, for 9.2 or > later. Thanks for your answer and the command. I knew there is an new option to = nfsstat, but I couldn=92t find it in the 9.1 man page, of course ;-). > 8K would be used if you specified "udp=94. I see. We=92re using tcp, so that would not be relevant then. Guess my = look at the mount vfsop was too quick... > For the client, it would be write requests that could be 64K. > You could try "wsize=3D32768,rsize=3D32768" (it is actually the > wsize that matters for this case, but you might as well set > rsize at the same time). With these options specified, you > know what the maximum value is (it will still be reduced for > udp or if the server wants it smaller). Ok. I checked this on a system that is still running under 9.2 (we had = to downgrade most production systems), and bingo, 64K wsize. So our nfs = server (a NetApp Cluster) seems to prefer 64K. This means NFS still is a potential trigger of the problem. Let=92s = pretend it is, despite I think I already tried disabling TSO. Would this = explain all the symptoms we were seeing? Why would ifconfig down/up = help? Do you have a theory on that? Markus From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 17:15:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D3037C1; Fri, 28 Feb 2014 17:15:18 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C11C41196; Fri, 28 Feb 2014 17:15:17 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:54315 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1WJR1n-0004m1-F1; Fri, 28 Feb 2014 18:15:15 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network loss From: Markus Gebert In-Reply-To: Date: Fri, 28 Feb 2014 18:15:11 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <29112316-5FC9-4DA1-BD0C-BCA61D3997E3@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> To: Johan Kooijman X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 17:15:18 -0000 On 27.02.2014, at 22:54, Johan Kooijman wrote: > Ok, so 9.1 is 100% OK then? Do you have any idea about 10.0 ? It=92s at least good enough and way better than 9.2. But you=92ll have = to test yourself, I don=92t think we=92re running the same hardware. Problem is, that 9.1 will go EOL at some point and we do not know if = this will be fixed until then. > On Thu, Feb 27, 2014 at 5:05 PM, Markus Gebert > wrote: >=20 >>=20 >> On 27.02.2014, at 02:00, Rick Macklem wrote: >>=20 >>> John Baldwin wrote: >>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>>> Hi all, >>>>>=20 >>>>> I have a weird situation here where I can't get my head around. >>>>>=20 >>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once = in >>>>> a while >>>>> the Linux clients loose their NFS connection: >>>>>=20 >>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>>>> timed out >>>>>=20 >>>>> Not all boxes, just one out of the cluster. The weird part is that >>>>> when I >>>>> try to ping a Linux client from the FreeBSD box, I have between 10 >>>>> and 30% >>>>> packetloss - all day long, no specific timeframe. If I ping the >>>>> Linux >>>>> clients - no loss. If I ping back from the Linux clients to FBSD >>>>> box - no >>>>> loss. >>>>>=20 >>>>> The errors I get when pinging a Linux client is this one: >>>>> ping: sendto: File too large >>=20 >> We were facing similar problems when upgrading to 9.2 and have stayed = with >> 9.1 on affected systems for now. We've seen this on HP G8 blades with >> 82599EB controllers: >>=20 >> ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 = rev=3D0x01 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' >> class =3D network >> subclass =3D ethernet >>=20 >> We didn't find a way to trigger the problem reliably. But when it = occurs, >> it usually affects only one interface. Symptoms include: >>=20 >> - socket functions return the 'File too large' error mentioned by = Johan >> - socket functions return 'No buffer space' available >> - heavy to full packet loss on the affected interface >> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that = should >> have timed out stick around forever (socket on the other side could = have >> been closed ours ago) >> - userland programs using the corresponding sockets usually got stuck = too >> (can't find kernel traces right now, but always in network related = syscalls) >>=20 >> Network is only lightly loaded on the affected systems (usually 5-20 = mbit, >> capped at 200 mbit, per server), and netstat never showed any = indication of >> ressource shortage (like mbufs). >>=20 >> What made the problem go away temporariliy was to ifconfig down/up = the >> affected interface. >>=20 >> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not = really >> stable. Also, we tested a few revisions between 9.1 and 9.2 to find = out >> when the problem started. Unfortunately, the ixgbe driver turned out = to be >> mostly unstable on our systems between these releases, worse than on = 9.2. >> The instability was introduced shortly after to 9.1 and fixed only = very >> shortly before 9.2 release. So no luck there. We ended up using 9.1 = with >> backports of 9.2 features we really need. >>=20 >> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe = driver >> or a combination of both that causes these problems. Unfortunately we = ran >> out of time (and ideas). >>=20 >>=20 >>>> EFBIG is sometimes used for drivers when a packet takes too many >>>> scatter/gather entries. Since you mentioned NFS, one thing you can >>>> try is to >>>> disable TSO on the intertface you are using for NFS to see if that >>>> "fixes" it. >>>>=20 >>> And please email if you try it and let us know if it helps. >>>=20 >>> I've think I've figured out how 64K NFS read replies can do this, >>> but I'll admit "ping" is a mystery? (Doesn't it just send a single >>> packet that would be in a single mbuf?) >>>=20 >>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >>> don't know if it can happen for an mbuf chain with < 32 entries? >>=20 >> We don't use the nfs server on our systems, but they're = (new)nfsclients. >> So I don't think our problem is nfs related, unless the default = rsize/wsize >> for client mounts is not 8K, which I thought it was. Can you confirm = this, >> Rick? >>=20 >> IIRC, disabling TSO did not make any difference in our case. >>=20 >>=20 >> Markus >>=20 >>=20 >=20 >=20 > --=20 > Met vriendelijke groeten / With kind regards, > Johan Kooijman >=20 > T +31(0) 6 43 44 45 27 > F +31(0) 162 82 00 01 > E mail@johankooijman.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 17:17:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7F709A9; Fri, 28 Feb 2014 17:17:31 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 656B611BE; Fri, 28 Feb 2014 17:17:31 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:56489 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1WJR3x-0004po-Ki; Fri, 28 Feb 2014 18:17:29 +0100 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network loss From: Markus Gebert In-Reply-To: <29112316-5FC9-4DA1-BD0C-BCA61D3997E3@hostpoint.ch> Date: Fri, 28 Feb 2014 18:17:28 +0100 Message-Id: <4D6D6C25-5521-449E-ABD7-63881183566D@hostpoint.ch> References: <532475749.13937791.1393462831884.JavaMail.root@uoguelph.ca> <76EBC5F0-DA4E-4A60-A10E-093F4E1BD1EF@hostpoint.ch> <29112316-5FC9-4DA1-BD0C-BCA61D3997E3@hostpoint.ch> To: Johan Kooijman X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 17:17:31 -0000 On 28.02.2014, at 18:15, Markus Gebert = wrote: >> Ok, so 9.1 is 100% OK then? Do you have any idea about 10.0 ? >=20 > It=92s at least good enough and way better than 9.2. But you=92ll have = to test yourself, I don=92t think we=92re running the same hardware. >=20 > Problem is, that 9.1 will go EOL at some point and we do not know if = this will be fixed until then. And no, we didn=92t test 10.0. It hadn=92t been released when we tested, = and the ixgbe driver was not newer in head at that point. Markus From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 21:15:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54E69B1B for ; Fri, 28 Feb 2014 21:15:00 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0F6CA18D9 for ; Fri, 28 Feb 2014 21:14:59 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id la4so1289983vcb.7 for ; Fri, 28 Feb 2014 13:14:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=ldKBtS7aaWQCRQ+W2ncHdPFg1acaiXri6KigIuC99mU=; b=d6iMDcGFqVtYSMEUHaLRJ4UkGsA/2HMWwiq5ynQRpFe7XlMEZCALZWheZbT+xIOcWa BKcyXTyuvH1PSKGzks9KDs9WZ0j7+N8OqB7JPervqnZnmUU42XGACg2iNUMyseYl8913 6a8l2uOzOVf1VqaFKJYnM2FBHmTjqAlhP4pnW7INsJPCMBo9uc1uwU4rrr/5YinrH4p6 lij8A8kve0OwBS79MJy/s5jPRu4WkV+ouKYMiJHwNYRFEAI4S1rwDfwXS8dt0jfWjblP /qezhQPVCPPx4A2L0hxcFFZM+wI2maWyVPs9IqH/fCDzBa32rWwbYkCCW+8tVu5Sfe85 BkBQ== X-Received: by 10.52.75.101 with SMTP id b5mr6716425vdw.40.1393622099101; Fri, 28 Feb 2014 13:14:59 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.188.35 with HTTP; Fri, 28 Feb 2014 13:14:39 -0800 (PST) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Fri, 28 Feb 2014 22:14:39 +0100 X-Google-Sender-Auth: GrbG1oJX7lr_XpUdXnEcQ62rkAo Message-ID: Subject: netmap's pkt-gen patch for IP and UDP checksum To: "freebsd-net@freebsd.org" , Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 21:15:00 -0000 I've found the problem with netmap's pkt-gen when using IP srt/dst or UDP port range: The function update_addresses() update the IP src/dst and/or the UDP port but forgot to update the IP and UDP checksum after these changes: Then only packets that correspond to the first value of the range are correct, all other have bad checksum. Some code is missing after the last comment "// update checksum" at the end of this function. I've did a dirty copy/past code from the initialize_packet() function for fixing this behavior and the problem is gone. Here is an example with this command-line: pkt-gen -f tx -l 60 -d 2.1.3.1-2.1.3.10 -s 1.1.3.1-1.1.3.10 Before the patch, on the receiving host, we see lot's of packet with bad checksum: 19:31:28.029702 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->31a1)!) 1.1.3.9.0 > 2.1.3.4.0: [bad udp cksum 0x2a23 -> 0x2a18!] UDP, length 18 19:31:28.029704 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->31a3)!) 1.1.3.10.0 > 2.1.3.1.0: [bad udp cksum 0x2a23 -> 0x2a1a!] UDP, length 18 19:31:28.029705 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->31a6)!) 1.1.3.4.0 > 2.1.3.4.0: [bad udp cksum 0x2a23 -> 0x2a1d!] UDP, length 18 19:31:28.029706 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->31a4)!) 1.1.3.7.0 > 2.1.3.3.0: [bad udp cksum 0x2a23 -> 0x2a1b!] UDP, length 18 19:31:28.029706 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->31a1)!) 1.1.3.6.0 > 2.1.3.7.0: [bad udp cksum 0x2a23 -> 0x2a18!] UDP, length 18 19:31:28.029707 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46, bad cksum 31ac (->319f)!) 1.1.3.5.0 > 2.1.3.10.0: [bad udp cksum 0x2a23 -> 0x2a16!] UDP, length 18 After the patch on the receiving host, no more bad checksum: 19:33:43.824502 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.9.0 > 2.1.3.2.0: [udp sum ok] UDP, length 18 19:33:43.824502 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.9.0 > 2.1.3.1.0: [udp sum ok] UDP, length 18 19:33:43.824505 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.4.0 > 2.1.3.5.0: [udp sum ok] UDP, length 18 19:33:43.824506 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.6.0 > 2.1.3.3.0: [udp sum ok] UDP, length 18 19:33:43.824507 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.4.0 > 2.1.3.7.0: [udp sum ok] UDP, length 18 19:33:43.824508 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.5.0 > 2.1.3.2.0: [udp sum ok] UDP, length 18 19:33:43.824510 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.4.0 > 2.1.3.8.0: [udp sum ok] UDP, length 18 19:33:43.824511 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.10.0 > 2.1.3.3.0: [udp sum ok] UDP, length 18 19:33:43.824512 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.6.0 > 2.1.3.4.0: [udp sum ok] UDP, length 18 19:33:43.824512 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 46) 1.1.3.9.0 > 2.1.3.4.0: [udp sum ok] UDP, length 18 The patch is here: http://www.freebsd.org/cgi/query-pr.cgi?pr=187149 Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 22:38:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 104198AC; Fri, 28 Feb 2014 22:38:38 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id B4E4110C8; Fri, 28 Feb 2014 22:38:37 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEABkPEVODaFve/2dsb2JhbABZg0FXgwO+HIErdIIlAQEFI1YbGAICDRkCIzYGE4dlAxENqmuZRg2HHReBKYsWgWIBMweCboFJBIlKjQODH4sxhUiDSx6Bbg X-IronPort-AV: E=Sophos;i="4.97,564,1389762000"; d="scan'208";a="100841733" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 28 Feb 2014 17:38:29 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id ACA7279283; Fri, 28 Feb 2014 17:38:29 -0500 (EST) Date: Fri, 28 Feb 2014 17:38:29 -0500 (EST) From: Rick Macklem To: Markus Gebert Message-ID: <1581623680.15157356.1393627109696.JavaMail.root@uoguelph.ca> In-Reply-To: <76CCBE2B-D89E-40AE-9A58-8F022D70913B@hostpoint.ch> Subject: Re: Network loss MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Johan Kooijman , FreeBSD Net , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 22:38:38 -0000 Markus Gebert wrote: >=20 >=20 >=20 > On 27.02.2014, at 18:02, Jack Vogel < jfvogel@gmail.com > wrote: >=20 >=20 > I would make SURE that you have enough mbuf resources of whatever > size pool > that you are > using (2, 4, 9K), and I would try the code in HEAD if you had not. >=20 > Jack >=20 > Thanks for the suggestion, but I do not think it has anything to do > with resource problems. We checked netstat -m among other things all > the time when the problem was occurring, and never saw any > indication of an mbuf shortage or anything similar. Looking at the > symptoms we experienced, especially the TCP connections that never > timed out, could that even be explained by mbuf shortage? >=20 >=20 > At the time we checked, 9.2 included the most recent driver AFAIR. > But this was 3 months ago, I=E2=80=99ll check if something was commited i= n > the meantime that could help us. >=20 >=20 > What I remember now is that we did see some error counter sysctl > within dev.ix rise during the network problem. It suggested that > something went wrong on the MAC layer when sending packages. I > disabled flow control, but that did not help. >=20 >=20 > Rick was so kind to point me to this other thread here: >=20 >=20 > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D279182+0+current/freebsd-n= et >=20 >=20 > It=E2=80=99s about FreeBSD 10 and his links are flapping, which they didn= =E2=80=99t > in our case, so I did not read that thread carefully at first. But > now the OP has found out that his issues were caused by bad > firmware, I=E2=80=99m not sure anymore, if our problems could be related = to > his (instead of the nfs/mbuf problem). What do you think Jack? Is > there a way to tell what firmware we have, and what=E2=80=99s has been fi= xed > in a more recent firmware release? >=20 In the above post, it was mentioned that booting Linux from a live-CD resulted in some info (he referred to it as "spam") in dmesg. I'd suggest trying that if you can have the box offline for a few minutes. rick >=20 >=20 >=20 > Markus >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 22:48:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC5B2A62; Fri, 28 Feb 2014 22:48:08 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7916F118F; Fri, 28 Feb 2014 22:48:08 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAPMREVODaFve/2dsb2JhbABZg0FXgwO9T0+BKnSCJQEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASHUAgNqmqgZReBKYxKEAIBDQ4BMweCLg8xgUkEiF1tjBqECJB5g0seMXsCBRkEHg X-IronPort-AV: E=Sophos;i="4.97,564,1389762000"; d="scan'208";a="101367595" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 28 Feb 2014 17:47:49 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 95A5CB3F2B; Fri, 28 Feb 2014 17:47:49 -0500 (EST) Date: Fri, 28 Feb 2014 17:47:49 -0500 (EST) From: Rick Macklem To: Markus Gebert Message-ID: <419352512.15161156.1393627669604.JavaMail.root@uoguelph.ca> In-Reply-To: <2FDC6123-5891-4DDA-AC41-FE4B639C0042@hostpoint.ch> Subject: Re: Network loss MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Johan Kooijman , freebsd-net@freebsd.org, Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 22:48:08 -0000 Markus Gebert wrote: >=20 > On 28.02.2014, at 00:13, Rick Macklem wrote: >=20 > > Markus Gebert wrote: > >>=20 > >> On 27.02.2014, at 02:00, Rick Macklem > >> wrote: > >>=20 > >>> John Baldwin wrote: > >>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>>>> Hi all, > >>>>>=20 > >>>>> I have a weird situation here where I can't get my head around. > >>>>>=20 > >>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. > >>>>> Once > >>>>> in > >>>>> a while > >>>>> the Linux clients loose their NFS connection: > >>>>>=20 > >>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not > >>>>> responding, > >>>>> timed out > >>>>>=20 > >>>>> Not all boxes, just one out of the cluster. The weird part is > >>>>> that > >>>>> when I > >>>>> try to ping a Linux client from the FreeBSD box, I have between > >>>>> 10 > >>>>> and 30% > >>>>> packetloss - all day long, no specific timeframe. If I ping the > >>>>> Linux > >>>>> clients - no loss. If I ping back from the Linux clients to > >>>>> FBSD > >>>>> box - no > >>>>> loss. > >>>>>=20 > >>>>> The errors I get when pinging a Linux client is this one: > >>>>> ping: sendto: File too large > >>=20 > >> We were facing similar problems when upgrading to 9.2 and have > >> stayed > >> with 9.1 on affected systems for now. We=E2=80=99ve seen this on HP G8 > >> blades with 82599EB controllers: > >>=20 > >> ix0@pci0:4:0:0:=09class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 > >> rev=3D0x01 hdr=3D0x00 > >> vendor =3D 'Intel Corporation' > >> device =3D '82599EB 10 Gigabit Dual Port Backplane > >> Connection' > >> class =3D network > >> subclass =3D ethernet > >>=20 > >> We didn=E2=80=99t find a way to trigger the problem reliably. But when= it > >> occurs, it usually affects only one interface. Symptoms include: > >>=20 > >> - socket functions return the 'File too large' error mentioned by > >> Johan > >> - socket functions return 'No buffer space=E2=80=99 available > >> - heavy to full packet loss on the affected interface > >> - =E2=80=9Cstuck=E2=80=9D TCP connection, i.e. ESTABLISHED TCP connect= ions that > >> should have timed out stick around forever (socket on the other > >> side > >> could have been closed ours ago) > >> - userland programs using the corresponding sockets usually got > >> stuck > >> too (can=E2=80=99t find kernel traces right now, but always in network > >> related syscalls) > >>=20 > >> Network is only lightly loaded on the affected systems (usually > >> 5-20 > >> mbit, capped at 200 mbit, per server), and netstat never showed > >> any > >> indication of ressource shortage (like mbufs). > >>=20 > >> What made the problem go away temporariliy was to ifconfig down/up > >> the affected interface. > >>=20 > >> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not > >> really stable. Also, we tested a few revisions between 9.1 and 9.2 > >> to find out when the problem started. Unfortunately, the ixgbe > >> driver turned out to be mostly unstable on our systems between > >> these > >> releases, worse than on 9.2. The instability was introduced > >> shortly > >> after to 9.1 and fixed only very shortly before 9.2 release. So no > >> luck there. We ended up using 9.1 with backports of 9.2 features > >> we > >> really need. > >>=20 > >> What we can=E2=80=99t tell is wether it=E2=80=99s the 9.2 kernel or th= e 9.2 ixgbe > >> driver or a combination of both that causes these problems. > >> Unfortunately we ran out of time (and ideas). > >>=20 > >>=20 > >>>> EFBIG is sometimes used for drivers when a packet takes too many > >>>> scatter/gather entries. Since you mentioned NFS, one thing you > >>>> can > >>>> try is to > >>>> disable TSO on the intertface you are using for NFS to see if > >>>> that > >>>> "fixes" it. > >>>>=20 > >>> And please email if you try it and let us know if it helps. > >>>=20 > >>> I've think I've figured out how 64K NFS read replies can do this, > >>> but I'll admit "ping" is a mystery? (Doesn't it just send a > >>> single > >>> packet that would be in a single mbuf?) > >>>=20 > >>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > >>> don't know if it can happen for an mbuf chain with < 32 entries? > >>=20 > >> We don=E2=80=99t use the nfs server on our systems, but they=E2=80=99r= e > >> (new)nfsclients. So I don=E2=80=99t think our problem is nfs related, > >> unless > >> the default rsize/wsize for client mounts is not 8K, which I > >> thought > >> it was. Can you confirm this, Rick? > >>=20 > > Well, if you don't specify any mount options, it will be > > min(64K, what-the-server-specifies). > >=20 > > "nfsstat -m" should show you what it actually is using, for 9.2 or > > later. >=20 > Thanks for your answer and the command. I knew there is an new option > to nfsstat, but I couldn=E2=80=99t find it in the 9.1 man page, of course > ;-). >=20 >=20 > > 8K would be used if you specified "udp=E2=80=9D. >=20 > I see. We=E2=80=99re using tcp, so that would not be relevant then. Guess= my > look at the mount vfsop was too quick... >=20 >=20 > > For the client, it would be write requests that could be 64K. > > You could try "wsize=3D32768,rsize=3D32768" (it is actually the > > wsize that matters for this case, but you might as well set > > rsize at the same time). With these options specified, you > > know what the maximum value is (it will still be reduced for > > udp or if the server wants it smaller). >=20 > Ok. I checked this on a system that is still running under 9.2 (we > had to downgrade most production systems), and bingo, 64K wsize. So > our nfs server (a NetApp Cluster) seems to prefer 64K. >=20 I suspect (don't know) that the NetApp Cluster prefers larger than 64K, but MAXBSIZE is set to 64K on FreeBSD at this time. (I do know that Solaris10 prefers 256K and handles 1M for UFS exports.) Sorry, this is basically irrelevant to this discussion, but I just wanted to note to readers that NFS servers like big IO sizes. (Or at least they say they do;-) > This means NFS still is a potential trigger of the problem. Let=E2=80=99s > pretend it is, despite I think I already tried disabling TSO. Would > this explain all the symptoms we were seeing? Why would ifconfig > down/up help? Do you have a theory on that? >=20 I have no idea what effect ifconfig down/up would have. It certainly shouldn't be relevant to the nfs related issue. Btw, if you are not running out of mbuf clusters, all the nfs issue does is introduce overhead from the m_defrag() calls. It is only if m_defrag() fails to copy the write request to 32 mbufs that things should fail (from my quick reading of the driver). As such, I have no idea if it is a trigger or not? rick >=20 > Markus >=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 23:01:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8C41D52; Fri, 28 Feb 2014 23:01:45 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 33AD512C5; Fri, 28 Feb 2014 23:01:44 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAE0UEVODaFve/2dsb2JhbABWA4NBV4MDvU9PgSp0giUBAQEEAQEBIAQnIAsbGBEZAgQlAQkmBggHBAEcBIdYDapnoGUXjX4GAQEbGQsQBxGCXYFJBIlKhn2FHYQIkHmDSx4xfAgXIg X-IronPort-AV: E=Sophos;i="4.97,564,1389762000"; d="scan'208";a="101371489" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 28 Feb 2014 18:01:43 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D55CAB404F; Fri, 28 Feb 2014 18:01:43 -0500 (EST) Date: Fri, 28 Feb 2014 18:01:43 -0500 (EST) From: Rick Macklem To: John-Mark Gurney Message-ID: <617657923.15165586.1393628503863.JavaMail.root@uoguelph.ca> In-Reply-To: <20140228064202.GN47921@funkthat.com> Subject: Re: TSO MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_15165584_195640140.1393628503861" X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Sami Halabi , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 23:01:45 -0000 ------=_Part_15165584_195640140.1393628503861 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit John-Mark Gurney wrote: > Julian Elischer wrote this message on Fri, Feb 28, 2014 at 13:24 > +0800: > > On 2/27/14, 5:24 AM, John-Mark Gurney wrote: > > >Jack Vogel wrote this message on Wed, Feb 26, 2014 at 10:27 -0800: > > >>Drivers have to work with whatever the requirements/limitations > > >>of the > > >>hardware, > > >>if you have a 5 lb sack you shouldn't be surprised if some drops > > >>when you > > >>shove > > >>6 lbs at it :) > > >But right now, when that happens, the nic just drops it instead of > > >telling the kernel to stop giving 6 lbs sacks.. :) It's only > > >after a > > >large amount of work by various people did we even find out that > > >this > > >is what was happening... > > so why not look at what would happen if it were people doing this.. > > > > person1 would hand person2 a 4 pound sack. > > all ok.. > > person 1 then hands person2 a 6 pound sack and person 2 replies > > "Ouch", that's too much!.. > > person 1 now knows not to do that.. until he forgets... > > part of the trick is knowing while assembling the packet, what > > interface is going to be used.. > > which from memory is not 100% guaranteed because routes can > > change... > > Umm... TSO depends upon knowlege that the interface supports it... > if it didn't we couldn't do it.. The default is that the person can > only ever accept a 1lb bag, but the TSO flag says, they can take > more.. > > If they forget, then they'd go back to the default of 1lb sacks at > a time... > > Plus, we already have something similar for the max size of the TSO, > so the code is mostly there already, see t_tsomax... > I've attached the untested (and I have nothing that does TSO, so I can't really take this patch any further) patch that adds tsomaxseg to be used along with t_tsomax by tcp_output() when deciding how big to make the TSO segment. The one thing John-Mark Gurney pointed out was that the default value for tsomaxseg needs to be large, so that it doesn't impact drivers that can handle more than 32 transmit segments. At this point few drivers set if_hw_tsomax and just use the default of 65535. Drivers would need to be fixed to set if_hw_tsomaxseg for this patch to be useful. (Assuming it is tested/debugged so it does what I meant it to;-) rick > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > ------=_Part_15165584_195640140.1393628503861 Content-Type: text/x-patch; name=tsomaxseg.patch Content-Disposition: attachment; filename=tsomaxseg.patch Content-Transfer-Encoding: base64 LS0tIGtlcm4vdWlwY19zb2NrYnVmLmMuc2F2CTIwMTQtMDEtMzAgMjA6Mjc6MTcuMDAwMDAwMDAw IC0wNTAwCisrKyBrZXJuL3VpcGNfc29ja2J1Zi5jCTIwMTQtMDEtMzAgMjI6MTI6MDguMDAwMDAw MDAwIC0wNTAwCkBAIC05NjUsNiArOTY1LDM5IEBAIHNic25kcHRyKHN0cnVjdCBzb2NrYnVmICpz YiwgdV9pbnQgb2ZmLCAKIH0KIAogLyoKKyAqIFJldHVybiB0aGUgZmlyc3QgbWJ1ZiBmb3IgdGhl IHByb3ZpZGVkIG9mZnNldC4KKyAqLworc3RydWN0IG1idWYgKgorc2JzbmRtYnVmKHN0cnVjdCBz b2NrYnVmICpzYiwgdV9pbnQgb2ZmLCBsb25nICpmaXJzdF9sZW4pCit7CisJc3RydWN0IG1idWYg Km07CisKKwlLQVNTRVJUKHNiLT5zYl9tYiAhPSBOVUxMLCAoIiVzOiBzYl9tYiBpcyBOVUxMIiwg X19mdW5jX18pKTsKKworCSpmaXJzdF9sZW4gPSAwOworCS8qCisJICogSXMgb2ZmIGJlbG93IHN0 b3JlZCBvZmZzZXQ/IEhhcHBlbnMgb24gcmV0cmFuc21pdHMuCisJICogSWYgc28sIGp1c3QgdXNl IHNiX21iLgorCSAqLworCWlmIChzYi0+c2Jfc25kcHRyID09IE5VTEwgfHwgc2ItPnNiX3NuZHB0 cm9mZiA+IG9mZikKKwkJbSA9IHNiLT5zYl9tYjsKKwllbHNlIHsKKwkJbSA9IHNiLT5zYl9zbmRw dHI7CisJCW9mZiAtPSBzYi0+c2Jfc25kcHRyb2ZmOworCX0KKwl3aGlsZSAob2ZmID4gMCAmJiBt ICE9IE5VTEwpIHsKKwkJaWYgKG9mZiA8IG0tPm1fbGVuKQorCQkJYnJlYWs7CisJCW9mZiAtPSBt LT5tX2xlbjsKKwkJbSA9IG0tPm1fbmV4dDsKKwl9CisJaWYgKG0gIT0gTlVMTCkKKwkJKmZpcnN0 X2xlbiA9IG0tPm1fbGVuIC0gb2ZmOworCisJcmV0dXJuIChtKTsKK30KKworLyoKICAqIERyb3Ag YSByZWNvcmQgb2ZmIHRoZSBmcm9udCBvZiBhIHNvY2tidWYgYW5kIG1vdmUgdGhlIG5leHQgcmVj b3JkIHRvIHRoZQogICogZnJvbnQuCiAgKi8KLS0tIHN5cy9zb2NrYnVmLmguc2F2CTIwMTQtMDEt MzAgMjA6NDI6MjguMDAwMDAwMDAwIC0wNTAwCisrKyBzeXMvc29ja2J1Zi5oCTIwMTQtMDEtMzAg MjI6MDg6NDMuMDAwMDAwMDAwIC0wNTAwCkBAIC0xNTMsNiArMTUzLDggQEAgaW50CXNicmVzZXJ2 ZV9sb2NrZWQoc3RydWN0IHNvY2tidWYgKnNiLAogCSAgICBzdHJ1Y3QgdGhyZWFkICp0ZCk7CiBz dHJ1Y3QgbWJ1ZiAqCiAJc2JzbmRwdHIoc3RydWN0IHNvY2tidWYgKnNiLCB1X2ludCBvZmYsIHVf aW50IGxlbiwgdV9pbnQgKm1vZmYpOworc3RydWN0IG1idWYgKgorCXNic25kbWJ1ZihzdHJ1Y3Qg c29ja2J1ZiAqc2IsIHVfaW50IG9mZiwgbG9uZyAqZmlyc3RfbGVuKTsKIHZvaWQJc2J0b3hzb2Nr YnVmKHN0cnVjdCBzb2NrYnVmICpzYiwgc3RydWN0IHhzb2NrYnVmICp4c2IpOwogaW50CXNid2Fp dChzdHJ1Y3Qgc29ja2J1ZiAqc2IpOwogaW50CXNibG9jayhzdHJ1Y3Qgc29ja2J1ZiAqc2IsIGlu dCBmbGFncyk7Ci0tLSBuZXRpbmV0L3RjcF9pbnB1dC5jLnNhdgkyMDE0LTAxLTMwIDE5OjM3OjUy LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3BfaW5wdXQuYwkyMDE0LTAxLTMwIDE5OjM5 OjA3LjAwMDAwMDAwMCAtMDUwMApAQCAtMzYyNyw2ICszNjI3LDcgQEAgdGNwX21zcyhzdHJ1Y3Qg dGNwY2IgKnRwLCBpbnQgb2ZmZXIpCiAJaWYgKGNhcC5pZmNhcCAmIENTVU1fVFNPKSB7CiAJCXRw LT50X2ZsYWdzIHw9IFRGX1RTTzsKIAkJdHAtPnRfdHNvbWF4ID0gY2FwLnRzb21heDsKKwkJdHAt PnRfdHNvbWF4c2VncyA9IGNhcC50c29tYXhzZWdzOwogCX0KIH0KIAotLS0gbmV0aW5ldC90Y3Bf b3V0cHV0LmMuc2F2CTIwMTQtMDEtMzAgMTg6NTU6MTUuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRp bmV0L3RjcF9vdXRwdXQuYwkyMDE0LTAxLTMwIDIyOjE4OjU2LjAwMDAwMDAwMCAtMDUwMApAQCAt MTY2LDggKzE2Niw4IEBAIGludAogdGNwX291dHB1dChzdHJ1Y3QgdGNwY2IgKnRwKQogewogCXN0 cnVjdCBzb2NrZXQgKnNvID0gdHAtPnRfaW5wY2ItPmlucF9zb2NrZXQ7Ci0JbG9uZyBsZW4sIHJl Y3dpbiwgc2VuZHdpbjsKLQlpbnQgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBp bGVyIGhhcHB5ICovCisJbG9uZyBsZW4sIHJlY3dpbiwgc2VuZHdpbiwgdHNvX3RsZW47CisJaW50 IGNudCwgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBpbGVyIGhhcHB5ICovCiAJ c3RydWN0IG1idWYgKm07CiAJc3RydWN0IGlwICppcCA9IE5VTEw7CiAJc3RydWN0IGlwb3ZseSAq aXBvdiA9IE5VTEw7CkBAIC03ODAsNiArNzgwLDI0IEBAIHNlbmQ6CiAJCQl9CiAKIAkJCS8qCisJ CQkgKiBMaW1pdCB0aGUgbnVtYmVyIG9mIFRTTyB0cmFuc21pdCBzZWdtZW50cyAobWJ1ZnMKKwkJ CSAqIGluIG1idWYgbGlzdCkgdG8gdHAtPnRfdHNvbWF4c2Vncy4KKwkJCSAqLworCQkJY250ID0g MDsKKwkJCW0gPSBzYnNuZG1idWYoJnNvLT5zb19zbmQsIG9mZiwgJnRzb190bGVuKTsKKwkJCXdo aWxlIChtICE9IE5VTEwgJiYgY250IDwgdHAtPnRfdHNvbWF4c2VncyAmJgorCQkJICAgIHRzb190 bGVuIDwgbGVuKSB7CisJCQkJaWYgKGNudCA+IDApCisJCQkJCXRzb190bGVuICs9IG0tPm1fbGVu OworCQkJCWNudCsrOworCQkJCW0gPSBtLT5tX25leHQ7CisJCQl9CisJCQlpZiAobSAhPSBOVUxM ICYmIHRzb190bGVuIDwgbGVuKSB7CisJCQkJbGVuID0gdHNvX3RsZW47CisJCQkJc2VuZGFsb3Qg PSAxOworCQkJfQorCisJCQkvKgogCQkJICogUHJldmVudCB0aGUgbGFzdCBzZWdtZW50IGZyb20g YmVpbmcKIAkJCSAqIGZyYWN0aW9uYWwgdW5sZXNzIHRoZSBzZW5kIHNvY2tidWYgY2FuCiAJCQkg KiBiZSBlbXB0aWVkLgotLS0gbmV0aW5ldC90Y3Bfc3Vici5jLnNhdgkyMDE0LTAxLTMwIDE5OjQ0 OjM1LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3Bfc3Vici5jCTIwMTQtMDEtMzAgMjA6 NTY6MTIuMDAwMDAwMDAwIC0wNTAwCkBAIC0xODAwLDYgKzE4MDAsMTIgQEAgdGNwX21heG10dShz dHJ1Y3QgaW5fY29ubmluZm8gKmluYywgc3RydQogCQkJICAgIGlmcC0+aWZfaHdhc3Npc3QgJiBD U1VNX1RTTykKIAkJCQljYXAtPmlmY2FwIHw9IENTVU1fVFNPOwogCQkJCWNhcC0+dHNvbWF4ID0g aWZwLT5pZl9od190c29tYXg7CisjaWZkZWYgbm90eWV0CisJCQkJY2FwLT50c29tYXhzZWdzID0g aWZwLT5pZl9od190c29tYXhzZWdzOworI2VuZGlmCisJCQkJaWYgKGNhcC0+dHNvbWF4c2VncyA9 PSAwKQorCQkJCQljYXAtPnRzb21heHNlZ3MgPQorCQkJCQkgICAgVENQVFNPX01BWF9UWF9TRUdT X0RFRkFVTFQ7CiAJCX0KIAkJUlRGUkVFKHNyby5yb19ydCk7CiAJfQotLS0gbmV0aW5ldC90Y3Bf dmFyLmguc2F2CTIwMTQtMDEtMzAgMTk6Mzk6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRpbmV0 L3RjcF92YXIuaAkyMDE0LTAxLTMwIDIwOjUyOjU3LjAwMDAwMDAwMCAtMDUwMApAQCAtMjA5LDYg KzIwOSw3IEBAIHN0cnVjdCB0Y3BjYiB7CiAJdV9pbnQJdF9rZWVwY250OwkJLyogbnVtYmVyIG9m IGtlZXBhbGl2ZXMgYmVmb3JlIGNsb3NlICovCiAKIAl1X2ludAl0X3Rzb21heDsJCS8qIHRzbyBi dXJzdCBsZW5ndGggbGltaXQgKi8KKwl1X2ludAl0X3Rzb21heHNlZ3M7CQkvKiB0c28gYnVyc3Qg c2VnbWVudCBsaW1pdCAqLwogCiAJdWludDMyX3QgdF9pc3BhcmVbOF07CQkvKiA1IFVUTywgMyBU QkQgKi8KIAl2b2lkCSp0X3BzcGFyZTJbNF07CQkvKiA0IFRCRCAqLwpAQCAtMjY4LDYgKzI2OSwx MSBAQCBzdHJ1Y3QgdGNwY2IgewogI2RlZmluZQlUQ1BPT0JfSEFWRURBVEEJMHgwMQogI2RlZmlu ZQlUQ1BPT0JfSEFEREFUQQkweDAyCiAKKy8qCisgKiBEZWZhdWx0IHZhbHVlIGZvciBUU08gbWF4 aW11bSBudW1iZXIgb2YgdHJhbnNtaXQgc2VnbWVudHMgKGNvdW50IG9mIG1idWZzKS4KKyAqLwor I2RlZmluZQlUQ1BUU09fTUFYX1RYX1NFR1NfREVGQVVMVAkzMAorCiAjaWZkZWYgVENQX1NJR05B VFVSRQogLyoKICAqIERlZmluZXMgd2hpY2ggYXJlIG5lZWRlZCBieSB0aGUgeGZvcm1fdGNwIG1v ZHVsZSBhbmQgdGNwX1tpbnxvdXRdcHV0CkBAIC0zMzMsNiArMzM5LDcgQEAgc3RydWN0IGhjX21l dHJpY3NfbGl0ZSB7CS8qIG11c3Qgc3RheSBpbgogc3RydWN0IHRjcF9pZmNhcCB7CiAJaW50CWlm Y2FwOwogCXVfaW50CXRzb21heDsKKwl1X2ludAl0c29tYXhzZWdzOwogfTsKIAogI2lmbmRlZiBf TkVUSU5FVF9JTl9QQ0JfSF8K ------=_Part_15165584_195640140.1393628503861-- From owner-freebsd-net@FreeBSD.ORG Fri Feb 28 23:15:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95B226EE; Fri, 28 Feb 2014 23:15:41 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3EAEB13D7; Fri, 28 Feb 2014 23:15:41 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id im17so1411244vcb.5 for ; Fri, 28 Feb 2014 15:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=VrO2uBkCiDqeexQC+BJ/PP65HD9Hs8tFnSaCmCIYMWc=; b=G7TEguZ4MqKnNpQmhB3Kvd2wSqqb+NjwAnv2a9AEDlK7VMAr1mbNj5rtAxBOIf3uj0 tp3rq8sC7dUdfW4O4scLEemJmDppeqRFlefZDBW7H959oaVIsi0ajNqFnU1UXBLhqZlX LlVP23G/VinPQ18BytSVj00rBRkGo36aXqnH95FlIa6Wf0Qo30um05zOGuEFf8uItnVs 1cVvq6Ntiw7LBQU45iQk8tDOd3xCSWT2HzVN06hqhJowBK0uCVFy+ezX1xb10Ao349Ll AwBVKG996OIC+19CD0JI5DQZLzddQwwS06L9tH/fMLjOOfyvRnZxCKwytnfNdNv2V+7K CY6w== MIME-Version: 1.0 X-Received: by 10.52.108.228 with SMTP id hn4mr2773395vdb.43.1393629340314; Fri, 28 Feb 2014 15:15:40 -0800 (PST) Received: by 10.221.11.135 with HTTP; Fri, 28 Feb 2014 15:15:40 -0800 (PST) In-Reply-To: <617657923.15165586.1393628503863.JavaMail.root@uoguelph.ca> References: <20140228064202.GN47921@funkthat.com> <617657923.15165586.1393628503863.JavaMail.root@uoguelph.ca> Date: Fri, 28 Feb 2014 15:15:40 -0800 Message-ID: Subject: Re: TSO From: Jack Vogel To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , John-Mark Gurney , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 23:15:41 -0000 Cool Rick, Would be easy enough to add the correct setting into the Intel drivers, thanks for doing this! Jack On Fri, Feb 28, 2014 at 3:01 PM, Rick Macklem wrote: > John-Mark Gurney wrote: > > Julian Elischer wrote this message on Fri, Feb 28, 2014 at 13:24 > > +0800: > > > On 2/27/14, 5:24 AM, John-Mark Gurney wrote: > > > >Jack Vogel wrote this message on Wed, Feb 26, 2014 at 10:27 -0800: > > > >>Drivers have to work with whatever the requirements/limitations > > > >>of the > > > >>hardware, > > > >>if you have a 5 lb sack you shouldn't be surprised if some drops > > > >>when you > > > >>shove > > > >>6 lbs at it :) > > > >But right now, when that happens, the nic just drops it instead of > > > >telling the kernel to stop giving 6 lbs sacks.. :) It's only > > > >after a > > > >large amount of work by various people did we even find out that > > > >this > > > >is what was happening... > > > so why not look at what would happen if it were people doing this.. > > > > > > person1 would hand person2 a 4 pound sack. > > > all ok.. > > > person 1 then hands person2 a 6 pound sack and person 2 replies > > > "Ouch", that's too much!.. > > > person 1 now knows not to do that.. until he forgets... > > > part of the trick is knowing while assembling the packet, what > > > interface is going to be used.. > > > which from memory is not 100% guaranteed because routes can > > > change... > > > > Umm... TSO depends upon knowlege that the interface supports it... > > if it didn't we couldn't do it.. The default is that the person can > > only ever accept a 1lb bag, but the TSO flag says, they can take > > more.. > > > > If they forget, then they'd go back to the default of 1lb sacks at > > a time... > > > > Plus, we already have something similar for the max size of the TSO, > > so the code is mostly there already, see t_tsomax... > > > I've attached the untested (and I have nothing that does TSO, so I can't > really take this patch any further) patch that adds tsomaxseg to be used > along with t_tsomax by tcp_output() when deciding how big to make the TSO > segment. > The one thing John-Mark Gurney pointed out was that the default value for > tsomaxseg needs to be large, so that it doesn't impact drivers that can > handle more than 32 transmit segments. > > At this point few drivers set if_hw_tsomax and just use the default of > 65535. Drivers would need to be fixed to set if_hw_tsomaxseg for this > patch to be useful. (Assuming it is tested/debugged so it does what I > meant it to;-) > > rick > > > -- > > John-Mark Gurney Voice: +1 415 225 5579 > > > > "All that I will do, has been done, All that I have, has not." > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Sat Mar 1 08:55:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB60BEF5 for ; Sat, 1 Mar 2014 08:55:13 +0000 (UTC) Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BD4E6185D for ; Sat, 1 Mar 2014 08:55:13 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id lj1so1786187pab.6 for ; Sat, 01 Mar 2014 00:55:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=sft76OCpZjVyQ6NO+tqIw13YTYaItBSaASVdz/nxyng=; b=FSmPg+7IuIoP/Gd6+0pUGVnEmzC5ZSmebw5YmoOoGmewOh6rbmYayMO2i6NaEon8VQ CgXmxm+jxp7T0oQnwsALbGNTmhbnT8SGSGZzICZA//AhuK2gUGOTmN2I+UAALaetLqWm foVTMPPQvj95tPBuIIKLP3NKJCbW5Y5ru9q3kdg3PQrEezr13V8szUzau5KcpOmECMB/ ZkBrP6II+pGEXAcSgjOynzrM52UQBLz5LLikLKET4pmJnjRT/e3NdSZf6baEecLt38L2 IZt4aR3Dy/obdunYZ9LTwnTLZKxn2rPVkTVR+EBUADi/uHd46IVbAX3K77gCcczNnvRI MgOg== X-Gm-Message-State: ALoCoQmjOBGj2QiaYAj+v7DpnAfQIimlkW8Nwcdb4b8Dk63djofbx5eShox6lHQplf09UcDwnQ1Q MIME-Version: 1.0 X-Received: by 10.68.130.202 with SMTP id og10mr8471319pbb.133.1393664106988; Sat, 01 Mar 2014 00:55:06 -0800 (PST) Received: by 10.68.111.37 with HTTP; Sat, 1 Mar 2014 00:55:06 -0800 (PST) In-Reply-To: <2FDC6123-5891-4DDA-AC41-FE4B639C0042@hostpoint.ch> References: <1673358278.14528789.1393542781747.JavaMail.root@uoguelph.ca> <2FDC6123-5891-4DDA-AC41-FE4B639C0042@hostpoint.ch> Date: Sat, 1 Mar 2014 09:55:06 +0100 Message-ID: Subject: Re: Network loss From: Johan Kooijman To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2014 08:55:14 -0000 I think NFS is part of the issue here. Everybody that seems to have this issue is running NFS. On top of that: the setup I'm having issues with, didn't have any issues for months when we were running istgt instead of NFS. On Fri, Feb 28, 2014 at 6:08 PM, Markus Gebert wrote: > > On 28.02.2014, at 00:13, Rick Macklem wrote: > > > Markus Gebert wrote: > >> > >> On 27.02.2014, at 02:00, Rick Macklem wrote: > >> > >>> John Baldwin wrote: > >>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>>>> Hi all, > >>>>> > >>>>> I have a weird situation here where I can't get my head around. > >>>>> > >>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once > >>>>> in > >>>>> a while > >>>>> the Linux clients loose their NFS connection: > >>>>> > >>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>>>> timed out > >>>>> > >>>>> Not all boxes, just one out of the cluster. The weird part is > >>>>> that > >>>>> when I > >>>>> try to ping a Linux client from the FreeBSD box, I have between > >>>>> 10 > >>>>> and 30% > >>>>> packetloss - all day long, no specific timeframe. If I ping the > >>>>> Linux > >>>>> clients - no loss. If I ping back from the Linux clients to FBSD > >>>>> box - no > >>>>> loss. > >>>>> > >>>>> The errors I get when pinging a Linux client is this one: > >>>>> ping: sendto: File too large > >> > >> We were facing similar problems when upgrading to 9.2 and have stayed > >> with 9.1 on affected systems for now. We've seen this on HP G8 > >> blades with 82599EB controllers: > >> > >> ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 > >> rev=0x01 hdr=0x00 > >> vendor = 'Intel Corporation' > >> device = '82599EB 10 Gigabit Dual Port Backplane Connection' > >> class = network > >> subclass = ethernet > >> > >> We didn't find a way to trigger the problem reliably. But when it > >> occurs, it usually affects only one interface. Symptoms include: > >> > >> - socket functions return the 'File too large' error mentioned by > >> Johan > >> - socket functions return 'No buffer space' available > >> - heavy to full packet loss on the affected interface > >> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that > >> should have timed out stick around forever (socket on the other side > >> could have been closed ours ago) > >> - userland programs using the corresponding sockets usually got stuck > >> too (can't find kernel traces right now, but always in network > >> related syscalls) > >> > >> Network is only lightly loaded on the affected systems (usually 5-20 > >> mbit, capped at 200 mbit, per server), and netstat never showed any > >> indication of ressource shortage (like mbufs). > >> > >> What made the problem go away temporariliy was to ifconfig down/up > >> the affected interface. > >> > >> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not > >> really stable. Also, we tested a few revisions between 9.1 and 9.2 > >> to find out when the problem started. Unfortunately, the ixgbe > >> driver turned out to be mostly unstable on our systems between these > >> releases, worse than on 9.2. The instability was introduced shortly > >> after to 9.1 and fixed only very shortly before 9.2 release. So no > >> luck there. We ended up using 9.1 with backports of 9.2 features we > >> really need. > >> > >> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe > >> driver or a combination of both that causes these problems. > >> Unfortunately we ran out of time (and ideas). > >> > >> > >>>> EFBIG is sometimes used for drivers when a packet takes too many > >>>> scatter/gather entries. Since you mentioned NFS, one thing you > >>>> can > >>>> try is to > >>>> disable TSO on the intertface you are using for NFS to see if that > >>>> "fixes" it. > >>>> > >>> And please email if you try it and let us know if it helps. > >>> > >>> I've think I've figured out how 64K NFS read replies can do this, > >>> but I'll admit "ping" is a mystery? (Doesn't it just send a single > >>> packet that would be in a single mbuf?) > >>> > >>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > >>> don't know if it can happen for an mbuf chain with < 32 entries? > >> > >> We don't use the nfs server on our systems, but they're > >> (new)nfsclients. So I don't think our problem is nfs related, unless > >> the default rsize/wsize for client mounts is not 8K, which I thought > >> it was. Can you confirm this, Rick? > >> > > Well, if you don't specify any mount options, it will be > > min(64K, what-the-server-specifies). > > > > "nfsstat -m" should show you what it actually is using, for 9.2 or > > later. > > Thanks for your answer and the command. I knew there is an new option to > nfsstat, but I couldn't find it in the 9.1 man page, of course ;-). > > > > 8K would be used if you specified "udp". > > I see. We're using tcp, so that would not be relevant then. Guess my look > at the mount vfsop was too quick... > > > > For the client, it would be write requests that could be 64K. > > You could try "wsize=32768,rsize=32768" (it is actually the > > wsize that matters for this case, but you might as well set > > rsize at the same time). With these options specified, you > > know what the maximum value is (it will still be reduced for > > udp or if the server wants it smaller). > > Ok. I checked this on a system that is still running under 9.2 (we had to > downgrade most production systems), and bingo, 64K wsize. So our nfs server > (a NetApp Cluster) seems to prefer 64K. > > This means NFS still is a potential trigger of the problem. Let's pretend > it is, despite I think I already tried disabling TSO. Would this explain > all the symptoms we were seeing? Why would ifconfig down/up help? Do you > have a theory on that? > > > Markus > > > -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Sun Mar 2 02:51:13 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34F055D9; Sun, 2 Mar 2014 02:51:13 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 083D91194; Sun, 2 Mar 2014 02:51:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s222pCaw068149; Sun, 2 Mar 2014 02:51:12 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s222pCE0068148; Sun, 2 Mar 2014 02:51:12 GMT (envelope-from linimon) Date: Sun, 2 Mar 2014 02:51:12 GMT Message-Id: <201403020251.s222pCE0068148@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/185387: [axe] if_axe usb ethernet interface no ssh, no http with 10.0-RC3 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 02:51:13 -0000 Old Synopsis: if_axe usb ethernet interface no ssh, no http with 10.0-RC3 New Synopsis: [axe] if_axe usb ethernet interface no ssh, no http with 10.0-RC3 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 2 02:50:49 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=185387 From owner-freebsd-net@FreeBSD.ORG Sun Mar 2 03:29:47 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00BA99F6; Sun, 2 Mar 2014 03:29:46 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C6E9514C5; Sun, 2 Mar 2014 03:29:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s223TkCu078875; Sun, 2 Mar 2014 03:29:46 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s223Tkwc078874; Sun, 2 Mar 2014 03:29:46 GMT (envelope-from linimon) Date: Sun, 2 Mar 2014 03:29:46 GMT Message-Id: <201403020329.s223Tkwc078874@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187068: [em] network data slow/stops with em driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 03:29:47 -0000 Old Synopsis: network data slow/stops with em driver New Synopsis: [em] network data slow/stops with em driver Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 2 03:29:33 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=187068 From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 10:17:33 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7062C2D; Mon, 3 Mar 2014 10:17:33 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8960B2C7; Mon, 3 Mar 2014 10:17:33 +0000 (UTC) Received: from Julian-MBP3.local ([12.157.112.67]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s23AHJZ1073549 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 3 Mar 2014 02:17:22 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <531456AA.50203@freebsd.org> Date: Mon, 03 Mar 2014 02:17:14 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Giovanni Mattera , freebsd-virtualization@freebsd.org Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> In-Reply-To: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Maurizio Marini , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 10:17:33 -0000 On 3/2/14, 2:49 AM, Giovanni Mattera wrote: > good morning > I installed freebsd 10.0 on Hyper -V Windows Server 2012 R2. > I would like to enable the CARP protocol but is still in a state of init . > The error in dmessage is " ifa_del_loopback_route : deletion failed: 48" . > How do you configure CARP on Hyper -V ? > I've used both the card " Legacy Network " and " Network" but the result is the same. > We are available to pay for this. have you tried this kernel with real hardware? > Thank you. > > Giovanni Mattera > ------------------------------------------------- > CoST - Computers Services and Technologies s.r.l. > Via Giuseppe Longhi 13 > 20137 Milano MI > tel: +39 02454461 > fax: +39 0245446333 > cell:+393351217481 > http://www.cost.it > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. > La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. > Grazie. > > This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. > Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. > Thank you > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCA16E28 for ; Mon, 3 Mar 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C7C29949 for ; Mon, 3 Mar 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s23B6mWu008579 for ; Mon, 3 Mar 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s23B6mVa008577 for freebsd-net@FreeBSD.org; Mon, 3 Mar 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Mar 2014 11:06:48 GMT Message-Id: <201403031106.s23B6mVa008577@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183390 net [ixgbe] 10gigabit networking problems o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) o kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis p kern/165903 net mbuf leak o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 467 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 11:29:05 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C04D4E05; Mon, 3 Mar 2014 11:29:05 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C47AC60; Mon, 3 Mar 2014 11:29:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 9DC3C9896A2; Mon, 3 Mar 2014 12:28:04 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 0sddf2AAXezu; Mon, 3 Mar 2014 12:28:03 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 2304F987879; Mon, 3 Mar 2014 12:28:03 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 2304F987879 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393846083; bh=V12ljh/XR75Nx9iwlKYQ4X1xlfEMYHL8it1EIAmDDHk=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=A6/LQlSsLGIN9aaevicpZfb3xS1athK2vLfHSBq/AY9rKlGOAA5271dUMizyKNh2b u9IywCKy2XIyFo9x8V55KDtwpO1xOGJ6kAeK3UbcHLNaRAQn1sX1dATZM2VHTxau4t +/jtFYNHsJ58laXV/5wnSgilYGN1iJKmbBWMqyhs= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oAIr9h4_evDI; Mon, 3 Mar 2014 12:28:02 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id 9D69B986590; Mon, 3 Mar 2014 12:28:01 +0100 (CET) Date: Mon, 3 Mar 2014 12:28:50 +0100 From: Maurizio Marini To: Julian Elischer Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> In-Reply-To: <531456AA.50203@freebsd.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/LfyNV3t=vub5UTNXSfWYW=F"; protocol="application/pkcs7-signature" Cc: FreeBSD Net , Manuel Martini , freebsd-virtualization@freebsd.org, Giovanni Mattera X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 11:29:05 -0000 --Sig_/LfyNV3t=vub5UTNXSfWYW=F Content-Type: multipart/mixed; boundary="MP_/0Q1XfTp_K/vlErc8C/IdV4+" --MP_/0Q1XfTp_K/vlErc8C/IdV4+ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Mon, 03 Mar 2014 02:17:14 -0800 Julian Elischer wrote: > On 3/2/14, 2:49 AM, Giovanni Mattera wrote: > > good morning > > I installed freebsd 10.0 on Hyper -V Windows Server 2012 R2. > > I would like to enable the CARP protocol but is still in a state of ini= t . > > The error in dmessage is " ifa_del_loopback_route : deletion failed: 48= " . > > How do you configure CARP on Hyper -V ? > > I've used both the card " Legacy Network " and " Network" but the resul= t is > > the same. We are available to pay for this. >=20 > have you tried this kernel with real hardware? >=20 Hello Julian CARP with this kernel in other environment does work fine. The issue is that carp interface never goes MASTER, it stay in INIT state w= /out going into MASTER state. The error we see on dmesg is: ifa_del_loopback_route : deletion failed: 48. There is no block on MAC-ADDRESSES on Hyper-v, no firewall, nothing. We have tried with de interface (lelacy) and hn interface: same issue. this is the hn interface: hn0: on vmbus0 this is the ifconfig output: hn0: flags=3D8943 metric 0 = mtu 1500 options=3D18 ether 00:15:5d:2c:11:21 inet6 fe80::215:5dff:fe2c:1121%hn0 prefixlen 64 scopeid 0x3=20 inet 192.168.222.201 netmask 0xffffff00 broadcast 192.168.222.255=20 inet 192.168.222.200 netmask 0xffffff00 broadcast 192.168.222.255 v= hid 1=20 nd6 options=3D29 carp: INIT vhid 1 advbase 1 advskew 0 In attach I provide you full dmesg output I can provide you an ssh access to this freebsd virtual machine, no problem= at all. Googling around we find many people facing the same issue (CARP on Hyper-v), none was able to provide a solution in forums or mailing lists. Looking forward to hear from you. Best Regards --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --MP_/0Q1XfTp_K/vlErc8C/IdV4+ Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename=carp-dmesg.txt Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 module vmbus already present! module storvsc already present! module hn already present! module atapci_dis already present! CPU: Intel(R) Xeon(R) CPU 5140 @ 2.33GHz (1262.62-MHz K8-class = CPU) Origin =3D "GenuineIntel" Id =3D 0x6f6 Family =3D 0x6 Model =3D 0xf S= tepping =3D 6 Features=3D0xf83fbff Features2=3D0x80002201 AMD Features=3D0x20100800 AMD Features2=3D0x1 real memory =3D 2147483648 (2048 MB) avail memory =3D 2052059136 (1956 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: ioapic0: Changing APIC ID to 0 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 random: initialized vmbus0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, f7f00000 (3) failed cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177= ,0x376,0xffa0-0xffaf at device 7.1 on pci0 ata0: at channel 0 on atapci0 ata1: at channel 1 on atapci0 pci0: at device 7.3 (no driver attached) vgapci0: mem 0xf8000000-0xfbffffff irq 11 at devic= e 8.0 on pci0 vgapci0: Boot video device de0: port 0xec00-0xec7f mem 0xfebff000-0xfeb= fffff irq 11 at device 10.0 on pci0 de0: 21140A [10-100Mb/s] pass 2.0 de0: Ethernet address: 00:15:5d:2c:11:1a atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on= acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 orm0: at iomem 0xc0000-0xcbfff,0xcc000-0xcc7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 Timecounters tick every 20.000 msec storvsc0 on vmbus0 hyperv-utils0 on vmbus0 hyperv-utils0: Hyper-V Service attaching: Hyper-V Heartbeat Service hyperv-kvp0 on vmbus0 hyperv-kvp0: Hyper-V Service attaching: Hyper-V KVP Service hyperv-utils1 on vmbus0 hyperv-utils1: Hyper-V Service attaching: Hyper-V Shutdown Service hv_kvp_negotiate_version hyperv-utils2 on vmbus0 hyperv-utils2: Hyper-V Service attaching: Hyper-V Time Synch Service storvsc1 on vmbus0 Netvsc probe... DONE=20 hn0: on vmbus0 Netvsc initializing... hn0: Ethernet address: 00:15:5d:2c:11:21 Netvsc probe... DONE=20 hn1: on vmbus0 Netvsc initializing... Already initialized! hn1: Ethernet address: 00:15:5d:2c:11:22 random: unblocking device. da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: Fixed Direct Access SCSI-4 device=20 da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) cd0 at ata1 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-5 device=20 cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... Already initialized! Timecounter "TSC" frequency 1262623248 Hz quality 800 Trying to mount root from ufs:/dev/da0p2 [rw]... ifa_del_loopback_route: deletion failed: 48 carp: demoted by -240 to 0 (vhid removed) hn0: promiscuous mode disabled Closing Device ... Closing Device ... hn0: promiscuous mode enabled carp: demoted by 240 to 240 (interface down) --MP_/0Q1XfTp_K/vlErc8C/IdV4+-- --Sig_/LfyNV3t=vub5UTNXSfWYW=F Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzAzMTEyODUx WjAjBgkqhkiG9w0BCQQxFgQUpd07idz2Dx6kI4x8HTjHAaFR0PUwKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAFDbrQzT0hoZQrjAUeDqmZGm/2bMwMr7qAVDAK52rfBENrGWRznddP9wutyc0 v52qsjBD4CJW705EZfqQexpyNdMVI/tn+JBOD0xuQvATam7II2vkEkjvtP35TMse X4JpRTx9SMlRMo06Pg8Oj6ixzLsaheQgyq91cDp+3/6zvPqfmcyWllRL7AwidPuA U4sOCZlveuPs53VdIgI+matpJrI8IvuG5HpVFZGbMIhpS4bgSI4EnLBETt7ITksY re6TFfA7Oq+ZVv+LedDWq72wXzKgpjSwQ2AztI7pHaoVb/Wa9+DRRU+FEVlCgB3i Z1oMn/kK3aVJCuY+mTKm2A3QIgAAAAAAAA== --Sig_/LfyNV3t=vub5UTNXSfWYW=F-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 15:05:04 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97CDFFD7 for ; Mon, 3 Mar 2014 15:05:04 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3D748214 for ; Mon, 3 Mar 2014 15:05:02 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA10890 for ; Mon, 03 Mar 2014 17:05:00 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WKUQN-000O3p-1v for freebsd-net@FreeBSD.org; Mon, 03 Mar 2014 17:04:59 +0200 Message-ID: <531499E2.2080109@FreeBSD.org> Date: Mon, 03 Mar 2014 17:04:02 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net Subject: delayed_ack and Nagle's algorithm interaction X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 15:05:04 -0000 Guys, my knowledge of TCP and our TCP/IP stack is quite basic. It seems that we are running into a problem described here or something with very similar symptoms: http://www.stuartcheshire.org/papers/NagleDelayedAck/ At least, if I either disable delayed_ack or use TCP_NODELAY option, then the problem goes away. Otherwise, we see quite significant extra latencies. Maybe an important detail: the problem is observed for communication between a local host client and a server in a jail. So, all traffic is local (via loopback interface). I have a pcap file that captures the problem. Ideas, suggestion, etc are welcome. Thanks! -- Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 17:16:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DD9B571 for ; Mon, 3 Mar 2014 17:16:12 +0000 (UTC) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 01D6E6B for ; Mon, 3 Mar 2014 17:16:11 +0000 (UTC) Received: by mail-qg0-f51.google.com with SMTP id q108so12129257qgd.10 for ; Mon, 03 Mar 2014 09:16:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=TBMLtI3yBGX08Hau6eGoubdDC6MxP1Rn/HeA1Zfe34M=; b=VBvnnflm5fckOm8O5PdJYwN/1HBMYvWi1uS1OEo6yxNL5UlHf7Oxkj/PkxXqJZMfwH ki220t60Nt9Vg62VoxHn7pAjXDZJcGN+THbgWtPmlggUMx07zFPd8mTkPu9W8OmMeF4F FKnSqIK0zfePUMC7GnxBxDJkJG4q96CcOLZ5VgKNPNILB+gEtqzmTH/wDubLwh1bvxu+ zHxjVdKyfgcth+jn5sGMLKNZyKefGQiwIUYJ39do/RwUNW3hs5u0kHyzlJ8XfBmmCoq8 a9nWm6BcbRXhJqHGJPQG1hux7Ir4G9gxBd6Cd7LTz58uw6nqaaHQnPpZM4vXBIAYAP+5 E3/g== MIME-Version: 1.0 X-Received: by 10.140.28.134 with SMTP id 6mr24090466qgz.11.1393866971011; Mon, 03 Mar 2014 09:16:11 -0800 (PST) Received: by 10.224.191.71 with HTTP; Mon, 3 Mar 2014 09:16:10 -0800 (PST) Date: Mon, 3 Mar 2014 12:16:10 -0500 Message-ID: Subject: VALE not forwarding on physical interfaces From: Nicholas Bastin To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:16:12 -0000 I'm sure this is a local misconfiguration, but I've been unable to figure this out. I have a VALE bridge with 5 physical interfaces connected: $ sudo ./vale-ctl -l bdg_ctl [98] bridge:0 port:0 vale0:eth12 bdg_ctl [98] bridge:0 port:1 eth12 bdg_ctl [98] bridge:0 port:2 vale0:eth19 bdg_ctl [98] bridge:0 port:3 eth19 bdg_ctl [98] bridge:0 port:4 vale0:eth20 bdg_ctl [98] bridge:0 port:5 eth20 bdg_ctl [98] bridge:0 port:6 vale0:eth21 bdg_ctl [98] bridge:0 port:7 eth21 bdg_ctl [98] bridge:0 port:8 vale0:eth22 bdg_ctl [98] bridge:0 port:9 eth22 (All of these interfaces were attached with vale-ctl -h vale0:ethXX, as per the README). eth12 is the uplink to the rest of the network, and eth19-22 are all directly attached to client servers. eth12 is an Intel I354 embedded controller, and eth19-22 are 82580EB, all using the igb driver. There is some learning going on: $ tail -F syslog|grep learning Mar 3 07:12:33 bsslnx kernel: [419434.805535] 753.688227 [1086] netmap_bdg_learning src c8:2a:14:1c:c5:c0 on port 0 Mar 3 07:12:33 bsslnx kernel: [419434.835356] 753.718022 [1086] netmap_bdg_learning src c8:2a:14:1c:c5:c0 on port 0 Mar 3 07:12:36 bsslnx kernel: [419437.297646] 756.178393 [1086] netmap_bdg_learning src 00:25:90:4c:35:c9 on port 4 Mar 3 07:12:37 bsslnx kernel: [419438.176860] 757.056920 [1086] netmap_bdg_learning src 00:0b:78:66:3a:0b on port 0 Mar 3 07:12:39 bsslnx kernel: [419440.140832] 759.019363 [1086] netmap_bdg_learning src 28:cf:e9:17:f6:ad on port 0 Mar 3 07:12:39 bsslnx kernel: [419440.172588] 759.051092 [1086] netmap_bdg_learning src 28:cf:e9:17:f6:ad on port 0 Mar 3 07:12:39 bsslnx kernel: [419440.357836] 759.236196 [1086] netmap_bdg_learning src 00:25:90:4c:35:c9 on port 4 But there is no reachability and while I haven't put in a tap yet (that is next), there doesn't appear to be any forwarding going on. When I ping from c8:2a:14:1c:c5:c0 (which then causes it to arp for the destination), I get the following: Mar 3 07:07:20 bsslnx kernel: [419121.098837] 440.226179 [1086] netmap_bdg_learning src c8:2a:14:1c:c5:c0 on port 0 Mar 3 07:07:20 bsslnx kernel: [419121.100163] 440.227506 [1216] nm_bdg_flush slot 0 port 0 -> 254 Mar 3 07:07:20 bsslnx kernel: [419121.101520] 440.228860 [1527] netmap_vp_txsync vale0:eth12 ring 0 flags 0 Mar 3 07:07:20 bsslnx kernel: [419121.101528] 440.228869 [2505] netmap_common_irq received TX queue 0 Mar 3 07:07:20 bsslnx kernel: [419121.101530] 440.228874 [1737] netmap_bwrap_intr_notify eth19 TX0 0x0 Mar 3 07:07:20 bsslnx kernel: [419121.101532] 440.228876 [2505] netmap_common_irq received RX queue 0 Mar 3 07:07:20 bsslnx kernel: [419121.101534] 440.228878 [1737] netmap_bwrap_intr_notify eth19 RX0 0x0 Mar 3 07:07:20 bsslnx kernel: [419121.101537] 440.228880 [1777] netmap_bwrap_intr_notify eth19 head 0 cur 0 tail 0 (kring 0 0 0) Mar 3 07:07:20 bsslnx kernel: [419121.101539] 440.228883 [1796] netmap_bwrap_intr_notify how strange, interrupt with no packets on eth19 I'm not sure if that has any relevant information... Thanks, Nick From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 20:55:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB013669; Mon, 3 Mar 2014 20:55:15 +0000 (UTC) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 885E5C4A; Mon, 3 Mar 2014 20:55:15 +0000 (UTC) Received: by mail-ve0-f173.google.com with SMTP id oy12so2098516veb.18 for ; Mon, 03 Mar 2014 12:55:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=JdSynC/ldzqDA39xlwY6fjDHiOS3aONJvt6gABRQleA=; b=enPkRVHZgbhuaN3++U/Vsh8BdcR+XZKxIMWRi5gtUmH7e2o4jDbyaTvssSlzA5cXK4 w+YBG9Cz6mLiIg+QnpA2LjsZo3LjyxJjOIdow9SE+5qYo5c4N9kteHdcSwVZmIKWu6WQ 8K/Ygmm+Gltvif7aq5DdlwcvVIvAqcSMjTdfqzBsBUqHHozpSp9lkzl53h+pBVFdGH3n h69FvylWFxp5fU5aQA5ChrOJPsfXN+n6C3qkyVKcKxeQj75TRvoajRReSN1rEatMeVMj cjPiblx8PSZE6oQ2g7AcV1o/Dr1dVCGhiw1JJO+HDFNa/lSplFFHZ/G3G1VQgmi7ZL9T sIOA== X-Received: by 10.220.250.203 with SMTP id mp11mr18781697vcb.2.1393880114469; Mon, 03 Mar 2014 12:55:14 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.188.35 with HTTP; Mon, 3 Mar 2014 12:54:54 -0800 (PST) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Mon, 3 Mar 2014 21:54:54 +0100 X-Google-Sender-Auth: XkFI0EertvH62-zkxCUmMHyRu-w Message-ID: Subject: Strange network performance on Intel Rangeley (8 cores Atom) To: "freebsd-net@freebsd.org" , freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 20:55:16 -0000 Hi all, I've got a new toy in my network bench lab: a SuperMicro SuperServer 5018A-FTN4. But I've got a problem for understanding and obtaining good throughput for "routing" or "firewalling" usages. I'm using only the embedded 4 gigabit ports of the Atom C2758 SoC. With the default igb(4) parameters which is to create 8 queues (because there is 8 cores) this server is not able to receive more than 585K packet-per-seconds into one port which is far from the gigabit line-rate (1.48Mpps): I was expecting better throughput with 8 cores. Then I did a bunch of new benchmarks by measuring the impact of number of queue and the results are here: http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4#graph => I've got better results with only 4 queues than 8... but still low throughput with only 938Kpps. Then I decided to measure the impact of pf and ipfw on the throughput with 4 and 8 queues. And the results are annoying: http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4#graph1 => With 8 queues, enabling pf or ipfw improve the input throughput of the igb(4) port. Why so low throughput with 8 queues ? Why better throughput with pf or ipfw enabled than without ? Thanks, Olivier From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 21:12:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BDED295; Mon, 3 Mar 2014 21:12:35 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 132A818C; Mon, 3 Mar 2014 21:12:34 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id ld13so4099497vcb.4 for ; Mon, 03 Mar 2014 13:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2YhHLr2yG3koVqsEuKa4NeHahw8jnCd+XtKlwDNlni4=; b=dtXgVReBZ736joZk6qUJsg+iyUPUkS2ThySZA30JdOIyQEC+Wpy/aT5ueIkk99RFou /kMXBtcEbaCi6FVtZUP/2NY1+lL1vohSVGFGO0TotkavBu/eevQov9Vc/iz2W+3gF8W9 kdgYmccCK1f52GCgXmOpohxcIC7ybAuM3m5XgXJFzqEDhURsbiAGtcL2TQWuabzGwFtK TNF5GTyGpx81952myd4xwB0zGDUmqjijdFuS1etnHvz/sulWeSVp2zHTJ5FYMnlZSgql p5CcTuW0aXVGgYy2dW8dqNe9clWU2gVCjXI1+SYD8+9ykmzpd5lmmWtz8UQmTRT7501B rJug== MIME-Version: 1.0 X-Received: by 10.58.123.70 with SMTP id ly6mr18857384veb.26.1393881154154; Mon, 03 Mar 2014 13:12:34 -0800 (PST) Received: by 10.221.11.135 with HTTP; Mon, 3 Mar 2014 13:12:34 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 13:12:34 -0800 Message-ID: Subject: Re: Strange network performance on Intel Rangeley (8 cores Atom) From: Jack Vogel To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , "freebsd-performance@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:12:35 -0000 What OS version are you running? Jack On Mon, Mar 3, 2014 at 12:54 PM, Olivier Cochard-Labb=E9 wrote: > Hi all, > > I've got a new toy in my network bench lab: a SuperMicro SuperServer > 5018A-FTN4. > But I've got a problem for understanding and obtaining good throughput fo= r > "routing" or "firewalling" usages. > > I'm using only the embedded 4 gigabit ports of the Atom C2758 SoC. > With the default igb(4) parameters which is to create 8 queues (because > there is 8 cores) this server is not able to receive more than 585K > packet-per-seconds into one port which is far from the gigabit line-rate > (1.48Mpps): I was expecting better throughput with 8 cores. > Then I did a bunch of new benchmarks by measuring the impact of number of > queue and the results are here: > > http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_s= uperserver_5018a-ftn4#graph > > =3D> I've got better results with only 4 queues than 8... but still low > throughput with only 938Kpps. > > Then I decided to measure the impact of pf and ipfw on the throughput wit= h > 4 and 8 queues. > And the results are annoying: > > http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_s= uperserver_5018a-ftn4#graph1 > > =3D> With 8 queues, enabling pf or ipfw improve the input throughput of t= he > igb(4) port. > > Why so low throughput with 8 queues ? > Why better throughput with pf or ipfw enabled than without ? > > Thanks, > > Olivier > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 3 21:30:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 315BFB8A; Mon, 3 Mar 2014 21:30:25 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D01AC2D8; Mon, 3 Mar 2014 21:30:24 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id pa12so4246304veb.15 for ; Mon, 03 Mar 2014 13:30:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=d1JpshKn+2/nVSeWNsMxW4IRotf6XO4I16IftypQjL4=; b=mRG3dMYYv/DFjHjLwK7BBytULYcui2Ugt4cPYH0VeOlegvWCSWlBRnfUhdAjReSjTS L7slnSsCDta4G92AnrXGh9JKybHE1fJSGl9spVCyA05ANPfqLT7B5+6sMJO5Q3RC/Kp2 VJA86Wm5PdVj0T6x2QgifgLtNxuf56dnJ5+5McuDdyywVKzwxeQ0283ZZRcLzgoqO7Wi hrnyccW9DORx9bAsr+QEMjTtIA3E6gNjO6IDKgUWYls5gJqMg8/R1/ymtCVMogch1h78 pj69zxgQcPDIb31rxoKdZl8MqnRuWKEjshCaFeGxyl7GAHCOo1V0NBbPHxMMvVMXxSSE lKmA== X-Received: by 10.58.37.67 with SMTP id w3mr18756606vej.22.1393882224066; Mon, 03 Mar 2014 13:30:24 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.188.35 with HTTP; Mon, 3 Mar 2014 13:30:03 -0800 (PST) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Mon, 3 Mar 2014 22:30:03 +0100 X-Google-Sender-Auth: gIU4ZUjLPiEuIWuMCMB-LK4AXUw Message-ID: Subject: Re: Strange network performance on Intel Rangeley (8 cores Atom) To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , "freebsd-performance@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:30:25 -0000 On Mon, Mar 3, 2014 at 10:12 PM, Jack Vogel wrote: > What OS version are you running? > Hi Jack, More information about the version used: FreeBSD 10.0-STABLE #0 r262601M And about the igb(4) parameters common to all my tests: hw.igb.rxd="2048" hw.igb.txd="2048" hw.igb.rx_process_limit="-1" hw.igb.max_interrupt_rate="16000" Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 01:11:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E14319A7; Tue, 4 Mar 2014 01:11:21 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 91220AF3; Tue, 4 Mar 2014 01:11:21 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so4646174qcx.20 for ; Mon, 03 Mar 2014 17:11:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=0lxip/oEUNs/BjXAsRe6KF4nzvPM1g+d9xVRZDnNR4s=; b=ZxzXpigoiLRvq3IR8en2Pl23/fpG+FrkJCUwLD58uRb1bczoowkajsz3MCHbfep3cx 4F7zMvUYoVPmTtdZ8e2gT3tBphiqsKccWvPXmH0RJ9zPtmUFa81vqlKvREeUkkltBs1g BqW6do9g+QHWVbDGLFvxhWgPdXXa4RX/IVr+E1Fx9PYaZnHpmHgyglYmm7Os2eM1n1qB zpYCabPwI6IrmDNmiUHPLlR1+I9CxAcfMeobY6Q+nYIXOl3fmfGuOB/yulJNYPwLLI8I zVdhwIdC+T4vQUh/dLixnhsabSGfiBlo58YCrF2iRUbxyukbyoIo9VGlIHsxTyPjDro1 lYVw== MIME-Version: 1.0 X-Received: by 10.140.42.21 with SMTP id b21mr729774qga.87.1393895480733; Mon, 03 Mar 2014 17:11:20 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Mon, 3 Mar 2014 17:11:20 -0800 (PST) In-Reply-To: <531499E2.2080109@FreeBSD.org> References: <531499E2.2080109@FreeBSD.org> Date: Mon, 3 Mar 2014 17:11:20 -0800 X-Google-Sender-Auth: iCAkPeBK4Uxr_0Ld5ud62HHmn9U Message-ID: Subject: Re: delayed_ack and Nagle's algorithm interaction From: Adrian Chadd To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 01:11:21 -0000 What's the time interval between transmissions? Is it something dumb like bad timer behaviour? -a On 3 March 2014 07:04, Andriy Gapon wrote: > > Guys, > > my knowledge of TCP and our TCP/IP stack is quite basic. > It seems that we are running into a problem described here or something with > very similar symptoms: > http://www.stuartcheshire.org/papers/NagleDelayedAck/ > At least, if I either disable delayed_ack or use TCP_NODELAY option, then the > problem goes away. Otherwise, we see quite significant extra latencies. > Maybe an important detail: the problem is observed for communication between a > local host client and a server in a jail. So, all traffic is local (via > loopback interface). > > I have a pcap file that captures the problem. > Ideas, suggestion, etc are welcome. > Thanks! > -- > Andriy Gapon > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 01:17:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCA3FD93; Tue, 4 Mar 2014 01:17:09 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 790E6B36; Tue, 4 Mar 2014 01:17:09 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so4256646qaj.39 for ; Mon, 03 Mar 2014 17:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/BSvHUXI/1mzcKfA02fxoe/7E4W8rPdRl519zAHMYvI=; b=VILdh2p3V1Tnetbgmee2b/7R2ceKK5LeZBFUsAYfaaTFB8Vb0r4bokQgMsc1XenZ0L SfAR+gqP+hk19O6awjdsAG3R9oLN4kUEMaShq4xYMEtStnn5sMEk9GmzQbqmkO0L2PoU DUFF4q2R0yXZp+j4zCvrTyY7V+8+LkFdR8RLrS+z0v/gUJcOtl3MLlJD0qiwFbf/bXbl PqaFLFMoTZXRJKW7uL34nxJIOhJ7CLkL25SHALGvrJRX3OQ6II2nnPdoKqVsTBqLt5Rt EloINpoVTN+SwBCtyBOrTw1gEq0NqFURX9n6ed3JgfTs1PKTgjUJGrDKzBVJMBl6W8+P AWOw== MIME-Version: 1.0 X-Received: by 10.140.42.138 with SMTP id c10mr26749023qga.24.1393895828549; Mon, 03 Mar 2014 17:17:08 -0800 (PST) Received: by 10.224.16.10 with HTTP; Mon, 3 Mar 2014 17:17:08 -0800 (PST) Date: Mon, 3 Mar 2014 17:17:08 -0800 Message-ID: Subject: UDP transmit and no flowid From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 01:17:09 -0000 Hi, So something I was told about whilst investigating RSS at Netflix was that the UDP path doesn't actually get set by anything unless you're using flowtable. So, if one doesn't define flowtable, the transmit path congests quite heavily through one TX queue. I was about to do up a simple hack to do a toeplitz hash in the UDP send path before it gets bumped up to ip_output() - that way it can set the flowid correctly. The other option is to do a topelitz hash in ip_output() if m_flowid isn't set. The downside there is that we'd have to check whether it's actually a TCP, UDP, or IP flow before we assign it a flowid. In any case, this would likely decongest the transmit path quite a bit for UDP send. It's still not completely RSS-y (we would still need to line up transmit and receive paths to minimise lock contention) but it seems like a necessary first step in that direction. What do people think? I'll try this out in the next week or two once I've sorted out my employment situation and I can reserve some time on the netperf cluster. (before people ask "why udp?" - I'm kinda fed up with memcached performing much worse on freebsd than linux. Since fixing the UDP side of things requires a subset of the same work needed for TCP, I figure we should tackle UDP first.) Thanks, -adrian From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 01:25:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F31921B4; Tue, 4 Mar 2014 01:25:29 +0000 (UTC) Received: from mail-ve0-x231.google.com (mail-ve0-x231.google.com [IPv6:2607:f8b0:400c:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9BFFFC0A; Tue, 4 Mar 2014 01:25:29 +0000 (UTC) Received: by mail-ve0-f177.google.com with SMTP id sa20so4597635veb.36 for ; Mon, 03 Mar 2014 17:25:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=w1yJ+v2OasfnlKYxCV23Os7Uko9DjCD+Y3Rg828bKMQ=; b=ITTZeWi9zO7FuUv7ciqzLQNKfPfmMByWCMfsS/dBJiT29bl4SAoKkCZTnqq1p1hpcW sY82ocjiCV7Ts4bB6cBvwbmRoAWX/dxuTyPVzC6s+Ei7HjaOcFihfaLE9AA8fLDd4GRb zsgHNiRCMvJ8O5kQ1GWpHFTPJ1I921karVXCZcI1Iik8cyYxGZP4bhz/f4bsahWLQcci bNomixxtGa8p4b+JlgZszfQI5t1obPPgI41bjPMRqCv+ap5SQfUe900TT0/XtmT4LE22 4USXIZOY6uni5QV9tSwTj7HtNy2Ku6msDJE+yS583sPBj1OaIElEC9KEsyoTwVAlAzAL 4SOg== MIME-Version: 1.0 X-Received: by 10.220.136.6 with SMTP id p6mr18945741vct.9.1393896328763; Mon, 03 Mar 2014 17:25:28 -0800 (PST) Received: by 10.221.11.135 with HTTP; Mon, 3 Mar 2014 17:25:28 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 17:25:28 -0800 Message-ID: Subject: Re: UDP transmit and no flowid From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 01:25:30 -0000 Funny, Jeff Pieper and I were just talking about how it seems UDP performance, errr is less than optimal, so hey trying something that might help sounds like a good idea to me. Will be curious to hear the results! Cheers, Jack On Mon, Mar 3, 2014 at 5:17 PM, Adrian Chadd wrote: > Hi, > > So something I was told about whilst investigating RSS at Netflix was > that the UDP path doesn't actually get set by anything unless you're > using flowtable. So, if one doesn't define flowtable, the transmit > path congests quite heavily through one TX queue. > > I was about to do up a simple hack to do a toeplitz hash in the UDP > send path before it gets bumped up to ip_output() - that way it can > set the flowid correctly. > > The other option is to do a topelitz hash in ip_output() if m_flowid > isn't set. The downside there is that we'd have to check whether it's > actually a TCP, UDP, or IP flow before we assign it a flowid. > > In any case, this would likely decongest the transmit path quite a bit > for UDP send. It's still not completely RSS-y (we would still need to > line up transmit and receive paths to minimise lock contention) but it > seems like a necessary first step in that direction. > > What do people think? > > I'll try this out in the next week or two once I've sorted out my > employment situation and I can reserve some time on the netperf > cluster. > > (before people ask "why udp?" - I'm kinda fed up with memcached > performing much worse on freebsd than linux. Since fixing the UDP side > of things requires a subset of the same work needed for TCP, I figure > we should tackle UDP first.) > > Thanks, > > > -adrian > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 05:58:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49A17D33 for ; Tue, 4 Mar 2014 05:58:15 +0000 (UTC) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 125422CA for ; Tue, 4 Mar 2014 05:58:15 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so8117958oag.27 for ; Mon, 03 Mar 2014 21:58:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=olHVIiMI29mXbf6QUVkdERS1H+9Q+NL6Dm/dGJxN0Bw=; b=upjYUqc8yLsv4uiIqN5Uc2hJCs3a1CtCZ5LBNe2sOU1SfLQiZB9PVqqDlwgDifXFQu 5tPU6GBGPjZOdX+1dtYWQbKQtWFcvJOcuz3N7UxpA/7pK3kDkuLTIfT6QJckeuvfs8g+ SBUbvL/kQ7StO4UtDvLYwd48IiyLLti5CVghXZvgiiNuROzqLVHO/wz+Gd9vuwcSF6Bt cdodYCmfcKl3KwAwbVq/ViNQcSujUi+FF+ewcZkBj+TCFdtg9i/GYB0LfRv66Awh8XH6 jhmN63g/XqHXNMnlytH3rtE5T1gYS8LDYZ/qyv/xBY0lHRpr0OGQbuSsQjQoW98Ib9be ZhLg== MIME-Version: 1.0 X-Received: by 10.182.233.228 with SMTP id tz4mr138030obc.56.1393912694377; Mon, 03 Mar 2014 21:58:14 -0800 (PST) Received: by 10.76.144.10 with HTTP; Mon, 3 Mar 2014 21:58:14 -0800 (PST) Date: Tue, 4 Mar 2014 06:58:14 +0100 Message-ID: Subject: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 05:58:15 -0000 Hello, I'm having a strange problem with ipfw and/or routing. I've only tested this on 9.2-RELEASE-p3, amd64. The machine is sort of acting as router. The ruleset is like (ipfw defaults to accept): $cmd="ipfw -fq " $cmd add 1 skipto 65534 log all from "table(1)" to any in recv "table(8)" ... $cmd add 65534 fwd tablearg all from "table(12)" to any Table 1 contains prefixes that should skip the normal rules and just pass through the box. Table 8 contains interface names. Table 12 is empty (so far). What happens is that packets that trigger the first rule never get to their destination. After looking at /var/log/security is see that packets trigger the rule, "never to be seen again". There is a route (ie not default) for the destination, but a tcpdump on the corresponding interface shows nothing. On changing the ruleset to $cmd="ipfw -fq " $cmd add 1 skipto 65533 log all from "table(1)" to any in recv "table(8)" ... $cmd add 65533 fwd x.y.z.w ip from "table(1)" to any in recv "table(8)" $cmd add 65534 fwd tablearg all from "table(12)" to any packets get to where they should. Why do I need the explict fwd rule? As far as I can see the ipfw man page says nothing about skipto changing the packets, and since the 65533 rule in the second ruleset triggers on the same thing as the skipto rule it would seem like packets are "intact". Why does the kernel not forward those packets? Best regards Andreas Nilsson From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 10:27:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7E4AF56 for ; Tue, 4 Mar 2014 10:27:29 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85E39E58 for ; Tue, 4 Mar 2014 10:27:29 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id u57so2740766wes.8 for ; Tue, 04 Mar 2014 02:27:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=goTYFgsRD0N6wIIl2sNXxT+HjEaeXqJF9uLgWCA1TTY=; b=Nx1rVSSH2y9rARPW1qMUzlyvDDJ6D2vVxWXu/x7utNtt8naUAqEOt/rC4HfhPBjxip tewEC3MhuD9vPW/7GusRWPXOwtye51JeGQTNjuWaX/yec1bIg8F81z/8tnzfX7VTprja iYIF9uqewW23mUih19WspJknZBWEXNh0KX6/Yx5cQOzbRwh29TQVRepQPyknNY4IL2LE AwCkQbBMVdyj5qLrRv9L/DiHMpqETdw6TPKjKWr5lgLYRGOx/4xSIIAmrZVhf98DnH1f k7wrYs7+h+MF+3a55FojPwMqwgg/LWXb7CNzwMb8dAP8GOWanwGFbP1//ybMrBU40AeL F67g== MIME-Version: 1.0 X-Received: by 10.194.63.228 with SMTP id j4mr26688619wjs.34.1393928843987; Tue, 04 Mar 2014 02:27:23 -0800 (PST) Received: by 10.194.29.163 with HTTP; Tue, 4 Mar 2014 02:27:23 -0800 (PST) Date: Tue, 4 Mar 2014 10:27:23 +0000 Message-ID: Subject: netmap-libpcap doesn't installs under FreeBSD10 From: "C. L. Martinez" To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 10:27:30 -0000 Hi all, When I try to compile netmap-libpcap, these errors appears: root@plzfsiem01:/tmp/j/netmap-libpcap # make cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-bpf.c cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-netmap.c ./pcap-netmap.c:117:9: warning: implicit declaration of function 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] ret = nm_dispatch((void *)d, cnt, (void *)pcap_netmap_filter, (void *)p); ^ ./pcap-netmap.c:131:9: warning: implicit declaration of function 'nm_inject' is invalid in C99 [-Wimplicit-function-declaration] return nm_inject(d, buf, size); ^ ./pcap-netmap.c:139:15: error: variable has incomplete type 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:139:9: note: forward declaration of 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:140:19: error: incomplete definition of type 'struct nm_desc' int error, fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:152:7: error: use of undeclared identifier 'SIOCSIFFLAGS' case SIOCSIFFLAGS: ^ ./pcap-netmap.c:157:10: warning: implicit declaration of function 'ioctl' is invalid in C99 [-Wimplicit-function-declaration] error = ioctl(fd, what, &ifr); ^ ./pcap-netmap.c:159:4: error: incomplete definition of type 'struct nm_desc' d->req.nr_name, what, error); ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:163:7: error: use of undeclared identifier 'SIOCGIFFLAGS' case SIOCGIFFLAGS: ^ ./pcap-netmap.c:177:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:178:18: error: use of undeclared identifier 'IFF_PPROMISC' if (if_flags & IFF_PPROMISC) { ^ ./pcap-netmap.c:179:17: error: use of undeclared identifier 'IFF_PPROMISC' if_flags &= ~IFF_PPROMISC; ^ ./pcap-netmap.c:180:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ ./pcap-netmap.c:183:2: warning: implicit declaration of function 'nm_close' is invalid in C99 [-Wimplicit-function-declaration] nm_close(d); ^ ./pcap-netmap.c:195:22: warning: implicit declaration of function 'nm_open' is invalid in C99 [-Wimplicit-function-declaration] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ./pcap-netmap.c:195:18: warning: incompatible integer to pointer conversion initializing 'struct nm_desc *' with an expression of type 'int' [-Wint-conversion] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./pcap-netmap.c:210:36: error: incomplete definition of type 'struct nm_desc' __FUNCTION__, p->opt.source, d, d->fd, ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:213:11: error: incomplete definition of type 'struct nm_desc' p->fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:27: error: incomplete definition of type 'struct nm_desc' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:45: error: use of undeclared identifier 'NETMAP_SW_RING' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ^ ./pcap-netmap.c:215:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:216:20: error: use of undeclared identifier 'IFF_PPROMISC' if (!(if_flags & IFF_PPROMISC)) { ^ ./pcap-netmap.c:218:16: error: use of undeclared identifier 'IFF_PPROMISC' if_flags |= IFF_PPROMISC; ^ ./pcap-netmap.c:219:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ 6 warnings and 17 errors generated. *** Error code 1 Stop. make: stopped in /tmp/j/netmap-libpcap Any patch?? From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 11:12:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF8C6B8E for ; Tue, 4 Mar 2014 11:12:01 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6FE732F9 for ; Tue, 4 Mar 2014 11:12:01 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id w62so2941003wes.14 for ; Tue, 04 Mar 2014 03:12:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=nXPnXlyeQRKYlylwro7a01o2ZgOSQZ2D9gOp3y8+0qA=; b=K5iYdiiJj85mAkG153e46ltRG89UdDuLmHfd9HlT9BqAQ/qpPVDG6NQ62zTjVrQ4jj ETkvCp50bIGh/biYKNeqb1jyBlVf215IcJNlR3J+EzJKFC2PFFjORsbQ1bvWSf9bO7BH aAmLnvPjvp8qrwEMfDWkeHMGjyP6YHsijONw2SUcR3zKzTZ+m/Q9oulvhe4QnulSI23p F1Gn+XqfK5tYYvZKnS5S/dVAUFrI7lPawLI5MQe8cYpw4KDBYzeNz3vXq1ryIUWVicCv 8RrMN6+tMnRmtTvzO8d3yqCs9fhw5Gv5muvZvaeqO/VGyOjHkLTKXj4AFv6LtAXkLCd6 L+Cw== MIME-Version: 1.0 X-Received: by 10.194.203.200 with SMTP id ks8mr15024105wjc.61.1393931519923; Tue, 04 Mar 2014 03:11:59 -0800 (PST) Received: by 10.194.29.163 with HTTP; Tue, 4 Mar 2014 03:11:59 -0800 (PST) Date: Tue, 4 Mar 2014 11:11:59 +0000 Message-ID: Subject: Bro doesn't builds using From: "C. L. Martinez" To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:12:01 -0000 Hi all, Using bro netmap patch provided (), Bro 2.2 doesn't compile under FreeBSD 10 amd64: [ 77%] Building CXX object src/CMakeFiles/bro.dir/PktSrc.cc.o /tmp/j/bro-2.2/src/PktSrc.cc:100:6: error: use of undeclared identifier 'IS_NETMAP_DESC' if (IS_NETMAP_DESC(pd)) ^ /tmp/j/bro-2.2/src/PktSrc.cc:101:22: error: use of undeclared identifier 'nm_nextpkt' data = last_data = nm_nextpkt((struct nm_desc *)pd, ^ /tmp/j/bro-2.2/src/PktSrc.cc:438:7: error: use of undeclared identifier 'IS_NETMAP_DESC' if (IS_NETMAP_DESC(pd)) ^ /tmp/j/bro-2.2/src/PktSrc.cc:439:4: error: use of undeclared identifier 'nm_close'; did you mean 'sem_close'? nm_close((struct nm_desc *)pd); ^~~~~~~~ sem_close /usr/include/semaphore.h:52:6: note: 'sem_close' declared here int sem_close(sem_t *); ^ /tmp/j/bro-2.2/src/PktSrc.cc:439:13: error: cannot initialize a parameter of type 'sem_t *' (aka '_sem *') with an rvalue of type 'struct nm_desc *' nm_close((struct nm_desc *)pd); ^~~~~~~~~~~~~~~~~~~~ /usr/include/semaphore.h:52:23: note: passing argument to parameter here int sem_close(sem_t *); ^ /tmp/j/bro-2.2/src/PktSrc.cc:479:7: error: use of undeclared identifier 'IS_NETMAP_DESC' if (IS_NETMAP_DESC(pd)) ^ /tmp/j/bro-2.2/src/PktSrc.cc:526:17: error: use of undeclared identifier 'nm_open' pd = (pcap_t *)nm_open(interface, getenv("NETMAP_RING_ID"), 0, 0); ^ /tmp/j/bro-2.2/src/PktSrc.cc:533:20: error: use of undeclared identifier 'NETMAP_FD' selectable_fd = NETMAP_FD(pd); ^ 8 errors generated. *** Error code 1 Stop. make[3]: stopped in /tmp/j/bro-2.2/build *** Error code 1 Stop. make[2]: stopped in /tmp/j/bro-2.2/build *** Error code 1 Stop. make[1]: stopped in /tmp/j/bro-2.2/build *** Error code 1 Stop. make: stopped in /tmp/j/bro-2.2 Do I need to add something?? Thanks. From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 11:16:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A53C7C9A; Tue, 4 Mar 2014 11:16:32 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 6608A355; Tue, 4 Mar 2014 11:16:32 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1WKnKp-000Nlf-0x; Tue, 04 Mar 2014 15:16:31 +0400 Date: Tue, 4 Mar 2014 15:16:31 +0400 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: UDP transmit and no flowid Message-ID: <20140304111630.GA84822@zxy.spb.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:16:32 -0000 On Mon, Mar 03, 2014 at 05:17:08PM -0800, Adrian Chadd wrote: > So something I was told about whilst investigating RSS at Netflix was > that the UDP path doesn't actually get set by anything unless you're > using flowtable. So, if one doesn't define flowtable, the transmit > path congests quite heavily through one TX queue. Now you touch UDP? Can you add "back pressure" for UDP sockets? (currenly UDP socket don't signaled about output buffer full and just drop packets, regardless) From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 11:45:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5B099AB for ; Tue, 4 Mar 2014 11:45:54 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6DADD87A for ; Tue, 4 Mar 2014 11:45:54 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id mc6so5410314lab.8 for ; Tue, 04 Mar 2014 03:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=lf8U7XSs5EJjpfI7a5zBLX86AohfTiDpum1D2YsYum8=; b=jNYjSx50bnhU3WDNSCWJJlbLZTlkP82jmmXc7zXjYbfXA8pM9aKIjezm6ABQQ5Pt1p hr2v7S4JmGb3OhT/iSlmQhSO/zgSBOYYPqwlrt9fuMGiCvMlvn2bfXOkIUZECi5hDTSs JU7HDgXmy/DTaMS+MWLcwGC2b6ho37MmltFk60wkocPFiG6tIRlN3Ie1h1gIF7f69uhN jOazW8kkQcW2UzU2k5u6zjSvU8CrO2WcSBHtEkyRamQRszH+HEkz1fGnpOT8yrzj/YTa hsvpjWIdWpZhTAnA1EfPnMv7OQtEKV+3a312isnfIeOLohI7BPHbJc6dn8yrmXxwBs38 fRmw== MIME-Version: 1.0 X-Received: by 10.152.28.129 with SMTP id b1mr27278362lah.9.1393933552541; Tue, 04 Mar 2014 03:45:52 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.115.4.162 with HTTP; Tue, 4 Mar 2014 03:45:52 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Mar 2014 12:45:52 +0100 X-Google-Sender-Auth: d5AMICNHgIKfGtSOPci9-UXTRZI Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: Luigi Rizzo To: "C. L. Martinez" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:45:55 -0000 On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez wrote: > Hi all, > > When I try to compile netmap-libpcap, these errors appears: > > root@plzfsiem01:/tmp/j/netmap-libpcap # make > cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > -c ./pcap-bpf.c > cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > -c ./pcap-netmap.c > ./pcap-netmap.c:117:9: warning: implicit declaration of function > 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] > ret = nm_dispatch((void *)d, cnt, (void > *)pcap_netmap_filter, (void *)p); > almost surely you have an old version of netmap.h and netmap_user.h in /usr/include/net You should update to the version in stable/10 (at the very least manually copy these two headers) cheers luigi From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 11:50:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46426B90 for ; Tue, 4 Mar 2014 11:50:31 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B0556912 for ; Tue, 4 Mar 2014 11:50:30 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id 10so5460598lbg.21 for ; Tue, 04 Mar 2014 03:50:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Rhm9xDcYMSnnHGu/gwuV5By9K7AcXlirUrF6vieaFmY=; b=jq7mrVE8y27qHooR1TL7O6KNuNGoICVpIZ/Rf/n4K8TDurvlDAySfppg7nc8GwWaFn qaQs3O2DMD8c2Ut/sEvvB7y6QVg82SyrBMMaXE+Dc9Up4bTEvSXcLC5onS8nUp2JuRDq U34IWZVP2v9RBX0ynH1ibMng02s+g4TA1o9ShRzSk5Pu57rQ5rXhNG2qf0wd6DFN0P8Z MZpqaCqGJ2P+4E7ZdNDJ+hchgzgJ+zC3Q7lof7kLGULlBxbK8QbLJ7q21eLHJF5rCqLO d9LGBH5KXChWHqU8ZWK31VSHK6SF+CqWYnj/p7fvixxfAKyHsKhLyTGK1tJWx2aG1ch2 qAlQ== MIME-Version: 1.0 X-Received: by 10.112.144.35 with SMTP id sj3mr1374347lbb.44.1393933828809; Tue, 04 Mar 2014 03:50:28 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.115.4.162 with HTTP; Tue, 4 Mar 2014 03:50:28 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Mar 2014 12:50:28 +0100 X-Google-Sender-Auth: 8CCA4retiYOa3XzC7-nFUwNRrmU Message-ID: Subject: Re: Bro doesn't builds using From: Luigi Rizzo To: "C. L. Martinez" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:50:31 -0000 On Tue, Mar 4, 2014 at 12:11 PM, C. L. Martinez wrote: > Hi all, > > Using bro netmap patch provided (), Bro 2.2 doesn't compile under > FreeBSD 10 amd64: > i am not sure if the bro patch is up to date, but likely you have the same outdated header as when you tried to build netmap-libpcap. This said, for bro you are probably better off just using it on the netmap-libpcap, there is almost no advantage in using the native API just for reading. cheers luigi > [ 77%] Building CXX object src/CMakeFiles/bro.dir/PktSrc.cc.o > /tmp/j/bro-2.2/src/PktSrc.cc:100:6: error: use of undeclared > identifier 'IS_NETMAP_DESC' > if (IS_NETMAP_DESC(pd)) > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:101:22: error: use of undeclared > identifier 'nm_nextpkt' > data = last_data = nm_nextpkt((struct nm_desc *)pd, > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:438:7: error: use of undeclared > identifier 'IS_NETMAP_DESC' > if (IS_NETMAP_DESC(pd)) > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:439:4: error: use of undeclared > identifier 'nm_close'; did you mean 'sem_close'? > nm_close((struct nm_desc *)pd); > ^~~~~~~~ > sem_close > /usr/include/semaphore.h:52:6: note: 'sem_close' declared here > int sem_close(sem_t *); > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:439:13: error: cannot initialize a > parameter of type 'sem_t *' (aka '_sem *') with an rvalue of type > 'struct nm_desc *' > nm_close((struct nm_desc *)pd); > ^~~~~~~~~~~~~~~~~~~~ > /usr/include/semaphore.h:52:23: note: passing argument to parameter here > int sem_close(sem_t *); > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:479:7: error: use of undeclared > identifier 'IS_NETMAP_DESC' > if (IS_NETMAP_DESC(pd)) > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:526:17: error: use of undeclared > identifier 'nm_open' > pd = (pcap_t *)nm_open(interface, getenv("NETMAP_RING_ID"), 0, 0); > ^ > /tmp/j/bro-2.2/src/PktSrc.cc:533:20: error: use of undeclared > identifier 'NETMAP_FD' > selectable_fd = NETMAP_FD(pd); > ^ > 8 errors generated. > *** Error code 1 > > Stop. > make[3]: stopped in /tmp/j/bro-2.2/build > *** Error code 1 > > Stop. > make[2]: stopped in /tmp/j/bro-2.2/build > *** Error code 1 > > Stop. > make[1]: stopped in /tmp/j/bro-2.2/build > *** Error code 1 > > Stop. > make: stopped in /tmp/j/bro-2.2 > > Do I need to add something?? > > Thanks. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 11:57:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E549D9E for ; Tue, 4 Mar 2014 11:57:21 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1A3AF9A3 for ; Tue, 4 Mar 2014 11:57:20 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id t61so2332883wes.31 for ; Tue, 04 Mar 2014 03:57:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=7CA22ghpPD7WenLIyugc/GHoKxE6ABwYl/KnBg2nMj8=; b=tLaWqQSgcNMwLXSljoLPoNSNAVVgJQLQL4w8hw48DSo+zY1qzEun/D9CDuvPlysYmR 6HstyigPdzOxXncZMse3Em+8hQ62ZcnHSGs2RLpPg1Ol+XGC/QAbFVBy6YI1h0VWDwaT i6ZkoXF3sdtLl0GOrJWHrGCb1RRJ3ve93OX4o/XSzADcf2cMCzqXcZ2Tk4WGSieq2NMz jlrlKtUaxkHlDCWfdqRxLATqyOatpTWVjqrKzF1CU/vGGH9H6vYEkV4HVaGiTPkxpQr2 Nck8GWgOPGTzSwhqVnAizA5MWkA8UZO+Vi6ZCvyyd5OT5epY6Pm3+gRQxcCHpZv2d3aR 4wRQ== MIME-Version: 1.0 X-Received: by 10.194.175.66 with SMTP id by2mr15432562wjc.59.1393934239550; Tue, 04 Mar 2014 03:57:19 -0800 (PST) Received: by 10.194.29.163 with HTTP; Tue, 4 Mar 2014 03:57:19 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Mar 2014 11:57:19 +0000 Message-ID: Subject: Re: Bro doesn't builds using From: "C. L. Martinez" To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:57:21 -0000 On Tue, Mar 4, 2014 at 11:50 AM, Luigi Rizzo wrote: > > > > On Tue, Mar 4, 2014 at 12:11 PM, C. L. Martinez > wrote: >> >> Hi all, >> >> Using bro netmap patch provided (), Bro 2.2 doesn't compile under >> FreeBSD 10 amd64: > > > i am not sure if the bro patch is up to date, > but likely you have the same outdated header as when you > tried to build netmap-libpcap. > > This said, for bro you are probably better off just using it > on the netmap-libpcap, there is almost no advantage in > using the native API just for reading. > > cheers > luigi > Thanks Luigi. Then the correct procedure is: a) update freebsd's netmap headers with provided by https://code.google.com/p/netmap b) build netmap-libpcap from https://code.google.com/p/netmap-libpcap c) build bro against netmap-libp-cap. Is this right?? From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 12:00:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 564141F9 for ; Tue, 4 Mar 2014 12:00:15 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E6C0E9E8 for ; Tue, 4 Mar 2014 12:00:14 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id a1so4886307wgh.22 for ; Tue, 04 Mar 2014 04:00:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=isIfRIqjSVlaKDoRa0vTN1ulPrulovApLqvHO+k/QFo=; b=wqwvWZZM9aHKndwUt0WUANOnBgDOa6gS0WPNu7Z1zTRlpp5I7vptJiiJ+ck/R9LI4n xRydQYFQmqNP7rmC1hI0UpCKVPQuGrMfXC9Nnoa6PeWD6leVF+0uIjTIaTImAOeckKI6 Di/f7NTvzIe/mUNj4NyrjcFX67wnMcJPPqRrdvk+N0fe3ncOe1P08wdp9r4pvZ2rtZ+L P3MK6P8Vx0EhMjBADCdeGKOUAMug1Fom+5z68zw/rYUTzpk5vGNRcyY/qPJI0hnKEY75 Ce7zJgsAN9bWpg+X/7f6d6HWakwJq4Sd1h9HGm5pr8CTX98dTP6gSyxY049peRqBWtEV cDXg== MIME-Version: 1.0 X-Received: by 10.194.82.35 with SMTP id f3mr25945330wjy.36.1393934413301; Tue, 04 Mar 2014 04:00:13 -0800 (PST) Received: by 10.194.29.163 with HTTP; Tue, 4 Mar 2014 04:00:13 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Mar 2014 12:00:13 +0000 Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: "C. L. Martinez" To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 12:00:15 -0000 On Tue, Mar 4, 2014 at 11:45 AM, Luigi Rizzo wrote: > > > > On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez > wrote: >> >> Hi all, >> >> When I try to compile netmap-libpcap, these errors appears: >> >> root@plzfsiem01:/tmp/j/netmap-libpcap # make >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> -c ./pcap-bpf.c >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> -c ./pcap-netmap.c >> ./pcap-netmap.c:117:9: warning: implicit declaration of function >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] >> ret = nm_dispatch((void *)d, cnt, (void >> *)pcap_netmap_filter, (void *)p); > > > almost surely you have an old version of netmap.h and netmap_user.h > in /usr/include/net > > You should update to the version in stable/10 (at the very least > manually copy these two headers) > > cheers > luigi Thanks Luigi. Only netmap.h and netmap_user.h?? Can I use default files provided by FreeBSD 10-RELEASE under sys/dev/netmap or do I need to update them?? From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 12:14:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 321B0475; Tue, 4 Mar 2014 12:14:15 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 00934B6F; Tue, 4 Mar 2014 12:14:13 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s24CE83e004033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 4 Mar 2014 13:14:09 +0100 (CET) (envelope-from egrosbein@rdtc.ru) X-Envelope-From: egrosbein@rdtc.ru X-Envelope-To: freebsd-arch@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s24CE2jF039046; Tue, 4 Mar 2014 19:14:02 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <5315C38A.1010508@rdtc.ru> Date: Tue, 04 Mar 2014 19:14:02 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP transmit and no flowid References: In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.2 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Report: * 2.5 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 12:14:15 -0000 On 04.03.2014 08:17, Adrian Chadd wrote: > I'll try this out in the next week or two once I've sorted out my > employment situation and I can reserve some time on the netperf > cluster. There is a patch written by melifaro@freebsd.org to introduce IP flow id generation and modified by me to add sysctl net.inet.ip.skip_flowid_gen to disable id generation in run time and return pre-patched behavior: http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff I've tested it in production for mpd5-based high loaded BRAS, it works just fine. It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() and to fill m->m_pkthdr.flowid. Eugene Grosbein. From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 15:09:17 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D13FFC9; Tue, 4 Mar 2014 15:09:17 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9F8A1FF6; Tue, 4 Mar 2014 15:09:17 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 29F1E21695; Tue, 4 Mar 2014 10:09:00 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute4.internal (MEProxy); Tue, 04 Mar 2014 10:09:01 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=8v/T5C0R0+ZtcwwGdCDWP6Jiuu0=; b=rpm MRBS/OfijibPVGLGWL5x6KvWYr6WeowwVCN224lG7yQvH5fekkSNOEnlDeHMoCZV X14FbzyjQG+OfXcbQ/t8Lg65x4Vubk74swZr8wgar0zY5zvibH1Sn5fsPvGxjK43 yuHF8cGriTXNPJL6/bWwVcZDRbbhGluCQM9sqYuM= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 124E4108B51; Tue, 4 Mar 2014 10:09:00 -0500 (EST) Message-Id: <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> X-Sasl-Enc: 54ObPpjiRiWTfPe+TVb7NF+qvRZF/OajidQ4sd4tl5tc 1393945740 From: Mark Felder To: Maurizio Marini , Julian Elischer MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-4527a23f In-Reply-To: <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Date: Tue, 04 Mar 2014 09:09:00 -0600 Cc: Giovanni Mattera , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 15:09:17 -0000 Did you try to enable spoofing of MAC Addresses in HyperV? http://vbry21.wordpress.com/2012/07/03/fixed-in-a-tick-solving-mulicast-issues-in-hyper-v-vms/ From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 16:01:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B70DDAF9 for ; Tue, 4 Mar 2014 16:01:37 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3E0547B7 for ; Tue, 4 Mar 2014 16:01:37 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id z11so5645177lbi.22 for ; Tue, 04 Mar 2014 08:01:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Clj1XHsVsUvRQfgVmEgMsfoTiJtKEmRhVegvwVqZSLs=; b=q+jg2nQlCxsWZMYE33zOjT1EkwU/Nh8c4SL1Dd7rYSuahDik3BpcE9xySg8vEVIeUr VezvPLy390W7OC4AgL8h8Z+DzDqmdqfaHAoqpcEZOOnDNvtGnq6xJ3F76xEBRghhWFo7 +n2Zhmp9rL1KH67qGOjaEzfvIFZFGJ9TzaTSFwI7caVpZGLf0HyqXMtzLd2eaesVW7Ng /ynQ7kc5GItriDFmBxGPd3QMvdu6mYWc3UfSYaT/BjCmN9Qheskhct9CSCOKkAwMMMoY Q5DwvzL6fDZSDQG3EWHoCjLIpfCGgdzZc9lnTaIkAPzLtktfxLwEp3cUUbgrBXljcrla m2iQ== MIME-Version: 1.0 X-Received: by 10.112.24.172 with SMTP id v12mr126433lbf.74.1393948895392; Tue, 04 Mar 2014 08:01:35 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.115.4.162 with HTTP; Tue, 4 Mar 2014 08:01:35 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Mar 2014 17:01:35 +0100 X-Google-Sender-Auth: 1ie1Av3y_ya6dAk-fdeJ8MZ9LiA Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: Luigi Rizzo To: "C. L. Martinez" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:01:37 -0000 On Tue, Mar 4, 2014 at 1:00 PM, C. L. Martinez wrote: > On Tue, Mar 4, 2014 at 11:45 AM, Luigi Rizzo wrote: > > > > > > > > On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez > > wrote: > >> > >> Hi all, > >> > >> When I try to compile netmap-libpcap, these errors appears: > >> > >> root@plzfsiem01:/tmp/j/netmap-libpcap # make > >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > >> -c ./pcap-bpf.c > >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > >> -c ./pcap-netmap.c > >> ./pcap-netmap.c:117:9: warning: implicit declaration of function > >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] > >> ret = nm_dispatch((void *)d, cnt, (void > >> *)pcap_netmap_filter, (void *)p); > > > > > > almost surely you have an old version of netmap.h and netmap_user.h > > in /usr/include/net > > > > You should update to the version in stable/10 (at the very least > > manually copy these two headers) > > > > cheers > > luigi > > Thanks Luigi. Only netmap.h and netmap_user.h?? Can I use default > files provided by FreeBSD 10-RELEASE under sys/dev/netmap or do I need > to update them?? > as i said, you need to update the files. you will also need the updated netmap kernel module so in the end it might be worthwhile upgrading to stable/10 cheers luigi From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 16:13:05 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22430D8C; Tue, 4 Mar 2014 16:13:05 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8A52488F; Tue, 4 Mar 2014 16:13:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 5AD74989E6C; Tue, 4 Mar 2014 17:11:57 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Kr5vdaqTAoye; Tue, 4 Mar 2014 17:11:55 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id DFFCB989E68; Tue, 4 Mar 2014 17:11:54 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it DFFCB989E68 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393949514; bh=zfEXUJhfA5gx8XDB87VaRig6MHnUdRHF48vxuQvckdM=; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type; b=GnnyPG37sGZukHPxa75Fvrv7eLDOGOz81Kesgt6Ru3bL/KZzOaGp8J589Ap2/gWoe fosTO/pJdHvrZTpd0RU6gPlP+dCmSMKZbshRU0qFE20+T3lmhrBa1ynYBFicloi65J d5PhGokUI9Y1WXldfrCwiPdalOGQ+gkhzkykIlgI= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id x37uxMDDlGGL; Tue, 4 Mar 2014 17:11:54 +0100 (CET) Received: from webmail.cost.it (webmail.cost.it [192.168.222.20]) by webmail.cost.it (Postfix) with ESMTP id 028B8989E58; Tue, 4 Mar 2014 17:11:53 +0100 (CET) Date: Tue, 4 Mar 2014 17:11:53 +0100 (CET) From: Giovanni Mattera To: Mark Felder Message-ID: <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> In-Reply-To: <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v MIME-Version: 1.0 X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF27 (Win)/8.0.6_GA_5922) Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: F9bKfPljswh1IpqjV+LhMNOi3TaKig== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Maurizio Marini , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:13:05 -0000 Yes i test it but not work=20 The CARP protocolol remain in state INIT.=20 ----- Messaggio originale ----- Da: "Mark Felder" =20 A: "Maurizio Marini" , "Julian Elischer" =20 Cc: "FreeBSD Net" , "Manuel Martini" , freebsd-virtualization@freebsd.org, "Giovanni Mattera" =20 Inviato: Marted=C3=AC, 4 marzo 2014 16:09:00=20 Oggetto: Re: freebsd 10.0 not work carp protocol on Hyper-v=20 Did you try to enable spoofing of MAC Addresses in HyperV?=20 http://vbry21.wordpress.com/2012/07/03/fixed-in-a-tick-solving-mulicast-iss= ues-in-hyper-v-vms/=20 From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 16:26:03 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60EA2507; Tue, 4 Mar 2014 16:26:03 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B3559AB; Tue, 4 Mar 2014 16:26:02 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6532C21597; Tue, 4 Mar 2014 11:25:44 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 04 Mar 2014 11:25:45 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id :references:to; s=smtpout; bh=5Oh9nrsiHKOpDdnZ/1EFZtDoikM=; b=Yc b5F2I/u0uYSv9EIGLbfMkjTz2qjwMpk2RR/F2Y4V/VmeDmRJVLaGVBONNbDxd0Mr mHOvN6oX64HFPya8qDDs6B/DTRl8eu5kfTDjIDsKw/QcQ5lino96MybeMfxFOYMk 0bnBTbO157FwYGLClxPfhklcjRCReMsNnpprvKIjE= X-Sasl-enc: chlEcHGw2AvwNNOsQqWdMoc7GbZUiKN52cWEkPlmrbxF 1393950344 Received: from [172.16.1.145] (unknown [68.117.126.78]) by mail.messagingengine.com (Postfix) with ESMTPA id 8B5FD68037C; Tue, 4 Mar 2014 11:25:43 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v From: Mark Felder In-Reply-To: <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> Date: Tue, 4 Mar 2014 10:25:42 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> To: Giovanni Mattera X-Mailer: Apple Mail (2.1874) Cc: Maurizio Marini , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:26:03 -0000 On Mar 4, 2014, at 10:11, Giovanni Mattera = wrote: > Yes i test it but not work > The CARP protocolol remain in state INIT. >=20 That's unfortunate. If you do a packet dump do you even see the other = node's advertisements? CARP is really at the mercy of these half-baked virtual switches; if = they refuse to pass the traffic there's not much that can be done from = the virtualized guest's perspective. I don't know the procedure for = opening a support case with Microsoft but if that's an option it would = be nice to know what their response is. I did run into this a while back on XenServer and wondered if it would = be possible to work around it by doing a GRE tunnel between the two = nodes and doing CARP over that, tying it all in with scripts that are = called by devd events. I have not had time to experiment and see if you = can do multicast over GRE tunnels though and it's probably a very bad = idea.= From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 17:43:11 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5DE499A; Tue, 4 Mar 2014 17:43:11 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 36EAD15B; Tue, 4 Mar 2014 17:43:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id A6509A03FDF; Tue, 4 Mar 2014 18:42:11 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 5tGbe4tKGbVX; Tue, 4 Mar 2014 18:42:10 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 7C133A03FD9; Tue, 4 Mar 2014 18:42:10 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 7C133A03FD9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393954930; bh=OiWj5LVWFH3Dq4ti55oio3OziMGH9Pjrk0VSJKK6SKo=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=Wu3+v+ErUuJENSWGF/Cdhp2hfEkevsiIU4qcBmDZOeVgdFqPbFZHh4+llqf88EVKB fsfhCepE4mZZp2Uc82sY8+ll6Aogn4WSU6tuvtGS59ZlivPPIZvpAEA9PM5eJ6xWiO n1x8Zl9UU0APJJaw3lZUpCdys817oPGObQuLoBkU= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id CD9_EYqv7O1X; Tue, 4 Mar 2014 18:42:10 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id 19345A03FCD; Tue, 4 Mar 2014 18:42:08 +0100 (CET) Date: Tue, 4 Mar 2014 18:43:03 +0100 From: Maurizio Marini To: Mark Felder Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> In-Reply-To: <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/mE=qoOl=qgn_g2GbQPFHogw"; protocol="application/pkcs7-signature" Cc: FreeBSD Net , freebsd-virtualization@freebsd.org, Manuel Martini , Giovanni Mattera X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 17:43:11 -0000 --Sig_/mE=qoOl=qgn_g2GbQPFHogw Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 10:25:42 -0600 Mark Felder wrote: =20 > That's unfortunate. If you do a packet dump do you even see the other nod= e's > advertisements? >=20 > CARP is really at the mercy of these half-baked virtual switc Dear sir: https://forum.pfsense.org/index.php?topic=3D44529.0 "But shouldn't the CARP state go to "master" anyway, even if it couldn't fi= nd a live partner due to network/Hyper-V problems?" this is the question, forget network/Hyper-V problems, CARP should go MASTER anyway --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --Sig_/mE=qoOl=qgn_g2GbQPFHogw Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTc0MzAz WjAjBgkqhkiG9w0BCQQxFgQUDAYo/UhToOPNzhTw4TAnmm1lIU0wKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAMaz3jZxd2VOAVqN8YVkJJsZFUQCBtJENANQP1DZYisgIvlJkF02MWgy9DJ3a 14Vf8hTABGRxrZqPA9pRGNp2q2JK9GWTuVRjIiSW3yo7aR/RfZuNfUi9C2bLxJMd K8iNtoZQ9RgIXWLqFWMUnoIr44Ga58aLjHwAB0ueZNoD33QIGS9YslyWlZEhqTwK +Y9b5l3csPk1nTJLz83yRNAW2yYPAWCZiS03I0Zn+URm5QLg30/jcdVZgCdUEAIp wMvrFvM91fjDvSphV7RGwLanCG3n6JEqPrP5/04ajsqrXfR+8UAdebwwk5lyUjP3 Jeq84lpUnVjUaz13Pu83iFlXCAAAAAAAAA== --Sig_/mE=qoOl=qgn_g2GbQPFHogw-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 18:01:05 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBDF5C5; Tue, 4 Mar 2014 18:01:05 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0209.outbound.protection.outlook.com [207.46.163.209]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 030E32B4; Tue, 4 Mar 2014 18:01:04 +0000 (UTC) Received: from BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) by BL2PR03MB593.namprd03.prod.outlook.com (10.255.109.36) with Microsoft SMTP Server (TLS) id 15.0.888.9; Tue, 4 Mar 2014 18:00:56 +0000 Received: from BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.130]) by BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.167]) with mapi id 15.00.0888.003; Tue, 4 Mar 2014 18:00:56 +0000 From: "Abhishek Gupta (LIS)" To: Maurizio Marini , Mark Felder Subject: RE: freebsd 10.0 not work carp protocol on Hyper-v Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: w1iYmb8i77+CCVNy/FC4kgfjKueHkO08X/8AgAAUAgCAAc/YAIAAEZKAgAAD3ACAABWdgIAABLBG Date: Tue, 4 Mar 2014 18:00:55 +0000 Message-ID: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org>, <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> In-Reply-To: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [71.227.189.220] x-forefront-prvs: 01401330D1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(377454003)(199002)(51704005)(189002)(24454002)(52314003)(164054003)(56776001)(69226001)(47736001)(54316002)(79102001)(95416001)(76786001)(86362001)(51856001)(94946001)(63696002)(80022001)(83322001)(19580395003)(92566001)(80976001)(46102001)(86612001)(95666003)(77096001)(94316002)(77982001)(85852003)(50986001)(47976001)(4396001)(76796001)(53806001)(49866001)(83072002)(59766001)(66066001)(76576001)(54356001)(56816005)(15975445006)(76482001)(19580405001)(93516002)(93136001)(33646001)(65816001)(47446002)(85306002)(81542001)(87936001)(2656002)(85806002)(81342001)(16799955002)(81816001)(74876001)(74502001)(31966008)(81686001)(90146001)(87266001)(74366001)(74316001)(74662001)(74706001)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR03MB593; H:BL2PR03MB210.namprd03.prod.outlook.com; CLIP:71.227.189.220; FPR:E665F15C.AC028582.68E5BCB3.40E57949.2037D; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: microsoft.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Cc: Giovanni Mattera , Manuel Martini , "freebsd-virtualization@freebsd.org" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:01:05 -0000 Hi Giovanni,=0A= =0A= Sorry to come late to the thread. Have you integrated the Hyper-V drivers y= et or are relying on emulated mode? Let's start from there and we shall see= if the problem can be resolved.=0A= Thanks,=0A= Abhishek Gupta=0A= PM, BSD Integration Services=0A= Microsoft Corporation=0A= ________________________________________=0A= From: owner-freebsd-virtualization@freebsd.org on behalf of Maurizio Marini =0A= Sent: Tuesday, March 04, 2014 9:43 AM=0A= To: Mark Felder=0A= Cc: FreeBSD Net; freebsd-virtualization@freebsd.org; Manuel Martini; Giovan= ni Mattera=0A= Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v=0A= =0A= On Tue, 4 Mar 2014 10:25:42 -0600=0A= Mark Felder wrote:=0A= =0A= =0A= > That's unfortunate. If you do a packet dump do you even see the other nod= e's=0A= > advertisements?=0A= >=0A= > CARP is really at the mercy of these half-baked virtual switc=0A= =0A= Dear sir:=0A= =0A= https://forum.pfsense.org/index.php?topic=3D44529.0=0A= =0A= "But shouldn't the CARP state go to "master" anyway, even if it couldn't fi= nd a=0A= live partner due to network/Hyper-V problems?"=0A= =0A= this is the question, forget network/Hyper-V problems, CARP should go MASTE= R=0A= anyway=0A= =0A= =0A= =0A= =0A= --=0A= Cordiali Saluti=0A= Maurizio Marini=0A= CoST - Computers Services and Technologies S.r.l.=0A= Via Longhi, 13 - 20137 Milano=0A= P. IVA 09585780159=0A= Tel +39 02 45446.207=0A= Fax +39 02 45446.333=0A= =0A= Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone=0A= indicate. La diffusione, copia o qualsiasi altra azione derivante dalla=0A= conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te=0A= ricevuto questo documento per errore siete cortesemente pregati di darne=0A= immediata comunicazione al mittente e di provvedere alla sua distruzione.= =0A= Grazie. This e-mail and any attachment are confidential and may contain=0A= privileged information intended for the addressee(s) only. Dissemination,= =0A= copying, printing or use by anybody else is unauthorized. If you are not th= e=0A= intended recipient, please delete this message and any attachment and advis= e=0A= the sender by return e-mail. Thank you=0A= =0A= =0A= =0A= From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 18:18:26 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14E30852; Tue, 4 Mar 2014 18:18:26 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id A024365B; Tue, 4 Mar 2014 18:18:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id BA819A0400C; Tue, 4 Mar 2014 19:17:25 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id xeMHZV-PQ6Mo; Tue, 4 Mar 2014 19:17:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 0A994A03FFA; Tue, 4 Mar 2014 19:17:25 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 0A994A03FFA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393957045; bh=SRiT5wO0mWWcgEPfGsNcigFpLeUqAhdIY5mmZgxkwyo=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=j2Q5htckY5JbMYsdjFWcK+zcKcv6i0DNHflbF3mJx8YVEaP5JvywnRPwgfiC9QcCl WDAbL+FUWf+Age2iRUim6ZTwe86LFPYBOA8ybFfnoyFMYhVCJv+T9AxjmCWTtSip/1 i1b5JVOzouTdtP8xd2Ztcm/I1K5BAy6WZtwUpKPk= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id K5QG3tG4JzRc; Tue, 4 Mar 2014 19:17:24 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id D8D7AA03FF5; Tue, 4 Mar 2014 19:17:23 +0100 (CET) Date: Tue, 4 Mar 2014 19:18:18 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" , Manuel Martini Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304191818.46acdd8d@tikal.homenet.telecomitalia.it> In-Reply-To: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/HXy/Or554.qUqGQLMk9DG5Y"; protocol="application/pkcs7-signature" Cc: Giovanni Mattera , "freebsd-virtualization@freebsd.org" , Mark Felder , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:18:26 -0000 --Sig_/HXy/Or554.qUqGQLMk9DG5Y Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 18:00:55 +0000 "Abhishek Gupta (LIS)" wrote: > Hi Giovanni, >=20 > Sorry to come late to the thread. Have you integrated the Hyper-V drivers= yet > or are relying on emulated mode? Let's start from there and we shall see = if Yes sir, we did Best rgds Looking forward to hear from you --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --Sig_/HXy/Or554.qUqGQLMk9DG5Y Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTgxODE4 WjAjBgkqhkiG9w0BCQQxFgQUi8B6JCYZL1z/dEsJ1TqpbVWz9R8wKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAE4pAcwZeoNgRkQZXlkydNEiRQ4h+quS4UxUp0AJkRBwBeRb7j6uBDz22m4v6 F4rx+2i3mhQHMqXTns2XnY9B7Ycp07g9oNVXt/ToYTDgmlctgRVkmSvN2usuzI/8 GQjuxgEJLafTqriQ/QYZQgIfUq/eknGOut7Ir653u3bXHvycQZsK2o18Ufzw019n F9ZSphx8hqpd/iUh0tdkHp34wDC7O6s8Hkk2Do84q0YyHQdv0EgMEu4xVOW9H1iw jvgHZXUAAFbZD1yotOH0g2dlFVfeW9iTjupmzIXTcp+fQwzp39MZuoDZ8ivE847p ok9DX/+yZFGKGffG8PIFpOzrVQAAAAAAAA== --Sig_/HXy/Or554.qUqGQLMk9DG5Y-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 18:21:01 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC9E29D8; Tue, 4 Mar 2014 18:21:01 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85BA7687; Tue, 4 Mar 2014 18:21:01 +0000 (UTC) Received: from compute2.internal (compute2.nyi.mail.srv.osa [10.202.2.42]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 0F25121524; Tue, 4 Mar 2014 13:20:59 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute2.internal (MEProxy); Tue, 04 Mar 2014 13:21:00 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id :references:to; s=smtpout; bh=PEGOvRkbJW77wgBMeQg3dA5yTmg=; b=X3 SS/HJbR5hcD5puhX71Xk8/jNXUbOA5yD2Yn5gR2Ax6+JhgtbOgKT2KQQc/VZIj9m FwcdQsYaudlTz56qfxBmD+fDeDBUTOUvwMJnRpf5bDrgI+TGnvIury2GIRVDiVuH spuSj4MECJxMWhq/VEV4EDyyGS1X1V/uQygFd8VS4= X-Sasl-enc: QnNOMgyTrrG7GHEmZazxglYqKynGO892VLXI7mlqifiJ 1393957259 Received: from [172.16.1.145] (unknown [68.117.126.78]) by mail.messagingengine.com (Postfix) with ESMTPA id A83C5680241; Tue, 4 Mar 2014 13:20:58 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v From: Mark Felder In-Reply-To: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> Date: Tue, 4 Mar 2014 12:20:55 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0968D615-5C7A-4D38-A441-676B05BE2A61@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> To: Maurizio Marini X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , freebsd-virtualization@freebsd.org, Manuel Martini , Giovanni Mattera X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:21:01 -0000 On Mar 4, 2014, at 11:43, Maurizio Marini = wrote: > "But shouldn't the CARP state go to "master" anyway, even if it = couldn't find a > live partner due to network/Hyper-V problems?" >=20 > this is the question, forget network/Hyper-V problems, CARP should go = MASTER > anyway I will have to test again in that XenServer environment soon but I = believe it also stayed INIT there unless you forced it to MASTER. I = could be wrong, though. You're right though -- it should go to MASTER if it can't talk to the = other members.= From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 18:27:12 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6090FCA2; Tue, 4 Mar 2014 18:27:12 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id ED7B977B; Tue, 4 Mar 2014 18:27:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 44EBBA04027; Tue, 4 Mar 2014 19:26:13 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id de2qwf8VFfZY; Tue, 4 Mar 2014 19:26:12 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 3CD01A04023; Tue, 4 Mar 2014 19:26:12 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 3CD01A04023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393957572; bh=gisVziYwWhl91+4Wl5nOaTukfAav3uWr9efLPmiHmEY=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=Lb/4NuHqdZS+NIWKNWB3TLN5ChAhWmEYKClrl4g7fZ7DhqkaEKHBvWozdeAVxX2a6 mnhiwRq7MFwZ2yjNH+yoe9K0y9UxX3WAhpeYQPE1yjjW1zQrjdgPiY1BretfFcQHUX YfF4kVUV3qU8C66bB7Kjuiix3LySM2SC0AvonE5c= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bVIITUlnFkOG; Tue, 4 Mar 2014 19:26:11 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id E8E70A03FFA; Tue, 4 Mar 2014 19:26:10 +0100 (CET) Date: Tue, 4 Mar 2014 19:27:05 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> In-Reply-To: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/DmSUn9FK1p31vlY+DkpCUmw"; protocol="application/pkcs7-signature" Cc: Giovanni Mattera , Manuel Martini , "freebsd-virtualization@freebsd.org" , Mark Felder , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:27:12 -0000 --Sig_/DmSUn9FK1p31vlY+DkpCUmw Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 18:00:55 +0000 "Abhishek Gupta (LIS)" wrote: > Hi Giovanni, >=20 > Sorry to come late to the thread. Have you integrated the Hyper-V drivers= yet > or are relying on emulated mode? Let's start from there and we shall see = if > the problem can be resolved. Thanks, Abhishek Gupta Hello all I can let you access to freebsd boxes with ssh and in rdp into Windows Serv= er 2012 R2 to give a look, let me know your source ip's brgds -m --Sig_/DmSUn9FK1p31vlY+DkpCUmw Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRDCCAkACAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTgyNzA1 WjAjBgkqhkiG9w0BCQQxFgQU1W9irarUh4Gx3Wzzyc/2r2BomtowKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE gf/CR/XCs0Kf6rSp8x0D10CvVVN0NJxdbfLrT0D76nDiEnBqjeRXu9CiKQjZSqaV bjr9ULm03oQLyTFwlD4P+i3SS56MC4VJ2Amu9goEfz4hipvl/fAteXZCbRoBcod3 bHwAuBmo+tf8KbxwghC06ablO2Gnrg8SBJQE0/j1H1u7tdXKmhOL8mm8ehft4CQ5 jQpisH4s5m9byJygd1vdW/1gx+8prPAg/2CyvMyjFExsUW0sNZm1j1obrnO1wM3+ UK8qxQc08sfyxWV6kYNgDOPd5NYMKxIIuVYxSZiLU4JDh7eObacqnjb3QALCXZQh inu4umHHC6nhh+0go9vAgeMAAAAAAAA= --Sig_/DmSUn9FK1p31vlY+DkpCUmw-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 19:39:35 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 410D9DC6; Tue, 4 Mar 2014 19:39:35 +0000 (UTC) Received: from mail-ee0-x230.google.com (mail-ee0-x230.google.com [IPv6:2a00:1450:4013:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 82A4CE57; Tue, 4 Mar 2014 19:39:34 +0000 (UTC) Received: by mail-ee0-f48.google.com with SMTP id e51so3873063eek.35 for ; Tue, 04 Mar 2014 11:39:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mNfRPKizHTuNrR+wt7tNgWZ7s9V/qsgFY4DYjPaNlis=; b=VRfIiUGHLLtDzbP3Ap9aCgCeThXuZePD5NTV9mLG6NQLsYDTB1ENynxdw1QwUvOc8a jRmPGxZICU6DqihGeXhSf61V2aYUuY7xY5c9sHI4t6GA7BPV1hwQQ9PCQrp28N6dnvHc 5bDZljZlHg/3trYpha1NaicymLIAK+XRTpIZR7ZYnXCVk8+c4zEA97FrHgLL7Ucaz+Mq hZh8YxOFxLbB5OrXSQmxatCDPNX24MOPV/gIAhmicLmDxhg87FYtYeeUVHyxXjeiceQL GerXHlVji8ADOMFRKC4HY2r9OCLeESpDlMXh5p4Z+ST4fvU5Uu3ZgQu8YMK3+VXpzVYu jlmA== MIME-Version: 1.0 X-Received: by 10.15.61.7 with SMTP id h7mr1189436eex.49.1393961972706; Tue, 04 Mar 2014 11:39:32 -0800 (PST) Received: by 10.14.65.4 with HTTP; Tue, 4 Mar 2014 11:39:32 -0800 (PST) In-Reply-To: <520C4F03.9040601@freebsd.org> References: <201307051458.r65EwObo066269@svn.freebsd.org> <520AED2F.4050001@freebsd.org> <520BB3F0.4020506@freebsd.org> <520C4F03.9040601@freebsd.org> Date: Tue, 4 Mar 2014 11:39:32 -0800 Message-ID: Subject: Re: TCP Initial Window 10 MFC From: hiren panchasara To: Lawrence Stewart Content-Type: text/plain; charset=UTF-8 Cc: Andre Oppermann , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 19:39:35 -0000 On Wed, Aug 14, 2013 at 8:46 PM, Lawrence Stewart wrote: > On 08/15/13 02:44, Andre Oppermann wrote: >> On 14.08.2013 04:36, Lawrence Stewart wrote: >>> Hi Andre, >>> >>> [RE team is BCCed so they're aware of this discussion] >>> >>> On 07/06/13 00:58, Andre Oppermann wrote: >>>> Author: andre >>>> Date: Fri Jul 5 14:58:24 2013 >>>> New Revision: 252789 >>>> URL: http://svnweb.freebsd.org/changeset/base/252789 >>>> >>>> Log: >>>> MFC r242266: >>>> >>>> Increase the initial CWND to 10 segments as defined in IETF TCPM >>>> draft-ietf-tcpm-initcwnd-05. It explains why the increased initial >>>> window improves the overall performance of many web services without >>>> risking congestion collapse. >>>> >>>> As long as it remains a draft it is placed under a sysctl marking it >>>> as experimental: >>>> net.inet.tcp.experimental.initcwnd10 = 1 >>>> When it becomes an official RFC soon the sysctl will be changed to >>>> the RFC number and moved to net.inet.tcp. >>>> >>>> This implementation differs from the RFC draft in that it is a bit >>>> more conservative in the case of packet loss on SYN or SYN|ACK >>>> because >>>> we haven't reduced the default RTO to 1 second yet. Also the >>>> restart >>>> window isn't yet increased as allowed. Both will be adjusted with >>>> upcoming changes. >>>> >>>> Is is enabled by default. In Linux it is enabled since kernel 3.0. >>> >>> I haven't been fully alert to FreeBSD happenings this year so apologies >>> for bringing this up so long after the MFC. >>> >>> I don't think this change should have been MFCed, at least not in its >>> current form. Enabling the switch to IW=10 on a stable branch is >>> inappropriate IMO. I also think the "net.inet.tcp.experimental" sysctl >>> branch is poorly named as per the important discussion we had back in >>> February [1]. I would really prefer we didn't get stuck having to keep >>> it around by making a stable release with it being present. >>> >>> I think this commit should be backed out of stable/9 and more >>> importantly, 9.2-RELEASE. >> >> Backing out the patch isn't really necessary, just flip the switch to >> off having it revert to the RFC5681 defaults. Those who want it anyway >> can simply enable it again. > > That doesn't address the sysctl tree naming concern or mechanism issue - > please refer back to the Feb discussion; specifically the proposal to > rename the experimental branch to "net.inet.tcp.nonstandard" and add an > "allowed" leaf which takes a list of non-standard behaviours to allow > tweaking in the stack. > > Leaving the sysctl branch named "experimental" conveys that the things > which live under the branch are being evaluated in some way for becoming > a default, which is very different to "nonstandard" which conveys that > the user is twiddling things in a way which normally shouldn't be. IW=10 > may become a FreeBSD default at some point, but the mechanism for > enabling it should be to specify the initial window as a value in > segments, and as such by allowing any non-standard value (IW=7, IW=50), > I strongly argue in favour for changing the branch name from > "experimental" to "nonstandard". > > In order to continue this discussion in the context of what we started > in Feb, I still request that this change be backed out of releng/9.2 so > that 9.2-RELEASE doesn't ship with it. We can continue discussion for > it's future in stable/9 and head after the backout so that 9.2 isn't > held up. > >> IW10 has become RFC6928 (experimental) in April 2013. > > Great for the draft authors, but irrelevant for this discussion. > >>> As an aside, I am intending to follow up to the Feb discussion with a >>> patch that implements the basic infrastructure I proposed so that we can >>> continue that discussion. >> >> Again I'm deeply concerned and opposed to giving end users direct control >> over the IW value. I've had and seen too many cases of totally bogus >> "tuning" >> by cranking up random sysctls to insane values and then complaining about >> FreeBSD being slow compared to Linux (and then ditching FreeBSD). > > Sorry, but referring to unspecified cases of stupidity resulting in loss > of unquantified numbers of users as a reason against providing a > controlled mechanism to change a default system parameter in a > potentially harmful way is not a rational argument. I do not subscribe to the idea of "Let's not make life of 98% of users better because 2% may do something stupid". I am revisiting this thread because at $work, we need to tweak initcwnd (other than 10) to see how it behaves but there is no easy way to tune it. (or am I missing something?) Can we please make initcwnd a sysctl tunable? cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 20:48:19 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4234DF7F; Tue, 4 Mar 2014 20:48:19 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0144.outbound.protection.outlook.com [207.46.163.144]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A28BB8DB; Tue, 4 Mar 2014 20:48:18 +0000 (UTC) Received: from BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) by BL2PR03MB593.namprd03.prod.outlook.com (10.255.109.36) with Microsoft SMTP Server (TLS) id 15.0.888.9; Tue, 4 Mar 2014 20:48:09 +0000 Received: from BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.130]) by BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.167]) with mapi id 15.00.0888.003; Tue, 4 Mar 2014 20:48:09 +0000 From: "Abhishek Gupta (LIS)" To: Maurizio Marini Subject: RE: freebsd 10.0 not work carp protocol on Hyper-v Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: Ac836vYWOoVgPPwqQUilqDGUim3o+A== Date: Tue, 4 Mar 2014 20:48:09 +0000 Message-ID: <93ad25323a2840639323393ccddc3af2@BL2PR03MB210.namprd03.prod.outlook.com> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> In-Reply-To: <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2001:4898:80e0:ee43::5] x-forefront-prvs: 01401330D1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(189002)(24454002)(164054003)(13464003)(53754006)(199002)(377454003)(51704005)(87936001)(2656002)(81542001)(65816001)(47446002)(85306002)(87266001)(31966008)(81686001)(90146001)(74706001)(74316001)(74366001)(81342001)(85806002)(81816001)(74876001)(74502001)(80022001)(19580395003)(80976001)(92566001)(83322001)(79102001)(69226001)(47736001)(51856001)(86362001)(94946001)(63696002)(54316002)(95416001)(76786001)(54356001)(56816005)(76576001)(33646001)(19580405001)(93136001)(93516002)(76482001)(95666003)(86612001)(77096001)(46102001)(49866001)(53806001)(76796001)(83072002)(59766001)(85852003)(47976001)(94316002)(77982001)(4396001)(50986001)(24736002)(3826001); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR03MB593; H:BL2PR03MB210.namprd03.prod.outlook.com; CLIP:2001:4898:80e0:ee43::5; FPR:FA1ED4DD.AC2394EA.9EF4FFAF.4CE2B889.2021A; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (: microsoft.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Cc: Giovanni Mattera , Manuel Martini , Mark Felder , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 20:48:19 -0000 -virtualization list. Hi Maurizio, Could you tell us how did you get the drivers? Did you obtain = them as part of FreeBSD 10 or did you get them from our ports repo? What version of FreeBSD are you running? Also if you don't mind, could you describe your problem once more to me? Th= e more detail the better. Thanks, Abhishek -----Original Message----- From: Maurizio Marini [mailto:maurizio.marini@cost.it]=20 Sent: Tuesday, March 4, 2014 10:27 AM To: Abhishek Gupta (LIS) Cc: Mark Felder; FreeBSD Net; freebsd-virtualization@freebsd.org; Manuel Ma= rtini; Giovanni Mattera Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v On Tue, 4 Mar 2014 18:00:55 +0000 "Abhishek Gupta (LIS)" wrote: > Hi Giovanni, >=20 > Sorry to come late to the thread. Have you integrated the Hyper-V drivers= yet > or are relying on emulated mode? Let's start from there and we shall see = if > the problem can be resolved. Thanks, Abhishek Gupta Hello all I can let you access to freebsd boxes with ssh and in rdp into Windows Serv= er 2012 R2 to give a look, let me know your source ip's brgds -m From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 21:10:29 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09B79B13; Tue, 4 Mar 2014 21:10:29 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C40F3AF6; Tue, 4 Mar 2014 21:10:28 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id lf10so84468pab.41 for ; Tue, 04 Mar 2014 13:10:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:reply-to:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=d23FWxEB8F5/W5fl8oasK7J8CMnjaFJstuhveUxqKS0=; b=B8pgNyXg65axwSqpUlkz7iyz3Sw9IkET8PhUlkitdSwf6CUeAT4QLso5jdGazNxiPE QLnk7gv24xaxY3KPjH/oVyvgF/ClMV1guo7xI++gso0O9mB5YweKIBdz8Xqn16mV175P Lfbbpcq7bF0AaljoJbwYB2UVPLC2bp/fdBw99ia+XSQ+gCwK7Y+eyuZIX8ZN3+6YEIYb d+Pq8rC64wFlhtuSTcmdtgk0xIR2rI1XsJ8mC+agEVpRTRw8NXJlMMaqGDeg4zUwDVgJ 83PGPPTCLY99F6VL0kE91DmEVvUTw0lvYbb8ZA1xWOPH7N3k5V/aAdgmxWQA2DLYxIBn eUKw== X-Received: by 10.66.243.131 with SMTP id wy3mr2187042pac.32.1393967428424; Tue, 04 Mar 2014 13:10:28 -0800 (PST) Received: from [192.168.1.7] (ppp59-167-128-11.static.internode.on.net. [59.167.128.11]) by mx.google.com with ESMTPSA id my6sm231177pbc.36.2014.03.04.13.10.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Mar 2014 13:10:27 -0800 (PST) Message-ID: <5316413D.7050000@FreeBSD.org> Date: Wed, 05 Mar 2014 08:10:21 +1100 From: Kubilay Kocak User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Thunderbird/27.0 MIME-Version: 1.0 To: hiren panchasara , Lawrence Stewart Subject: Re: TCP Initial Window 10 MFC References: <201307051458.r65EwObo066269@svn.freebsd.org> <520AED2F.4050001@freebsd.org> <520BB3F0.4020506@freebsd.org> <520C4F03.9040601@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Andre Oppermann , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: koobs@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 21:10:29 -0000 On 5/03/2014 6:39 AM, hiren panchasara wrote: > On Wed, Aug 14, 2013 at 8:46 PM, Lawrence Stewart wrote: >> On 08/15/13 02:44, Andre Oppermann wrote: >>> On 14.08.2013 04:36, Lawrence Stewart wrote: >>>> Hi Andre, >>>> >>>> [RE team is BCCed so they're aware of this discussion] >>>> >>>> On 07/06/13 00:58, Andre Oppermann wrote: >>>>> Author: andre >>>>> Date: Fri Jul 5 14:58:24 2013 >>>>> New Revision: 252789 >>>>> URL: http://svnweb.freebsd.org/changeset/base/252789 >>>>> >>>>> Log: >>>>> MFC r242266: >>>>> >>>>> Increase the initial CWND to 10 segments as defined in IETF TCPM >>>>> draft-ietf-tcpm-initcwnd-05. It explains why the increased initial >>>>> window improves the overall performance of many web services without >>>>> risking congestion collapse. >>>>> >>>>> As long as it remains a draft it is placed under a sysctl marking it >>>>> as experimental: >>>>> net.inet.tcp.experimental.initcwnd10 = 1 >>>>> When it becomes an official RFC soon the sysctl will be changed to >>>>> the RFC number and moved to net.inet.tcp. >>>>> >>>>> This implementation differs from the RFC draft in that it is a bit >>>>> more conservative in the case of packet loss on SYN or SYN|ACK >>>>> because >>>>> we haven't reduced the default RTO to 1 second yet. Also the >>>>> restart >>>>> window isn't yet increased as allowed. Both will be adjusted with >>>>> upcoming changes. >>>>> >>>>> Is is enabled by default. In Linux it is enabled since kernel 3.0. >>>> >>>> I haven't been fully alert to FreeBSD happenings this year so apologies >>>> for bringing this up so long after the MFC. >>>> >>>> I don't think this change should have been MFCed, at least not in its >>>> current form. Enabling the switch to IW=10 on a stable branch is >>>> inappropriate IMO. I also think the "net.inet.tcp.experimental" sysctl >>>> branch is poorly named as per the important discussion we had back in >>>> February [1]. I would really prefer we didn't get stuck having to keep >>>> it around by making a stable release with it being present. >>>> >>>> I think this commit should be backed out of stable/9 and more >>>> importantly, 9.2-RELEASE. >>> >>> Backing out the patch isn't really necessary, just flip the switch to >>> off having it revert to the RFC5681 defaults. Those who want it anyway >>> can simply enable it again. >> >> That doesn't address the sysctl tree naming concern or mechanism issue - >> please refer back to the Feb discussion; specifically the proposal to >> rename the experimental branch to "net.inet.tcp.nonstandard" and add an >> "allowed" leaf which takes a list of non-standard behaviours to allow >> tweaking in the stack. >> >> Leaving the sysctl branch named "experimental" conveys that the things >> which live under the branch are being evaluated in some way for becoming >> a default, which is very different to "nonstandard" which conveys that >> the user is twiddling things in a way which normally shouldn't be. IW=10 >> may become a FreeBSD default at some point, but the mechanism for >> enabling it should be to specify the initial window as a value in >> segments, and as such by allowing any non-standard value (IW=7, IW=50), >> I strongly argue in favour for changing the branch name from >> "experimental" to "nonstandard". >> >> In order to continue this discussion in the context of what we started >> in Feb, I still request that this change be backed out of releng/9.2 so >> that 9.2-RELEASE doesn't ship with it. We can continue discussion for >> it's future in stable/9 and head after the backout so that 9.2 isn't >> held up. >> >>> IW10 has become RFC6928 (experimental) in April 2013. >> >> Great for the draft authors, but irrelevant for this discussion. >> >>>> As an aside, I am intending to follow up to the Feb discussion with a >>>> patch that implements the basic infrastructure I proposed so that we can >>>> continue that discussion. >>> >>> Again I'm deeply concerned and opposed to giving end users direct control >>> over the IW value. I've had and seen too many cases of totally bogus >>> "tuning" >>> by cranking up random sysctls to insane values and then complaining about >>> FreeBSD being slow compared to Linux (and then ditching FreeBSD). >> >> Sorry, but referring to unspecified cases of stupidity resulting in loss >> of unquantified numbers of users as a reason against providing a >> controlled mechanism to change a default system parameter in a >> potentially harmful way is not a rational argument. > > I do not subscribe to the idea of "Let's not make life of 98% of users > better because 2% may do something stupid". > > I am revisiting this thread because at $work, we need to tweak > initcwnd (other than 10) to see how it behaves but there is no easy > way to tune it. (or am I missing something?) > > Can we please make initcwnd a sysctl tunable? > > cheers, > Hiren +1 - We'd like to at least be able to measure the difference and impact of different values @ work, with the choice to make informed configuration decisions too. From owner-freebsd-net@FreeBSD.ORG Tue Mar 4 21:11:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6987EBC2; Tue, 4 Mar 2014 21:11:34 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19C5AB77; Tue, 4 Mar 2014 21:11:34 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so128566qcy.28 for ; Tue, 04 Mar 2014 13:11:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qKQ4j+oxLmZOqFB3ih1qQC9E/xuWFYtMliMjPcAe5Xk=; b=CL670+jLHEWklOlz1EtT/mzMgZRor9IdQjT+8P3GHza7TtFHzT5LMsCNnx87deofrd zzMP0kLo6QHVKd30KR1+mXtm6xN8DiMB8kBvLV5A1T8OVBq07cCcrbNjyEw8CC0fkCus z+IrGGQbPIQ9W2F/V+CzLFeWp6IVK6l/2NRW/Gj+HvoAcSrf0eJhEQ5QAUmjhyyCdz+4 WMQbKwrAbo0rKuOeSwKeC5yiuvYvCQ1ksaUKKK1xO7bNqVoZLIXZhAjVuSI5rU4ZiKN7 l4FgzLT1PBGefAF3Pr133dIqakjiTalAV9Oa3ZFfQAYkFUWhiLgJuJDib1i12zE6LNpy 19Kw== MIME-Version: 1.0 X-Received: by 10.229.171.8 with SMTP id f8mr2433919qcz.13.1393967492543; Tue, 04 Mar 2014 13:11:32 -0800 (PST) Received: by 10.224.16.10 with HTTP; Tue, 4 Mar 2014 13:11:32 -0800 (PST) In-Reply-To: <5315C38A.1010508@rdtc.ru> References: <5315C38A.1010508@rdtc.ru> Date: Tue, 4 Mar 2014 13:11:32 -0800 Message-ID: Subject: Re: UDP transmit and no flowid From: Adrian Chadd To: Eugene Grosbein Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 21:11:34 -0000 Hi, On 4 March 2014 04:14, Eugene Grosbein wrote: > On 04.03.2014 08:17, Adrian Chadd wrote: > >> I'll try this out in the next week or two once I've sorted out my >> employment situation and I can reserve some time on the netperf >> cluster. > > There is a patch written by melifaro@freebsd.org to introduce > IP flow id generation and modified by me to add sysctl > net.inet.ip.skip_flowid_gen to disable id generation in run time > and return pre-patched behavior: > > http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff > > I've tested it in production for mpd5-based high loaded BRAS, it works just fine. > It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() > and to fill m->m_pkthdr.flowid. Hm, is this actually going to work for all paths through ip_output? Only a couple of paths go via netisr_queue(); the rest go via ifp->if_output. Is that going to loop back through the netisr outbound path? For some workloads we'll want to fill this in with the topelitz hash that matches the RX flow from the NIC, just to keep locking/processing of that flow on the same core. And to answer Slawa's email - yes, mainly because it's a subset of what's needed for doing this for TCP. In the TCP case, the hashing is already done for us on inbound connections; but there's still the whole missing bits from Robert's work to tie in the pcbgroup handling and the whole multi-queue / multi-listener thing that Linux and now DragonflyBSD does. But there's a handful of silly things that need to be first investigated and checked - like, ensuring that it works fine with fragmented IP frames and re-encapsulated things - just to ensure that they don't break the RSS model. Why'd you have to do an m_pullup() here in this path, which ideally should just be a lookup only path? Are you actually seeing the IP/TCP headers spread across multiple mbufs? They're not being snuck into mbuf headroom at all? Thanks, -a From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 02:19:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39E3DB06 for ; Wed, 5 Mar 2014 02:19:01 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EC83BD98 for ; Wed, 5 Mar 2014 02:19:00 +0000 (UTC) Received: from pool-71-174-185-176.bstnma.east.verizon.net ([71.174.185.176] helo=homobox.opal.com) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WL1QB-000KUP-Gd for freebsd-net@freebsd.org; Wed, 05 Mar 2014 02:18:59 +0000 Received: from shibato (shibato.opal.com [IPv6:2001:470:8cb8:4:221:63ff:fe5a:c9a7]) (authenticated bits=0) by homobox.opal.com (8.14.7/8.14.7) with ESMTP id s252Itiw003062 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 4 Mar 2014 21:18:55 -0500 (EST) (envelope-from fbsd@opal.com) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 71.174.185.176 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+vNYyLhy87p9b1UDGjZXXK Date: Tue, 4 Mar 2014 21:18:55 -0500 From: "J.R. Oldroyd" To: freebsd-net@freebsd.org Subject: Link DOWN/UP on RealTek 8169SC/8110SC <-> 8168/8111 Message-ID: <20140304211855.539990f9@shibato> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd9.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (homobox.opal.com [IPv6:2001:470:8cb8:4::1]); Tue, 04 Mar 2014 21:18:55 -0500 (EST) X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, RP_MATCHES_RCVD shortcircuit=no autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on homobox.opal.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 02:19:01 -0000 I'm having the "link state changed to DOWN/UP" problem on a GigE link between two FreeBSD re(4) devices. Both systems are releng/10.0. It is flapping a lot... as much as several times a minute. The link is down for 3 seconds, when it comes up dhclient takes a bit longer to re-init the IP. It is pretty much unusable most of the time! One end is a 8169SC/8110SC (RL_HWREV_8169_8110SC), the other end is a 8168/8111 (RL_HWREV_8168D). dmesg, devinfo and ifconfig for both ends follows. I've tried adding the RL_FLAG_PHYWAKE_PM flag on the 8169SC/8110SC device as was suggested before in similar discussions. No help. I don't have the technical docs (they require an NDA to download) so it was a guess anyway. Any suggestions for how to fix? They work just fine at 100baseTX. Thanks, -jr # dmesg | grep re1 re1: port 0xe800-0xe8ff mem 0xfebffc00-0xfebffcff irq 18 at device 4.0 on pci4 re1: Chip rev. 0x18000000 re1: MAC rev. 0x00000000 miibus1: on re1 re1: Ethernet address: 00:30:18:a5:5e:6c re1: link state changed to DOWN re1: link state changed to UP re1: link state changed to DOWN re1: link state changed to UP re1: link state changed to DOWN re1: link state changed to UP re1: watchdog timeout re1: link state changed to DOWN re1: link state changed to UP re1: link state changed to DOWN re1: link state changed to UP re1: link state changed to DOWN re1: link state changed to UP re1: link state changed to DOWN re1: link state changed to UP # devinfo -rv | grep re1 re1 pnpinfo vendor=0x10ec device=0x8167 subvendor=0x10ec subdevice=0x8167 class=0x020000 at slot=4 function=0 # devinfo -rv | grep rgephy1 rgephy1 pnpinfo oui=0xe04c model=0x11 rev=0x2 at phyno=1 # ifconfig re1 re1: flags=8843 metric 0 mtu 7418 options=8209b ether 00:30:18:a5:5e:6c inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::230:18ff:fea5:5e6c%re1 prefixlen 64 scopeid 0x3 inet6 2001:xxx:xxxx:1:230:18ff:fea5:5e6c prefixlen 64 inet6 2001:xxx:xxxx:1::1 prefixlen 64 nd6 options=21 media: Ethernet autoselect (1000baseT ) status: active # dmesg | grep re0 re0: port 0xe800-0xe8ff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff irq 18 at device 0.0 on pci4 re0: Using 1 MSI-X message re0: Chip rev. 0x28000000 re0: MAC rev. 0x00300000 miibus0: on re0 re0: Ethernet address: e0:cb:4e:5b:27:b6 re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP # devinfo -rv | grep re0 re0 pnpinfo vendor=0x10ec device=0x8168 subvendor=0x1043 subdevice=0x83a3 class=0x020000 a # devinfo -rv | grep rgephy0 rgephy0 pnpinfo oui=0xe04c model=0x11 rev=0x2 at phyno=1 # ifconfig re0 re0: flags=8843 metric 0 mtu 7418 options=82098 ether e0:cb:4e:5b:27:b6 inet6 fe80::e2cb:4eff:fe5b:27b6%re0 prefixlen 64 scopeid 0x1 inet6 2001:xxx:xxxx:1:e2cb:4eff:fe5b:27b6 prefixlen 64 autoconf inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=23 media: Ethernet autoselect (1000baseT ) status: active From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 03:24:23 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72199ED1; Wed, 5 Mar 2014 03:24:23 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 46E7F68A; Wed, 5 Mar 2014 03:24:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s253ONEm005222; Wed, 5 Mar 2014 03:24:23 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s253ONL1005221; Wed, 5 Mar 2014 03:24:23 GMT (envelope-from linimon) Date: Wed, 5 Mar 2014 03:24:23 GMT Message-Id: <201403050324.s253ONL1005221@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187258: [bxe] BCM57810 bxe(4) unstable/fails to initialize X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 03:24:23 -0000 Old Synopsis: BCM57810 bxe(4) unstable/fails to initialize New Synopsis: [bxe] BCM57810 bxe(4) unstable/fails to initialize Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Mar 5 03:24:00 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=187258 From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 03:38:13 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE125512; Wed, 5 Mar 2014 03:38:13 +0000 (UTC) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 45B28858; Wed, 5 Mar 2014 03:38:13 +0000 (UTC) Received: from lgwl-lstewart2.corp.netflix.com (unknown [69.53.237.72]) by lauren.room52.net (Postfix) with ESMTPSA id 106747E820; Wed, 5 Mar 2014 14:38:03 +1100 (EST) Message-ID: <53169C19.5020008@freebsd.org> Date: Tue, 04 Mar 2014 19:38:01 -0800 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: koobs@FreeBSD.org, hiren panchasara Subject: Re: TCP Initial Window 10 MFC References: <201307051458.r65EwObo066269@svn.freebsd.org> <520AED2F.4050001@freebsd.org> <520BB3F0.4020506@freebsd.org> <520C4F03.9040601@freebsd.org> <5316413D.7050000@FreeBSD.org> In-Reply-To: <5316413D.7050000@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: Andre Oppermann , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 03:38:13 -0000 On 03/04/14 13:10, Kubilay Kocak wrote: > On 5/03/2014 6:39 AM, hiren panchasara wrote: >> On Wed, Aug 14, 2013 at 8:46 PM, Lawrence Stewart wrote: >>> On 08/15/13 02:44, Andre Oppermann wrote: >>>> On 14.08.2013 04:36, Lawrence Stewart wrote: >>>>> Hi Andre, >>>>> >>>>> [RE team is BCCed so they're aware of this discussion] >>>>> >>>>> On 07/06/13 00:58, Andre Oppermann wrote: >>>>>> Author: andre >>>>>> Date: Fri Jul 5 14:58:24 2013 >>>>>> New Revision: 252789 >>>>>> URL: http://svnweb.freebsd.org/changeset/base/252789 >>>>>> >>>>>> Log: >>>>>> MFC r242266: >>>>>> >>>>>> Increase the initial CWND to 10 segments as defined in IETF TCPM >>>>>> draft-ietf-tcpm-initcwnd-05. It explains why the increased initial >>>>>> window improves the overall performance of many web services without >>>>>> risking congestion collapse. >>>>>> >>>>>> As long as it remains a draft it is placed under a sysctl marking it >>>>>> as experimental: >>>>>> net.inet.tcp.experimental.initcwnd10 = 1 >>>>>> When it becomes an official RFC soon the sysctl will be changed to >>>>>> the RFC number and moved to net.inet.tcp. >>>>>> >>>>>> This implementation differs from the RFC draft in that it is a bit >>>>>> more conservative in the case of packet loss on SYN or SYN|ACK >>>>>> because >>>>>> we haven't reduced the default RTO to 1 second yet. Also the >>>>>> restart >>>>>> window isn't yet increased as allowed. Both will be adjusted with >>>>>> upcoming changes. >>>>>> >>>>>> Is is enabled by default. In Linux it is enabled since kernel 3.0. >>>>> >>>>> I haven't been fully alert to FreeBSD happenings this year so apologies >>>>> for bringing this up so long after the MFC. >>>>> >>>>> I don't think this change should have been MFCed, at least not in its >>>>> current form. Enabling the switch to IW=10 on a stable branch is >>>>> inappropriate IMO. I also think the "net.inet.tcp.experimental" sysctl >>>>> branch is poorly named as per the important discussion we had back in >>>>> February [1]. I would really prefer we didn't get stuck having to keep >>>>> it around by making a stable release with it being present. >>>>> >>>>> I think this commit should be backed out of stable/9 and more >>>>> importantly, 9.2-RELEASE. >>>> >>>> Backing out the patch isn't really necessary, just flip the switch to >>>> off having it revert to the RFC5681 defaults. Those who want it anyway >>>> can simply enable it again. >>> >>> That doesn't address the sysctl tree naming concern or mechanism issue - >>> please refer back to the Feb discussion; specifically the proposal to >>> rename the experimental branch to "net.inet.tcp.nonstandard" and add an >>> "allowed" leaf which takes a list of non-standard behaviours to allow >>> tweaking in the stack. >>> >>> Leaving the sysctl branch named "experimental" conveys that the things >>> which live under the branch are being evaluated in some way for becoming >>> a default, which is very different to "nonstandard" which conveys that >>> the user is twiddling things in a way which normally shouldn't be. IW=10 >>> may become a FreeBSD default at some point, but the mechanism for >>> enabling it should be to specify the initial window as a value in >>> segments, and as such by allowing any non-standard value (IW=7, IW=50), >>> I strongly argue in favour for changing the branch name from >>> "experimental" to "nonstandard". >>> >>> In order to continue this discussion in the context of what we started >>> in Feb, I still request that this change be backed out of releng/9.2 so >>> that 9.2-RELEASE doesn't ship with it. We can continue discussion for >>> it's future in stable/9 and head after the backout so that 9.2 isn't >>> held up. >>> >>>> IW10 has become RFC6928 (experimental) in April 2013. >>> >>> Great for the draft authors, but irrelevant for this discussion. >>> >>>>> As an aside, I am intending to follow up to the Feb discussion with a >>>>> patch that implements the basic infrastructure I proposed so that we can >>>>> continue that discussion. >>>> >>>> Again I'm deeply concerned and opposed to giving end users direct control >>>> over the IW value. I've had and seen too many cases of totally bogus >>>> "tuning" >>>> by cranking up random sysctls to insane values and then complaining about >>>> FreeBSD being slow compared to Linux (and then ditching FreeBSD). >>> >>> Sorry, but referring to unspecified cases of stupidity resulting in loss >>> of unquantified numbers of users as a reason against providing a >>> controlled mechanism to change a default system parameter in a >>> potentially harmful way is not a rational argument. >> >> I do not subscribe to the idea of "Let's not make life of 98% of users >> better because 2% may do something stupid". >> >> I am revisiting this thread because at $work, we need to tweak >> initcwnd (other than 10) to see how it behaves but there is no easy >> way to tune it. (or am I missing something?) >> >> Can we please make initcwnd a sysctl tunable? >> >> cheers, >> Hiren > > +1 - We'd like to at least be able to measure the difference and impact > of different values @ work, with the choice to make informed > configuration decisions too. I lost the battle of wills on this topic and 10.0 shipped with IW10 enabled by default :( As for having it configurable, it is a trivial patch which perhaps, Hiren, you might be willing to take a stab at? I obviously did not manage to carve out the time last year to push forward with the agenda I proposed in this thread, but I will get back to it at some point. Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 03:52:30 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4905830; Wed, 5 Mar 2014 03:52:30 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 969699A9; Wed, 5 Mar 2014 03:52:30 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s253qU09014066; Wed, 5 Mar 2014 03:52:30 GMT (envelope-from glebius@freefall.freebsd.org) Received: (from glebius@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s253qSDr014065; Wed, 5 Mar 2014 03:52:28 GMT (envelope-from glebius) Date: Wed, 5 Mar 2014 03:52:28 GMT Message-Id: <201403050352.s253qSDr014065@freefall.freebsd.org> To: eugene@zhegan.in, glebius@FreeBSD.org, freebsd-net@FreeBSD.org From: glebius@FreeBSD.org Subject: Re: kern/165903: mbuf leak X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 03:52:30 -0000 Synopsis: mbuf leak State-Changed-From-To: patched->closed State-Changed-By: glebius State-Changed-When: Wed Mar 5 03:52:09 UTC 2014 State-Changed-Why: Doesn't affect 10.0-RELEASE. http://www.freebsd.org/cgi/query-pr.cgi?pr=165903 From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 04:15:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04F02CD4; Wed, 5 Mar 2014 04:15:15 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80403B67; Wed, 5 Mar 2014 04:15:13 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s254F3ZZ008167 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Mar 2014 05:15:03 +0100 (CET) (envelope-from egrosbein@rdtc.ru) X-Envelope-From: egrosbein@rdtc.ru X-Envelope-To: melifaro@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s254EwAj044959; Wed, 5 Mar 2014 11:14:58 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <5316A4C2.6040100@rdtc.ru> Date: Wed, 05 Mar 2014 11:14:58 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP transmit and no flowid References: <5315C38A.1010508@rdtc.ru> In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.7 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Report: * 3.0 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: FreeBSD Net , "Alexander V. Chernikov" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 04:15:15 -0000 On 05.03.2014 04:11, Adrian Chadd wrote: > Hi, > > On 4 March 2014 04:14, Eugene Grosbein wrote: >> On 04.03.2014 08:17, Adrian Chadd wrote: >> >>> I'll try this out in the next week or two once I've sorted out my >>> employment situation and I can reserve some time on the netperf >>> cluster. >> >> There is a patch written by melifaro@freebsd.org to introduce >> IP flow id generation and modified by me to add sysctl >> net.inet.ip.skip_flowid_gen to disable id generation in run time >> and return pre-patched behavior: >> >> http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff >> >> I've tested it in production for mpd5-based high loaded BRAS, it works just fine. >> It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() >> and to fill m->m_pkthdr.flowid. > > Hm, is this actually going to work for all paths through ip_output? > Only a couple of paths go via netisr_queue(); the rest go via > ifp->if_output. Is that going to loop back through the netisr outbound > path? > > For some workloads we'll want to fill this in with the topelitz hash > that matches the RX flow from the NIC, just to keep locking/processing > of that flow on the same core. > > And to answer Slawa's email - yes, mainly because it's a subset of > what's needed for doing this for TCP. In the TCP case, the hashing is > already done for us on inbound connections; but there's still the > whole missing bits from Robert's work to tie in the pcbgroup handling > and the whole multi-queue / multi-listener thing that Linux and now > DragonflyBSD does. > > But there's a handful of silly things that need to be first > investigated and checked - like, ensuring that it works fine with > fragmented IP frames and re-encapsulated things - just to ensure that > they don't break the RSS model. > > Why'd you have to do an m_pullup() here in this path, which ideally > should just be a lookup only path? Are you actually seeing the IP/TCP > headers spread across multiple mbufs? They're not being snuck into > mbuf headroom at all? I cannot answer these questions, CC'ing author of the patch, Alexander Chernikov. Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 06:22:52 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96D9FA6B; Wed, 5 Mar 2014 06:22:52 +0000 (UTC) Received: from mail-ee0-x22b.google.com (mail-ee0-x22b.google.com [IPv6:2a00:1450:4013:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B673099A; Wed, 5 Mar 2014 06:22:51 +0000 (UTC) Received: by mail-ee0-f43.google.com with SMTP id e53so201593eek.30 for ; Tue, 04 Mar 2014 22:22:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CASjv9kIrDHL110k6cX9m2WFXp35s0v08wnDDNy12tg=; b=panMCx9uioHzdU4TFM5SrR633e9xsQGQkI9LvKsXW+wYKh6j4IVb0KeQrtzTKXYwh3 cQqCH5Lj91bB7PeuVX/ErUZMZWGu3lr5yihZ2FOlA61KwuAdGlCxtAr1brX85SdeF4K9 +inIq6YH+m32ydsHRqbbHm4ZwH0K/uuzbHfaVxdVg6c0q90AdgUi6FkiQOzDzmjuFQtp GElj3M3ebd8a4q+i/LwN4yMggh27iVbA3x4mpKkPAq6SD+27SNFEwa4wt5jxBucEZyOJ R09BmeeGSDEoVa8IbH80R1+wav2tg4KlxwEo/7Dr+e8kYoGXqgvtYrnlIs8TjDABVRUQ uuYQ== MIME-Version: 1.0 X-Received: by 10.14.9.134 with SMTP id 6mr3612983eet.70.1394000570169; Tue, 04 Mar 2014 22:22:50 -0800 (PST) Received: by 10.14.65.4 with HTTP; Tue, 4 Mar 2014 22:22:50 -0800 (PST) In-Reply-To: <53169C19.5020008@freebsd.org> References: <201307051458.r65EwObo066269@svn.freebsd.org> <520AED2F.4050001@freebsd.org> <520BB3F0.4020506@freebsd.org> <520C4F03.9040601@freebsd.org> <5316413D.7050000@FreeBSD.org> <53169C19.5020008@freebsd.org> Date: Tue, 4 Mar 2014 22:22:50 -0800 Message-ID: Subject: Re: TCP Initial Window 10 MFC From: hiren panchasara To: Lawrence Stewart Content-Type: text/plain; charset=UTF-8 Cc: Andre Oppermann , koobs@freebsd.org, "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 06:22:52 -0000 On Tue, Mar 4, 2014 at 7:38 PM, Lawrence Stewart wrote: > I lost the battle of wills on this topic and 10.0 shipped with IW10 > enabled by default :( > > As for having it configurable, it is a trivial patch which perhaps, > Hiren, you might be willing to take a stab at? I obviously did not > manage to carve out the time last year to push forward with the agenda I > proposed in this thread, but I will get back to it at some point. Hi Lawrence, Let's fix it the right way if possible. Below is a rough/untested quick patch I came up with. Is this how you were planning to have "nonstandard" sysctl knob designed? Index: sys/netinet/tcp_input.c =================================================================== --- sys/netinet/tcp_input.c (revision 260833) +++ sys/netinet/tcp_input.c (working copy) @@ -164,6 +164,19 @@ &VNET_NAME(tcp_do_initcwnd10), 0, "Enable RFC 6928 (Increasing initial CWND to 10)"); +SYSCTL_NODE(_net_inet_tcp, OID_AUTO, nonstandard, CTLFLAG_RW, 0, + "Nonstandard TCP extensions"); + +VNET_DEFINE(int, tcp_nonstandard_allowed) = 0; +SYSCTL_VNET_INT(_net_inet_tcp_nonstandard, OID_AUTO, allowed, CTLFLAG_RW, + &VNET_NAME(tcp_nonstandard_allowed), 0, + "Allow nonstandard TCP extensions"); + +VNET_DEFINE(int, tcp_nonstandard_initcwnd) = 0; +SYSCTL_VNET_INT(_net_inet_tcp_nonstandard, OID_AUTO, initcwnd, CTLFLAG_RW, + &VNET_NAME(tcp_nonstandard_initcwnd), 0, + "Slow-start flight size (initial congestion window)"); + VNET_DEFINE(int, tcp_do_rfc3465) = 1; SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, rfc3465, CTLFLAG_RW, &VNET_NAME(tcp_do_rfc3465), 0, @@ -368,6 +381,8 @@ */ if (tp->snd_cwnd == 1) tp->snd_cwnd = tp->t_maxseg; /* SYN(-ACK) lost */ + else if (V_tcp_nonstandard_allowed && V_tcp_nonstandard_initcwnd) + tp->snd_cwnd = V_tcp_nonstandard_initcwnd * tp->t_maxseg; else if (V_tcp_do_initcwnd10) tp->snd_cwnd = min(10 * tp->t_maxseg, max(2 * tp->t_maxseg, 14600)); Index: sys/netinet/tcp_var.h =================================================================== --- sys/netinet/tcp_var.h (revision 260833) +++ sys/netinet/tcp_var.h (working copy) @@ -610,6 +610,7 @@ VNET_DECLARE(int, tcp_delack_enabled); VNET_DECLARE(int, tcp_do_rfc3390); VNET_DECLARE(int, tcp_do_initcwnd10); +VNET_DECLARE(int, tcp_nonstandard_allowed); +VNET_DECLARE(int, tcp_nonstandard_initcwnd); VNET_DECLARE(int, tcp_sendspace); VNET_DECLARE(int, tcp_recvspace); VNET_DECLARE(int, path_mtu_discovery); @@ -622,6 +623,7 @@ #define V_tcp_delack_enabled VNET(tcp_delack_enabled) #define V_tcp_do_rfc3390 VNET(tcp_do_rfc3390) #define V_tcp_do_initcwnd10 VNET(tcp_do_initcwnd10) +#define V_tcp_nonstandard_allowed VNET(tcp_nonstandard_allowed) +#define V_tcp_nonstandard_initcwnd VNET(tcp_nonstandard_initcwnd) #define V_tcp_sendspace VNET(tcp_sendspace) #define V_tcp_recvspace VNET(tcp_recvspace) #define V_path_mtu_discovery VNET(path_mtu_discovery) From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 13:03:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86DA0D8 for ; Wed, 5 Mar 2014 13:03:07 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 48B2A76D for ; Wed, 5 Mar 2014 13:03:07 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id i8so1019975qcq.17 for ; Wed, 05 Mar 2014 05:03:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=L4EOdcyFDpG7YOPq9kyP394OMPXTmEnz5IkxfnTh56g=; b=D/jGR/eFBDa0Ihh58F7zG0PkTgYGIPKdDp29GAlCkFWL4OkO0zisRr1AtaJLE/Y0LX DjqYJxXyCAB2RXjWBJAkv/8I4b28Q00W41VG1ER+FC0Rc+pYBsItZkucYdaoSK20vZ6Y 4Qd9H34561USAO8JnOKll6A2lsxK/24vEM3JEW5ghVAAT//hFfO8nWWfuOLjqOEnrZME qV1HGPc0eKWvg7fEIfCep+PGD0HSaB8z/yNaHcDd0YdVRHHftvmU/YqF4REZemYGkEZM 7gRtOLcWKXZ2G4w68KUIUV1n6Y2loPv1q5AZCLYos7w1mG5ufAtNFYEMjmD4ftB16FJo SWRQ== MIME-Version: 1.0 X-Received: by 10.140.20.17 with SMTP id 17mr6258172qgi.28.1394024586500; Wed, 05 Mar 2014 05:03:06 -0800 (PST) Received: by 10.224.191.71 with HTTP; Wed, 5 Mar 2014 05:03:06 -0800 (PST) Date: Wed, 5 Mar 2014 08:03:06 -0500 Message-ID: Subject: Basic recipe/expectations for VALE w/physical ports From: Nicholas Bastin To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 13:03:07 -0000 We're continuing to have problems with VALE not forwarding packets on physical ports, and I suspect at least some of the problem is local configuration - we've tried to cobble together the right workflow between fragments from various READMEs and the source code for the module and the tools, but without much luck. Is there a basic recipe available for building even a 2-port VALE bridge with netmap (igb) interfaces, with perhaps some output (log entries, etc.) to validate that the configuration is behaving as expected? We get lots of log messages that seem good, as well as many whose meaning is unclear (they only appear with "verbose" set, but seem to indicate unexpected happenings), and we see packets enter the bridge and the learning loop executed (although unfortunately there doesn't seem to be any way to inspect the current state of the FIB), but no packets actually egress from the device. Thanks, Nick From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 13:26:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2059956 for ; Wed, 5 Mar 2014 13:26:53 +0000 (UTC) Received: from mail-oa0-x22d.google.com (mail-oa0-x22d.google.com [IPv6:2607:f8b0:4003:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A91C93D for ; Wed, 5 Mar 2014 13:26:53 +0000 (UTC) Received: by mail-oa0-f45.google.com with SMTP id o6so995490oag.4 for ; Wed, 05 Mar 2014 05:26:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=bvxGWr8cyVu6z95VVlSfDaUBMq9d0cu66R20cdfSbNI=; b=qYA3v4X57Uk9+X1yNVaXbDMXL0xI1W4HOYklIvxF+8lFVSFajllfCq19FpB5fSUpXX mzqOqGjgtmnOQlQZMe5aYSnho0Xl44JxdM6HriGSG6xJhu8RyvztPXW3JLuJqrYCrl56 gMiWVaYrIY1J4Jv4o2Gw/ncx4K9wvnnUXi85dcUESfVxW67IzC0OkXCnGv0mJoE3+8WX 7YjVmGIQWl3PXeXiU/45itIusfsnb5xlyVu+Mh4FfhcqYfeT+zcaIZdidbQ0L/3ideBD d1CFTfKOrrJub3Hq8PutP4gatlBybckdODMpIp73flWkQom2Imx2NCdIub828ZpMPLZA L38g== X-Received: by 10.60.173.233 with SMTP id bn9mr398141oec.9.1394026012878; Wed, 05 Mar 2014 05:26:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.60.232.72 with HTTP; Wed, 5 Mar 2014 05:26:32 -0800 (PST) In-Reply-To: References: From: Raimundo Santos Date: Wed, 5 Mar 2014 10:26:32 -0300 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE To: Andreas Nilsson Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 13:26:53 -0000 Hello, Andreas. If table(12) is empty, how will fwd know where to send the packets that hits it? Best regards, Raimundo On 4 March 2014 02:58, Andreas Nilsson wrote: > Hello, > > I'm having a strange problem with ipfw and/or routing. I've only tested > this on 9.2-RELEASE-p3, amd64. The machine is sort of acting as router. The > ruleset is like (ipfw defaults to accept): > > $cmd="ipfw -fq " > > $cmd add 1 skipto 65534 log all from "table(1)" to any in recv "table(8)" > > ... > > $cmd add 65534 fwd tablearg all from "table(12)" to any > > Table 1 contains prefixes that should skip the normal rules and just pass > through the box. > > Table 8 contains interface names. > > Table 12 is empty (so far). > > What happens is that packets that trigger the first rule never get to their > destination. After looking at /var/log/security is see that packets trigger > the rule, "never to be seen again". There is a route (ie not default) for > the destination, but a tcpdump on the corresponding interface shows > nothing. > > > On changing the ruleset to > $cmd="ipfw -fq " > > $cmd add 1 skipto 65533 log all from "table(1)" to any in recv "table(8)" > > ... > > $cmd add 65533 fwd x.y.z.w ip from "table(1)" to any in recv "table(8)" > > $cmd add 65534 fwd tablearg all from "table(12)" to any > > packets get to where they should. > > > Why do I need the explict fwd rule? As far as I can see the ipfw man page > says nothing about skipto changing the packets, and since the 65533 rule in > the second ruleset triggers on the same thing as the skipto rule it would > seem like packets are "intact". Why does the kernel not forward those > packets? > > > Best regards > > Andreas Nilsson > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 14:28:29 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E2A2EDB; Wed, 5 Mar 2014 14:28:28 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 37DEEE7C; Wed, 5 Mar 2014 14:28:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 55BC8A04839; Wed, 5 Mar 2014 15:27:22 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 9zFJMVj0KeUx; Wed, 5 Mar 2014 15:27:21 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 626A3A04833; Wed, 5 Mar 2014 15:27:21 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 626A3A04833 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1394029641; bh=IBucDwMD8vNTEgsDEN6FKKdA2XYzVIbjXYoxjyBq8Q8=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=1k6HH+1WD1aewTKGt9iFKhslcn4uKzQbJqzMxKI+PJ7TszonGf0d5+aWSXmlIEAaO v3+Y4E2fZeyeQFMHRby6oFtMoDVzgI8YZZMqR+IktBtudW0c9iUjZV3ou0hOhRNo6f AD1bTym0XRv3ergeiAFAnBWk6b+YPeMUEq+5jxRY= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id JkG7sBYZ5Zyz; Wed, 5 Mar 2014 15:27:21 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id 3A212A04682; Wed, 5 Mar 2014 15:27:19 +0100 (CET) Date: Wed, 5 Mar 2014 15:28:16 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140305152816.2cfc4dde@tikal.homenet.telecomitalia.it> In-Reply-To: <93ad25323a2840639323393ccddc3af2@BL2PR03MB210.namprd03.prod.outlook.com> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> <93ad25323a2840639323393ccddc3af2@BL2PR03MB210.namprd03.prod.outlook.com> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/BhMI1DsFyhVo/X5W54sqYTJ"; protocol="application/pkcs7-signature" Cc: varanasi sainath , Giovanni Mattera , Manuel Martini , Mark Felder , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 14:28:29 -0000 --Sig_/BhMI1DsFyhVo/X5W54sqYTJ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 20:48:09 +0000 "Abhishek Gupta (LIS)" wrote: Hi Abhishek > -virtualization list. >=20 > Hi Maurizio, Could you tell us how did you get the drivers? Did you obtain > them as part of FreeBSD 10 or did you get them from our ports repo? your ports repo >=20 > What version of FreeBSD are you running? Freebsd 10 > Also if you don't mind, could you describe your problem once more to me? = The > more detail the better. we are packaging conf and screenshot to provide you with all the info u nee= d :) --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --Sig_/BhMI1DsFyhVo/X5W54sqYTJ Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA1MTQyODE3 WjAjBgkqhkiG9w0BCQQxFgQUFJIy1FjveFrjQXuteQ72GysO/EowKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAoMXPWzCmxRnvcKNBsI0j2ljftlWNJshuJUmozKvqrynB4hQa0RZIURaMQ3RU n+uOyrN3lcpKJ+y5rYJGZpgJ8qmuMnBrBLweROa+LqR86unEJMmppUAJZwEEgtSt PHwjdGk6dNVLJLrs4tSp0bFQ8+rizzo1VQ1dT+2og20l9m+TMFR+dufIWElem9Y6 IfGBtcDvniq5smLLb8X96LFxQ3Jwe2Jh0y52b2rpdA9ukSh+p2+qcjOvvWGfgxru w7H69dp4HshAqhWp4s57PdDo6UsEHGup47TG4pVQYhUvPTfqzScL9OV60PPf7/9H y/K23NGrfjmWeoiGl91ntOn0tgAAAAAAAA== --Sig_/BhMI1DsFyhVo/X5W54sqYTJ-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 15:09:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7176FFB for ; Wed, 5 Mar 2014 15:09:52 +0000 (UTC) Received: from mail-ob0-x22f.google.com (mail-ob0-x22f.google.com [IPv6:2607:f8b0:4003:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9EECF3B4 for ; Wed, 5 Mar 2014 15:09:52 +0000 (UTC) Received: by mail-ob0-f175.google.com with SMTP id uy5so1122343obc.34 for ; Wed, 05 Mar 2014 07:09:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XSoFxzu8YHbvpSFth7WptDaWTtkrnd588u+V3aqpLKY=; b=0WPTwM4jaqibT/6Fl3JwZKoxQDOQpoMCyxcF7GF+5I5PLWhsrUxCvBPFV+gBqYxI3w tJ13dC5QF2zmBWdIWYrJw0rRVff1UvgY7LumJX9BNTh/eVOlzT6+IBm3tFwRqJax/3yE sAcCjNPveLek+vEozbY9JauJCiP42q9I8KCW/x9KFWP1bK7bGYbHsLrFwyzbNZPDet7i Vm8bDhOmcpmrYnxNbvkp9I/KDJ9y0R+SrvrI2fsvENekSzIxXPNVz+yCaBb7ekkXJs0E BX7lqubRcS4UxhnqNVLZ8uqHhHApItp2dqYxq0D+8bq9c3zpf+RtfpRl9DhbUvX2j4b0 4eqg== MIME-Version: 1.0 X-Received: by 10.182.65.199 with SMTP id z7mr5021057obs.7.1394032192072; Wed, 05 Mar 2014 07:09:52 -0800 (PST) Received: by 10.76.144.10 with HTTP; Wed, 5 Mar 2014 07:09:51 -0800 (PST) In-Reply-To: References: Date: Wed, 5 Mar 2014 16:09:51 +0100 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: Raimundo Santos Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 15:09:52 -0000 Hello Raimundo On Wed, Mar 5, 2014 at 2:26 PM, Raimundo Santos wrote: > Hello, Andreas. > > If table(12) is empty, how will fwd know where to send the packets that > hits it? > My understanding is that the rule should not be triggered, as the "... from table(12)" will not match any packets. Other packets on the box that traverse the ruleset goes out just fine. > > Best regards, > Raimundo > > > > From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 16:11:17 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB022478; Wed, 5 Mar 2014 16:11:17 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 46166C8C; Wed, 5 Mar 2014 16:11:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 8C23AA0497E; Wed, 5 Mar 2014 17:10:13 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 3O0RgsoGJw66; Wed, 5 Mar 2014 17:10:10 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 8FB5DA0497B; Wed, 5 Mar 2014 17:10:10 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 8FB5DA0497B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1394035810; bh=NZwsGV+mCdcYrTCLcgeps4wQ/uLXbMv9bla4YcloKa8=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=2xZo/5wgHgGjHQifS7ctVWErTZqlgjxKaw7XS7IkFYJckN0PrnprR/99wj4NMjlQ5 9xzo+yINxp8H7nnPCD/mlAkGU07riy9cua2zlX3joyq1PVimPnTZPhsFHG0U3+HW68 jgu97Ou+UHldQ86OdK9x45gaPQIEmBUJlAy40HKk= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id FVvJxR7g_Pxz; Wed, 5 Mar 2014 17:10:09 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id B88D8A04963; Wed, 5 Mar 2014 17:10:06 +0100 (CET) Date: Wed, 5 Mar 2014 17:11:04 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140305171104.5b121ca2@tikal.homenet.telecomitalia.it> In-Reply-To: <93ad25323a2840639323393ccddc3af2@BL2PR03MB210.namprd03.prod.outlook.com> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> <93ad25323a2840639323393ccddc3af2@BL2PR03MB210.namprd03.prod.outlook.com> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/Wisok=1qi6wZhC57_dDouco"; protocol="application/pkcs7-signature" Cc: varanasi sainath , Giovanni Mattera , Manuel Martini , Mark Felder , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 16:11:17 -0000 --Sig_/Wisok=1qi6wZhC57_dDouco Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 20:48:09 +0000 "Abhishek Gupta (LIS)" wrote: > Also if you don't mind, could you describe your problem once more to me? = The > more detail the better. our test environment is following: https://www.dropbox.com/s/ovw0hv3cy4g2911/schema.pdf a video simulation of the issue is here: https://www.dropbox.com/s/9hh5ikyjqc49ad6/carp_on_hyperV.mov in a few words, the CARP interface never goes into MASTER/BACLUP state but keeps INIT state and it's not workable tech data and configurations 2 virtual machines FreeBSD 10-RELEASE amd64 on HyperV last version installed on Microsoft Windows 2012 R2 we have installed hv-kvp-1.0 on each vps freebsd we have configured network card as legacy (but even hn driver does not work) we have enabled mac spoofing networking of each VM does work fine on the same hyper-v 2 CentOS6 with keepalived (VRRP) wotk fine on the same hyper-v 2 PFSense (2.1 rel with fbsd 8.3) CARP doesn not work configuration files: #cat /etc/rc.conf hostname=3D"loadbal1" defaultrouter=3D"192.168.222.253" ifconfig_de0=3D"inet 192.168.222.201 netmask 255.255.255.0 media 100baseTX mediaopt full-duplex" ifconfig_de0_alias0=3D"inet 192.168.222.200/32 vhid 4= 00 pass f8398649753hl34875" ifconfig_de1=3D"inet 192.168.100.41 netmask 255.255.255.0 media 100baseTX mediaopt full-duplex" # ifconfig de0 de0: flags=3D8943 metric 0 = mtu 1500 ether 00:15:5d:2c:11:35 inet 192.168.222.201 netmask 0xffffff00 broadcast 192.168.222.255 inet6 fe80::215:5dff:fe2c:1135%de0 prefixlen 64 scopeid 0x1 inet 192.168.222.200 netmask 0xffffffff broadcast 192.168.222.200 vhid 200 nd6 options=3D29 media: Ethernet 100baseTX status: active carp: INIT vhid 200 advbase 1 advskew 0 # sysctl -a|grep carp net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 1 net.inet.carp.demotion: 240 net.inet.carp.senderr_demotion_factor: 240 net.inet.carp.ifdown_demotion_factor: 240 # uname -a FreeBSD loadbal1 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC a= md64 # pkg info ca_root_nss-3.15.4 The root certificate bundle from the Mozilla Project curl-7.35.0 Non-interactive tool to get files fr= om FTP, GOPHER, HTTP(S) servers hv-kvp-1.0 Hyper-V KVP Integration Components pkg-1.2.6 New generation package manager # ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D49 time=3D14.072 ms 64 bytes from 8.8.8.8: icmp_seq=3D1 ttl=3D49 time=3D16.979 ms ^C --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 14.072/15.525/16.979/1.453 ms # ping 192.168.222.202 PING 192.168.222.202 (192.168.222.202): 56 data bytes 64 bytes from 192.168.222.202: icmp_seq=3D0 ttl=3D64 time=3D2.045 ms 64 bytes from 192.168.222.202: icmp_seq=3D1 ttl=3D64 time=3D2.461 ms ^C --- 192.168.222.202 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 1.834/2.113/2.461/0.260 ms # ping 192.168.222.200 --- 192.168.222.200 ping statistics --- 2 packets transmitted, 0 packets received, 100.0% packet loss # cat /var/run/dmesg.boot Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 module vmbus already present! module storvsc already present! module hn already present! module atapci_dis already present! CPU: Intel(R) Xeon(R) CPU 5140 @ 2.33GHz (1300.97-MHz K8-class = CPU) Origin =3D "GenuineIntel" Id =3D 0x6f6 Family =3D 0x6 Model =3D 0xf S= tepping =3D 6 Features=3D0xf83fbff Features2=3D0x80002201 AMD Features=3D0x20100800 AMD Features2=3D0x1 real memory =3D 2147483648 (2048 MB) avail memory =3D 2052059136 (1956 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: ioapic0: Changing APIC ID to 0 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 random: initialized vmbus0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, f7f00000 (3) failed cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 7.1 on pci0 at= a0: at channel 0 on atapci0 ata1: at channel 1 on atapci0 pci0: at device 7.3 (no driver attached) vgapci0: mem 0xf8000000-0xfbffffff irq = 11 at device 8.0 on pci0 vgapci0: Boot video device de0: port 0xe880-0xe8ff mem 0xfebfe000-0xfebfefff irq 11 at device 10.0 on pci0 de0: 21140A [10-100Mb/= s] pass 2.0 de0: Ethernet address: 00:15:5d:2c:11:35 de1: port 0xec00-0xec7f mem 0xfebff000-0xfebfffff irq 11 at device 10.1 on pci0 de1: 21140A [10-100Mb/= s] pass 2.0 de1: Ethernet address: 00:15:5d:2c:11:36 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 orm0: at iomem 0xc0000-0xcbfff,0xcc000-0xcc7ff,0xcc800-0xccfff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 Timecounters tick every 20.000 msec storvsc0 on vmbus0 hyperv-utils0 on vmbus0 hyperv-utils0: Hyper-V Service attaching: Hyper-V Heartbeat Service hyperv-kvp0 on vmbus0 hyperv-kvp0: Hyper-V Service attaching: Hyper-V KVP Service hyperv-utils1 on vmbus0 hyperv-utils1: Hyper-V Service attaching: Hyper-V Shutdown Service hv_kvp_negotiate_version hyperv-utils2 on vmbus0 hyperv-utils2: Hyper-V Service attaching: Hyper-V Time Synch Service storvsc1 on vmbus0 random: unblocking device. cd0 at ata1 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-5 device cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: Fixed Direct Access SCSI-4 device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) Netvsc initializing... Timecounter "TSC" frequency 1300967194 Hz quality 800 Trying to mount root from ufs:/dev/da0p2 [rw]... Looking forward to hear from you --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 --Sig_/Wisok=1qi6wZhC57_dDouco Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA1MTYxMTA0 WjAjBgkqhkiG9w0BCQQxFgQURI7Qovb252FXyt2XooeNymAepCMwKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAhtazaidUJ8LGj1UcvZZJDnZpC/5l8yyrm2CUVdHu0beu9MQBBpKaLFkgMVb1 Ivj7EzO4RYA8Pdt/L37vPMW0odKFIBv8AdbsayVrkdqeLZRx8H9omNXAhLXhi0WG 9vX7N1FZgZmLF4ZF3CGVDZHBRvFajj8L3cyzWt5yMIiX9JziSzRcawdqK1N7hd7E Exd+FvNyF82kjFVJZA1xcvJsRLhES/YgjYgdGYlKdWRij+tlUHQTq2oXBIWCWxLg LbkN2H47fyUWu0FDte6tOJuBfNsJUESQLc4VWKwZzcXTUNIFAsEZTZQrr+rvWzAf cQA1b3/N+j0xiPFk/4a4JmdknQAAAAAAAA== --Sig_/Wisok=1qi6wZhC57_dDouco-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 18:50:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7074FD8E for ; Wed, 5 Mar 2014 18:50:26 +0000 (UTC) Received: from forward2l.mail.yandex.net (forward2l.mail.yandex.net [IPv6:2a02:6b8:0:1819::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2ACADDBA for ; Wed, 5 Mar 2014 18:50:26 +0000 (UTC) Received: from smtp17.mail.yandex.net (smtp17.mail.yandex.net [95.108.252.17]) by forward2l.mail.yandex.net (Yandex) with ESMTP id AA8891AC0D0D; Wed, 5 Mar 2014 22:50:22 +0400 (MSK) Received: from smtp17.mail.yandex.net (localhost [127.0.0.1]) by smtp17.mail.yandex.net (Yandex) with ESMTP id 591F71900564; Wed, 5 Mar 2014 22:50:22 +0400 (MSK) Received: from 84.201.166.137-vpn.dhcp.yndx.net (84.201.166.137-vpn.dhcp.yndx.net [84.201.166.137]) by smtp17.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id qvnalej3P0-oMI4SIwk; Wed, 5 Mar 2014 22:50:22 +0400 (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (Client certificate not present) X-Yandex-Uniq: bb8bb07b-6679-42f5-8a15-32c7519821ea DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1394045422; bh=CSLYa4DbVjVmO5bNiW4Fd5Wk0Fjb0vjl8d/UcfBBqnw=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=N2d7PcAe0q5BnCeIZpE7UWNCdATfGiBjXxLtO6EvUwtWwsJFed8P0pUtQKNjca6LC bwKbF9IIq6Cuz7mGDVuoJgKBBUnkLWXngcqtjAWO3DE1pp7rzAlvZpCCvPjJeam+kb yFQo6uJrU2SelIyBPovw8Jvumvqic9KJIe9xsUOI= Authentication-Results: smtp17.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <531771C8.1040207@yandex.ru> Date: Wed, 05 Mar 2014 22:49:44 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Andreas Nilsson , FreeBSD Net Subject: Re: ipfw / routing issue on 9.2-RELEASE References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 18:50:26 -0000 On 04.03.2014 09:58, Andreas Nilsson wrote: > Why do I need the explict fwd rule? As far as I can see the ipfw man page > says nothing about skipto changing the packets, and since the 65533 rule in > the second ruleset triggers on the same thing as the skipto rule it would > seem like packets are "intact". Why does the kernel not forward those > packets? What is the last rule? I suspect it is "deny all"? -- WBR, Andrey V. Elsukov From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 19:44:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F4F7A7E for ; Wed, 5 Mar 2014 19:44:52 +0000 (UTC) Received: from mail-oa0-x230.google.com (mail-oa0-x230.google.com [IPv6:2607:f8b0:4003:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CD050401 for ; Wed, 5 Mar 2014 19:44:51 +0000 (UTC) Received: by mail-oa0-f48.google.com with SMTP id m1so1533736oag.7 for ; Wed, 05 Mar 2014 11:44:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BHnpHZKlpPoMEVFTwH8uN5wE0J9P1mXnr94ExZ/1Geo=; b=lqFOlapn8Iw80nPwhpmHDLzxiCIqrjlw+N8GvA1JClr4nLtfFYxcC5bXQuNp6bhvf3 F7YoM584Dn1Rpvrb8xeR7//WS4rq4/kfN3UHVknTGZuwsKwwN9LWAqrik8+dE+v5DMFF rGm2vabXGNy3Lj/nbnTTR2YCmvNICWcE6zR7PV/SpnwAN6C1m0jL//hnbxq1b5r8Kh+T CtHRKAzDyM3GSYmM3W5TWN5Mlo4Dr7keVVAY6lXCid2+0kGaI1Y6sf/g5KPI2s4YuEE1 xeu7fKHqGxqRJ9MWahRZDVTPwIPLunoucoyn7/6I8jMxc39+G8rrIV6Lx25eiz3BusOF Ke2A== MIME-Version: 1.0 X-Received: by 10.182.43.161 with SMTP id x1mr6106732obl.5.1394048691220; Wed, 05 Mar 2014 11:44:51 -0800 (PST) Received: by 10.76.144.10 with HTTP; Wed, 5 Mar 2014 11:44:51 -0800 (PST) In-Reply-To: <531771C8.1040207@yandex.ru> References: <531771C8.1040207@yandex.ru> Date: Wed, 5 Mar 2014 20:44:51 +0100 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: "Andrey V. Elsukov" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 19:44:52 -0000 On Wed, Mar 5, 2014 at 7:49 PM, Andrey V. Elsukov wrote: > On 04.03.2014 09:58, Andreas Nilsson wrote: > > Why do I need the explict fwd rule? As far as I can see the ipfw man page > > says nothing about skipto changing the packets, and since the 65533 rule > in > > the second ruleset triggers on the same thing as the skipto rule it would > > seem like packets are "intact". Why does the kernel not forward those > > packets? > > What is the last rule? I suspect it is "deny all"? > No, last rule is allow any from any set via loader tunable net.inet.ip.fw.default_to_accept=1 For clarity : 00001 0 0 skipto 65534 log all from table(1) to any in recv table(8) 00002 6331546 601809038 skipto 13 ip from any to any in recv table(8) 00003 821402 247261846 allow ip from table(2) to any 00004 0 0 allow ip from table(3) to me dst-port 2121 00005 0 0 allow ip from table(4) to me dst-port 161 00006 0 0 allow ip from me to table(4) dst-port 162 00007 0 0 allow ip from me to table(5) dst-port 514 00008 20865 7823308 allow ip from table(6) to any dst-port 179 00009 6331564 753767359 allow { gre or ipencap } from table(6) to any 00010 3270 294972 allow icmp from table(7) to any 00011 4 617 allow icmp from any to me icmptypes 3 00012 5075 323759 deny ip from any to me 00013 1656214 123067475 divert tablearg tcp from any to any in recv table(8) 65534 0 0 fwd tablearg ip from table(12) to any 65535 11389470 1158795869 allow ip from any to any With the above ruleset a packet 1) triggering the first rule ( ie skipto no-op and the allow from any to any ) is lost. 2) triggering the second rule (ie skipto divert rule which returns it to the stack ) is forwarded. Best regards Andreas > > -- > WBR, Andrey V. Elsukov > From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 20:20:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 183DD9F2 for ; Wed, 5 Mar 2014 20:20:11 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E3D6D975 for ; Wed, 5 Mar 2014 20:20:10 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id y13so1486506pdi.5 for ; Wed, 05 Mar 2014 12:20:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=20THw0XPzBi4yQX0FYF+OzfAPT396q9p6VqC7foAUdI=; b=X8ZRccHyv9YUKmzwqZVG4ffW7CYxA8m/8CEQf2xBJIdxYxx4rQbJSnSWdB8sQxZXGz 0bonP6QrJ5MMw+f7GTNbBanXDpXsk/D3lcfmkC7Pt/berkTIOThP7h4GxTOqQry5fTOl Cdm3qXY7UeC/OQyfYwIzC/Kx+Ffoeka0d1CAt6qWAphd4e2VJs0UAyrNg8RrPHzKmqoR rSnorx5vKx807wUdYX29AYUj7pzzeA0VOMvQ2hNged4FIHRTNUVL39eOJTGH52qDP822 c7cHMFF4qAeO60YrninBqTDIdQ/r3ZXJmmEA8Sm3CHjsBZFhWI3296xPCRnPupCAI2H1 0o+A== X-Received: by 10.66.144.102 with SMTP id sl6mr9474527pab.96.1394050810548; Wed, 05 Mar 2014 12:20:10 -0800 (PST) Received: from h116-0-175-174.catv02.itscom.jp (h116-0-175-174.catv02.itscom.jp. [116.0.175.174]) by mx.google.com with ESMTPSA id ei4sm11683985pbb.42.2014.03.05.12.20.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 05 Mar 2014 12:20:09 -0800 (PST) From: Kenichi Mori Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: C library network C libc attack consle incoming Message-Id: Date: Thu, 6 Mar 2014 05:20:05 +0900 To: freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 20:20:11 -0000 Konfusion server freebsd server demorated animation server is SGI but = based Linux and AIX., tagments C library equal index.html.c Cyrillic dancers talk to mow mow=20 <><><><><><><><><><><> halfing tone equationing tag in Courc symbol spacing really in def C., c hyperlink everyone memo write copymants., index.html short version., C Library tagments double entry hyper linked bolicious network tag., link/html/three/tournaments/team/consin/depositty/ruther/bieble/ tagments in slashed be talk session., = <=C3=92=C3=B2=C3=8E=C3=A9=C3=A0=C3=80=C3=A9=C3=89=C3=A8=C3=8C=C3=AC=C3=8E=C3= =AE=C3=93=C3=B3/=C3=8C=C3=AC=C3=AE=C3=93=C3=B3=C3=B2=C3=92=C3=99=C3=B9=C3=AC= =C3=A8/=C3=80=C3=A0=C3=A9=C3=89=C3=88=C3=A8=C3=8C=C3=AC/=C3=AC=C3=8C=C3=8E= =E2=80=94/=C3=B9=C3=99=C3=92=C3=93=C3=B3=C3=8E=C3=AE=C3=AC=C3=8C=C3=88=C3=A8= /=C3=A9=C3=A0=C3=80=C3=89=C3=A9/=C3=A8=C3=88=C3=8C=C3=AC=C3=AE=C3=8E/=C3=B3= =C3=93=C3=92=C3=99=C3=B9=C3=B2/=C3=B3=C3=93=C3=8E=C3=AE=C3=8C=C3=AC/=C3=A8= =C3=88/> italiano formulate C library Latino alphred Italiano., = slashing connective pathfile fusion cocempt assessment tag match depost = tag link large slashing tag small beffing tag character line slash to slashed one = character.,Nee mon., Network tag basement hyperlink all internet protocol tag match back = Italiano tag slashes code in now., Kenichi Mori Miyake@E.v., From owner-freebsd-net@FreeBSD.ORG Wed Mar 5 20:28:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5BEABE3A for ; Wed, 5 Mar 2014 20:28:06 +0000 (UTC) Received: from forward7l.mail.yandex.net (forward7l.mail.yandex.net [84.201.143.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10B49A41 for ; Wed, 5 Mar 2014 20:28:05 +0000 (UTC) Received: from smtp16.mail.yandex.net (smtp16.mail.yandex.net [95.108.252.16]) by forward7l.mail.yandex.net (Yandex) with ESMTP id 46DF3BC0BF6; Thu, 6 Mar 2014 00:27:57 +0400 (MSK) Received: from smtp16.mail.yandex.net (localhost [127.0.0.1]) by smtp16.mail.yandex.net (Yandex) with ESMTP id E70386A07E6; Thu, 6 Mar 2014 00:27:56 +0400 (MSK) Received: from 84.201.166.137-vpn.dhcp.yndx.net (84.201.166.137-vpn.dhcp.yndx.net [84.201.166.137]) by smtp16.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id lcs27oCBdl-RuI8Zeta; Thu, 6 Mar 2014 00:27:56 +0400 (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (Client certificate not present) X-Yandex-Uniq: 91737883-4dab-4c68-90db-3df68436cb3b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1394051276; bh=Zp8PltqSlr2V7FxaDg0cwkRjrhbWMpRyRTkaDY3RAt0=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=dAczDT3A+tO991Ae7z7TEOzp8XgC8O7bw/1cStgJhIgEDOKaUfvqyG6n34N21iuUL 2LhLW/CZu+Pw1YisjrYaiL3MET+hwGO+blLdFsL9lMaB9pDgTFbVHu3VYsRhNjUoHD 9TnPL4enLHBtlfOp7lsoSl5nQfmf/SSb6zCF2cdo= Authentication-Results: smtp16.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <531788A7.2040504@yandex.ru> Date: Thu, 06 Mar 2014 00:27:19 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Andreas Nilsson Subject: Re: ipfw / routing issue on 9.2-RELEASE References: <531771C8.1040207@yandex.ru> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 20:28:06 -0000 On 05.03.2014 23:44, Andreas Nilsson wrote: > With the above ruleset a packet > 1) triggering the first rule ( ie skipto no-op and the allow from any to > any ) is lost. > 2) triggering the second rule (ie skipto divert rule which returns it to > the stack ) is forwarded. So, I don't see in the code how it can affect routing. Make sure: 1. net.inet.ip.forwarding=1 (gateway_enable="YES" in rc.conf) 2. you have route and gateway is reachable (route get/arp -n). -- WBR, Andrey V. Elsukov From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 04:00:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2467FFED for ; Thu, 6 Mar 2014 04:00:18 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E5D5C7A4 for ; Thu, 6 Mar 2014 04:00:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s2640AbI075423; Thu, 6 Mar 2014 15:00:11 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Thu, 6 Mar 2014 15:00:10 +1100 (EST) From: Ian Smith To: Andreas Nilsson Subject: Re: ipfw / routing issue on 9.2-RELEASE In-Reply-To: Message-ID: <20140306145231.Q75313@sola.nimnet.asn.au> References: <531771C8.1040207@yandex.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: FreeBSD Net , "Andrey V. Elsukov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 04:00:18 -0000 On Wed, 5 Mar 2014 20:44:51 +0100, Andreas Nilsson wrote: > On Wed, Mar 5, 2014 at 7:49 PM, Andrey V. Elsukov wrote: > > > On 04.03.2014 09:58, Andreas Nilsson wrote: > > > Why do I need the explict fwd rule? As far as I can see the ipfw man page > > > says nothing about skipto changing the packets, and since the 65533 rule > > in > > > the second ruleset triggers on the same thing as the skipto rule it would > > > seem like packets are "intact". Why does the kernel not forward those > > > packets? > > > > What is the last rule? I suspect it is "deny all"? > > > > No, last rule is allow any from any set via loader tunable > net.inet.ip.fw.default_to_accept=1 > > For clarity : > > 00001 0 0 skipto 65534 log all from table(1) to any in recv > table(8) > > 00002 6331546 601809038 skipto 13 ip from any to any in recv table(8) > > 00003 821402 247261846 allow ip from table(2) to any > > 00004 0 0 allow ip from table(3) to me dst-port 2121 > > 00005 0 0 allow ip from table(4) to me dst-port 161 > > 00006 0 0 allow ip from me to table(4) dst-port 162 > > 00007 0 0 allow ip from me to table(5) dst-port 514 > > 00008 20865 7823308 allow ip from table(6) to any dst-port 179 > > 00009 6331564 753767359 allow { gre or ipencap } from table(6) to any > > 00010 3270 294972 allow icmp from table(7) to any > > 00011 4 617 allow icmp from any to me icmptypes 3 > > 00012 5075 323759 deny ip from any to me > > 00013 1656214 123067475 divert tablearg tcp from any to any in recv > table(8) > > 65534 0 0 fwd tablearg ip from table(12) to any > > 65535 11389470 1158795869 allow ip from any to any > > With the above ruleset a packet > 1) triggering the first rule ( ie skipto no-op and the allow from any to > any ) is lost. The count on rule 1 is zero, so no packets matched it, not were 'lost'? > 2) triggering the second rule (ie skipto divert rule which returns it to > the stack ) is forwarded. > > Best regards > Andreas > > > > > -- > > WBR, Andrey V. Elsukov If at some other times rule 1 IS matched, I suggest some renumbering so you can put 'count log' rules both before and after the 'fwd tablearg' rule; then if they 'disappear' you can see exactly where. cheers, Ian From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 10:24:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E77232A; Thu, 6 Mar 2014 10:24:51 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 94837D61; Thu, 6 Mar 2014 10:24:50 +0000 (UTC) Received: from [2001:1620:2013:1:a8d5:dfb7:6d65:ba74] (port=58439 helo=[192.168.3.123]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WLVTr-000NCW-RP; Thu, 06 Mar 2014 11:24:47 +0100 From: Markus Gebert Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: 9.2 ixgbe tx queue hang (was: Network loss) Date: Thu, 6 Mar 2014 11:24:37 +0100 Message-Id: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> To: Jack Vogel , freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Cc: Johan Kooijman , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 10:24:51 -0000 (creating a new thread, because I=92m no longer sure this is related to = Johan=92s thread that I originally used to discuss this) On 27.02.2014, at 18:02, Jack Vogel wrote: > I would make SURE that you have enough mbuf resources of whatever size = pool > that you are > using (2, 4, 9K), and I would try the code in HEAD if you had not. >=20 > Jack Jack, we've upgraded some other systems on which I get more time to = debug (no impact for customers). Although those systems use the = nfsclient too, I no longer think that NFS is the source of the problem = (hence the new thread). I think it=92s the ixgbe driver and/or card. = When our problem occurs, it looks like it=92s a single tx queue that = gets stuck somehow (its buf_ring remains full). I tracked ping using dtrace to determine the source of ENOBUFS it = returns every few packets when things get weird: # dtrace -n 'fbt:::return / arg1 =3D=3D ENOBUFS && execname =3D=3D = "ping" / { stack(); }' dtrace: description 'fbt:::return ' matched 25476 probes CPU ID FUNCTION:NAME 26 7730 ixgbe_mq_start:return=20 if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`rip_output+0x229 kernel`sosend_generic+0x3f6 kernel`kern_sendit+0x1a3 kernel`sendit+0xdc kernel`sys_sendto+0x4d kernel`amd64_syscall+0x5ea kernel`0xffffffff80d35667 The only way ixgbe_mq_start could return ENOBUFS would be when = drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet = probably has no flow id, it should be assigned to a queue based on = curcpu, which made me try to pin ping to single cpus to check wether = it=92s always the same tx buf_ring that reports being full. This turned = out to be true: # cpuset -l 0 ping 10.0.4.5 PING 10.0.4.5 (10.0.4.5): 56 data bytes 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.347 ms 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.135 ms # cpuset -l 1 ping 10.0.4.5 PING 10.0.4.5 (10.0.4.5): 56 data bytes 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.184 ms 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.232 ms # cpuset -l 2 ping 10.0.4.5 PING 10.0.4.5 (10.0.4.5): 56 data bytes ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available # cpuset -l 3 ping 10.0.4.5 PING 10.0.4.5 (10.0.4.5): 56 data bytes 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.130 ms 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.126 ms [=85snip...] The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which use = the third tx buf_ring, ping reliably return ENOBUFS. If ping is run on = any other cpu using any other tx queue, it runs without any packet loss. So, when ENOBUFS is returned, this is not due to an mbuf shortage, it=92s = because the buf_ring is full. Not surprisingly, netstat -m looks pretty = normal: # netstat -m 38622/11823/50445 mbufs in use (current/cache/total) 32856/11642/44498/132096 mbuf clusters in use (current/cache/total/max) 32824/6344 mbuf+clusters out of packet secondary zone in use = (current/cache) 16/3906/3922/66048 4k (page size) jumbo clusters in use = (current/cache/total/max) 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) 75431K/41863K/117295K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines In the meantime I=92ve checked the commit log of the ixgbe driver in = HEAD and besides there are little differences between HEAD and 9.2, I = don=92t see a commit that fixes anything related to what were seeing=85 So, what=92s the conclusion here? Firmware bug that=92s only triggered = under 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue = stuff was added? Jack, how should we proceed? Markus On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert wrote: >=20 > On 27.02.2014, at 02:00, Rick Macklem wrote: >=20 >> John Baldwin wrote: >>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>> Hi all, >>>>=20 >>>> I have a weird situation here where I can't get my head around. >>>>=20 >>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in >>>> a while >>>> the Linux clients loose their NFS connection: >>>>=20 >>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>>> timed out >>>>=20 >>>> Not all boxes, just one out of the cluster. The weird part is that >>>> when I >>>> try to ping a Linux client from the FreeBSD box, I have between 10 >>>> and 30% >>>> packetloss - all day long, no specific timeframe. If I ping the >>>> Linux >>>> clients - no loss. If I ping back from the Linux clients to FBSD >>>> box - no >>>> loss. >>>>=20 >>>> The errors I get when pinging a Linux client is this one: >>>> ping: sendto: File too large >=20 > We were facing similar problems when upgrading to 9.2 and have stayed = with > 9.1 on affected systems for now. We've seen this on HP G8 blades with > 82599EB controllers: >=20 > ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 = rev=3D0x01 > hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' > class =3D network > subclass =3D ethernet >=20 > We didn't find a way to trigger the problem reliably. But when it = occurs, > it usually affects only one interface. Symptoms include: >=20 > - socket functions return the 'File too large' error mentioned by = Johan > - socket functions return 'No buffer space' available > - heavy to full packet loss on the affected interface > - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should > have timed out stick around forever (socket on the other side could = have > been closed ours ago) > - userland programs using the corresponding sockets usually got stuck = too > (can't find kernel traces right now, but always in network related = syscalls) >=20 > Network is only lightly loaded on the affected systems (usually 5-20 = mbit, > capped at 200 mbit, per server), and netstat never showed any = indication of > ressource shortage (like mbufs). >=20 > What made the problem go away temporariliy was to ifconfig down/up the > affected interface. >=20 > We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really > stable. Also, we tested a few revisions between 9.1 and 9.2 to find = out > when the problem started. Unfortunately, the ixgbe driver turned out = to be > mostly unstable on our systems between these releases, worse than on = 9.2. > The instability was introduced shortly after to 9.1 and fixed only = very > shortly before 9.2 release. So no luck there. We ended up using 9.1 = with > backports of 9.2 features we really need. >=20 > What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe = driver > or a combination of both that causes these problems. Unfortunately we = ran > out of time (and ideas). >=20 >=20 >>> EFBIG is sometimes used for drivers when a packet takes too many >>> scatter/gather entries. Since you mentioned NFS, one thing you can >>> try is to >>> disable TSO on the intertface you are using for NFS to see if that >>> "fixes" it. >>>=20 >> And please email if you try it and let us know if it helps. >>=20 >> I've think I've figured out how 64K NFS read replies can do this, >> but I'll admit "ping" is a mystery? (Doesn't it just send a single >> packet that would be in a single mbuf?) >>=20 >> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >> don't know if it can happen for an mbuf chain with < 32 entries? >=20 > We don't use the nfs server on our systems, but they're = (new)nfsclients. > So I don't think our problem is nfs related, unless the default = rsize/wsize > for client mounts is not 8K, which I thought it was. Can you confirm = this, > Rick? >=20 > IIRC, disabling TSO did not make any difference in our case. >=20 >=20 > Markus >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 15:00:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1524FCF for ; Thu, 6 Mar 2014 15:00:54 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 70334EA7 for ; Thu, 6 Mar 2014 15:00:54 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id b13so3303571wgh.20 for ; Thu, 06 Mar 2014 07:00:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=l7OgEkWWVRKijYvpYtc6632tUtRJPCGRLZiIFJYN9JA=; b=ztMneCFRx/yeCDpBAAijbcMZ++zlIrISE/UxQuQMSpKiGthcyy3be9hmhqZhbnrZ9V 3xnEZ6G1jtmR7KkD6ufZYPizZFfLMisnB7o7S9dI/CzPcbq933g7RGSyhv1Ko7mpRPlU SBLSycU9OAYPCv54f+8U3+vTK8l3SVbviOm/B/mhSlGbaUeh2Md6IvLkzmc/tOoac9xV qQ4qSyDJg/5ufJF/eq8Fz5yBrjildelf0Zo6Xgr9P1NJcvhx6XY/MaMd37wLDWjfSWJj A85q25eSemqTEx8jgJ2lvhStpcYkrWNMieY6UDByqoBflYzeb16V1LGgkThyHh6J7u/i I6ow== MIME-Version: 1.0 X-Received: by 10.195.13.234 with SMTP id fb10mr10915841wjd.50.1394118042414; Thu, 06 Mar 2014 07:00:42 -0800 (PST) Received: by 10.194.29.163 with HTTP; Thu, 6 Mar 2014 07:00:42 -0800 (PST) In-Reply-To: References: Date: Thu, 6 Mar 2014 15:00:42 +0000 Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: "C. L. Martinez" To: Luigi Rizzo , freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:00:54 -0000 Luigi, I have updated my system to FreeBSD 10 STABLE, and netmap-libpcap returns same errors: root@plzfnsm01:/tmp/n/netmap-libpcap # make cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-bpf.c cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-netmap.c ./pcap-netmap.c:117:9: warning: implicit declaration of function 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] ret = nm_dispatch((void *)d, cnt, (void *)pcap_netmap_filter, (void *)p); ^ ./pcap-netmap.c:131:9: warning: implicit declaration of function 'nm_inject' is invalid in C99 [-Wimplicit-function-declaration] return nm_inject(d, buf, size); ^ ./pcap-netmap.c:139:15: error: variable has incomplete type 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:139:9: note: forward declaration of 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:140:19: error: incomplete definition of type 'struct nm_desc' int error, fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:152:7: error: use of undeclared identifier 'SIOCSIFFLAGS' case SIOCSIFFLAGS: ^ ./pcap-netmap.c:157:10: warning: implicit declaration of function 'ioctl' is invalid in C99 [-Wimplicit-function-declaration] error = ioctl(fd, what, &ifr); ^ ./pcap-netmap.c:159:4: error: incomplete definition of type 'struct nm_desc' d->req.nr_name, what, error); ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:163:7: error: use of undeclared identifier 'SIOCGIFFLAGS' case SIOCGIFFLAGS: ^ ./pcap-netmap.c:177:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:178:18: error: use of undeclared identifier 'IFF_PPROMISC' if (if_flags & IFF_PPROMISC) { ^ ./pcap-netmap.c:179:17: error: use of undeclared identifier 'IFF_PPROMISC' if_flags &= ~IFF_PPROMISC; ^ ./pcap-netmap.c:180:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ ./pcap-netmap.c:183:2: warning: implicit declaration of function 'nm_close' is invalid in C99 [-Wimplicit-function-declaration] nm_close(d); ^ ./pcap-netmap.c:195:22: warning: implicit declaration of function 'nm_open' is invalid in C99 [-Wimplicit-function-declaration] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ./pcap-netmap.c:195:18: warning: incompatible integer to pointer conversion initializing 'struct nm_desc *' with an expression of type 'int' [-Wint-conversion] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./pcap-netmap.c:210:36: error: incomplete definition of type 'struct nm_desc' __FUNCTION__, p->opt.source, d, d->fd, ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:213:11: error: incomplete definition of type 'struct nm_desc' p->fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:27: error: incomplete definition of type 'struct nm_desc' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:45: error: use of undeclared identifier 'NETMAP_SW_RING' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ^ ./pcap-netmap.c:215:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:216:20: error: use of undeclared identifier 'IFF_PPROMISC' if (!(if_flags & IFF_PPROMISC)) { ^ ./pcap-netmap.c:218:16: error: use of undeclared identifier 'IFF_PPROMISC' if_flags |= IFF_PPROMISC; ^ ./pcap-netmap.c:219:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ 6 warnings and 17 errors generated. *** Error code 1 Stop. make: stopped in /tmp/n/netmap-libpcap cat /usr/src/sys/net/netmap.h: /* * Copyright (C) 2011-2014 Matteo Landi, Luigi Rizzo. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``S IS''AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * $FreeBSD: stable/10/sys/net/netmap.h 262151 2014-02-18 05:01:04Z luigi $ and cat /usr/src/sys/net/netmap_user.h /* * Copyright (C) 2011-2014 Universita` di Pisa. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * $FreeBSD: stable/10/sys/net/netmap_user.h 262151 2014-02-18 05:01:04Z luigi $ On Tue, Mar 4, 2014 at 4:01 PM, Luigi Rizzo wrote: > > > > On Tue, Mar 4, 2014 at 1:00 PM, C. L. Martinez wrote: >> >> On Tue, Mar 4, 2014 at 11:45 AM, Luigi Rizzo wrote: >> > >> > >> > >> > On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez >> > wrote: >> >> >> >> Hi all, >> >> >> >> When I try to compile netmap-libpcap, these errors appears: >> >> >> >> root@plzfsiem01:/tmp/j/netmap-libpcap # make >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> >> -c ./pcap-bpf.c >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> >> -c ./pcap-netmap.c >> >> ./pcap-netmap.c:117:9: warning: implicit declaration of function >> >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] >> >> ret = nm_dispatch((void *)d, cnt, (void >> >> *)pcap_netmap_filter, (void *)p); >> > >> > >> > almost surely you have an old version of netmap.h and netmap_user.h >> > in /usr/include/net >> > >> > You should update to the version in stable/10 (at the very least >> > manually copy these two headers) >> > >> > cheers >> > luigi >> >> Thanks Luigi. Only netmap.h and netmap_user.h?? Can I use default >> files provided by FreeBSD 10-RELEASE under sys/dev/netmap or do I need >> to update them?? > > > as i said, you need to update the files. > you will also need the updated netmap kernel module so in the end > it might be worthwhile upgrading to stable/10 > > cheers > luigi > From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 15:19:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C5928D6 for ; Thu, 6 Mar 2014 15:19:49 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 81DE0159 for ; Thu, 6 Mar 2014 15:19:48 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so1840214lab.41 for ; Thu, 06 Mar 2014 07:19:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=fiBdnQGTKNqs+u4a991oM7kL33O5ins7HVHXnTH4UC0=; b=G0IUZWvAlnQvsE2YWo0l3hpw+SUMmh8cJrezeR0TOGtAxV+arVppd37dMGUetKcD4L 1CS7LCRgNDvseecUDNxSPDO14XjGZ9hTCfzF4BhynVQbugihJszaOLET+/5klqULsxYe bdqu+qihzSBrhYd+sDluUC2g3B/Dv8GPRALUQMDVXLfY1Nr3RY8qmGOFMjS8TuJl3i4Q m96ewSpO6+PHOrJe575TuwOWykXhvHOeZUVB4RVgixfCM4HkvinqmCPGXPBY2tFLVN9K t2yszVG7sgZaAItQOU94gJ0t/ifhL2strbqyrL4LGocJHcEqNIPmcclmoGNS9gsAh0Ji vYWg== MIME-Version: 1.0 X-Received: by 10.112.138.233 with SMTP id qt9mr8095321lbb.34.1394119186652; Thu, 06 Mar 2014 07:19:46 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.115.4.162 with HTTP; Thu, 6 Mar 2014 07:19:46 -0800 (PST) In-Reply-To: References: Date: Thu, 6 Mar 2014 16:19:46 +0100 X-Google-Sender-Auth: kUtK5IL2EwM_JvY39rHceUpoUx8 Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: Luigi Rizzo To: "C. L. Martinez" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:19:49 -0000 On Thu, Mar 6, 2014 at 4:00 PM, C. L. Martinez wrote: > Luigi, > > I have updated my system to FreeBSD 10 STABLE, and netmap-libpcap > returns same errors: > this is because you haven't installed the headers in /usr/include/net cheers luigi > > root@plzfnsm01:/tmp/n/netmap-libpcap # make > cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > -c ./pcap-bpf.c > cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > -c ./pcap-netmap.c > ./pcap-netmap.c:117:9: warning: implicit declaration of function > 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] > ret = nm_dispatch((void *)d, cnt, (void > *)pcap_netmap_filter, (void *)p); > ^ > ./pcap-netmap.c:131:9: warning: implicit declaration of function > 'nm_inject' is invalid in C99 [-Wimplicit-function-declaration] > return nm_inject(d, buf, size); > ^ > ./pcap-netmap.c:139:15: error: variable has incomplete type 'struct ifreq' > struct ifreq ifr; > ^ > ./pcap-netmap.c:139:9: note: forward declaration of 'struct ifreq' > struct ifreq ifr; > ^ > ./pcap-netmap.c:140:19: error: incomplete definition of type 'struct > nm_desc' > int error, fd = d->fd; > ~^ > ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' > struct nm_desc *d; /* pointer returned by nm_open() */ > ^ > ./pcap-netmap.c:152:7: error: use of undeclared identifier 'SIOCSIFFLAGS' > case SIOCSIFFLAGS: > ^ > ./pcap-netmap.c:157:10: warning: implicit declaration of function > 'ioctl' is invalid in C99 [-Wimplicit-function-declaration] > error = ioctl(fd, what, &ifr); > ^ > ./pcap-netmap.c:159:4: error: incomplete definition of type 'struct > nm_desc' > d->req.nr_name, what, error); > ~^ > ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' > struct nm_desc *d; /* pointer returned by nm_open() */ > ^ > ./pcap-netmap.c:163:7: error: use of undeclared identifier 'SIOCGIFFLAGS' > case SIOCGIFFLAGS: > ^ > ./pcap-netmap.c:177:24: error: use of undeclared identifier 'SIOCGIFFLAGS' > pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch > flags */ > ^ > ./pcap-netmap.c:178:18: error: use of undeclared identifier 'IFF_PPROMISC' > if (if_flags & IFF_PPROMISC) { > ^ > ./pcap-netmap.c:179:17: error: use of undeclared identifier 'IFF_PPROMISC' > if_flags &= ~IFF_PPROMISC; > ^ > ./pcap-netmap.c:180:25: error: use of undeclared identifier 'SIOCSIFFLAGS' > pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); > ^ > ./pcap-netmap.c:183:2: warning: implicit declaration of function > 'nm_close' is invalid in C99 [-Wimplicit-function-declaration] > nm_close(d); > ^ > ./pcap-netmap.c:195:22: warning: implicit declaration of function > 'nm_open' is invalid in C99 [-Wimplicit-function-declaration] > struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); > ^ > ./pcap-netmap.c:195:18: warning: incompatible integer to pointer > conversion initializing 'struct nm_desc *' with an expression of type > 'int' [-Wint-conversion] > struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); > ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > ./pcap-netmap.c:210:36: error: incomplete definition of type 'struct > nm_desc' > __FUNCTION__, p->opt.source, d, d->fd, > ~^ > ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' > struct nm_desc *d; /* pointer returned by nm_open() */ > ^ > ./pcap-netmap.c:213:11: error: incomplete definition of type 'struct > nm_desc' > p->fd = d->fd; > ~^ > ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' > struct nm_desc *d; /* pointer returned by nm_open() */ > ^ > ./pcap-netmap.c:214:27: error: incomplete definition of type 'struct > nm_desc' > if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { > ~^ > ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' > struct nm_desc *d; /* pointer returned by nm_open() */ > ^ > ./pcap-netmap.c:214:45: error: use of undeclared identifier > 'NETMAP_SW_RING' > if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { > ^ > ./pcap-netmap.c:215:24: error: use of undeclared identifier 'SIOCGIFFLAGS' > pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch > flags */ > ^ > ./pcap-netmap.c:216:20: error: use of undeclared identifier 'IFF_PPROMISC' > if (!(if_flags & IFF_PPROMISC)) { > ^ > ./pcap-netmap.c:218:16: error: use of undeclared identifier 'IFF_PPROMISC' > if_flags |= IFF_PPROMISC; > ^ > ./pcap-netmap.c:219:25: error: use of undeclared identifier 'SIOCSIFFLAGS' > pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); > ^ > 6 warnings and 17 errors generated. > *** Error code 1 > > Stop. > make: stopped in /tmp/n/netmap-libpcap > > cat /usr/src/sys/net/netmap.h: > > /* > * Copyright (C) 2011-2014 Matteo Landi, Luigi Rizzo. All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > * modification, are permitted provided that the following conditions > * are met: > * > * 1. Redistributions of source code must retain the above copyright > * notice, this list of conditions and the following disclaimer. > * 2. Redistributions in binary form must reproduce the above copyright > * notice, this list of conditions and the following disclaimer in the > * documentation and/or other materials provided with the > distribution. > * > * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``S IS''AND > * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > PURPOSE > * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > CONSEQUENTIAL > * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > STRICT > * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY > WAY > * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > * SUCH DAMAGE. > */ > > /* > * $FreeBSD: stable/10/sys/net/netmap.h 262151 2014-02-18 05:01:04Z luigi $ > > and cat /usr/src/sys/net/netmap_user.h > > /* > * Copyright (C) 2011-2014 Universita` di Pisa. All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > * modification, are permitted provided that the following conditions > * are met: > * > * 1. Redistributions of source code must retain the above copyright > * notice, this list of conditions and the following disclaimer. > * 2. Redistributions in binary form must reproduce the above copyright > * notice, this list of conditions and the following disclaimer in the > * documentation and/or other materials provided with the > distribution. > * > * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > PURPOSE > * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > CONSEQUENTIAL > * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > STRICT > * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY > WAY > * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > * SUCH DAMAGE. > */ > > /* > * $FreeBSD: stable/10/sys/net/netmap_user.h 262151 2014-02-18 05:01:04Z > luigi $ > > On Tue, Mar 4, 2014 at 4:01 PM, Luigi Rizzo wrote: > > > > > > > > On Tue, Mar 4, 2014 at 1:00 PM, C. L. Martinez > wrote: > >> > >> On Tue, Mar 4, 2014 at 11:45 AM, Luigi Rizzo > wrote: > >> > > >> > > >> > > >> > On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez > > >> > wrote: > >> >> > >> >> Hi all, > >> >> > >> >> When I try to compile netmap-libpcap, these errors appears: > >> >> > >> >> root@plzfsiem01:/tmp/j/netmap-libpcap # make > >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > >> >> -c ./pcap-bpf.c > >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 > >> >> -c ./pcap-netmap.c > >> >> ./pcap-netmap.c:117:9: warning: implicit declaration of function > >> >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] > >> >> ret = nm_dispatch((void *)d, cnt, (void > >> >> *)pcap_netmap_filter, (void *)p); > >> > > >> > > >> > almost surely you have an old version of netmap.h and netmap_user.h > >> > in /usr/include/net > >> > > >> > You should update to the version in stable/10 (at the very least > >> > manually copy these two headers) > >> > > >> > cheers > >> > luigi > >> > >> Thanks Luigi. Only netmap.h and netmap_user.h?? Can I use default > >> files provided by FreeBSD 10-RELEASE under sys/dev/netmap or do I need > >> to update them?? > > > > > > as i said, you need to update the files. > > you will also need the updated netmap kernel module so in the end > > it might be worthwhile upgrading to stable/10 > > > > cheers > > luigi > > > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 15:36:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB47ECA for ; Thu, 6 Mar 2014 15:36:26 +0000 (UTC) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 39728332 for ; Thu, 6 Mar 2014 15:36:26 +0000 (UTC) Received: by mail-we0-f176.google.com with SMTP id x48so3333299wes.35 for ; Thu, 06 Mar 2014 07:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=2GuBxQot9YMDtBS4dxQmfH0Iij2ZQVVjyQDdRuLGEPA=; b=W/u/yubZOTTUnh9g2M3z3CzAk6bRdMhzTrG93GKgfCXnDMZ7wVprMFqaKxOmJQAa6X 92RJ7BuqIWSw7rRO3EUEEx5QtxeBUmfo/jWEE7NhZ/jnl5vAH5YRWjdxTtsNgx6P9v7+ 8QTLcYCFWseEZpha+vLC6ip+VKkBG1xZwDRssbmmoxW/tOvcvMFpxVqwX1YszsbtXEWh C3kCj9R0D3dDjENmdZJt93x+GTi+bHorBrHLKf1Fk/0/tQl96sE83Km3v7MUh7xn295Q 9/ZvAo7AQ+VaSzXgMQO+7DbXLarIhXsgm02Z10f1vp5Nb+ZVkgJooqzgjBr15Di2A4wJ k2gg== MIME-Version: 1.0 X-Received: by 10.194.175.66 with SMTP id by2mr10922604wjc.59.1394120183997; Thu, 06 Mar 2014 07:36:23 -0800 (PST) Received: by 10.194.29.163 with HTTP; Thu, 6 Mar 2014 07:36:23 -0800 (PST) In-Reply-To: References: Date: Thu, 6 Mar 2014 15:36:23 +0000 Message-ID: Subject: Re: netmap-libpcap doesn't installs under FreeBSD10 From: "C. L. Martinez" To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:36:26 -0000 Thanks Luigi. It is all ok now. On Thu, Mar 6, 2014 at 3:19 PM, Luigi Rizzo wrote: > > > > On Thu, Mar 6, 2014 at 4:00 PM, C. L. Martinez wrote: >> >> Luigi, >> >> I have updated my system to FreeBSD 10 STABLE, and netmap-libpcap >> returns same errors: > > > this is because you haven't installed the headers > in /usr/include/net > > cheers > luigi >> >> >> root@plzfnsm01:/tmp/n/netmap-libpcap # make >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> -c ./pcap-bpf.c >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> -c ./pcap-netmap.c >> ./pcap-netmap.c:117:9: warning: implicit declaration of function >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] >> ret = nm_dispatch((void *)d, cnt, (void >> *)pcap_netmap_filter, (void *)p); >> ^ >> ./pcap-netmap.c:131:9: warning: implicit declaration of function >> 'nm_inject' is invalid in C99 [-Wimplicit-function-declaration] >> return nm_inject(d, buf, size); >> ^ >> ./pcap-netmap.c:139:15: error: variable has incomplete type 'struct ifreq' >> struct ifreq ifr; >> ^ >> ./pcap-netmap.c:139:9: note: forward declaration of 'struct ifreq' >> struct ifreq ifr; >> ^ >> ./pcap-netmap.c:140:19: error: incomplete definition of type 'struct >> nm_desc' >> int error, fd = d->fd; >> ~^ >> ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' >> struct nm_desc *d; /* pointer returned by nm_open() */ >> ^ >> ./pcap-netmap.c:152:7: error: use of undeclared identifier 'SIOCSIFFLAGS' >> case SIOCSIFFLAGS: >> ^ >> ./pcap-netmap.c:157:10: warning: implicit declaration of function >> 'ioctl' is invalid in C99 [-Wimplicit-function-declaration] >> error = ioctl(fd, what, &ifr); >> ^ >> ./pcap-netmap.c:159:4: error: incomplete definition of type 'struct >> nm_desc' >> d->req.nr_name, what, error); >> ~^ >> ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' >> struct nm_desc *d; /* pointer returned by nm_open() */ >> ^ >> ./pcap-netmap.c:163:7: error: use of undeclared identifier 'SIOCGIFFLAGS' >> case SIOCGIFFLAGS: >> ^ >> ./pcap-netmap.c:177:24: error: use of undeclared identifier 'SIOCGIFFLAGS' >> pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch >> flags */ >> ^ >> ./pcap-netmap.c:178:18: error: use of undeclared identifier 'IFF_PPROMISC' >> if (if_flags & IFF_PPROMISC) { >> ^ >> ./pcap-netmap.c:179:17: error: use of undeclared identifier 'IFF_PPROMISC' >> if_flags &= ~IFF_PPROMISC; >> ^ >> ./pcap-netmap.c:180:25: error: use of undeclared identifier 'SIOCSIFFLAGS' >> pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); >> ^ >> ./pcap-netmap.c:183:2: warning: implicit declaration of function >> 'nm_close' is invalid in C99 [-Wimplicit-function-declaration] >> nm_close(d); >> ^ >> ./pcap-netmap.c:195:22: warning: implicit declaration of function >> 'nm_open' is invalid in C99 [-Wimplicit-function-declaration] >> struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); >> ^ >> ./pcap-netmap.c:195:18: warning: incompatible integer to pointer >> conversion initializing 'struct nm_desc *' with an expression of type >> 'int' [-Wint-conversion] >> struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); >> ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> ./pcap-netmap.c:210:36: error: incomplete definition of type 'struct >> nm_desc' >> __FUNCTION__, p->opt.source, d, d->fd, >> ~^ >> ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' >> struct nm_desc *d; /* pointer returned by nm_open() */ >> ^ >> ./pcap-netmap.c:213:11: error: incomplete definition of type 'struct >> nm_desc' >> p->fd = d->fd; >> ~^ >> ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' >> struct nm_desc *d; /* pointer returned by nm_open() */ >> ^ >> ./pcap-netmap.c:214:27: error: incomplete definition of type 'struct >> nm_desc' >> if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { >> ~^ >> ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' >> struct nm_desc *d; /* pointer returned by nm_open() */ >> ^ >> ./pcap-netmap.c:214:45: error: use of undeclared identifier >> 'NETMAP_SW_RING' >> if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { >> ^ >> ./pcap-netmap.c:215:24: error: use of undeclared identifier 'SIOCGIFFLAGS' >> pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch >> flags */ >> ^ >> ./pcap-netmap.c:216:20: error: use of undeclared identifier 'IFF_PPROMISC' >> if (!(if_flags & IFF_PPROMISC)) { >> ^ >> ./pcap-netmap.c:218:16: error: use of undeclared identifier 'IFF_PPROMISC' >> if_flags |= IFF_PPROMISC; >> ^ >> ./pcap-netmap.c:219:25: error: use of undeclared identifier 'SIOCSIFFLAGS' >> pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); >> ^ >> 6 warnings and 17 errors generated. >> *** Error code 1 >> >> Stop. >> make: stopped in /tmp/n/netmap-libpcap >> >> cat /usr/src/sys/net/netmap.h: >> >> /* >> * Copyright (C) 2011-2014 Matteo Landi, Luigi Rizzo. All rights reserved. >> * >> * Redistribution and use in source and binary forms, with or without >> * modification, are permitted provided that the following conditions >> * are met: >> * >> * 1. Redistributions of source code must retain the above copyright >> * notice, this list of conditions and the following disclaimer. >> * 2. Redistributions in binary form must reproduce the above copyright >> * notice, this list of conditions and the following disclaimer in >> the >> * documentation and/or other materials provided with the >> distribution. >> * >> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``S IS''AND >> * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >> * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >> PURPOSE >> * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE >> * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >> CONSEQUENTIAL >> * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS >> * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) >> * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, >> STRICT >> * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY >> WAY >> * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF >> * SUCH DAMAGE. >> */ >> >> /* >> * $FreeBSD: stable/10/sys/net/netmap.h 262151 2014-02-18 05:01:04Z luigi >> $ >> >> and cat /usr/src/sys/net/netmap_user.h >> >> /* >> * Copyright (C) 2011-2014 Universita` di Pisa. All rights reserved. >> * >> * Redistribution and use in source and binary forms, with or without >> * modification, are permitted provided that the following conditions >> * are met: >> * >> * 1. Redistributions of source code must retain the above copyright >> * notice, this list of conditions and the following disclaimer. >> * 2. Redistributions in binary form must reproduce the above copyright >> * notice, this list of conditions and the following disclaimer in >> the >> * documentation and/or other materials provided with the >> distribution. >> * >> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND >> * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >> * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >> PURPOSE >> * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE >> * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >> CONSEQUENTIAL >> * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS >> * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) >> * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, >> STRICT >> * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY >> WAY >> * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF >> * SUCH DAMAGE. >> */ >> >> /* >> * $FreeBSD: stable/10/sys/net/netmap_user.h 262151 2014-02-18 05:01:04Z >> luigi $ >> >> On Tue, Mar 4, 2014 at 4:01 PM, Luigi Rizzo wrote: >> > >> > >> > >> > On Tue, Mar 4, 2014 at 1:00 PM, C. L. Martinez >> > wrote: >> >> >> >> On Tue, Mar 4, 2014 at 11:45 AM, Luigi Rizzo >> >> wrote: >> >> > >> >> > >> >> > >> >> > On Tue, Mar 4, 2014 at 11:27 AM, C. L. Martinez >> >> > >> >> > wrote: >> >> >> >> >> >> Hi all, >> >> >> >> >> >> When I try to compile netmap-libpcap, these errors appears: >> >> >> >> >> >> root@plzfsiem01:/tmp/j/netmap-libpcap # make >> >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> >> >> -c ./pcap-bpf.c >> >> >> cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 >> >> >> -c ./pcap-netmap.c >> >> >> ./pcap-netmap.c:117:9: warning: implicit declaration of function >> >> >> 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] >> >> >> ret = nm_dispatch((void *)d, cnt, (void >> >> >> *)pcap_netmap_filter, (void *)p); >> >> > >> >> > >> >> > almost surely you have an old version of netmap.h and netmap_user.h >> >> > in /usr/include/net >> >> > >> >> > You should update to the version in stable/10 (at the very least >> >> > manually copy these two headers) >> >> > >> >> > cheers >> >> > luigi >> >> >> >> Thanks Luigi. Only netmap.h and netmap_user.h?? Can I use default >> >> files provided by FreeBSD 10-RELEASE under sys/dev/netmap or do I need >> >> to update them?? >> > >> > >> > as i said, you need to update the files. >> > you will also need the updated netmap kernel module so in the end >> > it might be worthwhile upgrading to stable/10 >> > >> > cheers >> > luigi >> > > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 18:33:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FBA55AA; Thu, 6 Mar 2014 18:33:46 +0000 (UTC) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 16C069FF; Thu, 6 Mar 2014 18:33:46 +0000 (UTC) Received: by mail-ve0-f175.google.com with SMTP id oz11so3028895veb.6 for ; Thu, 06 Mar 2014 10:33:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gvdAsCJXhuPCK5Y5K/nlwXvSCFbKYj4NBK+TzKwvmPM=; b=0frIT4hoN4PN0d1fz6N9ZMb28YxnjuxLaa938M/lgKp+yIYXCOcH8bhHvg3NLEaYUl /e1BQy9xBaBxOlrweu4aoA9BbtgB2+hCW75N1F5s15D8hgfqtSbC5fWNfTj+Dk6Z1SVQ PzQtuhZ6aG+am4FYHDalf9mzmkycQCvgUNlLAXNQ6OGnl96NqJY23qdexdraQfvCp2Mw OEq2P4bke5VN2OkFm7DOz6CpxkpXRWP9FWvzxBSi+2Jhmo9ET+98rV8hu/R/SVw35BTK Llb+mOD9Q8KprjbsVdeoWhj/hg4MF83nB6s2eG2U1vkHYuF69qr8Lc/Vb/2Aj0IF4N1m kHlA== MIME-Version: 1.0 X-Received: by 10.52.35.48 with SMTP id e16mr175645vdj.51.1394130825232; Thu, 06 Mar 2014 10:33:45 -0800 (PST) Received: by 10.221.11.135 with HTTP; Thu, 6 Mar 2014 10:33:45 -0800 (PST) In-Reply-To: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> References: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> Date: Thu, 6 Mar 2014 10:33:45 -0800 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang (was: Network loss) From: Jack Vogel To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:33:46 -0000 You did not make it explicit before, but I noticed in your dtrace info that you are using lagg, its been the source of lots of problems, so take it out of the setup and see if this queue problem still happens please. Jack On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert wrote: > (creating a new thread, because I'm no longer sure this is related to > Johan's thread that I originally used to discuss this) > > On 27.02.2014, at 18:02, Jack Vogel wrote: > > > I would make SURE that you have enough mbuf resources of whatever size > pool > > that you are > > using (2, 4, 9K), and I would try the code in HEAD if you had not. > > > > Jack > > Jack, we've upgraded some other systems on which I get more time to debug > (no impact for customers). Although those systems use the nfsclient too, I > no longer think that NFS is the source of the problem (hence the new > thread). I think it's the ixgbe driver and/or card. When our problem > occurs, it looks like it's a single tx queue that gets stuck somehow (its > buf_ring remains full). > > I tracked ping using dtrace to determine the source of ENOBUFS it returns > every few packets when things get weird: > > # dtrace -n 'fbt:::return / arg1 == ENOBUFS && execname == "ping" / { > stack(); }' > dtrace: description 'fbt:::return ' matched 25476 probes > CPU ID FUNCTION:NAME > 26 7730 ixgbe_mq_start:return > if_lagg.ko`lagg_transmit+0xc4 > kernel`ether_output_frame+0x33 > kernel`ether_output+0x4fe > kernel`ip_output+0xd74 > kernel`rip_output+0x229 > kernel`sosend_generic+0x3f6 > kernel`kern_sendit+0x1a3 > kernel`sendit+0xdc > kernel`sys_sendto+0x4d > kernel`amd64_syscall+0x5ea > kernel`0xffffffff80d35667 > > > > The only way ixgbe_mq_start could return ENOBUFS would be when > drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet > probably has no flow id, it should be assigned to a queue based on curcpu, > which made me try to pin ping to single cpus to check wether it's always > the same tx buf_ring that reports being full. This turned out to be true: > > # cpuset -l 0 ping 10.0.4.5 > PING 10.0.4.5 (10.0.4.5): 56 data bytes > 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.347 ms > 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.135 ms > > # cpuset -l 1 ping 10.0.4.5 > PING 10.0.4.5 (10.0.4.5): 56 data bytes > 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.184 ms > 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.232 ms > > # cpuset -l 2 ping 10.0.4.5 > PING 10.0.4.5 (10.0.4.5): 56 data bytes > ping: sendto: No buffer space available > ping: sendto: No buffer space available > ping: sendto: No buffer space available > ping: sendto: No buffer space available > ping: sendto: No buffer space available > > # cpuset -l 3 ping 10.0.4.5 > PING 10.0.4.5 (10.0.4.5): 56 data bytes > 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.130 ms > 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.126 ms > [...snip...] > > The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which use > the third tx buf_ring, ping reliably return ENOBUFS. If ping is run on any > other cpu using any other tx queue, it runs without any packet loss. > > So, when ENOBUFS is returned, this is not due to an mbuf shortage, it's > because the buf_ring is full. Not surprisingly, netstat -m looks pretty > normal: > > # netstat -m > 38622/11823/50445 mbufs in use (current/cache/total) > 32856/11642/44498/132096 mbuf clusters in use (current/cache/total/max) > 32824/6344 mbuf+clusters out of packet secondary zone in use > (current/cache) > 16/3906/3922/66048 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) > 75431K/41863K/117295K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > In the meantime I've checked the commit log of the ixgbe driver in HEAD > and besides there are little differences between HEAD and 9.2, I don't see > a commit that fixes anything related to what were seeing... > > So, what's the conclusion here? Firmware bug that's only triggered under > 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue stuff > was added? Jack, how should we proceed? > > > Markus > > > > On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert > wrote: > > > > > On 27.02.2014, at 02:00, Rick Macklem wrote: > > > >> John Baldwin wrote: > >>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>>> Hi all, > >>>> > >>>> I have a weird situation here where I can't get my head around. > >>>> > >>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in > >>>> a while > >>>> the Linux clients loose their NFS connection: > >>>> > >>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>>> timed out > >>>> > >>>> Not all boxes, just one out of the cluster. The weird part is that > >>>> when I > >>>> try to ping a Linux client from the FreeBSD box, I have between 10 > >>>> and 30% > >>>> packetloss - all day long, no specific timeframe. If I ping the > >>>> Linux > >>>> clients - no loss. If I ping back from the Linux clients to FBSD > >>>> box - no > >>>> loss. > >>>> > >>>> The errors I get when pinging a Linux client is this one: > >>>> ping: sendto: File too large > > > > We were facing similar problems when upgrading to 9.2 and have stayed > with > > 9.1 on affected systems for now. We've seen this on HP G8 blades with > > 82599EB controllers: > > > > ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 rev=0x01 > > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82599EB 10 Gigabit Dual Port Backplane Connection' > > class = network > > subclass = ethernet > > > > We didn't find a way to trigger the problem reliably. But when it occurs, > > it usually affects only one interface. Symptoms include: > > > > - socket functions return the 'File too large' error mentioned by Johan > > - socket functions return 'No buffer space' available > > - heavy to full packet loss on the affected interface > > - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should > > have timed out stick around forever (socket on the other side could have > > been closed ours ago) > > - userland programs using the corresponding sockets usually got stuck too > > (can't find kernel traces right now, but always in network related > syscalls) > > > > Network is only lightly loaded on the affected systems (usually 5-20 > mbit, > > capped at 200 mbit, per server), and netstat never showed any indication > of > > ressource shortage (like mbufs). > > > > What made the problem go away temporariliy was to ifconfig down/up the > > affected interface. > > > > We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really > > stable. Also, we tested a few revisions between 9.1 and 9.2 to find out > > when the problem started. Unfortunately, the ixgbe driver turned out to > be > > mostly unstable on our systems between these releases, worse than on 9.2. > > The instability was introduced shortly after to 9.1 and fixed only very > > shortly before 9.2 release. So no luck there. We ended up using 9.1 with > > backports of 9.2 features we really need. > > > > What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe driver > > or a combination of both that causes these problems. Unfortunately we ran > > out of time (and ideas). > > > > > >>> EFBIG is sometimes used for drivers when a packet takes too many > >>> scatter/gather entries. Since you mentioned NFS, one thing you can > >>> try is to > >>> disable TSO on the intertface you are using for NFS to see if that > >>> "fixes" it. > >>> > >> And please email if you try it and let us know if it helps. > >> > >> I've think I've figured out how 64K NFS read replies can do this, > >> but I'll admit "ping" is a mystery? (Doesn't it just send a single > >> packet that would be in a single mbuf?) > >> > >> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > >> don't know if it can happen for an mbuf chain with < 32 entries? > > > > We don't use the nfs server on our systems, but they're (new)nfsclients. > > So I don't think our problem is nfs related, unless the default > rsize/wsize > > for client mounts is not 8K, which I thought it was. Can you confirm > this, > > Rick? > > > > IIRC, disabling TSO did not make any difference in our case. > > > > > > Markus > > > > > > > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 20:58:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD5987AC; Thu, 6 Mar 2014 20:58:23 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 529F4A90; Thu, 6 Mar 2014 20:58:23 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:51770 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1WLfMw-000HI3-PW; Thu, 06 Mar 2014 21:58:18 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang (was: Network loss) From: Markus Gebert In-Reply-To: Date: Thu, 6 Mar 2014 21:58:16 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> References: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> To: Jack Vogel X-Mailer: Apple Mail (2.1874) Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 20:58:23 -0000 On 06.03.2014, at 19:33, Jack Vogel wrote: > You did not make it explicit before, but I noticed in your dtrace info = that > you are using > lagg, its been the source of lots of problems, so take it out of the = setup > and see if this > queue problem still happens please. >=20 > Jack Well, last year when upgrading another batch of servers (same hardware) = to 9.2, we tried find a solution to this network problem, and we = eliminated lagg where we had used it before, which did not help at all. = That=92s why I didn=92t mention it explicitly. My point is, I can confirm that 9.2 has network problems on this same = hardware with or without lagg, so it=92s unlikely that removing it will = bring immediate success. OTOH, I didn=92t have this tx queue theory back = then, so I cannot be sure that what we saw then without lagg, and what = we see now with lagg, really are the same problem. I guess, for the sake of simplicity I will remove lagg on these new = systems. But before I do that, to save time, I wanted to ask wether I = should remove vlan interfaces too? While that didn=92t help either last = year, my guess is that I should take them out of the picture, unless you = say otherwise. Thanks for looking into this. Markus > On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert = wrote: >=20 >> (creating a new thread, because I'm no longer sure this is related to >> Johan's thread that I originally used to discuss this) >>=20 >> On 27.02.2014, at 18:02, Jack Vogel wrote: >>=20 >>> I would make SURE that you have enough mbuf resources of whatever = size >> pool >>> that you are >>> using (2, 4, 9K), and I would try the code in HEAD if you had not. >>>=20 >>> Jack >>=20 >> Jack, we've upgraded some other systems on which I get more time to = debug >> (no impact for customers). Although those systems use the nfsclient = too, I >> no longer think that NFS is the source of the problem (hence the new >> thread). I think it's the ixgbe driver and/or card. When our problem >> occurs, it looks like it's a single tx queue that gets stuck somehow = (its >> buf_ring remains full). >>=20 >> I tracked ping using dtrace to determine the source of ENOBUFS it = returns >> every few packets when things get weird: >>=20 >> # dtrace -n 'fbt:::return / arg1 =3D=3D ENOBUFS && execname =3D=3D = "ping" / { >> stack(); }' >> dtrace: description 'fbt:::return ' matched 25476 probes >> CPU ID FUNCTION:NAME >> 26 7730 ixgbe_mq_start:return >> if_lagg.ko`lagg_transmit+0xc4 >> kernel`ether_output_frame+0x33 >> kernel`ether_output+0x4fe >> kernel`ip_output+0xd74 >> kernel`rip_output+0x229 >> kernel`sosend_generic+0x3f6 >> kernel`kern_sendit+0x1a3 >> kernel`sendit+0xdc >> kernel`sys_sendto+0x4d >> kernel`amd64_syscall+0x5ea >> kernel`0xffffffff80d35667 >>=20 >>=20 >>=20 >> The only way ixgbe_mq_start could return ENOBUFS would be when >> drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet >> probably has no flow id, it should be assigned to a queue based on = curcpu, >> which made me try to pin ping to single cpus to check wether it's = always >> the same tx buf_ring that reports being full. This turned out to be = true: >>=20 >> # cpuset -l 0 ping 10.0.4.5 >> PING 10.0.4.5 (10.0.4.5): 56 data bytes >> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.347 ms >> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.135 ms >>=20 >> # cpuset -l 1 ping 10.0.4.5 >> PING 10.0.4.5 (10.0.4.5): 56 data bytes >> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.184 ms >> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.232 ms >>=20 >> # cpuset -l 2 ping 10.0.4.5 >> PING 10.0.4.5 (10.0.4.5): 56 data bytes >> ping: sendto: No buffer space available >> ping: sendto: No buffer space available >> ping: sendto: No buffer space available >> ping: sendto: No buffer space available >> ping: sendto: No buffer space available >>=20 >> # cpuset -l 3 ping 10.0.4.5 >> PING 10.0.4.5 (10.0.4.5): 56 data bytes >> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.130 ms >> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.126 ms >> [...snip...] >>=20 >> The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which = use >> the third tx buf_ring, ping reliably return ENOBUFS. If ping is run = on any >> other cpu using any other tx queue, it runs without any packet loss. >>=20 >> So, when ENOBUFS is returned, this is not due to an mbuf shortage, = it's >> because the buf_ring is full. Not surprisingly, netstat -m looks = pretty >> normal: >>=20 >> # netstat -m >> 38622/11823/50445 mbufs in use (current/cache/total) >> 32856/11642/44498/132096 mbuf clusters in use = (current/cache/total/max) >> 32824/6344 mbuf+clusters out of packet secondary zone in use >> (current/cache) >> 16/3906/3922/66048 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) >> 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) >> 75431K/41863K/117295K bytes allocated to network = (current/cache/total) >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0/0/0 sfbufs in use (current/peak/max) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> 0 calls to protocol drain routines >>=20 >> In the meantime I've checked the commit log of the ixgbe driver in = HEAD >> and besides there are little differences between HEAD and 9.2, I = don't see >> a commit that fixes anything related to what were seeing... >>=20 >> So, what's the conclusion here? Firmware bug that's only triggered = under >> 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue = stuff >> was added? Jack, how should we proceed? >>=20 >>=20 >> Markus >>=20 >>=20 >>=20 >> On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert >> wrote: >>=20 >>>=20 >>> On 27.02.2014, at 02:00, Rick Macklem wrote: >>>=20 >>>> John Baldwin wrote: >>>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>>>> Hi all, >>>>>>=20 >>>>>> I have a weird situation here where I can't get my head around. >>>>>>=20 >>>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once = in >>>>>> a while >>>>>> the Linux clients loose their NFS connection: >>>>>>=20 >>>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>>>>> timed out >>>>>>=20 >>>>>> Not all boxes, just one out of the cluster. The weird part is = that >>>>>> when I >>>>>> try to ping a Linux client from the FreeBSD box, I have between = 10 >>>>>> and 30% >>>>>> packetloss - all day long, no specific timeframe. If I ping the >>>>>> Linux >>>>>> clients - no loss. If I ping back from the Linux clients to FBSD >>>>>> box - no >>>>>> loss. >>>>>>=20 >>>>>> The errors I get when pinging a Linux client is this one: >>>>>> ping: sendto: File too large >>>=20 >>> We were facing similar problems when upgrading to 9.2 and have = stayed >> with >>> 9.1 on affected systems for now. We've seen this on HP G8 blades = with >>> 82599EB controllers: >>>=20 >>> ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086 = rev=3D0x01 >>> hdr=3D0x00 >>> vendor =3D 'Intel Corporation' >>> device =3D '82599EB 10 Gigabit Dual Port Backplane Connection' >>> class =3D network >>> subclass =3D ethernet >>>=20 >>> We didn't find a way to trigger the problem reliably. But when it = occurs, >>> it usually affects only one interface. Symptoms include: >>>=20 >>> - socket functions return the 'File too large' error mentioned by = Johan >>> - socket functions return 'No buffer space' available >>> - heavy to full packet loss on the affected interface >>> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that = should >>> have timed out stick around forever (socket on the other side could = have >>> been closed ours ago) >>> - userland programs using the corresponding sockets usually got = stuck too >>> (can't find kernel traces right now, but always in network related >> syscalls) >>>=20 >>> Network is only lightly loaded on the affected systems (usually 5-20 >> mbit, >>> capped at 200 mbit, per server), and netstat never showed any = indication >> of >>> ressource shortage (like mbufs). >>>=20 >>> What made the problem go away temporariliy was to ifconfig down/up = the >>> affected interface. >>>=20 >>> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not = really >>> stable. Also, we tested a few revisions between 9.1 and 9.2 to find = out >>> when the problem started. Unfortunately, the ixgbe driver turned out = to >> be >>> mostly unstable on our systems between these releases, worse than on = 9.2. >>> The instability was introduced shortly after to 9.1 and fixed only = very >>> shortly before 9.2 release. So no luck there. We ended up using 9.1 = with >>> backports of 9.2 features we really need. >>>=20 >>> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe = driver >>> or a combination of both that causes these problems. Unfortunately = we ran >>> out of time (and ideas). >>>=20 >>>=20 >>>>> EFBIG is sometimes used for drivers when a packet takes too many >>>>> scatter/gather entries. Since you mentioned NFS, one thing you = can >>>>> try is to >>>>> disable TSO on the intertface you are using for NFS to see if that >>>>> "fixes" it. >>>>>=20 >>>> And please email if you try it and let us know if it helps. >>>>=20 >>>> I've think I've figured out how 64K NFS read replies can do this, >>>> but I'll admit "ping" is a mystery? (Doesn't it just send a single >>>> packet that would be in a single mbuf?) >>>>=20 >>>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >>>> don't know if it can happen for an mbuf chain with < 32 entries? >>>=20 >>> We don't use the nfs server on our systems, but they're = (new)nfsclients. >>> So I don't think our problem is nfs related, unless the default >> rsize/wsize >>> for client mounts is not 8K, which I thought it was. Can you confirm >> this, >>> Rick? >>>=20 >>> IIRC, disabling TSO did not make any difference in our case. >>>=20 >>>=20 >>> Markus >>>=20 >>>=20 >>=20 >>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 21:06:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 322F8A8B for ; Thu, 6 Mar 2014 21:06:26 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 09924B66 for ; Thu, 6 Mar 2014 21:06:26 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id x10so3053938pdj.34 for ; Thu, 06 Mar 2014 13:06:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=+B319KlDLy7MOc/R8zSLSJ3OBVofsx5kvmRb4Kgj82Q=; b=JvfeQ0F7GZrWbHwkD/T+dYCOFEYKPiw8ty2FCp2bnw1pi6qf9zJMD1ZG9TaFsu6LQi lbbQr73fJq4138UoReuaj5Q4+EmNlIBChn6+LVm5YM6fY3QgqKm57MpT6ALnf/X85LHN MCr2vJHqC5RAjDUSRC+U7+VEenb0EyBza7frpa4J1pp3wjq/AY//Fn9Tzipha9zHD1eS Mmlc+nuWB1H8iTyKSr7Jn5NoGEhQdWf6ICCi+t9osgnsmnmPO90CWh+oJq5j4HvUyNpC YzXepvaBnP6+APhyOVcZq4xj58CHGosUnHkX/2X05l3kk+2tAcdcBpPx+RgABSq3iCUM zKjA== X-Received: by 10.66.243.131 with SMTP id wy3mr17469153pac.32.1394139985755; Thu, 06 Mar 2014 13:06:25 -0800 (PST) Received: from h116-0-133-183.catv02.itscom.jp (h116-0-133-183.catv02.itscom.jp. [116.0.133.183]) by mx.google.com with ESMTPSA id xs1sm45250721pac.7.2014.03.06.13.06.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Mar 2014 13:06:23 -0800 (PST) From: Kenichi Mori Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Network C alphabet consign display return cosession., Message-Id: <7D6886D3-F826-44E6-BC88-F5FB206B31A2@gmail.com> Date: Fri, 7 Mar 2014 06:06:18 +0900 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 21:06:26 -0000 Header Crasher report in my knife user and pothic file., In Captcha system voicemail electric sounds., Many touch Smart phone Pad consign fighting mode., Three Playing Greek character dancing projective photoshop., Pony like just again kopsitty underground market dissolution dictionary = Kocept writements., Koalphabet in my tone generator Ah number extraction dictionary fished = blowing., Tenpo Proof Greek Konsider heaven Proof in ancient now., =E1=BF=AC=CF=81=CF=80=CF=8C=CE=BF=CE=BE=CE=BD=CE=BB=CE=AF=CE=B9 = =CE=B6=CE=AD=CE=B5=CE=B4=CE=B3=CE=B2=CE=AC=CE=B1 =CF=84=CE=A3=CF=85=CF=8D=CF= =86=CF=83=E1=BF=A5=CF=81 =CE=BC=CE=BB=CE=AF=CE=B9=CE=B8=CE=AE=CE=B7=CE=B6 = =CF=89=CF=8E=CF=9D=CF=9B=CF=9F=CF=A1=CF=85 =CF=81=CF=80=CF=8C=CE=BF=CE=BE=CE= =BD=CE=BC=CE=BB =CE=BA=CE=AF=CE=B9=CE=B8=CE=AE=E1=BF=A5=CF=82 =CF=8D=CF=86= =CF=88=CF=81 =CE=BF=CE=BE=CE=BD=CE=BC=CE=BB=CE=BA=CE=AF=CE=B9 = =CE=B8=CE=AE=CE=BD=CE=BE=CE=BF=CE=B8=CE=AE=CE=B7=CE=B6 = =CF=81=CF=80=CF=8C=CE=AD=CE=B4=CF=8D=CF=8D=CF=85=CF=86=CF=87=CF=88=CF=89 = =CE=BD=CE=BC=CE=BB=CE=BA=CE=AF=CE=B8=CE=AE=CE=B7=CE=B6=CE=AD=CE=B4=CE=B2 = =CE=BE=CE=BF=CF=8C=CF=80=CF=81=CF=83=CF=84=CF=85=CF=86 =CF=88=CF=89=CF=8E=CF= =9D=CF=8D=E1=BE=89 =CE=91=CE=96=CE=9D=CE=A3=CE=99=CE=93 =CE=9D=CF=85=CE=BC=CE=B2=CE=B5=CF=81=CE= =B5=CE=BF., =3D =CE=9C=CE=BC=CE=BD=CE=BE=CE=BF=CF=80=CF=81=E1=BF=A5=CF=82 = =CE=AE=CE=B7=CE=B6=CE=AD=CE=B5=CE=AF=CE=BE=CF=8C =CF=86=CF=87=CF=8C=CE=BF=CE= =BD=CE=BA=CE=BB=CE=B4 =CF=9F=CF=8E=CF=89=CF=87=CF=8D=CF=85=CF=83=E1=BF=A5 = =CE=BE=CE=BD=CE=BC=CE=BB=CE=AD=CF=84=CF=8D=CF=87=CF=86=20 =CE=9A=CE=BF=CE=BD=CF=83=CE=B9=CE=BD =CE=BB=CE=B1=CF=89 =CE=B9=CE=B4=CE=B5= =CE=BD=CF=84=CE=B9=CF=84=CF=85 =CE=B7=CE=BF=CE=BA=CE=B5 =CE=B9=CE=BD = =CE=BA=CE=BD=CE=B9=CF=86=CE=B5 =CF=80=CF=81=CE=BF=CE=BF=CF=86 =CE=BF=CE=BD= =CE=B5=CE=BA=CF=85=CE=B1=CE=BB =CE=B1=CE=BB=CF=86=CE=B1=CE=B2=CE=B5=CF=84= =CF=83=CE=B9=CE=B3=CE=BD =CE=B9=CE=BD=20 =CE=9A=CE=B5=CE=BD=CE=B9=CF=87=CE=B9 =CE=9C=CE=BF=CF=81=CE=B9 = =CE=9C=CE=B9=CF=85=CE=B1=CE=BA=CE=B5@=CE=95.=CF=85., Kenichi Mori Miyake@E.v., ccc.de From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 21:38:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11FD22E8; Thu, 6 Mar 2014 21:38:08 +0000 (UTC) Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9E0FEE2E; Thu, 6 Mar 2014 21:38:07 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id lg15so3209786vcb.40 for ; Thu, 06 Mar 2014 13:38:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kGl5c5qvau9BKUinObyPnuY4MaSt4s+hSmue64ugMwo=; b=NGIfkRWSEKM3ITf6OKOkjfC3fhNbR6LwNn7KJRPFZUC65wq6Y4dGn377Fcmfb8mKFh iYJlbJY0dei4W1akfpXX/br5GL24Z4LU5Lx4oZtpe2zZ4ad43/RHKiylTeLe1MSxC58h 5S9FUQM2oqGIRSCQ8HhLQ3mDgmKA55bPvUaq6jn0C7WA/9H6mQ9POkaiX0wYZnQh7XOy 1eP0jxueq3PaqBV6LZCRBROMcCuShOudxShm2Fe4y1Vu1+ATmIAYpPdMVdj8/m4cT0rQ LrQtNU+DCJXd7LKCevJbE8J0NUWAX+gT8V304cgunDLaSM7ZzCtR1GrBsS9CXMP4/q/s XhmA== MIME-Version: 1.0 X-Received: by 10.58.165.68 with SMTP id yw4mr7227891veb.17.1394141886671; Thu, 06 Mar 2014 13:38:06 -0800 (PST) Received: by 10.221.11.135 with HTTP; Thu, 6 Mar 2014 13:38:06 -0800 (PST) In-Reply-To: <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> References: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> Date: Thu, 6 Mar 2014 13:38:06 -0800 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang (was: Network loss) From: Jack Vogel To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 21:38:08 -0000 I suppose to be sure where the issue really occurs it would be best if both pseudo drivers were out of the picture. Once we see if it still occurs we can take the next step. Regards, Jack On Thu, Mar 6, 2014 at 12:58 PM, Markus Gebert wrote: > > On 06.03.2014, at 19:33, Jack Vogel wrote: > > > You did not make it explicit before, but I noticed in your dtrace info > that > > you are using > > lagg, its been the source of lots of problems, so take it out of the > setup > > and see if this > > queue problem still happens please. > > > > Jack > > Well, last year when upgrading another batch of servers (same hardware) to > 9.2, we tried find a solution to this network problem, and we eliminated > lagg where we had used it before, which did not help at all. That's why I > didn't mention it explicitly. > > My point is, I can confirm that 9.2 has network problems on this same > hardware with or without lagg, so it's unlikely that removing it will bring > immediate success. OTOH, I didn't have this tx queue theory back then, so I > cannot be sure that what we saw then without lagg, and what we see now with > lagg, really are the same problem. > > I guess, for the sake of simplicity I will remove lagg on these new > systems. But before I do that, to save time, I wanted to ask wether I > should remove vlan interfaces too? While that didn't help either last year, > my guess is that I should take them out of the picture, unless you say > otherwise. > > Thanks for looking into this. > > > Markus > > > > > On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert < > markus.gebert@hostpoint.ch>wrote: > > > >> (creating a new thread, because I'm no longer sure this is related to > >> Johan's thread that I originally used to discuss this) > >> > >> On 27.02.2014, at 18:02, Jack Vogel wrote: > >> > >>> I would make SURE that you have enough mbuf resources of whatever size > >> pool > >>> that you are > >>> using (2, 4, 9K), and I would try the code in HEAD if you had not. > >>> > >>> Jack > >> > >> Jack, we've upgraded some other systems on which I get more time to > debug > >> (no impact for customers). Although those systems use the nfsclient > too, I > >> no longer think that NFS is the source of the problem (hence the new > >> thread). I think it's the ixgbe driver and/or card. When our problem > >> occurs, it looks like it's a single tx queue that gets stuck somehow > (its > >> buf_ring remains full). > >> > >> I tracked ping using dtrace to determine the source of ENOBUFS it > returns > >> every few packets when things get weird: > >> > >> # dtrace -n 'fbt:::return / arg1 == ENOBUFS && execname == "ping" / { > >> stack(); }' > >> dtrace: description 'fbt:::return ' matched 25476 probes > >> CPU ID FUNCTION:NAME > >> 26 7730 ixgbe_mq_start:return > >> if_lagg.ko`lagg_transmit+0xc4 > >> kernel`ether_output_frame+0x33 > >> kernel`ether_output+0x4fe > >> kernel`ip_output+0xd74 > >> kernel`rip_output+0x229 > >> kernel`sosend_generic+0x3f6 > >> kernel`kern_sendit+0x1a3 > >> kernel`sendit+0xdc > >> kernel`sys_sendto+0x4d > >> kernel`amd64_syscall+0x5ea > >> kernel`0xffffffff80d35667 > >> > >> > >> > >> The only way ixgbe_mq_start could return ENOBUFS would be when > >> drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet > >> probably has no flow id, it should be assigned to a queue based on > curcpu, > >> which made me try to pin ping to single cpus to check wether it's always > >> the same tx buf_ring that reports being full. This turned out to be > true: > >> > >> # cpuset -l 0 ping 10.0.4.5 > >> PING 10.0.4.5 (10.0.4.5): 56 data bytes > >> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.347 ms > >> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.135 ms > >> > >> # cpuset -l 1 ping 10.0.4.5 > >> PING 10.0.4.5 (10.0.4.5): 56 data bytes > >> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.184 ms > >> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.232 ms > >> > >> # cpuset -l 2 ping 10.0.4.5 > >> PING 10.0.4.5 (10.0.4.5): 56 data bytes > >> ping: sendto: No buffer space available > >> ping: sendto: No buffer space available > >> ping: sendto: No buffer space available > >> ping: sendto: No buffer space available > >> ping: sendto: No buffer space available > >> > >> # cpuset -l 3 ping 10.0.4.5 > >> PING 10.0.4.5 (10.0.4.5): 56 data bytes > >> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.130 ms > >> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.126 ms > >> [...snip...] > >> > >> The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which use > >> the third tx buf_ring, ping reliably return ENOBUFS. If ping is run on > any > >> other cpu using any other tx queue, it runs without any packet loss. > >> > >> So, when ENOBUFS is returned, this is not due to an mbuf shortage, it's > >> because the buf_ring is full. Not surprisingly, netstat -m looks pretty > >> normal: > >> > >> # netstat -m > >> 38622/11823/50445 mbufs in use (current/cache/total) > >> 32856/11642/44498/132096 mbuf clusters in use (current/cache/total/max) > >> 32824/6344 mbuf+clusters out of packet secondary zone in use > >> (current/cache) > >> 16/3906/3922/66048 4k (page size) jumbo clusters in use > >> (current/cache/total/max) > >> 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) > >> 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) > >> 75431K/41863K/117295K bytes allocated to network (current/cache/total) > >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > >> 0/0/0 sfbufs in use (current/peak/max) > >> 0 requests for sfbufs denied > >> 0 requests for sfbufs delayed > >> 0 requests for I/O initiated by sendfile > >> 0 calls to protocol drain routines > >> > >> In the meantime I've checked the commit log of the ixgbe driver in HEAD > >> and besides there are little differences between HEAD and 9.2, I don't > see > >> a commit that fixes anything related to what were seeing... > >> > >> So, what's the conclusion here? Firmware bug that's only triggered under > >> 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue stuff > >> was added? Jack, how should we proceed? > >> > >> > >> Markus > >> > >> > >> > >> On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert > >> wrote: > >> > >>> > >>> On 27.02.2014, at 02:00, Rick Macklem wrote: > >>> > >>>> John Baldwin wrote: > >>>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: > >>>>>> Hi all, > >>>>>> > >>>>>> I have a weird situation here where I can't get my head around. > >>>>>> > >>>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in > >>>>>> a while > >>>>>> the Linux clients loose their NFS connection: > >>>>>> > >>>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, > >>>>>> timed out > >>>>>> > >>>>>> Not all boxes, just one out of the cluster. The weird part is that > >>>>>> when I > >>>>>> try to ping a Linux client from the FreeBSD box, I have between 10 > >>>>>> and 30% > >>>>>> packetloss - all day long, no specific timeframe. If I ping the > >>>>>> Linux > >>>>>> clients - no loss. If I ping back from the Linux clients to FBSD > >>>>>> box - no > >>>>>> loss. > >>>>>> > >>>>>> The errors I get when pinging a Linux client is this one: > >>>>>> ping: sendto: File too large > >>> > >>> We were facing similar problems when upgrading to 9.2 and have stayed > >> with > >>> 9.1 on affected systems for now. We've seen this on HP G8 blades with > >>> 82599EB controllers: > >>> > >>> ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 > rev=0x01 > >>> hdr=0x00 > >>> vendor = 'Intel Corporation' > >>> device = '82599EB 10 Gigabit Dual Port Backplane Connection' > >>> class = network > >>> subclass = ethernet > >>> > >>> We didn't find a way to trigger the problem reliably. But when it > occurs, > >>> it usually affects only one interface. Symptoms include: > >>> > >>> - socket functions return the 'File too large' error mentioned by Johan > >>> - socket functions return 'No buffer space' available > >>> - heavy to full packet loss on the affected interface > >>> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should > >>> have timed out stick around forever (socket on the other side could > have > >>> been closed ours ago) > >>> - userland programs using the corresponding sockets usually got stuck > too > >>> (can't find kernel traces right now, but always in network related > >> syscalls) > >>> > >>> Network is only lightly loaded on the affected systems (usually 5-20 > >> mbit, > >>> capped at 200 mbit, per server), and netstat never showed any > indication > >> of > >>> ressource shortage (like mbufs). > >>> > >>> What made the problem go away temporariliy was to ifconfig down/up the > >>> affected interface. > >>> > >>> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really > >>> stable. Also, we tested a few revisions between 9.1 and 9.2 to find out > >>> when the problem started. Unfortunately, the ixgbe driver turned out to > >> be > >>> mostly unstable on our systems between these releases, worse than on > 9.2. > >>> The instability was introduced shortly after to 9.1 and fixed only very > >>> shortly before 9.2 release. So no luck there. We ended up using 9.1 > with > >>> backports of 9.2 features we really need. > >>> > >>> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe > driver > >>> or a combination of both that causes these problems. Unfortunately we > ran > >>> out of time (and ideas). > >>> > >>> > >>>>> EFBIG is sometimes used for drivers when a packet takes too many > >>>>> scatter/gather entries. Since you mentioned NFS, one thing you can > >>>>> try is to > >>>>> disable TSO on the intertface you are using for NFS to see if that > >>>>> "fixes" it. > >>>>> > >>>> And please email if you try it and let us know if it helps. > >>>> > >>>> I've think I've figured out how 64K NFS read replies can do this, > >>>> but I'll admit "ping" is a mystery? (Doesn't it just send a single > >>>> packet that would be in a single mbuf?) > >>>> > >>>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I > >>>> don't know if it can happen for an mbuf chain with < 32 entries? > >>> > >>> We don't use the nfs server on our systems, but they're > (new)nfsclients. > >>> So I don't think our problem is nfs related, unless the default > >> rsize/wsize > >>> for client mounts is not 8K, which I thought it was. Can you confirm > >> this, > >>> Rick? > >>> > >>> IIRC, disabling TSO did not make any difference in our case. > >>> > >>> > >>> Markus > >>> > >>> > >> > >> > >> > >> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 21:21:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C791F92; Thu, 6 Mar 2014 21:21:23 +0000 (UTC) Received: from mail-we0-x234.google.com (mail-we0-x234.google.com [IPv6:2a00:1450:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6BD50C94; Thu, 6 Mar 2014 21:21:22 +0000 (UTC) Received: by mail-we0-f180.google.com with SMTP id p61so3855182wes.11 for ; Thu, 06 Mar 2014 13:21:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=V/rCI85u6latF+7E0Ku7bocyb5VIFwdMd8mUVIGGAUU=; b=IaPD0mc+F7YOkQT/XjJhd/1aHHro4lSljGcIU9UV0xLR6+MuDhmvGcko51obI36yBG z7yjheYgqGiDdg2++KQyuEDYOg3AcnE+fus99AcFjIe6NQBVGL40MCiFJj4HajeLFbbG YvNvotlPJM4Xmq20gcL0buA57i/d0WqmSu+xbB/I2o1sjXUCEAGAiHEJ5DfnMPNID7Pw rBf68ufLMLW3SqFOcByH4E/RNtjt0u0/YkXy4XxubMqgw97ucZcvC3kL8esGnxpdkebz UtEzfAnCmTwZser4/dypPKLVvXM5v87kHx8jczotizvQWYtiY2ULt8/pufxqAuK6E0G2 mkCA== MIME-Version: 1.0 X-Received: by 10.194.87.104 with SMTP id w8mr6873529wjz.90.1394140880847; Thu, 06 Mar 2014 13:21:20 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Thu, 6 Mar 2014 13:21:20 -0800 (PST) Date: Thu, 6 Mar 2014 14:21:20 -0700 X-Google-Sender-Auth: LK6PDRk4BK1iXRYVaSDq1fNvLxk Message-ID: Subject: Review: patch for kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN From: Alan Somers To: FreeBSD Net , rwatson@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 21:21:23 -0000 Replace 4.4BSD Lite's unix domain socket backpressure hack with a cleaner mechanism, based on the new SB_STOP sockbuf flag. The old hack dynamically changed the sending sockbuf's high water mark whenever adding or removing data from the receiving sockbuf. It worked for stream sockets, but it never worked for SOCK_SEQPACKET sockets because of their atomic nature. If the sockbuf was partially full, it might return EMSGSIZE instead of blocking. The new solution is based on DragonFlyBSD's fix from commit 3a6117bbe0ed6a87605c1e43e12a1438d8844380 on 2008-05-27. It adds an SB_STOP flag to sockbufs. Whenever uipc_send surpasses the socket's size limit, it sets SB_STOP on the sending sockbuf. sbspace() will then return 0 for that sockbuf, causing sosend_generic and friends to block. uipc_rcvd will likewise clear SB_STOP. There are two fringe benefits: uipc_{send,rcvd} no longer need to call chgsbsize() on every send and receive because they don't change the sockbuf's high water mark. Also, uipc_sense no longer needs to acquire the UIPC linkage lock, because it's simpler to compute the st_blksizes. There is one drawback: since sbspace() will only ever return 0 or the maximum, sosend_generic will allow the sockbuf to exceed its nominal maximum size by at most one packet of size less than the max. I don't think that's a serious problem. In fact, I'm not even positive that FreeBSD guarantees a socket will always stay within its nominal size limit. sys/sys/sockbuf.h Add the SB_STOP flag and adjust sbspace() sys/sys/unpcb.h Delete the obsolete unp_cc and unp_mbcnt fields from struct unpcb. sys/kern/uipc_usrreq.c Adjust uipc_rcvd, uipc_send, and uipc_sense to use the SB_STOP backpressure mechanism. Removing obsolete unpcb fields from db_show_unpcb. tests/sys/socket/unix_seqpacket/unix_seqpacket.c Clear expected failures from ATF. Obtained from: DragonFly BSD PR: kern/185812 Index: sys/kern/uipc_usrreq.c =================================================================== --- sys/kern/uipc_usrreq.c (revision 262867) +++ sys/kern/uipc_usrreq.c (working copy) @@ -51,7 +51,6 @@ * * TODO: * RDM - * distinguish datagram size limits from flow control limits in SEQPACKET * rethink name space problems * need a proper out-of-band */ @@ -789,7 +788,6 @@ struct unpcb *unp, *unp2; struct socket *so2; u_int mbcnt, sbcc; - u_long newhiwat; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_rcvd: unp == NULL")); @@ -811,6 +809,15 @@ mbcnt = so->so_rcv.sb_mbcnt; sbcc = so->so_rcv.sb_cc; SOCKBUF_UNLOCK(&so->so_rcv); + /* + * There is a benign race condition at this point. If we're planning to + * clear SB_STOP, but uipc_send is called on the connected socket at + * this instant, it might add data to the sockbuf and set SB_STOP. Then + * we would erroneously clear SB_STOP below, even though the sockbuf is + * full. The race is benign because the only ill effect is to allow the + * sockbuf to exceed its size limit, and the size limits are not + * strictly guaranteed anyway. + */ UNP_PCB_LOCK(unp); unp2 = unp->unp_conn; if (unp2 == NULL) { @@ -819,13 +826,9 @@ } so2 = unp2->unp_socket; SOCKBUF_LOCK(&so2->so_snd); - so2->so_snd.sb_mbmax += unp->unp_mbcnt - mbcnt; - newhiwat = so2->so_snd.sb_hiwat + unp->unp_cc - sbcc; - (void)chgsbsize(so2->so_cred->cr_uidinfo, &so2->so_snd.sb_hiwat, - newhiwat, RLIM_INFINITY); + if (sbcc < so2->so_snd.sb_hiwat && mbcnt < so2->so_snd.sb_mbmax) + so2->so_snd.sb_flags &= ~SB_STOP; sowwakeup_locked(so2); - unp->unp_mbcnt = mbcnt; - unp->unp_cc = sbcc; UNP_PCB_UNLOCK(unp); return (0); } @@ -836,8 +839,7 @@ { struct unpcb *unp, *unp2; struct socket *so2; - u_int mbcnt_delta, sbcc; - u_int newhiwat; + u_int mbcnt, sbcc; int error = 0; unp = sotounpcb(so); @@ -991,27 +993,21 @@ } } - /* - * XXXRW: While fine for SOCK_STREAM, this conflates maximum - * datagram size and back-pressure for SOCK_SEQPACKET, which - * can lead to undesired return of EMSGSIZE on send instead - * of more desirable blocking. - */ - mbcnt_delta = so2->so_rcv.sb_mbcnt - unp2->unp_mbcnt; - unp2->unp_mbcnt = so2->so_rcv.sb_mbcnt; + mbcnt = so2->so_rcv.sb_mbcnt; sbcc = so2->so_rcv.sb_cc; sorwakeup_locked(so2); + /* + * The PCB lock on unp2 protects the SB_STOP flag. Without it, + * it would be possible for uipc_rcvd to be called at this + * point, drain the receiving sockbuf, clear SB_STOP, and then + * we would set SB_STOP below. That could lead to an empty + * sockbuf having SB_STOP set + */ SOCKBUF_LOCK(&so->so_snd); - if ((int)so->so_snd.sb_hiwat >= (int)(sbcc - unp2->unp_cc)) - newhiwat = so->so_snd.sb_hiwat - (sbcc - unp2->unp_cc); - else - newhiwat = 0; - (void)chgsbsize(so->so_cred->cr_uidinfo, &so->so_snd.sb_hiwat, - newhiwat, RLIM_INFINITY); - so->so_snd.sb_mbmax -= mbcnt_delta; + if (sbcc >= so->so_snd.sb_hiwat || mbcnt >= so->so_snd.sb_mbmax) + so->so_snd.sb_flags |= SB_STOP; SOCKBUF_UNLOCK(&so->so_snd); - unp2->unp_cc = sbcc; UNP_PCB_UNLOCK(unp2); m = NULL; break; @@ -1049,27 +1045,18 @@ static int uipc_sense(struct socket *so, struct stat *sb) { - struct unpcb *unp, *unp2; - struct socket *so2; + struct unpcb *unp; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_sense: unp == NULL")); sb->st_blksize = so->so_snd.sb_hiwat; - UNP_LINK_RLOCK(); UNP_PCB_LOCK(unp); - unp2 = unp->unp_conn; - if ((so->so_type == SOCK_STREAM || so->so_type == SOCK_SEQPACKET) && - unp2 != NULL) { - so2 = unp2->unp_socket; - sb->st_blksize += so2->so_rcv.sb_cc; - } sb->st_dev = NODEV; if (unp->unp_ino == 0) unp->unp_ino = (++unp_ino == 0) ? ++unp_ino : unp_ino; sb->st_ino = unp->unp_ino; UNP_PCB_UNLOCK(unp); - UNP_LINK_RUNLOCK(); return (0); } @@ -2497,8 +2484,7 @@ /* XXXRW: Would be nice to print the full address, if any. */ db_printf("unp_addr: %p\n", unp->unp_addr); - db_printf("unp_cc: %d unp_mbcnt: %d unp_gencnt: %llu\n", - unp->unp_cc, unp->unp_mbcnt, + db_printf("unp_gencnt: %llu\n", (unsigned long long)unp->unp_gencnt); db_printf("unp_flags: %x (", unp->unp_flags); Index: sys/sys/sockbuf.h =================================================================== --- sys/sys/sockbuf.h (revision 262867) +++ sys/sys/sockbuf.h (working copy) @@ -52,6 +52,7 @@ #define SB_NOCOALESCE 0x200 /* don't coalesce new data into existing mbufs */ #define SB_IN_TOE 0x400 /* socket buffer is in the middle of an operation */ #define SB_AUTOSIZE 0x800 /* automatically size socket buffer */ +#define SB_STOP 0x1000 /* backpressure indicator */ #define SBS_CANTSENDMORE 0x0010 /* can't send more data to peer */ #define SBS_CANTRCVMORE 0x0020 /* can't receive more data from peer */ @@ -168,10 +169,20 @@ * still be negative (cc > hiwat or mbcnt > mbmax). Should detect * overflow and return 0. Should use "lmin" but it doesn't exist now. */ -#define sbspace(sb) \ - ((long) imin((int)((sb)->sb_hiwat - (sb)->sb_cc), \ - (int)((sb)->sb_mbmax - (sb)->sb_mbcnt))) +static __inline +long +sbspace(struct sockbuf *sb) +{ + long bleft; + long mleft; + if (sb->sb_flags & SB_STOP) + return(0); + bleft = sb->sb_hiwat - sb->sb_cc; + mleft = sb->sb_mbmax - sb->sb_mbcnt; + return((bleft < mleft) ? bleft : mleft); +} + /* adjust counters in sb reflecting allocation of m */ #define sballoc(sb, m) { \ (sb)->sb_cc += (m)->m_len; \ Index: sys/sys/unpcb.h =================================================================== --- sys/sys/unpcb.h (revision 262867) +++ sys/sys/unpcb.h (working copy) @@ -74,8 +74,8 @@ struct unp_head unp_refs; /* referencing socket linked list */ LIST_ENTRY(unpcb) unp_reflink; /* link in unp_refs list */ struct sockaddr_un *unp_addr; /* bound address of socket */ - int unp_cc; /* copy of rcv.sb_cc */ - int unp_mbcnt; /* copy of rcv.sb_mbcnt */ + int reserved1; + int reserved2; unp_gen_t unp_gencnt; /* generation count of this instance */ short unp_flags; /* flags */ short unp_gcflag; /* Garbage collector flags. */ Index: tests/sys/kern/unix_seqpacket_test.c =================================================================== --- tests/sys/kern/unix_seqpacket_test.c (revision 262867) +++ tests/sys/kern/unix_seqpacket_test.c (working copy) @@ -853,25 +853,21 @@ ATF_TC_WITHOUT_HEAD(eagain_8k_8k); ATF_TC_BODY(eagain_8k_8k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_eagain(8192, 8192); } ATF_TC_WITHOUT_HEAD(eagain_8k_128k); ATF_TC_BODY(eagain_8k_128k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_eagain(8192, 131072); } ATF_TC_WITHOUT_HEAD(eagain_128k_8k); ATF_TC_BODY(eagain_128k_8k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_eagain(131072, 8192); } ATF_TC_WITHOUT_HEAD(eagain_128k_128k); ATF_TC_BODY(eagain_128k_128k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_eagain(131072, 131072); } @@ -974,37 +970,24 @@ ATF_TC_WITHOUT_HEAD(pipe_8k_8k); ATF_TC_BODY(pipe_8k_8k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_pipe(8192, 8192); } ATF_TC_WITHOUT_HEAD(pipe_8k_128k); ATF_TC_BODY(pipe_8k_128k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_pipe(8192, 131072); } ATF_TC_WITHOUT_HEAD(pipe_128k_8k); ATF_TC_BODY(pipe_128k_8k, tc) { - /* - * kern/185812 causes this test case to both fail and timeout. The - * atf-c-api(3) doesn't have a way to set such an expectation. - * If you use atf_tc_expect_fail, then it will timeout. If you use - * atf_tc_expect_timeout, then it will fail. If you use both, then it - * will show up as an unexpected pass, which is much worse - * - * https://code.google.com/p/kyua/issues/detail?id=76 - */ - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_pipe(131072, 8192); } ATF_TC_WITHOUT_HEAD(pipe_128k_128k); ATF_TC_BODY(pipe_128k_128k, tc) { - atf_tc_expect_fail("PR kern/185812 send(2) on a UNIX domain SEQPACKET socket returns EMSGSIZE instead of EAGAIN"); test_pipe(131072, 131072); } From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 21:46:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAD7478D for ; Thu, 6 Mar 2014 21:46:11 +0000 (UTC) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::3]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1C5D1F03 for ; Thu, 6 Mar 2014 21:46:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1394142367; l=12673; s=domk; d=haakh.de; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References: Subject:To:MIME-Version:From:Date:X-RZG-CLASS-ID:X-RZG-AUTH; bh=q3ujKYdf+GrwghJ4ubilcG3+olk=; b=q+9ixqe7OhODAgoQ5+lHH9iz7jkx4RQa8ix/zWMEs2aYHrkklOS9kHqBLvCI3fb2QC9 GKH8xH6kiR++0q88DmBQQuCm9f4GwIHEcCLIj0cFTWVAthpju0RQSoa67QKqBq1AT5knL vgAsBw64w+utNNJIS8LXbVD/Okyz/Ny83x4= X-RZG-AUTH: :LWQcbViwW/e6OTbW0dHzwKkCepEs/ThuRG8zpeuciRNkwehqPJJjNL7gV97j X-RZG-CLASS-ID: mo00 Received: from abaton.Haakh.de (p57A72615.dip0.t-ipconnect.de [87.167.38.21]) by smtp.strato.de (RZmta 32.27 DYNA|AUTH) with ESMTPA id i073e8q26Ljunut for ; Thu, 6 Mar 2014 22:45:56 +0100 (CET) Received: from Crabberio.Haakh.de (crabberio.Haakh.de [192.168.63.16]) by abaton.Haakh.de (8.14.8/8.14.7) with ESMTP id s26LjtZA013040 for ; Thu, 6 Mar 2014 22:45:56 +0100 (CET) (envelope-from bugReporter@Haakh.de) Message-ID: <5318EC93.4000303@Haakh.de> Date: Thu, 06 Mar 2014 22:45:55 +0100 From: "Dr. A. Haakh" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:27.0) Gecko/20100101 Firefox/27.0 SeaMonkey/2.24 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: 9.2 ixgbe tx queue hang References: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> In-Reply-To: <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 21:46:11 -0000 Markus Gebert schrieb: > On 06.03.2014, at 19:33, Jack Vogel wrote: > >> You did not make it explicit before, but I noticed in your dtrace info that >> you are using >> lagg, its been the source of lots of problems, so take it out of the setup >> and see if this >> queue problem still happens please. >> >> Jack > Well, last year when upgrading another batch of servers (same hardware) to 9.2, we tried find a solution to this network problem, and we eliminated lagg where we had used it before, which did not help at all. That’s why I didn’t mention it explicitly. > > My point is, I can confirm that 9.2 has network problems on this same hardware with or without lagg, so it’s unlikely that removing it will bring immediate success. OTOH, I didn’t have this tx queue theory back then, so I cannot be sure that what we saw then without lagg, and what we see now with lagg, really are the same problem. > > I guess, for the sake of simplicity I will remove lagg on these new systems. But before I do that, to save time, I wanted to ask wether I should remove vlan interfaces too? While that didn’t help either last year, my guess is that I should take them out of the picture, unless you say otherwise. > > Thanks for looking into this. > > > Markus > I don't use ixgbe but this might be related to the discussed problem. I too realized network problems when I moved from 9.1 to 9.2 last october. Occasionally I use vlc to watch tv on udp://@224.0.0.1:7792 coming from an XP-system which displayed perfect on 9.1 but got scrambled on 9.2. By accident I realized that vlc worked fine again, when I had a cpu-intensiv job like portupgrade -a running. So I thought it might be a problem related to the scheduler. In the meantime I upgraded to 10.0-STABLE and things looks better now -- it still takes about 20 seconds for a video-stream get synchronized. My system is CPU: Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz (2675.02-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106e5 Family = 0x6 Model = 0x1e Stepping = 5 Features=0xbfebfbff Features2=0x98e3fd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 12884901888 (12288 MB) avail memory = 12438151168 (11861 MB) with this ethernet-card re0: port 0xd800-0xd8ff mem 0xf6fff000-0xf6ffffff,0xf6ff8000-0xf6ffbfff irq 19 at device 0.0 on pci2 re0: Using 1 MSI-X message re0: Chip rev. 0x28000000 re0: MAC rev. 0x00300000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Ethernet address: 90:e6:ba:bb:28:3e Andreas > >> On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert wrote: >> >>> (creating a new thread, because I'm no longer sure this is related to >>> Johan's thread that I originally used to discuss this) >>> >>> On 27.02.2014, at 18:02, Jack Vogel wrote: >>> >>>> I would make SURE that you have enough mbuf resources of whatever size >>> pool >>>> that you are >>>> using (2, 4, 9K), and I would try the code in HEAD if you had not. >>>> >>>> Jack >>> Jack, we've upgraded some other systems on which I get more time to debug >>> (no impact for customers). Although those systems use the nfsclient too, I >>> no longer think that NFS is the source of the problem (hence the new >>> thread). I think it's the ixgbe driver and/or card. When our problem >>> occurs, it looks like it's a single tx queue that gets stuck somehow (its >>> buf_ring remains full). >>> >>> I tracked ping using dtrace to determine the source of ENOBUFS it returns >>> every few packets when things get weird: >>> >>> # dtrace -n 'fbt:::return / arg1 == ENOBUFS && execname == "ping" / { >>> stack(); }' >>> dtrace: description 'fbt:::return ' matched 25476 probes >>> CPU ID FUNCTION:NAME >>> 26 7730 ixgbe_mq_start:return >>> if_lagg.ko`lagg_transmit+0xc4 >>> kernel`ether_output_frame+0x33 >>> kernel`ether_output+0x4fe >>> kernel`ip_output+0xd74 >>> kernel`rip_output+0x229 >>> kernel`sosend_generic+0x3f6 >>> kernel`kern_sendit+0x1a3 >>> kernel`sendit+0xdc >>> kernel`sys_sendto+0x4d >>> kernel`amd64_syscall+0x5ea >>> kernel`0xffffffff80d35667 >>> >>> >>> >>> The only way ixgbe_mq_start could return ENOBUFS would be when >>> drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet >>> probably has no flow id, it should be assigned to a queue based on curcpu, >>> which made me try to pin ping to single cpus to check wether it's always >>> the same tx buf_ring that reports being full. This turned out to be true: >>> >>> # cpuset -l 0 ping 10.0.4.5 >>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.347 ms >>> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.135 ms >>> >>> # cpuset -l 1 ping 10.0.4.5 >>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.184 ms >>> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.232 ms >>> >>> # cpuset -l 2 ping 10.0.4.5 >>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>> ping: sendto: No buffer space available >>> ping: sendto: No buffer space available >>> ping: sendto: No buffer space available >>> ping: sendto: No buffer space available >>> ping: sendto: No buffer space available >>> >>> # cpuset -l 3 ping 10.0.4.5 >>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.130 ms >>> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.126 ms >>> [...snip...] >>> >>> The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which use >>> the third tx buf_ring, ping reliably return ENOBUFS. If ping is run on any >>> other cpu using any other tx queue, it runs without any packet loss. >>> >>> So, when ENOBUFS is returned, this is not due to an mbuf shortage, it's >>> because the buf_ring is full. Not surprisingly, netstat -m looks pretty >>> normal: >>> >>> # netstat -m >>> 38622/11823/50445 mbufs in use (current/cache/total) >>> 32856/11642/44498/132096 mbuf clusters in use (current/cache/total/max) >>> 32824/6344 mbuf+clusters out of packet secondary zone in use >>> (current/cache) >>> 16/3906/3922/66048 4k (page size) jumbo clusters in use >>> (current/cache/total/max) >>> 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) >>> 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) >>> 75431K/41863K/117295K bytes allocated to network (current/cache/total) >>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >>> 0/0/0 sfbufs in use (current/peak/max) >>> 0 requests for sfbufs denied >>> 0 requests for sfbufs delayed >>> 0 requests for I/O initiated by sendfile >>> 0 calls to protocol drain routines >>> >>> In the meantime I've checked the commit log of the ixgbe driver in HEAD >>> and besides there are little differences between HEAD and 9.2, I don't see >>> a commit that fixes anything related to what were seeing... >>> >>> So, what's the conclusion here? Firmware bug that's only triggered under >>> 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue stuff >>> was added? Jack, how should we proceed? >>> >>> >>> Markus >>> >>> >>> >>> On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert >>> wrote: >>> >>>> On 27.02.2014, at 02:00, Rick Macklem wrote: >>>> >>>>> John Baldwin wrote: >>>>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> I have a weird situation here where I can't get my head around. >>>>>>> >>>>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. Once in >>>>>>> a while >>>>>>> the Linux clients loose their NFS connection: >>>>>>> >>>>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not responding, >>>>>>> timed out >>>>>>> >>>>>>> Not all boxes, just one out of the cluster. The weird part is that >>>>>>> when I >>>>>>> try to ping a Linux client from the FreeBSD box, I have between 10 >>>>>>> and 30% >>>>>>> packetloss - all day long, no specific timeframe. If I ping the >>>>>>> Linux >>>>>>> clients - no loss. If I ping back from the Linux clients to FBSD >>>>>>> box - no >>>>>>> loss. >>>>>>> >>>>>>> The errors I get when pinging a Linux client is this one: >>>>>>> ping: sendto: File too large >>>> We were facing similar problems when upgrading to 9.2 and have stayed >>> with >>>> 9.1 on affected systems for now. We've seen this on HP G8 blades with >>>> 82599EB controllers: >>>> >>>> ix0@pci0:4:0:0: class=0x020000 card=0x18d0103c chip=0x10f88086 rev=0x01 >>>> hdr=0x00 >>>> vendor = 'Intel Corporation' >>>> device = '82599EB 10 Gigabit Dual Port Backplane Connection' >>>> class = network >>>> subclass = ethernet >>>> >>>> We didn't find a way to trigger the problem reliably. But when it occurs, >>>> it usually affects only one interface. Symptoms include: >>>> >>>> - socket functions return the 'File too large' error mentioned by Johan >>>> - socket functions return 'No buffer space' available >>>> - heavy to full packet loss on the affected interface >>>> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that should >>>> have timed out stick around forever (socket on the other side could have >>>> been closed ours ago) >>>> - userland programs using the corresponding sockets usually got stuck too >>>> (can't find kernel traces right now, but always in network related >>> syscalls) >>>> Network is only lightly loaded on the affected systems (usually 5-20 >>> mbit, >>>> capped at 200 mbit, per server), and netstat never showed any indication >>> of >>>> ressource shortage (like mbufs). >>>> >>>> What made the problem go away temporariliy was to ifconfig down/up the >>>> affected interface. >>>> >>>> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not really >>>> stable. Also, we tested a few revisions between 9.1 and 9.2 to find out >>>> when the problem started. Unfortunately, the ixgbe driver turned out to >>> be >>>> mostly unstable on our systems between these releases, worse than on 9.2. >>>> The instability was introduced shortly after to 9.1 and fixed only very >>>> shortly before 9.2 release. So no luck there. We ended up using 9.1 with >>>> backports of 9.2 features we really need. >>>> >>>> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe driver >>>> or a combination of both that causes these problems. Unfortunately we ran >>>> out of time (and ideas). >>>> >>>> >>>>>> EFBIG is sometimes used for drivers when a packet takes too many >>>>>> scatter/gather entries. Since you mentioned NFS, one thing you can >>>>>> try is to >>>>>> disable TSO on the intertface you are using for NFS to see if that >>>>>> "fixes" it. >>>>>> >>>>> And please email if you try it and let us know if it helps. >>>>> >>>>> I've think I've figured out how 64K NFS read replies can do this, >>>>> but I'll admit "ping" is a mystery? (Doesn't it just send a single >>>>> packet that would be in a single mbuf?) >>>>> >>>>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >>>>> don't know if it can happen for an mbuf chain with < 32 entries? >>>> We don't use the nfs server on our systems, but they're (new)nfsclients. >>>> So I don't think our problem is nfs related, unless the default >>> rsize/wsize >>>> for client mounts is not 8K, which I thought it was. Can you confirm >>> this, >>>> Rick? >>>> >>>> IIRC, disabling TSO did not make any difference in our case. >>>> >>>> >>>> Markus >>>> >>>> >>> >>> >>> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > From owner-freebsd-net@FreeBSD.ORG Thu Mar 6 21:57:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2BB2A6B for ; Thu, 6 Mar 2014 21:57:45 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDC86FF8 for ; Thu, 6 Mar 2014 21:57:45 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s26LvhIY094513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Mar 2014 13:57:44 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s26LvhZi094512; Thu, 6 Mar 2014 13:57:43 -0800 (PST) (envelope-from jmg) Date: Thu, 6 Mar 2014 13:57:43 -0800 From: John-Mark Gurney To: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Message-ID: <20140306215743.GB47921@funkthat.com> Mail-Followup-To: Julien Charbon , freebsd-net@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 06 Mar 2014 13:57:44 -0800 (PST) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 21:57:46 -0000 Julien Charbon wrote this message on Thu, Feb 27, 2014 at 11:32 +0100: > Obviously, to mitigate this lock contention there are various solutions: > > - Introduce a new time-wait lock as proposed in joined patch > - Call tcp_tw_2msl_scan() more often in case of high workload > - Use INP_INFO_TRY_WLOCK() in tcp_tw_2msl_scan() to clean-up time-wait > objects only when nobody else handles INP_INFO lock > - Etc. > > The strategy being to prioritize packet reception over time-wait > objects cleaned-up as: > > - we hate dropping packet in reception when the bandwidth is far from > being full > - the maximum of used time-wait objects is configurable > (net.inet.tcp.maxtcptw) > - in case of time-wait objects memory exhaustion, the current behavior > is already optimal: The oldest time-wait object is recycled and > directly reused. > > We picked the time-wait lock way because it suits well our long-term > strategy to completely mitigate the INP_INFO lock contention everywhere > in TCP stack. > > Any thoughts on this particular behavior? One thing that I noticed is that you now lock/unlock the tw and inp lock a lot... Have you thought about grabing the TW lock once, grabbing some/all of the ones necessary to process and then process them in a second step? If the bulk processing is still an issue, then you could process them in blocks of 50 or so, that way the number of lock/unlock cycles is reduced... > +/* > + * Drop a refcount on an tw elevated using tw_pcbref(). If it is > + * valid, we return with the tw lock held. > + */ I assume you mean that you return with the tw lock unlocked? at least that's what the code reads to me... [...] > +static int > +tw_pcbrele(struct tcptw *tw) > +{ > + TW_WLOCK_ASSERT(V_tw_lock); > + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); > + > + if (!refcount_release(&tw->tw_refcount)) { > + TW_WUNLOCK(V_tw_lock); > + return (0); > + } > + > + uma_zfree(V_tcptw_zone, tw); > + TW_WUNLOCK(V_tw_lock); > + return (1); > +} [...] Otherwise looks like a good patch... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 02:48:23 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43A7CD16 for ; Fri, 7 Mar 2014 02:48:23 +0000 (UTC) Received: from hapkido.dreamhost.com (hapkido.dreamhost.com [66.33.216.122]) by mx1.freebsd.org (Postfix) with ESMTP id 1A90ED9C for ; Fri, 7 Mar 2014 02:48:22 +0000 (UTC) Received: from homiemail-a109.g.dreamhost.com (unknown [69.163.253.148]) by hapkido.dreamhost.com (Postfix) with ESMTP id 27DA2DC54E for ; Thu, 6 Mar 2014 18:33:17 -0800 (PST) Received: from homiemail-a109.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a109.g.dreamhost.com (Postfix) with ESMTP id A5CCC2005D827; Thu, 6 Mar 2014 18:48:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=saltant.com; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=saltant.com; bh=fVZ8LbxUKzCjdO/fIhGniM7B/mU=; b= Wu8ZAsBqXEegJwzSI8tPi40W7N8JVwTaiXUVoVIlBwWrAX+5AruyxDFb+qfFaT3O eKrstDVoKfEd3EZfR6wNyhiHtodiTj6rtzZyQVr3ud0dNip1PwCSSdKMqBjyhRHa PeR8WVx7itKZL+GMOf2BmYVZjiFA4b15s6SiBPEesp4= Received: from dreck.saltant.net (dreck.saltant.net [72.78.188.150]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: john@saltant.com) by homiemail-a109.g.dreamhost.com (Postfix) with ESMTPSA id 6A9F22005D826; Thu, 6 Mar 2014 18:48:15 -0800 (PST) Message-ID: <53193371.4090603@saltant.com> Date: Thu, 06 Mar 2014 21:48:17 -0500 From: "John W. O'Brien" Organization: Saltant Solutions User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Eric Masson Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> In-Reply-To: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="I40r9NONdAJJR9wJm29ARcJtAKvoRo7xb" Cc: Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 02:48:23 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --I40r9NONdAJJR9wJm29ARcJtAKvoRo7xb Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Eric, On 1/25/14 10:28 AM, Eric Masson wrote: > Hi, >=20 > I've setup a lab to experiment nat before ipsec scenario. > Architecture : > - 3 host only interfaces have been set up on the host > - 4 FreeBSD10 guests have been set up : > - 2 clients connected to their respective gateways via dedicated host= > only interfaces. > - 2 gateways connected together via dedicated host only interface Trimming configs for clarity > Gateway 1 setup : > <-----------------------------------------------------------------> > emss@gateway1:~ % more /etc/rc.conf > hostname=3D"gateway1" > ifconfig_em1=3D"inet 192.168.11.15 netmask 255.255.255.0" > ifconfig_em0=3D"inet 10.0.0.5 netmask 255.255.255.0" > gateway_enable=3D"YES" > ipsec_enable=3D"YES" > ipsec_file=3D"/etc/ipsec.conf" > firewall_enable=3D"YES" > firewall_script=3D"/etc/ipfw.rules" > firewall_logging=3D"YES" > emss@gateway1:~ % more /etc/ipfw.rules > #!/bin/sh > cmd=3D"/sbin/ipfw" > $cmd -f flush > $cmd add 00100 nat 100 all from 192.168.11.0/24 to 192.168.21.0/24 You also need to perform NAT processing on the traffic that returns to gateway1 from gateway2. $cmd add 200 nat 100 all from 192.168.21.0/24 to 172.16.0.1 > $cmd nat 100 config log ip 172.16.0.1 reverse > emss@gateway1:~ % more /etc/ipsec.conf > flush; > spdflush; >=20 > add 10.0.0.5 10.0.0.6 esp 0x1000 -E 3des-cbc "123456789012345678901234"= ; > add 10.0.0.6 10.0.0.5 esp 0x1001 -E 3des-cbc "432109876543210987654321"= ; >=20 > add 10.0.0.5 10.0.0.6 ipcomp 0x2000 -C deflate; > add 10.0.0.6 10.0.0.5 ipcomp 0x2001 -C deflate; >=20 > spdadd 192.168.21.0/24 172.16.0.1/32 any -P in ipsec > ipcomp/tunnel/10.0.0.6-10.0.0.5/require > esp/tunnel/10.0.0.6-10.0.0.5/require; >=20 > spdadd 172.16.0.1/32 192.168.21.0/24 any -P out ipsec > ipcomp/tunnel/10.0.0.5-10.0.0.6/require > esp/tunnel/10.0.0.5-10.0.0.6/require; > emss@gateway1:~ % more /boot/loader.conf > ipfw_load=3D"YES" > ipfw_nat_load=3D"YES" >=20 > net.inet.ip.fw.default_to_accept=3D"1" I'm curious to learn whether this is sufficient. I haven't tested any combination of NAT and IPsec. Regards, John --I40r9NONdAJJR9wJm29ARcJtAKvoRo7xb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBCgAGBQJTGTNxAAoJEBRzAKlhyP/FrsQIAL/4JxnWThM2C/U9+X6aH6En OmacOCP0Rq6rdYpa0qqtgnz49V4o7qMbSjYMKxBHGRPwlYpUKgdBlmkqpx1jtiJo CHM1mNJP5pu3yfzo74r1QrHdRIpsgGlXl0jRU00uG6YjYfdI3zjx0UWaN7qy9xbQ U5QjIvX3rzHUyTpGIlShCB2XJs0aT9a1W8fbJfYKf1CLdij93CYE7Bck9xT31fzy YYmSZUdBDh5nvOlfzXq8Hp4AOzPsfyBEZlpWGXEhgm/cbQDeAxY/cnrn2fDPgI0t fiwQ0Nrqm6WVOSx+j1o1nB7qm74V73C8qlo6qfYgaY6A2n3TgAE6ZG2WKAV2jDQ= =T34V -----END PGP SIGNATURE----- --I40r9NONdAJJR9wJm29ARcJtAKvoRo7xb-- From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 05:17:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27DC878F for ; Fri, 7 Mar 2014 05:17:18 +0000 (UTC) Received: from vms173003pub.verizon.net (vms173003pub.verizon.net [206.46.173.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0BC58B99 for ; Fri, 7 Mar 2014 05:17:17 +0000 (UTC) Received: from [192.168.1.3] ([unknown] [173.60.122.163]) by vms173003.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N210060FW0HOS20@vms173003.mailsrvcs.net> for freebsd-net@freebsd.org; Thu, 06 Mar 2014 23:17:10 -0600 (CST) Date: Thu, 06 Mar 2014 21:17:05 -0800 (PST) From: jcv To: freebsd-net@freebsd.org Subject: LAN network performance issues Message-id: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-version: 1.0 Content-type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 05:17:18 -0000 Hi - I am seeing some strange IPERF results.. Everything goes through my WIFI/GIGABIT router. For these tests everything is plugged directly into the router via Ethernet cable. My issue is the transfer rate from Windows to FreeBSD. There are 3 different computers in this lab running 3 different OS. Here are the results: FreeBSD as server: [vic@yeaguy ~] iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 4] local 192.168.1.3 port 5001 connected with 192.168.1.8 port 52505 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.1 sec 157 MBytes 131 Mbits/sec <----- WINDOWS 8.1 as client on same LAN/ROUTER [ 5] local 192.168.1.3 port 5001 connected with 192.168.1.12 port 60926 [ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec <------ MACBOOK PRO as client on same LAN/ROUTER Windows as the server: ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.3 port 60529 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.0 sec 1014 MBytes 850 Mbits/sec <--------- Freebsd 10 as client on same LAN/ROUTER [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.12 port 60933 [ 4] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec <------ MACBOOK PRO as client on same LAN/ROUTER Macbook Pro as the server: [ 3] local 192.168.1.8 port 52509 connected with 192.168.1.12 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 823 MBytes 690 Mbits/sec <------ WINDOWS 8.1 as client on same LAN/ROUTER [ 3] local 192.168.1.3 port 23190 connected with 192.168.1.12 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec <------ Freebsd 10 as client on same LAN/ROUTER With FreeBSD being the server, Windows transfer to FreeBSD is slow, compared to Macbook to FreeBSD transfer.. With Windows as the server, FreeBSD and Macbook to Windows transfer is great. With Macbook as server, Windows and FreeBSD transfer is good. The only bad transfer is Windows to FreeBSD. Windows transfer to Mac is good. Cant really blame Windows for the poor transfer to FreeBSD then. Macbook to FreeBSD is outstanding, cant really blame FreeBSD for poor receive performance. I am on version: FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 06:23:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B4EA2FE for ; Fri, 7 Mar 2014 06:23:52 +0000 (UTC) Received: from mail-ee0-x22d.google.com (mail-ee0-x22d.google.com [IPv6:2a00:1450:4013:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CB3C6AF for ; Fri, 7 Mar 2014 06:23:51 +0000 (UTC) Received: by mail-ee0-f45.google.com with SMTP id d17so1516907eek.32 for ; Thu, 06 Mar 2014 22:23:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6GjPKbtrw2EloCoFgG9WSv/0doF88lvBm/r383rJZeU=; b=eu6NKOSVQUtOR4tQkx/+M3JHscHBVD0hRjhs2Dytsvrwt/gjv2ylzLtCe5hr0IfrLo Ulr1CRbIlpTk8f7RJhKIwubCNz65/Hd++ZyEoc8bN1TrLCS8aLUHKylHBnfoY4yOy0wi dxBSuNQWH5yme9oPGYWhZqajTIV0+VJnTMrSo/YfEX234LGc4GAodw03zUqHgi3x7uUc unVP3plDPQpbqoJWO9Iwcx9xp6EPVvt1/JYs7qlmd0x4bIdR1sws4Fj8QmX22XzN90I+ AYXPgpbY8FBQxto9uOdIFQU8b3w82mEGd5UU4Vc31GAPkrtPii49cYT16XGCPapt+8Hp MpoQ== MIME-Version: 1.0 X-Received: by 10.14.39.3 with SMTP id c3mr16949577eeb.42.1394173430111; Thu, 06 Mar 2014 22:23:50 -0800 (PST) Received: by 10.14.65.4 with HTTP; Thu, 6 Mar 2014 22:23:50 -0800 (PST) Date: Thu, 6 Mar 2014 22:23:50 -0800 Message-ID: Subject: Include port number in "Listen queue overflow" messages From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 06:23:52 -0000 I am thinking of committing following change that includes port number in "Listen queue overflow" messages. New message would look something like: sonewconn: pcb 0xfffff8001b155760: Listen queue overflow on port 13120: 1 already in queue awaiting acceptance (454 occurrences) I've recently ran into a situation at $work where I could not catch the culprit application via "netstat -A" and had to dive into kgdb to find the port from pcb where this application was listening to. IMO, this change will make debugging easier. cheers, Hiren Index: sys/kern/uipc_socket.c =================================================================== --- sys/kern/uipc_socket.c (revision 262861) +++ sys/kern/uipc_socket.c (working copy) @@ -136,6 +136,7 @@ #include #include #include +#include #include @@ -491,8 +492,11 @@ static int overcount; struct socket *so; + struct inpcb *inp; int over; + inp = sotoinpcb(head); + ACCEPT_LOCK(); over = (head->so_qlen > 3 * head->so_qlimit / 2); ACCEPT_UNLOCK(); @@ -504,10 +508,12 @@ overcount++; if (ratecheck(&lastover, &overinterval)) { - log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow: " - "%i already in queue awaiting acceptance " + log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow on " + "port %d: %i already in queue awaiting acceptance " "(%d occurrences)\n", - __func__, head->so_pcb, head->so_qlen, overcount); + __func__, head->so_pcb, + ntohs(inp->inp_inc.inc_lport), head->so_qlen, + overcount); overcount = 0; } From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 06:55:36 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4168DAB4 for ; Fri, 7 Mar 2014 06:55:36 +0000 (UTC) Received: from mail.schmidp.com (mail.schmidp.com [IPv6:2a01:4f8:120:4ffe::9]) by mx1.freebsd.org (Postfix) with ESMTP id DF36532F for ; Fri, 7 Mar 2014 06:55:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.schmidp.com (Postfix) with ESMTP id 0DD485802CD; Fri, 7 Mar 2014 08:01:36 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mail.schmidp.com Received: from mail.schmidp.com ([127.0.0.1]) by localhost (dna.schmidp.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m8GzGYvjzTSt; Fri, 7 Mar 2014 08:01:32 +0100 (CET) Received: from charlie.lan (chello213047013064.west2.11.vie.surfer.at [213.47.13.64]) by mail.schmidp.com (Postfix) with ESMTPSA id B349C58016C; Fri, 7 Mar 2014 08:01:31 +0100 (CET) Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated From: Philipp Schmid In-Reply-To: <53193371.4090603@saltant.com> Date: Fri, 7 Mar 2014 07:55:22 +0100 Message-Id: <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> To: "John W. O'Brien" X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Eric Masson , Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 06:55:36 -0000 Hi Eric, FreeBSD 10 seems to have problems with IPSec and filtering/nat. Maybe your problem is related to: http://www.freebsd.org/cgi/query-pr.cgi?pr=185876 - Philipp On 07 Mar 2014, at 03:48, John W. O'Brien wrote: > Hi Eric, > > On 1/25/14 10:28 AM, Eric Masson wrote: >> Hi, >> >> I've setup a lab to experiment nat before ipsec scenario. >> Architecture : >> - 3 host only interfaces have been set up on the host >> - 4 FreeBSD10 guests have been set up : >> - 2 clients connected to their respective gateways via dedicated host >> only interfaces. >> - 2 gateways connected together via dedicated host only interface > > Trimming configs for clarity > >> Gateway 1 setup : >> <-----------------------------------------------------------------> >> emss@gateway1:~ % more /etc/rc.conf >> hostname="gateway1" >> ifconfig_em1="inet 192.168.11.15 netmask 255.255.255.0" >> ifconfig_em0="inet 10.0.0.5 netmask 255.255.255.0" >> gateway_enable="YES" >> ipsec_enable="YES" >> ipsec_file="/etc/ipsec.conf" >> firewall_enable="YES" >> firewall_script="/etc/ipfw.rules" >> firewall_logging="YES" >> emss@gateway1:~ % more /etc/ipfw.rules >> #!/bin/sh >> cmd="/sbin/ipfw" >> $cmd -f flush >> $cmd add 00100 nat 100 all from 192.168.11.0/24 to 192.168.21.0/24 > > You also need to perform NAT processing on the traffic that returns to > gateway1 from gateway2. > > $cmd add 200 nat 100 all from 192.168.21.0/24 to 172.16.0.1 > >> $cmd nat 100 config log ip 172.16.0.1 reverse >> emss@gateway1:~ % more /etc/ipsec.conf >> flush; >> spdflush; >> >> add 10.0.0.5 10.0.0.6 esp 0x1000 -E 3des-cbc "123456789012345678901234"; >> add 10.0.0.6 10.0.0.5 esp 0x1001 -E 3des-cbc "432109876543210987654321"; >> >> add 10.0.0.5 10.0.0.6 ipcomp 0x2000 -C deflate; >> add 10.0.0.6 10.0.0.5 ipcomp 0x2001 -C deflate; >> >> spdadd 192.168.21.0/24 172.16.0.1/32 any -P in ipsec >> ipcomp/tunnel/10.0.0.6-10.0.0.5/require >> esp/tunnel/10.0.0.6-10.0.0.5/require; >> >> spdadd 172.16.0.1/32 192.168.21.0/24 any -P out ipsec >> ipcomp/tunnel/10.0.0.5-10.0.0.6/require >> esp/tunnel/10.0.0.5-10.0.0.6/require; >> emss@gateway1:~ % more /boot/loader.conf >> ipfw_load="YES" >> ipfw_nat_load="YES" >> >> net.inet.ip.fw.default_to_accept="1" > > I'm curious to learn whether this is sufficient. I haven't tested any > combination of NAT and IPsec. > > Regards, > John > From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 10:29:13 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FEED6BC for ; Fri, 7 Mar 2014 10:29:13 +0000 (UTC) Received: from nk11p18im-asmtp002.me.com (nk11p18im-asmtp002.me.com [17.158.120.161]) by mx1.freebsd.org (Postfix) with ESMTP id 6A0CBB70 for ; Fri, 7 Mar 2014 10:29:13 +0000 (UTC) Received: from [10.42.0.51] (unknown [61.150.43.99]) by nk11p18im-asmtp002.mac.com (Oracle Communications Messaging Server 7u4-27.08(7.0.4.27.7) 64bit (built Aug 22 2013)) with ESMTPSA id <0N2200IEU7O7WG70@nk11p18im-asmtp002.mac.com> for net@freebsd.org; Fri, 07 Mar 2014 09:29:07 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87,1.0.14,0.0.0000 definitions=2014-03-07_03:2014-03-05,2014-03-07,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=1 spamscore=1 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1401130000 definitions=main-1403070013 Subject: join the maillist From: =?GB2312?B?19q35SDA7g==?= Content-type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (10B329) Message-id: Date: Fri, 07 Mar 2014 17:29:05 +0800 To: "net@freebsd.org" Content-transfer-encoding: 7bit MIME-version: 1.0 (1.0) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 10:29:13 -0000 I'm learning the netmap,hope to discuss in the netmap . From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 12:15:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8537E79C for ; Fri, 7 Mar 2014 12:15:59 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10235890 for ; Fri, 7 Mar 2014 12:15:58 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id t61so4885430wes.17 for ; Fri, 07 Mar 2014 04:15:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type:content-transfer-encoding; bh=Of0kP5mmL6ISLcam41qsX5Rr0czDY787vqDhfYWlz1k=; b=LMlYTeugHUw+2h3mWdd57YbmXMNbHaB5WPhMGbd4S6csGOztWWV0yd9zgZFEi3WYaI 43lYiVaFvWj2By5JIJNnjmi3gkkyC5yeOrAfUu9tRY4wypQVLWJrQdMDOgf7odioZ4yP fPvsxyqKEnKjhXVqP7UGI8IDWLN8N3/cFUz5w3dXXd7gc2+EqvY6L83kPAVpsdBTNEO9 +tJ3Eq5CZCGbhdlf4eamf39sMns6IGHzNHtvSA6a5jWyw00EvpJsJxeh8HS5RFQhQnFb QX1PycO1QRErQBIJox5iVYyczSH9d4p+dQjXFYkeYpVymqn8PdNmwVXph8EUzl9LwXm+ kAdg== X-Received: by 10.194.190.10 with SMTP id gm10mr18561257wjc.55.1394194557421; Fri, 07 Mar 2014 04:15:57 -0800 (PST) Received: from srvbsdfenssv.interne.associated-bears.org (LCaen-151-92-21-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by mx.google.com with ESMTPSA id f7sm8699308wjb.7.2014.03.07.04.15.55 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 07 Mar 2014 04:15:56 -0800 (PST) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id BCE29CF227; Fri, 7 Mar 2014 13:15:54 +0100 (CET) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I90xflpgUBoX; Fri, 7 Mar 2014 13:15:53 +0100 (CET) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id E4ADFCF0EF; Fri, 7 Mar 2014 13:15:52 +0100 (CET) From: Eric Masson To: "John W. O'Brien" Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated In-Reply-To: <53193371.4090603@saltant.com> (John W. O'Brien's message of "Thu, 06 Mar 2014 21:48:17 -0500") References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) X-Operating-System: FreeBSD 9.2-RELEASE-p3 amd64 Date: Fri, 07 Mar 2014 13:15:52 +0100 Message-ID: <8661nqmcg7.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 12:15:59 -0000 "John W. O'Brien" writes: Hi John, > You also need to perform NAT processing on the traffic that returns to > gateway1 from gateway2. > > $cmd add 200 nat 100 all from 192.168.21.0/24 to 172.16.0.1 I've been privately told about the return rule (I'm used to pf not ipfw), but no luck. Seems that http://www.freebsd.org/cgi/query-pr.cgi?pr=185876, as stated by Philipp could be an good candidate to explain failures even with return rule set up. > I'm curious to learn whether this is sufficient. I haven't tested any > combination of NAT and IPsec. bz@ seem to have used this kind of setup for a looong time ;) Regards Éric -- This is a multi-part message in MIME format. ... Content-Transfer-Encoding: quoted-printable ... J EN AI MARRE DES C... QUI NE RESPECTENT PAS LES CHARTES -+- R in: Guide du neuneu Usenet - bien respecter sa netiquette -+- From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 12:17:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 655D583E for ; Fri, 7 Mar 2014 12:17:03 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E34D58A1 for ; Fri, 7 Mar 2014 12:17:02 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id t61so4851114wes.3 for ; Fri, 07 Mar 2014 04:17:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type:content-transfer-encoding; bh=UMAyvX/eh80rRtDYr3fy6vHDTStsDazK6/zkIGJi8mc=; b=O1dnGh8VTnUjmrmK8Kv6E5k7Ws/reUCqOr41VuKgqyYt8z5neIa0KituquFWbbsy4b //yoRUXGoNryzjclxt6b372DApSAQbsZ7YeUfZUVPRlNm7Hm10DheWxwDnCwLmieoJxQ YrtwzmIssKZ3KzfcoivClBYfSA3MnYyWfUDcvGIuS5vip8ZdKtKlde0/1dLutB4/qpeU aT2kPlFt1BFU6MYcjxDUFOBSjyCxKzc38IbpMc37IYtYOBiGaJJbfhDZ+5vXhiw6s0Eu h0NRm7yZ4ohBk0lsw2iqiY/mH4vPYZB7ofG7SPpjqxfAEIMBT+o1z6xIFtRTnDZtr89n Eggg== X-Received: by 10.194.123.195 with SMTP id mc3mr1781501wjb.96.1394194621421; Fri, 07 Mar 2014 04:17:01 -0800 (PST) Received: from srvbsdfenssv.interne.associated-bears.org (LCaen-151-92-21-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by mx.google.com with ESMTPSA id az1sm8700171wjb.11.2014.03.07.04.17.00 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 07 Mar 2014 04:17:00 -0800 (PST) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id 8A84BCF0D1; Fri, 7 Mar 2014 13:16:59 +0100 (CET) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3gF-tz4I9jzV; Fri, 7 Mar 2014 13:16:56 +0100 (CET) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id A967ACF22F; Fri, 7 Mar 2014 13:16:56 +0100 (CET) From: Eric Masson To: Philipp Schmid Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated In-Reply-To: <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> (Philipp Schmid's message of "Fri, 7 Mar 2014 07:55:22 +0100") References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) X-Operating-System: FreeBSD 9.2-RELEASE-p3 amd64 Date: Fri, 07 Mar 2014 13:16:56 +0100 Message-ID: <861tyemcef.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: "John W. O'Brien" , Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 12:17:03 -0000 Philipp Schmid writes: Hi Philipp, > FreeBSD 10 seems to have problems with IPSec and filtering/nat. > Maybe your problem is related to: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=185876 I'll try the patch enclosed asap (overwhelmed by paid work these days). Regards Éric -- voila je suis depuis un certain temps sur l'essait du decryptage de certaines images X mises dans des magazines. Ceci juste afin d'apprendre la cryptologie et le decryptage. -+-S in - j'apprends aussi l'hypocrisyptage -+- From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 12:58:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 733DC637 for ; Fri, 7 Mar 2014 12:58:40 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3031AC14 for ; Fri, 7 Mar 2014 12:58:40 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WLu7e-0003aI-7s for freebsd-net@freebsd.org; Fri, 07 Mar 2014 13:43:30 +0100 Received: from 217.30.88.7 ([217.30.88.7]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 07 Mar 2014 13:43:30 +0100 Received: from jcharbon by 217.30.88.7 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 07 Mar 2014 13:43:30 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Fri, 07 Mar 2014 13:43:19 +0100 Lines: 85 Message-ID: <5319BEE7.607@verisign.com> References: <20140306215743.GB47921@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 217.30.88.7 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 In-Reply-To: <20140306215743.GB47921@funkthat.com> Cc: jmg@funkthat.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 12:58:40 -0000 Hi John, On 06/03/14 22:57, John-Mark Gurney wrote: > Julien Charbon wrote this message on Thu, Feb 27, 2014 at 11:32 +0100: >> [...] >> Any thoughts on this particular behavior? > > One thing that I noticed is that you now lock/unlock the tw and inp lock a > lot... Have you thought about grabing the TW lock once, grabbing some/all > of the ones necessary to process and then process them in a second step? > If the bulk processing is still an issue, then you could process them in > blocks of 50 or so, that way the number of lock/unlock cycles is reduced... First thanks, feedback are highly valuable to us. In first place, I indeed tried a kind of bulk processing enforcement with something like: tcp_tw_2msl_scan() { struct tcptw *tw; int i, end = 0, count = 100; for (;;) { INP_INFO_WLOCK(&V_tcbinfo); for (i = 0; i < count; ++i) { tw = TAILQ_FIRST(&V_twq_2msl); if (tw == NULL || (tw->tw_time - ticks) > 0)) { end = 1; break; } INP_WLOCK(tw->tw_inpcb); tcp_twclose(tw, 0); } if (end) break; INP_INFO_WUNLOCK(&V_tcbinfo); } return (NULL); } And I got best result with 'count' set to 1, this led us to current proposed patch. Thus main goal here is somehow to prioritize the NIC interruption handler calls against tcp_tw_2msl_scan() call in INP_INFO battle. As you proposed, we can add a sysctl(or a #define) to configure the maximum of tw objects to be cleaned up under a same INP_INFO_WLOCK() call, to have a finer control on how the tw objects are enforced. That said, our high level solution is to consider the NIC interruption code path (i.e. tcp_input()) as critical and remove almost most contention points on it, which is our long term goal. This change is just a step on this (long and not straightforward) path. >> +/* >> + * Drop a refcount on an tw elevated using tw_pcbref(). If it is >> + * valid, we return with the tw lock held. >> + */ > > I assume you mean that you return with the tw lock unlocked? at least > that's what the code reads to me... > >> +static int >> +tw_pcbrele(struct tcptw *tw) >> +{ >> + TW_WLOCK_ASSERT(V_tw_lock); >> + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); >> + >> + if (!refcount_release(&tw->tw_refcount)) { >> + TW_WUNLOCK(V_tw_lock); >> + return (0); >> + } >> + >> + uma_zfree(V_tcptw_zone, tw); >> + TW_WUNLOCK(V_tw_lock); >> + return (1); >> +} You are completely right, my mistake. I will update the comment. > Otherwise looks like a good patch... Thanks again for your time. -- Julien From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 13:55:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FD849B3; Fri, 7 Mar 2014 13:55:48 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C8B931F2; Fri, 7 Mar 2014 13:55:47 +0000 (UTC) Received: from [77.109.131.203] (port=49791 helo=[192.168.3.123]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WLvFZ-0004mv-41; Fri, 07 Mar 2014 14:55:45 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: 9.2 ixgbe tx queue hang (was: Network loss) From: Markus Gebert In-Reply-To: Date: Fri, 7 Mar 2014 14:55:44 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <9C5B43BD-9D80-49EA-8EDC-C7EF53D79C8D@hostpoint.ch> <02AD7510-C862-4C29-9420-25ABF1A6E744@hostpoint.ch> To: Jack Vogel X-Mailer: Apple Mail (2.1827) Cc: Johan Kooijman , FreeBSD Net , Rick Macklem , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 13:55:48 -0000 On 06.03.2014, at 22:38, Jack Vogel wrote: > I suppose to be sure where the issue really occurs it would be best if = both > pseudo drivers > were out of the picture. Once we see if it still occurs we can take = the > next step. >=20 > Regards, >=20 > Jack I removed lagg and vlan interfaces on four servers shortly after your = email. And one of these is already showing the symptoms. I think this = time we caught it earlier, and saw EFBIG instead of ENOBUFS. Still = coming from only two cpus (or two tx queues): # cpuset -l 0 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=3D0 ttl=3D255 time=3D1.150 ms [=85] # cpuset -l 1 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=3D0 ttl=3D255 time=3D0.385 ms [=85] # cpuset -l 2 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=3D0 ttl=3D255 time=3D0.385 ms [=85] # cpuset -l 3 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes ping: sendto: File too large ping: sendto: File too large ping: sendto: File too large [=85] # cpuset -l 4 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes ping: sendto: File too large ping: sendto: File too large ping: sendto: File too large [=85] # cpuset -l 5 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=3D0 ttl=3D255 time=3D0.316 ms [=85 (other cpus work)] So the fact that only a few cpus/tx queues are affected remains true = even without lagg/vlans. But we=92ve got a new stack trace: # dtrace -n 'fbt:::return / arg1 =3D=3D EFBIG && execname =3D=3D "ping" = / { stack(); }' dtrace: description 'fbt:::return ' matched 25400 probes [=85] 3 28502 _bus_dmamap_load_buffer:return=20 kernel`_bus_dmamap_load_mbuf_sg+0x5f kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`rip_output+0x229 kernel`sosend_generic+0x3f6 kernel`kern_sendit+0x1a3 kernel`sendit+0xdc kernel`sys_sendto+0x4d kernel`amd64_syscall+0x5ea kernel`0xffffffff80d35667 [=85] But after a while, everything went back to like it was yesterday = (ENOBUFS): # cpuset -l 3 ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes ping: sendto: No buffer space available [=85 (cpu4 the same, other cpus work] and the already known stacktrace: # dtrace -n 'fbt:::return / arg1 =3D=3D ENOBUFS && execname =3D=3D = "ping" / { stack(); }' dtrace: description 'fbt:::return ' matched 25400 probes CPU ID FUNCTION:NAME 3 7730 ixgbe_mq_start:return=20 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`rip_output+0x229 kernel`sosend_generic+0x3f6 kernel`kern_sendit+0x1a3 kernel`sendit+0xdc kernel`sys_sendto+0x4d kernel`amd64_syscall+0x5ea kernel`0xffffffff80d35667 So in my opinion, the different situation in the first few minutes was = not because lagg and vlans were removed, it=92s just that we caught it = early and the buf_ring was not full yet, but at least now we=92ve got a = stack trace of what happens before, which hopefully helps debugging = this. I also noticed that queue txd sysctls barely change or don=92t change at = all on affected queues: # while true; do sysctl dev.ix.1.queue3 dev.ix.1.queue4 | grep txd_; = echo ----; sleep 1; done dev.ix.1.queue3.txd_head: 717 dev.ix.1.queue3.txd_tail: 717 dev.ix.1.queue4.txd_head: 1517 dev.ix.1.queue4.txd_tail: 1517 ---- dev.ix.1.queue3.txd_head: 717 dev.ix.1.queue3.txd_tail: 717 dev.ix.1.queue4.txd_head: 1517 dev.ix.1.queue4.txd_tail: 1517 ---- dev.ix.1.queue3.txd_head: 717 dev.ix.1.queue3.txd_tail: 717 dev.ix.1.queue4.txd_head: 1517 dev.ix.1.queue4.txd_tail: 1517 ---- dev.ix.1.queue3.txd_head: 717 dev.ix.1.queue3.txd_tail: 717 dev.ix.1.queue4.txd_head: 1517 dev.ix.1.queue4.txd_tail: 1517 But I guess that=92s expected when the buf_ring is full=85 But maybe = it=92s important to you, that I might have seen these values rise just = once, when the problem already had started. And netstat -m: # netstat -m 37872/10263/48135 mbufs in use (current/cache/total) 33142/11680/44822/132096 mbuf clusters in use (current/cache/total/max) 33044/6508 mbuf+clusters out of packet secondary zone in use = (current/cache) 94/2995/3089/66048 4k (page size) jumbo clusters in use = (current/cache/total/max) 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) 76128K/37905K/114033K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Markus > On Thu, Mar 6, 2014 at 12:58 PM, Markus Gebert > wrote: >=20 >>=20 >> On 06.03.2014, at 19:33, Jack Vogel wrote: >>=20 >>> You did not make it explicit before, but I noticed in your dtrace = info >> that >>> you are using >>> lagg, its been the source of lots of problems, so take it out of the >> setup >>> and see if this >>> queue problem still happens please. >>>=20 >>> Jack >>=20 >> Well, last year when upgrading another batch of servers (same = hardware) to >> 9.2, we tried find a solution to this network problem, and we = eliminated >> lagg where we had used it before, which did not help at all. That's = why I >> didn't mention it explicitly. >>=20 >> My point is, I can confirm that 9.2 has network problems on this same >> hardware with or without lagg, so it's unlikely that removing it will = bring >> immediate success. OTOH, I didn't have this tx queue theory back = then, so I >> cannot be sure that what we saw then without lagg, and what we see = now with >> lagg, really are the same problem. >>=20 >> I guess, for the sake of simplicity I will remove lagg on these new >> systems. But before I do that, to save time, I wanted to ask wether I >> should remove vlan interfaces too? While that didn't help either last = year, >> my guess is that I should take them out of the picture, unless you = say >> otherwise. >>=20 >> Thanks for looking into this. >>=20 >>=20 >> Markus >>=20 >>=20 >>=20 >>> On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert < >> markus.gebert@hostpoint.ch>wrote: >>>=20 >>>> (creating a new thread, because I'm no longer sure this is related = to >>>> Johan's thread that I originally used to discuss this) >>>>=20 >>>> On 27.02.2014, at 18:02, Jack Vogel wrote: >>>>=20 >>>>> I would make SURE that you have enough mbuf resources of whatever = size >>>> pool >>>>> that you are >>>>> using (2, 4, 9K), and I would try the code in HEAD if you had not. >>>>>=20 >>>>> Jack >>>>=20 >>>> Jack, we've upgraded some other systems on which I get more time to >> debug >>>> (no impact for customers). Although those systems use the nfsclient >> too, I >>>> no longer think that NFS is the source of the problem (hence the = new >>>> thread). I think it's the ixgbe driver and/or card. When our = problem >>>> occurs, it looks like it's a single tx queue that gets stuck = somehow >> (its >>>> buf_ring remains full). >>>>=20 >>>> I tracked ping using dtrace to determine the source of ENOBUFS it >> returns >>>> every few packets when things get weird: >>>>=20 >>>> # dtrace -n 'fbt:::return / arg1 =3D=3D ENOBUFS && execname =3D=3D = "ping" / { >>>> stack(); }' >>>> dtrace: description 'fbt:::return ' matched 25476 probes >>>> CPU ID FUNCTION:NAME >>>> 26 7730 ixgbe_mq_start:return >>>> if_lagg.ko`lagg_transmit+0xc4 >>>> kernel`ether_output_frame+0x33 >>>> kernel`ether_output+0x4fe >>>> kernel`ip_output+0xd74 >>>> kernel`rip_output+0x229 >>>> kernel`sosend_generic+0x3f6 >>>> kernel`kern_sendit+0x1a3 >>>> kernel`sendit+0xdc >>>> kernel`sys_sendto+0x4d >>>> kernel`amd64_syscall+0x5ea >>>> kernel`0xffffffff80d35667 >>>>=20 >>>>=20 >>>>=20 >>>> The only way ixgbe_mq_start could return ENOBUFS would be when >>>> drbr_enqueue() encouters a full tx buf_ring. Since a new ping = packet >>>> probably has no flow id, it should be assigned to a queue based on >> curcpu, >>>> which made me try to pin ping to single cpus to check wether it's = always >>>> the same tx buf_ring that reports being full. This turned out to be >> true: >>>>=20 >>>> # cpuset -l 0 ping 10.0.4.5 >>>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>>> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.347 ms >>>> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.135 ms >>>>=20 >>>> # cpuset -l 1 ping 10.0.4.5 >>>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>>> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.184 ms >>>> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.232 ms >>>>=20 >>>> # cpuset -l 2 ping 10.0.4.5 >>>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>>> ping: sendto: No buffer space available >>>> ping: sendto: No buffer space available >>>> ping: sendto: No buffer space available >>>> ping: sendto: No buffer space available >>>> ping: sendto: No buffer space available >>>>=20 >>>> # cpuset -l 3 ping 10.0.4.5 >>>> PING 10.0.4.5 (10.0.4.5): 56 data bytes >>>> 64 bytes from 10.0.4.5: icmp_seq=3D0 ttl=3D255 time=3D0.130 ms >>>> 64 bytes from 10.0.4.5: icmp_seq=3D1 ttl=3D255 time=3D0.126 ms >>>> [...snip...] >>>>=20 >>>> The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which = use >>>> the third tx buf_ring, ping reliably return ENOBUFS. If ping is run = on >> any >>>> other cpu using any other tx queue, it runs without any packet = loss. >>>>=20 >>>> So, when ENOBUFS is returned, this is not due to an mbuf shortage, = it's >>>> because the buf_ring is full. Not surprisingly, netstat -m looks = pretty >>>> normal: >>>>=20 >>>> # netstat -m >>>> 38622/11823/50445 mbufs in use (current/cache/total) >>>> 32856/11642/44498/132096 mbuf clusters in use = (current/cache/total/max) >>>> 32824/6344 mbuf+clusters out of packet secondary zone in use >>>> (current/cache) >>>> 16/3906/3922/66048 4k (page size) jumbo clusters in use >>>> (current/cache/total/max) >>>> 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max) >>>> 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max) >>>> 75431K/41863K/117295K bytes allocated to network = (current/cache/total) >>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >>>> 0/0/0 sfbufs in use (current/peak/max) >>>> 0 requests for sfbufs denied >>>> 0 requests for sfbufs delayed >>>> 0 requests for I/O initiated by sendfile >>>> 0 calls to protocol drain routines >>>>=20 >>>> In the meantime I've checked the commit log of the ixgbe driver in = HEAD >>>> and besides there are little differences between HEAD and 9.2, I = don't >> see >>>> a commit that fixes anything related to what were seeing... >>>>=20 >>>> So, what's the conclusion here? Firmware bug that's only triggered = under >>>> 9.2? Driver bug introduced between 9.1 and 9.2 when new multiqueue = stuff >>>> was added? Jack, how should we proceed? >>>>=20 >>>>=20 >>>> Markus >>>>=20 >>>>=20 >>>>=20 >>>> On Thu, Feb 27, 2014 at 8:05 AM, Markus Gebert >>>> wrote: >>>>=20 >>>>>=20 >>>>> On 27.02.2014, at 02:00, Rick Macklem = wrote: >>>>>=20 >>>>>> John Baldwin wrote: >>>>>>> On Tuesday, February 25, 2014 2:19:01 am Johan Kooijman wrote: >>>>>>>> Hi all, >>>>>>>>=20 >>>>>>>> I have a weird situation here where I can't get my head around. >>>>>>>>=20 >>>>>>>> One FreeBSD 9.2-STABLE ZFS/NFS box, multiple Linux clients. = Once in >>>>>>>> a while >>>>>>>> the Linux clients loose their NFS connection: >>>>>>>>=20 >>>>>>>> Feb 25 06:24:09 hv3 kernel: nfs: server 10.0.24.1 not = responding, >>>>>>>> timed out >>>>>>>>=20 >>>>>>>> Not all boxes, just one out of the cluster. The weird part is = that >>>>>>>> when I >>>>>>>> try to ping a Linux client from the FreeBSD box, I have between = 10 >>>>>>>> and 30% >>>>>>>> packetloss - all day long, no specific timeframe. If I ping the >>>>>>>> Linux >>>>>>>> clients - no loss. If I ping back from the Linux clients to = FBSD >>>>>>>> box - no >>>>>>>> loss. >>>>>>>>=20 >>>>>>>> The errors I get when pinging a Linux client is this one: >>>>>>>> ping: sendto: File too large >>>>>=20 >>>>> We were facing similar problems when upgrading to 9.2 and have = stayed >>>> with >>>>> 9.1 on affected systems for now. We've seen this on HP G8 blades = with >>>>> 82599EB controllers: >>>>>=20 >>>>> ix0@pci0:4:0:0: class=3D0x020000 card=3D0x18d0103c chip=3D0x10f88086= >> rev=3D0x01 >>>>> hdr=3D0x00 >>>>> vendor =3D 'Intel Corporation' >>>>> device =3D '82599EB 10 Gigabit Dual Port Backplane = Connection' >>>>> class =3D network >>>>> subclass =3D ethernet >>>>>=20 >>>>> We didn't find a way to trigger the problem reliably. But when it >> occurs, >>>>> it usually affects only one interface. Symptoms include: >>>>>=20 >>>>> - socket functions return the 'File too large' error mentioned by = Johan >>>>> - socket functions return 'No buffer space' available >>>>> - heavy to full packet loss on the affected interface >>>>> - "stuck" TCP connection, i.e. ESTABLISHED TCP connections that = should >>>>> have timed out stick around forever (socket on the other side = could >> have >>>>> been closed ours ago) >>>>> - userland programs using the corresponding sockets usually got = stuck >> too >>>>> (can't find kernel traces right now, but always in network related >>>> syscalls) >>>>>=20 >>>>> Network is only lightly loaded on the affected systems (usually = 5-20 >>>> mbit, >>>>> capped at 200 mbit, per server), and netstat never showed any >> indication >>>> of >>>>> ressource shortage (like mbufs). >>>>>=20 >>>>> What made the problem go away temporariliy was to ifconfig down/up = the >>>>> affected interface. >>>>>=20 >>>>> We tested a 9.2 kernel with the 9.1 ixgbe driver, which was not = really >>>>> stable. Also, we tested a few revisions between 9.1 and 9.2 to = find out >>>>> when the problem started. Unfortunately, the ixgbe driver turned = out to >>>> be >>>>> mostly unstable on our systems between these releases, worse than = on >> 9.2. >>>>> The instability was introduced shortly after to 9.1 and fixed only = very >>>>> shortly before 9.2 release. So no luck there. We ended up using = 9.1 >> with >>>>> backports of 9.2 features we really need. >>>>>=20 >>>>> What we can't tell is wether it's the 9.2 kernel or the 9.2 ixgbe >> driver >>>>> or a combination of both that causes these problems. Unfortunately = we >> ran >>>>> out of time (and ideas). >>>>>=20 >>>>>=20 >>>>>>> EFBIG is sometimes used for drivers when a packet takes too many >>>>>>> scatter/gather entries. Since you mentioned NFS, one thing you = can >>>>>>> try is to >>>>>>> disable TSO on the intertface you are using for NFS to see if = that >>>>>>> "fixes" it. >>>>>>>=20 >>>>>> And please email if you try it and let us know if it helps. >>>>>>=20 >>>>>> I've think I've figured out how 64K NFS read replies can do this, >>>>>> but I'll admit "ping" is a mystery? (Doesn't it just send a = single >>>>>> packet that would be in a single mbuf?) >>>>>>=20 >>>>>> I think the EFBIG is replied by bus_dmamap_load_mbuf_sg(), but I >>>>>> don't know if it can happen for an mbuf chain with < 32 entries? >>>>>=20 >>>>> We don't use the nfs server on our systems, but they're >> (new)nfsclients. >>>>> So I don't think our problem is nfs related, unless the default >>>> rsize/wsize >>>>> for client mounts is not 8K, which I thought it was. Can you = confirm >>>> this, >>>>> Rick? >>>>>=20 >>>>> IIRC, disabling TSO did not make any difference in our case. >>>>>=20 >>>>>=20 >>>>> Markus >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 18:40:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7175A25F for ; Fri, 7 Mar 2014 18:40:13 +0000 (UTC) Received: from mail-wg0-x22a.google.com (mail-wg0-x22a.google.com [IPv6:2a00:1450:400c:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EEF28F7A for ; Fri, 7 Mar 2014 18:40:12 +0000 (UTC) Received: by mail-wg0-f42.google.com with SMTP id y10so5495234wgg.25 for ; Fri, 07 Mar 2014 10:40:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type:content-transfer-encoding; bh=4VcDNnxbkfN5dmovr58eFRGzIrF9OGQvKUYqkIII5a0=; b=f9qOAz1z0MPTydIkzpkjKfHFkLRgxl3gH6Q/iOimRNqT82KLfxkkZOaLCTI1DLF35J aPpP5Pnd+OK3aIFAoW+vnv9CdDHn4NdNe+pRA33iIDt7jYvEmFppwdeaAfZ2C4xx/ngv 2IyTMm+jjlt9Qb20HDaSClIV9v1HvqcghOjP7v+QLJ2FkylDk+Jh5w2InCgojGOEuuyq T2HeJ8y2qCHsdtXd9A4eBrtf4aXpCwY+x1Zd2K5nSjgeyApkqCSx7AQKrUqH9YCq2Td3 w0DjOWKlq93H0GUdD+25abzubk3gBLAoGPPK/MAuPphaqY+K88iLdpG28PCW+4BEnKW5 tgxA== X-Received: by 10.194.87.104 with SMTP id w8mr6253495wjz.90.1394217611421; Fri, 07 Mar 2014 10:40:11 -0800 (PST) Received: from srvbsdfenssv.interne.associated-bears.org (LCaen-151-92-21-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by mx.google.com with ESMTPSA id bj3sm13468932wjb.14.2014.03.07.10.40.10 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 07 Mar 2014 10:40:10 -0800 (PST) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id 25860CF240; Fri, 7 Mar 2014 19:40:09 +0100 (CET) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sxyfHRmGpq0S; Fri, 7 Mar 2014 19:40:07 +0100 (CET) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id C5635CF23C; Fri, 7 Mar 2014 19:40:07 +0100 (CET) From: Eric Masson To: Philipp Schmid Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated In-Reply-To: <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> (Philipp Schmid's message of "Fri, 7 Mar 2014 07:55:22 +0100") References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) X-Operating-System: FreeBSD 9.2-RELEASE-p3 amd64 Date: Fri, 07 Mar 2014 19:40:07 +0100 Message-ID: <86siqtluns.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: "John W. O'Brien" , Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 18:40:13 -0000 Philipp Schmid writes: Hi Philipp, > FreeBSD 10 seems to have problems with IPSec and filtering/nat. > Maybe your problem is related to: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=185876 I've rebuilt a kernel with the last patch available in the PR. It doesn't work (return nat rule in place). I think I'll try the following setup on gateway1 : - IIPTran https://www.ietf.org/rfc/rfc3884.txt (ipip tunnel in transport mode) - outside nat with pf on gif interface What bothers me is that ipfw reverse nat should work... Regards Éric Masson -- J'ai une dissert' en franįais : "Trouvez-vous regrettable que le camping sauvage soit interdit en France ?" Pouvez-vous m'aider, car je n'ai jamais campé !... -+- Laure in:- Youkaidi, youkaida -+- From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 18:48:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA325592 for ; Fri, 7 Mar 2014 18:48:16 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 83B6296 for ; Fri, 7 Mar 2014 18:48:16 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 763DBB9A3; Fri, 7 Mar 2014 13:48:15 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: LAN network performance issues Date: Fri, 7 Mar 2014 13:19:06 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403071319.06548.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Mar 2014 13:48:15 -0500 (EST) Cc: jcv X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 18:48:16 -0000 On Friday, March 07, 2014 12:17:05 am jcv wrote: > Hi - I am seeing some strange IPERF results.. Everything goes through my > WIFI/GIGABIT router. > > For these tests everything is plugged directly into the router via > Ethernet cable. > > My issue is the transfer rate from Windows to FreeBSD. > > There are 3 different computers in this lab running 3 different OS. > > Here are the results: > > > > FreeBSD as server: > > [vic@yeaguy ~] iperf -s > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 64.0 KByte (default) > ------------------------------------------------------------ > > > [ 4] local 192.168.1.3 port 5001 connected with 192.168.1.8 port 52505 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.1 sec 157 MBytes 131 Mbits/sec <----- WINDOWS 8.1 as > client on same LAN/ROUTER > > > > > [ 5] local 192.168.1.3 port 5001 connected with 192.168.1.12 port 60926 > [ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec <------ MACBOOK PRO as > client on same LAN/ROUTER > > > Windows as the server: > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 64.0 KByte (default) > ------------------------------------------------------------ > [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.3 port 60529 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.0 sec 1014 MBytes 850 Mbits/sec <--------- Freebsd 10 as > client on same LAN/ROUTER > > > > [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.12 port 60933 > [ 4] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec <------ MACBOOK PRO as > client on same LAN/ROUTER > > > > Macbook Pro as the server: > > [ 3] local 192.168.1.8 port 52509 connected with 192.168.1.12 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 823 MBytes 690 Mbits/sec <------ WINDOWS 8.1 as > client on same LAN/ROUTER > > [ 3] local 192.168.1.3 port 23190 connected with 192.168.1.12 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec <------ Freebsd 10 as > client on same LAN/ROUTER > > > With FreeBSD being the server, Windows transfer to FreeBSD is slow, > compared to Macbook to FreeBSD transfer.. > With Windows as the server, FreeBSD and Macbook to Windows transfer is > great. > With Macbook as server, Windows and FreeBSD transfer is good. > > The only bad transfer is Windows to FreeBSD. Windows transfer to Mac is > good. Cant really blame Windows for the poor transfer to FreeBSD then. > Macbook to FreeBSD is outstanding, cant really blame FreeBSD for poor > receive performance. Can you tell us more about the FreeBSD box such as the NIC being used? -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Mar 7 23:58:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08110FAC; Fri, 7 Mar 2014 23:58:03 +0000 (UTC) Received: from vms173019pub.verizon.net (vms173019pub.verizon.net [206.46.173.19]) by mx1.freebsd.org (Postfix) with ESMTP id D8B79E97; Fri, 7 Mar 2014 23:58:02 +0000 (UTC) Received: from [192.168.1.3] ([unknown] [173.60.122.163]) by vms173019.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N2300GRT93M0O90@vms173019.mailsrvcs.net>; Fri, 07 Mar 2014 16:57:27 -0600 (CST) Date: Fri, 07 Mar 2014 14:57:21 -0800 (PST) From: jcv To: John Baldwin Subject: Re: LAN network performance issues In-reply-to: <201403071319.06548.jhb@freebsd.org> Message-id: References: <201403071319.06548.jhb@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 23:58:03 -0000 On Fri, 7 Mar 2014, John Baldwin wrote: > On Friday, March 07, 2014 12:17:05 am jcv wrote: >> Hi - I am seeing some strange IPERF results.. Everything goes through my >> WIFI/GIGABIT router. >> >> For these tests everything is plugged directly into the router via >> Ethernet cable. >> >> My issue is the transfer rate from Windows to FreeBSD. >> >> There are 3 different computers in this lab running 3 different OS. >> >> Here are the results: >> >> >> >> FreeBSD as server: >> >> [vic@yeaguy ~] iperf -s >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 64.0 KByte (default) >> ------------------------------------------------------------ >> >> >> [ 4] local 192.168.1.3 port 5001 connected with 192.168.1.8 port 52505 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0-10.1 sec 157 MBytes 131 Mbits/sec <----- WINDOWS 8.1 as >> client on same LAN/ROUTER >> >> >> >> >> [ 5] local 192.168.1.3 port 5001 connected with 192.168.1.12 port 60926 >> [ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec <------ MACBOOK PRO as >> client on same LAN/ROUTER >> >> >> Windows as the server: >> >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> TCP window size: 64.0 KByte (default) >> ------------------------------------------------------------ >> [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.3 port 60529 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0-10.0 sec 1014 MBytes 850 Mbits/sec <--------- Freebsd 10 as >> client on same LAN/ROUTER >> >> >> >> [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.12 port 60933 >> [ 4] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec <------ MACBOOK PRO as >> client on same LAN/ROUTER >> >> >> >> Macbook Pro as the server: >> >> [ 3] local 192.168.1.8 port 52509 connected with 192.168.1.12 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 3] 0.0-10.0 sec 823 MBytes 690 Mbits/sec <------ WINDOWS 8.1 as >> client on same LAN/ROUTER >> >> [ 3] local 192.168.1.3 port 23190 connected with 192.168.1.12 port 5001 >> [ ID] Interval Transfer Bandwidth >> [ 3] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec <------ Freebsd 10 as >> client on same LAN/ROUTER >> >> >> With FreeBSD being the server, Windows transfer to FreeBSD is slow, >> compared to Macbook to FreeBSD transfer.. >> With Windows as the server, FreeBSD and Macbook to Windows transfer is >> great. >> With Macbook as server, Windows and FreeBSD transfer is good. >> >> The only bad transfer is Windows to FreeBSD. Windows transfer to Mac is >> good. Cant really blame Windows for the poor transfer to FreeBSD then. >> Macbook to FreeBSD is outstanding, cant really blame FreeBSD for poor >> receive performance. > > Can you tell us more about the FreeBSD box such as the NIC being used? > > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Sure John -- Here is the fbsd nic info: [vic@yeaguy ~] cat /var/run/dmesg.boot | grep re0 re0: port 0xe800-0xe8ff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff irq 18 at device 0.0 on pci3 re0: Using 1 MSI-X message re0: Chip rev. 0x48000000 re0: MAC rev. 0x00000000 miibus0: on re0 re0: Ethernet address: d8:50:e6:ba:c8:99 [vic@yeaguy ~] ifconfig re0: flags=8843 metric 0 mtu 1500 options=8209b ether d8:50:e6:ba:c8:99 inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::da50:e6ff:feba:c899%re0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 [vic@yeaguy ~] I tried to remove rxcsum and txcsum, but that didnt really improve the behavior.... I almost convinced its a iperf issue? maybe.. after iperf testing i did a FTP transfer and it exceeded what iperf is claiming the throughput is.. so im not sure what to make of it. From owner-freebsd-net@FreeBSD.ORG Sat Mar 8 00:09:56 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FCFA6FC for ; Sat, 8 Mar 2014 00:09:56 +0000 (UTC) Received: from hapkido.dreamhost.com (hapkido.dreamhost.com [66.33.216.122]) by mx1.freebsd.org (Postfix) with ESMTP id D9E2AF69 for ; Sat, 8 Mar 2014 00:09:53 +0000 (UTC) Received: from homiemail-a90.g.dreamhost.com (unknown [69.163.253.185]) by hapkido.dreamhost.com (Postfix) with ESMTP id CF4BF1836A for ; Fri, 7 Mar 2014 15:54:09 -0800 (PST) Received: from homiemail-a90.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a90.g.dreamhost.com (Postfix) with ESMTP id 43BCA2AC06A; Fri, 7 Mar 2014 16:09:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=saltant.com; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to: content-type; s=saltant.com; bh=qlJ/ZCUP+nvlw1hffbKXYl0OdpE=; b= hQY47Hea4SsHjsLU71qlToOawc7AeIli9+V8D1+U1pbhWophcWxdegpv6RpxsdED +E2E8CE4hPF2kuIFwesEzXMhgh7sedPSPLtsMjBoj5ld1OOf0pYDKL+5IZF+iIk0 sunNVi65rs6t+9BhUkzEe+evfcm1PU9oGeH95XniQ3g= Received: from dreck.saltant.net (dreck.saltant.net [72.78.188.150]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: john@saltant.com) by homiemail-a90.g.dreamhost.com (Postfix) with ESMTPSA id E70F72AC059; Fri, 7 Mar 2014 16:09:40 -0800 (PST) Message-ID: <531A5FBF.1000507@saltant.com> Date: Fri, 07 Mar 2014 19:09:35 -0500 From: "John W. O'Brien" Organization: Saltant Solutions User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Eric Masson , Philipp Schmid Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> <86siqtluns.fsf@srvbsdfenssv.interne.associated-bears.org> In-Reply-To: <86siqtluns.fsf@srvbsdfenssv.interne.associated-bears.org> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="cWGLsBurP5GE5HbDv99LDvLcEu5PJCB4p" Cc: Mailing List FreeBSD Network X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 00:09:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --cWGLsBurP5GE5HbDv99LDvLcEu5PJCB4p Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 3/7/14 1:40 PM, Eric Masson wrote: > Philipp Schmid writes: >=20 > Hi Philipp, >=20 >> FreeBSD 10 seems to have problems with IPSec and filtering/nat. >> Maybe your problem is related to: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D185876 >=20 > I've rebuilt a kernel with the last patch available in the PR. > It doesn't work (return nat rule in place). >=20 > I think I'll try the following setup on gateway1 : > - IIPTran https://www.ietf.org/rfc/rfc3884.txt (ipip tunnel in transpor= t > mode) > - outside nat with pf on gif interface >=20 > What bothers me is that ipfw reverse nat should work... I haven't done the mind meld with "reverse" yet. Could you comment on why you need to operate in a reversed NAT environment? What is it that's being reversed, and how does that apply to your use case? Regards, John --cWGLsBurP5GE5HbDv99LDvLcEu5PJCB4p Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBCgAGBQJTGl/FAAoJEBRzAKlhyP/F1xEIAMBRimMHSUueti5n8+Wl/7yb EdckI1x5W0+We4Egr/Syjq6vCpWitKpyVpv/M0Ud0+feOXJiCaOGY9LMtgcntINg 1W9OofYDI1VmLjvHi5VTtc5L/k108pa79wuBkZtRr7qD3QvgRTBZLe7PAea/C7h4 BJXrEBKgF14vr83emt/6dNC2mlYlwrgfPu5ZDftITQ3sjr+kjyJtoiLQHPESBC9B amW9P8EELBC+Sg75PdajaZcEigw8rtHnluTUF1FewnL2MgiAnLNxJT5GjavJH73W q9ZzFU35KtRZuPVWGSY5euhuUQ9vTIKejqeZVEERCj3FyVvAtwG+/RiXa6YwHGo= =t2dU -----END PGP SIGNATURE----- --cWGLsBurP5GE5HbDv99LDvLcEu5PJCB4p-- From owner-freebsd-net@FreeBSD.ORG Sat Mar 8 06:58:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 168D3C2B for ; Sat, 8 Mar 2014 06:58:55 +0000 (UTC) Received: from gator4068.hostgator.com (gator4068.hostgator.com [192.185.4.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E42AD1DC for ; Sat, 8 Mar 2014 06:58:54 +0000 (UTC) Received: from gesah2c by gator4068.hostgator.com with local (Exim 4.80.1) (envelope-from ) id 1WMBDh-0006zp-W6 for freebsd-net@freebsd.org; Sat, 08 Mar 2014 00:58:53 -0600 To: freebsd-net@freebsd.org Subject: =?windows-1256?B?z9rmySDj5CDVz+3e?= X-PHP-Script: www.gesah2.com/mail.php for 112.210.73.141 Date: Sat, 8 Mar 2014 00:58:53 -0600 From: April Joy Message-ID: <895db475b71d8745821ad489342e4657@www.gesah2.com> X-Priority: 3 X-Mailer: PHPMailer [version 1.73] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="windows-1256" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - gator4068.hostgator.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [32693 500] / [47 12] X-AntiAbuse: Sender Address Domain - gator4068.hostgator.com X-BWhitelist: no X-Source-IP: X-Source: /usr/bin/php X-Source-Args: /usr/bin/php /home1/gesah2c/public_html/mail.php X-Source-Dir: gesah2.com:/public_html X-Source-Sender: X-Source-Auth: gesah2c X-Email-Count: 251 X-Source-Cap: Z2VzYWgyYztnZXNhaDJjO2dhdG9yNDA2OC5ob3N0Z2F0b3IuY29t X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 06:58:55 -0000 Job Position Open.. Real Writing Jobs!! We are looking for some people that are interested in working from their home on a part or full-time basis. If you want to earn $100, $200 or even up to $500 a day, and you don't mind writing some short opinions up, this is the perfect opportunity for you! We work with hundreds of companies such as 20th Century Fox, Paramount Entertainment, Ford Motor Company, Google and more! We recruit people to fill 1000s of jobs for companies like this every year. Some of the positions that are available are: - Write Short Reviews Of Restaurants (Up To $150 per review) - Write Simple Blog Posts (Up to $30 per Blog Post) - Review Hollywood Movie Scripts (Up to $500 per Movie Script) - Write Short Articles About A Variety of Topics (Up to $200 per article) - Review Websites For Inappropriate Content (Up to $20 an hour) - Proofread Content For Mistakes (Up to $20 an hour) We are currently accepting new members. Sign Up Below. http://s.coop/joinnowdeme These companies are fighting for exposure on the internet and know the more people blogging about them, means the more exposure they are going to get, and ultimately the more money they are going to make. There has been an explosion in the need for online writers, regardless of skill. These companies are more interested in your honest genuine opinions when you're writing blog entries about their company... not if you are a very talented writer. If you're looking for work, or just want to make some part time money on the side, please come check out the jobs we have available. From owner-freebsd-net@FreeBSD.ORG Sat Mar 8 12:28:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0047671D for ; Sat, 8 Mar 2014 12:28:32 +0000 (UTC) Received: from mail-qg0-x243.google.com (mail-qg0-x243.google.com [IPv6:2607:f8b0:400d:c04::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B5103C77 for ; Sat, 8 Mar 2014 12:28:32 +0000 (UTC) Received: by mail-qg0-f67.google.com with SMTP id f51so2171169qge.2 for ; Sat, 08 Mar 2014 04:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=GSwyH1YpYtnYvEY9Q0+kdv1R297u/I4heYG933ABklA=; b=ATDuLFVNSoSh44+I9M4hgt++15O3MNjZeo2s/6uTslERi8H/4oteKpQU7XjfJXEcen +ixzRLkA5W23/3b1eOw0WaC7A8N5tuouSDhJVrOHiNxqeoqTUMrgnAQxLIc6jaSxM/Co AtF3KdhMzQ3SE4d8ppQEjSkyF9AAPB4lO4jwLoCNes9vhIlzG+Gac/zB3O1e95vXMKQo VEjYCVyNFxnb7iRRguNJ1SATXT+DwivKIE+E7kw83iVWbxCdLhkeEGnMG3EGMlHweFgm Bv+Z7GWqU9MjPWpb5EQEi8BEyTUlQVEZJbdzM6IV9gvHmMih93jJV5KLIACYPJGjjD14 aMfw== MIME-Version: 1.0 X-Received: by 10.224.43.200 with SMTP id x8mr20600866qae.43.1394281711688; Sat, 08 Mar 2014 04:28:31 -0800 (PST) Received: by 10.224.36.197 with HTTP; Sat, 8 Mar 2014 04:28:31 -0800 (PST) Date: Sat, 8 Mar 2014 13:28:31 +0100 Message-ID: Subject: lagg speed trouble From: vitaly rybalchenko To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 12:28:33 -0000 HI Any decision found? I have same problems with my server [root@VKTR123 /home/fish]# uname -a FreeBSD VKTR123.local 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50:31 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 [root@VKTR123 /home/fish]# ifconfig bge0: flags=8c43 metric 0 mtu 1500 options=c019b ether c8:1f:66:b9:00:8c inet6 fe80::ca1f:66ff:feb9:8c%bge0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active bge1: flags=8843 metric 0 mtu 1500 options=c019b ether c8:1f:66:b9:00:8c inet6 fe80::ca1f:66ff:feb9:8d%bge1 prefixlen 64 scopeid 0x2 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active bge2: flags=8843 metric 0 mtu 1500 options=c019b ether c8:1f:66:b9:00:8c inet6 fe80::ca1f:66ff:feb9:8a%bge2 prefixlen 64 scopeid 0x3 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active bge3: flags=8843 metric 0 mtu 1500 options=c019b ether c8:1f:66:b9:00:8c inet6 fe80::ca1f:66ff:feb9:8b%bge3 prefixlen 64 scopeid 0x4 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 lagg0: flags=8843 metric 0 mtu 1500 options=c019b ether c8:1f:66:b9:00:8c inet 109.69.56.95 netmask 0xfffffc00 broadcast 109.69.59.255 inet6 fe80::ca1f:66ff:feb9:8c%lagg0 prefixlen 64 scopeid 0x8 inet 109.69.58.65 netmask 0xffffffff broadcast 109.69.58.65 inet 109.69.58.41 netmask 0xffffffff broadcast 109.69.58.41 nd6 options=29 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: bge3 flags=1c laggport: bge2 flags=1c laggport: bge1 flags=1c laggport: bge0 flags=1c [root@VKTR123 /home/fish]# netstat -I lagg0 1 input (lagg0) output packets errs idrops bytes packets errs bytes colls 136519 0 0 16332341 98798 9106 212970406 0 144541 0 0 17706450 109218 0 233738869 0 133276 0 0 15775284 96646 4716 216895812 0 134745 0 0 16268020 101170 5175 215458117 0 134515 0 0 16332051 99530 8358 210910799 0 ^C From owner-freebsd-net@FreeBSD.ORG Sat Mar 8 17:56:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8C6DECF for ; Sat, 8 Mar 2014 17:56:50 +0000 (UTC) Received: from mail.shmtech.biz (unknown [IPv6:2001:41c8:10:8c::4:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E3AFA05 for ; Sat, 8 Mar 2014 17:56:50 +0000 (UTC) Received: from fleabag.domlan.talk2dom.com (188.28.195.85.threembb.co.uk [188.28.195.85]) (authenticated bits=0) by mail.shmtech.biz (8.14.8/8.14.5) with ESMTP id s28Huigk044500 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Sat, 8 Mar 2014 17:56:47 GMT (envelope-from misc+freebsd@talk2dom.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=shmtech1; t=1394301407; bh=D7FPV5+902wVSNhiJS20GD0jCsELVw4VZYDonJ1titc=; h=Date:From:To:Subject; b=rhumY5vMR2RD3S1/sg7IbE6ekQ2aI1YHOXXbgUpmfoWAnix6gCSUZYwHmTOzCs2IG +IUBr4LJqeZ1EN+D7q3uPS2s4ZxV+d2x19DYAwvXvOqhm1O1IEaVdjWdTCm1D69tFg E3rZdLmUZvaVSNMJNiPHD1B0Krq74TeieNPITQG0= X-Authentication-Warning: sendmail: Host 188.28.195.85.threembb.co.uk [188.28.195.85] claimed to be fleabag.domlan.talk2dom.com Message-ID: <531B59D6.40006@talk2dom.com> Date: Sat, 08 Mar 2014 17:56:38 +0000 From: Dom F User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: cas(4) PCI errors Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 17:56:51 -0000 Hello, I have a Sun quad gigaswift ethernet card in an ASUS K8V-SE-Deluxe motherboard. It's a 64-bit PCI card in a PCI v2.2 32-bit slot. It pretty much works fine except I see these errors in /var/log/messages: Mar 8 12:44:31 kernel: cas2: cas_eint: status 0x123c1010, PCI bus error 0x20 Mar 8 12:44:31 kernel: cas2: link state changed to DOWN Mar 8 12:44:33 kernel: cas2: link state changed to UP Mar 8 12:47:47 kernel: cas2: cas_eint: status 0x18541016, PCI bus error 0x20 Mar 8 12:47:47 kernel: cas2: link state changed to DOWN Mar 8 12:47:49 kernel: cas2: link state changed to UP Mar 8 12:50:02 kernel: cas2: cas_eint: status 0x9d41016, PCI bus error 0x20 Mar 8 12:50:02 kernel: cas2: link state changed to DOWN Mar 8 12:50:04 kernel: cas2: link state changed to UP Mar 8 12:50:17 kernel: cas2: cas_eint: status 0x128c1010, PCI bus error 0x20 Mar 8 12:50:17 kernel: cas2: link state changed to DOWN Mar 8 12:50:19 kernel: cas2: link state changed to UP Mar 8 12:59:45 kernel: cas2: cas_eint: status 0xf141010, PCI bus error 0x20 Mar 8 12:59:45 kernel: cas2: link state changed to DOWN Mar 8 12:59:47 kernel: cas2: link state changed to UP Mar 8 13:02:43 kernel: cas2: cas_eint: status 0x1f5c1016, PCI bus error 0x20 Mar 8 13:02:43 kernel: cas2: link state changed to DOWN Mar 8 13:02:45 kernel: cas2: link state changed to UP The 2-second ethernet renegotiation hurts the most. I have tried other replacement cards but the messages still pop up. Anyone else seen these? Currently running 9.1-RELEASE-p6. The "pciconf -lv" reports this: cas0@pci0:3:0:0: class=0x020000 card=0x00000000 chip=0x0035100b rev=0x30 hdr=0x00 vendor = 'National Semiconductor Corporation' device = 'DP83065 [Saturn] 10/100/1000 Ethernet Controller' class = network subclass = ethernet [ditto for cas1, cas2, cas3] Device attach logs from /var/log/messages: Mar 8 17:06:30 kernel: cas0: mem 0xfce00000-0xfcffffff irq 17 at device 0.0 on pci3 Mar 8 17:06:30 kernel: miibus0: on cas0 Mar 8 17:06:30 kernel: nsgphy0: PHY 1 on miibus0 Mar 8 17:06:30 kernel: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow Mar 8 17:06:30 kernel: cas0: 16kB RX FIFO, 9kB TX FIFO Mar 8 17:06:30 kernel: cas0: Ethernet address: 00:03:ba:db:ab:5f Mar 8 17:06:30 kernel: cas1: mem 0xfd600000-0xfd7fffff irq 18 at device 1.0 on pci3 Mar 8 17:06:30 kernel: miibus1: on cas1 Mar 8 17:06:30 kernel: nsgphy1: PHY 1 on miibus1 Mar 8 17:06:30 kernel: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow Mar 8 17:06:30 kernel: cas1: 16kB RX FIFO, 9kB TX FIFO Mar 8 17:06:30 kernel: cas1: Ethernet address: 00:03:ba:db:ab:60 Mar 8 17:06:30 kernel: cas2: mem 0xfd000000-0xfd1fffff irq 19 at device 2.0 on pci3 Mar 8 17:06:30 kernel: miibus2: on cas2 Mar 8 17:06:30 kernel: nsgphy2: PHY 1 on miibus2 Mar 8 17:06:30 kernel: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow Mar 8 17:06:30 kernel: cas2: 16kB RX FIFO, 9kB TX FIFO Mar 8 17:06:30 kernel: cas2: Ethernet address: 00:03:ba:db:ab:61 Mar 8 17:06:30 kernel: cas3: mem 0xfd400000-0xfd5fffff irq 16 at device 3.0 on pci3 Mar 8 17:06:30 kernel: miibus3: on cas3 Mar 8 17:06:30 kernel: nsgphy3: PHY 1 on miibus3 Mar 8 17:06:30 kernel: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow Mar 8 17:06:30 kernel: cas3: 16kB RX FIFO, 9kB TX FIFO Mar 8 17:06:30 kernel: cas3: Ethernet address: 00:03:ba:db:ab:62 Dom From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 11:33:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34FFE751 for ; Sun, 9 Mar 2014 11:33:25 +0000 (UTC) Received: from lagoon.freebsd.lublin.pl (lagoon.freebsd.lublin.pl [IPv6:2a02:2928:a::3]) by mx1.freebsd.org (Postfix) with ESMTP id E41CCE8C for ; Sun, 9 Mar 2014 11:33:24 +0000 (UTC) Received: from [IPv6:2a02:2928:a:ffff:91e7:ef13:bad7:28e4] (unknown [IPv6:2a02:2928:a:ffff:91e7:ef13:bad7:28e4]) by lagoon.freebsd.lublin.pl (Postfix) with ESMTPSA id 9897F479344; Sun, 9 Mar 2014 12:33:03 +0100 (CET) Message-ID: <531C517F.5020506@frasunek.com> Date: Sun, 09 Mar 2014 12:33:19 +0100 From: Przemyslaw Frasunek Organization: frasunek.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Mike Tancsa , Eugene Grosbein , "Bjoern A. Zeeb" , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> In-Reply-To: <53017B12.9050304@sentex.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 11:33:25 -0000 >> I've seen that Mike reported similar issues in October >> (http://lists.freebsd.org/pipermail/freebsd-stable/2013-October/075552.html). >> Did you managed to resolve it? > I worked around the crash by removing ipv6 from the kernel. The box has > been functioning without a crash since then. Hi, FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 enabled. From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 12:03:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53145CDF for ; Sun, 9 Mar 2014 12:03:09 +0000 (UTC) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1476414D for ; Sun, 9 Mar 2014 12:03:09 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.7/8.14.7) with ESMTP id s29C35hZ058623; Sun, 9 Mar 2014 08:03:05 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <531C5877.8050401@sentex.net> Date: Sun, 09 Mar 2014 08:03:03 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Przemyslaw Frasunek , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> <531C517F.5020506@frasunek.com> In-Reply-To: <531C517F.5020506@frasunek.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 12:03:09 -0000 On 3/9/2014 7:33 AM, Przemyslaw Frasunek wrote: >>> I've seen that Mike reported similar issues in October >>> (http://lists.freebsd.org/pipermail/freebsd-stable/2013-October/075552.html). >>> Did you managed to resolve it? >> I worked around the crash by removing ipv6 from the kernel. The box has >> been functioning without a crash since then. > > Hi, > > FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 > enabled. What sort of uptime have you seen with ipv6 enabled ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 15:38:22 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43068E90; Sun, 9 Mar 2014 15:38:22 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 150626E1; Sun, 9 Mar 2014 15:38:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s29FcLKa056034; Sun, 9 Mar 2014 15:38:21 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s29FcLXZ056033; Sun, 9 Mar 2014 15:38:21 GMT (envelope-from linimon) Date: Sun, 9 Mar 2014 15:38:21 GMT Message-Id: <201403091538.s29FcLXZ056033@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187341: [netinet] [patch] CARP addresses in backup state should't be used as source X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 15:38:22 -0000 Old Synopsis: CARP addresses in backup state should't be used as source New Synopsis: [netinet] [patch] CARP addresses in backup state should't be used as source Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 9 15:37:50 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=187341 From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 15:39:29 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6773F6F; Sun, 9 Mar 2014 15:39:29 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B291757; Sun, 9 Mar 2014 15:39:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s29FdTLE056186; Sun, 9 Mar 2014 15:39:29 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s29FdTrN056185; Sun, 9 Mar 2014 15:39:29 GMT (envelope-from linimon) Date: Sun, 9 Mar 2014 15:39:29 GMT Message-Id: <201403091539.s29FdTrN056185@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187194: Server hangs if -arp is present on an IP address X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 15:39:29 -0000 Synopsis: Server hangs if -arp is present on an IP address Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Mar 9 15:39:18 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=187194 From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 17:40:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D21D085D; Sun, 9 Mar 2014 17:40:56 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A1D5D197; Sun, 9 Mar 2014 17:40:56 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id v10so6138861pde.11 for ; Sun, 09 Mar 2014 10:40:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OTZhjfkSKcLvh7WTflPpqyj3yNiD2kpo0FzbdGXL/j0=; b=zfKQBi5tllSorQecUlOllSx9QxEedX7+dqfukb31N38vtU+/OKIg5Ube/E+bFW+Z7g OsQ9+E4qWfgo+OsDfHdQpCKWbeo/LsPZxLfWVaTD1PRjekXQUfD6QsyYj6hcBdZ82+Pm nVhVv1+TKmhFs54OW6bWyTOvHOhjNzd6QUqYjLQ3RGzx46flEAvApob2hLP2S3khC3RF eYDkrmqqeKMUtoihR3lrJo93I/SSr8L1waaR0lKtNydr+AQQEUSYorcpnlW0dMlv6W9K w5WO4rdvg2LhKcyRxvzjwQbC60xpmEzi/cnFbbwUTVvAEkfBtHfkRCd34WfV453tNvKY MHUQ== MIME-Version: 1.0 X-Received: by 10.67.8.102 with SMTP id dj6mr35288841pad.10.1394386856256; Sun, 09 Mar 2014 10:40:56 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.0.164 with HTTP; Sun, 9 Mar 2014 10:40:56 -0700 (PDT) In-Reply-To: References: <201403071319.06548.jhb@freebsd.org> Date: Sun, 9 Mar 2014 10:40:56 -0700 X-Google-Sender-Auth: 1OrYbn4Wtp6YCQXhZp0eQLgY6cE Message-ID: Subject: Re: LAN network performance issues From: Kevin Oberman To: jcv Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 17:40:56 -0000 On Fri, Mar 7, 2014 at 2:57 PM, jcv wrote: > > > On Fri, 7 Mar 2014, John Baldwin wrote: > > On Friday, March 07, 2014 12:17:05 am jcv wrote: >> >>> Hi - I am seeing some strange IPERF results.. Everything goes through my >>> WIFI/GIGABIT router. >>> >>> For these tests everything is plugged directly into the router via >>> Ethernet cable. >>> >>> My issue is the transfer rate from Windows to FreeBSD. >>> >>> There are 3 different computers in this lab running 3 different OS. >>> >>> Here are the results: >>> >>> >>> >>> FreeBSD as server: >>> >>> [vic@yeaguy ~] iperf -s >>> ------------------------------------------------------------ >>> Server listening on TCP port 5001 >>> TCP window size: 64.0 KByte (default) >>> ------------------------------------------------------------ >>> >>> >>> [ 4] local 192.168.1.3 port 5001 connected with 192.168.1.8 port 52505 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0-10.1 sec 157 MBytes 131 Mbits/sec <----- WINDOWS 8.1 as >>> client on same LAN/ROUTER >>> >>> >>> >>> >>> [ 5] local 192.168.1.3 port 5001 connected with 192.168.1.12 port 60926 >>> [ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec <------ MACBOOK PRO as >>> client on same LAN/ROUTER >>> >>> >>> Windows as the server: >>> >>> ------------------------------------------------------------ >>> Server listening on TCP port 5001 >>> TCP window size: 64.0 KByte (default) >>> ------------------------------------------------------------ >>> [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.3 port 60529 >>> [ ID] Interval Transfer Bandwidth >>> [ 4] 0.0-10.0 sec 1014 MBytes 850 Mbits/sec <--------- Freebsd 10 as >>> client on same LAN/ROUTER >>> >>> >>> >>> [ 4] local 192.168.1.8 port 5001 connected with 192.168.1.12 port 60933 >>> [ 4] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec <------ MACBOOK PRO as >>> client on same LAN/ROUTER >>> >>> >>> >>> Macbook Pro as the server: >>> >>> [ 3] local 192.168.1.8 port 52509 connected with 192.168.1.12 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 3] 0.0-10.0 sec 823 MBytes 690 Mbits/sec <------ WINDOWS 8.1 as >>> client on same LAN/ROUTER >>> >>> [ 3] local 192.168.1.3 port 23190 connected with 192.168.1.12 port 5001 >>> [ ID] Interval Transfer Bandwidth >>> [ 3] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec <------ Freebsd 10 as >>> client on same LAN/ROUTER >>> >>> >>> With FreeBSD being the server, Windows transfer to FreeBSD is slow, >>> compared to Macbook to FreeBSD transfer.. >>> With Windows as the server, FreeBSD and Macbook to Windows transfer is >>> great. >>> With Macbook as server, Windows and FreeBSD transfer is good. >>> >>> The only bad transfer is Windows to FreeBSD. Windows transfer to Mac is >>> good. Cant really blame Windows for the poor transfer to FreeBSD then. >>> Macbook to FreeBSD is outstanding, cant really blame FreeBSD for poor >>> receive performance. >>> >> >> Can you tell us more about the FreeBSD box such as the NIC being used? >> >> -- >> John Baldwin >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > Sure John -- > > Here is the fbsd nic info: > > [vic@yeaguy ~] cat /var/run/dmesg.boot | grep re0 > re0: port > 0xe800-0xe8ff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff irq 18 at > device 0.0 on pci3 > re0: Using 1 MSI-X message > re0: Chip rev. 0x48000000 > re0: MAC rev. 0x00000000 > miibus0: on re0 > re0: Ethernet address: d8:50:e6:ba:c8:99 > > > > [vic@yeaguy ~] ifconfig > re0: flags=8843 metric 0 mtu 1500 > > options=8209b HWCSUM,WOL_MAGIC,LINKSTATE> > ether d8:50:e6:ba:c8:99 > inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255 > inet6 fe80::da50:e6ff:feba:c899%re0 prefixlen 64 scopeid 0x1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > [vic@yeaguy ~] > > I tried to remove rxcsum and txcsum, but that didnt really improve the > behavior.... I almost convinced its a iperf issue? maybe.. after iperf > testing i did a FTP transfer and it exceeded what iperf is claiming the > throughput is.. so im not sure what to make of it. > You might try installing iperf3 and testing with that. iperf3 is a major rewrite of iperf and is totally incompatible with the older version, so you will need to install iperf3 on all systems I doubt iperf is the issue, but this is a way to check. -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 18:45:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33697D4E; Sun, 9 Mar 2014 18:45:06 +0000 (UTC) Received: from vms173019pub.verizon.net (vms173019pub.verizon.net [206.46.173.19]) by mx1.freebsd.org (Postfix) with ESMTP id 02DEA8DE; Sun, 9 Mar 2014 18:45:05 +0000 (UTC) Received: from yeaguy.com ([unknown] [173.60.122.163]) by vms173019.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N26002G4MQCZY30@vms173019.mailsrvcs.net>; Sun, 09 Mar 2014 13:44:41 -0500 (CDT) Received: from yeaguy.com (localhost [127.0.0.1]) by yeaguy.com (Postfix) with ESMTP id 01CD3112529; Sun, 09 Mar 2014 11:44:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at yeaguy.com Received: from yeaguy.com ([127.0.0.1]) by yeaguy.com (yeaguy.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oGga4nVMJuJu; Sun, 09 Mar 2014 11:44:34 -0700 (PDT) Received: from [192.168.1.8] (sabertooth.home [192.168.1.8]) by yeaguy.com (Postfix) with ESMTPSA id 3A016112524; Sun, 09 Mar 2014 11:44:34 -0700 (PDT) Message-id: <531CB691.8030001@yeaguy.com> Date: Sun, 09 Mar 2014 11:44:33 -0700 From: justin victoria User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-version: 1.0 To: Kevin Oberman Subject: Re: LAN network performance issues References: <201403071319.06548.jhb@freebsd.org> In-reply-to: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 18:45:06 -0000 On 3/9/2014 10:40 AM, Kevin Oberman wrote: > On Fri, Mar 7, 2014 at 2:57 PM, jcv > wrote: > > > > On Fri, 7 Mar 2014, John Baldwin wrote: > > On Friday, March 07, 2014 12:17:05 am jcv wrote: > > Hi - I am seeing some strange IPERF results.. Everything > goes through my > WIFI/GIGABIT router. > > For these tests everything is plugged directly into the > router via > Ethernet cable. > > My issue is the transfer rate from Windows to FreeBSD. > > There are 3 different computers in this lab running 3 > different OS. > > Here are the results: > > > > FreeBSD as server: > > [vic@yeaguy ~] iperf -s > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 64.0 KByte (default) > ------------------------------------------------------------ > > > [ 4] local 192.168.1.3 port 5001 connected with > 192.168.1.8 port 52505 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.1 sec 157 MBytes 131 Mbits/sec <----- > WINDOWS 8.1 as > client on same LAN/ROUTER > > > > > [ 5] local 192.168.1.3 port 5001 connected with > 192.168.1.12 port 60926 > [ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec <------ > MACBOOK PRO as > client on same LAN/ROUTER > > > Windows as the server: > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 64.0 KByte (default) > ------------------------------------------------------------ > [ 4] local 192.168.1.8 port 5001 connected with > 192.168.1.3 port 60529 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.0 sec 1014 MBytes 850 Mbits/sec > <--------- Freebsd 10 as > client on same LAN/ROUTER > > > > [ 4] local 192.168.1.8 port 5001 connected with > 192.168.1.12 port 60933 > [ 4] 0.0-10.0 sec 1.08 GBytes 931 Mbits/sec <------ > MACBOOK PRO as > client on same LAN/ROUTER > > > > Macbook Pro as the server: > > [ 3] local 192.168.1.8 port 52509 connected with > 192.168.1.12 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 823 MBytes 690 Mbits/sec <------ > WINDOWS 8.1 as > client on same LAN/ROUTER > > [ 3] local 192.168.1.3 port 23190 connected with > 192.168.1.12 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec <------ > Freebsd 10 as > client on same LAN/ROUTER > > > With FreeBSD being the server, Windows transfer to FreeBSD > is slow, > compared to Macbook to FreeBSD transfer.. > With Windows as the server, FreeBSD and Macbook to Windows > transfer is > great. > With Macbook as server, Windows and FreeBSD transfer is good. > > The only bad transfer is Windows to FreeBSD. Windows > transfer to Mac is > good. Cant really blame Windows for the poor transfer to > FreeBSD then. > Macbook to FreeBSD is outstanding, cant really blame > FreeBSD for poor > receive performance. > > > Can you tell us more about the FreeBSD box such as the NIC > being used? > > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org > mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org > " > > > > Sure John -- > > Here is the fbsd nic info: > > [vic@yeaguy ~] cat /var/run/dmesg.boot | grep re0 > re0: > port 0xe800-0xe8ff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff > irq 18 at device 0.0 on pci3 > re0: Using 1 MSI-X message > re0: Chip rev. 0x48000000 > re0: MAC rev. 0x00000000 > miibus0: on re0 > re0: Ethernet address: d8:50:e6:ba:c8:99 > > > > [vic@yeaguy ~] ifconfig > re0: flags=8843 metric 0 > mtu 1500 > > options=8209b > ether d8:50:e6:ba:c8:99 > inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255 > inet6 fe80::da50:e6ff:feba:c899%re0 prefixlen 64 scopeid 0x1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > [vic@yeaguy ~] > > I tried to remove rxcsum and txcsum, but that didnt really improve > the behavior.... I almost convinced its a iperf issue? maybe.. > after iperf testing i did a FTP transfer and it exceeded what > iperf is claiming the throughput is.. so im not sure what to make > of it. > > > You might try installing iperf3 and testing with that. iperf3 is a > major rewrite of iperf and is totally incompatible with the older > version, so you will need to install iperf3 on all systems > > I doubt iperf is the issue, but this is a way to check. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com iperf3 on windows isnt playing nice.. From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 19:37:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8E7DD0C for ; Sun, 9 Mar 2014 19:37:00 +0000 (UTC) Received: from mail-oa0-x22d.google.com (mail-oa0-x22d.google.com [IPv6:2607:f8b0:4003:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B3E59CBA for ; Sun, 9 Mar 2014 19:37:00 +0000 (UTC) Received: by mail-oa0-f45.google.com with SMTP id o6so6184254oag.18 for ; Sun, 09 Mar 2014 12:37:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=BcJu7jsZ7OPI6SqQnyR7p7tJjMdishSTDqCQCDiHiik=; b=Jul6rcfcYpLMKT52DxCzuK6ER6g+bwoqA/luSB3EY9FXkbqrSx/wOMOk7DoyYdONo6 raIKGRxUjuk2JJgu4GYCaBwBBDj6HP/Mag02zvHXzNm0UItA1IkQRZ7Tr5d4qTItLEwB 3V8rMQ4KoMVFt4PomBMusGVJ5pp+THjCS/agTQaudV/L5BbcQ4egstOV387UgZ8LlIYA FrO54HzjcV14/umj7UaSEJrxOBi1PuRSaQ0KxK/WF6ZTe5f6Lh5RHVrhfXcXXgCp52O0 WQ+ecquN4eYmeD2iH+ZbguiWjzvE9qN83U1fc/na5IZZ08QYbyBcmEr0Iz3InDiMiUzZ j7BQ== MIME-Version: 1.0 X-Received: by 10.182.47.100 with SMTP id c4mr25668836obn.38.1394393820098; Sun, 09 Mar 2014 12:37:00 -0700 (PDT) Received: by 10.182.76.201 with HTTP; Sun, 9 Mar 2014 12:36:59 -0700 (PDT) Date: Sun, 9 Mar 2014 15:36:59 -0400 Message-ID: Subject: Using pf.conf with public access points. From: Joe Nosay To: freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary=14dae93998a3530ad104f4319c78 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 19:37:01 -0000 --14dae93998a3530ad104f4319c78 Content-Type: text/plain; charset=ISO-8859-1 My pf.conf is attached. I would like to know: 1. Is everything set up properly? 2. How do I compensate for the use of public access points when the IP addresses will always be different? --14dae93998a3530ad104f4319c78 Content-Type: application/octet-stream; name="pf.conf" Content-Disposition: attachment; filename="pf.conf" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hskyn1wa0 ICAgIGV4dF9pZj0id2xhbjAiCiAgICBqYWlsX2lmPSJsbzEiCgogICAgCiAgICBJUF9KQUlMX1dX Vz0iMTI3LjEuMi43IgoKICAgIE5FVF9KQUlMPSIxMjcuMS4yLjcvMzIiCgogICAgUE9SVF9XV1c9 Ins4MH0iCgogICAgc2NydWIgaW4gYWxsCgogICAgIyBuYXQgYWxsIGphaWwgdHJhZmZpYwogICAg bmF0IHBhc3Mgb24gJGV4dF9pZiBmcm9tICRORVRfSkFJTCB0byBhbnkgLT4gJElQX1BVQgoKICAg ICMgV1dXCiAgICByZHIgcGFzcyBvbiAkZXh0X2lmIHByb3RvIHRjcCBmcm9tIGFueSB0byAkSVBf UFVCIHBvcnQgJFBPUlRfV1dXIC0+ICRJUF9KQUlMX1dXVwoKICAgICMgZGVtbyBvbmx5LCBwYXNz aW5nIGFsbCB0cmFmZmljCiAgICBwYXNzIG91dAogICAgcGFzcyBpbgo= --14dae93998a3530ad104f4319c78-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 19:40:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1551FFE0; Sun, 9 Mar 2014 19:40:52 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D8FA4D62; Sun, 9 Mar 2014 19:40:51 +0000 (UTC) Received: by mail-pd0-f173.google.com with SMTP id z10so6181247pdj.4 for ; Sun, 09 Mar 2014 12:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=23NQr9vnaasgCoa1lavZsgu1Vwk+CDoEFKh2FSbXXnw=; b=wUoIqHvivwaiyKOTVyPgjb2Cc52kn+435antSuMqKfGOBm9UG2rv0EB4538cU8/En8 Gcstm1uV31rNUxED8+Vmi7b8hdPJS888xKFSQZqp/xOwoKYDL4DezkWjSHL5AfFDnUKr 9ByjgxInxJ+fmSwn3ZS2B0qI7C0Ow75IL4noxI/+qrSKqo5r5AE4/ISjvz+fyNvUqxEp D+js+15vLD8DyQWBKBw030BCm9Zn1mLVs3IbK8xmv3vXP7LlGdkH14C+Ov+bh/nupGV+ iK9MHDrRl7yWWK7/NdDmFi8Z7ato3mRql2nckE8II5HZKXF9dzuY18bTfvtxc0Ub2l/W 6O/w== MIME-Version: 1.0 X-Received: by 10.67.8.102 with SMTP id dj6mr35757154pad.10.1394394051541; Sun, 09 Mar 2014 12:40:51 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.0.164 with HTTP; Sun, 9 Mar 2014 12:40:51 -0700 (PDT) In-Reply-To: <531CB691.8030001@yeaguy.com> References: <201403071319.06548.jhb@freebsd.org> <531CB691.8030001@yeaguy.com> Date: Sun, 9 Mar 2014 12:40:51 -0700 X-Google-Sender-Auth: X6c_734-WxhTsJc6fUuiZNC0Zmw Message-ID: Subject: Re: LAN network performance issues From: Kevin Oberman To: justin victoria Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 19:40:52 -0000 On Sun, Mar 9, 2014 at 11:44 AM, justin victoria wrote: > > On 3/9/2014 10:40 AM, Kevin Oberman wrote: > > You might try installing iperf3 and testing with that. iperf3 is a major > rewrite of iperf and is totally incompatible with the older version, so > you will need to install iperf3 on all systems > > I doubt iperf is the issue, but this is a way to check. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com > > > iperf3 on windows isnt playing nice.. > Can you provide more details on the problem on Windows? I know the primary developer of iperf3 (who also maintains iperf2) and would like to be sure he is aware of any and all problems on any platform. He's busy, but iperf3 is part of his day job, so I expect it will be looked at fairly quickly. -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 20:36:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6BDDC98; Sun, 9 Mar 2014 20:36:16 +0000 (UTC) Received: from vms173011pub.verizon.net (vms173011pub.verizon.net [206.46.173.11]) by mx1.freebsd.org (Postfix) with ESMTP id 9685B1D4; Sun, 9 Mar 2014 20:36:16 +0000 (UTC) Received: from yeaguy.com ([unknown] [173.60.122.163]) by vms173011.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N2600MCIRVIODY0@vms173011.mailsrvcs.net>; Sun, 09 Mar 2014 15:35:43 -0500 (CDT) Received: from yeaguy.com (localhost [127.0.0.1]) by yeaguy.com (Postfix) with ESMTP id 91E6F112525; Sun, 09 Mar 2014 13:35:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at yeaguy.com Received: from yeaguy.com ([127.0.0.1]) by yeaguy.com (yeaguy.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ozSVYU-Z53Ex; Sun, 09 Mar 2014 13:35:38 -0700 (PDT) Received: from [192.168.1.8] (sabertooth.home [192.168.1.8]) by yeaguy.com (Postfix) with ESMTPSA id 43DB1112524; Sun, 09 Mar 2014 13:35:38 -0700 (PDT) Message-id: <531CD099.50904@yeaguy.com> Date: Sun, 09 Mar 2014 13:35:37 -0700 From: justin victoria User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-version: 1.0 To: Kevin Oberman Subject: Re: LAN network performance issues References: <201403071319.06548.jhb@freebsd.org> <531CB691.8030001@yeaguy.com> In-reply-to: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 20:36:16 -0000 On 3/9/2014 12:40 PM, Kevin Oberman wrote: > On Sun, Mar 9, 2014 at 11:44 AM, justin victoria > wrote: > > > On 3/9/2014 10:40 AM, Kevin Oberman wrote: >> You might try installing iperf3 and testing with that. iperf3 is >> a major rewrite of iperf and is totally incompatible with the >> older version, so you will need to install iperf3 on all systems >> >> I doubt iperf is the issue, but this is a way to check. >> -- >> R. Kevin Oberman, Network Engineer, Retired >> E-mail: rkoberman@gmail.com > > iperf3 on windows isnt playing nice.. > > > Can you provide more details on the problem on Windows? I know the > primary developer of iperf3 (who also maintains iperf2) and would like > to be sure he is aware of any and all problems on any platform. He's > busy, but iperf3 is part of his day job, so I expect it will be looked > at fairly quickly. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com I keep getting this error: justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ ./iperf3.exe -c 192.168.1.3 2 [main] iperf3 3116 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer. Please report this problem to the public mailing list cygwin@cygwin.com Then when im the server: error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use and netstat on win: ustin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ netstat Active Connections Proto Local Address Foreign Address State TCP 127.0.0.1:5080 sabertooth:49221 ESTABLISHED TCP 127.0.0.1:49221 sabertooth:5080 ESTABLISHED TCP 127.0.0.1:49637 sabertooth:49638 ESTABLISHED TCP 127.0.0.1:49638 sabertooth:49637 ESTABLISHED TCP 192.168.1.8:49335 lax04s09-in-f22:https ESTABLISHED TCP 192.168.1.8:49639 yg:imap ESTABLISHED TCP 192.168.1.8:49641 yg:imap ESTABLISHED TCP 192.168.1.8:49648 yg:imap ESTABLISHED TCP 192.168.1.8:51457 yg:ssh ESTABLISHED TCP 192.168.1.8:51637 yg:ssh ESTABLISHED TCP 192.168.1.8:52390 bn1wns1011611:https ESTABLISHED TCP 192.168.1.8:52643 lax04s09-in-f1:https ESTABLISHED TCP 192.168.1.8:52647 yg:imap ESTABLISHED TCP 192.168.1.8:52649 yg:imap ESTABLISHED TCP 192.168.1.8:52741 103.10.4.40:ftp ESTABLISHED TCP 192.168.1.8:60435 176.32.96.103:http ESTABLISHED TCP 192.168.1.8:61874 yg:microsoft-ds ESTABLISHED TCP 192.168.1.8:63656 yg:ssh ESTABLISHED justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 20:38:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6C4ED3B; Sun, 9 Mar 2014 20:38:18 +0000 (UTC) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80F8E1E7; Sun, 9 Mar 2014 20:38:18 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id wp4so6117348obc.40 for ; Sun, 09 Mar 2014 13:38:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=txYTgEvfdl8HJi+rEGkvhy/TxdFYjdSOSHU8ERL79T4=; b=fwNr2htogjvGZnM5Ipp+cBphFGs6fkpAt9O/h3BlOSk/NVVinpKpY1+OHEGr+diONs itXhIULaOKonaWrcUwyXyaJscqir3bfwA6hCip5lnZGRL6YfxeLbNcTtjeIkpL0BkxiB TuCsPUjyV6kti81obFpqBHmxU/lIM8jhQIoZEKrBb6S+lbXC8jLtcpvjrNSfGTh57PSc iy6vYnUZ9qW2efVYmxk3SKMCPyxhIsBMbq/YXqu8Mm3kCmAy7YA+afPS0ZdQRbVVJkfW Wp2zNnflbdG9xp6b1qwRXW+j1BU9ZO13F+5KSr32G4IxpbdqmRMb+3f9+uos/O/qDKpj yuWQ== MIME-Version: 1.0 X-Received: by 10.182.43.161 with SMTP id x1mr25285615obl.5.1394397497916; Sun, 09 Mar 2014 13:38:17 -0700 (PDT) Received: by 10.182.232.166 with HTTP; Sun, 9 Mar 2014 13:38:17 -0700 (PDT) In-Reply-To: <531CD099.50904@yeaguy.com> References: <201403071319.06548.jhb@freebsd.org> <531CB691.8030001@yeaguy.com> <531CD099.50904@yeaguy.com> Date: Sun, 9 Mar 2014 13:38:17 -0700 Message-ID: Subject: Re: LAN network performance issues From: Nick Chevsky To: justin victoria Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Kevin Oberman , John Baldwin , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 20:38:18 -0000 On Sun, Mar 9, 2014 at 1:35 PM, justin victoria wrote: > > I keep getting this error: > > justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 > $ ./iperf3.exe -c 192.168.1.3 > 2 [main] iperf3 3116 find_fast_cwd: WARNING: Couldn't compute > FAST_CWD pointer. Please report this problem to > the public mailing list cygwin@cygwin.com You need to upgrade your cygwin1.dll to fix this problem on Windows 8+. -- Nick Chevsky From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 20:50:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA391FBA for ; Sun, 9 Mar 2014 20:50:17 +0000 (UTC) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) by mx1.freebsd.org (Postfix) with ESMTP id AF03B2D3 for ; Sun, 9 Mar 2014 20:50:17 +0000 (UTC) Received: from yeaguy.com ([unknown] [173.60.122.163]) by vms173005.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N26002XAPQ4G680@vms173005.mailsrvcs.net> for freebsd-net@freebsd.org; Sun, 09 Mar 2014 14:49:20 -0500 (CDT) Received: from yeaguy.com (localhost [127.0.0.1]) by yeaguy.com (Postfix) with ESMTP id E7006112525 for ; Sun, 09 Mar 2014 12:49:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at yeaguy.com Received: from yeaguy.com ([127.0.0.1]) by yeaguy.com (yeaguy.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nA3Jj4awDiWI for ; Sun, 09 Mar 2014 12:49:11 -0700 (PDT) Received: from [192.168.1.8] (sabertooth.home [192.168.1.8]) by yeaguy.com (Postfix) with ESMTPSA id A7070112524 for ; Sun, 09 Mar 2014 12:49:11 -0700 (PDT) Message-id: <531CC5B7.2000704@yeaguy.com> Date: Sun, 09 Mar 2014 12:49:11 -0700 From: justin victoria User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-version: 1.0 To: freebsd-net@freebsd.org Subject: Re: LAN network performance issues References: <201403071319.06548.jhb@freebsd.org> <531CB691.8030001@yeaguy.com> In-reply-to: Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 20:50:18 -0000 On 3/9/2014 12:40 PM, Kevin Oberman wrote: > On Sun, Mar 9, 2014 at 11:44 AM, justin victoria wrote: > >> On 3/9/2014 10:40 AM, Kevin Oberman wrote: >> >> You might try installing iperf3 and testing with that. iperf3 is a major >> rewrite of iperf and is totally incompatible with the older version, so >> you will need to install iperf3 on all systems >> >> I doubt iperf is the issue, but this is a way to check. >> -- >> R. Kevin Oberman, Network Engineer, Retired >> E-mail: rkoberman@gmail.com >> >> >> iperf3 on windows isnt playing nice.. >> > Can you provide more details on the problem on Windows? I know the primary > developer of iperf3 (who also maintains iperf2) and would like to be sure > he is aware of any and all problems on any platform. He's busy, but iperf3 > is part of his day job, so I expect it will be looked at fairly quickly. I keep getting this error: justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ ./iperf3.exe -c 192.168.1.3 2 [main] iperf3 3116 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer. Please report this problem to the public mailing list cygwin@cygwin.com Then when im the server: error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use error: Unable to start listener for connections: Address already in use and netstat on win: ustin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ netstat Active Connections Proto Local Address Foreign Address State TCP 127.0.0.1:5080 sabertooth:49221 ESTABLISHED TCP 127.0.0.1:49221 sabertooth:5080 ESTABLISHED TCP 127.0.0.1:49637 sabertooth:49638 ESTABLISHED TCP 127.0.0.1:49638 sabertooth:49637 ESTABLISHED TCP 192.168.1.8:49335 lax04s09-in-f22:https ESTABLISHED TCP 192.168.1.8:49639 yg:imap ESTABLISHED TCP 192.168.1.8:49641 yg:imap ESTABLISHED TCP 192.168.1.8:49648 yg:imap ESTABLISHED TCP 192.168.1.8:51457 yg:ssh ESTABLISHED TCP 192.168.1.8:51637 yg:ssh ESTABLISHED TCP 192.168.1.8:52390 bn1wns1011611:https ESTABLISHED TCP 192.168.1.8:52643 lax04s09-in-f1:https ESTABLISHED TCP 192.168.1.8:52647 yg:imap ESTABLISHED TCP 192.168.1.8:52649 yg:imap ESTABLISHED TCP 192.168.1.8:52741 103.10.4.40:ftp ESTABLISHED TCP 192.168.1.8:60435 176.32.96.103:http ESTABLISHED TCP 192.168.1.8:61874 yg:microsoft-ds ESTABLISHED TCP 192.168.1.8:63656 yg:ssh ESTABLISHED justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 21:53:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A989746 for ; Sun, 9 Mar 2014 21:53:10 +0000 (UTC) Received: from vms173019pub.verizon.net (vms173019pub.verizon.net [206.46.173.19]) by mx1.freebsd.org (Postfix) with ESMTP id 4A8A4A39 for ; Sun, 9 Mar 2014 21:53:09 +0000 (UTC) Received: from yeaguy.com ([unknown] [173.60.122.163]) by vms173019.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N26001DTVG1U640@vms173019.mailsrvcs.net> for freebsd-net@freebsd.org; Sun, 09 Mar 2014 16:52:54 -0500 (CDT) Received: from yeaguy.com (localhost [127.0.0.1]) by yeaguy.com (Postfix) with ESMTP id 5BE99112525 for ; Sun, 09 Mar 2014 14:52:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at yeaguy.com Received: from yeaguy.com ([127.0.0.1]) by yeaguy.com (yeaguy.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eFBHTiCPIa3X for ; Sun, 09 Mar 2014 14:52:48 -0700 (PDT) Received: from [192.168.1.8] (sabertooth.home [192.168.1.8]) by yeaguy.com (Postfix) with ESMTPSA id 3C511112524 for ; Sun, 09 Mar 2014 14:52:48 -0700 (PDT) Message-id: <531CE2AE.6000408@yeaguy.com> Date: Sun, 09 Mar 2014 14:52:46 -0700 From: justin victoria User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-version: 1.0 To: freebsd-net@freebsd.org Subject: Re: LAN network performance issues References: <201403071319.06548.jhb@freebsd.org> <531CB691.8030001@yeaguy.com> <531CD099.50904@yeaguy.com> In-reply-to: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 21:53:10 -0000 On 3/9/2014 1:38 PM, Nick Chevsky wrote: > On Sun, Mar 9, 2014 at 1:35 PM, justin victoria wrote: > >> I keep getting this error: >> >> justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 >> $ ./iperf3.exe -c 192.168.1.3 >> 2 [main] iperf3 3116 find_fast_cwd: WARNING: Couldn't compute >> FAST_CWD pointer. Please report this problem to >> the public mailing list cygwin@cygwin.com > > You need to upgrade your cygwin1.dll to fix this problem on Windows 8+. > > -- > Nick Chevsky > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" I just completely removed the old cygwin directory, downloaded a new copy of it and installed.. still seeing: justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ ./iperf3.exe -s 1 [main] iperf3 3380 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer. Please report this problem to the public mailing list cygwin@cygwin.com _Warning: There are multiple cygwin1.dlls on your path _ (not sure about this issue tho) base-cygwin 3.3-1 OK cygwin 1.7.28-2 OK cygwin-debuginfo 1.7.28-2 OK justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ echo $PATH /usr/local/bin:/usr/bin:/cygdrive/c/Program Files (x86)/AMD APP/bin/x86_64:/cygdrive/c/Program Files (x86)/AMD APP/bin/x86:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/cygdrive/c/Program Files (x86)/ATI Technologies/ATI.ACE/Core-Static justin@sabertooth /cygdrive/c/iperf3-3.0b4-cygwin-v2 $ From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 22:05:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C10DCD3 for ; Sun, 9 Mar 2014 22:05:07 +0000 (UTC) Received: from mail-ig0-x22a.google.com (mail-ig0-x22a.google.com [IPv6:2607:f8b0:4001:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDCE2B4F for ; Sun, 9 Mar 2014 22:05:06 +0000 (UTC) Received: by mail-ig0-f170.google.com with SMTP id uq10so7087354igb.1 for ; Sun, 09 Mar 2014 15:05:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=pS7B1nH7fKrRUDPAgkSpwTOK2To7Bg8Uq3aOJmVMSiE=; b=gOC9DXZiLhlnOGpg9vtuZgH6X7ML961krCusXT+ZbkClJ85R0CeD9lhGkj/pjoVvfA N9MXAR4jqw/q8aht8b5skyt9UDkHtVklBCIE+lFjiNVM4JtgaFesyYaSv5alc49Z/oEQ DHPlRsYUX1MNO+Bf2sL5vkaR7VaOPRtlvPuFc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=pS7B1nH7fKrRUDPAgkSpwTOK2To7Bg8Uq3aOJmVMSiE=; b=Q7pUN5R/x02O1TNoZT2jAvv7eSwo3Tso0TdCDcXpMygExkIE3GZO130Se1Hxl4V2MH gznboOx3UqctIBDDWd1qFsiyL3xWGjODY9y9f33AwoZzU0OSvHJHobISLMfjG42t+DaN JNcQCwOzrz0XH35xVVFDF/tQZYtN7qfg4py91uRP4Dyhq2/+VCjEPMsz57GsCPwxC1Go bG4QBT/q6mUUife6qBnker9U8FYbnC+eJ+6lkR1LwkC7M7CMrZFcCML6qmGFAch8tD7L mYxr8JUIlZoLBEY30OQxcyxbz6nagpBn6/C9fIyLdqelAizz7QSPqTjVK3sDJymEV3HP I5ew== X-Gm-Message-State: ALoCoQmvdMuweJ2YOh+K1UbBVvXeCE4x3pq8quq7VQeeH2DBF8nO43YhblddqBLBIPNmIgU6WzIG X-Received: by 10.43.146.69 with SMTP id jx5mr2663739icc.42.1394402706139; Sun, 09 Mar 2014 15:05:06 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id w9sm29122784iga.10.2014.03.09.15.05.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 09 Mar 2014 15:05:04 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-44141998-6346-42B3-A00B-1848B44C3146; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Using pf.conf with public access points. Date: Sun, 9 Mar 2014 18:05:02 -0400 To: Joe Nosay X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 22:05:07 -0000 --Apple-Mail-44141998-6346-42B3-A00B-1848B44C3146 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Sorry bit there is not enough information here. How can anyone know what you= are trying to accomplish by any of this. --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On Mar 9, 2014, at 15:36, Joe Nosay wrote: >=20 > My pf.conf is attached. I would like to know: > 1. Is everything set up properly? > 2. How do I compensate for the use of public access points when the IP > addresses will always be different? > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --Apple-Mail-44141998-6346-42B3-A00B-1848B44C3146 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDMwOTIyMDUwM1owIwYJKoZIhvcN AQkEMRYEFHUUONWvVvNOLPoNXxHgBWPenaS0MIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAIujPVECUTzwq6Dr/WYMx ghtLlpKrKoEkwxvdCP/DqY+wXiIo/l4guTHQZLdozBL4dNIGqTecEusBtipN8BZ2R5WFYg+w4Av7 aw3qh+y5re9mi6hzQ/S9D+yIHkrjMB4WLsipMXR1teXpYsfeLY6n1fyRtl1X//OfagE34RWEh0cv Zivvp7K65yxRihEENMvnDSUxO1WwvnXn2BR/+tAmeHnEZ+qbfFVa4FjGNC0iFOpMjH8/XBZ7+Uro 0uCEXQlRqBVApDDbUSU/qKiXqj6LDG1mDq7g611HloX1tGkAq1uUJRFqqcbJprof/JteFIGl/flC 2b+WDodbLq2/Q55jugAAAAAAAA== --Apple-Mail-44141998-6346-42B3-A00B-1848B44C3146-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 23:06:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9ECD3428 for ; Sun, 9 Mar 2014 23:06:51 +0000 (UTC) Received: from mail-oa0-x235.google.com (mail-oa0-x235.google.com [IPv6:2607:f8b0:4003:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 66F6CBD for ; Sun, 9 Mar 2014 23:06:51 +0000 (UTC) Received: by mail-oa0-f53.google.com with SMTP id j17so6336787oag.12 for ; Sun, 09 Mar 2014 16:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1iEJwlPXaeDGMOq3yanjzZ7BW95Yg5upszh4FOFgAQU=; b=G+TGsWXgY/IV2ONEJ6XgUshaL6l8pnSxMPKw7WNEUz2vEWJbje4G7B8tIKw0pdTqnG mZL/jwLtlYm8bqJi1uSwWpK1Fijsx2NwEwGKnK2K8qSsOWfCzylAPpZaLtQAQRJgif9S womsOMbLWQBtsvyGYQ0k3YvOLECPqChEUpYhsFikmxcFg9ZfYg+Z3VexRX9ay/RPXkv8 RjXVNdUao31Ii55a3Lb8Nk2sJ6C8s6W2lbkTPfzs/lmgTOQjfLnMAKyqlC0bBT7vqnE5 27I3QXgcykqpkdpFGhyRlJxyOBgTkkiNUvoxOiihIp+ztf3MyEF3X/VCnlKrltUoPmiq xoFw== MIME-Version: 1.0 X-Received: by 10.60.116.74 with SMTP id ju10mr21388100oeb.6.1394406410706; Sun, 09 Mar 2014 16:06:50 -0700 (PDT) Received: by 10.182.76.201 with HTTP; Sun, 9 Mar 2014 16:06:50 -0700 (PDT) In-Reply-To: References: Date: Sun, 9 Mar 2014 19:06:50 -0400 Message-ID: Subject: Re: Using pf.conf with public access points. From: Joe Nosay To: Jason Hellenthal Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 23:06:51 -0000 On Sun, Mar 9, 2014 at 6:05 PM, Jason Hellenthal wrote: > > Sorry bit there is not enough information here. How can anyone know what > you are trying to accomplish by any of this. > > -- > Jason Hellenthal > Voice: 95.30.17.6/616 > JJH48-ARIN > > On Mar 9, 2014, at 15:36, Joe Nosay wrote: > > My pf.conf is attached. I would like to know: > 1. Is everything set up properly? > 2. How do I compensate for the use of public access points when the IP > addresses will always be different? > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > 1. I can only access public wireless because I homeless. There is no "static" nor "home" ip address for me to use. 2. I am trying to setup a jail as a developer environment. 3. I need to be able to get the jail started. I do not know the proper commands nor am I able to find the reference to them. 4. I will need internet access in the jail. I need to test a UDP Lite patch and that means having to broadcast on the available subnet listed. 5. If there is a set of commands- I have to go back to setting up the proper sysctl jail references- to start everything, then where may I find them? Yes, I will look again. Now you know what I am working with and the conditions. From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 23:18:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C752B755 for ; Sun, 9 Mar 2014 23:18:30 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A449C1A8 for ; Sun, 9 Mar 2014 23:18:30 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s29NITSO054855 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 9 Mar 2014 16:18:30 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s29NIT6i054854; Sun, 9 Mar 2014 16:18:29 -0700 (PDT) (envelope-from jmg) Date: Sun, 9 Mar 2014 16:18:29 -0700 From: John-Mark Gurney To: Joe Nosay Subject: Re: Using pf.conf with public access points. Message-ID: <20140309231829.GG32089@funkthat.com> Mail-Followup-To: Joe Nosay , freebsd-net@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sun, 09 Mar 2014 16:18:30 -0700 (PDT) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 23:18:30 -0000 Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: > 2. How do I compensate for the use of public access points when the IP > addresses will always be different? it doesn't appear that pf has this ability, but it looks like ipfw has this, from ipfw(8): me matches any IP address configured on an interface in the system. So, maybe switching to ipfw might be an option.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Sun Mar 9 23:32:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A29CF990 for ; Sun, 9 Mar 2014 23:32:29 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71DAD2DB for ; Sun, 9 Mar 2014 23:32:29 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id v10so6335558pde.29 for ; Sun, 09 Mar 2014 16:32:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ae-35.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=SZBvQiEDZsV5DNgIVXwFWjQKrPf+h0EXLceJzrDtiWs=; b=IfDMCtvcuhcRlhhUnD6rc51aFZCpRBHdVOQnREBpNNmKVHSHfwFHto5LcYfc5PqoxZ KOPL2ZeswnMnq/u+ewYQ8Xa6J+b+pUd/eI5wYg75Qe3FaxsrFW4bUbUbL4qYX4Chxu5J 8yDgHKE0UMt90R1pxhtL1JQLLrDqMA/VeVHKE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=SZBvQiEDZsV5DNgIVXwFWjQKrPf+h0EXLceJzrDtiWs=; b=XwGXeU8bj/+a3gbSc0GIl08+xCIV9V/k3JOdm+2YBkykKp/ptOqBuVQtrSYrwhkyKK ZdIjyuQacecNkl3Pe0AdHbq5wr9sh1NFSFeLFZz6Ie74Wd3x/fmaUokkQ2WFdlWxp76n /3z+OHkq1RFX4knhC1orx7jE3rqkPfsIA/Q1+wk7bZ1V3ByOM61HiGvJbLZj3f0EBQfW FzvMuO7qKn/GNRIREH5QijU9khy/D4rXzbpl5Eb29N1gsXO5qvDpEkgmw/tpP135583G ryw/7DzddrKSpHJt2UQCI2uiYXCxcfB8fxSCAOZFfV1vcU1ypt0/C9/WA8cwyz6PVR7z 8XiQ== X-Gm-Message-State: ALoCoQkmB1U8ruNUW16qSt3Zini/Fvrvtiu+Ze3vdFV1xIQ/EKl5vjOsQ5hAaI3hajXN+7/c/Hop X-Received: by 10.66.151.205 with SMTP id us13mr36069224pab.93.1394407948689; Sun, 09 Mar 2014 16:32:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.98.132 with HTTP; Sun, 9 Mar 2014 16:32:08 -0700 (PDT) X-Originating-IP: [109.224.162.206] In-Reply-To: <20140309231829.GG32089@funkthat.com> References: <20140309231829.GG32089@funkthat.com> From: =?ISO-8859-1?Q?Andr=E9_Lucas?= Date: Sun, 9 Mar 2014 23:32:08 +0000 Message-ID: Subject: Re: Using pf.conf with public access points. To: Joe Nosay , freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 23:32:29 -0000 On 9 March 2014 23:18, John-Mark Gurney wrote: > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: > > 2. How do I compensate for the use of public access points when the IP > > addresses will always be different? > > it doesn't appear that pf has this ability, but it looks like ipfw > has this, from ipfw(8): > me matches any IP address configured on an interface in > the > system. > > So, maybe switching to ipfw might be an option.. > pf can follow the IP address of an interface. From the pf.conf(5) manual page, "When the interface name is surrounded by parentheses, the rule is automatically updated whenever the interface changes its address. The ruleset does not need to be reloaded. This is especially useful with nat." -Andr=E9 From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 04:27:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8623AB46 for ; Mon, 10 Mar 2014 04:27:50 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 43020F3C for ; Mon, 10 Mar 2014 04:27:50 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id lx4so6672946iec.37 for ; Sun, 09 Mar 2014 21:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=dHoU9ZzzrutX2wbbZl+t5f3dDvBxxVDrNA1bcNptvts=; b=MvV2554NwlK2mme91ROz4xuW9PUWkq1iYlLVDqagoZRuarQaQcSeV+qnb5FOR5rczC gMAe96FKn2634tGqtZd7uZhyH6UsDLGsgzHj/hhguIAlphjydgNQ3C9+jV48pZ4laZke bx5ChB2fV/O4lpHZDM1De63uzL18JtltzIqgY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=dHoU9ZzzrutX2wbbZl+t5f3dDvBxxVDrNA1bcNptvts=; b=dUYHRf58v8whUP19Bkn+QOHrQjHqxUGfd6xoHsUClUNFRqFl+D2N9AInWkVyWikcbK f0VVVOV3IuqimIIZvg3wTXlDDGf+LdH/IQIsAxzBF+r9jODVQQPMpOZqQXfZV5kurh8q J9N+QYCovwHvCqZ+Wt19WqLHrgSyBF5MXzN0/7bSMHA8+dlI0NBVwOH+8cxoelSwQQPI iO0qJ9Specf0qXiMMXlWMyTjLX/1FivyKSrvhvHhG3AH7sX0uUCgfm9r73cYSp6HSKpn m7fvZ1zqGZWLbkzZFfgQhqSxWLVdIa3dxcIF4u+2m2WaCED+NLUx1sIj5OWDCKsJJjan t8Bw== X-Gm-Message-State: ALoCoQno9bzV6XJtUSDf1wSzy//krj/AbD9nOZhXmy2VfQyrWuDUjjyD/NtJh62hzF90eP6K7DiP X-Received: by 10.42.121.147 with SMTP id j19mr25595146icr.13.1394425669693; Sun, 09 Mar 2014 21:27:49 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id c17sm31909033igo.4.2014.03.09.21.27.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 09 Mar 2014 21:27:48 -0700 (PDT) References: <20140309231829.GG32089@funkthat.com> Mime-Version: 1.0 (1.0) In-Reply-To: <20140309231829.GG32089@funkthat.com> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-A7CBF3BC-2155-4090-BF89-2F3411F1FD9C; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Using pf.conf with public access points. Date: Mon, 10 Mar 2014 00:27:45 -0400 To: John-Mark Gurney X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Joe Nosay , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 04:27:50 -0000 --Apple-Mail-A7CBF3BC-2155-4090-BF89-2F3411F1FD9C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable You'll want to not use up addresses in your pf.conf Block on default and then open up by definition of ports instead. Forget the= whole IPAddr thing and treat this as a roaming client firewall. --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: >=20 > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: >> 2. How do I compensate for the use of public access points when the IP >> addresses will always be different? >=20 > it doesn't appear that pf has this ability, but it looks like ipfw > has this, from ipfw(8): > me matches any IP address configured on an interface in t= he > system. >=20 > So, maybe switching to ipfw might be an option.. >=20 > --=20 > John-Mark Gurney Voice: +1 415 225 5579 >=20 > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --Apple-Mail-A7CBF3BC-2155-4090-BF89-2F3411F1FD9C Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDMxMDA0Mjc0NlowIwYJKoZIhvcN AQkEMRYEFPJ8DD1OyKh6cqAmRJJv7j3XAfqrMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAggNB89XH/U/E3a4RBvhe SDSzk7oahWv71LFgUXEsfRgWjWKeGLS9UGWQugVkeweIA15rkEo2hbS8s/mL0JkvmxJIV8/r5rgb LL26Q1iWrLDt0AOVWtiA0TZ7ZWQse+hKy2XoRlgVfuJWfl8Z+nFE1IYEeVZN5GHtFcqi+rlYd5W6 SRs+lLKqh//xkoQPoqsCD8Sfae3dyK+P59QPpc18oJrOWmnWcwe1pqW9qRKkcm1O3sjlKfu/kiQd HD/CMyihKPPFK0e1kXeSDmhkNqumLhGUSxSTAqW929qUjbvNY7G0erqcw4sFS/IZPkpZqMRo8EzK OkptzClbtT1d12JplwAAAAAAAA== --Apple-Mail-A7CBF3BC-2155-4090-BF89-2F3411F1FD9C-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 06:22:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D37E78E for ; Mon, 10 Mar 2014 06:22:12 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D0196A20 for ; Mon, 10 Mar 2014 06:22:11 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id cm18so2610799qab.8 for ; Sun, 09 Mar 2014 23:22:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jPJkcQ/4cpUKuHhpi6scSCgm0+V7o4dd8bwt455BBFw=; b=weR6nfhdPGFtLIlvV68pjCwgJ0RMWB+QDG3kL3ZPEl2HFtqN6dIzP5xT2jZYUgNQsV +C0XDgw9NPGNb408YCA408/B7I0r6WQiVtyO/ngvjOC5JZMRgDXck3/JmgfJZN52caCg L8jHML9yUYoH3zuOxr620Lit/2TPEQNRuGc/Qm2YuCp1Nv7QJCsOcx5zpfT8kyzVxSdJ 7ZcNf1/fBMZNWbH/1FpNTXmJw0Eg38aXimk4hLAM3oR0IIgwTiG1OkYK2nMG9pzYFRLM kDlqqYfitDyDYGlF6NvUEF4xNiswVXDS56XFQwI5ZbqHPD3M3ukuLm+X7FjLPFKV8V7s ghIQ== MIME-Version: 1.0 X-Received: by 10.140.85.35 with SMTP id m32mr36605427qgd.40.1394432530960; Sun, 09 Mar 2014 23:22:10 -0700 (PDT) Received: by 10.224.191.66 with HTTP; Sun, 9 Mar 2014 23:22:10 -0700 (PDT) Date: Mon, 10 Mar 2014 14:22:10 +0800 Message-ID: Subject: A problem on SCTP From: Niu Zhixiong To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 06:22:12 -0000 Dear freebsd-net all, I am a beginner of FreeBSD. I meet a problem in SCTP testing. I have two virtual machine, both installed FreeBSD 10 / i386 / with generic KERNEL (the host is CentOS 6.5 x86-64 with KVM) The first virtual machine named freetest0 and second is freetest1 freetest0 =3D freebsd 10 / i386 / the IF is vtnet2 192.168.6.100 freetest1 =3D freebsd 10 / i386 / the IF is vtnet2 192.168.6.110 I want to test the speed between to freetest(s) IF. But, the problem is they cannot get connected by SCTP. the TCP and UDP are well. Whatever I use iperf3 (with SCTP support) and netperfmeter, they cannot get connected by SCTP. #the server is freetest1 root@freetest1:~ # netstat -an -f inet Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp46 0 0 *.9000 *.* LISTE= N tcp4 0 0 192.168.0.110.22 192.168.0.1.39754 ESTABLISHED tcp4 0 0 192.168.0.110.22 192.168.0.1.39752 ESTABLISHED tcp4 0 0 127.0.0.1.25 *.* LISTE= N tcp4 0 0 *.22 *.* LISTE= N udp46 0 0 *.9000 *.* udp4 0 0 *.514 *.* Active SCTP associations (including servers) Proto Type Local Address Foreign Address (state) sctp46 1to1 fe80::5054:ff:fe.9000 LISTEN 192.168.8.110.9000 fe80::5054:ff:fe.9000 192.168.6.110.9000 fe80::5054:ff:fe.9000 192.168.0.110.9000 127.0.0.1.9000 fe80::1.9000 ::1.9000 sctp46 1toN fe80::5054:ff:fe.9001 LISTEN 192.168.8.110.9001 fe80::5054:ff:fe.9001 192.168.6.110.9001 fe80::5054:ff:fe.9001 192.168.0.110.9001 127.0.0.1.9001 fe80::1.9001 ::1.9001 root@freetest0:~ # netperfmeter 192.168.6.110:9000 Network Performance Meter - Version 1.0 --------------------------------------- Active Mode: - Measurement ID =3D 4bc75bae - Remote Address =3D 192.168.6.110:9000 - Control Address =3D 192.168.6.110:9001 - connecting ... # root@freetest0:~ # tcpdump -i vtnet2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vtnet2, link-type EN10MB (Ethernet), capture size 65535 bytes 14:11:03.839031 IP 192.168.6.100.55228 > 192.168.6.110.5201: Flags [S], seq 1318388212, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2652085 ecr 0], length 0 14:11:03.868787 IP 192.168.6.110.5201 > 192.168.6.100.55228: Flags [R.], seq 0, ack 1318388213, win 0, length 0 14:11:35.235362 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:38.256378 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:40.256418 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:44.256099 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:52.254442 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] root@freetest1:~ # tcpdump -i vtnet2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vtnet2, link-type EN10MB (Ethernet), capture size 65535 bytes 14:11:35.979349 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:39.000411 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:41.000495 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:45.000116 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] 14:11:52.998491 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp (1) [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] [init TSN: 332259025] root@freetest0:~ # uname -a FreeBSD freetest0 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Fri Jan 17 01:46:25 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC i386 Could somebody help me? Thank you in advance. Regards, Niu Zhixiong =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF= =BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D kaiaixi@gmail.com From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 08:37:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 364F3622 for ; Mon, 10 Mar 2014 08:37:10 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4000818 for ; Mon, 10 Mar 2014 08:37:09 +0000 (UTC) Received: from [192.168.1.200] (p508F0570.dip0.t-ipconnect.de [80.143.5.112]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 6D2A71C1040F7; Mon, 10 Mar 2014 09:37:06 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: A problem on SCTP From: Michael Tuexen In-Reply-To: Date: Mon, 10 Mar 2014 09:37:01 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Niu Zhixiong X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 08:37:10 -0000 On 10 Mar 2014, at 07:22, Niu Zhixiong wrote: > Dear freebsd-net all, > I am a beginner of FreeBSD. I meet a problem in SCTP testing. >=20 > I have two virtual machine, both installed FreeBSD 10 / i386 / with = generic > KERNEL > (the host is CentOS 6.5 x86-64 with KVM) >=20 > The first virtual machine named freetest0 and second is freetest1 >=20 > freetest0 =3D freebsd 10 / i386 / the IF is vtnet2 192.168.6.100 > freetest1 =3D freebsd 10 / i386 / the IF is vtnet2 192.168.6.110 >=20 > I want to test the speed between to freetest(s) IF. > But, the problem is they cannot get connected by SCTP. the TCP and UDP = are > well. >=20 > Whatever I use iperf3 (with SCTP support) and netperfmeter, they = cannot get > connected by SCTP. >=20 > #the server is freetest1 > root@freetest1:~ # netstat -an -f inet > Active Internet connections (including servers) > Proto Recv-Q Send-Q Local Address Foreign Address > (state) > tcp46 0 0 *.9000 *.* = LISTEN > tcp4 0 0 192.168.0.110.22 192.168.0.1.39754 > ESTABLISHED > tcp4 0 0 192.168.0.110.22 192.168.0.1.39752 > ESTABLISHED > tcp4 0 0 127.0.0.1.25 *.* = LISTEN > tcp4 0 0 *.22 *.* = LISTEN > udp46 0 0 *.9000 *.* > udp4 0 0 *.514 *.* > Active SCTP associations (including servers) > Proto Type Local Address Foreign Address (state) > sctp46 1to1 fe80::5054:ff:fe.9000 LISTEN > 192.168.8.110.9000 > fe80::5054:ff:fe.9000 > 192.168.6.110.9000 > fe80::5054:ff:fe.9000 > 192.168.0.110.9000 > 127.0.0.1.9000 > fe80::1.9000 > ::1.9000 > sctp46 1toN fe80::5054:ff:fe.9001 LISTEN > 192.168.8.110.9001 > fe80::5054:ff:fe.9001 > 192.168.6.110.9001 > fe80::5054:ff:fe.9001 > 192.168.0.110.9001 > 127.0.0.1.9001 > fe80::1.9001 > ::1.9001 >=20 >=20 >=20 >=20 >=20 > root@freetest0:~ # netperfmeter 192.168.6.110:9000 > Network Performance Meter - Version 1.0 > --------------------------------------- >=20 > Active Mode: > - Measurement ID =3D 4bc75bae > - Remote Address =3D 192.168.6.110:9000 > - Control Address =3D 192.168.6.110:9001 - connecting ... >=20 > # >=20 >=20 >=20 > root@freetest0:~ # tcpdump -i vtnet2 > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > listening on vtnet2, link-type EN10MB (Ethernet), capture size = 65535 > bytes > 14:11:03.839031 IP 192.168.6.100.55228 > 192.168.6.110.5201: Flags = [S], > seq 1318388212, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS = val > 2652085 ecr 0], length 0 > 14:11:03.868787 IP 192.168.6.110.5201 > 192.168.6.100.55228: Flags > [R.], seq 0, ack 1318388213, win 0, length 0 > 14:11:35.235362 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:38.256378 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:40.256418 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:44.256099 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:52.254442 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] >=20 > root@freetest1:~ # tcpdump -i vtnet2 > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > listening on vtnet2, link-type EN10MB (Ethernet), capture size = 65535 > bytes > 14:11:35.979349 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:39.000411 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:41.000495 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:45.000116 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] > 14:11:52.998491 IP 192.168.6.100.52018 > 192.168.6.110.9001: sctp = (1) > [INIT] [init tag: 3995201801] [rwnd: 1864135] [OS: 10] [MIS: 2048] = [init > TSN: 332259025] >=20 > root@freetest0:~ # uname -a > FreeBSD freetest0 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Fri = Jan 17 > 01:46:25 UTC 2014 = root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC > i386 >=20 > Could somebody help me? Thank you in advance. I haven't used KVM, so I hope there is no NAT involved between the two FreeBSD machines. Can you build a kernel with SCTP_DEBUG added to the options? Then use sudo sysctl -w net.inet.sctp.debug=3D1 to enable it on the server. This should provide some information what is = going on. Best regards Michael >=20 > Regards, > Niu Zhixiong > =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D= =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D > kaiaixi@gmail.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F9C3188 for ; Mon, 10 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7B00E810 for ; Mon, 10 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2AB6n65043277 for ; Mon, 10 Mar 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2AB6nh0043275 for freebsd-net@FreeBSD.org; Mon, 10 Mar 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Mar 2014 11:06:49 GMT Message-Id: <201403101106.s2AB6nh0043275@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183390 net [ixgbe] 10gigabit networking problems o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) o kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 469 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 14:20:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93E923FB for ; Mon, 10 Mar 2014 14:20:03 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 67D5BE1B for ; Mon, 10 Mar 2014 14:20:03 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id md12so7341249pbc.35 for ; Mon, 10 Mar 2014 07:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Fya5DgDjkT0b34aYSfkJcGEL4k6wTnyIzkaiU4D66NA=; b=aKZn/zlB4x7MTZ1FO2irIn4T9iEHF1nsz9mx73oSs4C7IRq3+7eWvK2tHUPP/PiuQI iOI/h/soV6KbhiIVcPI2mhKe++CDqTyOS+ZjgYb+jefcmOUlC1tlQH46IAr2B1gJN2f6 5/tBwNCgwBfF4hN+PaL39TO4lL0nXVdDF2P4YCxDuo06+se/3t3uKKQiaVO+9Fur5kDF zzpGTVJ6/daWnMu4yAPfyFf/hBQILe7pN0fNYbEqemzUr7ldTkNrbdGtO2oFqxbzff61 aior7N4NIJxSZdOGyKjT9tn2NAQsT0z8hH2j0FPQExPhtYAzK2mLL4K/Z9OrXKSke8MV eyZg== MIME-Version: 1.0 X-Received: by 10.66.217.133 with SMTP id oy5mr41254005pac.46.1394461201748; Mon, 10 Mar 2014 07:20:01 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.70.6.36 with HTTP; Mon, 10 Mar 2014 07:20:01 -0700 (PDT) In-Reply-To: <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> References: <20140309231829.GG32089@funkthat.com> <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> Date: Mon, 10 Mar 2014 09:20:01 -0500 X-Google-Sender-Auth: 8yIPAFnr3zR67SDBFEAAiXVVKHs Message-ID: Subject: Re: Using pf.conf with public access points. From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Jason Hellenthal Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Joe Nosay , John-Mark Gurney , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 14:20:03 -0000 Usually pf(4) does support having dynamic ips inside its ruleset. For example just putting the interface name as address or putting $iface:0 for first address etc... Take a look an man page of pf.conf and search for the string 'Interface names and interface group names can' On Sun, Mar 9, 2014 at 11:27 PM, Jason Hellenthal wrote: > You'll want to not use up addresses in your pf.conf > > Block on default and then open up by definition of ports instead. Forget > the whole IPAddr thing and treat this as a roaming client firewall. > > > -- > Jason Hellenthal > Voice: 95.30.17.6/616 > JJH48-ARIN > > > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: > > > > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: > >> 2. How do I compensate for the use of public access points when the IP > >> addresses will always be different? > > > > it doesn't appear that pf has this ability, but it looks like ipfw > > has this, from ipfw(8): > > me matches any IP address configured on an interface in > the > > system. > > > > So, maybe switching to ipfw might be an option.. > > > > -- > > John-Mark Gurney Voice: +1 415 225 5579 > > > > "All that I will do, has been done, All that I have, has not." > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Ermal From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 18:25:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCF4B38D for ; Mon, 10 Mar 2014 18:25:59 +0000 (UTC) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6753FCCD for ; Mon, 10 Mar 2014 18:25:59 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id t61so9015169wes.2 for ; Mon, 10 Mar 2014 11:25:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type:content-transfer-encoding; bh=d4JqTV4rBW2g3GO7XmfQhY9iPTvHuP1QhQLqHJqIhr0=; b=rv02LP/9qVLfsHBMgtd9QGOzxSOP4SVOCwCj7nZcUU7cx4on3sDm16JaGJeWqdpOH2 J+r7vS6azMCv8ozXKJS9kKtaJxoUl2A+aP524P62g2FSz4NJUvcqAkJaPBYeAA0tyhBl NGNCNdZIk1wWCYY+X7twakIj9ha3ZjwwAi5DLHjTsvQ5fGra75cWEF4ryLpa861d2Ayh GTEo16GL4NR47h6avGIJ8rTN6ha7LMkVvZWPzCalAtYMesra3C0N6HnRcMEatzE1lhBY 0YpkFZi4z12g3GOnp8sVAxI0hQ8aTY2e2qZF2ALN5rnenfIq27iqAXzSu9cm/893KuWj i7ig== X-Received: by 10.194.63.103 with SMTP id f7mr9953698wjs.38.1394475957654; Mon, 10 Mar 2014 11:25:57 -0700 (PDT) Received: from srvbsdfenssv.interne.associated-bears.org (LCaen-151-92-21-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by mx.google.com with ESMTPSA id h9sm54063957wjz.16.2014.03.10.11.25.56 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 10 Mar 2014 11:25:56 -0700 (PDT) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id 014E5CF294; Mon, 10 Mar 2014 19:25:55 +0100 (CET) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N4gj7IBD3dbA; Mon, 10 Mar 2014 19:25:54 +0100 (CET) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id 104F1CF28F; Mon, 10 Mar 2014 19:25:54 +0100 (CET) From: Eric Masson To: "John W. O'Brien" Subject: Re: [FreeBSD 10.0] nat before vpn, incoming packets not translated In-Reply-To: <531A5FBF.1000507@saltant.com> (John W. O'Brien's message of "Fri, 07 Mar 2014 19:09:35 -0500") References: <868uu4rshh.fsf@srvbsdfenssv.interne.associated-bears.org> <53193371.4090603@saltant.com> <09B6BE02-2F04-41A1-AC0D-9A7943F88086@openresearch.com> <86siqtluns.fsf@srvbsdfenssv.interne.associated-bears.org> <531A5FBF.1000507@saltant.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) X-Operating-System: FreeBSD 9.2-RELEASE-p3 amd64 Date: Mon, 10 Mar 2014 19:25:53 +0100 Message-ID: <86siqpj4ge.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Mailing List FreeBSD Network , Philipp Schmid X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 18:25:59 -0000 "John W. O'Brien" writes: Hi John, > I haven't done the mind meld with "reverse" yet. > Could you comment on why you need to operate in a reversed NAT > environment? In this particular case, this is a test lab. The purpose of this kind of setup is the following : - administrator of the remote lan demands your endpoint to be seen as a unique ip address on his ipsec device. - subnet ranges on each side conflict, so one must be natted. > What is it that's being reversed, and how does that apply to your use > case? Packets from local lan to remote lan are natted on the internal interface of gateway1 (source address is translated to match the ipsec policy) Regards Éric From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 18:56:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA785C1E for ; Mon, 10 Mar 2014 18:56:56 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 94178F82 for ; Mon, 10 Mar 2014 18:56:56 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id rd18so7733288iec.1 for ; Mon, 10 Mar 2014 11:56:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=nWPt/Tblppcj0IIftnHE0nNH0s6G8ac3MnP2R5O+3qo=; b=Jb3K3M3rAu4uD4uslEhmtSxKHeUlnl/jej9UAxtSEYCsK20At29e9G/J4TVuGGVpd0 bXHz09O40RdCxjzE7o4DUN+qakazrwEyOi2xx5sXjxxPntkJXgoX89PogMoDya2t1F8s UM1bvoizXAEhhEK6gjUevzN7A5IS2Wlgw+Vwg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=nWPt/Tblppcj0IIftnHE0nNH0s6G8ac3MnP2R5O+3qo=; b=VngrLUN59sW7LhxTZwl61X7ROj+cMf08I3pE52chqHLOPoNViLvFd6QLfxhCkntfBT T7ZKIfI8RT1PULB2n7dVd9wm573yFiW8837bqdAKzJvt5eekcix+tIc6k6nFRcL+oAzz 7GpkiHIEZ8S3EMeRv7HLxOD7/9IKCjg51NsS7DsHhlJPMfoUAQKWXxqhiyVysMS22nlU Ol504txBLbumHCOPsO3roU359/5aghs6iMv1fmf5CG45dTNBHJVtyFCE6Hs1HDJpESfV gKHVT5X7+PghIOzyUjoNNpglO6tHo07++ipoqVkOxUI2BXi5YAoq1mgpkDgp01kLQvI0 iadw== X-Gm-Message-State: ALoCoQlkfs/pWv0Z4i+gfvvEKRtv9fetgNThhsnYNO9GPGc3Fn9dJcHSrPa2ILLNIwRCdipxhItk X-Received: by 10.50.43.165 with SMTP id x5mr20354873igl.40.1394477815922; Mon, 10 Mar 2014 11:56:55 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id ai4sm38680800igd.3.2014.03.10.11.56.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 10 Mar 2014 11:56:54 -0700 (PDT) References: <20140309231829.GG32089@funkthat.com> <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-033E70FA-0CCB-4665-AC61-4EB08AB56CA4; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: <71CCF277-8BF7-4C3B-9F9E-2095EA4CC060@dataix.net> X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Using pf.conf with public access points. Date: Mon, 10 Mar 2014 14:56:51 -0400 To: =?utf-8?Q?Ermal_Lu=C3=A7i?= X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Joe Nosay , John-Mark Gurney , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 18:56:56 -0000 --Apple-Mail-033E70FA-0CCB-4665-AC61-4EB08AB56CA4 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I nearly forgot all about that feature thank you for the reminder. --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On Mar 10, 2014, at 10:20, Ermal Lu=C3=A7i wrote: >=20 > Usually pf(4) does support having dynamic ips inside its ruleset. > For example just putting the interface name as address or putting $iface:0= for first address etc... >=20 > Take a look an man page of pf.conf and search for the string 'Interface na= mes and interface group names can' >=20 >=20 >> On Sun, Mar 9, 2014 at 11:27 PM, Jason Hellenthal wrote: >> You'll want to not use up addresses in your pf.conf >>=20 >> Block on default and then open up by definition of ports instead. Forget t= he whole IPAddr thing and treat this as a roaming client firewall. >>=20 >>=20 >> -- >> Jason Hellenthal >> Voice: 95.30.17.6/616 >> JJH48-ARIN >>=20 >> > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: >> > >> > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: >> >> 2. How do I compensate for the use of public access points when the IP= >> >> addresses will always be different? >> > >> > it doesn't appear that pf has this ability, but it looks like ipfw >> > has this, from ipfw(8): >> > me matches any IP address configured on an interface i= n the >> > system. >> > >> > So, maybe switching to ipfw might be an option.. >> > >> > -- >> > John-Mark Gurney Voice: +1 415 225 5579 >> > >> > "All that I will do, has been done, All that I have, has not." >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 >=20 >=20 > --=20 > Ermal --Apple-Mail-033E70FA-0CCB-4665-AC61-4EB08AB56CA4 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDMxMDE4NTY1M1owIwYJKoZIhvcN AQkEMRYEFOQF2WrEI5I2bPvZJI8iR8XIGZuXMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAqt0YKP5m0HI7DVi7M//z rOLzNTWgBgnxLK1P+n9KKkAoudbVFiTYroVn/Gd5hu7rx56J+8sRMZRN0t/YwL5PXeVz1nTEzrET xVrM7uEpuBSFMTy/iybPU1nqYD+6noBuRkDyd6dJHInzWtzHuer90NNk0mrfg8Nd+JWVG+UYzu1B /5k8uoZh4/mXW0dD8xHbDzQvjyi7VECYiD8PopgC+lcqL5WwqVk5rSoJZW1s3ZVfSK7Ux8sO2zzp bBdbY1Mwr56yyOCDHxEtw4RRwFSkzWnIjiwFGGl7tGSOpqbkJgf96Cc5lBYQViLp7DG6/wAl43Ki JwDpQA3d+E8hrnzUXgAAAAAAAA== --Apple-Mail-033E70FA-0CCB-4665-AC61-4EB08AB56CA4-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 20:41:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A73D8678; Mon, 10 Mar 2014 20:41:57 +0000 (UTC) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 608DCCAF; Mon, 10 Mar 2014 20:41:57 +0000 (UTC) Received: by mail-ob0-f171.google.com with SMTP id wn1so7599031obc.2 for ; Mon, 10 Mar 2014 13:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Kek6YqZDBVV8rjNj161Q9Wvt7LASK9PrsLB3C1LjmcY=; b=lF7XKY8ec4CTow8/90mPP5u48pYJ5+9z9T7/BSpQsuJRLlIwga+NZPjb79GzwTpg8+ BG+gfTP5cRC4QjE+VwX0XMUxpcu0PDazvB8B2kYNl+yZNvUNS//cAbI8EV74ImnncHe4 NUWUg7a13CBmCCaGjCJw8/V++M3K0SJiOrQzTyzM7I1ZD4Tl1IxGJFpAvQ35ugwEZYpI hPB4VBQd0lT0ehTRYpmjiljEOj+N0tAz3eU0F42EbrKOY5n+Ogdr+cOzdavdProKToK7 P2fuCLq66MxX5txEXZEKFLikEDozY/8V3uOY0gW0mjJsWfkyjDiHoz9n8VDxKfZN9DuL /hJw== MIME-Version: 1.0 X-Received: by 10.182.18.102 with SMTP id v6mr2112157obd.71.1394484116725; Mon, 10 Mar 2014 13:41:56 -0700 (PDT) Received: by 10.182.76.201 with HTTP; Mon, 10 Mar 2014 13:41:56 -0700 (PDT) In-Reply-To: <71CCF277-8BF7-4C3B-9F9E-2095EA4CC060@dataix.net> References: <20140309231829.GG32089@funkthat.com> <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> <71CCF277-8BF7-4C3B-9F9E-2095EA4CC060@dataix.net> Date: Mon, 10 Mar 2014 16:41:56 -0400 Message-ID: Subject: Re: Using pf.conf with public access points. From: Joe Nosay To: Jason Hellenthal Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: =?ISO-8859-1?Q?Ermal_Lu=E7i?= , "freebsd-net@freebsd.org" , John-Mark Gurney X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 20:41:57 -0000 On Mon, Mar 10, 2014 at 2:56 PM, Jason Hellenthal w= rote: > I nearly forgot all about that feature thank you for the reminder. > > > -- > Jason Hellenthal > Voice: 95.30.17.6/616 > JJH48-ARIN > > On Mar 10, 2014, at 10:20, Ermal Lu=E7i wrote: > > Usually pf(4) does support having dynamic ips inside its ruleset. > For example just putting the interface name as address or putting $iface:= 0 > for first address etc... > > Take a look an man page of pf.conf and search for the string 'Interface > names and interface group names can' > > > On Sun, Mar 9, 2014 at 11:27 PM, Jason Hellenthal wrote: > >> You'll want to not use up addresses in your pf.conf >> >> Block on default and then open up by definition of ports instead. Forget >> the whole IPAddr thing and treat this as a roaming client firewall. >> >> >> -- >> Jason Hellenthal >> Voice: 95.30.17.6/616 >> JJH48-ARIN >> >> > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: >> > >> > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: >> >> 2. How do I compensate for the use of public access points when the I= P >> >> addresses will always be different? >> > >> > it doesn't appear that pf has this ability, but it looks like ipfw >> > has this, from ipfw(8): >> > me matches any IP address configured on an interface >> in the >> > system. >> > >> > So, maybe switching to ipfw might be an option.. >> > >> > -- >> > John-Mark Gurney Voice: +1 415 225 5579 >> > >> > "All that I will do, has been done, All that I have, has not." >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Ermal > > Has anyone thought about putting themselves in an environment similar to mine- not everything- when it comes to networking? You would have to set everything up with the following parameters: 1. Because you are at more than one place, you cannot setup wlanX or the wlandev in rc.conf. They must always be created after booting and logging in. 2. Dhclient cannot be automatic because a public access area may have more than one available bssid for connecting. 3. Since each public access will have different firewalls, streaming and web services may not be able to be ran. 4. A script would probably work better than static settings in this case. From owner-freebsd-net@FreeBSD.ORG Mon Mar 10 23:57:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85EE864D for ; Mon, 10 Mar 2014 23:57:37 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3E2B91A6 for ; Mon, 10 Mar 2014 23:57:37 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id tp5so8132046ieb.12 for ; Mon, 10 Mar 2014 16:57:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=P6oT63PF8Kb3W6F5UPrlHRbxTjwA9Fc64AkLz2bcJUQ=; b=d4De/pOiFAwdM2NuXaGpYEE3Ao8O3lsziWngeCSj856SITZWTfbJdLVgO6JkM0GsZN ltG+02Ay2SuMxFtR5HDpYjO3Yp0jFY5ITAIQyn4d0SHXLsBAWxh5zYo7bIIXZjpgh1Dp WTDttw+qluT0pAPFGQ1VPV2YrtUi4wWZz7N7o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=P6oT63PF8Kb3W6F5UPrlHRbxTjwA9Fc64AkLz2bcJUQ=; b=icWonozJ1OkJDPmhRn0sUVb4P7Lz8HNEHnX1cUoR9mU4+aLYBBZPYetvYN2R6IXtT7 gDn8jlCcpIHnF8XFxb8i12zO+mujPGczbKWLthLOS5I74nuUMFoSVtqcBDTKofxcEPmk w1gD8D5r6QjYHuRIH/Bi53OE2L41MCexOGFQ07IY5Dt9x4+N8z9ClYOgk5Vt9++uEYTw VJuRtPSJ0tfjNHo8j3NTJgSpQTXECGBpzYqsVoEs2u+pWchUQLJlDFBhJ3IKSyuJTNFN 3TdrldH5o03/3Jhf5uCejlc1IU0QBaURlr3Rm7phFKLzXWhjxKjoaCrpk/zOyIcPQq/X dkOw== X-Gm-Message-State: ALoCoQlVZDPVny+f29grbgDe6eR4vwp69NL/g5qwwKL2ul1nElwbvU9zWKQxsuaxkN8LR8XpWlmg X-Received: by 10.50.33.11 with SMTP id n11mr21110495igi.0.1394495856569; Mon, 10 Mar 2014 16:57:36 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id pn6sm41320785igb.4.2014.03.10.16.57.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 10 Mar 2014 16:57:35 -0700 (PDT) References: <20140309231829.GG32089@funkthat.com> <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> <71CCF277-8BF7-4C3B-9F9E-2095EA4CC060@dataix.net> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-BA12FC3E-6159-4800-8F22-43FEE71DDB58; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Using pf.conf with public access points. Date: Mon, 10 Mar 2014 19:57:33 -0400 To: Joe Nosay X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: =?utf-8?Q?Ermal_Lu=C3=A7i?= , "freebsd-net@freebsd.org" , John-Mark Gurney X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 23:57:37 -0000 --Apple-Mail-BA12FC3E-6159-4800-8F22-43FEE71DDB58 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I feel as if you are over thinking this project just a little. dhclient has nothing to do with the bssid. wlanX can be setup to use DHCP and for wep or wpa or open connections in rc.= conf. You can't control others firewalls only your own so why the worry about that= ? --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On Mar 10, 2014, at 16:41, Joe Nosay wrote: >=20 >=20 >=20 >=20 >> On Mon, Mar 10, 2014 at 2:56 PM, Jason Hellenthal wrote: >> I nearly forgot all about that feature thank you for the reminder. >>=20 >>=20 >> --=20 >> Jason Hellenthal >> Voice: 95.30.17.6/616 >> JJH48-ARIN >>=20 >>> On Mar 10, 2014, at 10:20, Ermal Lu=C3=A7i wrote: >>>=20 >>> Usually pf(4) does support having dynamic ips inside its ruleset. >>> For example just putting the interface name as address or putting $iface= :0 for first address etc... >>>=20 >>> Take a look an man page of pf.conf and search for the string 'Interface n= ames and interface group names can' >>>=20 >>>=20 >>>> On Sun, Mar 9, 2014 at 11:27 PM, Jason Hellenthal wrote: >>>> You'll want to not use up addresses in your pf.conf >>>>=20 >>>> Block on default and then open up by definition of ports instead. Forge= t the whole IPAddr thing and treat this as a roaming client firewall. >>>>=20 >>>>=20 >>>> -- >>>> Jason Hellenthal >>>> Voice: 95.30.17.6/616 >>>> JJH48-ARIN >>>>=20 >>>> > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: >>>> > >>>> > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: >>>> >> 2. How do I compensate for the use of public access points when the I= P >>>> >> addresses will always be different? >>>> > >>>> > it doesn't appear that pf has this ability, but it looks like ipfw >>>> > has this, from ipfw(8): >>>> > me matches any IP address configured on an interface= in the >>>> > system. >>>> > >>>> > So, maybe switching to ipfw might be an option.. >>>> > >>>> > -- >>>> > John-Mark Gurney Voice: +1 415 225 5579 >>>> > >>>> > "All that I will do, has been done, All that I have, has not." >>>> > _______________________________________________ >>>> > freebsd-net@freebsd.org mailing list >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>>=20 >>>=20 >>>=20 >>> --=20 >>> Ermal >=20 >=20 > Has anyone thought about putting themselves in an environment similar to m= ine- not everything- when it comes to networking? You would have to set ever= ything up with the following parameters: > 1. Because you are at more than one place, you cannot setup wlanX or the w= landev in rc.conf. They must always be created after booting and logging in.= > 2. Dhclient cannot be automatic because a public access area may have more= than one available bssid for connecting. > 3. Since each public access will have different firewalls, streaming and w= eb services may not be able to be ran. > 4. A script would probably work better than static settings in this case. >=20 >=20 --Apple-Mail-BA12FC3E-6159-4800-8F22-43FEE71DDB58 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDMxMDIzNTczNFowIwYJKoZIhvcN AQkEMRYEFIZ4H3uz9v2DVTqapZ/4gsRnezp0MIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAV8M3MCFvnjeRWdFnoTZP q6mWW2IBMaZkbQuyLM9v53GrMoXCd5TxXcaW9tcDmm70ayYf/Kl4BiGVSmGmFBn0AhU/RCQfG3+P DDYGEFKQN+zFQbzSbPfJ83STz6eQXPIUKcO4vUzB7ycWRsBhx9n1/XSR+T4/SolWvvfJuCC+oMC7 21nm3vj417Cr7yzMRrBv6o4KuVVK+cbS3gzxbNBEkHK5uxUNmZ3l3+fhbEWBeH9D/haS+tDG6Mlp V55IeONZFJvX+hMS3PGLi/eSrfsS91Nte5kXUh341pIXi3bMWVxRKu3ihKUDe8yxjLbsqZc3N01k QiyIQNKTTF9wXS2fewAAAAAAAA== --Apple-Mail-BA12FC3E-6159-4800-8F22-43FEE71DDB58-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 00:41:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1F042B8; Tue, 11 Mar 2014 00:41:44 +0000 (UTC) Received: from mail-ob0-x230.google.com (mail-ob0-x230.google.com [IPv6:2607:f8b0:4003:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7905B768; Tue, 11 Mar 2014 00:41:44 +0000 (UTC) Received: by mail-ob0-f176.google.com with SMTP id wp18so7764448obc.21 for ; Mon, 10 Mar 2014 17:41:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QRf0nMnh5911BtVhPXSvCK6dM3OYMeFZNt4jUrtHy1M=; b=pjVUH+cMuh6HqrYmj+HJhCe9tkQjBnvXYE/JeCgyp9ul0Mie30tdkN0mKVGZ6p6Fmi AZFFeqD1+6ugjmzqnCvoV36MpEDctdf1OBN631suF7fb9x3xePBwUeL9KX57rjjW7Dwb rKEybQ7tNVCY7OK25Ee3SGRpfUJNVwPgP9wisFnJWGLuwNd2HRWPNs6zlJtODnHAt7Fc DLs29svZS/MMHaoMzxh2lo5dbp2irbmpwSFcNNbXLVMZ4BwKG2+TMuIolsoFjYBjNaI7 uVFDz47lh+r9kBaKa0ht5P1zjOqIg+IhLPy5XEAc/9jhBPU7lFJwXk61aCfn275npedV O/uQ== MIME-Version: 1.0 X-Received: by 10.182.19.132 with SMTP id f4mr30662871obe.14.1394498503775; Mon, 10 Mar 2014 17:41:43 -0700 (PDT) Received: by 10.182.76.201 with HTTP; Mon, 10 Mar 2014 17:41:43 -0700 (PDT) In-Reply-To: References: <20140309231829.GG32089@funkthat.com> <9C40270E-18E0-4993-B7C5-BD8B5A24C95D@dataix.net> <71CCF277-8BF7-4C3B-9F9E-2095EA4CC060@dataix.net> Date: Mon, 10 Mar 2014 20:41:43 -0400 Message-ID: Subject: Re: Using pf.conf with public access points. From: Joe Nosay To: Jason Hellenthal Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: =?ISO-8859-1?Q?Ermal_Lu=E7i?= , "freebsd-net@freebsd.org" , John-Mark Gurney X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 00:41:44 -0000 On Mon, Mar 10, 2014 at 7:57 PM, Jason Hellenthal w= rote: > I feel as if you are over thinking this project just a little. > > dhclient has nothing to do with the bssid. > wlanX can be setup to use DHCP and for wep or wpa or open connections in > rc.conf. > You can't control others firewalls only your own so why the worry about > that ? > > > -- > Jason Hellenthal > Voice: 95.30.17.6/616 > JJH48-ARIN > > On Mar 10, 2014, at 16:41, Joe Nosay wrote: > > > > > On Mon, Mar 10, 2014 at 2:56 PM, Jason Hellenthal wrote: > >> I nearly forgot all about that feature thank you for the reminder. >> >> >> -- >> Jason Hellenthal >> Voice: 95.30.17.6/616 >> JJH48-ARIN >> >> On Mar 10, 2014, at 10:20, Ermal Lu=E7i wrote: >> >> Usually pf(4) does support having dynamic ips inside its ruleset. >> For example just putting the interface name as address or putting >> $iface:0 for first address etc... >> >> Take a look an man page of pf.conf and search for the string 'Interface >> names and interface group names can' >> >> >> On Sun, Mar 9, 2014 at 11:27 PM, Jason Hellenthal > > wrote: >> >>> You'll want to not use up addresses in your pf.conf >>> >>> Block on default and then open up by definition of ports instead. Forge= t >>> the whole IPAddr thing and treat this as a roaming client firewall. >>> >>> >>> -- >>> Jason Hellenthal >>> Voice: 95.30.17.6/616 >>> JJH48-ARIN >>> >>> > On Mar 9, 2014, at 19:18, John-Mark Gurney wrote: >>> > >>> > Joe Nosay wrote this message on Sun, Mar 09, 2014 at 15:36 -0400: >>> >> 2. How do I compensate for the use of public access points when the = IP >>> >> addresses will always be different? >>> > >>> > it doesn't appear that pf has this ability, but it looks like ipfw >>> > has this, from ipfw(8): >>> > me matches any IP address configured on an interface >>> in the >>> > system. >>> > >>> > So, maybe switching to ipfw might be an option.. >>> > >>> > -- >>> > John-Mark Gurney Voice: +1 415 225 5579 >>> > >>> > "All that I will do, has been done, All that I have, has not." >>> > _______________________________________________ >>> > freebsd-net@freebsd.org mailing list >>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>> >> >> >> >> -- >> Ermal >> >> > > Has anyone thought about putting themselves in an environment similar to > mine- not everything- when it comes to networking? You would have to set > everything up with the following parameters: > 1. Because you are at more than one place, you cannot setup wlanX or the > wlandev in rc.conf. They must always be created after booting and logging > in. > 2. Dhclient cannot be automatic because a public access area may have mor= e > than one available bssid for connecting. > 3. Since each public access will have different firewalls, streaming and > web services may not be able to be ran. > 4. A script would probably work better than static settings in this case. > > > Apologies. I am trying different ways of setting up jailed networking. After setting up the sysctl variables and chrooting into the jail, the difficulty comes in connecting. I am going to try what is suggested by the ezjail page and see if that helps. Stepping back, I see that I should enable wlan0 to be created in rc.conf but not enable dhcp on it. Would that be the proper thing to do? From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 17:58:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E6E5440 for ; Tue, 11 Mar 2014 17:58:49 +0000 (UTC) Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19546663 for ; Tue, 11 Mar 2014 17:58:49 +0000 (UTC) Received: by mail-ob0-f177.google.com with SMTP id wo20so8513586obc.22 for ; Tue, 11 Mar 2014 10:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=4ilOenvPPQA83erPFjWqB4xZBvvlaBt642FBa8Fm/tI=; b=hOlr9FX5ojJM/tuWfupjFbdXwomnL4PiGpiXtfAysImUecvcNpTBJvTnobQZwdi6yf aqGG20yr1KEG8W73C6jd3C+N6pCD2FR2gVJ8atuq1jtvXGO8mdY7SvONMMIHaqZRaxh6 oJFPnE4ne1pwpsPRZuGdyru12XXZXN/ZtCyYN89+mggxRIU0KzEr0oaDyhGZP47syv8q uD9Jp0hMKUBbblwZaiKZnldrazodZqSA9P6zgjZeKnbDwxdW9HeJrA2EdXsGVt2Ex0aU PFVzwOE3b2m3egpdGFD/SD8Tsox/QA/z+psNQzMX5UrC6+BpDrVAVUNii2dgOWd+SWyV sBzA== MIME-Version: 1.0 X-Received: by 10.60.39.129 with SMTP id p1mr2844661oek.54.1394560728413; Tue, 11 Mar 2014 10:58:48 -0700 (PDT) Received: by 10.182.74.4 with HTTP; Tue, 11 Mar 2014 10:58:48 -0700 (PDT) Date: Tue, 11 Mar 2014 10:58:48 -0700 Message-ID: Subject: socket receive buffer size & window updates From: Vijay Singh To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 17:58:49 -0000 The socket option handler currently doesn't prevent connecting or connected sockets from changing their receive buffer sizes. In particular, I ran into a an application that sets the receive buffer size lower than what it originally was. In tcp_output(), if no data is being sent, there is code that is trying to decide if a window update is needed. If the socket receive buffer size was reduced after a larger window was already advertized, or perhaps even when there is data in the receive buffer, it seems to me that the computation in 592 could go -ve, and be interpreted as a large window update. I was led to this issue after observing in packet traces that duplicate FINs were being sent on close. I tracked it down to this check. Should this be changed to a check like such? if (adv >= oldwin) adv -= oldwin; else adv = 0; 586 long adv ; 587 int oldwin; 588 589 adv = min (recwin, (long)TCP_MAXWIN << tp ->rcv_scale); 590 if (SEQ_GT (tp ->rcv_adv, tp ->rcv_nxt )) { 591 oldwin = (tp ->rcv_adv - tp ->rcv_nxt ); 592 adv -= oldwin; 593 } else 594 oldwin = 0; -vijay From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 18:25:00 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F411711; Tue, 11 Mar 2014 18:25:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 248EEC86; Tue, 11 Mar 2014 18:25:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2BIOxFK064455; Tue, 11 Mar 2014 18:24:59 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2BIOxMG064454; Tue, 11 Mar 2014 18:24:59 GMT (envelope-from edavis) Date: Tue, 11 Mar 2014 18:24:59 GMT Message-Id: <201403111824.s2BIOxMG064454@freefall.freebsd.org> To: mike_karels@mcafee.com, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/174850: [bxe] [patch] bxe driver does not receive multicasts X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 18:25:00 -0000 Synopsis: [bxe] [patch] bxe driver does not receive multicasts State-Changed-From-To: open->feedback State-Changed-By: edavis State-Changed-When: Tue Mar 11 18:16:37 UTC 2014 State-Changed-Why: The bxe driver has been re-written and committed. This multicast issue should no longer exit. Please verify on latest head or the tip of any stable branch. http://www.freebsd.org/cgi/query-pr.cgi?pr=174850 From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 18:27:57 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4EFE7C5; Tue, 11 Mar 2014 18:27:57 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 889BACA4; Tue, 11 Mar 2014 18:27:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2BIRvIP064569; Tue, 11 Mar 2014 18:27:57 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2BIRvGO064568; Tue, 11 Mar 2014 18:27:57 GMT (envelope-from edavis) Date: Tue, 11 Mar 2014 18:27:57 GMT Message-Id: <201403111827.s2BIRvGO064568@freefall.freebsd.org> To: mike_karels@mcafee.com, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/174851: [bxe] [patch] UDP checksum offload is wrong in bxe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 18:27:57 -0000 Synopsis: [bxe] [patch] UDP checksum offload is wrong in bxe driver State-Changed-From-To: open->feedback State-Changed-By: edavis State-Changed-When: Tue Mar 11 18:26:37 UTC 2014 State-Changed-Why: The bxe driver has been re-written and committed. UDP checksums are working properly. Please verify on latest head or the tip of any stable branch. http://www.freebsd.org/cgi/query-pr.cgi?pr=174851 From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 18:30:55 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB2E4897; Tue, 11 Mar 2014 18:30:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BF28BCCF; Tue, 11 Mar 2014 18:30:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2BIUtGo064773; Tue, 11 Mar 2014 18:30:55 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2BIUruL064772; Tue, 11 Mar 2014 18:30:53 GMT (envelope-from edavis) Date: Tue, 11 Mar 2014 18:30:53 GMT Message-Id: <201403111830.s2BIUruL064772@freefall.freebsd.org> To: snelius@tsu.ru, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/165526: [bxe] UDP packets checksum calculation whithin if_bxe driver is not correct may be. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 18:30:56 -0000 Synopsis: [bxe] UDP packets checksum calculation whithin if_bxe driver is not correct may be. State-Changed-From-To: open->closed State-Changed-By: edavis State-Changed-When: Tue Mar 11 18:28:35 UTC 2014 State-Changed-Why: The bxe driver has been re-written and committed. UDP checksums are working properly in the new version. http://www.freebsd.org/cgi/query-pr.cgi?pr=165526 From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 18:33:13 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35E71977; Tue, 11 Mar 2014 18:33:13 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 09DE8D76; Tue, 11 Mar 2014 18:33:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2BIXCt3067384; Tue, 11 Mar 2014 18:33:12 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2BIXC25067383; Tue, 11 Mar 2014 18:33:12 GMT (envelope-from edavis) Date: Tue, 11 Mar 2014 18:33:12 GMT Message-Id: <201403111833.s2BIXC25067383@freefall.freebsd.org> To: sr@swisscenter.com, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/180775: [bxe] if_bxe driver broken with Broadcom BCM57711 cards X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 18:33:13 -0000 Synopsis: [bxe] if_bxe driver broken with Broadcom BCM57711 cards State-Changed-From-To: open->feedback State-Changed-By: edavis State-Changed-When: Tue Mar 11 18:31:40 UTC 2014 State-Changed-Why: The bxe driver has been re-written and committed. Please verify this issue on latest head or the tip of any stable branch. We could not reproduce this issue on 57711 using the new bxe driver. http://www.freebsd.org/cgi/query-pr.cgi?pr=180775 From owner-freebsd-net@FreeBSD.ORG Tue Mar 11 18:39:04 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87D41ABB; Tue, 11 Mar 2014 18:39:04 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5C615DBE; Tue, 11 Mar 2014 18:39:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2BId44H067589; Tue, 11 Mar 2014 18:39:04 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2BId48f067588; Tue, 11 Mar 2014 18:39:04 GMT (envelope-from edavis) Date: Tue, 11 Mar 2014 18:39:04 GMT Message-Id: <201403111839.s2BId48f067588@freefall.freebsd.org> To: edavis@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/183666: Compiled-in bxe(4) breaks kgzip(1) kernel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 18:39:04 -0000 Synopsis: Compiled-in bxe(4) breaks kgzip(1) kernel Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: edavis Responsible-Changed-When: Tue Mar 11 18:38:38 UTC 2014 Responsible-Changed-Why: changed responsible to -net http://www.freebsd.org/cgi/query-pr.cgi?pr=183666 From owner-freebsd-net@FreeBSD.ORG Wed Mar 12 20:40:05 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DEF541F; Wed, 12 Mar 2014 20:40:05 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23580E2F; Wed, 12 Mar 2014 20:40:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2CKe50M082621; Wed, 12 Mar 2014 20:40:05 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2CKe4N0082620; Wed, 12 Mar 2014 20:40:04 GMT (envelope-from edavis) Date: Wed, 12 Mar 2014 20:40:04 GMT Message-Id: <201403122040.s2CKe4N0082620@freefall.freebsd.org> To: mike_karels@mcafee.com, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/174851: [bxe] [patch] UDP checksum offload is wrong in bxe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Mar 2014 20:40:05 -0000 Synopsis: [bxe] [patch] UDP checksum offload is wrong in bxe driver State-Changed-From-To: feedback->closed State-Changed-By: edavis State-Changed-When: Wed Mar 12 20:39:00 UTC 2014 State-Changed-Why: fixed in new driver version http://www.freebsd.org/cgi/query-pr.cgi?pr=174851 From owner-freebsd-net@FreeBSD.ORG Wed Mar 12 20:40:31 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BE0B4F1; Wed, 12 Mar 2014 20:40:31 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 616BAE49; Wed, 12 Mar 2014 20:40:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2CKeVYf082669; Wed, 12 Mar 2014 20:40:31 GMT (envelope-from edavis@freefall.freebsd.org) Received: (from edavis@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2CKeV14082668; Wed, 12 Mar 2014 20:40:31 GMT (envelope-from edavis) Date: Wed, 12 Mar 2014 20:40:31 GMT Message-Id: <201403122040.s2CKeV14082668@freefall.freebsd.org> To: mike_karels@mcafee.com, edavis@FreeBSD.org, freebsd-net@FreeBSD.org From: edavis@FreeBSD.org Subject: Re: kern/174850: [bxe] [patch] bxe driver does not receive multicasts X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Mar 2014 20:40:31 -0000 Synopsis: [bxe] [patch] bxe driver does not receive multicasts State-Changed-From-To: feedback->closed State-Changed-By: edavis State-Changed-When: Wed Mar 12 20:40:19 UTC 2014 State-Changed-Why: fixed in new driver version http://www.freebsd.org/cgi/query-pr.cgi?pr=174850 From owner-freebsd-net@FreeBSD.ORG Thu Mar 13 16:22:02 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3285996D; Thu, 13 Mar 2014 16:22:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0829FDC6; Thu, 13 Mar 2014 16:22:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2DGM1OA074446; Thu, 13 Mar 2014 16:22:01 GMT (envelope-from asomers@freefall.freebsd.org) Received: (from asomers@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2DGM1fs074445; Thu, 13 Mar 2014 16:22:01 GMT (envelope-from asomers) Date: Thu, 13 Mar 2014 16:22:01 GMT Message-Id: <201403131622.s2DGM1fs074445@freefall.freebsd.org> To: kes-kes@yandex.ru, asomers@FreeBSD.org, freebsd-net@FreeBSD.org From: asomers@FreeBSD.org Subject: Re: conf/132851: [patch] rc.conf(5): allow to setfib(1) for service running from rc.conf X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 16:22:02 -0000 Synopsis: [patch] rc.conf(5): allow to setfib(1) for service running from rc.conf State-Changed-From-To: open->closed State-Changed-By: asomers State-Changed-When: Thu Mar 13 16:22:01 UTC 2014 State-Changed-Why: Independently fixed by hrs in revision 242184 on 27-October-2012 http://www.freebsd.org/cgi/query-pr.cgi?pr=132851 From owner-freebsd-net@FreeBSD.ORG Thu Mar 13 16:25:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E67AACC for ; Thu, 13 Mar 2014 16:25:43 +0000 (UTC) Received: from mail-ve0-x230.google.com (mail-ve0-x230.google.com [IPv6:2607:f8b0:400c:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 41F10E22 for ; Thu, 13 Mar 2014 16:25:43 +0000 (UTC) Received: by mail-ve0-f176.google.com with SMTP id cz12so1346824veb.35 for ; Thu, 13 Mar 2014 09:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=mUuRCHRDv11SUNen5Koqzsxw19uoMqp7QIS6ctZWi1w=; b=lBgSYI37wXqq90D/iKs2xVCkOzqDzm+E8aQbAsLvePyARMM35f2irbbLu5yjp8IeN1 3bvaPhN+b4Ne6SSveuT4Etr/KgMFpCK7waR1y/ptlvkHOpapIznBPHxDeQafHEVkdpiD GaoQCknN6TfCfysq8FLhb1LDpqMLNgVrlUAGb7Z4onfsI3v3RtCcGJCpyrCvRW9Nw2R7 ZaHLQUT/d2byDiD1ZeCZxvWkss5yghieiedL/ZWmYWvaVpfeugIuqVonOGnF5dXd3dbu XajlI2v3d0u4ynfPTLBXEyWN09EvqoEUPSND00u+3jdOcQJlZoH9Ha2SKXqXrejgiDBa 1jhA== MIME-Version: 1.0 X-Received: by 10.58.207.34 with SMTP id lt2mr196295vec.49.1394727942433; Thu, 13 Mar 2014 09:25:42 -0700 (PDT) Received: by 10.52.139.66 with HTTP; Thu, 13 Mar 2014 09:25:42 -0700 (PDT) Date: Thu, 13 Mar 2014 10:25:42 -0600 Message-ID: Subject: NMap scans extremely slow on FreeBSD 10, possibly BIOCIMMEDIATE From: Ken Harvey To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 16:25:43 -0000 I am attempting to troubleshoot a problem with nmap on FreeBSD 10. The issue that I am having is that when running nmap -O 10.1.2.3 it is taking around 220 seconds to complete. While if I run that same command using Windows or Linux the command completes in around 2.3 seconds. Currently FreeBSD is 100 times slower for nmap scans then Linux or Windows. After reading through the forums and the mailing list archives I think the problem may be associated with BIOCIMMEDIATE. bpf is waiting for the buffer to fill, or for the ttl to expire before it processes the packets, rather than processing them upon receiving them. I may be incorrect in this theory, but I am unsure how to verify plausibility. While looking at /usr/includes/net/bpf.h I do see that BIOCIMMEDIATE is implemented. So I am now wondering if nmap or libpcap is sending the proper switch to bpf for it to enable BIOCIMMEDIATE. Is there a way for me to verify whether BIOCIMMEDIATE is being called in bpf? Is there a better way for me to try and troubleshoot this issue? You can view my forum post at https://forums.freebsd.org/viewtopic.php?f=7&t=45286 It has a little bit more detail then this post, but it also has a lot of my random troubleshooting steps as well. Currently I am a little over my head, and I am unsure how to or where to begin troubleshooting this problem. While I do want to get this issue resolved, I also would like to learn how to troubleshoot issues like these. Any help or guidance would be greatly appreciated. From owner-freebsd-net@FreeBSD.ORG Fri Mar 14 12:18:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F1A7463 for ; Fri, 14 Mar 2014 12:18:22 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A694E62 for ; Fri, 14 Mar 2014 12:18:21 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WOR44-0001zd-W8 for freebsd-net@freebsd.org; Fri, 14 Mar 2014 13:18:17 +0100 Received: from col74-1-88-183-113-97.fbx.proxad.net ([88.183.113.97]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Mar 2014 13:18:16 +0100 Received: from jcharbon by col74-1-88-183-113-97.fbx.proxad.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 14 Mar 2014 13:18:16 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Fri, 14 Mar 2014 13:18:06 +0100 Lines: 441 Message-ID: <5322F37E.5040700@verisign.com> References: <20140306215743.GB47921@funkthat.com> <5319BEE7.607@verisign.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------090708080906090808070008" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: col74-1-88-183-113-97.fbx.proxad.net User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 In-Reply-To: <5319BEE7.607@verisign.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 12:18:22 -0000 This is a multi-part message in MIME format. --------------090708080906090808070008 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi John, On 07/03/14 13:43, Julien Charbon wrote: > On 06/03/14 22:57, John-Mark Gurney wrote: >> Julien Charbon wrote this message on Thu, Feb 27, 2014 at 11:32 +0100: >>> [...] >>> Any thoughts on this particular behavior? >> >> One thing that I noticed is that you now lock/unlock the tw and inp >> lock a >> lot... Have you thought about grabing the TW lock once, grabbing >> some/all >> of the ones necessary to process and then process them in a second step? >> If the bulk processing is still an issue, then you could process them in >> blocks of 50 or so, that way the number of lock/unlock cycles is >> reduced... > > First thanks, feedback are highly valuable to us. In first place, I > indeed tried a kind of bulk processing enforcement with something like: > > tcp_tw_2msl_scan() { > > struct tcptw *tw; > int i, end = 0, count = 100; > > for (;;) { > INP_INFO_WLOCK(&V_tcbinfo); > for (i = 0; i < count; ++i) { > tw = TAILQ_FIRST(&V_twq_2msl); > if (tw == NULL || (tw->tw_time - ticks) > 0)) { > end = 1; > break; > } > INP_WLOCK(tw->tw_inpcb); > tcp_twclose(tw, 0); > } > if (end) > break; > INP_INFO_WUNLOCK(&V_tcbinfo); > } > return (NULL); > } > > And I got best result with 'count' set to 1, this led us to current > proposed patch. [...] I would like to continue the discussion about tcp_tw_2msl_scan(). The proposed patch makes tcp_tw_2msl_scan() less disturbing for the TCP stack (another way to say it makes tcp_tw_2msl_scan() less efficient to prioritize other TCP related tasks), and continuing on this way I have tested below change on top of previous patch: diff --git a/sys/netinet/tcp_timewait.c b/sys/netinet/tcp_timewait.c index 0230b88..da79120 100644 --- a/sys/netinet/tcp_timewait.c +++ b/sys/netinet/tcp_timewait.c @@ -728,7 +728,7 @@ tcp_tw_2msl_scan(void) TW_RUNLOCK(V_tw_lock); /* Close timewait state */ - INP_INFO_WLOCK(&V_tcbinfo); + if(INP_INFO_TRY_WLOCK(&V_tcbinfo)) { TW_WLOCK(V_tw_lock); if(tw_pcbrele(tw)) @@ -740,5 +740,9 @@ tcp_tw_2msl_scan(void) INP_WLOCK(tw->tw_inpcb); tcp_twclose(tw, 0); INP_INFO_WUNLOCK(&V_tcbinfo); + } else { + /* INP_INFO lock is busy, continue later */ + break; + } } } Basically, it changes tcp_tw_2msl_scan() purpose from: - Let's clean up all expired tw objects in a row. to: - Let's clean up expired tw objects as long as the INP_INFO lock is not busy. (See joined tw-lock-v2.patch). And corresponding performance results being: Maximum short-lived TCP connection rate without dropping a single packet: - FreeBSD 10.0-RELEASE: 40.0k - FreeBSD 10.0 + tw patch-v1: 52.8k (+32%) - FreeBSD 10.0 + tw patch-v2: 56.8k (+42%) Thus, a significant improvement and it makes the tcp_tw_2msl_scan() purpose clear. Moreover, it seems that two members of the struct tcptw are never used: 'tw_pspare' and 'tw_spare': struct tcptw { [...] u_int t_starttime; int tw_time; TAILQ_ENTRY(tcptw) tw_2msl; void *tw_pspare; /* TCP_SIGNATURE */ u_int *tw_spare; /* TCP_SIGNATURE */ u_int tw_refcount; /* refcount */ }; I found no references of these members anywhere, even when setting the TCP_SIGNATURE kernel option the kernel builds and runs just fine. As the tcptw struct is private to kernel and its size should be as small as possible, I would propose to remove them. Joined tw-clock-v2.patch which includes: - Use TRY_WLOCK (Performance improvements) - Fix comment (After John-Mark Gurney's review) - Two unused fields removed (After Mike Bentkofsky's review) -- Julien --------------090708080906090808070008 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="tw-lock-v2.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tw-lock-v2.patch" diff --git a/sys/netinet/tcp_timer.c b/sys/netinet/tcp_timer.c index bde7503..b45a9ea 100644 --- a/sys/netinet/tcp_timer.c +++ b/sys/netinet/tcp_timer.c @@ -144,9 +144,7 @@ tcp_slowtimo(void) VNET_LIST_RLOCK_NOSLEEP(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); - INP_INFO_WLOCK(&V_tcbinfo); - (void) tcp_tw_2msl_scan(0); - INP_INFO_WUNLOCK(&V_tcbinfo); + tcp_tw_2msl_scan(); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); diff --git a/sys/netinet/tcp_timer.h b/sys/netinet/tcp_timer.h index 3115fb3..c04723a 100644 --- a/sys/netinet/tcp_timer.h +++ b/sys/netinet/tcp_timer.h @@ -178,7 +178,8 @@ extern int tcp_fast_finwait2_recycle; void tcp_timer_init(void); void tcp_timer_2msl(void *xtp); struct tcptw * - tcp_tw_2msl_scan(int _reuse); /* XXX temporary */ + tcp_tw_2msl_reuse(void); /* XXX temporary? */ +void tcp_tw_2msl_scan(void); void tcp_timer_keep(void *xtp); void tcp_timer_persist(void *xtp); void tcp_timer_rexmt(void *xtp); diff --git a/sys/netinet/tcp_timewait.c b/sys/netinet/tcp_timewait.c index 7e6128b..afaf5f9 100644 --- a/sys/netinet/tcp_timewait.c +++ b/sys/netinet/tcp_timewait.c @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include @@ -98,13 +99,59 @@ static int maxtcptw; * The timed wait queue contains references to each of the TCP sessions * currently in the TIME_WAIT state. The queue pointers, including the * queue pointers in each tcptw structure, are protected using the global - * tcbinfo lock, which must be held over queue iteration and modification. + * timewait lock, which must be held over queue iteration and modification. */ static VNET_DEFINE(TAILQ_HEAD(, tcptw), twq_2msl); #define V_twq_2msl VNET(twq_2msl) -static void tcp_tw_2msl_reset(struct tcptw *, int); -static void tcp_tw_2msl_stop(struct tcptw *); +/* Global timewait lock */ +static VNET_DEFINE(struct rwlock, tw_lock); +#define V_tw_lock VNET(tw_lock) + +#define TW_LOCK_INIT(tw, d) rw_init_flags(&(tw), (d), 0) +#define TW_LOCK_DESTROY(tw) rw_destroy(&(tw)) +#define TW_RLOCK(tw) rw_rlock(&(tw)) +#define TW_WLOCK(tw) rw_wlock(&(tw)) +#define TW_RUNLOCK(tw) rw_runlock(&(tw)) +#define TW_WUNLOCK(tw) rw_wunlock(&(tw)) +#define TW_LOCK_ASSERT(tw) rw_assert(&(tw), RA_LOCKED) +#define TW_RLOCK_ASSERT(tw) rw_assert(&(tw), RA_RLOCKED) +#define TW_WLOCK_ASSERT(tw) rw_assert(&(tw), RA_WLOCKED) +#define TW_UNLOCK_ASSERT(tw) rw_assert(&(tw), RA_UNLOCKED) + +/* + * tw_pcbref() bumps the reference count on an tw in order to maintain + * stability of an tw pointer despite the tw lock being released. + */ +static void +tw_pcbref(struct tcptw *tw) +{ + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); + refcount_acquire(&tw->tw_refcount); +} + +/* + * Drop a refcount on an tw elevated using tw_pcbref(). Return + * the tw lock released. + */ +static int +tw_pcbrele(struct tcptw *tw) +{ + TW_WLOCK_ASSERT(V_tw_lock); + KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); + + if (!refcount_release(&tw->tw_refcount)) { + TW_WUNLOCK(V_tw_lock); + return (0); + } + + uma_zfree(V_tcptw_zone, tw); + TW_WUNLOCK(V_tw_lock); + return (1); +} + +static void tcp_tw_2msl_reset(struct tcptw *, int ream); +static void tcp_tw_2msl_stop(struct tcptw *, int reuse); static int tcptw_auto_size(void) @@ -171,6 +218,7 @@ tcp_tw_init(void) else uma_zone_set_max(V_tcptw_zone, maxtcptw); TAILQ_INIT(&V_twq_2msl); + TW_LOCK_INIT(V_tw_lock, "tcptw"); } #ifdef VIMAGE @@ -179,11 +227,14 @@ tcp_tw_destroy(void) { struct tcptw *tw; - INP_INFO_WLOCK(&V_tcbinfo); - while((tw = TAILQ_FIRST(&V_twq_2msl)) != NULL) - tcp_twclose(tw, 0); - INP_INFO_WUNLOCK(&V_tcbinfo); + TW_WLOCK(V_tw_lock); + while((tw = TAILQ_FIRST(&V_twq_2msl)) != NULL) { + tcp_twclose(tw, 0, 1); + TW_WLOCK(V_tw_lock); + } + TW_WUNLOCK(V_tw_lock); + TW_LOCK_DESTROY(V_tw_lock); uma_zdestroy(V_tcptw_zone); } #endif @@ -204,7 +255,7 @@ tcp_twstart(struct tcpcb *tp) int isipv6 = inp->inp_inc.inc_flags & INC_ISIPV6; #endif - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* tcp_tw_2msl_reset(). */ + INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); if (V_nolocaltimewait) { @@ -229,7 +280,7 @@ tcp_twstart(struct tcpcb *tp) tw = uma_zalloc(V_tcptw_zone, M_NOWAIT); if (tw == NULL) { - tw = tcp_tw_2msl_scan(1); + tw = tcp_tw_2msl_reuse(); if (tw == NULL) { tp = tcp_close(tp); if (tp != NULL) @@ -238,6 +289,7 @@ tcp_twstart(struct tcpcb *tp) } } tw->tw_inpcb = inp; + refcount_init(&tw->tw_refcount, 1); /* * Recover last window size sent. @@ -356,7 +408,6 @@ tcp_twcheck(struct inpcb *inp, struct tcpopt *to, struct tcphdr *th, int thflags; tcp_seq seq; - /* tcbinfo lock required for tcp_twclose(), tcp_tw_2msl_reset(). */ INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); @@ -458,11 +509,11 @@ tcp_twclose(struct tcptw *tw, int reuse) inp = tw->tw_inpcb; KASSERT((inp->inp_flags & INP_TIMEWAIT), ("tcp_twclose: !timewait")); KASSERT(intotw(inp) == tw, ("tcp_twclose: inp_ppcb != tw")); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* tcp_tw_2msl_stop(). */ + INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); tw->tw_inpcb = NULL; - tcp_tw_2msl_stop(tw); + tcp_tw_2msl_stop(tw, reuse); inp->inp_ppcb = NULL; in_pcbdrop(inp); @@ -494,11 +545,6 @@ tcp_twclose(struct tcptw *tw, int reuse) } else in_pcbfree(inp); TCPSTAT_INC(tcps_closed); - crfree(tw->tw_cred); - tw->tw_cred = NULL; - if (reuse) - return; - uma_zfree(V_tcptw_zone, tw); } int @@ -616,34 +662,87 @@ tcp_tw_2msl_reset(struct tcptw *tw, int rearm) INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tw->tw_inpcb); + + TW_WLOCK(V_tw_lock); if (rearm) TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); tw->tw_time = ticks + 2 * tcp_msl; TAILQ_INSERT_TAIL(&V_twq_2msl, tw, tw_2msl); + TW_WUNLOCK(V_tw_lock); } static void -tcp_tw_2msl_stop(struct tcptw *tw) +tcp_tw_2msl_stop(struct tcptw *tw, int reuse) { INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + + TW_WLOCK(V_tw_lock); TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); + crfree(tw->tw_cred); + tw->tw_cred = NULL; + + if (!reuse) { + tw_pcbrele(tw); + return; + } + + TW_WUNLOCK(V_tw_lock); } struct tcptw * -tcp_tw_2msl_scan(int reuse) +tcp_tw_2msl_reuse(void) { - struct tcptw *tw; INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + + struct tcptw *tw; + + TW_WLOCK(V_tw_lock); + tw = TAILQ_FIRST(&V_twq_2msl); + if (tw == NULL) { + TW_WUNLOCK(V_tw_lock); + return NULL; + } + TW_WUNLOCK(V_tw_lock); + + INP_WLOCK(tw->tw_inpcb); + tcp_twclose(tw, 1); + + return (tw); +} + +void +tcp_tw_2msl_scan(void) +{ + + struct tcptw *tw; for (;;) { + TW_RLOCK(V_tw_lock); tw = TAILQ_FIRST(&V_twq_2msl); - if (tw == NULL || (!reuse && (tw->tw_time - ticks) > 0)) + if (tw == NULL || ((tw->tw_time - ticks) > 0)) { + TW_RUNLOCK(V_tw_lock); + break; + } + tw_pcbref(tw); + TW_RUNLOCK(V_tw_lock); + + /* Close timewait state */ + if(INP_INFO_TRY_WLOCK(&V_tcbinfo)) { + + TW_WLOCK(V_tw_lock); + if(tw_pcbrele(tw)) + continue; + + KASSERT(tw->tw_inpcb != NULL, + ("%s: tw->tw_inpcb == NULL", __func__)); + + INP_WLOCK(tw->tw_inpcb); + tcp_twclose(tw, 0); + INP_INFO_WUNLOCK(&V_tcbinfo); + } else { + /* INP_INFO lock is busy, continue later */ break; - INP_WLOCK(tw->tw_inpcb); - tcp_twclose(tw, reuse); - if (reuse) - return (tw); + } } - return (NULL); } diff --git a/sys/netinet/tcp_var.h b/sys/netinet/tcp_var.h index e3197e5..f13c1f4 100644 --- a/sys/netinet/tcp_var.h +++ b/sys/netinet/tcp_var.h @@ -353,8 +353,7 @@ struct tcptw { u_int t_starttime; int tw_time; TAILQ_ENTRY(tcptw) tw_2msl; - void *tw_pspare; /* TCP_SIGNATURE */ - u_int *tw_spare; /* TCP_SIGNATURE */ + u_int tw_refcount; /* refcount */ }; #define intotcpcb(ip) ((struct tcpcb *)(ip)->inp_ppcb) --------------090708080906090808070008-- From owner-freebsd-net@FreeBSD.ORG Fri Mar 14 15:31:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4722B3BC for ; Fri, 14 Mar 2014 15:31:53 +0000 (UTC) Received: from mail-oa0-x232.google.com (mail-oa0-x232.google.com [IPv6:2607:f8b0:4003:c02::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0E4509C4 for ; Fri, 14 Mar 2014 15:31:53 +0000 (UTC) Received: by mail-oa0-f50.google.com with SMTP id i7so2764306oag.23 for ; Fri, 14 Mar 2014 08:31:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=wRp/WUcKzNL/f5mlbZcSOEjjtgkmGnIzkyncqu3mrLI=; b=CFN8NRzJpbVgqW87nS4yf1WO7ZEqeyOL2TR+rL9ud2R2dSJJsKPqh/FbR7HthbgfB/ 6+ez/uDBSGJc8KbDncXb4tGzHT+tR4iZKMBWi5u6RKZpxBK3RZiFQ4sY6vv0HuBch2xM 1Sjd8xQNpv4eA1IjZHT0wFUvO4rLBl28WArhIsqV7oyKGZLq0R8SVgx94JL/4CiHDBFr QFTm27AF0n1oWEaJFnQ5DAp8+t8Z1rOSqO4lOUMGRh+tenqgOODHl7QFLe1T6XPqKUyg +V6duWzeNiwteRUjCtfhs23p1IvJtWenELejPLX8pbQuWT7MubQysgMaCJVtARaeVdgg iLnA== MIME-Version: 1.0 X-Received: by 10.182.131.170 with SMTP id on10mr7420541obb.2.1394811112376; Fri, 14 Mar 2014 08:31:52 -0700 (PDT) Received: by 10.76.72.5 with HTTP; Fri, 14 Mar 2014 08:31:52 -0700 (PDT) In-Reply-To: <20140307024724.T75313@sola.nimnet.asn.au> References: <531771C8.1040207@yandex.ru> <20140306145231.Q75313@sola.nimnet.asn.au> <20140307024724.T75313@sola.nimnet.asn.au> Date: Fri, 14 Mar 2014 16:31:52 +0100 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: Ian Smith , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 15:31:53 -0000 ... snip ... > Ah. Well it was good to see the rules listed anyway, always helps. > > > Was the count rules something like: > > > > 00001 901 46132 skipto 63000 ip from table(1) to any in recv > > table(8) > > > > ... same as before ... > > > > 63500 895 45844 count log logamount 100 ip from table(1) to > any in > > recv table(8) > > > > 64000 0 0 fwd tablearg ip from table(12) to any > > > > 64500 895 45844 count log logamount 100 ip from table(1) to > any in > > recv table(8) > > 65535 3849164 1234591611 allow ip from any to any > > > > So everything looks nice enough, but still no go. Interestingly enough, > > since this is a production machine I can't fool around to much ( that's > why > > it took a while to get the above results ) I tried to reproduce this on > > another machine. Lo and behold, I couldn't. There it worked with the > > skipto-version. I'll investigate this and see if I can find any > differences. > > I'm wondering what's happening on the outbound path, most of your rules > handle inbound (to kernel) and it seems that rule 65535 deals with most > outbound, except those specifically acting on both paths. > So do I :) > > Maybe try adding to the above: > ipfw add 63510 count log ip from table(1) to any out recv table(8) > and > ipfw add 64510 count log ip from table(1) to any out recv table(8) > > which should catch them on the outbound path before the default rule; > outbound packets have both receive and transmit interfaces available. > > They never get that far :( Those rules log nothing. So I see the packet coming in, triggering the skipto, triggering the count log ... in recv but not the count log ... out. > I guess you're sure that these packets are actually going out to some > external address, not a localhost or alias address (which of course are > received and not sent out anywhere)? > Yes, when the go through they go to external address, which is in the routing table. > > I guess the above new log lines should show the interface (if any) > these packets are leaving on, after routing (maybe a routing issue?) > > Good luck hunting. If in doubt, throw ever more logging at it! :) And > please let the list know if you solve it - or not! > > cheers, Ian > To make it even more interesting, it tested this on another machine, where I'm unable to reproduce this "problem". I'm thinking that a reboot might help, but this is kind of fascinating so I would like to actually find the cause. I will investigate further. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Fri Mar 14 17:00:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E941834F for ; Fri, 14 Mar 2014 17:00:43 +0000 (UTC) Received: from lagoon.freebsd.lublin.pl (lagoon.freebsd.lublin.pl [IPv6:2a02:2928:a::3]) by mx1.freebsd.org (Postfix) with ESMTP id 82D38304 for ; Fri, 14 Mar 2014 17:00:43 +0000 (UTC) Received: from [10.66.254.3] (mgmt.nette.pl [193.138.118.12]) by lagoon.freebsd.lublin.pl (Postfix) with ESMTPSA id 45C9447B6A3; Fri, 14 Mar 2014 18:00:26 +0100 (CET) Message-ID: <532335B6.7070105@frasunek.com> Date: Fri, 14 Mar 2014 18:00:38 +0100 From: Przemyslaw Frasunek Organization: frasunek.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Mike Tancsa , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> <531C517F.5020506@frasunek.com> <531C5877.8050401@sentex.net> In-Reply-To: <531C5877.8050401@sentex.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 17:00:44 -0000 >> FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 >> enabled. > What sort of uptime have you seen with ipv6 enabled ? Now it's 19 days and still no crash occurred. From owner-freebsd-net@FreeBSD.ORG Fri Mar 14 17:13:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D1F4A7E for ; Fri, 14 Mar 2014 17:13:08 +0000 (UTC) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3F211647 for ; Fri, 14 Mar 2014 17:13:08 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.8/8.14.8) with ESMTP id s2EHD32c054145; Fri, 14 Mar 2014 13:13:04 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <5323389B.9060109@sentex.net> Date: Fri, 14 Mar 2014 13:12:59 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Przemyslaw Frasunek , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> <531C517F.5020506@frasunek.com> <531C5877.8050401@sentex.net> <532335B6.7070105@frasunek.com> In-Reply-To: <532335B6.7070105@frasunek.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 17:13:08 -0000 On 3/14/2014 1:00 PM, Przemyslaw Frasunek wrote: >>> FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 >>> enabled. >> What sort of uptime have you seen with ipv6 enabled ? > > Now it's 19 days and still no crash occurred. I would sometimes get a month on a box with ~ 500 users ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Sat Mar 15 12:01:40 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3656EA30 for ; Sat, 15 Mar 2014 12:01:40 +0000 (UTC) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id AC29E134 for ; Sat, 15 Mar 2014 12:01:39 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [198.74.231.63]) by cyrus.watson.org (Postfix) with ESMTPS id 80B5446B2A for ; Sat, 15 Mar 2014 08:01:31 -0400 (EDT) Date: Sat, 15 Mar 2014 12:01:31 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: net@FreeBSD.org Subject: RSS patch merged (was: svn commit: r263198 - in head/sys: amd64/conf conf net netinet netinet6 sys (fwd)) Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 12:01:40 -0000 FYI -- I've had this RSS work sitting as a patchset locally for several years, and it appears to be in use at a number of FreeBSD-based shops. I didn't merge it as I felt that a number of aspects were immature -- especially, the KPI/KBI to RSS-aware device drivers. However, it seems useful to commit it so those with local enhancements have a starting point to submit patches. Improvements on this work would be much appreciated! Although the below commit shows it being enabled by default in GENERIC, that was actually a merge error on my part; connection groups and RSS support remain disabled by default (something we might think about revisting prior to 11.0). Robert ---------- Forwarded message ---------- Date: Sat, 15 Mar 2014 00:57:50 +0000 (UTC) From: Robert Watson To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263198 - in head/sys: amd64/conf conf net netinet netinet6 sys Author: rwatson Date: Sat Mar 15 00:57:50 2014 New Revision: 263198 URL: http://svnweb.freebsd.org/changeset/base/263198 Log: Several years after initial development, merge prototype support for linking NIC Receive Side Scaling (RSS) to the network stack's connection-group implementation. This prototype (and derived patches) are in use at Juniper and several other FreeBSD-using companies, so despite some reservations about its maturity, merge the patch to the base tree so that it can be iteratively refined in collaboration rather than maintained as a set of gradually diverging patch sets. (1) Merge a software implementation of the Toeplitz hash specified in RSS implemented by David Malone. This is used to allow suitable pcbgroup placement of connections before the first packet is received from the NIC. Software hashing is generally avoided, however, due to high cost of the hash on general-purpose CPUs. (2) In in_rss.c, maintain authoritative versions of RSS state intended to be pushed to each NIC, including keying material, hash algorithm/ configuration, and buckets. Provide software-facing interfaces to hash 2- and 4-tuples for IPv4 and IPv6 using both the RSS standardised Toeplitz and a 'naive' variation with a hash efficient in software but with poor distribution properties. Implement rss_m2cpuid()to be used by netisr and other load balancing code to look up the CPU on which an mbuf should be processed. (3) In the Ethernet link layer, allow netisr distribution using RSS as a source of policy as an alternative to source ordering; continue to default to direct dispatch (i.e., don't try and requeue packets for processing on the 'right' CPU if they arrive in a directly dispatchable context). (4) Allow RSS to control tuning of connection groups in order to align groups with RSS buckets. If a packet arrives on a protocol using connection groups, and contains a suitable hardware-generated hash, use that hash value to select the connection group for pcb lookup for both IPv4 and IPv6. If no hardware-generated Toeplitz hash is available, we fall back on regular PCB lookup risking contention rather than pay the cost of Toeplitz in software -- this is a less scalable but, at my last measurement, faster approach. As core counts go up, we may want to revise this strategy despite CPU overhead. Where device drivers suitably configure NICs, and connection groups / RSS are enabled, this should avoid both lock and line contention during connection lookup for TCP. This commit does not modify any device drivers to tune device RSS configuration to the global RSS configuration; patches are in circulation to do this for at least Chelsio T3 and Intel 1G/10G drivers. Currently, the KPI for device drivers is not particularly robust, nor aware of more advanced features such as runtime reconfiguration/rebalancing. This will hopefully prove a useful starting point for refinement. No MFC is scheduled as we will first want to nail down a more mature and maintainable KPI/KBI for device drivers. Sponsored by: Juniper Networks (original work) Sponsored by: EMC/Isilon (patch update and merge) Added: head/sys/netinet/in_rss.c (contents, props changed) head/sys/netinet/in_rss.h (contents, props changed) head/sys/netinet/toeplitz.c (contents, props changed) head/sys/netinet/toeplitz.h (contents, props changed) Modified: head/sys/amd64/conf/GENERIC head/sys/conf/files head/sys/conf/options head/sys/net/if_ethersubr.c head/sys/netinet/in_pcb.c head/sys/netinet/in_pcbgroup.c head/sys/netinet6/in6_pcb.c head/sys/netinet6/in6_pcbgroup.c head/sys/sys/priv.h Modified: head/sys/amd64/conf/GENERIC ============================================================================== --- head/sys/amd64/conf/GENERIC Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/amd64/conf/GENERIC Sat Mar 15 00:57:50 2014 (r263198) @@ -28,6 +28,8 @@ options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols +options PCBGROUP # Protocol control-block groups +options RSS # Receive-side scaling support options TCP_OFFLOAD # TCP offload options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/conf/files Sat Mar 15 00:57:50 2014 (r263198) @@ -3267,6 +3267,7 @@ netinet/in_pcb.c optional inet | inet6 netinet/in_pcbgroup.c optional inet pcbgroup | inet6 pcbgroup netinet/in_proto.c optional inet | inet6 netinet/in_rmx.c optional inet +netinet/in_rss.c optional inet rss | inet6 rss netinet/ip_divert.c optional inet ipdivert ipfirewall netinet/ip_ecn.c optional inet | inet6 netinet/ip_encap.c optional inet | inet6 @@ -3308,6 +3309,7 @@ netinet/tcp_syncache.c optional inet | netinet/tcp_timer.c optional inet | inet6 netinet/tcp_timewait.c optional inet | inet6 netinet/tcp_usrreq.c optional inet | inet6 +netinet/toeplitz.c optional inet rss | inet6 rss netinet/udp_usrreq.c optional inet | inet6 netinet/libalias/alias.c optional libalias inet | netgraph_nat inet netinet/libalias/alias_db.c optional libalias inet | netgraph_nat inet Modified: head/sys/conf/options ============================================================================== --- head/sys/conf/options Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/conf/options Sat Mar 15 00:57:50 2014 (r263198) @@ -427,6 +427,7 @@ PCBGROUP opt_pcbgroup.h PF_DEFAULT_TO_DROP opt_pf.h RADIX_MPATH opt_mpath.h ROUTETABLES opt_route.h +RSS opt_rss.h SLIP_IFF_OPTS opt_slip.h TCPDEBUG TCP_OFFLOAD opt_inet.h # Enable code to dispatch TCP offloading Modified: head/sys/net/if_ethersubr.c ============================================================================== --- head/sys/net/if_ethersubr.c Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/net/if_ethersubr.c Sat Mar 15 00:57:50 2014 (r263198) @@ -34,6 +34,7 @@ #include "opt_inet6.h" #include "opt_netgraph.h" #include "opt_mbuf_profiling.h" +#include "opt_rss.h" #include #include @@ -70,6 +71,7 @@ #include #include #include +#include #include #include #endif @@ -583,7 +585,22 @@ ether_input_internal(struct ifnet *ifp, /* * Ethernet input dispatch; by default, direct dispatch here regardless of - * global configuration. + * global configuration. However, if RSS is enabled, hook up RSS affinity + * so that when deferred or hybrid dispatch is enabled, we can redistribute + * load based on RSS. + * + * XXXRW: Would be nice if the ifnet passed up a flag indicating whether or + * not it had already done work distribution via multi-queue. Then we could + * direct dispatch in the event load balancing was already complete and + * handle the case of interfaces with different capabilities better. + * + * XXXRW: Sort of want an M_DISTRIBUTED flag to avoid multiple distributions + * at multiple layers? + * + * XXXRW: For now, enable all this only if RSS is compiled in, although it + * works fine without RSS. Need to characterise the performance overhead + * of the detour through the netisr code in the event the result is always + * direct dispatch. */ static void ether_nh_input(struct mbuf *m) @@ -596,8 +613,14 @@ static struct netisr_handler ether_nh = .nh_name = "ether", .nh_handler = ether_nh_input, .nh_proto = NETISR_ETHER, +#ifdef RSS + .nh_policy = NETISR_POLICY_CPU, + .nh_dispatch = NETISR_DISPATCH_DIRECT, + .nh_m2cpuid = rss_m2cpuid, +#else .nh_policy = NETISR_POLICY_SOURCE, .nh_dispatch = NETISR_DISPATCH_DIRECT, +#endif }; static void Modified: head/sys/netinet/in_pcb.c ============================================================================== --- head/sys/netinet/in_pcb.c Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/netinet/in_pcb.c Sat Mar 15 00:57:50 2014 (r263198) @@ -43,6 +43,7 @@ __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include "opt_pcbgroup.h" +#include "opt_rss.h" #include #include @@ -75,6 +76,7 @@ __FBSDID("$FreeBSD$"); #if defined(INET) || defined(INET6) #include #include +#include #include #include #include @@ -1820,7 +1822,7 @@ struct inpcb * in_pcblookup(struct inpcbinfo *pcbinfo, struct in_addr faddr, u_int fport, struct in_addr laddr, u_int lport, int lookupflags, struct ifnet *ifp) { -#if defined(PCBGROUP) +#if defined(PCBGROUP) && !defined(RSS) struct inpcbgroup *pcbgroup; #endif @@ -1829,7 +1831,17 @@ in_pcblookup(struct inpcbinfo *pcbinfo, KASSERT((lookupflags & (INPLOOKUP_RLOCKPCB | INPLOOKUP_WLOCKPCB)) != 0, ("%s: LOCKPCB not set", __func__)); -#if defined(PCBGROUP) + /* + * When not using RSS, use connection groups in preference to the + * reservation table when looking up 4-tuples. When using RSS, just + * use the reservation table, due to the cost of the Toeplitz hash + * in software. + * + * XXXRW: This policy belongs in the pcbgroup code, as in principle + * we could be doing RSS with a non-Toeplitz hash that is affordable + * in software. + */ +#if defined(PCBGROUP) && !defined(RSS) if (in_pcbgroup_enabled(pcbinfo)) { pcbgroup = in_pcbgroup_bytuple(pcbinfo, laddr, lport, faddr, fport); @@ -1856,16 +1868,27 @@ in_pcblookup_mbuf(struct inpcbinfo *pcbi ("%s: LOCKPCB not set", __func__)); #ifdef PCBGROUP - if (in_pcbgroup_enabled(pcbinfo)) { + /* + * If we can use a hardware-generated hash to look up the connection + * group, use that connection group to find the inpcb. Otherwise + * fall back on a software hash -- or the reservation table if we're + * using RSS. + * + * XXXRW: As above, that policy belongs in the pcbgroup code. + */ + if (in_pcbgroup_enabled(pcbinfo) && + !(M_HASHTYPE_TEST(m, M_HASHTYPE_NONE))) { pcbgroup = in_pcbgroup_byhash(pcbinfo, M_HASHTYPE_GET(m), m->m_pkthdr.flowid); if (pcbgroup != NULL) return (in_pcblookup_group(pcbinfo, pcbgroup, faddr, fport, laddr, lport, lookupflags, ifp)); +#ifndef RSS pcbgroup = in_pcbgroup_bytuple(pcbinfo, laddr, lport, faddr, fport); return (in_pcblookup_group(pcbinfo, pcbgroup, faddr, fport, laddr, lport, lookupflags, ifp)); +#endif } #endif return (in_pcblookup_hash(pcbinfo, faddr, fport, laddr, lport, Modified: head/sys/netinet/in_pcbgroup.c ============================================================================== --- head/sys/netinet/in_pcbgroup.c Sat Mar 15 00:23:35 2014 (r263197) +++ head/sys/netinet/in_pcbgroup.c Sat Mar 15 00:57:50 2014 (r263198) @@ -32,6 +32,7 @@ __FBSDID("$FreeBSD$"); #include "opt_inet6.h" +#include "opt_rss.h" #include #include @@ -43,6 +44,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #ifdef INET6 #include #endif /* INET6 */ @@ -60,6 +62,13 @@ __FBSDID("$FreeBSD$"); * minimal cache line migration and lock contention during steady state * operation. * + * Hardware-offloaded checksums are often inefficient in software -- for + * example, Toeplitz, specified by RSS, introduced a significant overhead if + * performed during per-packge processing. It is therefore desirable to fall + * back on traditional reservation table lookups without affinity where + * hardware-offloaded checksums aren't available, such as for traffic over + * non-RSS interfaces. + * * Internet protocols, such as UDP and TCP, register to use connection groups * by providing an ipi_hashfields value other than IPI_HASHFIELDS_NONE; this * indicates to the connection group code whether a 2-tuple or 4-tuple is @@ -72,6 +81,11 @@ __FBSDID("$FreeBSD$"); * signficantly more expensive than without connection groups, but that * steady-state processing can be significantly faster. * + * When RSS is used, certain connection group parameters, such as the number + * of groups, are provided by the RSS implementation, found in in_rss.c. + * Otherwise, in_pcbgroup.c selects possible sensible parameters + * corresponding to the degree of parallelism exposed by netisr. + * * Most of the implementation of connection groups is in this file; however, * connection group lookup is implemented in in_pcb.c alongside reservation * table lookups -- see in_pcblookup_group(). @@ -126,11 +140,25 @@ in_pcbgroup_init(struct inpcbinfo *pcbin if (mp_ncpus == 1) return; +#ifdef RSS /* - * Use one group per CPU for now. If we decide to do dynamic - * rebalancing a la RSS, we'll need to shift left by at least 1. + * If we're using RSS, then RSS determines the number of connection + * groups to use: one connection group per RSS bucket. If for some + * reason RSS isn't able to provide a number of buckets, disable + * connection groups entirely. + * + * XXXRW: Can this ever happen? + */ + numpcbgroups = rss_getnumbuckets(); + if (numpcbgroups == 0) + return; +#else + /* + * Otherwise, we'll just use one per CPU for now. If we decide to + * do dynamic rebalancing a la RSS, we'll need similar logic here. */ numpcbgroups = mp_ncpus; +#endif pcbinfo->ipi_hashfields = hashfields; pcbinfo->ipi_pcbgroups = malloc(numpcbgroups * @@ -146,10 +174,19 @@ in_pcbgroup_init(struct inpcbinfo *pcbin /* * Initialise notional affinity of the pcbgroup -- for RSS, - * we want the same notion of affinity as NICs to be used. - * Just round robin for the time being. + * we want the same notion of affinity as NICs to be used. In + * the non-RSS case, just round robin for the time being. + * + * XXXRW: The notion of a bucket to CPU mapping is common at + * both pcbgroup and RSS layers -- does that mean that we + * should migrate it all from RSS to here, and just leave RSS + * responsible only for providing hashing and mapping funtions? */ +#ifdef RSS + pcbgroup->ipg_cpu = rss_getcpu(pgn); +#else pcbgroup->ipg_cpu = (pgn % mp_ncpus); +#endif } } @@ -179,23 +216,38 @@ in_pcbgroup_destroy(struct inpcbinfo *pc /* * Given a hash of whatever the covered tuple might be, return a pcbgroup - * index. + * index. Where RSS is supported, try to align bucket selection with RSS CPU + * affinity strategy. */ static __inline u_int in_pcbgroup_getbucket(struct inpcbinfo *pcbinfo, uint32_t hash) { +#ifdef RSS + return (rss_getbucket(hash)); +#else return (hash % pcbinfo->ipi_npcbgroups); +#endif } /* * Map a (hashtype, hash) tuple into a connection group, or NULL if the hash - * information is insufficient to identify the pcbgroup. + * information is insufficient to identify the pcbgroup. This might occur if + * a TCP packet turns up with a 2-tuple hash, or if an RSS hash is present but + * RSS is not compiled into the kernel. */ struct inpcbgroup * in_pcbgroup_byhash(struct inpcbinfo *pcbinfo, u_int hashtype, uint32_t hash) { +#ifdef RSS + if ((pcbinfo->ipi_hashfields == IPI_HASHFIELDS_4TUPLE && + hashtype == M_HASHTYPE_RSS_TCP_IPV4) || + (pcbinfo->ipi_hashfields == IPI_HASHFIELDS_2TUPLE && + hashtype == M_HASHTYPE_RSS_IPV4)) + return (&pcbinfo->ipi_pcbgroups[ + in_pcbgroup_getbucket(pcbinfo, hash)]); +#endif return (NULL); } @@ -213,13 +265,26 @@ in_pcbgroup_bytuple(struct inpcbinfo *pc { uint32_t hash; + /* + * RSS note: we pass foreign addr/port as source, and local addr/port + * as destination, as we want to align with what the hardware is + * doing. + */ switch (pcbinfo->ipi_hashfields) { case IPI_HASHFIELDS_4TUPLE: +#ifdef RSS + hash = rss_hash_ip4_4tuple(faddr, fport, laddr, lport); +#else hash = faddr.s_addr ^ fport; +#endif break; case IPI_HASHFIELDS_2TUPLE: +#ifdef RSS + hash = rss_hash_ip4_2tuple(faddr, laddr); +#else hash = faddr.s_addr ^ laddr.s_addr; +#endif break; default: Added: head/sys/netinet/in_rss.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/netinet/in_rss.c Sat Mar 15 00:57:50 2014 (r263198) @@ -0,0 +1,505 @@ +/*- + * Copyright (c) 2010-2011 Juniper Networks, Inc. + * All rights reserved. + * + * This software was developed by Robert N. M. Watson under contract + * to Juniper Networks, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include + +__FBSDID("$FreeBSD$"); + +#include "opt_inet6.h" +#include "opt_pcbgroup.h" + +#ifndef PCBGROUP +#error "options RSS depends on options PCBGROUP" +#endif + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include +#include + +/*- + * Operating system parts of receiver-side scaling (RSS), which allows + * network cards to direct flows to particular receive queues based on hashes + * of header tuples. This implementation aligns RSS buckets with connection + * groups at the TCP/IP layer, so each bucket is associated with exactly one + * group. As a result, the group lookup structures (and lock) should have an + * effective affinity with exactly one CPU. + * + * Network device drivers needing to configure RSS will query this framework + * for parameters, such as the current RSS key, hashing policies, number of + * bits, and indirection table mapping hashes to buckets and CPUs. They may + * provide their own supplementary information, such as queue<->CPU bindings. + * It is the responsibility of the network device driver to inject packets + * into the stack on as close to the right CPU as possible, if playing by RSS + * rules. + * + * TODO: + * + * - Synchronization for rss_key and other future-configurable parameters. + * - Event handler drivers can register to pick up RSS configuration changes. + * - Should we allow rss_basecpu to be configured? + * - Randomize key on boot. + * - IPv6 support. + * - Statistics on how often there's a misalignment between hardware + * placement and pcbgroup expectations. + */ + +SYSCTL_NODE(_net_inet, OID_AUTO, rss, CTLFLAG_RW, 0, "Receive-side steering"); + +/* + * Toeplitz is the only required hash function in the RSS spec, so use it by + * default. + */ +static u_int rss_hashalgo = RSS_HASH_TOEPLITZ; +SYSCTL_INT(_net_inet_rss, OID_AUTO, hashalgo, CTLFLAG_RD, &rss_hashalgo, 0, + "RSS hash algorithm"); +TUNABLE_INT("net.inet.rss.hashalgo", &rss_hashalgo); + +/* + * Size of the indirection table; at most 128 entries per the RSS spec. We + * size it to at least 2 times the number of CPUs by default to allow useful + * rebalancing. If not set explicitly with a loader tunable, we tune based + * on the number of CPUs present. + * + * XXXRW: buckets might be better to use for the tunable than bits. + */ +static u_int rss_bits; +SYSCTL_INT(_net_inet_rss, OID_AUTO, bits, CTLFLAG_RD, &rss_bits, 0, + "RSS bits"); +TUNABLE_INT("net.inet.rss.bits", &rss_bits); + +static u_int rss_mask; +SYSCTL_INT(_net_inet_rss, OID_AUTO, mask, CTLFLAG_RD, &rss_mask, 0, + "RSS mask"); + +static const u_int rss_maxbits = RSS_MAXBITS; +SYSCTL_INT(_net_inet_rss, OID_AUTO, maxbits, CTLFLAG_RD, + __DECONST(int *, &rss_maxbits), 0, "RSS maximum bits"); + +/* + * RSS's own count of the number of CPUs it could be using for processing. + * Bounded to 64 by RSS constants. + */ +static u_int rss_ncpus; +SYSCTL_INT(_net_inet_rss, OID_AUTO, ncpus, CTLFLAG_RD, &rss_ncpus, 0, + "Number of CPUs available to RSS"); + +#define RSS_MAXCPUS (1 << (RSS_MAXBITS - 1)) +static const u_int rss_maxcpus = RSS_MAXCPUS; +SYSCTL_INT(_net_inet_rss, OID_AUTO, maxcpus, CTLFLAG_RD, + __DECONST(int *, &rss_maxcpus), 0, "RSS maximum CPUs that can be used"); + +/* + * Variable exists just for reporting rss_bits in a user-friendly way. + */ +static u_int rss_buckets; +SYSCTL_INT(_net_inet_rss, OID_AUTO, buckets, CTLFLAG_RD, &rss_buckets, 0, + "RSS buckets"); + +/* + * Base CPU number; devices will add this to all CPU numbers returned by the + * RSS indirection table. Currently unmodifable in FreeBSD. + */ +static const u_int rss_basecpu; +SYSCTL_INT(_net_inet_rss, OID_AUTO, basecpu, CTLFLAG_RD, + __DECONST(int *, &rss_basecpu), 0, "RSS base CPU"); + +/* + * RSS secret key, intended to prevent attacks on load-balancing. Its + * effectiveness may be limited by algorithm choice and available entropy + * during the boot. + * + * XXXRW: And that we don't randomize it yet! + * + * XXXRW: This default is actually the default key from Chelsio T3 cards, as + * it offers reasonable distribution, unlike all-0 keys which always + * generate a hash of 0 (upsettingly). + */ +static uint8_t rss_key[RSS_KEYSIZE] = { + 0x43, 0xa3, 0x8f, 0xb0, 0x41, 0x67, 0x25, 0x3d, + 0x25, 0x5b, 0x0e, 0xc2, 0x6d, 0x5a, 0x56, 0xda, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +/* + * RSS hash->CPU table, which maps hashed packet headers to particular CPUs. + * Drivers may supplement this table with a seperate CPU<->queue table when + * programming devices. + */ +struct rss_table_entry { + uint8_t rte_cpu; /* CPU affinity of bucket. */ +}; +static struct rss_table_entry rss_table[RSS_TABLE_MAXLEN]; + +static void +rss_init(__unused void *arg) +{ + u_int i; + + /* + * Validate tunables, coerce to sensible values. + */ + switch (rss_hashalgo) { + case RSS_HASH_TOEPLITZ: + case RSS_HASH_NAIVE: + break; + + default: + printf("%s: invalid RSS hashalgo %u, coercing to %u", + __func__, rss_hashalgo, RSS_HASH_TOEPLITZ); + rss_hashalgo = RSS_HASH_TOEPLITZ; + } + + /* + * Count available CPUs. + * + * XXXRW: Note incorrect assumptions regarding contiguity of this set + * elsewhere. + */ + rss_ncpus = 0; + for (i = 0; i <= mp_maxid; i++) { + if (CPU_ABSENT(i)) + continue; + rss_ncpus++; + } + if (rss_ncpus > RSS_MAXCPUS) + rss_ncpus = RSS_MAXCPUS; + + /* + * Tune RSS table entries to be no less than 2x the number of CPUs + * -- unless we're running uniprocessor, in which case there's not + * much point in having buckets to rearrange for load-balancing! + */ + if (rss_ncpus > 1) { + if (rss_bits == 0) + rss_bits = fls(rss_ncpus - 1) + 1; + + /* + * Microsoft limits RSS table entries to 128, so apply that + * limit to both auto-detected CPU counts and user-configured + * ones. + */ + if (rss_bits == 0 || rss_bits > RSS_MAXBITS) { + printf("%s: RSS bits %u not valid, coercing to %u", + __func__, rss_bits, RSS_MAXBITS); + rss_bits = RSS_MAXBITS; + } + + /* + * Figure out how many buckets to use; warn if less than the + * number of configured CPUs, although this is not a fatal + * problem. + */ + rss_buckets = (1 << rss_bits); + if (rss_buckets < rss_ncpus) + printf("%s: WARNING: rss_buckets (%u) less than " + "rss_ncpus (%u)\n", __func__, rss_buckets, + rss_ncpus); + rss_mask = rss_buckets - 1; + } else { + rss_bits = 0; + rss_buckets = 1; + rss_mask = 0; + } + + /* + * Set up initial CPU assignments: round-robin by default. + * + * XXXRW: Need a mapping to non-contiguous IDs here. + */ + for (i = 0; i < rss_buckets; i++) + rss_table[i].rte_cpu = i % rss_ncpus; + + /* + * Randomize rrs_key. + * + * XXXRW: Not yet. If nothing else, will require an rss_isbadkey() + * loop to check for "bad" RSS keys. + */ +} +SYSINIT(rss_init, SI_SUB_SOFTINTR, SI_ORDER_SECOND, rss_init, NULL); + +static uint32_t +rss_naive_hash(u_int keylen, const uint8_t *key, u_int datalen, + const uint8_t *data) +{ + uint32_t v; + u_int i; + + v = 0; + for (i = 0; i < keylen; i++) + v += key[i]; + for (i = 0; i < datalen; i++) + v += data[i]; + return (v); +} + +static uint32_t +rss_hash(u_int datalen, const uint8_t *data) +{ + + switch (rss_hashalgo) { + case RSS_HASH_TOEPLITZ: + return (toeplitz_hash(sizeof(rss_key), rss_key, datalen, + data)); + + case RSS_HASH_NAIVE: + return (rss_naive_hash(sizeof(rss_key), rss_key, datalen, + data)); + + default: + panic("%s: unsupported/unknown hashalgo %d", __func__, + rss_hashalgo); + } +} + +/* + * Hash an IPv4 2-tuple. + */ +uint32_t +rss_hash_ip4_2tuple(struct in_addr src, struct in_addr dst) +{ + uint8_t data[sizeof(src) + sizeof(dst)]; + u_int datalen; + + datalen = 0; + bcopy(&src, &data[datalen], sizeof(src)); + datalen += sizeof(src); + bcopy(&dst, &data[datalen], sizeof(dst)); + datalen += sizeof(dst); + return (rss_hash(datalen, data)); +} + +/* + * Hash an IPv4 4-tuple. + */ +uint32_t +rss_hash_ip4_4tuple(struct in_addr src, u_short srcport, struct in_addr dst, + u_short dstport) +{ + uint8_t data[sizeof(src) + sizeof(dst) + sizeof(srcport) + + sizeof(dstport)]; + u_int datalen; + + datalen = 0; + bcopy(&src, &data[datalen], sizeof(src)); + datalen += sizeof(src); + bcopy(&dst, &data[datalen], sizeof(dst)); + datalen += sizeof(dst); + bcopy(&srcport, &data[datalen], sizeof(srcport)); + datalen += sizeof(srcport); + bcopy(&dstport, &data[datalen], sizeof(dstport)); + datalen += sizeof(dstport); + return (rss_hash(datalen, data)); +} + +#ifdef INET6 +/* + * Hash an IPv6 2-tuple. + */ +uint32_t +rss_hash_ip6_2tuple(struct in6_addr src, struct in6_addr dst) +{ + uint8_t data[sizeof(src) + sizeof(dst)]; + u_int datalen; + + datalen = 0; + bcopy(&src, &data[datalen], sizeof(src)); + datalen += sizeof(src); + bcopy(&dst, &data[datalen], sizeof(dst)); + datalen += sizeof(dst); + return (rss_hash(datalen, data)); +} + +/* + * Hash an IPv6 4-tuple. + */ +uint32_t +rss_hash_ip6_4tuple(struct in6_addr src, u_short srcport, + struct in6_addr dst, u_short dstport) +{ + uint8_t data[sizeof(src) + sizeof(dst) + sizeof(srcport) + + sizeof(dstport)]; + u_int datalen; + + datalen = 0; + bcopy(&src, &data[datalen], sizeof(src)); + datalen += sizeof(src); + bcopy(&dst, &data[datalen], sizeof(dst)); + datalen += sizeof(dst); + bcopy(&srcport, &data[datalen], sizeof(srcport)); + datalen += sizeof(srcport); + bcopy(&dstport, &data[datalen], sizeof(dstport)); + datalen += sizeof(dstport); + return (rss_hash(datalen, data)); +} +#endif /* INET6 */ + +/* + * Query the number of RSS bits in use. + */ +u_int +rss_getbits(void) +{ + + return (rss_bits); +} + +/* + * Query the RSS bucket associated with an RSS hash. + */ +u_int +rss_getbucket(u_int hash) +{ + + return (hash & rss_mask); +} + +/* + * Query the RSS CPU associated with an RSS bucket. + */ +u_int +rss_getcpu(u_int bucket) +{ + + return (rss_table[bucket].rte_cpu); +} + +/* + * netisr CPU affinity lookup routine for use by protocols. + */ +struct mbuf * +rss_m2cpuid(struct mbuf *m, uintptr_t source, u_int *cpuid) +{ + + M_ASSERTPKTHDR(m); + + switch (M_HASHTYPE_GET(m)) { + case M_HASHTYPE_RSS_IPV4: + case M_HASHTYPE_RSS_TCP_IPV4: + *cpuid = rss_getcpu(rss_getbucket(m->m_pkthdr.flowid)); + return (m); + + default: + *cpuid = NETISR_CPUID_NONE; + return (m); + } +} + +/* + * Query the RSS hash algorithm. + */ +u_int +rss_gethashalgo(void) +{ + + return (rss_hashalgo); +} + +/* + * Query the current RSS key; likely to be used by device drivers when + * configuring hardware RSS. Caller must pass an array of size RSS_KEYSIZE. + * + * XXXRW: Perhaps we should do the accept-a-length-and-truncate thing? + */ +void +rss_getkey(uint8_t *key) +{ + + bcopy(rss_key, key, sizeof(rss_key)); +} + +/* + * Query the number of buckets; this may be used by both network device + * drivers, which will need to populate hardware shadows of the software + * indirection table, and the network stack itself (such as when deciding how + * many connection groups to allocate). + */ +u_int +rss_getnumbuckets(void) +{ + + return (rss_buckets); +} + +/* + * Query the number of CPUs in use by RSS; may be useful to device drivers + * trying to figure out how to map a larger number of CPUs into a smaller + * number of receive queues. + */ +u_int +rss_getnumcpus(void) +{ + + return (rss_ncpus); +} + +/* + * XXXRW: Confirm that sysctl -a won't dump this keying material, don't want + * it appearing in debugging output unnecessarily. + */ +static int +sysctl_rss_key(SYSCTL_HANDLER_ARGS) +{ + uint8_t temp_rss_key[RSS_KEYSIZE]; + int error; + + error = priv_check(req->td, PRIV_NETINET_HASHKEY); + if (error) + return (error); + + bcopy(rss_key, temp_rss_key, sizeof(temp_rss_key)); + error = sysctl_handle_opaque(oidp, temp_rss_key, + sizeof(temp_rss_key), req); + if (error) + return (error); + if (req->newptr != NULL) { + /* XXXRW: Not yet. */ + return (EINVAL); + } + return (0); +} +SYSCTL_PROC(_net_inet_rss, OID_AUTO, key, + CTLTYPE_OPAQUE | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, sysctl_rss_key, + "", "RSS keying material"); Added: head/sys/netinet/in_rss.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/netinet/in_rss.h Sat Mar 15 00:57:50 2014 (r263198) @@ -0,0 +1,94 @@ +/*- + * Copyright (c) 2010-2011 Juniper Networks, Inc. + * All rights reserved. + * + * This software was developed by Robert N. M. Watson under contract + * to Juniper Networks, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _NETINET_IN_RSS_H_ +#define _NETINET_IN_RSS_H_ + +#include /* in_addr_t */ + +/* + * Supported RSS hash functions. + */ +#define RSS_HASH_NAIVE 0x00000001 /* Poor but fast hash. */ +#define RSS_HASH_TOEPLITZ 0x00000002 /* Required by RSS. */ +#define RSS_HASH_CRC32 0x00000004 /* Future; some NICs do it. */ + +#define RSS_HASH_MASK (RSS_HASH_NAIVE | RSS_HASH_TOEPLITZ) + +/* + * Instances of struct inpcbinfo declare an RSS hash type indicating what + * header fields are covered. + */ +#define RSS_HASHFIELDS_NONE 0 +#define RSS_HASHFIELDS_4TUPLE 1 +#define RSS_HASHFIELDS_2TUPLE 2 + +/* + * Compile-time limits on the size of the indirection table. + */ +#define RSS_MAXBITS 7 +#define RSS_TABLE_MAXLEN (1 << RSS_MAXBITS) + +/* + * Maximum key size used throughout. It's OK for hardware to use only the + * first 16 bytes, which is all that's required for IPv4. + */ +#define RSS_KEYSIZE 40 + +/* + * Device driver interfaces to query RSS properties that must be programmed + * into hardware. + */ +u_int rss_getbits(void); +u_int rss_getbucket(u_int hash); +u_int rss_getcpu(u_int bucket); +void rss_getkey(uint8_t *key); +u_int rss_gethashalgo(void); +u_int rss_getnumbuckets(void); +u_int rss_getnumcpus(void); + +/* + * Network stack interface to generate a hash for a protocol tuple. + */ +uint32_t rss_hash_ip4_4tuple(struct in_addr src, u_short srcport, + struct in_addr dst, u_short dstport); +uint32_t rss_hash_ip4_2tuple(struct in_addr src, struct in_addr dst); +uint32_t rss_hash_ip6_4tuple(struct in6_addr src, u_short srcport, + struct in6_addr dst, u_short dstport); +uint32_t rss_hash_ip6_2tuple(struct in6_addr src, + struct in6_addr dst); + +/* + * Network stack interface to query desired CPU affinity of a packet. + */ +struct mbuf *rss_m2cpuid(struct mbuf *m, uintptr_t source, u_int *cpuid); + +#endif /* !_NETINET_IN_RSS_H_ */ Added: head/sys/netinet/toeplitz.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/netinet/toeplitz.c Sat Mar 15 00:57:50 2014 (r263198) @@ -0,0 +1,58 @@ +/*- + * Copyright (c) 2010 David Malone + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include + +#include +#include + +#include + +uint32_t +toeplitz_hash(u_int keylen, const uint8_t *key, u_int datalen, + const uint8_t *data) +{ + uint32_t hash = 0, v; + u_int i, b; + + /* XXXRW: Perhaps an assertion about key length vs. data length? */ + + v = (key[0]<<24) + (key[1]<<16) + (key[2] <<8) + key[3]; + for (i = 0; i < datalen; i++) { + for (b = 0; b < 8; b++) { + if (data[i] & (1<<(7-b))) + hash ^= v; + v <<= 1; + if ((i + 4) < RSS_KEYSIZE && + (key[i+4] & (1<<(7-b)))) + v |= 1; + } + } + return (hash); +} Added: head/sys/netinet/toeplitz.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-freebsd-net@FreeBSD.ORG Sun Mar 16 03:08:51 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 037F0775 for ; Sun, 16 Mar 2014 03:08:51 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [195.131.132.186]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9BB8E62F for ; Sun, 16 Mar 2014 03:08:50 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 2E22F7F45D; Sat, 15 Mar 2014 17:13:06 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 2E22F7F45D DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1394889186; bh=kHuflD9fbl1E8NGYsUN0aBMeYazJ97SEVlj8/uqSFWI=; l=957; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type; b=R23PPyKxMnECk/YQHncb2XmPrrWBokMQl6QsSNV1okwn+37a8TQ0aQnbWs1MGh6KV QaTKfZHkOuHejBDvzUxrdUVA7795YDG/nJ98DLRzJ9UmLo6sl5Cwbt1rWsYQdEycNC k027Oz77cZ29K7uDeL+DpaJHWlsgP/aLXTcZUqAw= Message-ID: <532451E2.20407@oktetlabs.ru> Date: Sat, 15 Mar 2014 17:13:06 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [patch] [1/6] sfxge: fix mbuf leak if it does not fit in software queue Content-Type: multipart/mixed; boundary="------------040304000606070504070008" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 03:08:51 -0000 This is a multi-part message in MIME format. --------------040304000606070504070008 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit --------------040304000606070504070008 Content-Type: text/x-patch; name="1-sfxge-leak.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="1-sfxge-leak.patch" sfxge: fix mbuf leak if it does not fit in software queue mbuf should be owned by if_transmit function in any case. Submitted-by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r e2bc8f64f1b2 -r ff9f5d3dbafe src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:13:05 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:15:13 2014 +0400 @@ -536,6 +536,7 @@ return (0); fail: + m_freem(m); return (rc); } --------------040304000606070504070008-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 03:04:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 688EDF35 for ; Mon, 17 Mar 2014 03:04:23 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 235AD392 for ; Mon, 17 Mar 2014 03:04:22 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WPNqY-0006nS-5L for freebsd-net@freebsd.org; Mon, 17 Mar 2014 04:04:14 +0100 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Mar 2014 04:04:14 +0100 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Mar 2014 04:04:14 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: VNET, if_bridge, if_epair, vlans and bridged phy? Date: Sun, 16 Mar 2014 20:04:01 -0700 Lines: 34 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Thunderbird/28.0 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 03:04:23 -0000 I'm trying a somewhat elaborate VNET jails setup and for the most part it's working. I'm using if_epairs, one side that gets passed into the jail, and the other side that attaches to an if_bridge. The if_bridge has a member on a vlan interface. So far so good. cloned_interfaces="bridge0 bridge1 bridge2 vlan0 vlan1" ifconfig_ix0="inet netmask 255.255.255.240 up" ifconfig_vlan0="vlan 1010 vlandev ix0" ifconfig_vlan1="vlan 1011 vlandev ix0" ifconfig_bridge1="inet 10.10.10.55/24 addm vlan0 description vlan0" ifconfig_bridge2="inet 10.10.11.55/24 addm vlan1 description vlan1" The above works fine, the VNET jails are able to access the outside world and vis versa (NAT happens on a dedicated router, not this host). Now, if I instead do something like this to add the public IP to a bridge: ifconfig_ix0="up" ifconfig_vlan0="vlan 1010 vlandev ix0" ifconfig_vlan1="vlan 1011 vlandev ix0" ifconfig_bridge0="inet netmask 255.255.255.240 addm ix0 description ix0" ifconfig_bridge1="inet 10.10.10.55/24 addm vlan0 description vlan0" ifconfig_bridge2="inet 10.10.11.55/24 addm vlan1 description vlan1" A VNET jail on bridge0 in the public IP space works fine, but bridge1 and bridge2 are no longer accessible from the outside, including the host interface like 10.10.10.55. Any ideas on what could be going wrong? Is there a way to use an untagged interface like this in addition to the tagged ones? Regards, Kevin From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 08:02:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 032D5276 for ; Mon, 17 Mar 2014 08:02:46 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AF366F67 for ; Mon, 17 Mar 2014 08:02:45 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WPSVO-0007jQ-83 for freebsd-net@freebsd.org; Mon, 17 Mar 2014 09:02:42 +0100 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Mar 2014 09:02:42 +0100 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Mar 2014 09:02:42 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: Re: VNET, if_bridge, if_epair, vlans and bridged phy? Date: Mon, 17 Mar 2014 01:02:29 -0700 Lines: 48 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Thunderbird/28.0 In-Reply-To: Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 08:02:46 -0000 On 3/16/2014 8:04 PM, Kevin Bowling wrote: > I'm trying a somewhat elaborate VNET jails setup and for the most part > it's working. I'm using if_epairs, one side that gets passed into the > jail, and the other side that attaches to an if_bridge. The if_bridge > has a member on a vlan interface. So far so good. > > cloned_interfaces="bridge0 bridge1 bridge2 vlan0 vlan1" > ifconfig_ix0="inet netmask 255.255.255.240 up" > ifconfig_vlan0="vlan 1010 vlandev ix0" > ifconfig_vlan1="vlan 1011 vlandev ix0" > ifconfig_bridge1="inet 10.10.10.55/24 addm vlan0 description vlan0" > ifconfig_bridge2="inet 10.10.11.55/24 addm vlan1 description vlan1" > > The above works fine, the VNET jails are able to access the outside > world and vis versa (NAT happens on a dedicated router, not this host). > > Now, if I instead do something like this to add the public IP to a bridge: > > ifconfig_ix0="up" > ifconfig_vlan0="vlan 1010 vlandev ix0" > ifconfig_vlan1="vlan 1011 vlandev ix0" > ifconfig_bridge0="inet netmask 255.255.255.240 addm ix0 > description ix0" > ifconfig_bridge1="inet 10.10.10.55/24 addm vlan0 description vlan0" > ifconfig_bridge2="inet 10.10.11.55/24 addm vlan1 description vlan1" > > A VNET jail on bridge0 in the public IP space works fine, but bridge1 > and bridge2 are no longer accessible from the outside, including the > host interface like 10.10.10.55. > > Any ideas on what could be going wrong? Is there a way to use an > untagged interface like this in addition to the tagged ones? > > Regards, > Kevin I'm able to work around this by setting the native VLAN on the switch to a bogus value and using another tagged interface for the public IP (now nothing uses untagged interface). I'm guessing it might be rstp/mstp related since STP does not happen on the VLAN interfaces, but it does on the native port when added to a bridge. When they're all VLANs, I don't think if_bridge will send any BPDUs to the switch. Regards, Kevin From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F625A73 for ; Mon, 17 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4BA7629A for ; Mon, 17 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2HB6nAP011323 for ; Mon, 17 Mar 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2HB6m7R011321 for freebsd-net@FreeBSD.org; Mon, 17 Mar 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Mar 2014 11:06:48 GMT Message-Id: <201403171106.s2HB6m7R011321@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183390 net [ixgbe] 10gigabit networking problems o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 466 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 13:35:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87B695CB for ; Mon, 17 Mar 2014 13:35:22 +0000 (UTC) Received: from blu0-omc2-s2.blu0.hotmail.com (blu0-omc2-s2.blu0.hotmail.com [65.55.111.77]) by mx1.freebsd.org (Postfix) with ESMTP id 5412C3A4 for ; Mon, 17 Mar 2014 13:35:22 +0000 (UTC) Received: from BLU172-W35 ([65.55.111.72]) by blu0-omc2-s2.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 17 Mar 2014 06:34:16 -0700 X-TMN: [p4Dzf7QiQAW840rXcEiqXP2ChexgllPa] X-Originating-Email: [testdog@hotmail.com] Message-ID: From: Wayne Hotmail To: "freebsd-net@freebsd.org" Subject: Intel 10 gig cards mtu 9000 Date: Mon, 17 Mar 2014 13:34:16 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 17 Mar 2014 13:34:16.0841 (UTC) FILETIME=[97E04F90:01CF41E5] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 13:35:22 -0000 I am unable to add 2 Intel 10 gig nics to my Freebsd 9.1 at 9000 MTU. I tri= ed to add them last night and was only able to set only a max of 4000 MTU. = I get an error telling me kernel: ix1: Could not setup receive structures = and the interface will not pass traffic. I found these changes on the Intel readme but this did solve my issue. kern.ipc.nmbclusters=3D262144 kern.ipc.nmbjumbop=3D262144 There seems to be some calculation that needs to be done with multiple CPUs= . I am running 12 cpus what tweak do I need to make so I can run my MTUs at= 9000 on my 1 NIC with no vlans and 1 NIC with 3 vlans on it.=20 Wayne = From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 17:40:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62A9473C for ; Mon, 17 Mar 2014 17:40:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4EEFE138 for ; Mon, 17 Mar 2014 17:40:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2HHe0Q9035534 for ; Mon, 17 Mar 2014 17:40:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2HHe08t035533; Mon, 17 Mar 2014 17:40:00 GMT (envelope-from gnats) Date: Mon, 17 Mar 2014 17:40:00 GMT Message-Id: <201403171740.s2HHe08t035533@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Charbon, Julien" Subject: Re: kern/183659: [tcp] TCP stack lock contention with short-lived connections X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: "Charbon, Julien" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 17:40:01 -0000 The following reply was made to PR kern/183659; it has been noted by GNATS. From: "Charbon, Julien" To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/183659: [tcp] TCP stack lock contention with short-lived connections Date: Mon, 17 Mar 2014 15:54:03 +0100 Just a follow-up that updates lock profiling results with short-lived TCP connection traffic on FreeBSD-10.0 RELEASE: (Previous results were made on FreeBSD-9.2 RELEASE) o FreeBSD-10 RELEASE: # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -5 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 37 321900 3049892 13033648 610019 4 21 0 588013 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 51 115462 3240265 12270984 553157 5 22 0 545293 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 29 62577 1170617 8754815 305885 3 28 0 296845 sys/netinet/tcp_usrreq.c:728 (rw:tcp) tcp_usr_close() 6 62645 146544 8548857 292058 0 29 0 283587 sys/netinet/tcp_usrreq.c:984 (rw:tcp) tcp_usr_shutdown() 11 62595 198811 6525067 309009 0 21 0 304522 sys/netinet/tcp_usrreq.c:635 (rw:tcp) tcp_usr_accept() - If lock contention spots moved a little between 9.2 and 10.0, nothing major as the top 5 still belongs to (rw:tcp) lock (a.k.a. TCP INP_INFO). o FreeBSD-10 RELEASE + PCBGROUP kernel option (by popular demand): # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -5 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 58 84250 2970633 13154832 622401 4 21 0 598964 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 47 224326 3375328 12945466 562451 6 23 0 554567 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 22 84332 1193078 9693951 311555 3 31 0 302420 sys/netinet/tcp_usrreq.c:728 (rw:tcp) tcp_usr_close() 6 84307 151411 9137383 298120 0 30 0 289496 sys/netinet/tcp_usrreq.c:984 (rw:tcp) tcp_usr_shutdown() 15 84351 201705 6504520 314353 0 20 0 310270 sys/netinet/tcp_usrreq.c:635 (rw:tcp) tcp_usr_accept() - No changes at all in first ranks by using PCBGROUP option on FreeBSD-10 RELEASE. I have indeed checked that PCBGROUP was in use as at #36 rank there is the specific pcbgroup lock: 11 9 289817 4815 1505626 0 0 0 16054 sys/netinet/in_pcb.c:1530 (sleep mutex:pcbgroup) o FreeBSD-10 RELEASE + current lock mitigation patches [1][2]: # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -20 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 29 297 3781629 13476466 734686 5 18 0 715214 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 35 287 3817278 12301410 672907 5 18 0 669324 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 18 170 1392058 2494823 367131 3 6 0 357888 sys/netinet/tcp_usrreq.c:719 (rw:tcp) tcp_usr_shutdown() 7 141 182209 2433120 350488 0 6 0 344878 sys/netinet/tcp_usrreq.c:975 (rw:tcp) tcp_usr_close() 10 259 26786 933073 38101 0 24 0 37624 sys/netinet/tcp_timer.c:493 (rw:tcp) tcp_timer_rexmt() - No more tcp_usr_accept() (expected) o Global results: Maximum short-lived TCP connection rate without dropping a single packet: - FreeBSD 10.0 RELEASE: 40.0k - FreeBSD 10.0 RELEASE + PCBGROUP: 40.0k - FreeBSD 10.0 RELEASE + patches: 56.8k [1] Decrease lock contention within the TCP accept case by removing the INP_INFO lock from tcp_usr_accept. http://svnweb.freebsd.org/base?view=revision&revision=261242 [2] tw-clock-v2.patch attached in: http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038124.html -- Julien From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 18:40:02 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA7E67B0 for ; Mon, 17 Mar 2014 18:40:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A6F1F9A7 for ; Mon, 17 Mar 2014 18:40:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2HIe2A2054551 for ; Mon, 17 Mar 2014 18:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2HIe2ed054550; Mon, 17 Mar 2014 18:40:02 GMT (envelope-from gnats) Date: Mon, 17 Mar 2014 18:40:02 GMT Message-Id: <201403171840.s2HIe2ed054550@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Charbon, Julien" Subject: Re: kern/183659: [tcp] ]TCP stack lock contention with short-lived connections X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: "Charbon, Julien" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 18:40:02 -0000 The following reply was made to PR kern/183659; it has been noted by GNATS. From: "Charbon, Julien" To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/183659: [tcp] ]TCP stack lock contention with short-lived connections Date: Mon, 17 Mar 2014 17:39:19 +0100 Just a follow-up that updates lock profiling results with short-lived TCP connection traffic on FreeBSD-10.0 RELEASE: (Previous results were made on FreeBSD-9.2 RELEASE) o FreeBSD-10 RELEASE: # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -5 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 37 321900 3049892 13033648 610019 4 21 0 588013 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 51 115462 3240265 12270984 553157 5 22 0 545293 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 29 62577 1170617 8754815 305885 3 28 0 296845 sys/netinet/tcp_usrreq.c:728 (rw:tcp) tcp_usr_close() 6 62645 146544 8548857 292058 0 29 0 283587 sys/netinet/tcp_usrreq.c:984 (rw:tcp) tcp_usr_shutdown() 11 62595 198811 6525067 309009 0 21 0 304522 sys/netinet/tcp_usrreq.c:635 (rw:tcp) tcp_usr_accept() - If lock contention spots moved a little between 9.2 and 10.0, nothing major as the top 5 still belongs to (rw:tcp) lock (a.k.a. TCP INP_INFO). o FreeBSD-10 RELEASE + PCBGROUP kernel option (by popular demand): # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -5 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 58 84250 2970633 13154832 622401 4 21 0 598964 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 47 224326 3375328 12945466 562451 6 23 0 554567 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 22 84332 1193078 9693951 311555 3 31 0 302420 sys/netinet/tcp_usrreq.c:728 (rw:tcp) tcp_usr_close() 6 84307 151411 9137383 298120 0 30 0 289496 sys/netinet/tcp_usrreq.c:984 (rw:tcp) tcp_usr_shutdown() 15 84351 201705 6504520 314353 0 20 0 310270 sys/netinet/tcp_usrreq.c:635 (rw:tcp) tcp_usr_accept() - No changes at all in first ranks by using PCBGROUP option on FreeBSD-10 RELEASE. I have indeed checked that PCBGROUP was in use as at #36 rank there is the specific pcbgroup lock: 11 9 289817 4815 1505626 0 0 0 16054 sys/netinet/in_pcb.c:1530 (sleep mutex:pcbgroup) o FreeBSD-10 RELEASE + current lock mitigation patches [1][2]: # sysctl debug.lock.prof.stats | head -2; sysctl debug.lock.prof.stats | sort -n -k 4 -r | head -20 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 29 297 3781629 13476466 734686 5 18 0 715214 sys/netinet/tcp_input.c:778 (rw:tcp) tcp_input() (SYN|FIN|RST) 35 287 3817278 12301410 672907 5 18 0 669324 sys/netinet/tcp_input.c:1013 (rw:tcp) tcp_input() (state != ESTABLISHED) 18 170 1392058 2494823 367131 3 6 0 357888 sys/netinet/tcp_usrreq.c:719 (rw:tcp) tcp_usr_shutdown() 7 141 182209 2433120 350488 0 6 0 344878 sys/netinet/tcp_usrreq.c:975 (rw:tcp) tcp_usr_close() 10 259 26786 933073 38101 0 24 0 37624 sys/netinet/tcp_timer.c:493 (rw:tcp) tcp_timer_rexmt() - No more tcp_usr_accept() (expected) o Global results: Maximum short-lived TCP connection rate without dropping a single packet: - FreeBSD 10.0 RELEASE: 40.0k - FreeBSD 10.0 RELEASE + PCBGROUP: 40.0k - FreeBSD 10.0 RELEASE + patches: 56.8k [1] Decrease lock contention within the TCP accept case by removing the INP_INFO lock from tcp_usr_accept. http://svnweb.freebsd.org/base?view=revision&revision=261242 [2] tw-clock-v2.patch attached in: http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038124.html -- Julien From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 20:38:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF02C911 for ; Mon, 17 Mar 2014 20:38:32 +0000 (UTC) Received: from mail-ve0-x229.google.com (mail-ve0-x229.google.com [IPv6:2607:f8b0:400c:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A4BE7A9 for ; Mon, 17 Mar 2014 20:38:32 +0000 (UTC) Received: by mail-ve0-f169.google.com with SMTP id pa12so6124412veb.14 for ; Mon, 17 Mar 2014 13:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lrGKmUNELlUnqN8q1KZLQ3xB30T+oOFHj/IgRadI7jU=; b=xUBmhVmqDS8x31cnRAMYOlyV/PNJDUIkt7XJJdYDWpeO1a0R1pvVgPiRHKeXDMsOyk tI62bmGKSeXJ/lgHM5koY5f7+RmaywA96LVY+h/yzNRTRQIm35ZNrE6kKrN76/aN0LPD BpckXXt9cpL2jDVhqlG/y8GxffA5Wf+Jql2qylwWUVMSllY3VvXKGG2FKN7FdGKPNTHj eJbC6voJlKPJwEpN+MYlLdCYx+e0477i1uu/+hGTmknplPXLI+ASQ2ZfuqGw0aTV+7NM QZuHSZvbgw80tECyMzz2IW4QP764dvTzlh9dz6Ko946842OzvmC/mSpD5Bo5OnlXfele mEQA== MIME-Version: 1.0 X-Received: by 10.58.238.35 with SMTP id vh3mr20941138vec.16.1395088711704; Mon, 17 Mar 2014 13:38:31 -0700 (PDT) Received: by 10.221.11.135 with HTTP; Mon, 17 Mar 2014 13:38:31 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Mar 2014 13:38:31 -0700 Message-ID: Subject: Re: Intel 10 gig cards mtu 9000 From: Jack Vogel To: Wayne Hotmail Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 20:38:33 -0000 There's only one reason RX structures fail, and that's insufficient mbuf pool. You will find the driver probably uses the 9K mbuf pool in that driver, so look at how many queues it wants to set up, how big your ring is, and do the math. Jack On Mon, Mar 17, 2014 at 6:34 AM, Wayne Hotmail wrote: > I am unable to add 2 Intel 10 gig nics to my Freebsd 9.1 at 9000 MTU. I > tried to add them last night and was only able to set only a max of 4000 > MTU. I get an error telling me kernel: ix1: Could not setup receive > structures and the interface will not pass traffic. > > I found these changes on the Intel readme but this did solve my issue. > kern.ipc.nmbclusters=262144 > kern.ipc.nmbjumbop=262144 > > There seems to be some calculation that needs to be done with multiple > CPUs. I am running 12 cpus what tweak do I need to make so I can run my > MTUs at 9000 on my 1 NIC with no vlans and 1 NIC with 3 vlans on it. > > Wayne > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 17 22:42:08 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49F8181E; Mon, 17 Mar 2014 22:42:08 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1F1E632A; Mon, 17 Mar 2014 22:42:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2HMg785031045; Mon, 17 Mar 2014 22:42:07 GMT (envelope-from delphij@freefall.freebsd.org) Received: (from delphij@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2HMg7xP031044; Mon, 17 Mar 2014 22:42:07 GMT (envelope-from delphij) Date: Mon, 17 Mar 2014 22:42:07 GMT Message-Id: <201403172242.s2HMg7xP031044@freefall.freebsd.org> To: delphij@FreeBSD.org, freebsd-net@FreeBSD.org, jfv@FreeBSD.org From: delphij@FreeBSD.org Subject: Re: kern/183390: [ixgbe] 10gigabit networking problems X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 22:42:08 -0000 Synopsis: [ixgbe] 10gigabit networking problems Responsible-Changed-From-To: freebsd-net->jfv Responsible-Changed-By: delphij Responsible-Changed-When: Mon Mar 17 22:41:15 UTC 2014 Responsible-Changed-Why: Hi, Jack, Some FreeNAS users [1] have encountered similar issue too, can you take a look at this one? Thanks in advance! [1] https://bugs.freenas.org/issues/4560 http://www.freebsd.org/cgi/query-pr.cgi?pr=183390 From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 05:42:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68859B67 for ; Tue, 18 Mar 2014 05:42:49 +0000 (UTC) Received: from smtp.new-ukraine.org (smtp.new-ukraine.org [148.251.53.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E7948C5E for ; Tue, 18 Mar 2014 05:42:48 +0000 (UTC) Received: from new-ukraine.org (smtp.new-ukraine.org [148.251.53.51]) by smtp.new-ukraine.org with ESMTP id s2I5gkru008521 for ; Tue, 18 Mar 2014 07:42:46 +0200 (EET) Message-ID: <20140318074246.8519@smtp.new-ukraine.org> Date: Tue, 18 Mar 2014 07:42:46 +0200 From: "Zeus Panchenko" To: cc: Subject: vlanXXX vs ifXX.YY notation Organization: I.B.S. LLC X-Attribution: zeus Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEWxsbGdnZ3U1NQTExN cXFzx8fG/v7+f8hyWAAACXUlEQVQ4jUWSwXYiIRBFi4yyhtjtWpmRdTL0ZC3TJOukDa6Rc+T/P2F eFepwtFvr8upVFVDua8mLWw6La4VIKTuMdAPOebdU55sQs3n/D1xFFPFGVGh4AHKttr5K0bS6g7N ZCge7qpVLB+f1Z2WAj2OKXwIWt/bXpdXSiu8KXbviWkHxF5td9+lg2e3xlI2SCvatK8YLfHyh9lw 15yrad8Va5eXg4Llr7QmAaC+dL9sDt9iad/DX3OKvLMBf+dm0A0QuMrTvYIevSik1IaSVvgjIHt5 lSCG2ynNRpEcBZ8cgDWk+Ns99qzsYYV3MZoppWzGtYlTO9+meG6m/g92iNO9LfQB2JZsMpoJs7QG ku2KtabRK0bZRwDLyBDvwlxTm6ZlP7qyOqLcfqtLexpDSB4M0H3I/PQy1emvjjzgK+A0LmMKl6Lq zlqzh0VGAw440F6MJd8cY0nI7wiF/fVIBGY7UNCAXy6DmfYGCLLI0wtDbVcDUMqtJLmAhLqODQAe riERAxXJ1/QYGpa0ymqyytpKC19MNXHjvFmEsfcHIrncFR4xdbYWgmfEGLCcZokpGbGj1egMR+6M 1BkNX1pDdhPcOXpAnAeLQUwQLYepgQoZVNGS61yaE8CYA7gYAcWKzwGstACY2HTFvvOwk4FXAG/a mKHni/EcA/GkOk7I0IK7UMIf3+SahU8/FJdiE7KcuWdM3MFocUDEEIX9LfJoo4xV5tnNKc3jJuSs SZWgnnhepgU1zN4Hii18yW4RwDX52CXUtk0Hqz6cHOIUkWaX8fDcB+J7y1y2xDHwjv/8Buu8Ekz6 7tXQAAAAASUVORK5CYII= X-Mailer: MH-E 8.3.1; GNU Mailutils 2.99.98; GNU Emacs 24.3.1 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Zeus Panchenko List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 05:42:49 -0000 =2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, is there any advantage of using vlanXXX vs ifXX.YY notation? I mean > ifconfig em0 vlan777: flags=3D8843 metric 0 mtu = 1500 ether 00:1b:b9:8b:ca:33 ... vlan: 777 parent interface: em0 vs em0.555: flags=3D8843 metric 0 mtu = 1500 ether 00:1b:b9:8b:ca:33 ... vlan: 555 parent interface: em0 =2D --=20 Zeus V. Panchenko jid:zeus@im.ibs.dn.ua IT Dpt., I.B.S. LLC GMT+2 (EET) =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlMn3NYACgkQr3jpPg/3oyqFEgCgj5te5rIwlKqZmlzENBBjLbdG j5kAoPv8vFLgrsJRVPeIAzhCvpEC+Xxj =3DBUzc =2D----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 09:49:08 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEA4E1E0 for ; Tue, 18 Mar 2014 09:49:08 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A604B3E6 for ; Tue, 18 Mar 2014 09:49:08 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 353187F4AE; Tue, 18 Mar 2014 13:11:14 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 353187F4AE DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395133874; bh=GaBnJsNunYZ58EGb+jVqjEkIOJY4qf0HTdRVNtBhpxA=; l=547; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=xg4s5/GZEgziJgnpfixJ2+hrDa+k03hvSflztZIZSoTKQFLTNuK7VMFagq9OgIlDI ND++n+Z4h58+RNNgBh5A1m7JC+4IDR59iW6wnXIf9ROtpyl64QyzaV59Cyd5zvrIaM 5ezCINcgLOlev9v564bWrNgK2OaxaQ5IaEF/dvU8= Message-ID: <53280DB3.4080900@oktetlabs.ru> Date: Tue, 18 Mar 2014 13:11:15 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 09:49:09 -0000 sfxge: fix mbuf leak if it does not fit in software queue mbuf should be owned by if_transmit function in any case. Submitted-by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r e2bc8f64f1b2 -r ff9f5d3dbafe src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:13:05 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:15:13 2014 +0400 @@ -536,6 +536,7 @@ return (0); fail: + m_freem(m); return (rc); } From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 09:55:03 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D184E38F for ; Tue, 18 Mar 2014 09:55:03 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8A20B6A5 for ; Tue, 18 Mar 2014 09:55:03 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id B3BC57F57C; Tue, 18 Mar 2014 13:55:00 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru B3BC57F57C DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395136500; bh=8MszJOk6k9O2Zb6wQVR8MBIm4aUDiotW/5dkTw60uHc=; l=2069; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=goV4R/xsBLdAjCu0zy937J8aJz4eGhOgQuD01IOCwN/QDERKbk89GFREAOnoRPpwA LqJhf4t7tnoiSU9s6Q0FT8Uzb6EndBFfss8Ko8LNV8NrUIJXTDa468l6zOIKtu0vNI CTb5a1aInP6AvAYSoXJBR0+nZbgAPp9+Qdp+3pqM= Message-ID: <532817F5.8010505@oktetlabs.ru> Date: Tue, 18 Mar 2014 13:55:01 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 2/6] sfxge: limit software Tx queue size Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 09:55:03 -0000 sfxge: limit software Tx queue size Previous implementation limits put queue size only (when Tx lock can't be acquired), but get queue may grow unboundedly which results in mbuf pools exhaustion and latency growth. Submitted-by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r ff9f5d3dbafe -r 7632a3355224 src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:15:13 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Wed Mar 05 09:06:01 2014 +0400 @@ -461,6 +461,9 @@ sfxge_tx_qdpl_swizzle(txq); + if (stdp->std_count >= SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT) + return ENOBUFS; + *(stdp->std_getp) = mbuf; stdp->std_getp = &mbuf->m_nextpkt; stdp->std_count++; @@ -480,7 +483,7 @@ old_len = mp->m_pkthdr.csum_data; } else old_len = 0; - if (old_len >= SFXGE_TX_MAX_DEFERRED) + if (old_len >= SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT) return ENOBUFS; mbuf->m_pkthdr.csum_data = old_len + 1; mbuf->m_nextpkt = (void *)old; @@ -507,12 +510,9 @@ */ locked = mtx_trylock(&txq->lock); - /* - * Can only fail if we weren't able to get the lock. - */ if (sfxge_tx_qdpl_put(txq, m, locked) != 0) { - KASSERT(!locked, - ("sfxge_tx_qdpl_put() failed locked")); + if (locked) + mtx_unlock(&txq->lock); rc = ENOBUFS; goto fail; } diff -r ff9f5d3dbafe -r 7632a3355224 src/driver/freebsd/sfxge_tx.h --- a/head/sys/dev/sfxge/sfxge_tx.h Tue Mar 04 13:15:13 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.h Wed Mar 05 09:06:01 2014 +0400 @@ -75,7 +75,8 @@ enum sfxge_tx_buf_flags flags; }; -#define SFXGE_TX_MAX_DEFERRED 64 +#define SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT 64 +#define SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT 64 /* * Deferred packet list. From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 09:56:24 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E5B842E for ; Tue, 18 Mar 2014 09:56:24 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 459836B3 for ; Tue, 18 Mar 2014 09:56:24 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 449227F570; Tue, 18 Mar 2014 13:56:21 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 449227F570 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395136581; bh=w1Ul4B6kEWBSzOe3PRXqLAm9r6KKwuS/T9fbY4wWZEA=; l=575; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=haK5MAm1xydMX4vx3nNkSvTc/50e7lwJpOHIoO0GhMNXfNNQkaoMAaDs1n9ju5KGQ J7Z9cGo42+L1kVxMYY4pyTHXgN9i+bIyUeAYMoam+WIp2kLBFXSGkPMY/WG/v8bA4S FA4DEJ1QjfJRzfrkRDnapR6laROwQhc/z6OG5tFY= Message-ID: <53281846.2070308@oktetlabs.ru> Date: Tue, 18 Mar 2014 13:56:22 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 3/6] sfxge: return error when packet is dropped because of link down Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 09:56:24 -0000 sfxge: return error when packet is dropped because of link down Submitted-by: Boris Misenov Sponsored by: Solarflare Communications, Inc. diff -r d292c9f51d36 -r 53935db50f8a src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Thu Mar 06 13:38:55 2014 +0000 +++ b/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 @@ -589,7 +589,7 @@ if (!SFXGE_LINK_UP(sc)) { m_freem(m); - return (0); + return (ENETDOWN); } /* Pick the desired transmit queue. */ From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 09:58:42 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8C144E3 for ; Tue, 18 Mar 2014 09:58:42 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 70DB46D2 for ; Tue, 18 Mar 2014 09:58:42 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 3F05F7F54C; Tue, 18 Mar 2014 13:58:39 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 3F05F7F54C DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395136719; bh=A2B23ZS4UG739CN9KKv/82ASKIDcm2Svo/3q16yTa6Y=; l=1882; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=gOyjzLP1Dt8hHFZFQ2yqn350PCPkoST62rHPX2sTPZLBqMBgz90trEFsbT4WDkvbr Z8Hr9++SjA4DjO5YyQS7EJ0Cjf5NAfhqsGtUUxNDkjgHis3yEaJy/y8F6JRTkau1Ic 1ryNmvkV0aeQxmrtRrylTfnIh71Vz22EeUdYFDlE= Message-ID: <532818D0.4080006@oktetlabs.ru> Date: Tue, 18 Mar 2014 13:58:40 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 4/6] sfxge: add counter for Tx errors returned from if_transmit Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 09:58:42 -0000 sfxge: add counter for Tx errors returned from if_transmit Submitted-by: Boris Misenov Sponsored by: Solarflare Communications, Inc. diff -r 53935db50f8a -r af2586a023d8 src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 @@ -503,6 +503,11 @@ int locked; int rc; + if (!SFXGE_LINK_UP(txq->sc)) { + rc = ENETDOWN; + goto fail; + } + /* * Try to grab the txq lock. If we are able to get the lock, * the packet will be appended to the "get list" of the deferred @@ -537,6 +542,7 @@ fail: m_freem(m); + atomic_add_long(&txq->early_drops, 1); return (rc); } @@ -587,11 +593,6 @@ KASSERT(ifp->if_flags & IFF_UP, ("interface not up")); - if (!SFXGE_LINK_UP(sc)) { - m_freem(m); - return (ENETDOWN); - } - /* Pick the desired transmit queue. */ if (m->m_pkthdr.csum_flags & (CSUM_DELAY_DATA | CSUM_TSO)) { int index = 0; @@ -1391,6 +1392,7 @@ SFXGE_TX_STAT(tso_long_headers, tso_long_headers), SFXGE_TX_STAT(tx_collapses, collapses), SFXGE_TX_STAT(tx_drops, drops), + SFXGE_TX_STAT(tx_early_drops, early_drops), }; static int diff -r 53935db50f8a -r af2586a023d8 src/driver/freebsd/sfxge_tx.h --- a/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 @@ -160,6 +160,7 @@ unsigned long tso_long_headers; unsigned long collapses; unsigned long drops; + unsigned long early_drops; /* The following fields change more often, and are used mostly * on the completion path From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 09:59:50 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAFC8588 for ; Tue, 18 Mar 2014 09:59:50 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 729EB6E3 for ; Tue, 18 Mar 2014 09:59:50 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 68EE07F572; Tue, 18 Mar 2014 13:59:47 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 68EE07F572 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395136787; bh=w418HQ5g99hSGrA6pyEIUA6adONHXRjOhdmKWP97TOk=; l=1943; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=eEo4Rb5kS/XTsC5C5BgeEYQFojfp650s1sYD+WUtg3RdgC5U+CYza9hqZThZDatkF fNQSYMiWyVt1ldgHi53cY+mw/s2TOQygbAqUbpF7hN9zniAZur5dx5gw6j3il892p1 OFJjO8YUBApON3oNxZWZl+XpevyhkRisHIPPTxX0= Message-ID: <53281914.1000309@oktetlabs.ru> Date: Tue, 18 Mar 2014 13:59:48 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 5/6] sfxge: access statistics buffers under port lock Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 09:59:50 -0000 sfxge: access statistics buffers under port lock Allow access to statistics data not only from sysctl handlers. Submitted-by: Boris Misenov Sponsored by: Solarflare Communications, Inc. diff -r af2586a023d8 -r 7f58b1a5ea60 src/driver/freebsd/sfxge_port.c --- a/head/sys/dev/sfxge/sfxge_port.c Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_port.c Mon Mar 10 11:37:12 2014 +0400 @@ -48,7 +48,7 @@ unsigned int count; int rc; - mtx_lock(&port->lock); + mtx_assert(&port->lock, MA_OWNED); if (port->init_state != SFXGE_PORT_STARTED) { rc = 0; @@ -82,7 +82,6 @@ rc = ETIMEDOUT; out: - mtx_unlock(&port->lock); return rc; } @@ -93,12 +92,16 @@ unsigned int id = arg2; int rc; + mtx_lock(&sc->port.lock); if ((rc = sfxge_mac_stat_update(sc)) != 0) - return rc; + goto out; - return SYSCTL_OUT(req, + rc = SYSCTL_OUT(req, (uint64_t *)sc->port.mac_stats.decode_buf + id, sizeof(uint64_t)); +out: + mtx_unlock(&sc->port.lock); + return rc; } static void @@ -442,7 +445,7 @@ unsigned int count; int rc; - mtx_lock(&port->lock); + mtx_assert(&port->lock, MA_OWNED); if (port->init_state != SFXGE_PORT_STARTED) { rc = 0; @@ -476,7 +479,6 @@ rc = ETIMEDOUT; out: - mtx_unlock(&port->lock); return rc; } @@ -487,12 +489,16 @@ unsigned int id = arg2; int rc; + mtx_lock(&sc->port.lock); if ((rc = sfxge_phy_stat_update(sc)) != 0) - return rc; + goto out; - return SYSCTL_OUT(req, + rc = SYSCTL_OUT(req, (uint32_t *)sc->port.phy_stats.decode_buf + id, sizeof(uint32_t)); +out: + mtx_unlock(&sc->port.lock); + return rc; } static void From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 10:00:45 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C4BC63B for ; Tue, 18 Mar 2014 10:00:45 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD4E5753 for ; Tue, 18 Mar 2014 10:00:44 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 3CEB37F4AE; Tue, 18 Mar 2014 14:00:43 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 3CEB37F4AE DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395136843; bh=OZuqSmyhldE0c0jTDLXWmFrh7CxszXxbOWPFa5kdl9M=; l=6413; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=uwJXHfFmUEnYOYs2qkXh43RKLPJ+0Nh1FdtoDgpBbxrtpM+nSGTMMGxzIvaVJPXy+ C0I41e7luVj5IZH1NCTVkk7mdGQmimh8/UkjWcMnQ1/b3JWR3hc7EDSCvQ+opb+9Hw 5UnYfsDeDUTPJvn5+5C2ggNvsuIAet+z5tGzrlHg= Message-ID: <5328194B.60707@oktetlabs.ru> Date: Tue, 18 Mar 2014 14:00:43 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 6/6] sfxge: implement interface statistics shown by netstat Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 10:00:45 -0000 sfxge: implement interface statistics shown by netstat netstat directly reads interface statistics collected in the ifnet structure members: if_ipackets, if_ierrors, if_iqdrops, if_opackets, if_oerrors, if_collisions. The if_oerrors counter should include both errors reported by hardware and errors happened in software before posting to hardware. Since statistics is retrieved periodically its counters could be smaller than counters provided by kernel for IP addresses. Report IFCAP_HWSTATS capability since driver manages if_ibytes. Submitted-by: Boris Misenov Sponsored by: Solarflare Communications, Inc. diff -r 7f58b1a5ea60 -r a3ab0749ffa3 src/driver/freebsd/sfxge.c --- a/head/sys/dev/sfxge/sfxge.c Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge.c Mon Mar 10 11:37:12 2014 +0400 @@ -60,10 +60,10 @@ #define SFXGE_CAP (IFCAP_VLAN_MTU | \ IFCAP_HWCSUM | IFCAP_VLAN_HWCSUM | IFCAP_TSO | \ IFCAP_JUMBO_MTU | IFCAP_LRO | \ - IFCAP_VLAN_HWTSO | IFCAP_LINKSTATE) + IFCAP_VLAN_HWTSO | IFCAP_LINKSTATE | IFCAP_HWSTATS) #define SFXGE_CAP_ENABLE SFXGE_CAP #define SFXGE_CAP_FIXED (IFCAP_VLAN_MTU | IFCAP_HWCSUM | IFCAP_VLAN_HWCSUM | \ - IFCAP_JUMBO_MTU | IFCAP_LINKSTATE) + IFCAP_JUMBO_MTU | IFCAP_LINKSTATE | IFCAP_HWSTATS) MALLOC_DEFINE(M_SFXGE, "sfxge", "Solarflare 10GigE driver"); @@ -274,10 +274,23 @@ } static void +sfxge_tick(void *arg) +{ + struct sfxge_softc *sc = arg; + + sfxge_port_update_stats(sc); + sfxge_tx_update_stats(sc); + + callout_reset(&sc->tick_callout, SFXGE_CALLOUT_TICKS, sfxge_tick, sc); +} + +static void sfxge_ifnet_fini(struct ifnet *ifp) { struct sfxge_softc *sc = ifp->if_softc; + callout_drain(&sc->tick_callout); + sx_xlock(&sc->softc_lock); sfxge_stop(sc); sx_xunlock(&sc->softc_lock); @@ -321,9 +334,12 @@ mtx_init(&sc->tx_lock, "txq", NULL, MTX_DEF); #endif + callout_init(&sc->tick_callout, TRUE); + if ((rc = sfxge_port_ifmedia_init(sc)) != 0) goto fail; + callout_reset(&sc->tick_callout, SFXGE_CALLOUT_TICKS, sfxge_tick, sc); return 0; fail: diff -r 7f58b1a5ea60 -r a3ab0749ffa3 src/driver/freebsd/sfxge.h --- a/head/sys/dev/sfxge/sfxge.h Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge.h Mon Mar 10 11:37:12 2014 +0400 @@ -61,6 +61,9 @@ #ifndef IFCAP_VLAN_HWTSO #define IFCAP_VLAN_HWTSO 0 #endif +#ifndef IFCAP_HWSTATS +#define IFCAP_HWSTATS 0 +#endif #ifndef IFM_10G_T #define IFM_10G_T IFM_UNKNOWN #endif @@ -100,6 +103,8 @@ #define SFXGE_EV_BATCH 16384 +#define SFXGE_CALLOUT_TICKS 10 + struct sfxge_evq { struct sfxge_softc *sc __aligned(CACHE_LINE_SIZE); struct mtx lock __aligned(CACHE_LINE_SIZE); @@ -241,6 +246,7 @@ #ifndef SFXGE_HAVE_MQ struct mtx tx_lock __aligned(CACHE_LINE_SIZE); #endif + struct callout tick_callout; }; #define SFXGE_LINK_UP(sc) ((sc)->port.link_mode != EFX_LINK_DOWN) @@ -298,6 +304,7 @@ efx_link_mode_t mode); extern int sfxge_mac_filter_set(struct sfxge_softc *sc); extern int sfxge_port_ifmedia_init(struct sfxge_softc *sc); +extern void sfxge_port_update_stats(struct sfxge_softc *sc); #define SFXGE_MAX_MTU (9 * 1024) diff -r 7f58b1a5ea60 -r a3ab0749ffa3 src/driver/freebsd/sfxge_port.c --- a/head/sys/dev/sfxge/sfxge_port.c Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_port.c Mon Mar 10 11:37:12 2014 +0400 @@ -85,6 +85,40 @@ return rc; } +void +sfxge_port_update_stats(struct sfxge_softc *sc) +{ + struct ifnet *ifp; + uint64_t *mac_stats; + + mtx_lock(&sc->port.lock); + /* Ignore error and use old values */ + (void)sfxge_mac_stat_update(sc); + + ifp = sc->ifnet; + mac_stats = (uint64_t *)sc->port.mac_stats.decode_buf; + + ifp->if_ipackets = mac_stats[EFX_MAC_RX_PKTS]; + ifp->if_ierrors = mac_stats[EFX_MAC_RX_ERRORS]; + ifp->if_opackets = mac_stats[EFX_MAC_TX_PKTS]; + ifp->if_oerrors = mac_stats[EFX_MAC_TX_ERRORS]; + ifp->if_collisions = + mac_stats[EFX_MAC_TX_SGL_COL_PKTS] + + mac_stats[EFX_MAC_TX_MULT_COL_PKTS] + + mac_stats[EFX_MAC_TX_EX_COL_PKTS] + + mac_stats[EFX_MAC_TX_LATE_COL_PKTS]; + ifp->if_ibytes = mac_stats[EFX_MAC_RX_OCTETS]; + ifp->if_obytes = mac_stats[EFX_MAC_TX_OCTETS]; + /* if_imcasts is maintained in net/if_ethersubr.c */ + ifp->if_omcasts = + mac_stats[EFX_MAC_TX_MULTICST_PKTS] + + mac_stats[EFX_MAC_TX_BRDCST_PKTS]; + /* if_iqdrops is maintained in net/if_ethersubr.c */ + /* if_noproto is maintained in net/if_ethersubr.c */ + + mtx_unlock(&sc->port.lock); +} + static int sfxge_mac_stat_handler(SYSCTL_HANDLER_ARGS) { diff -r 7f58b1a5ea60 -r a3ab0749ffa3 src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 @@ -1436,6 +1436,28 @@ } void +sfxge_tx_update_stats(struct sfxge_softc *sc) +{ + unsigned int index; + u_long oerrors = 0; + struct sfxge_txq *txq; + + /* Sum across all TX queues */ + for (index = 0; + index < SFXGE_TXQ_IP_TCP_UDP_CKSUM + SFXGE_TX_SCALE(sc); + index++) { + txq = sc->txq[index]; + /* + * In theory, txq->drops should be obtained under txq lock, + * and txq->early_drops should use atomic operation, + * but it is just statistics. + */ + oerrors += txq->drops + txq->early_drops; + } + sc->ifnet->if_oerrors += oerrors; +} + +void sfxge_tx_fini(struct sfxge_softc *sc) { int index; diff -r 7f58b1a5ea60 -r a3ab0749ffa3 src/driver/freebsd/sfxge_tx.h --- a/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 @@ -170,6 +170,7 @@ }; extern int sfxge_tx_packet_add(struct sfxge_txq *, struct mbuf *); +extern void sfxge_tx_update_stats(struct sfxge_softc *sc); extern int sfxge_tx_init(struct sfxge_softc *sc); extern void sfxge_tx_fini(struct sfxge_softc *sc); From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 11:10:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCDBE718 for ; Tue, 18 Mar 2014 11:10:50 +0000 (UTC) Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B0615E67 for ; Tue, 18 Mar 2014 11:10:50 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id p10so6943233pdj.17 for ; Tue, 18 Mar 2014 04:10:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=dU/W8yp+fVvZT0v3xX+0E/SUvpMuC9XBzCDEhStU1Xo=; b=MZi4sWlv2pUl5u7ZePLLzXPl4eFVR2w9Rlsa+r2Jj3vU9yRGu3CkBHQgbZvwInUQ3y 5xX0xFRdUEwQQUZsuvCkViWDcBRMjzaXV0qBz2VnUKz3/A9hQhUqwaEKDPeJHBuTVaC8 SJuLOsML+8kI5EHRE19PmD8onl+sclBF53DDUdBU/wJK+om35BR0noeTcdj57jtHarM7 hZelgmKjjihqw+bgrsXDyhg3ofdyMEURc4YmwfXg9PeuNnDY7XS8QIU7ge4tvv3CEfm6 9yDGH2tC0KirGByU010Z9byV5SfEYhnrUnGpmXFaEJMcQbOg5ll4Nl/FGpguUay+clEO lXow== X-Gm-Message-State: ALoCoQl/jRcxRK+1R0uF9OB0C+schpNR+Kx/s3PLCsGVKddMeFrXGoLzvbzanAlEuhAmpUwDuG3e MIME-Version: 1.0 X-Received: by 10.68.239.137 with SMTP id vs9mr33456824pbc.84.1395141049838; Tue, 18 Mar 2014 04:10:49 -0700 (PDT) Received: by 10.68.111.37 with HTTP; Tue, 18 Mar 2014 04:10:49 -0700 (PDT) In-Reply-To: <201403172242.s2HMg7xP031044@freefall.freebsd.org> References: <201403172242.s2HMg7xP031044@freefall.freebsd.org> Date: Tue, 18 Mar 2014 12:10:49 +0100 Message-ID: Subject: Re: kern/183390: [ixgbe] 10gigabit networking problems From: Johan Kooijman To: delphij@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: jfv@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 11:10:50 -0000 Keep these two threads in mind, I believe it's the same issue: http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037967.html http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038061.html On Mon, Mar 17, 2014 at 11:42 PM, wrote: > Synopsis: [ixgbe] 10gigabit networking problems > > Responsible-Changed-From-To: freebsd-net->jfv > Responsible-Changed-By: delphij > Responsible-Changed-When: Mon Mar 17 22:41:15 UTC 2014 > Responsible-Changed-Why: > Hi, Jack, > > Some FreeNAS users [1] have encountered similar issue too, can you take > a look at this one? > > Thanks in advance! > > [1] https://bugs.freenas.org/issues/4560 > > http://www.freebsd.org/cgi/query-pr.cgi?pr=183390 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 11:44:01 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5DF4F7B for ; Tue, 18 Mar 2014 11:44:01 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 78925275 for ; Tue, 18 Mar 2014 11:44:01 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so1709987qcz.30 for ; Tue, 18 Mar 2014 04:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ljcZ16yVhEe6EOGOofFP1YyW63KoCEZ4Ob8KKJeImW8=; b=UYBgbOjclo1TapwCnRFvok4Y5wbOmKCmnhA5ekkSRfSNQEersCI+trRl+66bzlyp0V AO7cFeGYQMGqu8SkqInZ+NECGXFVQvksQBb84Z2rXs3/aCQyp//zbIyrxNj0bD0b6X+C MtC5oBShst5+8doqbkCWzaObkqqHphUwNNCrTThNz6sapwbUecqr17g4SOkjED5ktDnx lAT6hEqjGp/Z2lAkFVThumSQ4aYcz86cTfBE8LPNH36yPRgINFST+ITC4YxiUyGcRA1Q /ne2E1txJPwzn/XkMDSywUE+xFyYB0/JoIPEdxaFcQEDAThqryJ9xkS/1J05nbGTwgtj lCWg== MIME-Version: 1.0 X-Received: by 10.140.50.169 with SMTP id s38mr33061253qga.12.1395143040700; Tue, 18 Mar 2014 04:44:00 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.8.137 with HTTP; Tue, 18 Mar 2014 04:44:00 -0700 (PDT) In-Reply-To: <53280DB3.4080900@oktetlabs.ru> References: <53280DB3.4080900@oktetlabs.ru> Date: Tue, 18 Mar 2014 04:44:00 -0700 X-Google-Sender-Auth: Xw2xnxzTM5RUTWOnAwHN5pW8SgI Message-ID: Subject: Re: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue From: Adrian Chadd To: Andrew Rybchenko Content-Type: text/plain; charset=ISO-8859-1 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 11:44:01 -0000 Hi! Who's the solarflare driver maintainer? Ah, there isn't one. The closest is Ben Hutchings . I can commit these if no-one else is willing but I don't have any solarflare hardware to test it on. -a On 18 March 2014 02:11, Andrew Rybchenko wrote: > > sfxge: fix mbuf leak if it does not fit in software queue > > mbuf should be owned by if_transmit function in any case. > > Submitted-by: Andrew Rybchenko > Sponsored by: Solarflare Communications, Inc. > > diff -r e2bc8f64f1b2 -r ff9f5d3dbafe src/driver/freebsd/sfxge_tx.c > --- a/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:13:05 2014 +0400 > +++ b/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:15:13 2014 +0400 > @@ -536,6 +536,7 @@ > return (0); > > fail: > + m_freem(m); > return (rc); > > } > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 12:46:50 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F7D530C for ; Tue, 18 Mar 2014 12:46:50 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 046C3B8E for ; Tue, 18 Mar 2014 12:46:49 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2ICkOUC003846 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Mar 2014 16:46:24 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2ICkOOv003845; Tue, 18 Mar 2014 16:46:24 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 18 Mar 2014 16:46:24 +0400 From: Gleb Smirnoff To: Andrew Rybchenko Subject: Re: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue Message-ID: <20140318124624.GD1499@FreeBSD.org> References: <53280DB3.4080900@oktetlabs.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="T4sUOijqQbZv57TR" Content-Disposition: inline In-Reply-To: <53280DB3.4080900@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 12:46:50 -0000 --T4sUOijqQbZv57TR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Andrew, On Tue, Mar 18, 2014 at 01:11:15PM +0400, Andrew Rybchenko wrote: A> A> sfxge: fix mbuf leak if it does not fit in software queue A> A> mbuf should be owned by if_transmit function in any case. A> A> Submitted-by: Andrew Rybchenko A> Sponsored by: Solarflare Communications, Inc. Can we simplify the function while here? -- Totus tuus, Glebius. --T4sUOijqQbZv57TR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="1.diff" Index: sfxge_tx.c =================================================================== --- sfxge_tx.c (revision 263296) +++ sfxge_tx.c (working copy) @@ -498,7 +498,6 @@ int sfxge_tx_packet_add(struct sfxge_txq *txq, struct mbuf *m) { int locked; - int rc; /* * Try to grab the txq lock. If we are able to get the lock, @@ -511,10 +510,9 @@ sfxge_tx_packet_add(struct sfxge_txq *txq, struct * Can only fail if we weren't able to get the lock. */ if (sfxge_tx_qdpl_put(txq, m, locked) != 0) { - KASSERT(!locked, - ("sfxge_tx_qdpl_put() failed locked")); - rc = ENOBUFS; - goto fail; + KASSERT(!locked, ("sfxge_tx_qdpl_put() failed locked")); + m_freem(m); + return (ENOBUFS); } /* @@ -534,10 +532,6 @@ sfxge_tx_packet_add(struct sfxge_txq *txq, struct } return (0); - -fail: - return (rc); - } static void --T4sUOijqQbZv57TR-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 12:50:20 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D632E3E3 for ; Tue, 18 Mar 2014 12:50:20 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 46345C27 for ; Tue, 18 Mar 2014 12:50:19 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2ICoI10003863 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Mar 2014 16:50:18 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2ICoIHH003862; Tue, 18 Mar 2014 16:50:18 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 18 Mar 2014 16:50:18 +0400 From: Gleb Smirnoff To: Andrew Rybchenko Subject: Re: [PATCH 3/6] sfxge: return error when packet is dropped because of link down Message-ID: <20140318125018.GE1499@FreeBSD.org> References: <53281846.2070308@oktetlabs.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53281846.2070308@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 12:50:20 -0000 On Tue, Mar 18, 2014 at 01:56:22PM +0400, Andrew Rybchenko wrote: A> sfxge: return error when packet is dropped because of link down A> A> Submitted-by: Boris Misenov A> Sponsored by: Solarflare Communications, Inc. Committed, thanks. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 12:59:23 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C08EF939 for ; Tue, 18 Mar 2014 12:59:23 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5BDF7CD7 for ; Tue, 18 Mar 2014 12:59:22 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2ICxL7L003917 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Mar 2014 16:59:21 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2ICxLYL003916; Tue, 18 Mar 2014 16:59:21 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 18 Mar 2014 16:59:21 +0400 From: Gleb Smirnoff To: Andrew Rybchenko , Boris Misenov Subject: Re: [PATCH 4/6] sfxge: add counter for Tx errors returned from if_transmit Message-ID: <20140318125921.GF1499@FreeBSD.org> References: <532818D0.4080006@oktetlabs.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <532818D0.4080006@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 12:59:23 -0000 Andrew, Boris, On Tue, Mar 18, 2014 at 01:58:40PM +0400, Andrew Rybchenko wrote: A> sfxge: add counter for Tx errors returned from if_transmit A> A> Submitted-by: Boris Misenov A> Sponsored by: Solarflare Communications, Inc. I'd suggest not to use atomic(9) increment there, since it locks memory bus and thus has performance impact. Using ++ would be fine, since we probably don't care about absolute precision of this counter. However, if you are interested in precision and also in performance, I'd suggest you to convert all statistics in sfxge(4) that are shared between CPUs to the counter(9) framework. More info on it can be found in manual page and some measurements are available here: http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html If you insist, I can apply your patch as is. The only problem is that when you send patches inlined into email, your MUA mangles all TABs to spaces, so I can't apply your patches. If it possible, next time send them as attachments. A> diff -r 53935db50f8a -r af2586a023d8 src/driver/freebsd/sfxge_tx.c A> --- a/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 A> +++ b/head/sys/dev/sfxge/sfxge_tx.c Mon Mar 10 11:37:12 2014 +0400 A> @@ -503,6 +503,11 @@ A> int locked; A> int rc; A> A> + if (!SFXGE_LINK_UP(txq->sc)) { A> + rc = ENETDOWN; A> + goto fail; A> + } A> + A> /* A> * Try to grab the txq lock. If we are able to get the lock, A> * the packet will be appended to the "get list" of the deferred A> @@ -537,6 +542,7 @@ A> A> fail: A> m_freem(m); A> + atomic_add_long(&txq->early_drops, 1); A> return (rc); A> A> } A> @@ -587,11 +593,6 @@ A> A> KASSERT(ifp->if_flags & IFF_UP, ("interface not up")); A> A> - if (!SFXGE_LINK_UP(sc)) { A> - m_freem(m); A> - return (ENETDOWN); A> - } A> - A> /* Pick the desired transmit queue. */ A> if (m->m_pkthdr.csum_flags & (CSUM_DELAY_DATA | CSUM_TSO)) { A> int index = 0; A> @@ -1391,6 +1392,7 @@ A> SFXGE_TX_STAT(tso_long_headers, tso_long_headers), A> SFXGE_TX_STAT(tx_collapses, collapses), A> SFXGE_TX_STAT(tx_drops, drops), A> + SFXGE_TX_STAT(tx_early_drops, early_drops), A> }; A> A> static int A> diff -r 53935db50f8a -r af2586a023d8 src/driver/freebsd/sfxge_tx.h A> --- a/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 A> +++ b/head/sys/dev/sfxge/sfxge_tx.h Mon Mar 10 11:37:12 2014 +0400 A> @@ -160,6 +160,7 @@ A> unsigned long tso_long_headers; A> unsigned long collapses; A> unsigned long drops; A> + unsigned long early_drops; A> A> /* The following fields change more often, and are used mostly A> * on the completion path A> A> _______________________________________________ A> freebsd-net@freebsd.org mailing list A> http://lists.freebsd.org/mailman/listinfo/freebsd-net A> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 13:05:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67D7ABE4 for ; Tue, 18 Mar 2014 13:05:53 +0000 (UTC) Received: from mail.egr.msu.edu (dauterive.egr.msu.edu [35.9.37.168]) by mx1.freebsd.org (Postfix) with ESMTP id 3CA35DB4 for ; Tue, 18 Mar 2014 13:05:52 +0000 (UTC) Received: from dauterive (localhost [127.0.0.1]) by mail.egr.msu.edu (Postfix) with ESMTP id 59922293B5 for ; Tue, 18 Mar 2014 09:05:45 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mail.egr.msu.edu ([127.0.0.1]) by dauterive (dauterive.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1nSHxv_Q6BjB for ; Tue, 18 Mar 2014 09:05:45 -0400 (EDT) Received: from EGR authenticated sender Message-ID: <532844A9.3020100@egr.msu.edu> Date: Tue, 18 Mar 2014 09:05:45 -0400 From: Adam McDougall User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: vlanXXX vs ifXX.YY notation References: <20140318074246.8519@smtp.new-ukraine.org> In-Reply-To: <20140318074246.8519@smtp.new-ukraine.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 13:05:53 -0000 On 03/18/2014 01:42, Zeus Panchenko wrote: > hi, > > is there any advantage of using vlanXXX vs ifXX.YY notation? > > I mean > >> ifconfig em0 > vlan777: flags=8843 metric 0 mtu 1500 > ether 00:1b:b9:8b:ca:33 > ... > vlan: 777 parent interface: em0 > > vs > > em0.555: flags=8843 metric 0 mtu 1500 > ether 00:1b:b9:8b:ca:33 > ... > vlan: 555 parent interface: em0 > > - -- > Zeus V. Panchenko jid:zeus@im.ibs.dn.ua > IT Dpt., I.B.S. LLC GMT+2 (EET) It would seem to have an advantage on a bridge where you are passing a vlan through multiple interfaces since 'vlan777' cannot represent both the incoming and outgoing vlan interfaces, but em0.555 and em1.555 seem quite logical. It gives an easy way to differentiate the same vlan on more than one parent interface. Additionally the native names are shorter which often helps in status reporting utilities that truncate the name for display. From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 13:24:43 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B907D1BF for ; Tue, 18 Mar 2014 13:24:43 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 524BCFB3 for ; Tue, 18 Mar 2014 13:24:42 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2IDOe36004081 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Mar 2014 17:24:40 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2IDOeFx004080; Tue, 18 Mar 2014 17:24:40 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 18 Mar 2014 17:24:40 +0400 From: Gleb Smirnoff To: Andrew Rybchenko Subject: Re: [PATCH 2/6] sfxge: limit software Tx queue size Message-ID: <20140318132440.GG1499@FreeBSD.org> References: <532817F5.8010505@oktetlabs.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="JYK4vJDZwFMowpUq" Content-Disposition: inline In-Reply-To: <532817F5.8010505@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 13:24:43 -0000 --JYK4vJDZwFMowpUq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Andrew, On Tue, Mar 18, 2014 at 01:55:01PM +0400, Andrew Rybchenko wrote: A> sfxge: limit software Tx queue size A> A> Previous implementation limits put queue size only (when Tx lock can't A> be acquired), A> but get queue may grow unboundedly which results in mbuf pools A> exhaustion and A> latency growth. A> A> Submitted-by: Andrew Rybchenko A> Sponsored by: Solarflare Communications, Inc. The interaction between sfxge_tx_qdpl_put() and sfxge_tx_packet_add() is quite complex and I couldn't resist from suggesting you to simplify the code. Can you please look into attached patch? - Inline sfxge_tx_qdpl_put() into sfxge_tx_packet_add(). - Simplify the 'locked' logic. - Add your PATCH 1/6, the mbuf leak fix. - Add your PATCH 2/6, the SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT check. -- Totus tuus, Glebius. --JYK4vJDZwFMowpUq Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="2.diff" Index: sfxge_tx.c =================================================================== --- sfxge_tx.c (revision 263297) +++ sfxge_tx.c (working copy) @@ -437,32 +437,39 @@ sfxge_tx_qdpl_service(struct sfxge_txq *txq) } /* - * Put a packet on the deferred packet list. - * - * If we are called with the txq lock held, we put the packet on the "get - * list", otherwise we atomically push it on the "put list". The swizzle - * function takes care of ordering. - * - * The length of the put list is bounded by SFXGE_TX_MAX_DEFFERED. We - * overload the csum_data field in the mbuf to keep track of this length - * because there is no cheap alternative to avoid races. + * Called from if_transmit - will try to grab the txq lock and enqueue to the + * put list if it succeeds, otherwise will push onto the defer list. */ -static inline int -sfxge_tx_qdpl_put(struct sfxge_txq *txq, struct mbuf *mbuf, int locked) +int +sfxge_tx_packet_add(struct sfxge_txq *txq, struct mbuf *m) { - struct sfxge_tx_dpl *stdp; + struct sfxge_tx_dpl *stdp = &txq->dpl; - stdp = &txq->dpl; + KASSERT(m->m_nextpkt == NULL, ("m->m_nextpkt != NULL")); - KASSERT(mbuf->m_nextpkt == NULL, ("mbuf->m_nextpkt != NULL")); + /* + * Try to grab the txq lock. If we are able to get the lock, + * the packet will be appended to the "get list" of the deferred + * packet list. Otherwise, it will be pushed on the "put list". + * The swizzle function takes care of ordering. + * + * The length of the get and put lists are bounded by + * SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT and + * SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT respectively. + * We overload the csum_data field in the mbuf to keep track of this + * length because there is no cheap alternative to avoid races. + */ + if (mtx_trylock(&txq->lock)) { + sfxge_tx_qdpl_swizzle(txq); - if (locked) { - mtx_assert(&txq->lock, MA_OWNED); + if (stdp->std_count >= SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT) { + mtx_unlock(&txq->lock); + m_freem(m); + return (ENOBUFS); + } - sfxge_tx_qdpl_swizzle(txq); - - *(stdp->std_getp) = mbuf; - stdp->std_getp = &mbuf->m_nextpkt; + *(stdp->std_getp) = m; + stdp->std_getp = &m->m_nextpkt; stdp->std_count++; } else { volatile uintptr_t *putp; @@ -471,7 +478,7 @@ sfxge_tx_qdpl_service(struct sfxge_txq *txq) unsigned old_len; putp = &stdp->std_put; - new = (uintptr_t)mbuf; + new = (uintptr_t)m; do { old = *putp; @@ -480,64 +487,30 @@ sfxge_tx_qdpl_service(struct sfxge_txq *txq) old_len = mp->m_pkthdr.csum_data; } else old_len = 0; - if (old_len >= SFXGE_TX_MAX_DEFERRED) - return ENOBUFS; - mbuf->m_pkthdr.csum_data = old_len + 1; - mbuf->m_nextpkt = (void *)old; + if (old_len >= SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT) { + m_freem(m); + return (ENOBUFS); + } + m->m_pkthdr.csum_data = old_len + 1; + m->m_nextpkt = (void *)old; } while (atomic_cmpset_ptr(putp, old, new) == 0); - } - return (0); -} - -/* - * Called from if_transmit - will try to grab the txq lock and enqueue to the - * put list if it succeeds, otherwise will push onto the defer list. - */ -int -sfxge_tx_packet_add(struct sfxge_txq *txq, struct mbuf *m) -{ - int locked; - int rc; - - /* - * Try to grab the txq lock. If we are able to get the lock, - * the packet will be appended to the "get list" of the deferred - * packet list. Otherwise, it will be pushed on the "put list". - */ - locked = mtx_trylock(&txq->lock); - - /* - * Can only fail if we weren't able to get the lock. - */ - if (sfxge_tx_qdpl_put(txq, m, locked) != 0) { - KASSERT(!locked, - ("sfxge_tx_qdpl_put() failed locked")); - rc = ENOBUFS; - goto fail; + /* + * Again try to grab the lock. + * + * If we are able to get the lock, we need to process the + * deferred packet list. If we are not able to get the lock, + * another thread is processing the list, and we return. + */ + if (mtx_trylock(&txq->lock) == 0) + return (0); } - /* - * Try to grab the lock again. - * - * If we are able to get the lock, we need to process the deferred - * packet list. If we are not able to get the lock, another thread - * is processing the list. - */ - if (!locked) - locked = mtx_trylock(&txq->lock); + /* Try to service the list. */ + sfxge_tx_qdpl_service(txq); + /* Lock has been dropped. */ - if (locked) { - /* Try to service the list. */ - sfxge_tx_qdpl_service(txq); - /* Lock has been dropped. */ - } - return (0); - -fail: - return (rc); - } static void Index: sfxge_tx.h =================================================================== --- sfxge_tx.h (revision 263297) +++ sfxge_tx.h (working copy) @@ -75,7 +75,8 @@ struct sfxge_tx_mapping { enum sfxge_tx_buf_flags flags; }; -#define SFXGE_TX_MAX_DEFERRED 64 +#define SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT 64 +#define SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT 64 /* * Deferred packet list. --JYK4vJDZwFMowpUq-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 18 14:54:43 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3FE01BC; Tue, 18 Mar 2014 14:54:43 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0FDCBC42; Tue, 18 Mar 2014 14:54:42 +0000 (UTC) Received: from ip-64-134-43-150.public.wayport.net ([64.134.43.150]:63186 helo=[192.168.10.234]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WPvPb-0007Mp-VA; Tue, 18 Mar 2014 10:54:42 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue From: George Neville-Neil In-Reply-To: Date: Tue, 18 Mar 2014 10:54:28 -0400 X-Mao-Original-Outgoing-Id: 416847268.772767-5a020d8fcacefbccfce9804b4a21d578 Content-Transfer-Encoding: quoted-printable Message-Id: <940D7950-3FE7-453F-BAD1-6E4D5F79DEBD@neville-neil.com> References: <53280DB3.4080900@oktetlabs.ru> To: Adrian Chadd X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "net@freebsd.org" , Andrew Rybchenko X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 14:54:43 -0000 On Mar 18, 2014, at 7:44 , Adrian Chadd wrote: > Hi! >=20 > Who's the solarflare driver maintainer? >=20 > Ah, there isn't one. The closest is Ben Hutchings = . >=20 > I can commit these if no-one else is willing but I don't have any > solarflare hardware to test it on. >=20 >=20 I am taking care of this. I=92ll be testing and commit these patches. There is hardware inbound = to Sentex. Best, George From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 03:28:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0CB9DFA9 for ; Wed, 19 Mar 2014 03:28:40 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7222F75B for ; Wed, 19 Mar 2014 03:28:39 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:64430 helo=minion.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WQ7BD-0003ve-7H; Tue, 18 Mar 2014 23:28:36 -0400 Content-Type: multipart/signed; boundary="Apple-Mail=_77A09F73-CDA1-495E-9871-B5F8C833DB81"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: DCTCP for FreeBSD From: George Neville-Neil In-Reply-To: Date: Tue, 18 Mar 2014 23:28:33 -0400 X-Mao-Original-Outgoing-Id: 416892514.288427-5a74c5f5436bd212f0ff7f4c7430cefe Message-Id: <273DE766-0AAA-4DB3-A3EA-1CFBDE0DBB4F@neville-neil.com> References: To: "Eggert, Lars" X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "freebsd-net@freebsd.org" , Midori Kato X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 03:28:40 -0000 --Apple-Mail=_77A09F73-CDA1-495E-9871-B5F8C833DB81 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Feb 19, 2014, at 4:18 , Eggert, Lars wrote: > Hi, >=20 > Midori Kato has implemented Microsoft's/Stanford's Datacenter TCP = (DCTCP) for FreeBSD as part of her MS thesis with me. Find a patch = attached. >=20 Thanks! Any hints on how best to test this code? Best, George > Also note that we're documenting a specification for DCTCP in an IETF = draft: http://tools.ietf.org/html/draft-bensley-tcpm-dctcp >=20 > Microsoft has made a licensing statement (RAND-Z) on the technology to = the IETF: https://datatracker.ietf.org/ipr/2319/ (I'm not sure what this = means for an eventual inclusion in FreeBSD.) >=20 > Roughly, Midori's patch consists of an extension of the modular = congestion control framework to expose ECN information to the modules, a = module to implement DCTCP, and a few experimental variants. See Midori's = explanation: >=20 >> [1] A change for the modular congestion control framework (See = Section 4.1 if needed) >> DCTCP uses the difference ECN processing from RFC3168. We need to = prepare three functions to do the following ECN processing.=20 >> a) The kernel decides whether an ECE flag should be set in the next = outgoing TCP segment by snooping reserved bits in IP and TCP headers. = (tcp_input.c) >> b) The kernel controls a congestion if an ECE flag is set in an = arriving TCP segment. (tcp_input.c) >> c) After the outgoing TCP segment is generated, the kernel decides = whether an ECT bit should be set in an ECN field of IP header in the = outgoing packet. (tcp_output.c) >> The current framework has no housekeeping functions for (a) and (b). = Therefore, I add two functions into the moduler cc framework: = ecnpkt_handler() and ect_handler(). >>=20 >> - ecnpkt_handler() allows the kernel to do the additional ECN = processing by snooping ECN field in IP and TCP headers. As an option, = this function takes a flag, which tells whether this function is in the = delayed ACK. This function returns an integer value. When the return = value is set, the kernel force to disable delayed ACK. >> - ect_handler() allows the kernel to use different rule from RFC3168 = in terms of an ECT marking in the outgoing segment. This function = returns an integer value. If the value is set, an ECT bit is set to the = outgoing segment. >>=20 >>=20 >> [2] Five changes from the original DCTCP algorithm >> In order to reflect the DCTCP motivation, I modified the following = processing. First four modifications are for senders and the last = modification is for receivers. >>=20 >> (1) no congestion recovery in the receipt of ECE flags (See section = 4.2.1 if needed) >> FreeBSD handles ECN as a congestion event but it's not true for DCTCP = senders. A DCTCP sender uses ECN as a means to understand the extent of = congestions. Therefore, I remove congestion recovery mode in any = situation for DCTCP senders. >>=20 >> (2) selective initial alpha value (See section 4.2.2 if needed)=20 >> DCTCP defines alpha as a parameter to see the depth of a congestion. = When the alpha value is large, it allows a saw-toothed CWND behavior to = a DCTCP sender. >> A problem is that the alpha value is not reliable during a dozen of = RTTs because there is no way to identify the depth of a congestion over = a network from the beginning. When considering the alpha reliability, I = think the initial alpha should be selective for applications by users. = When a user chooses DCTCP for latency-sensitive applications, the = initial alpha is preferred. Otherwise, DCTCP senders had better to set = the initial alpha value to zero from my experimental result (See section = 7.2 of attaching file). >> The default alpha value is set to zero in my implementation. >>=20 >> (3) alpha value initialization after an idle period (See section = 4.2.3 if needed) >> How long an idle period is no longer predictable. Therefore, for a = DCTCP sender, using the out-dated alpha after an idle period is not good = idea. A DCTCP sender resets alpha to the initial value when an idle = period occurs. >>=20 >> The following changes is applied to eliminate a compatibility issue = to standard ECN defined in RFC3465. DCTCP and standard ECN servers have = no way to identify which mechanism is working on the peer. Thus, we need = to eliminate the worst situation in a network mixing DCTCP = senders/receivers and standard ECN senders/receivers. >> (4) using CWR flag when the ECE flag is found for a RTT (See section = 5.1 if needed) >> This change is applied for a situation when a sender uses DCTCP and a = reciever uses standard ECN.=20 >> Under the situation, I find that a DCTCP sender minimizes CWND. The = detailed technical reason is described in section 4.2 of my attaching = file. Fortunately, the current tcp_input() function complement this = change, thus, there is no modification in my patch. >>=20 >> (5) enabling delayed ACK in the receipt of the CWR flag (See section = 5.2 if needed) >> This change is applied for a situation when a sender uses standard = ECN and a reciever uses DCTCP. Under the situation, I find that a = standard ECN sender increases smaller CWND than expected without this = change. The detailed technical reason is described in section 5.2 of my = attaching file. >=20 >=20 > The patch is attached and should apply to a recent -CURRENT. Midori's = thesis (which she refers to in the quoted text above) is at = https://eggert.org/students/kato-thesis.pdf >=20 > Lars >=20 > --Apple-Mail=_77A09F73-CDA1-495E-9871-B5F8C833DB81 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlMpDuIACgkQYdh2wUQKM9KudgCgqLHf+KnuHBnGbH/YNLSd543X FoMAnRW+zY7r8L0tTQFxlBzusREn5U2O =ln+h -----END PGP SIGNATURE----- --Apple-Mail=_77A09F73-CDA1-495E-9871-B5F8C833DB81-- From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 03:33:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 990B0192 for ; Wed, 19 Mar 2014 03:33:25 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 69C8C7F8 for ; Wed, 19 Mar 2014 03:33:25 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:64451 helo=minion.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WQ7Fs-0004Rj-1p; Tue, 18 Mar 2014 23:33:24 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Include port number in "Listen queue overflow" messages From: George Neville-Neil In-Reply-To: Date: Tue, 18 Mar 2014 23:33:23 -0400 X-Mao-Original-Outgoing-Id: 416892803.279549-30537c8e51b987b5d4f253032f0e1ff0 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: hiren panchasara X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 03:33:25 -0000 On Mar 7, 2014, at 1:23 , hiren panchasara = wrote: > I am thinking of committing following change that includes port number > in "Listen queue overflow" messages. >=20 I like it. Best, George > New message would look something like: > sonewconn: pcb 0xfffff8001b155760: Listen queue overflow on port > 13120: 1 already in queue awaiting acceptance (454 occurrences) >=20 > I've recently ran into a situation at $work where I could not catch > the culprit application via "netstat -A" and had to dive into kgdb to > find the port from pcb where this application was listening to. >=20 > IMO, this change will make debugging easier. >=20 > cheers, > Hiren >=20 > Index: sys/kern/uipc_socket.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/kern/uipc_socket.c (revision 262861) > +++ sys/kern/uipc_socket.c (working copy) > @@ -136,6 +136,7 @@ > #include > #include > #include > +#include >=20 > #include >=20 > @@ -491,8 +492,11 @@ > static int overcount; >=20 > struct socket *so; > + struct inpcb *inp; > int over; >=20 > + inp =3D sotoinpcb(head); > + > ACCEPT_LOCK(); > over =3D (head->so_qlen > 3 * head->so_qlimit / 2); > ACCEPT_UNLOCK(); > @@ -504,10 +508,12 @@ > overcount++; >=20 > if (ratecheck(&lastover, &overinterval)) { > - log(LOG_DEBUG, "%s: pcb %p: Listen queue = overflow: " > - "%i already in queue awaiting acceptance " > + log(LOG_DEBUG, "%s: pcb %p: Listen queue = overflow on " > + "port %d: %i already in queue awaiting = acceptance " > "(%d occurrences)\n", > - __func__, head->so_pcb, head->so_qlen, = overcount); > + __func__, head->so_pcb, > + ntohs(inp->inp_inc.inc_lport), = head->so_qlen, > + overcount); >=20 > overcount =3D 0; > } > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 03:34:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1661365 for ; Wed, 19 Mar 2014 03:34:40 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A2B7780C for ; Wed, 19 Mar 2014 03:34:40 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:64545 helo=minion.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WQ7H5-0004ZO-9j; Tue, 18 Mar 2014 23:34:39 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: socket receive buffer size & window updates From: George Neville-Neil In-Reply-To: Date: Tue, 18 Mar 2014 23:34:38 -0400 X-Mao-Original-Outgoing-Id: 416892878.511522-e5bf2ba144ca9fcb67482620684b286e Content-Transfer-Encoding: 7bit Message-Id: <91ECA466-61D8-4F7C-ABAD-D7565B8034DF@neville-neil.com> References: To: Vijay Singh X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 03:34:40 -0000 On Mar 11, 2014, at 13:58 , Vijay Singh wrote: > The socket option handler currently doesn't prevent connecting or connected > sockets from changing their receive buffer sizes. In particular, I ran into > a an application that sets the receive buffer size lower than what it > originally was. > > In tcp_output(), if no data is being sent, there is code that is trying to > decide if a window update is needed. > > If the socket receive buffer size was reduced after a larger window was > already advertized, or perhaps even when there is data in the receive > buffer, it seems to me that the computation in 592 could go -ve, and be > interpreted as a large window update. > > I was led to this issue after observing in packet traces that duplicate > FINs were being sent on close. I tracked it down to this check. Should this > be changed to a check like such? > Interesting. Do you have a bit of test code? Best, George From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 03:35:59 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCCA140F for ; Wed, 19 Mar 2014 03:35:59 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9EEB9823 for ; Wed, 19 Mar 2014 03:35:59 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:64545 helo=minion.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WQ7IL-0004ZO-E3; Tue, 18 Mar 2014 23:35:57 -0400 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [patch] [1/6] sfxge: fix mbuf leak if it does not fit in software queue From: George Neville-Neil In-Reply-To: <532451E2.20407@oktetlabs.ru> Date: Tue, 18 Mar 2014 23:35:56 -0400 X-Mao-Original-Outgoing-Id: 416892956.859391-a7a41149908ddacbdbc58089a04d4810 Content-Transfer-Encoding: quoted-printable Message-Id: <27DF6B25-ABD9-408E-9631-92018CE1F302@neville-neil.com> References: <532451E2.20407@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 03:35:59 -0000 On Mar 15, 2014, at 9:13 , Andrew Rybchenko = wrote: >=20 > <1-sfxge-leak.patch> Committed this morning. Best, George From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 05:00:36 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9B4AED6; Wed, 19 Mar 2014 05:00:36 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6C27DE80; Wed, 19 Mar 2014 05:00:36 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 2CC9E7F4D4; Wed, 19 Mar 2014 09:00:32 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 2CC9E7F4D4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395205232; bh=KeidqrIDIyPqvY5pDucX6Fzmyz9NZXabhozdzrcYXqA=; l=835; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=OshDlL7a90aXhOGo0DqwMz3pe37GI3iydQ/vB1UGs2i8eEtIkcDo945sTQPPQk4WR /f1BlRAt2LLECPlt3PGKpSOJjaptaVa8eLHiTrIu8R2ANCK8UlH5N0PUJ4mVbVNtKn A9ymwbZWqp7xZm9lNbRjzZ1YXyP2Gtd9un7I+pfU= Message-ID: <53292470.801@oktetlabs.ru> Date: Wed, 19 Mar 2014 09:00:32 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Gleb Smirnoff Subject: Re: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue References: <53280DB3.4080900@oktetlabs.ru> <20140318124624.GD1499@FreeBSD.org> In-Reply-To: <20140318124624.GD1499@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 05:00:36 -0000 Gleb, On 03/18/2014 04:46 PM, Gleb Smirnoff wrote: > Andrew, > > On Tue, Mar 18, 2014 at 01:11:15PM +0400, Andrew Rybchenko wrote: > A> > A> sfxge: fix mbuf leak if it does not fit in software queue > A> > A> mbuf should be owned by if_transmit function in any case. > A> > A> Submitted-by: Andrew Rybchenko > A> Sponsored by: Solarflare Communications, Inc. > > Can we simplify the function while here? One of the next patches (4/6) moves link down check to the function and uses "fail" label to increment early drops statistics and free mbuf. IMHO, it is really nice to have single place to do it. Thanks, Andrew. -- Andrew Rybchenko OKTET Labs, St.-Petersburg, Russia Web: www.oktetlabs.ru Office: +7 812 7832191 Fax: +7 812 7846591 Mobile: +7 921 7479683 From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 05:44:51 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 243575FC for ; Wed, 19 Mar 2014 05:44:51 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A1731383 for ; Wed, 19 Mar 2014 05:44:49 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2J5iVl0008787 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 19 Mar 2014 09:44:31 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2J5iV1W008786; Wed, 19 Mar 2014 09:44:31 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 19 Mar 2014 09:44:31 +0400 From: Gleb Smirnoff To: Andrew Rybchenko Subject: Re: [PATCH 1/6] sfxge: fix mbuf leak if it does not fit in software queue Message-ID: <20140319054431.GN1499@glebius.int.ru> References: <53280DB3.4080900@oktetlabs.ru> <20140318124624.GD1499@FreeBSD.org> <53292470.801@oktetlabs.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53292470.801@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 05:44:51 -0000 On Wed, Mar 19, 2014 at 09:00:32AM +0400, Andrew Rybchenko wrote: A> Gleb, A> A> On 03/18/2014 04:46 PM, Gleb Smirnoff wrote: A> > Andrew, A> > A> > On Tue, Mar 18, 2014 at 01:11:15PM +0400, Andrew Rybchenko wrote: A> > A> A> > A> sfxge: fix mbuf leak if it does not fit in software queue A> > A> A> > A> mbuf should be owned by if_transmit function in any case. A> > A> A> > A> Submitted-by: Andrew Rybchenko A> > A> Sponsored by: Solarflare Communications, Inc. A> > A> > Can we simplify the function while here? A> One of the next patches (4/6) moves link down check to the function and A> uses "fail" label to increment early drops statistics and free mbuf. A> IMHO, it is really nice to have single place to do it. I've replied to 4/6 and to 3/6 as well. Please consider my suggestions. If you insist on committing your patches as is, then please resubmit them as email attachments. When submitting them inlined, your MUA mangles patches, and they fail to apply. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 05:44:56 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2905683; Wed, 19 Mar 2014 05:44:56 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5E7F1384; Wed, 19 Mar 2014 05:44:56 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id D3ED57F545; Wed, 19 Mar 2014 09:44:53 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru D3ED57F545 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395207893; bh=Ez2bfqoCwD0BFSCu4AMYCMA+yI7Cu6o7uTmhkYfrJtg=; l=5478; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=GKfe2lFtBf+oRI2mNIlZMsU30YEY44B5MIIGl6WUl4qxK+abJ6Ap4NwhwxSgCReKu Tgq3pwfQfxZkXb7rXf6xmGKmpsKVDiPnrt0RwsKCZGKHupWE2xRySxV5Yg97NDKE1B xmdmUshuBTz/DXq4ijW1zESUQc2t3lWRN/bKZVjQ= Message-ID: <53292ED6.5020102@oktetlabs.ru> Date: Wed, 19 Mar 2014 09:44:54 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Gleb Smirnoff , Boris Misenov Subject: Re: [PATCH 4/6] sfxge: add counter for Tx errors returned from if_transmit References: <532818D0.4080006@oktetlabs.ru> <20140318125921.GF1499@FreeBSD.org> In-Reply-To: <20140318125921.GF1499@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 05:44:56 -0000 Gleb, On 03/18/2014 04:59 PM, Gleb Smirnoff wrote: > Andrew, Boris, > > On Tue, Mar 18, 2014 at 01:58:40PM +0400, Andrew Rybchenko wrote: > A> sfxge: add counter for Tx errors returned from if_transmit > A> > A> Submitted-by: Boris Misenov > A> Sponsored by: Solarflare Communications, Inc. > > I'd suggest not to use atomic(9) increment there, since it locks > memory bus and thus has performance impact. Using ++ would be fine, > since we probably don't care about absolute precision of this > counter. We think that usage of atomic here is appropriate since it is not on fast path. CPU has already overfilled both HW and SW Tx queues. > However, if you are interested in precision and also in performance, > I'd suggest you to convert all statistics in sfxge(4) that are shared > between CPUs to the counter(9) framework. Yes, we have seen the framework. It looks really good and would use it in the case of fast path counter. Other problem that it is available since 10.0 only. > More info on it can be found in manual page and some measurements > are available here: > > http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html Thanks a lot. We'll have a look. > If you insist, I can apply your patch as is. The only problem is that > when you send patches inlined into email, your MUA mangles all TABs > to spaces, so I can't apply your patches. If it possible, next time > send them as attachments. I didn't know it. I guess you have seen that I've sent the first patch twice (I'm sorry for that). The first one has patch in attachment, but it is not shown in mailman web interface and I need to download the patch to have a look and I've decided that it is not I think it is very inconvenient. I can include both inline and attached patch, but it will double mail size. Is the any best practice how to send patches to mailing list? Thanks a lot, Andrew. From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 06:05:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC459A31 for ; Wed, 19 Mar 2014 06:05:29 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 87DD4799 for ; Wed, 19 Mar 2014 06:05:29 +0000 (UTC) Received: from Julian-MBP3.local ([12.157.112.67]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s2J65QVq088020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 18 Mar 2014 23:05:27 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <532933A3.4030403@freebsd.org> Date: Tue, 18 Mar 2014 23:05:23 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: George Neville-Neil , hiren panchasara Subject: Re: Include port number in "Listen queue overflow" messages References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 06:05:29 -0000 On 3/18/14, 8:33 PM, George Neville-Neil wrote: > On Mar 7, 2014, at 1:23 , hiren panchasara wrote: > >> I am thinking of committing following change that includes port number >> in "Listen queue overflow" messages. I think it's a good idea. There is even more information available but this is probably enough. >> > I like it. > > Best, > George > >> New message would look something like: >> sonewconn: pcb 0xfffff8001b155760: Listen queue overflow on port >> 13120: 1 already in queue awaiting acceptance (454 occurrences) >> >> I've recently ran into a situation at $work where I could not catch >> the culprit application via "netstat -A" and had to dive into kgdb to >> find the port from pcb where this application was listening to. >> >> IMO, this change will make debugging easier. >> >> cheers, >> Hiren >> >> Index: sys/kern/uipc_socket.c >> =================================================================== >> --- sys/kern/uipc_socket.c (revision 262861) >> +++ sys/kern/uipc_socket.c (working copy) >> @@ -136,6 +136,7 @@ >> #include >> #include >> #include >> +#include >> >> #include >> >> @@ -491,8 +492,11 @@ >> static int overcount; >> >> struct socket *so; >> + struct inpcb *inp; >> int over; >> >> + inp = sotoinpcb(head); >> + >> ACCEPT_LOCK(); >> over = (head->so_qlen > 3 * head->so_qlimit / 2); >> ACCEPT_UNLOCK(); >> @@ -504,10 +508,12 @@ >> overcount++; >> >> if (ratecheck(&lastover, &overinterval)) { >> - log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow: " >> - "%i already in queue awaiting acceptance " >> + log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow on " >> + "port %d: %i already in queue awaiting acceptance " >> "(%d occurrences)\n", >> - __func__, head->so_pcb, head->so_qlen, overcount); >> + __func__, head->so_pcb, >> + ntohs(inp->inp_inc.inc_lport), head->so_qlen, >> + overcount); >> >> overcount = 0; >> } >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 08:37:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C55743BB for ; Wed, 19 Mar 2014 08:37:58 +0000 (UTC) Received: from frv197.fwdcdn.com (frv197.fwdcdn.com [212.42.77.197]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 81934782 for ; Wed, 19 Mar 2014 08:37:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:To:Subject:From:Date; bh=EV+oPZFZRInIGqMssRuFz+i1pdDsBYVMEPf6Q/ieBVo=; b=jiiVCXfiXSR+PlSyo4+COJMBsz+zO82n/3p6+M18TWcYECPwotpGXNSgvk2VPKAjpA+Kh/vW4iVbn3dsNyiE084U416he3KLG1J6U6ihXVOpAUkv4BisPsfx1G6L+AXf5mXBDVQ33oyX7IwemeHmCV6rg7yKIWKD68PGj8lp+Oo=; Received: from [10.10.10.34] (helo=frv34.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1WQC0a-000Fkh-Dw for freebsd-net@freebsd.org; Wed, 19 Mar 2014 10:37:56 +0200 Date: Wed, 19 Mar 2014 10:37:55 +0200 From: wishmaster Subject: sysctl net.inet.ip.fw broken in 10-STABLE with VIMAGE in kernel To: freebsd-net@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1395217932.132431687.7yfij1gv@frv34.fwdcdn.com> MIME-Version: 1.0 Received: from artemrts@ukr.net by frv34.fwdcdn.com; Wed, 19 Mar 2014 10:37:56 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 08:37:58 -0000 Hi, list. Guys, I think somebody have broken something in IPFW/network stack and in the last stable revision I am unable to disable IPFW nor in base system nor in jail. As workaround is adding permission rule. In release-10 this problem is absent. PR here http://www.freebsd.org/cgi/query-pr.cgi?pr=187665 From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 11:23:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6A59C09 for ; Wed, 19 Mar 2014 11:23:21 +0000 (UTC) Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46]) by mx1.freebsd.org (Postfix) with SMTP id 8961C8F7 for ; Wed, 19 Mar 2014 11:23:20 +0000 (UTC) Received: (qmail 94230 invoked by uid 1001); 19 Mar 2014 11:16:38 -0000 Delivered-To: qmda-intercept-freebsd-net@freebsd.org DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; b=NbvrnPGHnOh2i/J3Q+KKpgufsbMcgIjdcpF3oOUr4+IwsABjNqmDxJrYqYKaQ4K8; Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys DomainKey-Trace-MD: h=10; b=31; l=C18R71D32M65F38T27S42M17C39C27; Comments: QMDA 0.3 Received: (qmail 94223 invoked by uid 1001); 19 Mar 2014 11:16:38 -0000 Date: 19 Mar 2014 11:16:38 +0000 Message-ID: <20140319111638.94222.qmail@f5-external.bushwire.net> From: "Mark Delany" To: freebsd-net@freebsd.org Subject: Minor nits with netmap(4) manpage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 11:23:21 -0000 (Luigi's page suggests posting here.) Very recent freebsd10 (r263256) 1) the manpage says "SEE TRANSPARENT MODE" but no such section exists. 2) the manpage refers to NR_RING_NIC_SW when I think it means NR_REG_NIC_SW. 3) No mention is made of access control. I think earlier documentation suggested that you had to run as root, but now it appears to work for any user that has rw access to /dev/netmap. Obvious I guess but just mentioning that access is controlled by the file system, not your uid. 4) epoll/kqueue has conflicting information. An early para says "... and standard OS mechanisms such as select(2), poll(2), epoll(2), kqueue(2)." But a later para says "epoll(2) and kqueue(2) are not supported on netmap file descriptors.". On the matter of transparent mode, it seems that all an application has to do to have a packet proceed up into the host stack is set NS_FORWARD in the ring flags. That's a super-nice feature as is netmap in general. Mark. From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 11:53:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A62136F for ; Wed, 19 Mar 2014 11:53:45 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 863F4B8F for ; Wed, 19 Mar 2014 11:53:44 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id c11so5675291lbj.26 for ; Wed, 19 Mar 2014 04:53:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jNS152ntnu5B1fMM7LS0Q644f9OOTUcKR2nf/sjdHsw=; b=sI7BOkfgdN8MKk4ccmkxSBqqHPcmFl1WRAUUMxlNVv+1NtvfP+xVXaiUwGNBFuv1iB qM7QDwABx8mx/FeXF8S9Ax6APNiGKQ7gE01wcuxIJk9jupG6DIui7BIeKM7gSJdi0kc2 AiBCEDe3W9iAeE0kAjeifTiZUYbGUtcgTY57h1m0tA4jKi9ufXqdsWM5Frk7rkWOKaov UEAaK2V7WvmOhDVQW0q9+oDK57JpXjYNypD5mQX01L1C5+hs8r1fB5ADIZCVdjoliIY3 bh7DavUyF325Y+QosYCoi4stqPcAALf/RcmZqjzw9e7BjFi9SIuDUwjxTq7jPFs3/qCH FZjw== MIME-Version: 1.0 X-Received: by 10.112.150.233 with SMTP id ul9mr24394900lbb.2.1395230022090; Wed, 19 Mar 2014 04:53:42 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Wed, 19 Mar 2014 04:53:42 -0700 (PDT) In-Reply-To: <20140319111638.94222.qmail@f5-external.bushwire.net> References: <20140319111638.94222.qmail@f5-external.bushwire.net> Date: Wed, 19 Mar 2014 12:53:42 +0100 X-Google-Sender-Auth: l_PNE_V6Ns3JXLKbeWFSiTJXFDA Message-ID: Subject: Re: Minor nits with netmap(4) manpage From: Luigi Rizzo To: Mark Delany Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 11:53:45 -0000 thanks for the suggestions we'll integrate them cheers luigi On Wed, Mar 19, 2014 at 12:16 PM, Mark Delany wrote: > (Luigi's page suggests posting here.) > > Very recent freebsd10 (r263256) > > 1) the manpage says "SEE TRANSPARENT MODE" but no such section > exists. > > 2) the manpage refers to NR_RING_NIC_SW when I think it means > NR_REG_NIC_SW. > > 3) No mention is made of access control. I think earlier documentation > suggested that you had to run as root, but now it appears to work > for any user that has rw access to /dev/netmap. Obvious I guess but > just mentioning that access is controlled by the file system, not > your uid. > > 4) epoll/kqueue has conflicting information. An early para says > "... and standard OS mechanisms such as select(2), poll(2), > epoll(2), kqueue(2)." > > But a later para says "epoll(2) and kqueue(2) are not supported on > netmap file descriptors.". > > > On the matter of transparent mode, it seems that all an application > has to do to have a packet proceed up into the host stack is set > NS_FORWARD in the ring flags. That's a super-nice feature as is netmap > in general. > > > Mark. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 16:35:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96071943; Wed, 19 Mar 2014 16:35:48 +0000 (UTC) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 604E4DCF; Wed, 19 Mar 2014 16:35:48 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id rq2so9181099pbb.33 for ; Wed, 19 Mar 2014 09:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=f8eSW6iS8reDprmjdco7GlMWFx6uD0Vbg0050So2TqY=; b=rwzLfslZxKOFHXRnk3N1uGQ7+lC3r3EDSHuSt78CdHZf77sstuUc+OemTijWVOXG8P UZwKB9K18LvNmGGJqepbL2IqoafnZJiY53p68A4kE+zIyl4LYzeZZylPQJM7mCFOGrt3 YmPujfwVW+iUfnPIzeCIeQ8upoEJLxhe9XVYDCY3mRWjwx72VjsqeXgnJi148b4JNE/R jKdrpOXgCBXWTyjuTO2AI1Rxwp+jaZBt3KJNlpAFQEA3wClxUKF7wthVZKBv73M2HnbV 1fSTVYgcGjPJtp5MhVByn9kAzv7CurkYDaRyrfQWh4dCAC7tIeVezyyBLA+lj4s9C553 1J2Q== X-Received: by 10.66.122.36 with SMTP id lp4mr41395445pab.82.1395246948044; Wed, 19 Mar 2014 09:35:48 -0700 (PDT) Received: from ox (c-24-6-44-228.hsd1.ca.comcast.net. [24.6.44.228]) by mx.google.com with ESMTPSA id ha2sm63778356pbb.8.2014.03.19.09.35.46 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 19 Mar 2014 09:35:47 -0700 (PDT) Date: Wed, 19 Mar 2014 09:35:44 -0700 From: Navdeep Parhar To: Julian Elischer Subject: Re: Include port number in "Listen queue overflow" messages Message-ID: <20140319163544.GB11935@ox> Mail-Followup-To: Julian Elischer , George Neville-Neil , hiren panchasara , "freebsd-net@freebsd.org" References: <532933A3.4030403@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <532933A3.4030403@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" , hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 16:35:48 -0000 On Tue, Mar 18, 2014 at 11:05:23PM -0700, Julian Elischer wrote: > On 3/18/14, 8:33 PM, George Neville-Neil wrote: > >On Mar 7, 2014, at 1:23 , hiren panchasara wrote: > > > >>I am thinking of committing following change that includes port number > >>in "Listen queue overflow" messages. > I think it's a good idea. There is even more information available > but this is probably enough. > >> > >I like it. > > > >Best, > >George > > I think the suggested change isn't correct as is assumes every socket's pcb is an inpcb. Navdeep > >>New message would look something like: > >>sonewconn: pcb 0xfffff8001b155760: Listen queue overflow on port > >>13120: 1 already in queue awaiting acceptance (454 occurrences) > >> > >>I've recently ran into a situation at $work where I could not catch > >>the culprit application via "netstat -A" and had to dive into kgdb to > >>find the port from pcb where this application was listening to. > >> > >>IMO, this change will make debugging easier. > >> > >>cheers, > >>Hiren > >> > >>Index: sys/kern/uipc_socket.c > >>=================================================================== > >>--- sys/kern/uipc_socket.c (revision 262861) > >>+++ sys/kern/uipc_socket.c (working copy) > >>@@ -136,6 +136,7 @@ > >>#include > >>#include > >>#include > >>+#include > >> > >>#include > >> > >>@@ -491,8 +492,11 @@ > >> static int overcount; > >> > >> struct socket *so; > >>+ struct inpcb *inp; > >> int over; > >> > >>+ inp = sotoinpcb(head); > >>+ > >> ACCEPT_LOCK(); > >> over = (head->so_qlen > 3 * head->so_qlimit / 2); > >> ACCEPT_UNLOCK(); > >>@@ -504,10 +508,12 @@ > >> overcount++; > >> > >> if (ratecheck(&lastover, &overinterval)) { > >>- log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow: " > >>- "%i already in queue awaiting acceptance " > >>+ log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow on " > >>+ "port %d: %i already in queue awaiting acceptance " > >> "(%d occurrences)\n", > >>- __func__, head->so_pcb, head->so_qlen, overcount); > >>+ __func__, head->so_pcb, > >>+ ntohs(inp->inp_inc.inc_lport), head->so_qlen, > >>+ overcount); > >> > >> overcount = 0; > >> } > >>_______________________________________________ > >>freebsd-net@freebsd.org mailing list > >>http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >_______________________________________________ > >freebsd-net@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-net > >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 19:17:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5EB22EB0 for ; Wed, 19 Mar 2014 19:17:35 +0000 (UTC) Received: from mail-qa0-x243.google.com (mail-qa0-x243.google.com [IPv6:2607:f8b0:400d:c00::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 215B2106 for ; Wed, 19 Mar 2014 19:17:35 +0000 (UTC) Received: by mail-qa0-f67.google.com with SMTP id w5so2997969qac.2 for ; Wed, 19 Mar 2014 12:17:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=82pdbe5YYiQaA2qdly2HBmFz/x4cU7GbNw3xJ4adrBI=; b=pMqBf/wfz8W7xIbj77PyJOmxzcgi1dRtlknQA7GTVvMt3V9oHvbvvMI9pvsv4MyqVV fOhY4pTNtTS5jYANscNwCaXFIxTJnfica1CeXPv0SDXgB8xP0dns2BlC7bXK5Z6ypFgf h3PLNBpNGld0yWG/qatooI//ZaIi+NhwUvCZFm0QQREguQ3BfXYC7ETh6ouLBGaIbHcl ZTH+JIGM7ipva2s+dsPxQiz1lWP5tsMO8VNDrjgnZvup523jXBGqbZYqnLI8SgRJwrC0 OfWFoNL3vAGatZsLMWF7rxLs0ZZnqAwhnu/DyWiGfFFuM1gFFdq1ot8FVK25WNizTjzQ WzGA== MIME-Version: 1.0 X-Received: by 10.224.65.194 with SMTP id k2mr24754qai.59.1395256654371; Wed, 19 Mar 2014 12:17:34 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Wed, 19 Mar 2014 12:17:34 -0700 (PDT) Date: Wed, 19 Mar 2014 16:17:34 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 19:17:35 -0000 Hello, I can report this problem as well on 10.0-RELEASE. I think it's the same as kern/183390? I have two physically identical machines, one running 9.2-STABLE, and one on 10.0-RELEASE. My 10.0 machine used to be running 9.0-STABLE for over a year without any problems. I'm not having the problems with 9.2-STABLE as far as I can tell, but it does seem to be a load-based issue more than anything. Since my 9.2 system is in production, I'm unable to load it to see if the problem exists there. I have a ping_logger.py running on it now to see if it's experiencing problems briefly or not. I am able to reproduce it fairly reliably within 15 min of a reboot by loading the server via NFS with iometer and some large NFS file copies at the same time. I seem to need to sustain ~2 Gbps for a few minutes. It will happen with just ix0 (no lagg) or with lagg enabled across ix0 and ix1. I've been load-testing new FreeBSD-10.0-RELEASE SAN's for production use here, so I'm quite willing to put time into this to help find out where it's coming from. It took me a day to track down my iometer issues as being network related, and another day to isolate and write scripts to reproduce. The symptom I notice is: - A running flood ping (ping -f 172.16.0.31) to the same hardware (running 9.2) will come back with "ping: sendto: File too large" when the problem occurs - Network connectivity is very spotty during these incidents - It can run with sporadic ping errors, or it can run a straight set of errors for minutes at a time - After a long run of ping errors, ESXi will show a disconnect from the hosted NFS stores on this machine. - I've yet to see it happen right after boot. Fastest is around 5 min, normally it's within 15 min. System Specs: - Dell PowerEdge M610x Blade - 2 Xeon 6600 @ 2.40GHz (24 Cores total) - 96 Gig RAM - 35.3 TB ZFS Mirrored pool, lz4 compression on my test pool (ZFS pool is the latest) - Intel 520-DA2 10 Gb dual-port Blade Mezz. Cards Currently this 10.0 testing machine is clean for all sysctl's other than hw.intr_storm_threshold=9900. I have the problem if that's set or not, so I leave it on. ( I used to set manual nmbclusters, etc. as per the Intel Readme.doc, but I notice that the defaults on the new 10.0 system are larger. I did try using all of the old sysctl's from an older 9.0-STABLE, and still had the problem, but it did seem to take longer to occur? I haven't run enough tests to confirm that time observation is true. ) What logs / info can I provide to help? I have written a small script called ping_logger.py that pings an IP, and checks to see if there is an error. On error it will execute and log: - netstat -m - sysctl hw.ix - sysctl dev.ix then go back to pinging. It will also log those values on the startup of the script, and every 5 min (so you can see the progression on the system). I can add any number of things to the reporting, so I'm looking for suggestions. This results in some large log files, but I can email a .gz directly to anyone who need them, or perhaps put it up on a website. I will also make the ping_logger.py script available if anyone else wants it. LASTLY: The one thing I can see that is different in my 10.0 System and my 9.2 is: 9.2's netstat -m: 37965/16290/54255 mbufs in use (current/cache/total) 4080/8360/12440/524288 mbuf clusters in use (current/cache/total/max) 4080/4751 mbuf+clusters out of packet secondary zone in use (current/cache) 0/452/452/262144 4k (page size) jumbo clusters in use (current/cache/total/max) 32773/4129/36902/96000 9k jumbo clusters in use (current/cache/total/max) 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) 312608K/59761K/372369K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines 10.0's netstat -m: 21512/24448/45960 mbufs in use (current/cache/total) 4080/16976/21056/6127254 mbuf clusters in use (current/cache/total/max) 4080/16384 mbuf+clusters out of packet secondary zone in use (current/cache) 0/23/23/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16384/158/16542/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 160994K/41578K/202572K bytes allocated to network (current/cache/total) 17488/13290/20464 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 7/16462/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile Way more mbuf clusters in use, but also I never get denied/delayed results in 9.2 - but I have them in 10.0 right away after a reboot. Thanks for any help.. From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 19:41:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96EDEAB4 for ; Wed, 19 Mar 2014 19:41:41 +0000 (UTC) Received: from mail-qa0-x243.google.com (mail-qa0-x243.google.com [IPv6:2607:f8b0:400d:c00::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5B5F1397 for ; Wed, 19 Mar 2014 19:41:41 +0000 (UTC) Received: by mail-qa0-f67.google.com with SMTP id w5so3004903qac.6 for ; Wed, 19 Mar 2014 12:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=tvvy/uIyvwQ1cE2Y91HlbNbO7CXKHsr12ooQpw7oTus=; b=YmfIHzS0NVfhGRe3MUYqCG6RXNV1rUuQTM4lcztvJ+1zgiQ2ekwlXMwa/7I7qKLWdU 88YyE7VhtJivoIQ2GVdZO9vUaHh3UWm6vToviPeyfRq+alMm4slMKnI2yE9Gjr42TVcz 8kufm40amD8c+EBDONqBEWlGAk1CjcjPppfC8r9jnLCNZacLBjbkdQuOJNE8fl5o2iH6 PVvxSlv9yMBKygz9Pb4E0lVcLlDRNh88VP4ZZcSZGWZnuiXdCVAL5tT5nQfZL90qx01H k8oUHEcuudqo8cae3aOdoi9Gr1HHS6CWaZA6a1DPVnzDK06HnZjYpYSSFwgSUoxwNRml Wfaw== MIME-Version: 1.0 X-Received: by 10.224.172.133 with SMTP id l5mr46229488qaz.25.1395258100596; Wed, 19 Mar 2014 12:41:40 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Wed, 19 Mar 2014 12:41:40 -0700 (PDT) Date: Wed, 19 Mar 2014 16:41:40 -0300 Message-ID: Subject: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 19:41:41 -0000 (Sorry for the formatting on that last message, that was weird) Today I wanted to test the assertion that this is a NFS issue, since we all seem to be running NFS. I shut down my NFS daemon in rc.conf, configured the FreeBSD10 iSCSI ctld, rebooted, and then ran all my tests exclusively from the iSCSI connection to my FreeBSD-10 box. The only other thing I had active on the FreeBSD-10 box was a flood-ping so I could see if/when the problem occurred. I still had the problem. It triggered around 40 min into my iometer benchmark. Continued sporadically for a while, then settled down and didn't give me problems. That error behaviour isn't abnormal - I really need to push the SAN to put it's network into a 'dead and not coming back' mode. Any thoughts? From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 19:48:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FC44C16 for ; Wed, 19 Mar 2014 19:48:57 +0000 (UTC) Received: from mail-qc0-x241.google.com (mail-qc0-x241.google.com [IPv6:2607:f8b0:400d:c01::241]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 339533EE for ; Wed, 19 Mar 2014 19:48:57 +0000 (UTC) Received: by mail-qc0-f193.google.com with SMTP id e16so3470554qcx.0 for ; Wed, 19 Mar 2014 12:48:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=JwZDKGEGgCzYdv++N21sNICJZ+E0+8Hc2lBl0QtT25w=; b=EOwUBDvS5Yr1yX2uHsJ8rIh6AbvjSlFlLOmsEpHYVT5vVPVlax5X9SfYrW09napFIT P9dQfE+P05xqpS8u1BuvgccfdAZ54R545vJb6vsNwULcSHX8hMdDW7j3HBtGinVYU+Qh LkCjcUrcVh2IKpH2MZ/iiQauF/gc4ChsQ/znDF06scD+psAWEgs1p2PUmbsyQRDpb6+u ZHDKCiY6OkZDgkN757IAnTICAbjoN8VI/GwNYLVK0/dSUDAtnvzs3NPAmlQnoM/YYUna IxijnyvrWJMBjVjkw5zsVffwWrduqtKPR300VN1syXi4QL2PPte6/jGTLWb37qfksJle rk0A== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr29099630qga.57.1395258536475; Wed, 19 Mar 2014 12:48:56 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Wed, 19 Mar 2014 12:48:56 -0700 (PDT) Date: Wed, 19 Mar 2014 16:48:56 -0300 Message-ID: Subject: Network loss From: Christopher Forgeron To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 19:48:57 -0000 Hello, I'm replying on the "9.2 ixgbe tx queue hang" thread, and wanted to tie in here the fact that I was able to replicate this problem with iSCSI only, no NFS. I assume the conversation will continue on the other thread. Johan Kooijman @ Sat Mar 1 08:55:14 UTC 2014 > >I think NFS is part of the issue here. Everybody that seems to have this >issue is running NFS. On top of that: the setup I'm having issues with, >didn't have any issues for months when we were running istgt instead of NFS. From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 21:01:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24AC17D5 for ; Wed, 19 Mar 2014 21:01:39 +0000 (UTC) Received: from golf.bouncedomains.com (golf.bouncedomains.com [103.18.252.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3010CD4 for ; Wed, 19 Mar 2014 21:01:38 +0000 (UTC) Received: from [103.18.252.70] (port=50675 helo=[192.168.77.202]) by golf.bouncedomains.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WPoTe-00050r-71; Tue, 18 Mar 2014 18:30:22 +1100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: vlanXXX vs ifXX.YY notation From: Mark van der Meulen In-Reply-To: <20140318074246.8519@smtp.new-ukraine.org> Date: Tue, 18 Mar 2014 18:30:21 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: <803C5DD5-47C1-44C9-9212-E17894B15168@fivenynes.com> References: <20140318074246.8519@smtp.new-ukraine.org> To: Zeus Panchenko X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - golf.bouncedomains.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - fivenynes.com X-Get-Message-Sender-Via: golf.bouncedomains.com: authenticated_id: mark@fivenynes.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 21:01:39 -0000 Hi, I have noticed net graph has issues with device that have a period in = their name. Either that is the case or something similar. I couldn=92t enable netflow on lagg0.X devices and I also couldn=92t use = some of mpd's features(such as LAC) either. I believe there were other = examples but those are the two that came to mind and they both use net = graph - I couldn=92t work out why things would work on em0 or lagg0 but = not on em0.X or lagg0.X and as i did some trawling through logs I saw = something which suggested an issue with the names so I changed all my = sub interfaces into =93vlan=94 interfaces and I was able to get full = functionality in MPD and with net graph netflow. Now that I have made the change, I would also say it is a much more = manageable way of doing things as well. Mark=20 On 18 Mar 2014, at 4:42 pm, Zeus Panchenko wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > hi, >=20 > is there any advantage of using vlanXXX vs ifXX.YY notation? >=20 > I mean >=20 >> ifconfig em0 > vlan777: flags=3D8843 metric 0 = mtu 1500 > ether 00:1b:b9:8b:ca:33 > ... > vlan: 777 parent interface: em0 >=20 > vs >=20 > em0.555: flags=3D8843 metric 0 = mtu 1500 > ether 00:1b:b9:8b:ca:33 > ... > vlan: 555 parent interface: em0 >=20 > - --=20 > Zeus V. Panchenko jid:zeus@im.ibs.dn.ua > IT Dpt., I.B.S. LLC GMT+2 (EET) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 >=20 > iEYEARECAAYFAlMn3NYACgkQr3jpPg/3oyqFEgCgj5te5rIwlKqZmlzENBBjLbdG > j5kAoPv8vFLgrsJRVPeIAzhCvpEC+Xxj > =3DBUzc > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" Mark van der Meulen: Fivenynes - Network Engineer Mob: 0459 312 495 Phone: 1300 661 902 Fax: 1300 855 328 Address: Unit 4, 23 Leeds St Rhodes NSW 2138 Web: www.fivenynes.com From owner-freebsd-net@FreeBSD.ORG Wed Mar 19 21:08:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4DB5B87 for ; Wed, 19 Mar 2014 21:08:16 +0000 (UTC) Received: from mail-pa0-f52.google.com (mail-pa0-f52.google.com [209.85.220.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B384D9C for ; Wed, 19 Mar 2014 21:08:16 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id rd3so9546784pab.11 for ; Wed, 19 Mar 2014 14:08:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=V+Hh/FuAJtHEGjtt6dZBw0XpgTAfkNagyDuU75YTf4E=; b=lmZyLWO9wFQg4/PKw8SBeoeDxyeqtK2ms2UAGsjJFgeCRiUoRJq7VXp0RvcdOS6DDB j/xWqBxP3QkoDej8rEa0vQGmPeXleEziBcr5/y1I3mxPNQPE9z9yURQo6SOzcWGg33Yd MTkpG76IWMfB3+Punl1ErIr3zgLsosePHNVu9xx8L8LUxj51OnSDCIMRaY8DtGPgcyrE boNCayzEwH9aRavbCUFKh/l25YGsJS/6IaCw5UPPvWXV+jl6STBtVbty2Hp1Zz8q0tXr lRGQ5CKKA3aZq8b43GowEcodSRTyQOnAN9pcP5IeZ9EkRJtQFYC+TabKuZGCFMxpow1K 2kYw== X-Gm-Message-State: ALoCoQn7ZItSVeknVwrI4DEDuJv2ZUqQ0rcBHEgYIwkrxNnGF7nuxfTceW/ZGBLFO/zPogiJyoXY MIME-Version: 1.0 X-Received: by 10.66.180.200 with SMTP id dq8mr42107820pac.104.1395263284538; Wed, 19 Mar 2014 14:08:04 -0700 (PDT) Received: by 10.68.111.37 with HTTP; Wed, 19 Mar 2014 14:08:04 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Mar 2014 22:08:04 +0100 Message-ID: Subject: Re: Network loss From: Johan Kooijman To: Christopher Forgeron Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 21:08:16 -0000 Hey Christopher, Can you paste the output of "uname -a" as a reply to this email? Thx! On Wed, Mar 19, 2014 at 8:48 PM, Christopher Forgeron wrote: > Hello, > > I'm replying on the "9.2 ixgbe tx queue hang" thread, and wanted to tie in > here the fact that I was able to replicate this problem with iSCSI only, no > NFS. > > I assume the conversation will continue on the other thread. > > Johan Kooijman @ Sat Mar 1 08:55:14 UTC 2014 > > > >I think NFS is part of the issue here. Everybody that seems to have this > >issue is running NFS. On top of that: the setup I'm having issues with, > >didn't have any issues for months when we were running istgt instead of > NFS. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 02:29:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86D0916E for ; Thu, 20 Mar 2014 02:29:44 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 1E951E for ; Thu, 20 Mar 2014 02:29:43 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAEpRKlODaFve/2dsb2JhbABag0FXgwe4EoZjUYE0dIIlAQEBAwEBAQEgBCcgCxsOChEZAgQlAQkmBggHBAEcBIdQCA2tRKJeF44DEAIBGxkbB4JvgUkEkFGFH4QJkH6DSSExgT0 X-IronPort-AV: E=Sophos;i="4.97,691,1389762000"; d="scan'208";a="107352963" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 19 Mar 2014 22:29:36 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C77EEB4043; Wed, 19 Mar 2014 22:29:36 -0400 (EDT) Date: Wed, 19 Mar 2014 22:29:36 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_25490919_2110583616.1395282576803" X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 02:29:44 -0000 ------=_Part_25490919_2110583616.1395282576803 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > Hello, > > > > I can report this problem as well on 10.0-RELEASE. > > > > I think it's the same as kern/183390? > > > > I have two physically identical machines, one running 9.2-STABLE, and > one > on 10.0-RELEASE. > > > > My 10.0 machine used to be running 9.0-STABLE for over a year without > any > problems. > > > > I'm not having the problems with 9.2-STABLE as far as I can tell, but > it > does seem to be a load-based issue more than anything. Since my 9.2 > system > is in production, I'm unable to load it to see if the problem exists > there. > I have a ping_logger.py running on it now to see if it's experiencing > problems briefly or not. > > > > I am able to reproduce it fairly reliably within 15 min of a reboot > by > loading the server via NFS with iometer and some large NFS file > copies at > the same time. I seem to need to sustain ~2 Gbps for a few minutes. > If you can easily do so, testing with the attached patch might shed some light on the problem. It just adds a couple of diagnostic checks before and after m_defrag() is called when bus_dmamap_load_mbuf_sg() returns EFBIG. If the "before" printf happens, it would suggest a problem with the loop in tcp_output() that creates TSO segments. If the "after" printf happens, it would suggest that m_defrag() somehow doesn't create a list of 32 or fewer mbufs for the TSO segment. I don't have any ix hardware, so this patch is completely untested. Just something maybe worth trying, rick > > > It will happen with just ix0 (no lagg) or with lagg enabled across > ix0 and > ix1. > > > > I've been load-testing new FreeBSD-10.0-RELEASE SAN's for production > use > here, so I'm quite willing to put time into this to help find out > where > it's coming from. It took me a day to track down my iometer issues > as > being network related, and another day to isolate and write scripts > to > reproduce. > > > > The symptom I notice is: > > - A running flood ping (ping -f 172.16.0.31) to the same > hardware > (running 9.2) will come back with "ping: sendto: File too large" when > the > problem occurs > > - Network connectivity is very spotty during these incidents > > - It can run with sporadic ping errors, or it can run a > straight > set of errors for minutes at a time > > - After a long run of ping errors, ESXi will show a > disconnect > from the hosted NFS stores on this machine. > > - I've yet to see it happen right after boot. Fastest is > around 5 > min, normally it's within 15 min. > > > > System Specs: > > > > - Dell PowerEdge M610x Blade > > - 2 Xeon 6600 @ 2.40GHz (24 Cores total) > > - 96 Gig RAM > > - 35.3 TB ZFS Mirrored pool, lz4 compression on my test pool > (ZFS > pool is the latest) > > - Intel 520-DA2 10 Gb dual-port Blade Mezz. Cards > > > > Currently this 10.0 testing machine is clean for all sysctl's other > than > hw.intr_storm_threshold=9900. I have the problem if that's set or > not, so I > leave it on. > > > > ( I used to set manual nmbclusters, etc. as per the Intel Readme.doc, > but I > notice that the defaults on the new 10.0 system are larger. I did try > using > all of the old sysctl's from an older 9.0-STABLE, and still had the > problem, but it did seem to take longer to occur? I haven't run > enough > tests to confirm that time observation is true. ) > > > > What logs / info can I provide to help? > > > > I have written a small script called ping_logger.py that pings an IP, > and > checks to see if there is an error. On error it will execute and log: > > - netstat -m > > - sysctl hw.ix > > - sysctl dev.ix > > > > then go back to pinging. It will also log those values on the startup > of > the script, and every 5 min (so you can see the progression on the > system). > I can add any number of things to the reporting, so I'm looking for > suggestions. > > > > This results in some large log files, but I can email a .gz directly > to > anyone who need them, or perhaps put it up on a website. > > > > I will also make the ping_logger.py script available if anyone else > wants > it. > > > > > > LASTLY: > > > > The one thing I can see that is different in my 10.0 System and my > 9.2 is: > > > > 9.2's netstat -m: > > > > 37965/16290/54255 mbufs in use (current/cache/total) > > 4080/8360/12440/524288 mbuf clusters in use (current/cache/total/max) > > 4080/4751 mbuf+clusters out of packet secondary zone in use > (current/cache) > > 0/452/452/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > > 32773/4129/36902/96000 9k jumbo clusters in use > (current/cache/total/max) > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > 312608K/59761K/372369K bytes allocated to network > (current/cache/total) > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > > > > 10.0's netstat -m: > > > > 21512/24448/45960 mbufs in use (current/cache/total) > > 4080/16976/21056/6127254 mbuf clusters in use > (current/cache/total/max) > > 4080/16384 mbuf+clusters out of packet secondary zone in use > (current/cache) > > 0/23/23/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > > 16384/158/16542/907741 9k jumbo clusters in use > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 160994K/41578K/202572K bytes allocated to network > (current/cache/total) > > 17488/13290/20464 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 7/16462/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > Way more mbuf clusters in use, but also I never get denied/delayed > results > in 9.2 - but I have them in 10.0 right away after a reboot. > > > > Thanks for any help.. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > ------=_Part_25490919_2110583616.1395282576803 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0xOSAxOToyMDowOC41OTcxMzcwMDAg LTA0MDAKQEAgLTE3NzEsNiArMTc3MSwxNyBAQCByZXRyeToKIAkJCS8qIFRyeSBpdCBhZ2Fpbj8g LSBvbmUgdHJ5ICovCiAJCQlpZiAocmVtYXAgPT0gVFJVRSkgewogCQkJCXJlbWFwID0gRkFMU0U7 Cit7IGludCBpaWk7IHN0cnVjdCBtYnVmICptbW07CittbW0gPSAqbV9oZWFkcDsKK2lpaSA9IDA7 Cit3aGlsZSAobW1tICE9IE5VTEwpIHsKKwlpaWkgKz0gbW1tLT5tX2xlbjsKKwltbW0gPSBtbW0t Pm1fbmV4dDsKK30KK21tbSA9ICptX2hlYWRwOworaWYgKGlpaSAhPSBtbW0tPm1fcGt0aGRyLmxl biB8fCBpaWkgPiA2NTUzNSkKK3ByaW50ZigiYmVmb3JlIHBrbGVuPSVkIGFjdGw9JWRcbiIsIG1t bS0+bV9wa3RoZHIubGVuLCBpaWkpOworfQogCQkJCW0gPSBtX2RlZnJhZygqbV9oZWFkcCwgTV9O T1dBSVQpOwogCQkJCWlmIChtID09IE5VTEwpIHsKIAkJCQkJYWRhcHRlci0+bWJ1Zl9kZWZyYWdf ZmFpbGVkKys7CkBAIC0xNzc4LDYgKzE3ODksMTggQEAgcmV0cnk6CiAJCQkJCSptX2hlYWRwID0g TlVMTDsKIAkJCQkJcmV0dXJuIChFTk9CVUZTKTsKIAkJCQl9Cit7IGludCBpaWksIGpqajsgc3Ry dWN0IG1idWYgKm1tbTsKK21tbSA9IG07CitqamogPSBpaWkgPSAwOword2hpbGUgKG1tbSAhPSBO VUxMKSB7CisJampqKys7CisJaWlpICs9IG1tbS0+bV9sZW47CisJbW1tID0gbW1tLT5tX25leHQ7 Cit9CittbW0gPSBtOworaWYgKGlpaSAhPSBtbW0tPm1fcGt0aGRyLmxlbiB8fCBpaWkgPiA2NTUz NSB8fCBqamogPiAzMikKK3ByaW50ZigiYWZ0ZXIgbWJjbnQ9JWQgcGtsZW49JWQgYWN0bD0lZFxu IiwgampqLCBtbW0tPm1fcGt0aGRyLmxlbiwgaWlpKTsKK30KIAkJCQkqbV9oZWFkcCA9IG07CiAJ CQkJZ290byByZXRyeTsKIAkJCX0gZWxzZQo= ------=_Part_25490919_2110583616.1395282576803-- From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 07:36:29 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F93EEB3 for ; Thu, 20 Mar 2014 07:36:29 +0000 (UTC) Received: from mail-ig0-x234.google.com (mail-ig0-x234.google.com [IPv6:2607:f8b0:4001:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6DE73DEB for ; Thu, 20 Mar 2014 07:36:29 +0000 (UTC) Received: by mail-ig0-f180.google.com with SMTP id hl1so1311722igb.1 for ; Thu, 20 Mar 2014 00:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=xrAFMmchWacSrsj/qg7KMqfI9lX9Sb3nnc2PcCo1kic=; b=APOgYd+yODNbWyMwDbYA8ZMJYfPtdoXHR7lnkCJruOowMCACXp3nCuDeRHEIJd0lh1 vg7ZYZDc8C32tDkuai5B5TRM0XHzHa4ipfmWpuEVsqDDdqisIUrRyQrXEnoEUUW8UopZ pdXqm4menEliRehIdmfKW7GoHLXlUui/ghR+2rLZ3de19OZrqOwNdqIe1u2zH+ajEhgc NzTxBjX+AfyxK6u7GkcalSpGnCUPOKeF9oMjTXG6r6N+cLJGc+ga+swqUdUQk3vK3CHW RoYtAgoLjdHhGbuftUGjBWJco66vEZr/bYp4ZbMY4QIhYxDtKI8UgiWrNGX4pqxN1wdY fEIA== MIME-Version: 1.0 X-Received: by 10.50.50.41 with SMTP id z9mr29634697ign.16.1395300988968; Thu, 20 Mar 2014 00:36:28 -0700 (PDT) Sender: andrei.moraru.n@gmail.com Received: by 10.64.238.112 with HTTP; Thu, 20 Mar 2014 00:36:28 -0700 (PDT) Date: Thu, 20 Mar 2014 09:36:28 +0200 X-Google-Sender-Auth: BMgtpenoq1vZ1afwRjcUUKU3Cno Message-ID: Subject: FreeBSD netmap build from ports error: no member named '_Ios_Openmode' in namespace 'std' From: FreeBSD ML Dev To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 07:36:29 -0000 Hello Friends, On fresh installed FreeBSD 10.0-Release, there is an error on building netmap from ports: root@test01:/usr/ports/net/netmap # *make install clean* ===> Building for netmap-0.1.3_1 gmake[1]: Entering directory `/usr/ports/net/netmap/work/netmap-0.1.3' gmake -C belgolib gmake[2]: Entering directory `/usr/ports/net/netmap/work/netmap-0.1.3/belgolib' c++ -O2 -pipe -fno-strict-aliasing -c -o files.o files.c c++: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated files.c:21:41: error: no member named '_Ios_Openmode' in namespace 'std' : ifstream(file_name.c_str(), (std::_Ios_Openmode)mode) ~~~~~^ files.c:52:35: error: no member named '_Ios_Openmode' in namespace 'std' open(file_name.c_str(), (std::_Ios_Openmode)mode); ~~~~~^ 2 errors generated. gmake[2]: *** [files.o] Error 1 gmake[2]: Leaving directory `/usr/ports/net/netmap/work/netmap-0.1.3/belgolib' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/usr/ports/net/netmap/work/netmap-0.1.3' ===> Compilation failed unexpectedly. Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 Stop. make: stopped in /usr/ports/net/netmap root@test01:/usr/ports/net/netmap # *make MAKE_JOBS_UNSAFE=yes install clean* ===> Building for netmap-0.1.3_1 gmake[1]: Entering directory `/usr/ports/net/netmap/work/netmap-0.1.3' gmake -C belgolib gmake[2]: Entering directory `/usr/ports/net/netmap/work/netmap-0.1.3/belgolib' c++ -O2 -pipe -fno-strict-aliasing -c -o files.o files.c c++: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated files.c:21:41: error: no member named '_Ios_Openmode' in namespace 'std' : ifstream(file_name.c_str(), (std::_Ios_Openmode)mode) ~~~~~^ files.c:52:35: error: no member named '_Ios_Openmode' in namespace 'std' open(file_name.c_str(), (std::_Ios_Openmode)mode); ~~~~~^ 2 errors generated. gmake[2]: *** [files.o] Error 1 gmake[2]: Leaving directory `/usr/ports/net/netmap/work/netmap-0.1.3/belgolib' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/usr/ports/net/netmap/work/netmap-0.1.3' *** Error code 1 Stop. make: stopped in /usr/ports/net/netmap root@test01:/usr/ports/net/netmap # uname FreeBSD root@test01:/usr/ports/net/netmap # uname -a FreeBSD test01 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 -- -------------- Andrei Moraru Happy Bacula Admin From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 10:03:48 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 347B9BCD for ; Thu, 20 Mar 2014 10:03:48 +0000 (UTC) Received: from m13-3.163.com (m13-3.163.com [220.181.13.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8C4E8D64 for ; Thu, 20 Mar 2014 10:03:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=Date:From:Subject:MIME-Version:Message-ID; bh=9XEyg 2wyA83ISDJfSegLBHI7n/0Qu4Ax1o+6mPXNAWU=; b=mHAhPNEmlz/jTTmkMeKTq 0a72pUHhiL/IuuSUMfbDHeSOXYNjsvXCn2oC7RcGWpdI8I0/6Bx1lQpzFXSJ/yA+ cD+hsQzoCDnreA2kZt8vxLjbf9M4XFA2XG6FsfNiydpc4Odu9cBSSVSgnSej2oUN KQk9xcI3SvDcip63k7K55k= Received: from mstian88$163.com ( [113.135.117.59] ) by ajax-webmail-wmsvr3 (Coremail) ; Thu, 20 Mar 2014 18:03:43 +0800 (CST) X-Originating-IP: [113.135.117.59] Date: Thu, 20 Mar 2014 18:03:43 +0800 (CST) From: mstian88 To: net@freebsd.org Subject: some problem about netmap X-Priority: 3 X-Mailer: Coremail Webmail Server Version SP_ntes V3.5 build 20131204(24406.5820.5783) Copyright (c) 2002-2014 www.mailtech.cn 163com X-CM-CTRLDATA: oKFSuWZvb3Rlcl9odG09ODg5Ojgx MIME-Version: 1.0 Message-ID: <59d70d80.2b3cc.144def24445.Coremail.mstian88@163.com> X-CM-TRANSID: A8GowAB3clUAvSpTP4AkAA--.4278W X-CM-SenderInfo: 5pvwxtrqyyqiywtou0bp/1tbiNBRUJVC-A-XTyAAAsT X-Coremail-Antispam: 1U5529EdanIXcx71UUUUU7vcSsGvfC2KfnxnUU== Content-Type: text/plain; charset=GBK Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 10:03:48 -0000 Li9wa3QtZ2VuIC1pIGV0aDEgLWYgcnggLVgKdGhlIHByaW50IGluZm8gc2hvd3Mgc2xvdC0+bGVu IGlzIDIwNDg/ICB3aHk/CnRoZSBwYWNrZXRzIHdhcyBzZW5kZWQgd2l0aCB0Y3ByZXBsYXkuCndo ZW4gTXVsdGlwbGUgcGFja2V0cyBsZW4gaXMgMTUxNCwgcGt0LWdlbiByZWNlaXZlIGF2aWFsID4x LCAgdGhlIHNsb3QtPmxlbiBpcyAyMDQ4CgoKbGlrZSB0aGlzOgpbMjIxM106c2xvdGxlblsxNTE0 XSxjdXJpZHhbMTg5XSxidWZpZHhbMjQ3NjddLGF2YWlsWzNdLGl3aGlsZVs0NzVdLGlmb3JbMTZd LGJ1Zl9vZnNbNTYwMzMyOF0scmluZ3NbMzJdICAKWzIyMTRdOnNsb3RsZW5bMjA0OF0sY3VyaWR4 WzE5MF0sYnVmaWR4WzI0NzY4XSxhdmFpbFsyXSxpd2hpbGVbNDc1XSxpZm9yWzE2XSxidWZfb2Zz WzU2MDMzMjhdLHJpbmdzWzMyXSAgClsyMjE1XTpzbG90bGVuWzkyNl0sY3VyaWR4WzE5MV0sYnVm aWR4WzI0NzY5XSxhdmFpbFsxXSxpd2hpbGVbNDc1XSxpZm9yWzE2XSxidWZfb2ZzWzU2MDMzMjhd LHJpbmdzWzMyXSAgClsyMjE2XTpzbG90bGVuWzE1MTRdLGN1cmlkeFsxOTJdLGJ1ZmlkeFsyNDc3 MF0sYXZhaWxbMV0saXdoaWxlWzQ3Nl0saWZvclsxNl0sYnVmX29mc1s1NjAzMzI4XSxyaW5nc1sz Ml0gIA== From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 10:41:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 648345D2 for ; Thu, 20 Mar 2014 10:41:00 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 230A8101 for ; Thu, 20 Mar 2014 10:40:59 +0000 (UTC) Received: from [2001:1620:2013:1:4535:ed23:3991:6e11] (port=62880) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WQaPB-000Ee0-7e; Thu, 20 Mar 2014 11:40:57 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Thu, 20 Mar 2014 11:40:18 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 10:41:00 -0000 On 19.03.2014, at 20:17, Christopher Forgeron = wrote: > Hello, >=20 >=20 >=20 > I can report this problem as well on 10.0-RELEASE. >=20 >=20 >=20 > I think it's the same as kern/183390? Possible. We still see this on nfsclients only, but I=92m not convinced = that nfs is the only trigger. > I have two physically identical machines, one running 9.2-STABLE, and = one > on 10.0-RELEASE. >=20 >=20 >=20 > My 10.0 machine used to be running 9.0-STABLE for over a year without = any > problems. >=20 >=20 >=20 > I'm not having the problems with 9.2-STABLE as far as I can tell, but = it > does seem to be a load-based issue more than anything. Since my 9.2 = system > is in production, I'm unable to load it to see if the problem exists = there. > I have a ping_logger.py running on it now to see if it's experiencing > problems briefly or not. I our case, when it happens, the problem persists for quite some time = (minutes or hours) if we don=92t interact (ifconfig or reboot). > I am able to reproduce it fairly reliably within 15 min of a reboot by > loading the server via NFS with iometer and some large NFS file copies = at > the same time. I seem to need to sustain ~2 Gbps for a few minutes. That=92s probably why we can=92t reproduce it reliably here. Although = having 10gig cards in our blade servers, the ones affected are connected = to a 1gig switch. > It will happen with just ix0 (no lagg) or with lagg enabled across ix0 = and > ix1. Same here. > I've been load-testing new FreeBSD-10.0-RELEASE SAN's for production = use > here, so I'm quite willing to put time into this to help find out = where > it's coming from. It took me a day to track down my iometer issues as > being network related, and another day to isolate and write scripts to > reproduce. >=20 >=20 >=20 > The symptom I notice is: >=20 > - A running flood ping (ping -f 172.16.0.31) to the same = hardware > (running 9.2) will come back with "ping: sendto: File too large" when = the > problem occurs >=20 > - Network connectivity is very spotty during these incidents >=20 > - It can run with sporadic ping errors, or it can run a = straight > set of errors for minutes at a time >=20 > - After a long run of ping errors, ESXi will show a = disconnect > from the hosted NFS stores on this machine. >=20 > - I've yet to see it happen right after boot. Fastest is = around 5 > min, normally it's within 15 min. Can you try this when the problem occurs? for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 -c = 2 -W 1 10.0.0.1 | grep sendto; done It will tie ping to certain cpus to test the different tx queues of your = ix interface. If the pings reliably fail only on some queues, then your = problem is more likely to be the same as ours. Also, if you have dtrace available: kldload dtraceall dtrace -n 'fbt:::return / arg1 =3D=3D EFBIG && execname =3D=3D "ping" / = { stack(); }' while you run pings over the interface affected. This will give you = hints about where the EFBIG error comes from. > [=85] Markus From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 11:36:16 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E7D81D8 for ; Thu, 20 Mar 2014 11:36:16 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E9DA97C5 for ; Thu, 20 Mar 2014 11:36:15 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so485019lab.41 for ; Thu, 20 Mar 2014 04:36:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=AGiE4Q+VhPUzSeagXuXk5L/yb+5r/9NsI5nheBLNnuU=; b=YkVYQJU0B70EQoN7IIyNYdfEBzAvNU2XRMCvLGPf4v2ohK3SneXs9DIPJp+IUpGMc7 Q0jIIj1uEmLtFez9vG++Bx1bWlE9tgo6saudl7Jr6S1mz7P5tfsF83u4vJ7RagudJXOP L4ZDH7tK8lf358WpWUEG5CNErDpGQgxGWt1c6InSdenqv4FqqgdkdQhvCOppuIq4KUK6 QXxaAcpGE2WR2lXGLf9HBdIjOvsCFOOXyHsPwypN7/Q6nJTwOsQoOldS8zL5V+uQwj4P xqlcpM0HMOqI1pT35e1VQMTwsU4Ik86fJAIVsvSSUIj5sm1GQD72xRMBdfr9i61Wtd3N pJIw== MIME-Version: 1.0 X-Received: by 10.112.147.36 with SMTP id th4mr13896099lbb.32.1395315374025; Thu, 20 Mar 2014 04:36:14 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Thu, 20 Mar 2014 04:36:13 -0700 (PDT) In-Reply-To: <59d70d80.2b3cc.144def24445.Coremail.mstian88@163.com> References: <59d70d80.2b3cc.144def24445.Coremail.mstian88@163.com> Date: Thu, 20 Mar 2014 12:36:13 +0100 X-Google-Sender-Auth: -lIAxOBeTJJhozELtraAP7puMow Message-ID: Subject: Re: some problem about netmap From: Luigi Rizzo To: mstian88 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 11:36:16 -0000 what os, device driver ? are you using emulated or native netmap mode ? Do you have tso and receive side coalescing enabled ? (you can disable them with ethtool). cheers luigi On Thu, Mar 20, 2014 at 11:03 AM, mstian88 wrote: > ./pkt-gen -i eth1 -f rx -X > the print info shows slot->len is 2048? why? > the packets was sended with tcpreplay. > when Multiple packets len is 1514, pkt-gen receive avial >1, the > slot->len is 2048 > > > like this: > > [2213]:slotlen[1514],curidx[189],bufidx[24767],avail[3],iwhile[475],ifor[16],buf_ofs[5603328],rings[32] > > [2214]:slotlen[2048],curidx[190],bufidx[24768],avail[2],iwhile[475],ifor[16],buf_ofs[5603328],rings[32] > > [2215]:slotlen[926],curidx[191],bufidx[24769],avail[1],iwhile[475],ifor[16],buf_ofs[5603328],rings[32] > > [2216]:slotlen[1514],curidx[192],bufidx[24770],avail[1],iwhile[476],ifor[16],buf_ofs[5603328],rings[32] > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > ---- From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 13:26:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B8BFA6A; Thu, 20 Mar 2014 13:26:42 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1C00F372; Thu, 20 Mar 2014 13:26:41 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s2KDQd8J079804; Thu, 20 Mar 2014 09:26:39 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s2KDQduJ079801; Thu, 20 Mar 2014 09:26:39 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21290.60558.750106.630804@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 09:26:38 -0400 From: Garrett Wollman To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Network stack returning EFBIG? X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 20 Mar 2014 09:26:39 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: jackv@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 13:26:42 -0000 I recently put a new server running 9.2 (with a local patches for NFS) into production, and it's immediately started to fail in an odd way. Since I pounded this server pretty heavily and never saw the error in testing, I'm more than a little bit taken aback. We have identical hardware in production with 9.1, and I have the same kernel running just peachy on a machine with Chelsio T4 NICs. The problem machine has ixgbe(4): ix0: port 0x9c00-0x9c1f mem 0xdef80000-0xdeffffff,0xdef7c000-0xdef7ffff irq 24 at device 0.0 on pci2 ix0: Using MSIX interrupts with 7 vectors ix0: Ethernet address: 04:7d:7b:a5:87:32 ix0: PCI Express Bus: Speed 5.0GT/s Width x4 ix1: port 0x9880-0x989f mem 0xdee80000-0xdeefffff,0xdee7c000-0xdee7ffff irq 34 at device 0.1 on pci2 ix1: Using MSIX interrupts with 7 vectors ix1: Ethernet address: 04:7d:7b:a5:87:33 ix1: PCI Express Bus: Speed 5.0GT/s Width x4 (pciconf tells me these are "82599EB 10-Gigabit SFI/SFP+ Network Connection". It's a bug that the driver doesn't tell me that.) These are glued together in a lagg(4) using LACP. Since we put this server into production, random network system calls have started failing with [EFBIG] or maybe sometimes [EIO]. I've observed this with a simple ping, but various daemons also log the errors: Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File too large [preauth] Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete SSL handshake. 5 The machine eventually becomes unreachable and has to be rebooted from the console. So, can anyone tell me how this is possible, and what changed between 9.1 and 9.2 to cause it? -GAWollman From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 13:32:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC630CFA; Thu, 20 Mar 2014 13:32:06 +0000 (UTC) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8D47F62A; Thu, 20 Mar 2014 13:32:05 +0000 (UTC) Received: from th-04.cs.huji.ac.il ([132.65.80.125]) by kabab.cs.huji.ac.il with esmtp id 1WQd4k-0008gR-Vb; Thu, 20 Mar 2014 15:32:03 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network stack returning EFBIG? From: Daniel Braniss In-Reply-To: <21290.60558.750106.630804@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 15:32:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <868FFD0A-106E-4C5E-A61C-10C3895C3281@cs.huji.ac.il> References: <21290.60558.750106.630804@hergotha.csail.mit.edu> To: Garrett Wollman X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org, jackv@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 13:32:06 -0000 turn off TSO the problems sound similar to the one I reported a while back. truing = off tso fixed it. danny On Mar 20, 2014, at 3:26 PM, Garrett Wollman = wrote: > I recently put a new server running 9.2 (with a local patches for NFS) > into production, and it's immediately started to fail in an odd way. > Since I pounded this server pretty heavily and never saw the error in > testing, I'm more than a little bit taken aback. We have identical > hardware in production with 9.1, and I have the same kernel running > just peachy on a machine with Chelsio T4 NICs. The problem machine = has > ixgbe(4): >=20 > ix0: = port 0x9c00-0x9c1f mem 0xdef80000-0xdeffffff,0xdef7c000-0xdef7ffff irq = 24 at device 0.0 on pci2 > ix0: Using MSIX interrupts with 7 vectors > ix0: Ethernet address: 04:7d:7b:a5:87:32 > ix0: PCI Express Bus: Speed 5.0GT/s Width x4 > ix1: = port 0x9880-0x989f mem 0xdee80000-0xdeefffff,0xdee7c000-0xdee7ffff irq = 34 at device 0.1 on pci2 > ix1: Using MSIX interrupts with 7 vectors > ix1: Ethernet address: 04:7d:7b:a5:87:33 > ix1: PCI Express Bus: Speed 5.0GT/s Width x4 >=20 > (pciconf tells me these are "82599EB 10-Gigabit SFI/SFP+ Network > Connection". It's a bug that the driver doesn't tell me that.) >=20 > These are glued together in a lagg(4) using LACP. >=20 > Since we put this server into production, random network system calls > have started failing with [EFBIG] or maybe sometimes [EIO]. I've > observed this with a simple ping, but various daemons also log the > errors: > Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File too = large [preauth] > Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete SSL = handshake. 5 >=20 > The machine eventually becomes unreachable and has to be rebooted from > the console. >=20 > So, can anyone tell me how this is possible, and what changed between > 9.1 and 9.2 to cause it? >=20 > -GAWollman > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 13:51:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 939501ED; Thu, 20 Mar 2014 13:51:44 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A8997DD; Thu, 20 Mar 2014 13:51:44 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s2KDpg27080117; Thu, 20 Mar 2014 09:51:42 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s2KDpghe080116; Thu, 20 Mar 2014 09:51:42 -0400 (EDT) (envelope-from wollman) Date: Thu, 20 Mar 2014 09:51:42 -0400 (EDT) Message-Id: <201403201351.s2KDpghe080116@hergotha.csail.mit.edu> From: wollman@bimajority.org To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Network stack returning EFBIG? In-Reply-To: <21290.60558.750106.630804@hergotha.csail.mit.edu> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 20 Mar 2014 09:51:42 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: jfv@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 13:51:44 -0000 In article <21290.60558.750106.630804@hergotha.csail.mit.edu>, I wrote: >Since we put this server into production, random network system calls >have started failing with [EFBIG] or maybe sometimes [EIO]. I've >observed this with a simple ping, but various daemons also log the >errors: >Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File too >large [preauth] >Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete SSL >handshake. 5 I found at least one call stack where this happens and it does get returned all the way to userspace: 17 15547 _bus_dmamap_load_buffer:return kernel`_bus_dmamap_load_mbuf_sg+0x5f kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 8863 _bus_dmamap_load_mbuf_sg:return kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 25315 bus_dmamap_load_mbuf_sg:return kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 15547 _bus_dmamap_load_buffer:return kernel`_bus_dmamap_load_mbuf_sg+0x5f kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 8863 _bus_dmamap_load_mbuf_sg:return kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 25315 bus_dmamap_load_mbuf_sg:return kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 4206 ixgbe_xmit:return kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 4208 ixgbe_mq_start_locked:return kernel`ixgbe_mq_start+0x12a if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 4212 ixgbe_mq_start:return if_lagg.ko`lagg_transmit+0xc4 kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 36017 lagg_transmit:return kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 23948 ether_output_frame:return kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 18849 ether_output:return kernel`ip_output+0xd74 kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 30895 ip_output:return kernel`tcp_output+0xfea kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 20356 tcp_output:return kernel`tcp_usr_send+0x325 kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 10923 tcp_usr_send:return kernel`sosend_generic+0x3f6 kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 19509 sosend_generic:return kernel`soo_write+0x5e kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 26794 soo_write:return kernel`dofilewrite+0x85 kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 9141 dofilewrite:return kernel`kern_writev+0x6c kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 25665 kern_writev:return kernel`sys_write+0x64 kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 17 24390 sys_write:return kernel`amd64_syscall+0x5ea kernel`0xffffffff808443c7 The MTU here is 9120, and the ixgbe driver has one local modification, to prevent it from using large contiguous mbufs in its receive queue: Index: ixgbe.c =================================================================== --- ixgbe.c (revision 261091) +++ ixgbe.c (working copy) @@ -1117,12 +1117,8 @@ */ if (adapter->max_frame_size <= 2048) adapter->rx_mbuf_sz = MCLBYTES; - else if (adapter->max_frame_size <= 4096) + else adapter->rx_mbuf_sz = MJUMPAGESIZE; - else if (adapter->max_frame_size <= 9216) - adapter->rx_mbuf_sz = MJUM9BYTES; - else - adapter->rx_mbuf_sz = MJUM16BYTES; /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { -GAWollman From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 15:06:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D5D95F4 for ; Thu, 20 Mar 2014 15:06:18 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EDBE1BF for ; Thu, 20 Mar 2014 15:06:17 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so1124776qcx.34 for ; Thu, 20 Mar 2014 08:06:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=owmisOEDXz+klm3Mi+M7Iy9b23/qvr+bvmZYbs3QxJM=; b=kk6YbBFXmmBwsL+kZLjWfKAcCXDG+Bg6xq/5qd1Z9eRFzmF0wa0ECuunR0vepKlHbP Fqo6PeIyjBCHa8zel7l3mTyMg6M9GNQ/QBfhCmgT+CTmUCodfbyykpNTWEgbSui+34XT DVklJgJQbvEA4+yA8TrMo5cDnJPtqv9XmZtm/VDfOsvKwiyJSYlEW2++ZsEDhTJiBmyN 7CHA2Xe13YBDIQhnQcFLhv0w3QtsPx2u/QSewScxj7OkYGWeNYAvOJw2uk1/UJOar7KW hy5MvqP9GCJRUdF5YxrYnN/Lw3gDKnHugW6cFUGDnBWzLxHu92gCzF81lwmKlRSyFBVo 1ppg== MIME-Version: 1.0 X-Received: by 10.224.65.194 with SMTP id k2mr5465173qai.59.1395327977013; Thu, 20 Mar 2014 08:06:17 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 08:06:16 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Mar 2014 12:06:16 -0300 Message-ID: Subject: Re: Network loss From: Christopher Forgeron To: Johan Kooijman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 15:06:18 -0000 Sure: $ uname -a FreeBSD SAN0.XXXXXX 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 I normally run a slightly tweaked NFS kernel, but I'm back on the default build for now until this problem is resolved. On Wed, Mar 19, 2014 at 6:08 PM, Johan Kooijman wrote: > Hey Christopher, > > Can you paste the output of "uname -a" as a reply to this email? Thx! > > > On Wed, Mar 19, 2014 at 8:48 PM, Christopher Forgeron < > csforgeron@gmail.com> wrote: > >> Hello, >> >> I'm replying on the "9.2 ixgbe tx queue hang" thread, and wanted to tie >> in >> here the fact that I was able to replicate this problem with iSCSI only, >> no >> NFS. >> >> I assume the conversation will continue on the other thread. >> >> Johan Kooijman @ Sat Mar 1 08:55:14 UTC 2014 >> > >> >I think NFS is part of the issue here. Everybody that seems to have this >> >issue is running NFS. On top of that: the setup I'm having issues with, >> >didn't have any issues for months when we were running istgt instead of >> NFS. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Met vriendelijke groeten / With kind regards, > Johan Kooijman > > T +31(0) 6 43 44 45 27 > F +31(0) 162 82 00 01 > E mail@johankooijman.com > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 15:22:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B8F573; Thu, 20 Mar 2014 15:22:39 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A8092CC; Thu, 20 Mar 2014 15:22:39 +0000 (UTC) Received: from [2001:1620:2013:1:4535:ed23:3991:6e11] (port=54749) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WQenl-000NhV-2M; Thu, 20 Mar 2014 16:22:37 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network stack returning EFBIG? From: Markus Gebert In-Reply-To: <201403201351.s2KDpghe080116@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 16:21:56 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201403201351.s2KDpghe080116@hergotha.csail.mit.edu> To: wollman@bimajority.org X-Mailer: Apple Mail (2.1874) Cc: jfv@freebsd.org, freebsd-net@freebsd.org, freebsd-stable@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 15:22:39 -0000 On 20.03.2014, at 14:51, wollman@bimajority.org wrote: > In article <21290.60558.750106.630804@hergotha.csail.mit.edu>, I = wrote: >=20 >> Since we put this server into production, random network system calls >> have started failing with [EFBIG] or maybe sometimes [EIO]. I've >> observed this with a simple ping, but various daemons also log the >> errors: >> Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File too >> large [preauth] >> Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete SSL >> handshake. 5 >=20 > I found at least one call stack where this happens and it does get > returned all the way to userspace: >=20 > 17 15547 _bus_dmamap_load_buffer:return=20 > kernel`_bus_dmamap_load_mbuf_sg+0x5f > kernel`bus_dmamap_load_mbuf_sg+0x38 > kernel`ixgbe_xmit+0xcf > kernel`ixgbe_mq_start_locked+0x94 > kernel`ixgbe_mq_start+0x12a > if_lagg.ko`lagg_transmit+0xc4 > kernel`ether_output_frame+0x33 > kernel`ether_output+0x4fe > kernel`ip_output+0xd74 > kernel`tcp_output+0xfea > kernel`tcp_usr_send+0x325 > kernel`sosend_generic+0x3f6 > kernel`soo_write+0x5e > kernel`dofilewrite+0x85 > kernel`kern_writev+0x6c > kernel`sys_write+0x64 > kernel`amd64_syscall+0x5ea > kernel`0xffffffff808443c7 This looks pretty similar to what we=92ve seen when we got EFBIG: 3 28502 _bus_dmamap_load_buffer:return=20 kernel`_bus_dmamap_load_mbuf_sg+0x5f kernel`bus_dmamap_load_mbuf_sg+0x38 kernel`ixgbe_xmit+0xcf kernel`ixgbe_mq_start_locked+0x94 kernel`ixgbe_mq_start+0x12a kernel`ether_output_frame+0x33 kernel`ether_output+0x4fe kernel`ip_output+0xd74 kernel`rip_output+0x229 kernel`sosend_generic+0x3f6 kernel`kern_sendit+0x1a3 kernel`sendit+0xdc kernel`sys_sendto+0x4d kernel`amd64_syscall+0x5ea kernel`0xffffffff80d35667 In our case it looks like some of the ixgbe tx queues get stuck, and = some don=92t. You can test, wether your server shows the same symptoms = with this command: # for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.5 = -c 2 -W 1 10.0.0.1 | grep sendto; done We also use 82599EB based ixgbe controllers on affected systems. Also see these two threads on freebsd-net: http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037967.html http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038061.html I have started the second one, and there are some more details of what = we were seeing in case you=92re interested. Then there is: http://www.freebsd.org/cgi/query-pr.cgi?pr=3D183390 and: https://bugs.freenas.org/issues/4560 Markus= From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 15:34:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 387CA857 for ; Thu, 20 Mar 2014 15:34:50 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB65B5E6 for ; Thu, 20 Mar 2014 15:34:49 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id j7so1016144qaq.8 for ; Thu, 20 Mar 2014 08:34:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gwenBx+mn6mFUGA24eNU9sDezZM6KnwRCv0Er/ThM40=; b=Tr240WrFYp30jEDfEAfIY+mA4UqA+85s58jcWaxl5oMdoqEipVonKdj/640I0+jU0y HGTYuaCikgYN8fKCfz6lQtpAQbI1TZow2wQytEk7gDX1VoHTJo2x8WCj9qs4AXCBzGMz cW7UN1luq9Na6TTi6364df6KkDJFek10Dltmyy10LWuLaK2MNS+EAljvaTI9gmo37dEX TwFpE5AQIwUEhlt0RoEeQ4UStVQfkMovvbffEHlRO/dmzQ/ZV1wzShuD2IKY3Mzt+Mxg TaVpXiWU9b+bMOusMk7M8XzciF0YI6YMXJfyiyZZ5g7mK3XC+pn0zmn09FlahufPloUy ww6g== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr3471628qab.93.1395329689168; Thu, 20 Mar 2014 08:34:49 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 08:34:49 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Mar 2014 12:34:49 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 15:34:50 -0000 On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert wrote: > > > Possible. We still see this on nfsclients only, but I'm not convinced that > nfs is the only trigger. > > Just to clarify, I'm experiencing this error with NFS, but also with iSCSI - I turned off my NFS server in rc.conf and rebooted, and I'm still able to create the error. This is not just a NFS issue on my machine. I our case, when it happens, the problem persists for quite some time > (minutes or hours) if we don't interact (ifconfig or reboot). > > The first few times that I ran into it, I had similar issues - Because I was keeping my system up and treating it like a temporary problem/issue. Worst case scenario resulted in reboots to reset the NIC. Then again, I find the ix's to be cranky if you ifconfig them too much. Now, I'm trying to find a root cause, so as soon as I start seeing any errors, I abort and reboot the machine to test the next theory. Additionally, I'm often able to create the problem with just 1 VM running iometer on the SAN storage. When the problem occurs, that connection is broken temporarily, taking network load off the SAN - That may improve my chances of keeping this running. > > > I am able to reproduce it fairly reliably within 15 min of a reboot by > > loading the server via NFS with iometer and some large NFS file copies at > > the same time. I seem to need to sustain ~2 Gbps for a few minutes. > > That's probably why we can't reproduce it reliably here. Although having > 10gig cards in our blade servers, the ones affected are connected to a 1gig > switch. > > It seems that it needs a lot of traffic. I have a 10 gig backbone between my SANs and my ESXi machines, so I can saturate quite quickly (just now I hit a record.. the error occurred within ~5 min of reboot and testing). In your case, I recommend firing up multiple VM's running iometer on different 1 gig connections and see if you can make it pop. I also often turn off ix1 to drive all traffic through ix0 - I've noticed it happens faster this way, but once again I'm not taking enough observations to make decent time predictions. > > > Can you try this when the problem occurs? > > for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 -c 2 > -W 1 10.0.0.1 | grep sendto; done > > It will tie ping to certain cpus to test the different tx queues of your > ix interface. If the pings reliably fail only on some queues, then your > problem is more likely to be the same as ours. > > Also, if you have dtrace available: > > kldload dtraceall > dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { stack(); > }' > > while you run pings over the interface affected. This will give you hints > about where the EFBIG error comes from. > > > [...] > > > Markus > > Will do. I'm not sure what shell the first script was written for, it's not working in csh, here's a re-write that does work in csh in case others are using the default shell: #!/bin/csh foreach CPU (`seq 0 23`) echo "CPU$CPU"; cpuset -l $CPU ping -i 0.2 -c 2 -W 1 10.0.0.1 | grep sendto; end Thanks for your input. I should have results to post to the list shortly. From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 15:50:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7AD2CD for ; Thu, 20 Mar 2014 15:50:22 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A817D7A9 for ; Thu, 20 Mar 2014 15:50:22 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id e9so1215731qcy.12 for ; Thu, 20 Mar 2014 08:50:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ez1TLfWULNwdqoiJ7aPFmS7F7YoTIVjDdfR2IJCYEYM=; b=gx9LigzKE+efApEJekUJcIe4Ri+A38W29Vo8rqHdwUwopuldoDy4E2ORlFHDijrB8M yAURL6xnGS456jEP98ChJu4zJYq3U7aT7qp1VLK2H8IBkERubViRMvcJeDepQn1H8Igz 1zwnhPdLaZ+Rc3GQgpCQfvKVH9pc0KK1YdolihBxi4beBmnm1XGEyQz6VMJODWUT/0eR av4pAhS9XO64dOzoJpmau8jxSt36MFWFGRWUIvvTQ0vt21r8hZCATL9ZR+poyFsJrAVN O7HhPvunDPLyfDae7ciSuhV9MHGsfFCHBf5+lSe3fZMhzqUXJbtahj58+HA2H93zVplJ IXRQ== MIME-Version: 1.0 X-Received: by 10.224.53.198 with SMTP id n6mr50793477qag.41.1395330621917; Thu, 20 Mar 2014 08:50:21 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 08:50:21 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Mar 2014 12:50:21 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 15:50:23 -0000 Markus, I just wanted to clarify what dtrace will output in a 'no-error' situation. I'm seeing the following during a normal ping (no errors) on ix0, or even on a non-problematic bge NIC: On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert wrote: > Also, if you have dtrace available: > > kldload dtraceall > dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { stack(); > }' > > while you run pings over the interface affected. This will give you hints > about where the EFBIG error comes from. > > > > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 15:52:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65433273 for ; Thu, 20 Mar 2014 15:52:16 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23ED383C for ; Thu, 20 Mar 2014 15:52:16 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id hw13so1053318qab.4 for ; Thu, 20 Mar 2014 08:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mIFZHodHH8bjq8Znz3b4xvygguHkwevE0kUa7LUzNz0=; b=UYUWfOAyZ4l4S+h93m07qSdoKqkdBuwqYC5is5yc5u4BW9VkpxUYFvrpGv1jywCRS+ uO//dFuVrJeJ9n7DAJxJftXrvINjtGnc0Fx3MfNdlcJ+vjCimze5+La4NJPXVt5i4cDH 0Um6nafXv9c8+/pTWSDhj6TCgBbUAObJ5s3nm9f1fDJ0VbamjsMMRK1wYVSSV+ebz0mJ 0xsU+UiB6GdkUrcq4VR5o3FmZpINzZEIxEtZdmY1MTeV94B/WrTrV7mCXgTn6fMz4usi R7qaaZET9QDQHHELO5Puuy73bskJAw5GYxVoZ1hJ9FFVU/jzsHZADJtqEtPhWgjpX/j/ OjBQ== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr34388312qga.57.1395330735374; Thu, 20 Mar 2014 08:52:15 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 08:52:15 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Mar 2014 12:52:15 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 15:52:16 -0000 (Struggling with this mail client for some reason, sorry, here's the paste) # dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { stack(); }' dtrace: description 'fbt:::return ' matched 24892 probes CPU ID FUNCTION:NAME 19 29656 maybe_yield:return kernel`uiomove_faultflag+0xab kernel`m_uiotombuf+0xfd kernel`sosend_generic+0x367 kernel`kern_sendit+0x224 kernel`sendit+0x116 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x357 kernel`0xffffffff80c7567b 19 29656 maybe_yield:return kernel`uiomove_faultflag+0xab kernel`ttydisc_write+0xde kernel`ttydev_write+0x143 kernel`devfs_write_f+0xef kernel`dofilewrite+0x8a kernel`kern_writev+0x65 kernel`sys_write+0x63 kernel`amd64_syscall+0x357 kernel`0xffffffff80c7567b 11 29656 maybe_yield:return kernel`uiomove_faultflag+0xab kernel`m_uiotombuf+0xfd kernel`sosend_generic+0x367 kernel`kern_sendit+0x224 kernel`sendit+0x116 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x357 kernel`0xffffffff80c7567b 11 29656 maybe_yield:return kernel`uiomove_faultflag+0xab kernel`ttydisc_write+0xde kernel`ttydev_write+0x143 kernel`devfs_write_f+0xef kernel`dofilewrite+0x8a kernel`kern_writev+0x65 kernel`sys_write+0x63 kernel`amd64_syscall+0x357 kernel`0xffffffff80c7567b On Thu, Mar 20, 2014 at 12:50 PM, Christopher Forgeron wrote: > Markus, > > I just wanted to clarify what dtrace will output in a 'no-error' > situation. I'm seeing the following during a normal ping (no errors) on > ix0, or even on a non-problematic bge NIC: > > > > > > On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert > wrote: > >> Also, if you have dtrace available: >> >> kldload dtraceall >> dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { >> stack(); }' >> >> while you run pings over the interface affected. This will give you hints >> about where the EFBIG error comes from. >> >> >> >> >> > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 17:02:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 653FA67F for ; Thu, 20 Mar 2014 17:02:13 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 21C92FA for ; Thu, 20 Mar 2014 17:02:13 +0000 (UTC) Received: from [2001:1620:2013:1:4535:ed23:3991:6e11] (port=57438) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WQgM6-0001Hj-He; Thu, 20 Mar 2014 18:02:10 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Thu, 20 Mar 2014 18:01:31 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 17:02:13 -0000 On 20.03.2014, at 16:50, Christopher Forgeron = wrote: > Markus, >=20 > I just wanted to clarify what dtrace will output in a 'no-error' > situation. I'm seeing the following during a normal ping (no errors) = on > ix0, or even on a non-problematic bge NIC: >=20 >=20 This is expected. This dtrace probe will fire if any kernel function = that is run in the context of a process named =93ping=94 returns 27, = which is what EFBIG stands for. Kernel functions can return 27 for many = reasons, not just an error, or they can return void (not sure how dtrace = handles this case). Anyway, this dtrace one-liner is only meant to be = used in the error case. Otherwise it=92s probably useless. And even when = the problem occurs, you need to pick the right stack trace, and dig = around kernel sources to verify that the functions indeed return EFBIG = and not any integer that happens to be 27 by accident. Markus > On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert > wrote: >=20 >> Also, if you have dtrace available: >>=20 >> kldload dtraceall >> dtrace -n 'fbt:::return / arg1 =3D=3D EFBIG && execname =3D=3D "ping" = / { stack(); >> }' >>=20 >> while you run pings over the interface affected. This will give you = hints >> about where the EFBIG error comes from. >>=20 >>=20 >>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 19:38:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99524CF8 for ; Thu, 20 Mar 2014 19:38:42 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 57BD4855 for ; Thu, 20 Mar 2014 19:38:42 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id i50so4008184qgf.0 for ; Thu, 20 Mar 2014 12:38:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1EcXEasuSEo+9cku2hohR/bzTBd7YjghVc9BsW1O9JE=; b=Dwyo6tkhC7WEeYf6XAtQ8merHRsjBOIM/GK/BCPary8JeUAZMepmhAXGdMBfKWOj/9 QXTl51mHRdaGNRXR5hZcprvH3LSgwllLvrzwmvTG0n46fV1Jb3DZJ5lZ7XearSFmrht0 MlW5WHt+0qzRpZBHpfcKcvlK9TMOH5q8kDxX4nMRJZx1TKt5j6CXM8hQd1jHIICERnTQ O7PUHU2Tc1SIK6A49DEWTpdAjK5bbhiAA0aOr5bypT2/Q7TJ8MsiUVrwkoKOzrOFRK5l 1/c13a3OxhTvlGsiUgZIo5gxQ5Wvr0hWb1W9AfuaFKPvxpUqOGjvX3TqyErPdUM3z/YN QQWQ== MIME-Version: 1.0 X-Received: by 10.229.58.68 with SMTP id f4mr27776985qch.18.1395344321534; Thu, 20 Mar 2014 12:38:41 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 12:38:41 -0700 (PDT) In-Reply-To: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 16:38:41 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 19:38:42 -0000 Output from the patch you gave me (I have screens of it.. let me know what you're hoping to see. Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:23 SAN0 kernel: before pklen=65538 actl=65538 Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65 On Wed, Mar 19, 2014 at 11:29 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > Hello, > > > > > > > > I can report this problem as well on 10.0-RELEASE. > > > > > > > > I think it's the same as kern/183390? > > > > > > > > I have two physically identical machines, one running 9.2-STABLE, and > > one > > on 10.0-RELEASE. > > > > > > > > My 10.0 machine used to be running 9.0-STABLE for over a year without > > any > > problems. > > > > > > > > I'm not having the problems with 9.2-STABLE as far as I can tell, but > > it > > does seem to be a load-based issue more than anything. Since my 9.2 > > system > > is in production, I'm unable to load it to see if the problem exists > > there. > > I have a ping_logger.py running on it now to see if it's experiencing > > problems briefly or not. > > > > > > > > I am able to reproduce it fairly reliably within 15 min of a reboot > > by > > loading the server via NFS with iometer and some large NFS file > > copies at > > the same time. I seem to need to sustain ~2 Gbps for a few minutes. > > > If you can easily do so, testing with the attached patch might shed > some light on the problem. It just adds a couple of diagnostic checks > before and after m_defrag() is called when bus_dmamap_load_mbuf_sg() > returns EFBIG. > > If the "before" printf happens, it would suggest a problem with the > loop in tcp_output() that creates TSO segments. > > If the "after" printf happens, it would suggest that m_defrag() somehow > doesn't create a list of 32 or fewer mbufs for the TSO segment. > > I don't have any ix hardware, so this patch is completely untested. > > Just something maybe worth trying, rick > > > > > > > It will happen with just ix0 (no lagg) or with lagg enabled across > > ix0 and > > ix1. > > > > > > > > I've been load-testing new FreeBSD-10.0-RELEASE SAN's for production > > use > > here, so I'm quite willing to put time into this to help find out > > where > > it's coming from. It took me a day to track down my iometer issues > > as > > being network related, and another day to isolate and write scripts > > to > > reproduce. > > > > > > > > The symptom I notice is: > > > > - A running flood ping (ping -f 172.16.0.31) to the same > > hardware > > (running 9.2) will come back with "ping: sendto: File too large" when > > the > > problem occurs > > > > - Network connectivity is very spotty during these incidents > > > > - It can run with sporadic ping errors, or it can run a > > straight > > set of errors for minutes at a time > > > > - After a long run of ping errors, ESXi will show a > > disconnect > > from the hosted NFS stores on this machine. > > > > - I've yet to see it happen right after boot. Fastest is > > around 5 > > min, normally it's within 15 min. > > > > > > > > System Specs: > > > > > > > > - Dell PowerEdge M610x Blade > > > > - 2 Xeon 6600 @ 2.40GHz (24 Cores total) > > > > - 96 Gig RAM > > > > - 35.3 TB ZFS Mirrored pool, lz4 compression on my test pool > > (ZFS > > pool is the latest) > > > > - Intel 520-DA2 10 Gb dual-port Blade Mezz. Cards > > > > > > > > Currently this 10.0 testing machine is clean for all sysctl's other > > than > > hw.intr_storm_threshold=9900. I have the problem if that's set or > > not, so I > > leave it on. > > > > > > > > ( I used to set manual nmbclusters, etc. as per the Intel Readme.doc, > > but I > > notice that the defaults on the new 10.0 system are larger. I did try > > using > > all of the old sysctl's from an older 9.0-STABLE, and still had the > > problem, but it did seem to take longer to occur? I haven't run > > enough > > tests to confirm that time observation is true. ) > > > > > > > > What logs / info can I provide to help? > > > > > > > > I have written a small script called ping_logger.py that pings an IP, > > and > > checks to see if there is an error. On error it will execute and log: > > > > - netstat -m > > > > - sysctl hw.ix > > > > - sysctl dev.ix > > > > > > > > then go back to pinging. It will also log those values on the startup > > of > > the script, and every 5 min (so you can see the progression on the > > system). > > I can add any number of things to the reporting, so I'm looking for > > suggestions. > > > > > > > > This results in some large log files, but I can email a .gz directly > > to > > anyone who need them, or perhaps put it up on a website. > > > > > > > > I will also make the ping_logger.py script available if anyone else > > wants > > it. > > > > > > > > > > > > LASTLY: > > > > > > > > The one thing I can see that is different in my 10.0 System and my > > 9.2 is: > > > > > > > > 9.2's netstat -m: > > > > > > > > 37965/16290/54255 mbufs in use (current/cache/total) > > > > 4080/8360/12440/524288 mbuf clusters in use (current/cache/total/max) > > > > 4080/4751 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > > > 0/452/452/262144 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > > > 32773/4129/36902/96000 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > > > 312608K/59761K/372369K bytes allocated to network > > (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > > > > > 10.0's netstat -m: > > > > > > > > 21512/24448/45960 mbufs in use (current/cache/total) > > > > 4080/16976/21056/6127254 mbuf clusters in use > > (current/cache/total/max) > > > > 4080/16384 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > > > 0/23/23/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > > > 16384/158/16542/907741 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > > 160994K/41578K/202572K bytes allocated to network > > (current/cache/total) > > > > 17488/13290/20464 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 7/16462/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > Way more mbuf clusters in use, but also I never get denied/delayed > > results > > in 9.2 - but I have them in 10.0 right away after a reboot. > > > > > > > > Thanks for any help.. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 19:56:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7CE03F4 for ; Thu, 20 Mar 2014 19:56:45 +0000 (UTC) Received: from mail-qa0-x235.google.com (mail-qa0-x235.google.com [IPv6:2607:f8b0:400d:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 84DBFA50 for ; Thu, 20 Mar 2014 19:56:45 +0000 (UTC) Received: by mail-qa0-f53.google.com with SMTP id w8so1412989qac.26 for ; Thu, 20 Mar 2014 12:56:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XenrVhIzH2jsKUafcLGxRI8MCslkxLkBTwu6okaXZzA=; b=K6LkhDXhRr/LXhf8nH8/BGE/cUry7yVfLMa1SY2mDnn336mg0N2bPUu5eglgmw8rmR OaHz3w7Cj3jia9VJrEdtaxNKEQ+YUugyIUdlO/CEVK4SZn5Z9Q+GDU7Jc+4MEoiu9Li5 wpvxF9Tj2SHu7M2Ei5Pwz1TUVS2H7ELdOsjFzcL8fTox/xF/irXur+lbY6hkyESko5WP z9OB/pUIuQqy0RCMdcFt4YH6ptoOijaIoFBWwKQHG/jeV5pO20t2mId537hETq62q+cF bD2v9gYK5okp5F9QomK89JlpVRtigLRfAJ/zL7MGVDmoQjzoXT0tyiGKQFPP7TxAzDoT +lUA== MIME-Version: 1.0 X-Received: by 10.140.107.10 with SMTP id g10mr50519430qgf.63.1395345404745; Thu, 20 Mar 2014 12:56:44 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 12:56:44 -0700 (PDT) In-Reply-To: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 16:56:44 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 19:56:45 -0000 BTW, When I have the problem, this is what I see from netstat -m 4080/2956/7036/6127254 mbuf clusters in use (current/cache/total/max) 4080/2636 mbuf+clusters out of packet secondary zone in use (current/cache) 0/50/50/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 32768/155/32923/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 312541K/9182K/321724K bytes allocated to network (current/cache/total) 34481/2600/4091 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 50/27433/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile It doesn't look that bad to me, other than all of the denied counts - But I can't see sysctl buffer numbers that look too low... For those who are interested, here is a dump of hw.ix and dev.ix.0 (I have ix.1 off) hw.ix.enable_aim: 1 hw.ix.max_interrupt_rate: 31250 hw.ix.rx_process_limit: 256 hw.ix.tx_process_limit: 256 hw.ix.enable_msix: 1 hw.ix.num_queues: 8 hw.ix.txd: 2048 hw.ix.rxd: 2048 2014-03-20 16:29:05.291 - INFO - dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.15 dev.ix.0.%driver: ix dev.ix.0.%location: slot=0 function=0 dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10f8 subvendor=0x8086 subdevice=0x000c class=0x020000 dev.ix.0.%parent: pci5 dev.ix.0.fc: 3 dev.ix.0.enable_aim: 1 dev.ix.0.advertise_speed: 0 dev.ix.0.dropped: 0 dev.ix.0.mbuf_defrag_failed: 0 dev.ix.0.watchdog_events: 0 dev.ix.0.link_irq: 5 dev.ix.0.queue0.interrupt_rate: 500000 dev.ix.0.queue0.irqs: 452969 dev.ix.0.queue0.txd_head: 319 dev.ix.0.queue0.txd_tail: 319 dev.ix.0.queue0.tso_tx: 61107 dev.ix.0.queue0.no_tx_dma_setup: 0 dev.ix.0.queue0.no_desc_avail: 0 dev.ix.0.queue0.tx_packets: 257636 dev.ix.0.queue0.rxd_head: 531 dev.ix.0.queue0.rxd_tail: 530 dev.ix.0.queue0.rx_packets: 522771 dev.ix.0.queue0.rx_bytes: 1318022421 dev.ix.0.queue0.rx_copies: 224837 dev.ix.0.queue0.lro_queued: 424583 dev.ix.0.queue0.lro_flushed: 181580 dev.ix.0.queue1.interrupt_rate: 125000 dev.ix.0.queue1.irqs: 22756 dev.ix.0.queue1.txd_head: 1169 dev.ix.0.queue1.txd_tail: 1169 dev.ix.0.queue1.tso_tx: 0 dev.ix.0.queue1.no_tx_dma_setup: 0 dev.ix.0.queue1.no_desc_avail: 0 dev.ix.0.queue1.tx_packets: 23202 dev.ix.0.queue1.rxd_head: 337 dev.ix.0.queue1.rxd_tail: 336 dev.ix.0.queue1.rx_packets: 337 dev.ix.0.queue1.rx_bytes: 32988 dev.ix.0.queue1.rx_copies: 225 dev.ix.0.queue1.lro_queued: 335 dev.ix.0.queue1.lro_flushed: 320 dev.ix.0.queue2.interrupt_rate: 500000 dev.ix.0.queue2.irqs: 20256 dev.ix.0.queue2.txd_head: 1201 dev.ix.0.queue2.txd_tail: 1201 dev.ix.0.queue2.tso_tx: 0 dev.ix.0.queue2.no_tx_dma_setup: 0 dev.ix.0.queue2.no_desc_avail: 0 dev.ix.0.queue2.tx_packets: 20962 dev.ix.0.queue2.rxd_head: 1021 dev.ix.0.queue2.rxd_tail: 1020 dev.ix.0.queue2.rx_packets: 1021 dev.ix.0.queue2.rx_bytes: 99126 dev.ix.0.queue2.rx_copies: 891 dev.ix.0.queue2.lro_queued: 396 dev.ix.0.queue2.lro_flushed: 391 dev.ix.0.queue3.interrupt_rate: 71428 dev.ix.0.queue3.irqs: 25072 dev.ix.0.queue3.txd_head: 1465 dev.ix.0.queue3.txd_tail: 1465 dev.ix.0.queue3.tso_tx: 0 dev.ix.0.queue3.no_tx_dma_setup: 0 dev.ix.0.queue3.no_desc_avail: 0 dev.ix.0.queue3.tx_packets: 25726 dev.ix.0.queue3.rxd_head: 310 dev.ix.0.queue3.rxd_tail: 309 dev.ix.0.queue3.rx_packets: 310 dev.ix.0.queue3.rx_bytes: 36886 dev.ix.0.queue3.rx_copies: 150 dev.ix.0.queue3.lro_queued: 309 dev.ix.0.queue3.lro_flushed: 286 dev.ix.0.queue4.interrupt_rate: 500000 dev.ix.0.queue4.irqs: 21251 dev.ix.0.queue4.txd_head: 308 dev.ix.0.queue4.txd_tail: 308 dev.ix.0.queue4.tso_tx: 0 dev.ix.0.queue4.no_tx_dma_setup: 0 dev.ix.0.queue4.no_desc_avail: 0 dev.ix.0.queue4.tx_packets: 22090 dev.ix.0.queue4.rxd_head: 589 dev.ix.0.queue4.rxd_tail: 588 dev.ix.0.queue4.rx_packets: 589 dev.ix.0.queue4.rx_bytes: 57938 dev.ix.0.queue4.rx_copies: 558 dev.ix.0.queue4.lro_queued: 585 dev.ix.0.queue4.lro_flushed: 585 dev.ix.0.queue5.interrupt_rate: 41666 dev.ix.0.queue5.irqs: 20123 dev.ix.0.queue5.txd_head: 314 dev.ix.0.queue5.txd_tail: 314 dev.ix.0.queue5.tso_tx: 0 dev.ix.0.queue5.no_tx_dma_setup: 0 dev.ix.0.queue5.no_desc_avail: 0 dev.ix.0.queue5.tx_packets: 20618 dev.ix.0.queue5.rxd_head: 112 dev.ix.0.queue5.rxd_tail: 111 dev.ix.0.queue5.rx_packets: 112 dev.ix.0.queue5.rx_bytes: 10224 dev.ix.0.queue5.rx_copies: 84 dev.ix.0.queue5.lro_queued: 109 dev.ix.0.queue5.lro_flushed: 109 dev.ix.0.queue6.interrupt_rate: 71428 dev.ix.0.queue6.irqs: 18418 dev.ix.0.queue6.txd_head: 732 dev.ix.0.queue6.txd_tail: 732 dev.ix.0.queue6.tso_tx: 45 dev.ix.0.queue6.no_tx_dma_setup: 0 dev.ix.0.queue6.no_desc_avail: 0 dev.ix.0.queue6.tx_packets: 19137 dev.ix.0.queue6.rxd_head: 824 dev.ix.0.queue6.rxd_tail: 823 dev.ix.0.queue6.rx_packets: 824 dev.ix.0.queue6.rx_bytes: 92838 dev.ix.0.queue6.rx_copies: 583 dev.ix.0.queue6.lro_queued: 818 dev.ix.0.queue6.lro_flushed: 716 dev.ix.0.queue7.interrupt_rate: 62500 dev.ix.0.queue7.irqs: 17681 dev.ix.0.queue7.txd_head: 721 dev.ix.0.queue7.txd_tail: 721 dev.ix.0.queue7.tso_tx: 0 dev.ix.0.queue7.no_tx_dma_setup: 0 dev.ix.0.queue7.no_desc_avail: 0 dev.ix.0.queue7.tx_packets: 18067 dev.ix.0.queue7.rxd_head: 1407 dev.ix.0.queue7.rxd_tail: 1406 dev.ix.0.queue7.rx_packets: 1407 dev.ix.0.queue7.rx_bytes: 252631 dev.ix.0.queue7.rx_copies: 884 dev.ix.0.queue7.lro_queued: 1400 dev.ix.0.queue7.lro_flushed: 1390 dev.ix.0.mac_stats.crc_errs: 0 dev.ix.0.mac_stats.ill_errs: 0 dev.ix.0.mac_stats.byte_errs: 0 dev.ix.0.mac_stats.short_discards: 0 dev.ix.0.mac_stats.local_faults: 2 dev.ix.0.mac_stats.remote_faults: 3 dev.ix.0.mac_stats.rec_len_errs: 0 dev.ix.0.mac_stats.xon_txd: 0 dev.ix.0.mac_stats.xon_recvd: 0 dev.ix.0.mac_stats.xoff_txd: 0 dev.ix.0.mac_stats.xoff_recvd: 0 dev.ix.0.mac_stats.total_octets_rcvd: 1320732697 dev.ix.0.mac_stats.good_octets_rcvd: 1320713370 dev.ix.0.mac_stats.total_pkts_rcvd: 527648 dev.ix.0.mac_stats.good_pkts_rcvd: 527365 dev.ix.0.mac_stats.mcast_pkts_rcvd: 25 dev.ix.0.mac_stats.bcast_pkts_rcvd: 75 dev.ix.0.mac_stats.rx_frames_64: 128032 dev.ix.0.mac_stats.rx_frames_65_127: 100057 dev.ix.0.mac_stats.rx_frames_128_255: 115733 dev.ix.0.mac_stats.rx_frames_256_511: 1210 dev.ix.0.mac_stats.rx_frames_512_1023: 3075 dev.ix.0.mac_stats.rx_frames_1024_1522: 179258 dev.ix.0.mac_stats.recv_undersized: 0 dev.ix.0.mac_stats.recv_fragmented: 0 dev.ix.0.mac_stats.recv_oversized: 0 dev.ix.0.mac_stats.recv_jabberd: 0 dev.ix.0.mac_stats.management_pkts_rcvd: 0 dev.ix.0.mac_stats.management_pkts_drpd: 0 dev.ix.0.mac_stats.checksum_errs: 0 dev.ix.0.mac_stats.good_octets_txd: 2815129453 dev.ix.0.mac_stats.total_pkts_txd: 640355 dev.ix.0.mac_stats.good_pkts_txd: 640355 dev.ix.0.mac_stats.bcast_pkts_txd: 2 dev.ix.0.mac_stats.mcast_pkts_txd: 25 dev.ix.0.mac_stats.management_pkts_txd: 0 dev.ix.0.mac_stats.tx_frames_64: 39831 dev.ix.0.mac_stats.tx_frames_65_127: 166390 dev.ix.0.mac_stats.tx_frames_128_255: 72116 dev.ix.0.mac_stats.tx_frames_256_511: 2072 dev.ix.0.mac_stats.tx_frames_512_1023: 1339 dev.ix.0.mac_stats.tx_frames_1024_1522: 358607 and lastly, the default sysctl kern.ipc: kern.ipc.maxsockbuf: 2097152 kern.ipc.sockbuf_waste_factor: 8 kern.ipc.max_linkhdr: 16 kern.ipc.max_protohdr: 60 kern.ipc.max_hdr: 76 kern.ipc.max_datalen: 92 kern.ipc.maxmbufmem: 50194468864 kern.ipc.nmbclusters: 6127254 kern.ipc.nmbjumbop: 3063627 kern.ipc.nmbjumbo9: 2723223 kern.ipc.nmbjumbo16: 2042416 kern.ipc.nmbufs: 39214440 kern.ipc.maxpipekva: 1610002432 kern.ipc.pipekva: 147456 kern.ipc.pipefragretry: 0 kern.ipc.pipeallocfail: 0 kern.ipc.piperesizefail: 0 kern.ipc.piperesizeallowed: 1 kern.ipc.msgmax: 16384 kern.ipc.msgmni: 40 kern.ipc.msgmnb: 2048 kern.ipc.msgtql: 40 kern.ipc.msgssz: 8 kern.ipc.msgseg: 2048 kern.ipc.semmni: 50 kern.ipc.semmns: 340 kern.ipc.semmnu: 150 kern.ipc.semmsl: 340 kern.ipc.semopm: 100 kern.ipc.semume: 50 kern.ipc.semusz: 632 kern.ipc.semvmx: 32767 kern.ipc.semaem: 16384 kern.ipc.shmmax: 536870912 kern.ipc.shmmin: 1 kern.ipc.shmmni: 192 kern.ipc.shmseg: 128 kern.ipc.shmall: 131072 kern.ipc.shm_use_phys: 0 kern.ipc.shm_allow_removed: 0 kern.ipc.soacceptqueue: 128 kern.ipc.numopensockets: 79 kern.ipc.maxsockets: 3144540 kern.ipc.sendfile.readahead: 1 From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 20:05:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D42D87EA for ; Thu, 20 Mar 2014 20:05:23 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 93E75B25 for ; Thu, 20 Mar 2014 20:05:23 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so1664998qcr.22 for ; Thu, 20 Mar 2014 13:05:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9tYhD43R7RKD6VljbOAp9fsTegFok4nYMyFAiw8CFfM=; b=g86OFqqb1b5f79vFYVVMxSIEaowcvmV4zFhPlZ8HEcsWTJ3llR6CLrGDJjqInpim5D gipuLdM8K9tTcUmYeAeiqdMbYmQaRe+p0p3mqBccjQgBD8BlUzX4TBrYDi0Z4mvpICGf +qwLGLolp28vNaeQSkG/DDkB2T9IA8vdPTERvUjpnuGFLObYf4/3cCOpuGDIdXokZLeD oUfbmOMim5IrHgZsg6r2tEZEfQ4oO32N2zIBXnkPqFPjCn2ISuWETOWBCHPgttMsHWoI sPh1Sb/cRgutRLH/hWWCWmbhGB8NY/RYlUcgrh+zCY5zoKY1RmObeuj8fLDxSj89J2Va fCOQ== MIME-Version: 1.0 X-Received: by 10.224.123.7 with SMTP id n7mr571055qar.18.1395345922844; Thu, 20 Mar 2014 13:05:22 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 13:05:22 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Mar 2014 17:05:22 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 20:05:24 -0000 Re: cpuset ping I can report that I do not get any fails with this ping - I have screens of failed flood pings on the ix0 nic, but these always pass (i have that cpuset ping looping constantly). I can't report about the dtrace yet, as I'm running Rick's ixgbe patch, and there seems to be a .ko conflict someplace that keeps dtrace from running. I'm going to try locking my flood pings down to specific CPU's to see if there is any pattern there. After that I'll restore GENERIC and try the dtrace line. On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert wrote: > > > Can you try this when the problem occurs? > > for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 -c 2 > -W 1 10.0.0.1 | grep sendto; done > > It will tie ping to certain cpus to test the different tx queues of your > ix interface. If the pings reliably fail only on some queues, then your > problem is more likely to be the same as ours. > > Also, if you have dtrace available: > > kldload dtraceall > dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { stack(); > }' > > while you run pings over the interface affected. This will give you hints > about where the EFBIG error comes from. > > > [...] > > > Markus > > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 21:13:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4619413A for ; Thu, 20 Mar 2014 21:13:10 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F41D7387 for ; Thu, 20 Mar 2014 21:13:09 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s2KLD7LA085086; Thu, 20 Mar 2014 17:13:07 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s2KLD7GB085085; Thu, 20 Mar 2014 17:13:07 -0400 (EDT) (envelope-from wollman) Date: Thu, 20 Mar 2014 17:13:07 -0400 (EDT) From: Garrett Wollman Message-Id: <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> To: csforgeron@gmail.com Subject: Re: 9.2 ixgbe tx queue hang X-Newsgroups: mit.lcs.mail.freebsd-net In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 20 Mar 2014 17:13:07 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:13:10 -0000 In article , csforgeron@gmail.com writes: >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) This is going to screw you. You need to make sure that no NIC driver ever allocates 9k jumbo pages -- unless you are using one of those mythical drivers that can't do scatter/gather DMA on receive, which you don't appear to be. These failures occur when the driver is trying to replenish its receive queue, but is unable to allocate three *physically* contiguous pages of RAM to construct the 9k jumbo cluster (of which the remaining 3k is simply wasted). This happens on any moderately active server, once physical memory gets checkerboarded with active single pages, particularly with ZFS where those pages are wired in kernel memory and so can't be evicted. -GAWollman From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 21:22:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33C1465B for ; Thu, 20 Mar 2014 21:22:48 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D25926A3 for ; Thu, 20 Mar 2014 21:22:47 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id q108so4385878qgd.9 for ; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Gyx+UwU3f8EhNHE+wAk73s4QM6ZroZMnOco5Uis+G2k=; b=S2nKIgnwrWshUTwiigtalPwlUjOfhiHZL13lYyAOktTf1STFGrn4d83OFsoEiR9Ioi uW/K3UXH1v5f9eTInnRukXINN5TkbmzI4Tr5fbiw8AcTrZLbiHR0pKgMwwsFX1FKHv0L 1BEFe2qfakhp3Uv4MrkFOBmUUMZoJnIAi47WM7fIkRLwRTat82bxB3dP0tixivazzh2f l4PFG9Jvgsqq4jXHkJxB6Gh70Qi6auL5RL5QscNxLaMBwVk8iJ1IKm57YD+c9EFrIgid 8VWzzYCbQ4P3eDauhhrlxUA5e6X5qO/1YvZ6lLMvCzuPconsxXbPG2VINRTwUdooqmEm u8IA== MIME-Version: 1.0 X-Received: by 10.224.123.7 with SMTP id n7mr982734qar.18.1395350566609; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) In-Reply-To: <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 18:22:46 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:22:48 -0000 Any recommendations on what to do? I'm experimenting with disabling TSO right now, but it's too early to tell if it fixes my problem. On my 9.2 box, we don't see this number climbing. With TSO off on 10.0, I also see the number is not climbing. I'd appreciate any links you may have so I can read up on this. Thanks for the comment. On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > , > csforgeron@gmail.com writes: > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > This is going to screw you. You need to make sure that no NIC driver > ever allocates 9k jumbo pages -- unless you are using one of those > mythical drivers that can't do scatter/gather DMA on receive, which > you don't appear to be. > > These failures occur when the driver is trying to replenish its > receive queue, but is unable to allocate three *physically* contiguous > pages of RAM to construct the 9k jumbo cluster (of which the remaining > 3k is simply wasted). This happens on any moderately active server, > once physical memory gets checkerboarded with active single pages, > particularly with ZFS where those pages are wired in kernel memory and > so can't be evicted. > > -GAWollman > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 21:34:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BC399B5 for ; Thu, 20 Mar 2014 21:34:05 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 09F9E7F4 for ; Thu, 20 Mar 2014 21:34:04 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id j107so4426148qga.7 for ; Thu, 20 Mar 2014 14:34:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yJ8Z9YpMhDk+5D4Pr9AnlL3TOvMIxeNPEBsAmvQclwA=; b=bifBoCkW/FFfqFJFY6dN6lA3ALS8oqnx/3UUdNd3RW70u8Ybs/bMa8lGRqqnqv4elm hJ7o005QHya+oeKuiL4xqcoN5AXzuhsXCmQirNqYSFSgOjcdCg/oJbTfFJLwS03vy9ul WyljwD5G701bdxU6y+vmunqPrQOgRD6hss1L2ud3zjEFlkgAXhypthT+FOWr854RFJys +3qe21hIQChqpOc3S2m1S7Bsxc2rlKpggx0QlCUVlCt2HZokDLHZB56v3+6U5VwUagyP 7XjMfB+7fWkOIhoss6pBo6rJWnz+1loSPiJARMOBTyl1H4t8T3RWgvAQVMYLDpNBUqiK vYiQ== MIME-Version: 1.0 X-Received: by 10.224.53.198 with SMTP id n6mr52909695qag.41.1395351244261; Thu, 20 Mar 2014 14:34:04 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 14:34:04 -0700 (PDT) In-Reply-To: <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 18:34:04 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:34:05 -0000 I have found this: http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html I think what you're saying is that; - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster - modern NIC drivers can gather 9000 bytes of data from various memory locations - The fact that I'm seeing 9k jumbo clusters is showing me that my driver is trying to allocate 9k of contiguous space, and it's failing. Please correct me if I'm off here, I'd love to understand more. On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > , > csforgeron@gmail.com writes: > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > This is going to screw you. You need to make sure that no NIC driver > ever allocates 9k jumbo pages -- unless you are using one of those > mythical drivers that can't do scatter/gather DMA on receive, which > you don't appear to be. > > These failures occur when the driver is trying to replenish its > receive queue, but is unable to allocate three *physically* contiguous > pages of RAM to construct the 9k jumbo cluster (of which the remaining > 3k is simply wasted). This happens on any moderately active server, > once physical memory gets checkerboarded with active single pages, > particularly with ZFS where those pages are wired in kernel memory and > so can't be evicted. > > -GAWollman > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 22:00:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71BDD57D for ; Thu, 20 Mar 2014 22:00:08 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE7CA83 for ; Thu, 20 Mar 2014 22:00:08 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id il7so1731112vcb.18 for ; Thu, 20 Mar 2014 15:00:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=g33YzIHmHW1EB8nAbiYTV3jucIf6xJOPbs4vVpzPHzs=; b=ymIO2pMC1WHcAdEexGp/je3M79s68G8GIahYfD4nFi7U6PSJiGfYfIv1sCYj91si7/ b+S3PVJLm1a3ufzUVvW5M2+FC1uCnNU5HQVMrhXHbfsivGQJiMBjQPVOMqbhARraAIic CH9mQKTu0mm82dtlyPM2q/F89n4whvP4WqU2sbM3HhLnKLOqv3vLbvf0XqHWoZ0eSgqG UOQJbi1dROgVY8nkriaT89siov2rWi15+PkrCevjKhrBbf+vcCXrxsMjHxAhm31h0E79 AjThDVKS8YGzJsKAAvraCgNL8PqRm2GzJ44hBwtPFucG48AUDHCwO6APb1D8O0ZuqMQm 6Fig== MIME-Version: 1.0 X-Received: by 10.221.20.199 with SMTP id qp7mr17215035vcb.24.1395352807292; Thu, 20 Mar 2014 15:00:07 -0700 (PDT) Received: by 10.221.11.135 with HTTP; Thu, 20 Mar 2014 15:00:07 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 15:00:07 -0700 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Jack Vogel To: Christopher Forgeron Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:00:08 -0000 What he's saying is that the driver should not be using 9K mbuf clusters, I thought this had been changed but I see the code in HEAD is still using the larger clusters when you up the mtu. I will put it on my list to change with the next update to HEAD. What version of ixgbe are you using? Jack On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron wrote: > I have found this: > > http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html > > I think what you're saying is that; > - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster > - modern NIC drivers can gather 9000 bytes of data from various memory > locations > - The fact that I'm seeing 9k jumbo clusters is showing me that my driver > is trying to allocate 9k of contiguous space, and it's failing. > > Please correct me if I'm off here, I'd love to understand more. > > > On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < > wollman@hergotha.csail.mit.edu> wrote: > > > In article > > , > > csforgeron@gmail.com writes: > > > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > > > This is going to screw you. You need to make sure that no NIC driver > > ever allocates 9k jumbo pages -- unless you are using one of those > > mythical drivers that can't do scatter/gather DMA on receive, which > > you don't appear to be. > > > > These failures occur when the driver is trying to replenish its > > receive queue, but is unable to allocate three *physically* contiguous > > pages of RAM to construct the 9k jumbo cluster (of which the remaining > > 3k is simply wasted). This happens on any moderately active server, > > once physical memory gets checkerboarded with active single pages, > > particularly with ZFS where those pages are wired in kernel memory and > > so can't be evicted. > > > > -GAWollman > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 22:12:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C944BAA0 for ; Thu, 20 Mar 2014 22:12:50 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 82065C33 for ; Thu, 20 Mar 2014 22:12:50 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so1633042qaq.0 for ; Thu, 20 Mar 2014 15:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nqDCyCg/A6aKZJ11dh5H0+QyGvdDoKgwybTCDmNqvhU=; b=Prhvz5ZJawjbMu5G6/cnRsqTYDaMXmzWgDHz6/lArdtGALYh/ruPQq6yWkAd4x4jju qcHf1YTiiqS1ys6Hug0n2afslW+ufzdYyWjjUCxTsTMJ6mPBbo6y3z7KBRvLMcBBZm5W hAPgDjyCppgfNP0rdna5h4AmYdSnfoJQtIq6gFr4rJV0AzHBkSZasWuZ6kyEqsh/T2BG RccRo7fAzP9WAO/GmBXBCu02ZnhR/9aro1akYvSsFnOQbn74rFKnWGZph5q97o2KGVae dERBYRw2UkGZquIpaGbglCyLdZj5wHAJaEPFElufXhABOFa5oFWXQwL2CQUSlN1ZgTgv vFPA== MIME-Version: 1.0 X-Received: by 10.140.20.167 with SMTP id 36mr20700789qgj.54.1395353569731; Thu, 20 Mar 2014 15:12:49 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 15:12:49 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 19:12:49 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:12:50 -0000 Hi Jack, I'm on ixgbe 2.5.15 I see a few other threads about using MJUMPAGESIZE instead of MJUM9BYTES. If you have a patch you'd like me to test, I'll compile it in and let you know. I was just looking at Garrett's if_em.c patch and thinking about applying it to ixgbe.. As it stands I seem to not be having the problem now that I have disabled TSO on ix0, but I still need more test runs to confirm - Which is also in line (i think) with what you are all saying. On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel wrote: > What he's saying is that the driver should not be using 9K mbuf clusters, > I thought > this had been changed but I see the code in HEAD is still using the larger > clusters > when you up the mtu. I will put it on my list to change with the next > update to HEAD. > > > What version of ixgbe are you using? > > Jack > > > > On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron < > csforgeron@gmail.com> wrote: > >> I have found this: >> >> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html >> >> I think what you're saying is that; >> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster >> - modern NIC drivers can gather 9000 bytes of data from various memory >> locations >> - The fact that I'm seeing 9k jumbo clusters is showing me that my driver >> is trying to allocate 9k of contiguous space, and it's failing. >> >> Please correct me if I'm off here, I'd love to understand more. >> >> >> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < >> wollman@hergotha.csail.mit.edu> wrote: >> >> > In article >> > , >> > csforgeron@gmail.com writes: >> > >> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) >> > >> > This is going to screw you. You need to make sure that no NIC driver >> > ever allocates 9k jumbo pages -- unless you are using one of those >> > mythical drivers that can't do scatter/gather DMA on receive, which >> > you don't appear to be. >> > >> > These failures occur when the driver is trying to replenish its >> > receive queue, but is unable to allocate three *physically* contiguous >> > pages of RAM to construct the 9k jumbo cluster (of which the remaining >> > 3k is simply wasted). This happens on any moderately active server, >> > once physical memory gets checkerboarded with active single pages, >> > particularly with ZFS where those pages are wired in kernel memory and >> > so can't be evicted. >> > >> > -GAWollman >> > >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 22:24:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A20F0DD3 for ; Thu, 20 Mar 2014 22:24:47 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A0A5D2B for ; Thu, 20 Mar 2014 22:24:47 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id ld13so1758765vcb.33 for ; Thu, 20 Mar 2014 15:24:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zIBnyawiKBP5Iv8jxAmTnEmI+JAL1oLjovA9WlkrKGs=; b=w52Sz/MBh6B6W7WS5YonnDg9WQ8PBaCR6kbBEUTTBO1LPNUjDHLdJ/B+D1Cb9kjsDz nI0ioL8Aj3mSRvJTL1Rqe+wCltmM2JTxu7uYlQOCrCpFSjAq6Mf0qNybLkob8nQmSO3X 77oRvEK94YDwcX81uwPP6t9/4Q1W8hcelkv4CYJX/IDct8+Hyhhb2UcsnSAzFlW11F1G b1hzVL2c8tOyknAJpgOkN4gOY5XrnnLi8/KN6EugEmoExLJsXXwGUKW1kJqGuK19+9Je glpuyGQnIXkKpEu19S9tsKvO61ZaHfJkpmjSgb56KNQnYBSsbTzAo84S2eS56LnzxGeE sIew== MIME-Version: 1.0 X-Received: by 10.52.241.106 with SMTP id wh10mr30009421vdc.16.1395354286558; Thu, 20 Mar 2014 15:24:46 -0700 (PDT) Received: by 10.221.11.135 with HTTP; Thu, 20 Mar 2014 15:24:46 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 15:24:46 -0700 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Jack Vogel To: Christopher Forgeron Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:24:47 -0000 I strongly discourage anyone from disabling TSO on 10G, its necessary to get the performance one wants to see on the hardware. Here is a patch to do what i'm talking about: *** ixgbe.c Fri Jan 10 18:12:20 2014 --- ixgbe.jfv.c Thu Mar 20 23:04:15 2014 *************** ixgbe_init_locked(struct adapter *adapte *** 1140,1151 **** */ if (adapter->max_frame_size <= 2048) adapter->rx_mbuf_sz = MCLBYTES; - else if (adapter->max_frame_size <= 4096) - adapter->rx_mbuf_sz = MJUMPAGESIZE; - else if (adapter->max_frame_size <= 9216) - adapter->rx_mbuf_sz = MJUM9BYTES; else ! adapter->rx_mbuf_sz = MJUM16BYTES; /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { --- 1140,1147 ---- */ if (adapter->max_frame_size <= 2048) adapter->rx_mbuf_sz = MCLBYTES; else ! adapter->rx_mbuf_sz = MJUMPAGESIZE; /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron wrote: > Hi Jack, > > I'm on ixgbe 2.5.15 > > I see a few other threads about using MJUMPAGESIZE instead of MJUM9BYTES. > > If you have a patch you'd like me to test, I'll compile it in and let you > know. I was just looking at Garrett's if_em.c patch and thinking about > applying it to ixgbe.. > > As it stands I seem to not be having the problem now that I have disabled > TSO on ix0, but I still need more test runs to confirm - Which is also in > line (i think) with what you are all saying. > > > > > On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel wrote: > >> What he's saying is that the driver should not be using 9K mbuf clusters, >> I thought >> this had been changed but I see the code in HEAD is still using the >> larger clusters >> when you up the mtu. I will put it on my list to change with the next >> update to HEAD. >> >> >> What version of ixgbe are you using? >> >> Jack >> >> >> >> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron < >> csforgeron@gmail.com> wrote: >> >>> I have found this: >>> >>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html >>> >>> I think what you're saying is that; >>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster >>> - modern NIC drivers can gather 9000 bytes of data from various memory >>> locations >>> - The fact that I'm seeing 9k jumbo clusters is showing me that my driver >>> is trying to allocate 9k of contiguous space, and it's failing. >>> >>> Please correct me if I'm off here, I'd love to understand more. >>> >>> >>> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < >>> wollman@hergotha.csail.mit.edu> wrote: >>> >>> > In article >>> > , >>> > csforgeron@gmail.com writes: >>> > >>> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) >>> > >>> > This is going to screw you. You need to make sure that no NIC driver >>> > ever allocates 9k jumbo pages -- unless you are using one of those >>> > mythical drivers that can't do scatter/gather DMA on receive, which >>> > you don't appear to be. >>> > >>> > These failures occur when the driver is trying to replenish its >>> > receive queue, but is unable to allocate three *physically* contiguous >>> > pages of RAM to construct the 9k jumbo cluster (of which the remaining >>> > 3k is simply wasted). This happens on any moderately active server, >>> > once physical memory gets checkerboarded with active single pages, >>> > particularly with ZFS where those pages are wired in kernel memory and >>> > so can't be evicted. >>> > >>> > -GAWollman >>> > >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 22:32:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 811E2246 for ; Thu, 20 Mar 2014 22:32:18 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 39F68E30 for ; Thu, 20 Mar 2014 22:32:18 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id o15so1619339qap.37 for ; Thu, 20 Mar 2014 15:32:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=J8rd6wIdjmVReBrSAn7difcBlde/mlL6d37uqlSPzus=; b=w74w3Y83opq4q4cGBmH+0H/PNBrgh9fuDWlPaaXMRh6F7MVckjEzMOIsWnmWRSwdcv W6Aj/Kc6E8fVQXHLdnTzPcp7C2Z5JPdjP5jkyZxcRQFuVtxhgCyHmQObjBFx26CYeQhB v9xkH2ipq9azFKb2UftKQztRgKJFzb0Ki1Lx7V+vHmUPqUqv+xenYoEtF7/SFe4Row1J cuuhDwOY0OPj5iLJNaf8yf2ZdgQdrvaz8a2gbcNm7KtpW8ot3Bwo8rz3PjIFoWJxHM3W F1HgOyMjl9USz783JDPPZNCZKYTE9SRWW6fgnJraQSdONCqVe4JMsKJhOnTf39yLOACK X6QQ== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr36662338qga.57.1395354737422; Thu, 20 Mar 2014 15:32:17 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 15:32:17 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 19:32:17 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:32:18 -0000 I agree, performance is noticeably worse with TSO off, but I thought it would be a good step in troubleshooting. I'm glad you're a regular reader of the list, so I don't have to settle for slow performance. :-) I'm applying your patch now, I think it will fix it - but I'll report in after it's run iometer for the night regardless. On another note: What's so different about memory allocation in 10 that is making this an issue? On Thu, Mar 20, 2014 at 7:24 PM, Jack Vogel wrote: > I strongly discourage anyone from disabling TSO on 10G, its necessary to > get the > performance one wants to see on the hardware. > > Here is a patch to do what i'm talking about: > > *** ixgbe.c Fri Jan 10 18:12:20 2014 > --- ixgbe.jfv.c Thu Mar 20 23:04:15 2014 > *************** ixgbe_init_locked(struct adapter *adapte > *** 1140,1151 **** > */ > if (adapter->max_frame_size <= 2048) > adapter->rx_mbuf_sz = MCLBYTES; > - else if (adapter->max_frame_size <= 4096) > - adapter->rx_mbuf_sz = MJUMPAGESIZE; > - else if (adapter->max_frame_size <= 9216) > - adapter->rx_mbuf_sz = MJUM9BYTES; > else > ! adapter->rx_mbuf_sz = MJUM16BYTES; > > /* Prepare receive descriptors and buffers */ > if (ixgbe_setup_receive_structures(adapter)) { > --- 1140,1147 ---- > */ > if (adapter->max_frame_size <= 2048) > adapter->rx_mbuf_sz = MCLBYTES; > else > ! adapter->rx_mbuf_sz = MJUMPAGESIZE; > > /* Prepare receive descriptors and buffers */ > if (ixgbe_setup_receive_structures(adapter)) { > > > > > > > On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron < > csforgeron@gmail.com> wrote: > >> Hi Jack, >> >> I'm on ixgbe 2.5.15 >> >> I see a few other threads about using MJUMPAGESIZE instead of MJUM9BYTES. >> >> If you have a patch you'd like me to test, I'll compile it in and let >> you know. I was just looking at Garrett's if_em.c patch and thinking about >> applying it to ixgbe.. >> >> As it stands I seem to not be having the problem now that I have >> disabled TSO on ix0, but I still need more test runs to confirm - Which is >> also in line (i think) with what you are all saying. >> >> >> >> >> On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel wrote: >> >>> What he's saying is that the driver should not be using 9K mbuf >>> clusters, I thought >>> this had been changed but I see the code in HEAD is still using the >>> larger clusters >>> when you up the mtu. I will put it on my list to change with the next >>> update to HEAD. >>> >>> >>> What version of ixgbe are you using? >>> >>> Jack >>> >>> >>> >>> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron < >>> csforgeron@gmail.com> wrote: >>> >>>> I have found this: >>>> >>>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html >>>> >>>> I think what you're saying is that; >>>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster >>>> - modern NIC drivers can gather 9000 bytes of data from various memory >>>> locations >>>> - The fact that I'm seeing 9k jumbo clusters is showing me that my >>>> driver >>>> is trying to allocate 9k of contiguous space, and it's failing. >>>> >>>> Please correct me if I'm off here, I'd love to understand more. >>>> >>>> >>>> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < >>>> wollman@hergotha.csail.mit.edu> wrote: >>>> >>>> > In article >>>> > , >>>> > csforgeron@gmail.com writes: >>>> > >>>> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) >>>> > >>>> > This is going to screw you. You need to make sure that no NIC driver >>>> > ever allocates 9k jumbo pages -- unless you are using one of those >>>> > mythical drivers that can't do scatter/gather DMA on receive, which >>>> > you don't appear to be. >>>> > >>>> > These failures occur when the driver is trying to replenish its >>>> > receive queue, but is unable to allocate three *physically* contiguous >>>> > pages of RAM to construct the 9k jumbo cluster (of which the remaining >>>> > 3k is simply wasted). This happens on any moderately active server, >>>> > once physical memory gets checkerboarded with active single pages, >>>> > particularly with ZFS where those pages are wired in kernel memory and >>>> > so can't be evicted. >>>> > >>>> > -GAWollman >>>> > >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>> >>> >> > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 22:42:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDEBA70C for ; Thu, 20 Mar 2014 22:42:02 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 83846F1E for ; Thu, 20 Mar 2014 22:42:02 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id la4so1783898vcb.31 for ; Thu, 20 Mar 2014 15:42:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2Dty+3mq2+dv4gVdt6LbFcAEWij4f5hxBpRFCKisKRs=; b=Hz0F0/BVteXZaxPDFE0tnW41p0Jnjj5xIbIc0HQHr2amY7WRXTtZLftYXLdqmhBcwP UGip47ecZkfBLYAl+rGv1aeJfPyVfD8OBhZld7D4OE/lEZGel1CrPU4PREDjVZBAg4y3 DfPmXlTsbqBIMqGk8NzAcLaiy5kbKGdDUQdcV8E49Sd00VRN83bsGGVGo9rU/5UwDca6 1ENNzW2oIsakzTBVozGSYnrS9t6DrQf8sowwviBxEke0Hn6C9P1SPQ6buQN8pJu5rLzg eC57dsfNSi6OXmqHhr1MPzyqdAO14CozRZSrgiro2xeQ14u3aCZfuLFhAWDBLppC3Y3n Vc+w== MIME-Version: 1.0 X-Received: by 10.52.137.74 with SMTP id qg10mr33426vdb.61.1395355321667; Thu, 20 Mar 2014 15:42:01 -0700 (PDT) Received: by 10.221.11.135 with HTTP; Thu, 20 Mar 2014 15:42:01 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 15:42:01 -0700 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Jack Vogel To: Christopher Forgeron Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:42:02 -0000 Your 4K mbuf pool is not being used, make sure you increase the size once you are using that or you'll just be having the same issue with a different pool. Oh, and that patch was against the code in HEAD, it might need some manual hacking if you're using anything older. Not sure what you mean about memory allocation in 10, this change is not 10 specific, its something I intended on doing and it just slipped between the cracks. Jack On Thu, Mar 20, 2014 at 3:32 PM, Christopher Forgeron wrote: > I agree, performance is noticeably worse with TSO off, but I thought it > would be a good step in troubleshooting. I'm glad you're a regular reader > of the list, so I don't have to settle for slow performance. :-) > > I'm applying your patch now, I think it will fix it - but I'll report in > after it's run iometer for the night regardless. > > On another note: What's so different about memory allocation in 10 that is > making this an issue? > > > > > On Thu, Mar 20, 2014 at 7:24 PM, Jack Vogel wrote: > >> I strongly discourage anyone from disabling TSO on 10G, its necessary to >> get the >> performance one wants to see on the hardware. >> >> Here is a patch to do what i'm talking about: >> >> *** ixgbe.c Fri Jan 10 18:12:20 2014 >> --- ixgbe.jfv.c Thu Mar 20 23:04:15 2014 >> *************** ixgbe_init_locked(struct adapter *adapte >> *** 1140,1151 **** >> */ >> if (adapter->max_frame_size <= 2048) >> adapter->rx_mbuf_sz = MCLBYTES; >> - else if (adapter->max_frame_size <= 4096) >> - adapter->rx_mbuf_sz = MJUMPAGESIZE; >> - else if (adapter->max_frame_size <= 9216) >> - adapter->rx_mbuf_sz = MJUM9BYTES; >> else >> ! adapter->rx_mbuf_sz = MJUM16BYTES; >> >> /* Prepare receive descriptors and buffers */ >> if (ixgbe_setup_receive_structures(adapter)) { >> --- 1140,1147 ---- >> */ >> if (adapter->max_frame_size <= 2048) >> adapter->rx_mbuf_sz = MCLBYTES; >> else >> ! adapter->rx_mbuf_sz = MJUMPAGESIZE; >> >> /* Prepare receive descriptors and buffers */ >> if (ixgbe_setup_receive_structures(adapter)) { >> >> >> >> >> >> >> On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron < >> csforgeron@gmail.com> wrote: >> >>> Hi Jack, >>> >>> I'm on ixgbe 2.5.15 >>> >>> I see a few other threads about using MJUMPAGESIZE instead of >>> MJUM9BYTES. >>> >>> If you have a patch you'd like me to test, I'll compile it in and let >>> you know. I was just looking at Garrett's if_em.c patch and thinking about >>> applying it to ixgbe.. >>> >>> As it stands I seem to not be having the problem now that I have >>> disabled TSO on ix0, but I still need more test runs to confirm - Which is >>> also in line (i think) with what you are all saying. >>> >>> >>> >>> >>> On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel wrote: >>> >>>> What he's saying is that the driver should not be using 9K mbuf >>>> clusters, I thought >>>> this had been changed but I see the code in HEAD is still using the >>>> larger clusters >>>> when you up the mtu. I will put it on my list to change with the next >>>> update to HEAD. >>>> >>>> >>>> What version of ixgbe are you using? >>>> >>>> Jack >>>> >>>> >>>> >>>> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron < >>>> csforgeron@gmail.com> wrote: >>>> >>>>> I have found this: >>>>> >>>>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html >>>>> >>>>> I think what you're saying is that; >>>>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster >>>>> - modern NIC drivers can gather 9000 bytes of data from various memory >>>>> locations >>>>> - The fact that I'm seeing 9k jumbo clusters is showing me that my >>>>> driver >>>>> is trying to allocate 9k of contiguous space, and it's failing. >>>>> >>>>> Please correct me if I'm off here, I'd love to understand more. >>>>> >>>>> >>>>> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < >>>>> wollman@hergotha.csail.mit.edu> wrote: >>>>> >>>>> > In article >>>>> > >>>> >, >>>>> > csforgeron@gmail.com writes: >>>>> > >>>>> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) >>>>> > >>>>> > This is going to screw you. You need to make sure that no NIC driver >>>>> > ever allocates 9k jumbo pages -- unless you are using one of those >>>>> > mythical drivers that can't do scatter/gather DMA on receive, which >>>>> > you don't appear to be. >>>>> > >>>>> > These failures occur when the driver is trying to replenish its >>>>> > receive queue, but is unable to allocate three *physically* >>>>> contiguous >>>>> > pages of RAM to construct the 9k jumbo cluster (of which the >>>>> remaining >>>>> > 3k is simply wasted). This happens on any moderately active server, >>>>> > once physical memory gets checkerboarded with active single pages, >>>>> > particularly with ZFS where those pages are wired in kernel memory >>>>> and >>>>> > so can't be evicted. >>>>> > >>>>> > -GAWollman >>>>> > >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>>> >>>> >>>> >>> >> > From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 23:01:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A49DAEC for ; Thu, 20 Mar 2014 23:01:01 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E5334F9 for ; Thu, 20 Mar 2014 23:01:00 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id e89so4728423qgf.5 for ; Thu, 20 Mar 2014 16:01:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=z1j12ve1hsVDfQmU6s8JoitgRkdFveHzdh+KYBvs//k=; b=IS5v8MHVmzUSLsS4Znwvg4aXYWNkP8/v3k/GMlbMUoDOfH4wVE78X4SVp+9sUZnQ+g O0YZiL9c6wAuTDjLUizP79vAvY/VlibeMl+phDo2ipLbIP6tkCVt3Ae3g7WBniKYiIrh N9AURKyd6WZyfFbDfjqjQlCwBksavF30oG7QHcSecAH8JzvDfBMjYwPNJvdx18J2q2VZ D4zCzHT+KfSg4hTm8xKKiuZYsDAT0FuivjdnHqgPxUVeMnELwTA64pFYnKEhcJvI36Sg Cj2apwrKqhLYs4k2rFrWEzUaXoTTOEzYrymZ+uvwohpGnrBzgn7bQ9iYSAkmTG9cxbmq 8xgw== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr6146514qab.93.1395356460126; Thu, 20 Mar 2014 16:01:00 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 16:01:00 -0700 (PDT) In-Reply-To: References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 20:01:00 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 23:01:01 -0000 Ah, good point about the 4k buff size : I will allocate more to kern.ipc.nmbjumbop , perhaps taking it from 9 and 16. Yes, I did have to tweak the patch slightly to work on 10.0, but it's basically the same thing I was trying after looking at Garrett's notes. I see this is part of a larger problem, but I didn't see any issues with a 9.0 system for over a year, and my 9.2 system seems to be stable (all the same hardware, same use). I was thinking it was an issue with later 9.2's or 10, but ultimately I guess it's just a problem on any system that can't allocate 3 contiguous 4k memory pages quickly enough. (?).. I do notice ~ 30% more NFS speed to my ZFS pool with 10.0 - Perhaps that's the key performance to start noticing this problem. Then again, my 10.0 system starts out with denied 9k bufs at boot, where my 9.2 doesn't. There's no real memory pressure on boot when I have 96G of RAM I would expect. (I also wonder if I I shouldn't be considering a MTU that fits inside a MJUMPAGESIZE. I don't think my switches support a MTU that will == 3 or 4 full MJUMPAGESIZE. Then again, wasting a bit of memory on the server may be worth it to have slightly fewer TCP frames. ) What should be done about the other network drivers that still call MJUM9BYTES? http://fxr.watson.org/fxr/ident?im=excerpts;i=MJUM9BYTES I have a collection of a number of different NICs, I could test a few of these to verify they work okay with the same sort of patch we're talking about. I appreciate the help everyone gives me here, so I'm willing to help out if it's needed. Thanks again. On Thu, Mar 20, 2014 at 7:42 PM, Jack Vogel wrote: > Your 4K mbuf pool is not being used, make sure you increase the size once > you are > using that or you'll just be having the same issue with a different pool. > > Oh, and that patch was against the code in HEAD, it might need some manual > hacking > if you're using anything older. > > Not sure what you mean about memory allocation in 10, this change is not > 10 specific, its > something I intended on doing and it just slipped between the cracks. > > Jack > > > > On Thu, Mar 20, 2014 at 3:32 PM, Christopher Forgeron < > csforgeron@gmail.com> wrote: > >> I agree, performance is noticeably worse with TSO off, but I thought it >> would be a good step in troubleshooting. I'm glad you're a regular reader >> of the list, so I don't have to settle for slow performance. :-) >> >> I'm applying your patch now, I think it will fix it - but I'll report in >> after it's run iometer for the night regardless. >> >> On another note: What's so different about memory allocation in 10 that >> is making this an issue? >> >> >> >> >> On Thu, Mar 20, 2014 at 7:24 PM, Jack Vogel wrote: >> >>> I strongly discourage anyone from disabling TSO on 10G, its necessary to >>> get the >>> performance one wants to see on the hardware. >>> >>> Here is a patch to do what i'm talking about: >>> >>> *** ixgbe.c Fri Jan 10 18:12:20 2014 >>> --- ixgbe.jfv.c Thu Mar 20 23:04:15 2014 >>> *************** ixgbe_init_locked(struct adapter *adapte >>> *** 1140,1151 **** >>> */ >>> if (adapter->max_frame_size <= 2048) >>> adapter->rx_mbuf_sz = MCLBYTES; >>> - else if (adapter->max_frame_size <= 4096) >>> - adapter->rx_mbuf_sz = MJUMPAGESIZE; >>> - else if (adapter->max_frame_size <= 9216) >>> - adapter->rx_mbuf_sz = MJUM9BYTES; >>> else >>> ! adapter->rx_mbuf_sz = MJUM16BYTES; >>> >>> /* Prepare receive descriptors and buffers */ >>> if (ixgbe_setup_receive_structures(adapter)) { >>> --- 1140,1147 ---- >>> */ >>> if (adapter->max_frame_size <= 2048) >>> adapter->rx_mbuf_sz = MCLBYTES; >>> else >>> ! adapter->rx_mbuf_sz = MJUMPAGESIZE; >>> >>> /* Prepare receive descriptors and buffers */ >>> if (ixgbe_setup_receive_structures(adapter)) { >>> >>> >>> >>> >>> >>> >>> On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron < >>> csforgeron@gmail.com> wrote: >>> >>>> Hi Jack, >>>> >>>> I'm on ixgbe 2.5.15 >>>> >>>> I see a few other threads about using MJUMPAGESIZE instead of >>>> MJUM9BYTES. >>>> >>>> If you have a patch you'd like me to test, I'll compile it in and let >>>> you know. I was just looking at Garrett's if_em.c patch and thinking about >>>> applying it to ixgbe.. >>>> >>>> As it stands I seem to not be having the problem now that I have >>>> disabled TSO on ix0, but I still need more test runs to confirm - Which is >>>> also in line (i think) with what you are all saying. >>>> >>>> >>>> >>>> >>>> On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel wrote: >>>> >>>>> What he's saying is that the driver should not be using 9K mbuf >>>>> clusters, I thought >>>>> this had been changed but I see the code in HEAD is still using the >>>>> larger clusters >>>>> when you up the mtu. I will put it on my list to change with the next >>>>> update to HEAD. >>>>> >>>>> >>>>> What version of ixgbe are you using? >>>>> >>>>> Jack >>>>> >>>>> >>>>> >>>>> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron < >>>>> csforgeron@gmail.com> wrote: >>>>> >>>>>> I have found this: >>>>>> >>>>>> >>>>>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html >>>>>> >>>>>> I think what you're saying is that; >>>>>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster >>>>>> - modern NIC drivers can gather 9000 bytes of data from various memory >>>>>> locations >>>>>> - The fact that I'm seeing 9k jumbo clusters is showing me that my >>>>>> driver >>>>>> is trying to allocate 9k of contiguous space, and it's failing. >>>>>> >>>>>> Please correct me if I'm off here, I'd love to understand more. >>>>>> >>>>>> >>>>>> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < >>>>>> wollman@hergotha.csail.mit.edu> wrote: >>>>>> >>>>>> > In article >>>>>> > >>>>> >, >>>>>> > csforgeron@gmail.com writes: >>>>>> > >>>>>> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) >>>>>> > >>>>>> > This is going to screw you. You need to make sure that no NIC >>>>>> driver >>>>>> > ever allocates 9k jumbo pages -- unless you are using one of those >>>>>> > mythical drivers that can't do scatter/gather DMA on receive, which >>>>>> > you don't appear to be. >>>>>> > >>>>>> > These failures occur when the driver is trying to replenish its >>>>>> > receive queue, but is unable to allocate three *physically* >>>>>> contiguous >>>>>> > pages of RAM to construct the 9k jumbo cluster (of which the >>>>>> remaining >>>>>> > 3k is simply wasted). This happens on any moderately active server, >>>>>> > once physical memory gets checkerboarded with active single pages, >>>>>> > particularly with ZFS where those pages are wired in kernel memory >>>>>> and >>>>>> > so can't be evicted. >>>>>> > >>>>>> > -GAWollman >>>>>> > >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org >>>>>> " >>>>>> >>>>> >>>>> >>>> >>> >> > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:13:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A0CCD67 for ; Fri, 21 Mar 2014 02:13:31 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id B1CF07BE for ; Fri, 21 Mar 2014 02:13:30 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAP2eK1ODaFve/2dsb2JhbABZg0FXgwe4J4ZjUYErdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1AIDa0UomAXgSmMWhACARs0B4JvgUkElXKECZB/g0khMYE9 X-IronPort-AV: E=Sophos;i="4.97,700,1389762000"; d="scan'208";a="107524175" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 20 Mar 2014 22:13:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3D667B3F19; Thu, 20 Mar 2014 22:13:22 -0400 (EDT) Date: Thu, 20 Mar 2014 22:13:22 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:13:31 -0000 Christopher Forgeron wrote: > > Output from the patch you gave me (I have screens of it.. let me know > what you're hoping to see. > > > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 Hmm. I think this means that the loop that generates TSO segments in tcp_output() is broken, since I'm pretty sure that the maximum size should be is IP_MAXPACKET (65535). Either that or some non-TCP socket is trying to send a packet that exceeds IP_MAXPACKET for some reason. Would it be possible to add a printf() for m->m_pkthdr.csum_flags to the before case, in the "if" that generates the before printf? I didn't think to put this in, but CSUM_TSO will be set if it is a TSO segment, I think? My networking is very rusty. (If how to add this isn't obvious, just email and I'll update the patch.) Thanks for doing this, rick > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:23 SAN0 kernel: before pklen=65538 actl=65538 > Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 20 16:37:23 SAN0 kernel: before pklen=65542 actl=65542 > Mar 20 16:37:23 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65 > > > > > > On Wed, Mar 19, 2014 at 11:29 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > Christopher Forgeron wrote: > > Hello, > > > > > > > > I can report this problem as well on 10.0-RELEASE. > > > > > > > > I think it's the same as kern/183390? > > > > > > > > I have two physically identical machines, one running 9.2-STABLE, > > and > > one > > on 10.0-RELEASE. > > > > > > > > My 10.0 machine used to be running 9.0-STABLE for over a year > > without > > any > > problems. > > > > > > > > I'm not having the problems with 9.2-STABLE as far as I can tell, > > but > > it > > does seem to be a load-based issue more than anything. Since my 9.2 > > system > > is in production, I'm unable to load it to see if the problem > > exists > > there. > > I have a ping_logger.py running on it now to see if it's > > experiencing > > problems briefly or not. > > > > > > > > I am able to reproduce it fairly reliably within 15 min of a reboot > > by > > loading the server via NFS with iometer and some large NFS file > > copies at > > the same time. I seem to need to sustain ~2 Gbps for a few minutes. > > > If you can easily do so, testing with the attached patch might shed > some light on the problem. It just adds a couple of diagnostic checks > before and after m_defrag() is called when bus_dmamap_load_mbuf_sg() > returns EFBIG. > > If the "before" printf happens, it would suggest a problem with the > loop in tcp_output() that creates TSO segments. > > If the "after" printf happens, it would suggest that m_defrag() > somehow > doesn't create a list of 32 or fewer mbufs for the TSO segment. > > I don't have any ix hardware, so this patch is completely untested. > > Just something maybe worth trying, rick > > > > > > > > It will happen with just ix0 (no lagg) or with lagg enabled across > > ix0 and > > ix1. > > > > > > > > I've been load-testing new FreeBSD-10.0-RELEASE SAN's for > > production > > use > > here, so I'm quite willing to put time into this to help find out > > where > > it's coming from. It took me a day to track down my iometer issues > > as > > being network related, and another day to isolate and write scripts > > to > > reproduce. > > > > > > > > The symptom I notice is: > > > > - A running flood ping (ping -f 172.16.0.31) to the same > > > > hardware > > (running 9.2) will come back with "ping: sendto: File too large" > > when > > the > > problem occurs > > > > - Network connectivity is very spotty during these incidents > > > > - It can run with sporadic ping errors, or it can run a > > straight > > set of errors for minutes at a time > > > > - After a long run of ping errors, ESXi will show a > > disconnect > > from the hosted NFS stores on this machine. > > > > - I've yet to see it happen right after boot. Fastest is > > around 5 > > min, normally it's within 15 min. > > > > > > > > System Specs: > > > > > > > > - Dell PowerEdge M610x Blade > > > > - 2 Xeon 6600 @ 2.40GHz (24 Cores total) > > > > - 96 Gig RAM > > > > - 35.3 TB ZFS Mirrored pool, lz4 compression on my test pool > > (ZFS > > pool is the latest) > > > > - Intel 520-DA2 10 Gb dual-port Blade Mezz. Cards > > > > > > > > Currently this 10.0 testing machine is clean for all sysctl's other > > than > > hw.intr_storm_threshold=9900. I have the problem if that's set or > > not, so I > > leave it on. > > > > > > > > ( I used to set manual nmbclusters, etc. as per the Intel > > Readme.doc, > > but I > > notice that the defaults on the new 10.0 system are larger. I did > > try > > using > > all of the old sysctl's from an older 9.0-STABLE, and still had the > > problem, but it did seem to take longer to occur? I haven't run > > enough > > tests to confirm that time observation is true. ) > > > > > > > > What logs / info can I provide to help? > > > > > > > > I have written a small script called ping_logger.py that pings an > > IP, > > and > > checks to see if there is an error. On error it will execute and > > log: > > > > - netstat -m > > > > > - sysctl hw.ix > > > > - sysctl dev.ix > > > > > > > > then go back to pinging. It will also log those values on the > > startup > > of > > the script, and every 5 min (so you can see the progression on the > > system). > > I can add any number of things to the reporting, so I'm looking for > > suggestions. > > > > > > > > This results in some large log files, but I can email a .gz > > directly > > to > > anyone who need them, or perhaps put it up on a website. > > > > > > > > I will also make the ping_logger.py script available if anyone else > > wants > > it. > > > > > > > > > > > > LASTLY: > > > > > > > > The one thing I can see that is different in my 10.0 System and my > > 9.2 is: > > > > > > > > 9.2's netstat -m: > > > > > > > > > 37965/16290/54255 mbufs in use (current/cache/total) > > > > 4080/8360/12440/524288 mbuf clusters in use > > (current/cache/total/max) > > > > 4080/4751 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > > > 0/452/452/262144 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > > > 32773/4129/36902/96000 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > > > 312608K/59761K/372369K bytes allocated to network > > (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > > > > > 10.0's netstat -m: > > > > > > > > > 21512/24448/45960 mbufs in use (current/cache/total) > > > > 4080/16976/21056/6127254 mbuf clusters in use > > (current/cache/total/max) > > > > 4080/16384 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > > > 0/23/23/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > > > 16384/158/16542/907741 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > > 160994K/41578K/202572K bytes allocated to network > > (current/cache/total) > > > > 17488/13290/20464 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 7/16462/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > Way more mbuf clusters in use, but also I never get denied/delayed > > results > > in 9.2 - but I have them in 10.0 right away after a reboot. > > > > > > > > Thanks for any help.. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > " freebsd-net-unsubscribe@freebsd.org " > > > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:25:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2083EB2 for ; Fri, 21 Mar 2014 02:25:53 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7144B877 for ; Fri, 21 Mar 2014 02:25:52 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAH2iK1ODaFve/2dsb2JhbABZhBiDB79fgSt0giUBAQEEI1YbDgoCAg0ZAlkGE4d5rR+iYheBKYxzFQEzB4JvgUkEqnqDSSGBLAEfBB4 X-IronPort-AV: E=Sophos;i="4.97,700,1389762000"; d="scan'208";a="107781963" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 20 Mar 2014 22:25:51 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id AA2F279283; Thu, 20 Mar 2014 22:25:51 -0400 (EDT) Date: Thu, 20 Mar 2014 22:25:51 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <477992488.642193.1395368751685.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:25:53 -0000 Christopher Forgeron wrote: >=20 >=20 >=20 >=20 >=20 >=20 > On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert < > markus.gebert@hostpoint.ch > wrote: >=20 >=20 >=20 >=20 >=20 > Possible. We still see this on nfsclients only, but I=E2=80=99m not convi= nced > that nfs is the only trigger. >=20 >=20 Since Christopher is getting a bunch of the "before" printf()s from my patch, it indicates that a packet/TSO segment that is > 65535 bytes in length is showing up at ixgbe_xmit(). I've asked him to add a printf() for the m_pkthdr.csum_flags field to see if it is really a TSO segment. If it is a TSO segment, that indicates to me that the code in tcp_output() = that should generate a TSO segment no greater than 65535 bytes in length is busted. And this would imply just about any app doing large sosend()s could cause this, I think? (NFS read replies/write requests of 64K would be one of them= .) rick >=20 >=20 >=20 > Just to clarify, I'm experiencing this error with NFS, but also with > iSCSI - I turned off my NFS server in rc.conf and rebooted, and I'm > still able to create the error. This is not just a NFS issue on my > machine. >=20 >=20 >=20 > I our case, when it happens, the problem persists for quite some time > (minutes or hours) if we don=E2=80=99t interact (ifconfig or reboot). >=20 >=20 >=20 > The first few times that I ran into it, I had similar issues - > Because I was keeping my system up and treating it like a temporary > problem/issue. Worst case scenario resulted in reboots to reset the > NIC. Then again, I find the ix's to be cranky if you ifconfig them > too much. >=20 > Now, I'm trying to find a root cause, so as soon as I start seeing > any errors, I abort and reboot the machine to test the next theory. >=20 >=20 > Additionally, I'm often able to create the problem with just 1 VM > running iometer on the SAN storage. When the problem occurs, that > connection is broken temporarily, taking network load off the SAN - > That may improve my chances of keeping this running. >=20 >=20 >=20 >=20 >=20 > > I am able to reproduce it fairly reliably within 15 min of a reboot > > by > > loading the server via NFS with iometer and some large NFS file > > copies at > > the same time. I seem to need to sustain ~2 Gbps for a few minutes. >=20 > That=E2=80=99s probably why we can=E2=80=99t reproduce it reliably here. = Although > having 10gig cards in our blade servers, the ones affected are > connected to a 1gig switch. >=20 >=20 >=20 >=20 >=20 > It seems that it needs a lot of traffic. I have a 10 gig backbone > between my SANs and my ESXi machines, so I can saturate quite > quickly (just now I hit a record.. the error occurred within ~5 min > of reboot and testing). In your case, I recommend firing up multiple > VM's running iometer on different 1 gig connections and see if you > can make it pop. I also often turn off ix1 to drive all traffic > through ix0 - I've noticed it happens faster this way, but once > again I'm not taking enough observations to make decent time > predictions. >=20 >=20 >=20 >=20 >=20 >=20 > Can you try this when the problem occurs? >=20 > for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 > -c 2 -W 1 10.0.0.1 | grep sendto; done >=20 > It will tie ping to certain cpus to test the different tx queues of > your ix interface. If the pings reliably fail only on some queues, > then your problem is more likely to be the same as ours. >=20 > Also, if you have dtrace available: >=20 > kldload dtraceall > dtrace -n 'fbt:::return / arg1 =3D=3D EFBIG && execname =3D=3D "ping" / { > stack(); }' >=20 > while you run pings over the interface affected. This will give you > hints about where the EFBIG error comes from. >=20 > > [=E2=80=A6] >=20 >=20 > Markus >=20 >=20 >=20 >=20 > Will do. I'm not sure what shell the first script was written for, > it's not working in csh, here's a re-write that does work in csh in > case others are using the default shell: >=20 > #!/bin/csh > foreach CPU (`seq 0 23`) > echo "CPU$CPU"; > cpuset -l $CPU ping -i 0.2 -c 2 -W 1 10.0.0.1 | grep sendto; > end >=20 >=20 > Thanks for your input. I should have results to post to the list > shortly. >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:33:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61627FEA for ; Fri, 21 Mar 2014 02:33:00 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 217D790F for ; Fri, 21 Mar 2014 02:33:00 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id z60so5272801qgd.8 for ; Thu, 20 Mar 2014 19:32:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Xm3kc1LMOQV5BrYxQ27Z8xGBNRYt/O4yHCNJrDV1Fq4=; b=GbWXKeKVj/7LH+JwT+axl5I5f6cFYYGbAFvUG1/s1Mw7ZeWO0SpKV6iOWe1dSvieZN hQ2amLEgjIdT5jqu35BIs7cLrR3rP8LqGLJR3yqukhmrNIYTttYl56uANnpih0AClyue 3/eXtifOg7o0t9W857muaUBn8R9ipNyrYky4GyaKDqw84aw3AO33PvDgUSASVcMv33G9 ewoIusFEiVHY2d+in6tdlkWHDFfPPuklMcjk3LbTRzFeF4/ir8EJkHjroSCzICXcY1JT hdGT2BsMsXD0JHziSaMuuRE64iP1nd+cYd+VgJOV+k2+lQ66wDugI+8ve5w57pZHc8Wd XPPA== MIME-Version: 1.0 X-Received: by 10.140.95.65 with SMTP id h59mr51900609qge.2.1395369179137; Thu, 20 Mar 2014 19:32:59 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 19:32:59 -0700 (PDT) In-Reply-To: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 23:32:59 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:33:00 -0000 Yes, there is something broken in TSO for sure, as disabling it allows me to run without error. It is possible that the drop in performance is allowing me to stay under a critical threshold for the problem, but I'd feel happier testing to make sure. I understand what you're asking for in the patch, I'll make the edits tomorrow and recompile a test kernel and see. Right now I'm running tests on the ixgbe that Jack sent. Even if his patch fixes the issue, I wonder if something else isn't broken in TSO, as the ixgbe code has had these lines for a long time, and it's only on this 10.0 build that I have issues. I'll be following up tomorrow with info on either outcome. Thanks for your help.. your rusty networking is still better than mine. :-) On Thu, Mar 20, 2014 at 11:13 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > Output from the patch you gave me (I have screens of it.. let me know > > what you're hoping to see. > > > > > > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > Hmm. I think this means that the loop that generates TSO segments in > tcp_output() is broken, since I'm pretty sure that the maximum size > should be is IP_MAXPACKET (65535). > > Either that or some non-TCP socket is trying to send a packet that > exceeds IP_MAXPACKET for some reason. > > Would it be possible to add a printf() for m->m_pkthdr.csum_flags > to the before case, in the "if" that generates the before printf? > I didn't think to put this in, but CSUM_TSO will be set if it > is a TSO segment, I think? My networking is very rusty. > (If how to add this isn't obvious, just email and I'll update > the patch.) > > Thanks for doing this, rick > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:45:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E6E92DF; Fri, 21 Mar 2014 02:45:17 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id A08FF9C0; Fri, 21 Mar 2014 02:45:16 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAD2nK1ODaFve/2dsb2JhbABZg0FXgwe4FIZjUYErdIIlAQEBAwEBAQEgBCcgCxsYAgINGQIpAQkmBgEHBwQBCBQEh1AIDa0XomAXgSmMZQYBARsBMweCb4FJBJVyhAmQf4NJITF8CBci X-IronPort-AV: E=Sophos;i="4.97,700,1389762000"; d="scan'208";a="107529372" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 20 Mar 2014 22:45:15 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 8430BB3F19; Thu, 20 Mar 2014 22:45:15 -0400 (EDT) Date: Thu, 20 Mar 2014 22:45:15 -0400 (EDT) From: Rick Macklem To: Markus Gebert , Christopher Forgeron Message-ID: <429006400.647323.1395369915529.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Network stack returning EFBIG? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: jfv@freebsd.org, freebsd-net@freebsd.org, freebsd-stable@freebsd.org, wollman@bimajority.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:45:17 -0000 Markus Gebert wrote: >=20 > On 20.03.2014, at 14:51, wollman@bimajority.org wrote: >=20 > > In article <21290.60558.750106.630804@hergotha.csail.mit.edu>, I > > wrote: > >=20 > >> Since we put this server into production, random network system > >> calls > >> have started failing with [EFBIG] or maybe sometimes [EIO]. I've > >> observed this with a simple ping, but various daemons also log the > >> errors: > >> Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File > >> too > >> large [preauth] > >> Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete > >> SSL > >> handshake. 5 > >=20 > > I found at least one call stack where this happens and it does get > > returned all the way to userspace: > >=20 > > 17 15547 _bus_dmamap_load_buffer:return > > kernel`_bus_dmamap_load_mbuf_sg+0x5f > > kernel`bus_dmamap_load_mbuf_sg+0x38 > > kernel`ixgbe_xmit+0xcf > > kernel`ixgbe_mq_start_locked+0x94 > > kernel`ixgbe_mq_start+0x12a > > if_lagg.ko`lagg_transmit+0xc4 > > kernel`ether_output_frame+0x33 > > kernel`ether_output+0x4fe > > kernel`ip_output+0xd74 > > kernel`tcp_output+0xfea > > kernel`tcp_usr_send+0x325 > > kernel`sosend_generic+0x3f6 > > kernel`soo_write+0x5e > > kernel`dofilewrite+0x85 > > kernel`kern_writev+0x6c > > kernel`sys_write+0x64 > > kernel`amd64_syscall+0x5ea > > kernel`0xffffffff808443c7 >=20 > This looks pretty similar to what we=E2=80=99ve seen when we got EFBIG: >=20 > 3 28502 _bus_dmamap_load_buffer:return > kernel`_bus_dmamap_load_mbuf_sg+0x5f > kernel`bus_dmamap_load_mbuf_sg+0x38 > kernel`ixgbe_xmit+0xcf > kernel`ixgbe_mq_start_locked+0x94 > kernel`ixgbe_mq_start+0x12a > kernel`ether_output_frame+0x33 > kernel`ether_output+0x4fe > kernel`ip_output+0xd74 > kernel`rip_output+0x229 > kernel`sosend_generic+0x3f6 > kernel`kern_sendit+0x1a3 > kernel`sendit+0xdc > kernel`sys_sendto+0x4d > kernel`amd64_syscall+0x5ea > kernel`0xffffffff80d35667 >=20 > In our case it looks like some of the ixgbe tx queues get stuck, and > some don=E2=80=99t. You can test, wether your server shows the same sympt= oms > with this command: >=20 > # for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i > 0.5 -c 2 -W 1 10.0.0.1 | grep sendto; done >=20 > We also use 82599EB based ixgbe controllers on affected systems. >=20 > Also see these two threads on freebsd-net: >=20 > http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037967.html > http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038061.html >=20 > I have started the second one, and there are some more details of > what we were seeing in case you=E2=80=99re interested. >=20 > Then there is: >=20 > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D183390 > and: > https://bugs.freenas.org/issues/4560 >=20 Well, the "before" printf() from my patch is indicating a packet > 65535 and that will definitely result in a EFBIG. (There is no way that m_defrag(= ) can squeeze > 64K into 32 MCLBYTES mbufs.) Note that the EFBIG will be returned by the call that dequeues this packet and tries to transmit it (not necessarily the one that generated/queued the packet). This was pointed out by Ryan in a previous discussion of this. The code snippet from sys/netinet/tcp_output.c looks pretty straightforward= : /* 772 =09* Limit a burst to t_tsomax minus IP, 773 =09* TCP and options length to keep ip->ip_len 774 =09* from overflowing or exceeding the maximum 775 =09* length allowed by the network interface. 776 =09*/ 777 =09if (len > tp->t_tsomax - hdrlen) { 778 =09 len =3D tp->t_tsomax - hdrlen; 779 =09 sendalot =3D 1; 780 =09} If it is a TSO segment of > 65535, at a glance it would seem that this "if" is busted. Just to see, you could try replacing line# 777-778 with if (len > IP_MAXPACKET - hdrlen) { len =3D IP_MAXPACKET - hdrlen; which was what it was in 9.1. (Maybe t_tsomax isn't set correctly or someho= w screws up the calculation? rick >=20 > Markus > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:47:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C6F847D for ; Fri, 21 Mar 2014 02:47:45 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B4669EF for ; Fri, 21 Mar 2014 02:47:45 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so2159824qcr.11 for ; Thu, 20 Mar 2014 19:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Scma7ukSPD53COwzAmXmTA23/pRgd1PDg9805tca3rg=; b=XJUwWDTZ61JWf1Bna8vEKShVlwJUmh+R0oSte6GlCQ48D6Eoofr89tuD/7IZUnvET7 3BkgofuvixLvMoIPLkOGnvUO/Jn0Z0h0VPeTZI1NQu4smj13e+/tmgeaiDMIafvKQB8M w5vBBfUI9ZAuvJafUfInVpUfbRHRregDf8pXO4k2ds7nXAAc6nBWeez0xR+C5nZhS3gY 2q4JReKwjjJa6gBFp6ekpz3OO/D18xnKQNlx1ZzIOt55N11pkdeOT1eUWcsMI7FBuWng qzSA4DepD0LfLwiFUiIY7GFWHeH5KYhoQZZYzt7eptUFSp8bVNsJ9cgM1elI+rPYHsN9 sCig== MIME-Version: 1.0 X-Received: by 10.224.57.81 with SMTP id b17mr55221730qah.44.1395370064385; Thu, 20 Mar 2014 19:47:44 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 19:47:44 -0700 (PDT) In-Reply-To: References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 23:47:44 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:47:45 -0000 BTW - I think this will end up being a TSO issue, not the patch that Jack applied. When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m shows: 21489/2886/24375 mbufs in use (current/cache/total) 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) 4080/587 mbuf+clusters out of packet secondary zone in use (current/cache) 16384/50/16434/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 79068K/2173K/81241K bytes allocated to network (current/cache/total) 18831/545/4542 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile Here is an un-patched boot: 21550/7400/28950 mbufs in use (current/cache/total) 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) 4080/2769 mbuf+clusters out of packet secondary zone in use (current/cache) 0/42/42/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16439/129/16568/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 161498K/10699K/172197K bytes allocated to network (current/cache/total) 18345/155/4099 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile See how removing the MJUM9BYTES is just pushing the problem from the 9k jumbo cluster into the 4k jumbo cluster? Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact same hardware, revisions, zpool size, etc. Just it's running an older FreeBSD. # uname -a FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 15:12:14 AST 2013 aatech@FreeBSD-Update Server:/usr/obj/usr/src/sys/GENERIC amd64 root@SAN1:/san1 # uptime 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 root@SAN1:/san1 # netstat -m 37930/15755/53685 mbufs in use (current/cache/total) 4080/10996/15076/524288 mbuf clusters in use (current/cache/total/max) 4080/5775 mbuf+clusters out of packet secondary zone in use (current/cache) 0/692/692/262144 4k (page size) jumbo clusters in use (current/cache/total/max) 32773/4257/37030/96000 9k jumbo clusters in use (current/cache/total/max) 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) 312599K/67011K/379611K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Lastly, please note this link: http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html It's so old that I assume the TSO leak that he speaks of has been patched, but perhaps not. More things to look into tomorrow. On Thu, Mar 20, 2014 at 11:32 PM, Christopher Forgeron wrote: > Yes, there is something broken in TSO for sure, as disabling it allows me > to run without error. It is possible that the drop in performance is > allowing me to stay under a critical threshold for the problem, but I'd > feel happier testing to make sure. > > I understand what you're asking for in the patch, I'll make the edits > tomorrow and recompile a test kernel and see. > > Right now I'm running tests on the ixgbe that Jack sent. Even if his patch > fixes the issue, I wonder if something else isn't broken in TSO, as the > ixgbe code has had these lines for a long time, and it's only on this 10.0 > build that I have issues. > > I'll be following up tomorrow with info on either outcome. > > Thanks for your help.. your rusty networking is still better than mine. :-) > > > On Thu, Mar 20, 2014 at 11:13 PM, Rick Macklem wrote: > >> Christopher Forgeron wrote: >> > >> > Output from the patch you gave me (I have screens of it.. let me know >> > what you're hoping to see. >> > >> > >> > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 >> > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 >> Hmm. I think this means that the loop that generates TSO segments in >> tcp_output() is broken, since I'm pretty sure that the maximum size >> should be is IP_MAXPACKET (65535). >> >> Either that or some non-TCP socket is trying to send a packet that >> exceeds IP_MAXPACKET for some reason. >> >> Would it be possible to add a printf() for m->m_pkthdr.csum_flags >> to the before case, in the "if" that generates the before printf? >> I didn't think to put this in, but CSUM_TSO will be set if it >> is a TSO segment, I think? My networking is very rusty. >> (If how to add this isn't obvious, just email and I'll update >> the patch.) >> >> Thanks for doing this, rick >> >> > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:48:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07A4E507 for ; Fri, 21 Mar 2014 02:48:00 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id C0EC79F3 for ; Fri, 21 Mar 2014 02:47:59 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAD2nK1ODaFve/2dsb2JhbABZg0FXgwe4FIZjUYErdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1AIDa0XomAXgSmMWhACARs0B4JvgUkElXKECZB/g0khMYE9 X-IronPort-AV: E=Sophos;i="4.97,700,1389762000"; d="scan'208";a="107530109" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 20 Mar 2014 22:47:58 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 4297BB3F18; Thu, 20 Mar 2014 22:47:58 -0400 (EDT) Date: Thu, 20 Mar 2014 22:47:58 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <729766714.648507.1395370078262.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:48:00 -0000 Christopher Forgeron wrote: > Yes, there is something broken in TSO for sure, as disabling it > allows me > to run without error. It is possible that the drop in performance is > allowing me to stay under a critical threshold for the problem, but > I'd > feel happier testing to make sure. > > I understand what you're asking for in the patch, I'll make the edits > tomorrow and recompile a test kernel and see. > I also suggested a small change (basically reverting it to the 9.1 code) for tcp_output() in sys/netinet/tcp_output.c (around line# 777-778). You might as well throw that in at the same time. Thanks for all your work with this (and this applies to others that have been working on this as well.) rick > Right now I'm running tests on the ixgbe that Jack sent. Even if his > patch > fixes the issue, I wonder if something else isn't broken in TSO, as > the > ixgbe code has had these lines for a long time, and it's only on this > 10.0 > build that I have issues. > > I'll be following up tomorrow with info on either outcome. > > Thanks for your help.. your rusty networking is still better than > mine. :-) > > > On Thu, Mar 20, 2014 at 11:13 PM, Rick Macklem > wrote: > > > Christopher Forgeron wrote: > > > > > > Output from the patch you gave me (I have screens of it.. let me > > > know > > > what you're hoping to see. > > > > > > > > > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 > > > actl=65538 > > > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > > Hmm. I think this means that the loop that generates TSO segments > > in > > tcp_output() is broken, since I'm pretty sure that the maximum size > > should be is IP_MAXPACKET (65535). > > > > Either that or some non-TCP socket is trying to send a packet that > > exceeds IP_MAXPACKET for some reason. > > > > Would it be possible to add a printf() for m->m_pkthdr.csum_flags > > to the before case, in the "if" that generates the before printf? > > I didn't think to put this in, but CSUM_TSO will be set if it > > is a TSO segment, I think? My networking is very rusty. > > (If how to add this isn't obvious, just email and I'll update > > the patch.) > > > > Thanks for doing this, rick > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:51:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF290633 for ; Fri, 21 Mar 2014 02:51:45 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9EF83A7E for ; Fri, 21 Mar 2014 02:51:45 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so2095735qcz.5 for ; Thu, 20 Mar 2014 19:51:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ic5L5mtGf60+Mj7iT9+ey3Kzpy2xRvN9lEJ/Kja36VY=; b=D+gLz44tgynU9zKgrRpxO9l/77+7ytQv2nbh0oDjppWjpV/F61qJ82L/OEqKHwSoa1 O7ONq0BRkCngkrB0wuNmYEABFgt3aPQ5RtLuo7i3tl1ZfFjp4TmAknv40qsdOyIaYS5Q MKFteVyRTSuz+m51Pzc9d2jZAY4DjqhuCIhRpEwQ7YSdhnwcgbsvvkCh7oFYTxUIpsMB 2BRWKy7tgUcP5nuhBNr3ECqsG1JkzxcBHPvpe2TocLSUsCLcI8X7GXkmu31heO1inDzb wLp+Fi4NZ7l0DP0pgB6exI8K/Luvxv/JlJA+CPfwJjkcrpPCEFQmw0jkR12xHq5VmxPp voWQ== MIME-Version: 1.0 X-Received: by 10.140.100.129 with SMTP id s1mr52028504qge.43.1395370304863; Thu, 20 Mar 2014 19:51:44 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 19:51:44 -0700 (PDT) In-Reply-To: <729766714.648507.1395370078262.JavaMail.root@uoguelph.ca> References: <729766714.648507.1395370078262.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 23:51:44 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:51:45 -0000 Sorry Rick, what's the small change you wanted in sys/netinet/tcp_output.c at 777-778? I see it's calc'ing length... or did you want me to take the whole file back to 9.1-RELEASE ? On Thu, Mar 20, 2014 at 11:47 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > Yes, there is something broken in TSO for sure, as disabling it > > allows me > > to run without error. It is possible that the drop in performance is > > allowing me to stay under a critical threshold for the problem, but > > I'd > > feel happier testing to make sure. > > > > I understand what you're asking for in the patch, I'll make the edits > > tomorrow and recompile a test kernel and see. > > > I also suggested a small change (basically reverting it to the 9.1 code) > for tcp_output() in sys/netinet/tcp_output.c (around line# 777-778). > You might as well throw that in at the same time. > > Thanks for all your work with this (and this applies to others that > have been working on this as well.) > > rick > > > Right now I'm running tests on the ixgbe that Jack sent. Even if his > > patch > > fixes the issue, I wonder if something else isn't broken in TSO, as > > the > > ixgbe code has had these lines for a long time, and it's only on this > > 10.0 > > build that I have issues. > > > > I'll be following up tomorrow with info on either outcome. > > > > Thanks for your help.. your rusty networking is still better than > > mine. :-) > > > > > > On Thu, Mar 20, 2014 at 11:13 PM, Rick Macklem > > wrote: > > > > > Christopher Forgeron wrote: > > > > > > > > Output from the patch you gave me (I have screens of it.. let me > > > > know > > > > what you're hoping to see. > > > > > > > > > > > > Mar 20 16:37:22 SAN0 kernel: after mbcnt=33 pklen=65538 > > > > actl=65538 > > > > Mar 20 16:37:22 SAN0 kernel: before pklen=65538 actl=65538 > > > Hmm. I think this means that the loop that generates TSO segments > > > in > > > tcp_output() is broken, since I'm pretty sure that the maximum size > > > should be is IP_MAXPACKET (65535). > > > > > > Either that or some non-TCP socket is trying to send a packet that > > > exceeds IP_MAXPACKET for some reason. > > > > > > Would it be possible to add a printf() for m->m_pkthdr.csum_flags > > > to the before case, in the "if" that generates the before printf? > > > I didn't think to put this in, but CSUM_TSO will be set if it > > > is a TSO segment, I think? My networking is very rusty. > > > (If how to add this isn't obvious, just email and I'll update > > > the patch.) > > > > > > Thanks for doing this, rick > > > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 02:53:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCAED6C4 for ; Fri, 21 Mar 2014 02:53:22 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9C3F5A89 for ; Fri, 21 Mar 2014 02:53:22 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id q108so5336618qgd.9 for ; Thu, 20 Mar 2014 19:53:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=KEt9EKXcivcVqZztNp3gVgOiea8MD78yCexQ0qitf6s=; b=e93EqhEGKJOfEeQHMbvkp+tMnfH/0aH9H7J5/NFxt7qBnZRU6kdog30FtWk/O/xP3N OIAwNALgH7qVS5O3qnaivXr1ROTy1hlT5/cQKWCGD1yBBFlbzT5TUrvUf+Z3aSwpR0WH cEAsP4MUfo4i3w7dtkfxbXO71XvoKLqIYl3/DdV8KLpe+dY6p1EaifkFywCRCNSUScEU xULqUGI6tHKKCDTRtSmBeE2VFbGgvnp2qcetYKgyUZEZglAiqIbhvCZgaTM0nhtVaJl/ lENpK5d3GG5hPevm/v2hNR1n0A6XMDzSYWTbrt8LebfqX1KMyZ73Q//UP+jAV4UJjVz+ EmGw== MIME-Version: 1.0 X-Received: by 10.229.193.136 with SMTP id du8mr54671815qcb.11.1395370401871; Thu, 20 Mar 2014 19:53:21 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 19:53:21 -0700 (PDT) In-Reply-To: References: <729766714.648507.1395370078262.JavaMail.root@uoguelph.ca> Date: Thu, 20 Mar 2014 23:53:21 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:53:22 -0000 Pardon.. delay in recv'ing messages. I see your edits for 777-778 .. will attempt tomorrow. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 10:33:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3FD5F51B; Fri, 21 Mar 2014 10:33:12 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C531E78F; Fri, 21 Mar 2014 10:33:11 +0000 (UTC) Received: from [2001:1620:2013:1:98ae:107d:2646:4979] (port=52493) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WQwl9-0009wR-7b; Fri, 21 Mar 2014 11:33:07 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network stack returning EFBIG? From: Markus Gebert In-Reply-To: <429006400.647323.1395369915529.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 11:32:27 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <429006400.647323.1395369915529.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1874) Cc: jfv@freebsd.org, freebsd-net@freebsd.org, freebsd-stable@freebsd.org, wollman@bimajority.org, Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 10:33:12 -0000 On 21.03.2014, at 03:45, Rick Macklem wrote: > Markus Gebert wrote: >>=20 >> On 20.03.2014, at 14:51, wollman@bimajority.org wrote: >>=20 >>> In article <21290.60558.750106.630804@hergotha.csail.mit.edu>, I >>> wrote: >>>=20 >>>> Since we put this server into production, random network system >>>> calls >>>> have started failing with [EFBIG] or maybe sometimes [EIO]. I've >>>> observed this with a simple ping, but various daemons also log the >>>> errors: >>>> Mar 20 09:22:04 nfs-prod-4 sshd[42487]: fatal: Write failed: File >>>> too >>>> large [preauth] >>>> Mar 20 09:23:44 nfs-prod-4 nrpe[42492]: Error: Could not complete >>>> SSL >>>> handshake. 5 >>>=20 >>> I found at least one call stack where this happens and it does get >>> returned all the way to userspace: >>>=20 >>> 17 15547 _bus_dmamap_load_buffer:return >>> kernel`_bus_dmamap_load_mbuf_sg+0x5f >>> kernel`bus_dmamap_load_mbuf_sg+0x38 >>> kernel`ixgbe_xmit+0xcf >>> kernel`ixgbe_mq_start_locked+0x94 >>> kernel`ixgbe_mq_start+0x12a >>> if_lagg.ko`lagg_transmit+0xc4 >>> kernel`ether_output_frame+0x33 >>> kernel`ether_output+0x4fe >>> kernel`ip_output+0xd74 >>> kernel`tcp_output+0xfea >>> kernel`tcp_usr_send+0x325 >>> kernel`sosend_generic+0x3f6 >>> kernel`soo_write+0x5e >>> kernel`dofilewrite+0x85 >>> kernel`kern_writev+0x6c >>> kernel`sys_write+0x64 >>> kernel`amd64_syscall+0x5ea >>> kernel`0xffffffff808443c7 >>=20 >> This looks pretty similar to what we=92ve seen when we got EFBIG: >>=20 >> 3 28502 _bus_dmamap_load_buffer:return >> kernel`_bus_dmamap_load_mbuf_sg+0x5f >> kernel`bus_dmamap_load_mbuf_sg+0x38 >> kernel`ixgbe_xmit+0xcf >> kernel`ixgbe_mq_start_locked+0x94 >> kernel`ixgbe_mq_start+0x12a >> kernel`ether_output_frame+0x33 >> kernel`ether_output+0x4fe >> kernel`ip_output+0xd74 >> kernel`rip_output+0x229 >> kernel`sosend_generic+0x3f6 >> kernel`kern_sendit+0x1a3 >> kernel`sendit+0xdc >> kernel`sys_sendto+0x4d >> kernel`amd64_syscall+0x5ea >> kernel`0xffffffff80d35667 >>=20 >> In our case it looks like some of the ixgbe tx queues get stuck, and >> some don=92t. You can test, wether your server shows the same = symptoms >> with this command: >>=20 >> # for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i >> 0.5 -c 2 -W 1 10.0.0.1 | grep sendto; done >>=20 >> We also use 82599EB based ixgbe controllers on affected systems. >>=20 >> Also see these two threads on freebsd-net: >>=20 >> = http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037967.html >> http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038061.html >>=20 >> I have started the second one, and there are some more details of >> what we were seeing in case you=92re interested. >>=20 >> Then there is: >>=20 >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D183390 >> and: >> https://bugs.freenas.org/issues/4560 >>=20 > Well, the "before" printf() from my patch is indicating a packet > = 65535 > and that will definitely result in a EFBIG. (There is no way that = m_defrag() > can squeeze > 64K into 32 MCLBYTES mbufs.) Makes sense. > Note that the EFBIG will be returned by the call that dequeues this = packet > and tries to transmit it (not necessarily the one that = generated/queued the > packet). This was pointed out by Ryan in a previous discussion of = this. I remember that email, and it also explains why a ping could fail when = it happens to be on the same queue. On the other hand, would it explain = why every single ping on certain queues starts to fail, while other = queues are unaffected? Of course it could be that whatever triggers the = problem, resends the huge segment immediately over the same TCP = connection, and blocks one queue for some time by repeating this over = and over quickly enough to kill every single ping packet. However this = sounds unlikely to me. And once we saw the problem, I umounted all NFS = shares and therefore eliminated all sources of huge packets, and the = problem persisted. So, in my opinion, there must be more to it than just = a packet too big once in a while. > The code snippet from sys/netinet/tcp_output.c looks pretty = straightforward: > /* > 772 * Limit a burst to t_tsomax minus IP, > 773 * TCP and options length to keep ip->ip_len > 774 * from overflowing or exceeding the maximum > 775 * length allowed by the network interface. > 776 */ > 777 if (len > tp->t_tsomax - hdrlen) { > 778 len =3D tp->t_tsomax - hdrlen; > 779 sendalot =3D 1; > 780 } > If it is a TSO segment of > 65535, at a glance it would seem that this = "if" > is busted. Just to see, you could try replacing line# 777-778 with > if (len > IP_MAXPACKET - hdrlen) { > len =3D IP_MAXPACKET - hdrlen; > which was what it was in 9.1. (Maybe t_tsomax isn't set correctly or = somehow > screws up the calculation? I cannot answer your question, but this is an interesting catch. I=92ll = get this and your printfs in our 9.2 kernel as soon as I can. Markus > rick >=20 >>=20 >> Markus >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 11:47:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1367C9A for ; Fri, 21 Mar 2014 11:47:36 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7DBFADB1 for ; Fri, 21 Mar 2014 11:47:36 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id m20so2571688qcx.35 for ; Fri, 21 Mar 2014 04:47:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GjB1ootpns5FqHPLd+EuzF8oDBtbnO+5tvNOK0xtAXc=; b=TYlnfqJLUq42EXp0m8i572FO0V400HE/tcZ/ih8AVvE2PFSCp9OkGL+OhqwlVRJDOr A9Dw0OP76jv8pZcqfheuyw24ydM8eVCMrGw+2B4ALyjRFkFJ6BaEAaEmcXncUQ29B5QP nPclY6+x9zF2/B2QhAGtMvZrFBr2kkDebB5oaEUuftAOU2aivSqsDQAXX9ATog6O7zZp 4xayL6BQ52vIR8KLUwFhIOQhigf/rRYPwhPDtukWy5aczR1E8Y9KHkdvfFiqAmnPmpq0 llCAXN+XNjvgLlXjmZqAvdttDYd70UtUXkucGoCo6A+NeCfQB2Ojxdl2faMdgdafAdVP DbvA== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr39684401qga.57.1395402455764; Fri, 21 Mar 2014 04:47:35 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 04:47:35 -0700 (PDT) In-Reply-To: References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 08:47:35 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem , Jack Vogel , Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 11:47:36 -0000 Hello all, I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer away at the NFS store overnight - But the problem is still there. >From what I read, I think the MJUM9BYTES removal is probably good cleanup (as long as it doesn't trade performance on a lightly memory loaded system for performance on a heavily memory loaded system). If I can stabilize my system, I may attempt those benchmarks. I think the fix will be obvious at boot for me - My 9.2 has a 'clean' netstat - Until I can boot and see a 'netstat -m' that looks similar to that, I'm going to have this problem. Markus: Do your systems show denied mbufs at boot like mine does? Turning off TSO works for me, but at a performance hit. I'll compile Rick's patch (and extra debugging) this morning and let you know soon. On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron wrote: > BTW - I think this will end up being a TSO issue, not the patch that Jack > applied. > > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m > shows: > > 21489/2886/24375 mbufs in use (current/cache/total) > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) > 4080/587 mbuf+clusters out of packet secondary zone in use (current/cache) > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 79068K/2173K/81241K bytes allocated to network (current/cache/total) > 18831/545/4542 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > Here is an un-patched boot: > > 21550/7400/28950 mbufs in use (current/cache/total) > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2769 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/42/42/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16439/129/16568/907741 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161498K/10699K/172197K bytes allocated to network (current/cache/total) > 18345/155/4099 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > > See how removing the MJUM9BYTES is just pushing the problem from the 9k > jumbo cluster into the 4k jumbo cluster? > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact same > hardware, revisions, zpool size, etc. Just it's running an older FreeBSD. > > # uname -a > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 15:12:14 > AST 2013 aatech@FreeBSD-Update Server:/usr/obj/usr/src/sys/GENERIC > amd64 > > root@SAN1:/san1 # uptime > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 > > root@SAN1:/san1 # netstat -m > 37930/15755/53685 mbufs in use (current/cache/total) > 4080/10996/15076/524288 mbuf clusters in use (current/cache/total/max) > 4080/5775 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/692/692/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > 32773/4257/37030/96000 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > 312599K/67011K/379611K bytes allocated to network (current/cache/total) > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > Lastly, please note this link: > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > It's so old that I assume the TSO leak that he speaks of has been patched, > but perhaps not. More things to look into tomorrow. > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 13:04:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95A3EE04 for ; Fri, 21 Mar 2014 13:04:53 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23EFC7E4 for ; Fri, 21 Mar 2014 13:04:52 +0000 (UTC) Received: from [2001:1620:2013:1:98ae:107d:2646:4979] (port=56494) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WQz7w-000H6F-MU; Fri, 21 Mar 2014 14:04:48 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Fri, 21 Mar 2014 14:04:07 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 13:04:53 -0000 On 21.03.2014, at 12:47, Christopher Forgeron = wrote: > Hello all, >=20 > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer = away > at the NFS store overnight - But the problem is still there. >=20 > =46rom what I read, I think the MJUM9BYTES removal is probably good = cleanup > (as long as it doesn't trade performance on a lightly memory loaded = system > for performance on a heavily memory loaded system). If I can stabilize = my > system, I may attempt those benchmarks. >=20 > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' > netstat > - Until I can boot and see a 'netstat -m' that looks similar to that, = I'm > going to have this problem. >=20 > Markus: Do your systems show denied mbufs at boot like mine does? No. Our systems never show denied mbufs. Not on boot, not during normal = operations and also not when the problem is occuring. I don=92t know = what you do differently, but in our case neither 4k nor 9k mbufs get = used, only the normal ones. I=92m beginning to think that we look at different problems and at least = quite different symptoms of a similar problem. Have you had luck in = trying to find out, where EFBIG originates from in your case? Markus > Turning off TSO works for me, but at a performance hit. >=20 > I'll compile Rick's patch (and extra debugging) this morning and let = you > know soon. >=20 >=20 >=20 >=20 > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron = > wrote: >=20 >> BTW - I think this will end up being a TSO issue, not the patch that = Jack >> applied. >>=20 >> When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m >> shows: >>=20 >> 21489/2886/24375 mbufs in use (current/cache/total) >> 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) >> 4080/587 mbuf+clusters out of packet secondary zone in use = (current/cache) >> 16384/50/16434/3063627 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) >>=20 >> 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) >> 79068K/2173K/81241K bytes allocated to network (current/cache/total) >> 18831/545/4542 requests for mbufs denied = (mbufs/clusters/mbuf+clusters) >>=20 >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) >>=20 >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >>=20 >> Here is an un-patched boot: >>=20 >> 21550/7400/28950 mbufs in use (current/cache/total) >> 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) >> 4080/2769 mbuf+clusters out of packet secondary zone in use = (current/cache) >> 0/42/42/3063627 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 16439/129/16568/907741 9k jumbo clusters in use = (current/cache/total/max) >>=20 >> 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) >> 161498K/10699K/172197K bytes allocated to network = (current/cache/total) >> 18345/155/4099 requests for mbufs denied = (mbufs/clusters/mbuf+clusters) >>=20 >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) >>=20 >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >>=20 >>=20 >>=20 >> See how removing the MJUM9BYTES is just pushing the problem from the = 9k >> jumbo cluster into the 4k jumbo cluster? >>=20 >> Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact = same >> hardware, revisions, zpool size, etc. Just it's running an older = FreeBSD. >>=20 >> # uname -a >> FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 = 15:12:14 >> AST 2013 aatech@FreeBSD-Update = Server:/usr/obj/usr/src/sys/GENERIC >> amd64 >>=20 >> root@SAN1:/san1 # uptime >> 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 >>=20 >> root@SAN1:/san1 # netstat -m >> 37930/15755/53685 mbufs in use (current/cache/total) >> 4080/10996/15076/524288 mbuf clusters in use = (current/cache/total/max) >> 4080/5775 mbuf+clusters out of packet secondary zone in use = (current/cache) >> 0/692/692/262144 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 32773/4257/37030/96000 9k jumbo clusters in use = (current/cache/total/max) >>=20 >> 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) >> 312599K/67011K/379611K bytes allocated to network = (current/cache/total) >>=20 >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0/0/0 sfbufs in use (current/peak/max) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> 0 calls to protocol drain routines >>=20 >> Lastly, please note this link: >>=20 >> = http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html >>=20 >> It's so old that I assume the TSO leak that he speaks of has been = patched, >> but perhaps not. More things to look into tomorrow. >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 13:16:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DFD89195 for ; Fri, 21 Mar 2014 13:16:49 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 90FE790F for ; Fri, 21 Mar 2014 13:16:49 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so2685828qcz.2 for ; Fri, 21 Mar 2014 06:16:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wJhHShv/Ex2JzBgqgxOqLjVCthtp1B6eB6GcuAEj2h8=; b=UTu1dV1mzpbyABZpIHdLp0mnRAxl+c2c2hxJmB14KP1yCCsCQX6UmAOzCpi558dJmM OdlTh0Ej0k6LgsNlzIrV3zCYBV9VCe3qZl+UqeC6fPFiND/YG5l9bE2mpXPT7DWvwEQl aLolf9elszFBc8ldJ5Ii2B1ooy2sDg6cNlrA5MZN3p2hh4JAV4Q786IiiNP8rNVHO1la +C/7pXBfu/41gnUCPeDZqIFsHVbSSxhtdId8aLsHFeOjaJL4R8Wj97Kx2t0nWs/XYzjf o48qL++34mI21O7RBBB6jtHohQHG5x7bZD2tkjWQpfAisdcHhYgPxUsQ/tnd2r0Rdkib SBzg== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr9742434qab.93.1395407807898; Fri, 21 Mar 2014 06:16:47 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 06:16:47 -0700 (PDT) In-Reply-To: References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 10:16:47 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 13:16:49 -0000 Hi Markus, Yes, we may have different problems, or perhaps the same problem is manifesting itself in different ways in our systems. Have you tried a 10.0-RELEASE system yet? If we were on the same OS version, we could then compare system specs a bit deeper, and see what is different. Perhaps under 10.0 your symptoms would be closer to mine, which may not be progress, but would be something. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 14:22:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17278361 for ; Fri, 21 Mar 2014 14:22:20 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C4488FD8 for ; Fri, 21 Mar 2014 14:22:19 +0000 (UTC) Received: from [2001:1620:2013:1:98ae:107d:2646:4979] (port=58718) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WR0Ku-000LJD-9k; Fri, 21 Mar 2014 15:22:16 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Fri, 21 Mar 2014 15:21:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:22:20 -0000 On 21.03.2014, at 14:16, Christopher Forgeron = wrote: > Hi Markus, >=20 > Yes, we may have different problems, or perhaps the same problem is = manifesting itself in different ways in our systems.=20 >=20 > Have you tried a 10.0-RELEASE system yet? If we were on the same OS = version, we could then compare system specs a bit deeper, and see what = is different. Perhaps under 10.0 your symptoms would be closer to mine, = which may not be progress, but would be something.=20 I=92m afraid, we can=92t. We can only reproduce this on production = systems. And they cannot be easily upgraded right now. For one, we don=92t= have our build infrastructure ready for 10.x, because we usually skip = the X.0 release, which means no poudriere/pkg repo with locally patched = packages I need for these systems go into production. Then there=92s = testing, configuration changes needed, etc. It will probably be a year = until we can consider upgrading anything productive to FreeBSD 10.x. I could setup a test system on identical spare hardware, but as we = cannot trigger the problem without production load so far, there=92s no = point. Markus From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 14:48:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 504FD240 for ; Fri, 21 Mar 2014 14:48:41 +0000 (UTC) Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46]) by mx1.freebsd.org (Postfix) with SMTP id 23A6230B for ; Fri, 21 Mar 2014 14:48:40 +0000 (UTC) Received: (qmail 52764 invoked by uid 1001); 21 Mar 2014 14:48:33 -0000 Delivered-To: qmda-intercept-freebsd-net@freebsd.org DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; b=1x1XyopHoh+9agItEx2jCAd1cil/IT8j8zeO8ZdBcgiIwfyHBCrf5an2xKnfQNad; Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys DomainKey-Trace-MD: h=10; b=67; l=C18R71D32M65F38T27S66M17C58C27; Comments: QMDA 0.3 Received: (qmail 52747 invoked by uid 1001); 21 Mar 2014 14:48:32 -0000 Date: 21 Mar 2014 14:48:32 +0000 Message-ID: <20140321144832.52746.qmail@f5-external.bushwire.net> From: "Mark Delany" To: freebsd-net@freebsd.org Subject: Patch: Should netmap prototypes use const where possible? MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="5vNYLRcllDrimb99" Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:48:41 -0000 --5vNYLRcllDrimb99 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Subject line says it all. I don't know what the convention is, but I presume everything should be declared const whenever possible, thus the appended patch. Mark. --5vNYLRcllDrimb99 Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="diff.txt" *** /usr/include/net/netmap_user.h Sun Mar 16 12:01:36 2014 --- /tmp/./netmap_user.h Fri Mar 21 07:39:16 2014 *************** *** 97,103 **** static inline uint32_t ! nm_ring_next(struct netmap_ring *r, uint32_t i) { return ( unlikely(i + 1 == r->num_slots) ? 0 : i + 1); } --- 97,103 ---- static inline uint32_t ! nm_ring_next(const struct netmap_ring *r, uint32_t i) { return ( unlikely(i + 1 == r->num_slots) ? 0 : i + 1); } *************** *** 108,121 **** * When everything is complete ring->head = ring->tail + 1 (modulo ring size) */ static inline int ! nm_tx_pending(struct netmap_ring *r) { return nm_ring_next(r, r->tail) != r->head; } static inline uint32_t ! nm_ring_space(struct netmap_ring *ring) { int ret = ring->tail - ring->cur; if (ret < 0) --- 108,121 ---- * When everything is complete ring->head = ring->tail + 1 (modulo ring size) */ static inline int ! nm_tx_pending(const struct netmap_ring *r) { return nm_ring_next(r, r->tail) != r->head; } static inline uint32_t ! nm_ring_space(const struct netmap_ring *ring) { int ret = ring->tail - ring->cur; if (ret < 0) --5vNYLRcllDrimb99-- From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 14:49:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9759308 for ; Fri, 21 Mar 2014 14:49:27 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 91F7E321 for ; Fri, 21 Mar 2014 14:49:27 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so2861756qcz.2 for ; Fri, 21 Mar 2014 07:49:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MLTtMz0XUfN/qwFOcdPKgHeBnClAncTGQm0MNLRiK1o=; b=gZYfvH7xNdlUjFhGjoAVLXbN4eywUYSVnXD2bXzjsDRvrWWObWyjv6wJH3qgpvFLC/ mTl00yE/qvakQiK5bVhzknW7IRD4SARtqKemE2uqQVj2Dw06p9dIG/DZw1o6nOdpIrA9 02zEpR0kA4/cIAVqsrNMUEmDl0wzgJ95p4AWvZ7PWv06WRP7wgCP1gJwdDDHGbq+xXZ+ vjt/7sHQcK+sW3ucnZ79SkV5tD2mSq5q/B8KTPHpLy5x9RIUrkG3ItIVvWLTcgSY5Phr 417LDjx0Gzp8PeIUaoYy/mBiBOCDdK4RUralhHM577BN2rwLBcHE1J7nV1a3IWEPRKVQ gtMw== MIME-Version: 1.0 X-Received: by 10.140.48.199 with SMTP id o65mr19822675qga.16.1395413366882; Fri, 21 Mar 2014 07:49:26 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 07:49:26 -0700 (PDT) In-Reply-To: <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> Date: Fri, 21 Mar 2014 11:49:26 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:49:27 -0000 Ah, I understand the difficulties of testing production systems. However, if you can make a spare tester of the same hardware, that's perfect - And you can generate all the load you need with benchmark software like iometer, large NFS copies, or perhaps a small replica of your network. Synthetic load is easier to control, and thus easier to reproduce the issue and speed testing. Heck, you may be able to do it all by looping through your two ix adapters and never using an external client. It's a bit of a pain to setup, but it's worth the effort imo. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 14:54:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 338C04AA for ; Fri, 21 Mar 2014 14:54:38 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E44E25EA for ; Fri, 21 Mar 2014 14:54:37 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so2550513qaj.11 for ; Fri, 21 Mar 2014 07:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+APTIXkpM1feiAV010YT1ZjNNbTNbk8ILFjA7F8UIAQ=; b=IdF2dc+PTT/Ib+PBnn3bMoe1cUNIJP0nAXlJk29xguJANfkYK2EVXFfzPAPZfykF4T ntO1+3UCAwiAUkRNmq3Zun3sOKDbhszdJ3AiLzlBKf4JwH2cRXwMWsCz+WILq+8S3AYx kgzisfK2TpjPfnXDtpyXg2TIgR5cbugmYbtq9anTeikRzEAXupB42mqd8T/xm7kM9dt7 Qfi4FMbbz+KTsGk/kHg9tRezG95aVCNqprbUyr43LnBNIpmubeQihyMKvrWyLum9YDTZ Ys/luqYDKFCv6nN3MGtsM5QL/n25uieMz6/Yyfq80944x9ELjMSpQCxSv7AKo2/8Shub OlCQ== MIME-Version: 1.0 X-Received: by 10.140.48.199 with SMTP id o65mr19852141qga.16.1395413676890; Fri, 21 Mar 2014 07:54:36 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 07:54:36 -0700 (PDT) In-Reply-To: <477992488.642193.1395368751685.JavaMail.root@uoguelph.ca> References: <477992488.642193.1395368751685.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 11:54:36 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:54:38 -0000 Rick: Unfortunately your patch didn't work. I expected as much as soon as I saw my boot time 'netstat -m', but I wanted to run the tests to make sure. First, here is where I put in your additional line - Let me know if that's what you were hoping for, as I'm using mmm->m_pkthdr.csum_flags, as m doesn't exist until the call to m_defrag a few lines below. printf("before pklen=%d actl=%d csum=%lu\n", mmm->m_pkthdr.len, iii, mmm->m_pkthdr.csum_flags); With this in place, here is the first set of logs after ~ 5min of load: On Thu, Mar 20, 2014 at 11:25 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert < > > markus.gebert@hostpoint.ch > wrote: > > > > > > > > > > > > Possible. We still see this on nfsclients only, but I'm not convinced > > that nfs is the only trigger. > > > > > Since Christopher is getting a bunch of the "before" printf()s from > my patch, it indicates that a packet/TSO segment that is > 65535 bytes > in length is showing up at ixgbe_xmit(). I've asked him to add a printf() > for the m_pkthdr.csum_flags field to see if it is really a TSO segment. > > If it is a TSO segment, that indicates to me that the code in tcp_output() > that should > generate a TSO segment no greater than 65535 bytes in length is busted. > And this would imply just about any app doing large sosend()s could cause > this, I think? (NFS read replies/write requests of 64K would be one of > them.) > > rick > > > > > > > > > Just to clarify, I'm experiencing this error with NFS, but also with > > iSCSI - I turned off my NFS server in rc.conf and rebooted, and I'm > > still able to create the error. This is not just a NFS issue on my > > machine. > > > > > > > > I our case, when it happens, the problem persists for quite some time > > (minutes or hours) if we don't interact (ifconfig or reboot). > > > > > > > > The first few times that I ran into it, I had similar issues - > > Because I was keeping my system up and treating it like a temporary > > problem/issue. Worst case scenario resulted in reboots to reset the > > NIC. Then again, I find the ix's to be cranky if you ifconfig them > > too much. > > > > Now, I'm trying to find a root cause, so as soon as I start seeing > > any errors, I abort and reboot the machine to test the next theory. > > > > > > Additionally, I'm often able to create the problem with just 1 VM > > running iometer on the SAN storage. When the problem occurs, that > > connection is broken temporarily, taking network load off the SAN - > > That may improve my chances of keeping this running. > > > > > > > > > > > > > I am able to reproduce it fairly reliably within 15 min of a reboot > > > by > > > loading the server via NFS with iometer and some large NFS file > > > copies at > > > the same time. I seem to need to sustain ~2 Gbps for a few minutes. > > > > That's probably why we can't reproduce it reliably here. Although > > having 10gig cards in our blade servers, the ones affected are > > connected to a 1gig switch. > > > > > > > > > > > > It seems that it needs a lot of traffic. I have a 10 gig backbone > > between my SANs and my ESXi machines, so I can saturate quite > > quickly (just now I hit a record.. the error occurred within ~5 min > > of reboot and testing). In your case, I recommend firing up multiple > > VM's running iometer on different 1 gig connections and see if you > > can make it pop. I also often turn off ix1 to drive all traffic > > through ix0 - I've noticed it happens faster this way, but once > > again I'm not taking enough observations to make decent time > > predictions. > > > > > > > > > > > > > > Can you try this when the problem occurs? > > > > for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 > > -c 2 -W 1 10.0.0.1 | grep sendto; done > > > > It will tie ping to certain cpus to test the different tx queues of > > your ix interface. If the pings reliably fail only on some queues, > > then your problem is more likely to be the same as ours. > > > > Also, if you have dtrace available: > > > > kldload dtraceall > > dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { > > stack(); }' > > > > while you run pings over the interface affected. This will give you > > hints about where the EFBIG error comes from. > > > > > [...] > > > > > > Markus > > > > > > > > > > Will do. I'm not sure what shell the first script was written for, > > it's not working in csh, here's a re-write that does work in csh in > > case others are using the default shell: > > > > #!/bin/csh > > foreach CPU (`seq 0 23`) > > echo "CPU$CPU"; > > cpuset -l $CPU ping -i 0.2 -c 2 -W 1 10.0.0.1 | grep sendto; > > end > > > > > > Thanks for your input. I should have results to post to the list > > shortly. > > > > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 15:01:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8570275C for ; Fri, 21 Mar 2014 15:01:09 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4106A6D2 for ; Fri, 21 Mar 2014 15:01:09 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so7386975qgd.1 for ; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pDdGRsGOdTRlmNfUPp4mQDxh8LGQyj9jVSYzW4p3VrA=; b=nRnmVeVNdVc4ZolccF18hzo9N1FvY9KkpzVUi89fYSULL+VRDIrUxBjIEvd0cHHUfZ a4kgiLDLxB+LxoZ4cziD1WkaW1h8Aaw806kDpugpuZr0yUc6hyHZQJ7PGBHYcj/ztSJo 9Cj0vC2Tf1+o5ECB4M1KTbpf76VwLe38etTbs1KoJlcS221PZc9LK6S46t2+g3mHIpvl 0znF612l1EalVRcBwsrRDjdUO1yCWc4JFJyc26lnN94n5bT9Kb+hBEZhAoUkVlk/m0Ey gq14FaiGyvA9Ls/9T8gIys3h/gqBN0y8Hc2MuihfKsaOakCQRI/BTH0Ne3+LEg/9N0Uo Wp2g== MIME-Version: 1.0 X-Received: by 10.224.163.12 with SMTP id y12mr8648917qax.25.1395414068545; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) In-Reply-To: References: <477992488.642193.1395368751685.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 12:01:08 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 15:01:09 -0000 (Pardon me, for some reason my gmail is sending on my cut-n-pastes if I cr down too fast) First set of logs: Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Here's a few later on. Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:23:00 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:01 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:01 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:03 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:03 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:04 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:04 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 To be clear, I changed tp->t_tsomax to IP_MAXPACKET at ~ 777 in sys/netinet/tcp_output.c like so: if (len > IP_MAXPACKET - hdrlen) { len = IP_MAXPACKET - hdrlen; sendalot = 1; } I notice there is more that is different between 9.1 and 10 for this file: http://fxr.watson.org/fxr/diff/netinet/tcp_output.c?v=FREEBSD10;diffval=FREEBSD91;diffvar=v I'm going to attempt inserting a 9.1 tcp_output.c and see if that makes any difference. Otherwise, I wait further ideas from the list. Thanks. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 15:22:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AC6EE01 for ; Fri, 21 Mar 2014 15:22:03 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CC23591A for ; Fri, 21 Mar 2014 15:22:02 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id i50so7452052qgf.0 for ; Fri, 21 Mar 2014 08:22:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FQeETi4/16zB4drpIdQwxo6eQzQrBfOam2a4OyNUjjo=; b=q1ldr0xnFKkTXaB9CkOF+hJwoL8p+7II/k+QE5Z8njuOtwdQDDC+uR7YbW9HZucjT/ NGI2CP5NOssTW4S4X4RJfOzLECM8Y8fNZgecxJVBLmMjOHP5Oo+cvns9enW4DONkd83+ GLavAxsgiD5xe/xcUgApE782SMe2fJ1mwrVWYhAW3h32ckyNh9wOKuIebrd6ffVYVwbe Odm/sQH6TTj+Vn0N5G6IdGdMjfHiIBL70ZnOS3s/EuE3Z7oF4MHnjzxlUx4LjzHYnlHu SwL788EC0rZ4Sp6hSbpF+eeCKo7w/nup3p1qbcfwH4Jf2tWqGH+0MyAKsBpWj/L9FX1w dbUw== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr10557332qab.93.1395415321712; Fri, 21 Mar 2014 08:22:01 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 08:22:01 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Mar 2014 12:22:01 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 15:22:03 -0000 Markus, I don't know why I didn't notice this before.. I copied your cpuset ping verbatim, not realizing that I should be using 172.16.0.x as that's my network on the ix's On this tester box, 10.0.0.1 goes out a different interface, thus it never reported back any problems. Now that I've corrected that, I see I have problems on the same queues: CPU0 ping: sendto: No buffer space available ping: sendto: No buffer space available CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 ping: sendto: No buffer space available ping: sendto: No buffer space available CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15 CPU16 ping: sendto: No buffer space available ping: sendto: No buffer space available CPU17 CPU18 CPU19 CPU20 CPU21 CPU22 CPU23 I can run that three times and get the same CPU's. I'll try a reboot and see if they always fail on the same queues, tho I don't know if that would show anything. At this stage, NFS connections coming into the box are down, but I can still ping out. Incoming pings show 'host is down' Here is the dump of ix0 's sysctls (only ix0 is in use on this machine for testing) dev.ix.0.queue0.interrupt_rate: 500000 dev.ix.0.queue0.irqs: 100179 dev.ix.0.queue0.txd_head: 0 dev.ix.0.queue0.txd_tail: 0 dev.ix.0.queue0.tso_tx: 104156 dev.ix.0.queue0.no_tx_dma_setup: 0 dev.ix.0.queue0.no_desc_avail: 5 dev.ix.0.queue0.tx_packets: 279480 dev.ix.0.queue0.rxd_head: 513 dev.ix.0.queue0.rxd_tail: 512 dev.ix.0.queue0.rx_packets: 774424 dev.ix.0.queue0.rx_bytes: 281916 dev.ix.0.queue0.rx_copies: 4609 dev.ix.0.queue0.lro_queued: 0 dev.ix.0.queue0.lro_flushed: 0 dev.ix.0.queue1.interrupt_rate: 71428 dev.ix.0.queue1.irqs: 540682 dev.ix.0.queue1.txd_head: 1295 dev.ix.0.queue1.txd_tail: 1295 dev.ix.0.queue1.tso_tx: 15 dev.ix.0.queue1.no_tx_dma_setup: 0 dev.ix.0.queue1.no_desc_avail: 0 dev.ix.0.queue1.tx_packets: 93248 dev.ix.0.queue1.rxd_head: 0 dev.ix.0.queue1.rxd_tail: 2047 dev.ix.0.queue1.rx_packets: 462225 dev.ix.0.queue1.rx_bytes: 0 dev.ix.0.queue1.rx_copies: 0 dev.ix.0.queue1.lro_queued: 0 dev.ix.0.queue1.lro_flushed: 0 dev.ix.0.queue2.interrupt_rate: 71428 dev.ix.0.queue2.irqs: 282801 dev.ix.0.queue2.txd_head: 367 dev.ix.0.queue2.txd_tail: 367 dev.ix.0.queue2.tso_tx: 312757 dev.ix.0.queue2.no_tx_dma_setup: 0 dev.ix.0.queue2.no_desc_avail: 0 dev.ix.0.queue2.tx_packets: 876533 dev.ix.0.queue2.rxd_head: 0 dev.ix.0.queue2.rxd_tail: 2047 dev.ix.0.queue2.rx_packets: 2324954 dev.ix.0.queue2.rx_bytes: 0 dev.ix.0.queue2.rx_copies: 0 dev.ix.0.queue2.lro_queued: 0 dev.ix.0.queue2.lro_flushed: 0 dev.ix.0.queue3.interrupt_rate: 71428 dev.ix.0.queue3.irqs: 1424108 dev.ix.0.queue3.txd_head: 499 dev.ix.0.queue3.txd_tail: 499 dev.ix.0.queue3.tso_tx: 1263116 dev.ix.0.queue3.no_tx_dma_setup: 0 dev.ix.0.queue3.no_desc_avail: 0 dev.ix.0.queue3.tx_packets: 1590798 dev.ix.0.queue3.rxd_head: 0 dev.ix.0.queue3.rxd_tail: 2047 dev.ix.0.queue3.rx_packets: 8319143 dev.ix.0.queue3.rx_bytes: 0 dev.ix.0.queue3.rx_copies: 0 dev.ix.0.queue3.lro_queued: 0 dev.ix.0.queue3.lro_flushed: 0 dev.ix.0.queue4.interrupt_rate: 71428 dev.ix.0.queue4.irqs: 138019 dev.ix.0.queue4.txd_head: 1620 dev.ix.0.queue4.txd_tail: 1620 dev.ix.0.queue4.tso_tx: 29235 dev.ix.0.queue4.no_tx_dma_setup: 0 dev.ix.0.queue4.no_desc_avail: 0 dev.ix.0.queue4.tx_packets: 200853 dev.ix.0.queue4.rxd_head: 6 dev.ix.0.queue4.rxd_tail: 5 dev.ix.0.queue4.rx_packets: 218327 dev.ix.0.queue4.rx_bytes: 1527 dev.ix.0.queue4.rx_copies: 0 dev.ix.0.queue4.lro_queued: 0 dev.ix.0.queue4.lro_flushed: 0 dev.ix.0.queue5.interrupt_rate: 71428 dev.ix.0.queue5.irqs: 131367 dev.ix.0.queue5.txd_head: 330 dev.ix.0.queue5.txd_tail: 330 dev.ix.0.queue5.tso_tx: 9907 dev.ix.0.queue5.no_tx_dma_setup: 0 dev.ix.0.queue5.no_desc_avail: 0 dev.ix.0.queue5.tx_packets: 150955 dev.ix.0.queue5.rxd_head: 0 dev.ix.0.queue5.rxd_tail: 2047 dev.ix.0.queue5.rx_packets: 72814 dev.ix.0.queue5.rx_bytes: 0 dev.ix.0.queue5.rx_copies: 0 dev.ix.0.queue5.lro_queued: 0 dev.ix.0.queue5.lro_flushed: 0 dev.ix.0.queue6.interrupt_rate: 71428 dev.ix.0.queue6.irqs: 839814 dev.ix.0.queue6.txd_head: 1402 dev.ix.0.queue6.txd_tail: 1402 dev.ix.0.queue6.tso_tx: 327633 dev.ix.0.queue6.no_tx_dma_setup: 0 dev.ix.0.queue6.no_desc_avail: 0 dev.ix.0.queue6.tx_packets: 1371262 dev.ix.0.queue6.rxd_head: 0 dev.ix.0.queue6.rxd_tail: 2047 dev.ix.0.queue6.rx_packets: 2559592 dev.ix.0.queue6.rx_bytes: 0 dev.ix.0.queue6.rx_copies: 0 dev.ix.0.queue6.lro_queued: 0 dev.ix.0.queue6.lro_flushed: 0 dev.ix.0.queue7.interrupt_rate: 71428 dev.ix.0.queue7.irqs: 150693 dev.ix.0.queue7.txd_head: 1965 dev.ix.0.queue7.txd_tail: 1965 dev.ix.0.queue7.tso_tx: 248 dev.ix.0.queue7.no_tx_dma_setup: 0 dev.ix.0.queue7.no_desc_avail: 0 dev.ix.0.queue7.tx_packets: 145736 dev.ix.0.queue7.rxd_head: 0 dev.ix.0.queue7.rxd_tail: 2047 dev.ix.0.queue7.rx_packets: 19030 dev.ix.0.queue7.rx_bytes: 0 dev.ix.0.queue7.rx_copies: 0 dev.ix.0.queue7.lro_queued: 0 dev.ix.0.queue7.lro_flushed: 0 On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert wrote: > > > Can you try this when the problem occurs? > > for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 -c 2 > -W 1 10.0.0.1 | grep sendto; done > > It will tie ping to certain cpus to test the different tx queues of your > ix interface. If the pings reliably fail only on some queues, then your > problem is more likely to be the same as ours. > > Also, if you have dtrace available: > > kldload dtraceall > dtrace -n 'fbt:::return / arg1 == EFBIG && execname == "ping" / { stack(); > }' > > while you run pings over the interface affected. This will give you hints > about where the EFBIG error comes from. > > > [...] > > > Markus > > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 15:23:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65620EC7 for ; Fri, 21 Mar 2014 15:23:13 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D4CFA936 for ; Fri, 21 Mar 2014 15:23:12 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id y1so1734501lam.23 for ; Fri, 21 Mar 2014 08:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=SMWJLHbyCSgG8y+BY20MXybosbQTMPTOJfGVC0A/KoU=; b=At5jslLY/wWEKRQP+GnDBtMzW9Ys1usnHFlowd/MwR81109CW80B4xUPJecBi7IMzO 8Q3XeiEniCApKkHMgYvZqJeEz2dWf1QuzaGYfSxst0YeP3LrPHPTfqaKOi5+L3f2+WzV pqvsYFhaecog8M+rYVCX7dmuSyDIMOZ81ewXaNcJ3sgCKCKGF3LEIhtREXpVzJrGzWEh 6hier4F8hEMo8nwKO31SlvvM0lUVW+DpBv8cHX89+MimOMkCRF8JBy4yA/KbpCBX/98g eU2RE7MnEJnHqc4+9Y0zQskK2kTO2+oiyOO730CucR+xsFbHaeStqHOArdlHu4ATeEjK xOhw== MIME-Version: 1.0 X-Received: by 10.112.46.225 with SMTP id y1mr32501210lbm.12.1395415390925; Fri, 21 Mar 2014 08:23:10 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Fri, 21 Mar 2014 08:23:10 -0700 (PDT) In-Reply-To: <20140321144832.52746.qmail@f5-external.bushwire.net> References: <20140321144832.52746.qmail@f5-external.bushwire.net> Date: Fri, 21 Mar 2014 16:23:10 +0100 X-Google-Sender-Auth: vNBS6x5W1AoehEDw5WUtyGGyrqs Message-ID: Subject: Re: Patch: Should netmap prototypes use const where possible? From: Luigi Rizzo To: Mark Delany Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 15:23:13 -0000 On Fri, Mar 21, 2014 at 3:48 PM, Mark Delany wrote: > Subject line says it all. I don't know what the convention is, but I > presume everything should be declared const whenever possible, thus > the appended patch. > > yes makes sense. cheers luigi From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 16:30:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9D084B0 for ; Fri, 21 Mar 2014 16:30:58 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 680B9FC for ; Fri, 21 Mar 2014 16:30:58 +0000 (UTC) Received: from [2001:1620:2013:1:98ae:107d:2646:4979] (port=62456) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WR2LP-0001F1-Hk; Fri, 21 Mar 2014 17:30:55 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Fri, 21 Mar 2014 17:30:15 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 16:30:58 -0000 On 21.03.2014, at 16:22, Christopher Forgeron = wrote: > Markus, >=20 > I don't know why I didn't notice this before.. I copied your cpuset = ping > verbatim, not realizing that I should be using 172.16.0.x as that's my > network on the ix's >=20 > On this tester box, 10.0.0.1 goes out a different interface, thus it = never > reported back any problems. I=92m sorry, I could have mentioned that. The good news is, this makes = our two problems look very similar again. > Now that I've corrected that, I see I have problems on the same = queues: >=20 > CPU0 > ping: sendto: No buffer space available > ping: sendto: No buffer space available > CPU1 > CPU2 > CPU3 > CPU4 > CPU5 > CPU6 > CPU7 > CPU8 > ping: sendto: No buffer space available > ping: sendto: No buffer space available > CPU9 > CPU10 > CPU11 > CPU12 > CPU13 > CPU14 > CPU15 > CPU16 > ping: sendto: No buffer space available > ping: sendto: No buffer space available > CPU17 > CPU18 > CPU19 > CPU20 > CPU21 > CPU22 > CPU23 While this is not EFBIG, we=92ve seen this too. It usually starts out as = EFBIG because _bus_dmamap_load_mbuf_sg fails, and at some point turns = into ENOBUFS when the software tx queue fills up too. If there=92s no flow id, CPU cores and tx queues have a direct = relationship, which seems to be the case with ping packets. ixgbe.c 798 /* Which queue to use */ 799 if ((m->m_flags & M_FLOWID) !=3D 0) 800 i =3D m->m_pkthdr.flowid % adapter->num_queues; 801 else 802 i =3D curcpu % adapter->num_queues; In your example, queue 0 got stuck, which is the one that cpus 0, 8 and = 16 will queue their ping packets in. num_queues defaults to 8 on systems = with 8 cores or more. So it=92s actually enough to run this test for the = first 8 cpus to actually cover all tx queues. > I can run that three times and get the same CPU's. I'll try a reboot = and > see if they always fail on the same queues, tho I don't know if that = would > show anything. I=92ve seen it happen on 2 queues at the same time. And I=92ve seen it = go away after a couple of hours leaving the system idle. > At this stage, NFS connections coming into the box are down, but I can > still ping out. Incoming pings show 'host is down=92 I guess what will still work or not really depends on which queue is = affected and which flows are tied to that queue. Markus > Here is the dump of ix0 's sysctls (only ix0 is in use on this machine = for > testing) >=20 > dev.ix.0.queue0.interrupt_rate: 500000 > dev.ix.0.queue0.irqs: 100179 > dev.ix.0.queue0.txd_head: 0 > dev.ix.0.queue0.txd_tail: 0 > dev.ix.0.queue0.tso_tx: 104156 > dev.ix.0.queue0.no_tx_dma_setup: 0 > dev.ix.0.queue0.no_desc_avail: 5 > dev.ix.0.queue0.tx_packets: 279480 > dev.ix.0.queue0.rxd_head: 513 > dev.ix.0.queue0.rxd_tail: 512 > dev.ix.0.queue0.rx_packets: 774424 > dev.ix.0.queue0.rx_bytes: 281916 > dev.ix.0.queue0.rx_copies: 4609 > dev.ix.0.queue0.lro_queued: 0 > dev.ix.0.queue0.lro_flushed: 0 > dev.ix.0.queue1.interrupt_rate: 71428 > dev.ix.0.queue1.irqs: 540682 > dev.ix.0.queue1.txd_head: 1295 > dev.ix.0.queue1.txd_tail: 1295 > dev.ix.0.queue1.tso_tx: 15 > dev.ix.0.queue1.no_tx_dma_setup: 0 > dev.ix.0.queue1.no_desc_avail: 0 > dev.ix.0.queue1.tx_packets: 93248 > dev.ix.0.queue1.rxd_head: 0 > dev.ix.0.queue1.rxd_tail: 2047 > dev.ix.0.queue1.rx_packets: 462225 > dev.ix.0.queue1.rx_bytes: 0 > dev.ix.0.queue1.rx_copies: 0 > dev.ix.0.queue1.lro_queued: 0 > dev.ix.0.queue1.lro_flushed: 0 > dev.ix.0.queue2.interrupt_rate: 71428 > dev.ix.0.queue2.irqs: 282801 > dev.ix.0.queue2.txd_head: 367 > dev.ix.0.queue2.txd_tail: 367 > dev.ix.0.queue2.tso_tx: 312757 > dev.ix.0.queue2.no_tx_dma_setup: 0 > dev.ix.0.queue2.no_desc_avail: 0 > dev.ix.0.queue2.tx_packets: 876533 > dev.ix.0.queue2.rxd_head: 0 > dev.ix.0.queue2.rxd_tail: 2047 > dev.ix.0.queue2.rx_packets: 2324954 > dev.ix.0.queue2.rx_bytes: 0 > dev.ix.0.queue2.rx_copies: 0 > dev.ix.0.queue2.lro_queued: 0 > dev.ix.0.queue2.lro_flushed: 0 > dev.ix.0.queue3.interrupt_rate: 71428 > dev.ix.0.queue3.irqs: 1424108 > dev.ix.0.queue3.txd_head: 499 > dev.ix.0.queue3.txd_tail: 499 > dev.ix.0.queue3.tso_tx: 1263116 > dev.ix.0.queue3.no_tx_dma_setup: 0 > dev.ix.0.queue3.no_desc_avail: 0 > dev.ix.0.queue3.tx_packets: 1590798 > dev.ix.0.queue3.rxd_head: 0 > dev.ix.0.queue3.rxd_tail: 2047 > dev.ix.0.queue3.rx_packets: 8319143 > dev.ix.0.queue3.rx_bytes: 0 > dev.ix.0.queue3.rx_copies: 0 > dev.ix.0.queue3.lro_queued: 0 > dev.ix.0.queue3.lro_flushed: 0 > dev.ix.0.queue4.interrupt_rate: 71428 > dev.ix.0.queue4.irqs: 138019 > dev.ix.0.queue4.txd_head: 1620 > dev.ix.0.queue4.txd_tail: 1620 > dev.ix.0.queue4.tso_tx: 29235 > dev.ix.0.queue4.no_tx_dma_setup: 0 > dev.ix.0.queue4.no_desc_avail: 0 > dev.ix.0.queue4.tx_packets: 200853 > dev.ix.0.queue4.rxd_head: 6 > dev.ix.0.queue4.rxd_tail: 5 > dev.ix.0.queue4.rx_packets: 218327 > dev.ix.0.queue4.rx_bytes: 1527 > dev.ix.0.queue4.rx_copies: 0 > dev.ix.0.queue4.lro_queued: 0 > dev.ix.0.queue4.lro_flushed: 0 > dev.ix.0.queue5.interrupt_rate: 71428 > dev.ix.0.queue5.irqs: 131367 > dev.ix.0.queue5.txd_head: 330 > dev.ix.0.queue5.txd_tail: 330 > dev.ix.0.queue5.tso_tx: 9907 > dev.ix.0.queue5.no_tx_dma_setup: 0 > dev.ix.0.queue5.no_desc_avail: 0 > dev.ix.0.queue5.tx_packets: 150955 > dev.ix.0.queue5.rxd_head: 0 > dev.ix.0.queue5.rxd_tail: 2047 > dev.ix.0.queue5.rx_packets: 72814 > dev.ix.0.queue5.rx_bytes: 0 > dev.ix.0.queue5.rx_copies: 0 > dev.ix.0.queue5.lro_queued: 0 > dev.ix.0.queue5.lro_flushed: 0 > dev.ix.0.queue6.interrupt_rate: 71428 > dev.ix.0.queue6.irqs: 839814 > dev.ix.0.queue6.txd_head: 1402 > dev.ix.0.queue6.txd_tail: 1402 > dev.ix.0.queue6.tso_tx: 327633 > dev.ix.0.queue6.no_tx_dma_setup: 0 > dev.ix.0.queue6.no_desc_avail: 0 > dev.ix.0.queue6.tx_packets: 1371262 > dev.ix.0.queue6.rxd_head: 0 > dev.ix.0.queue6.rxd_tail: 2047 > dev.ix.0.queue6.rx_packets: 2559592 > dev.ix.0.queue6.rx_bytes: 0 > dev.ix.0.queue6.rx_copies: 0 > dev.ix.0.queue6.lro_queued: 0 > dev.ix.0.queue6.lro_flushed: 0 > dev.ix.0.queue7.interrupt_rate: 71428 > dev.ix.0.queue7.irqs: 150693 > dev.ix.0.queue7.txd_head: 1965 > dev.ix.0.queue7.txd_tail: 1965 > dev.ix.0.queue7.tso_tx: 248 > dev.ix.0.queue7.no_tx_dma_setup: 0 > dev.ix.0.queue7.no_desc_avail: 0 > dev.ix.0.queue7.tx_packets: 145736 > dev.ix.0.queue7.rxd_head: 0 > dev.ix.0.queue7.rxd_tail: 2047 > dev.ix.0.queue7.rx_packets: 19030 > dev.ix.0.queue7.rx_bytes: 0 > dev.ix.0.queue7.rx_copies: 0 > dev.ix.0.queue7.lro_queued: 0 > dev.ix.0.queue7.lro_flushed: 0 >=20 >=20 >=20 > On Thu, Mar 20, 2014 at 7:40 AM, Markus Gebert > wrote: >=20 >>=20 >>=20 >> Can you try this when the problem occurs? >>=20 >> for CPU in {0..7}; do echo "CPU${CPU}"; cpuset -l ${CPU} ping -i 0.2 = -c 2 >> -W 1 10.0.0.1 | grep sendto; done >>=20 >> It will tie ping to certain cpus to test the different tx queues of = your >> ix interface. If the pings reliably fail only on some queues, then = your >> problem is more likely to be the same as ours. >>=20 >> Also, if you have dtrace available: >>=20 >> kldload dtraceall >> dtrace -n 'fbt:::return / arg1 =3D=3D EFBIG && execname =3D=3D "ping" = / { stack(); >> }' >>=20 >> while you run pings over the interface affected. This will give you = hints >> about where the EFBIG error comes from. >>=20 >>> [...] >>=20 >>=20 >> Markus >>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 16:41:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F187990 for ; Fri, 21 Mar 2014 16:41:23 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 570491E8 for ; Fri, 21 Mar 2014 16:41:23 +0000 (UTC) Received: from [2001:1620:2013:1:98ae:107d:2646:4979] (port=62719) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WR2VV-0001hf-AP; Fri, 21 Mar 2014 17:41:21 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Fri, 21 Mar 2014 17:40:40 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <7B586D3A-BD6B-40B1-980E-7F9FD4A49F6A@hostpoint.ch> References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 16:41:23 -0000 On 21.03.2014, at 15:49, Christopher Forgeron = wrote: > However, if you can make a spare tester of the same hardware, that's > perfect - And you can generate all the load you need with benchmark > software like iometer, large NFS copies, or perhaps a small replica of = your > network. Synthetic load is easier to control, and thus easier to = reproduce > the issue and speed testing. Heck, you may be able to do it all by = looping > through your two ix adapters and never using an external client. >=20 > It's a bit of a pain to setup, but it's worth the effort imo. The main problem is, that all the affected systems are blades which are = only connected 1gig. I think that=92s the main reason we have trouble = reproducing the problem and I cannot change that, because we simply lack = the parts to produce any kind of 10gig connection between blades. So I = will postpone this idea, especially since our problems seem very similar = again. 9.2 or 10.0 does not seem to matter, at least for now. Markus From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 17:22:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4958F491 for ; Fri, 21 Mar 2014 17:22:19 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0197D890 for ; Fri, 21 Mar 2014 17:22:18 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id e9so3091694qcy.26 for ; Fri, 21 Mar 2014 10:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RgsYMf4VG4FYvZVclBpFU9ApjeE9ppd6gMbKdUZD8yI=; b=O8RU9v+21pcQAQDyMWpUjZ87qKQ9205gWwOTxNofc4EUWvKjo/zAKXrpfMmxdPoSxU Byvj94fbghhIL6fBFaYY2JyQMGB6qXHCl7kp2qOgenzQIK3EK9tVin6iykaGIZiK0Rmw lffIcAWME5auPbdAToao45IN2Nq6ULRM/BWtj7Edqh7uZeMlfcFPAJDbAq0F/OHc9k5x ML1el6m9O76e8LrOoZe6TeFqfaOL+PfqggDG40rmpyL4CEhb34jbo5JGtmFHlXY4UaAX ViUz4YpZvrgpwLQHVy+C7pCzRxw3yMbpTdWHPIuV+lFNiy7YRhuzexzGjifFZIY16WSf kwrg== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr41708868qga.57.1395422538237; Fri, 21 Mar 2014 10:22:18 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 10:22:18 -0700 (PDT) In-Reply-To: <7B586D3A-BD6B-40B1-980E-7F9FD4A49F6A@hostpoint.ch> References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> <7B586D3A-BD6B-40B1-980E-7F9FD4A49F6A@hostpoint.ch> Date: Fri, 21 Mar 2014 14:22:18 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 17:22:19 -0000 Fair enough. Have you tried disabling tso on the ix's ? That does fix the problem for me, however there is a performance penalty to be paid. I'm now regressing through the ixgbe drivers - I see there's been changes to how the queues are drained between 9.1 - 10.0, will see if the older ixgbe 2.4.8 works under 10.0 On Fri, Mar 21, 2014 at 1:40 PM, Markus Gebert wrote: > > On 21.03.2014, at 15:49, Christopher Forgeron > wrote: > > > However, if you can make a spare tester of the same hardware, that's > > perfect - And you can generate all the load you need with benchmark > > software like iometer, large NFS copies, or perhaps a small replica of > your > > network. Synthetic load is easier to control, and thus easier to > reproduce > > the issue and speed testing. Heck, you may be able to do it all by > looping > > through your two ix adapters and never using an external client. > > > > It's a bit of a pain to setup, but it's worth the effort imo. > > The main problem is, that all the affected systems are blades which are > only connected 1gig. I think that's the main reason we have trouble > reproducing the problem and I cannot change that, because we simply lack > the parts to produce any kind of 10gig connection between blades. So I will > postpone this idea, especially since our problems seem very similar again. > 9.2 or 10.0 does not seem to matter, at least for now. > > > Markus > > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 18:23:58 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C06C878; Fri, 21 Mar 2014 18:23:58 +0000 (UTC) Received: from melamine.cuivre.fr.eu.org (houdart.cuivre.fr.eu.org [81.57.40.110]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DE476ED7; Fri, 21 Mar 2014 18:23:57 +0000 (UTC) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id 12166161AA; Fri, 21 Mar 2014 19:16:16 +0100 (CET) Date: Fri, 21 Mar 2014 19:16:16 +0100 From: Thomas Quinot To: Jimmy Olgeni Subject: Re: ipsec packets apparently not getting to destination Message-ID: <20140321181616.GB29989@melamine.cuivre.fr.eu.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-net@FreeBSD.org, freebsd-questions@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 18:23:58 -0000 * Jimmy Olgeni, 2013-12-03 : > I cannot imagine any obvious reason for packets getting "lost" after enc0, > so any hint would be much appreciated :) Chances are "netstat -s -p udp" will show you an increasing count of packets with bad checksum. See PRs kern/145737 and kern/146190. Thomas. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 18:47:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AAAAC79D; Fri, 21 Mar 2014 18:47:25 +0000 (UTC) Received: from mail-ee0-x22c.google.com (mail-ee0-x22c.google.com [IPv6:2a00:1450:4013:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1DE9118C; Fri, 21 Mar 2014 18:47:24 +0000 (UTC) Received: by mail-ee0-f44.google.com with SMTP id e49so2125547eek.17 for ; Fri, 21 Mar 2014 11:47:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=2RLvp/1HK5/5NsgIxzzTS0H284SXDTqOGc5tknKm8vU=; b=zP0niGnhXSLzGkw6BvMT08JuDpPrmJX9qJFju/GkOp2ylfANy95PMMCxytz6emz7Ae uFLZi9vNLv763D/Az1yc88XndNnjMjDUbBi9MQTfrl1du0I5Lh3XiNxcogMNUK4rXhpW 5b48A2BsN/hcTijGsirahnnDv4jBBNAEpWtNErdDUV+CprJSkh4Hc0ru/47ZeY94vkDR GHkVLhcrHi/Qyk2CnDaavpZlFrEMBZe+yZFgZUG9mn1xkfrK6SUA5Vd7ZfIau9uxhaYK XQOwMyI2Fwz2/AY30vUcyUIeEUq56j7Orob3gnkKE+YR+4STSBoSe2CgOCAbmsFyyhCT i5Rg== MIME-Version: 1.0 X-Received: by 10.15.31.70 with SMTP id x46mr2591427eeu.26.1395427642639; Fri, 21 Mar 2014 11:47:22 -0700 (PDT) Received: by 10.14.219.68 with HTTP; Fri, 21 Mar 2014 11:47:22 -0700 (PDT) In-Reply-To: <20140319163544.GB11935@ox> References: <532933A3.4030403@freebsd.org> <20140319163544.GB11935@ox> Date: Fri, 21 Mar 2014 18:47:22 +0000 Message-ID: Subject: Re: Include port number in "Listen queue overflow" messages From: hiren panchasara To: Julian Elischer , George Neville-Neil , hiren panchasara , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 18:47:25 -0000 On Wed, Mar 19, 2014 at 4:35 PM, Navdeep Parhar wrote: > On Tue, Mar 18, 2014 at 11:05:23PM -0700, Julian Elischer wrote: >> On 3/18/14, 8:33 PM, George Neville-Neil wrote: >> >On Mar 7, 2014, at 1:23 , hiren panchasara wrote: >> > >> >>I am thinking of committing following change that includes port number >> >>in "Listen queue overflow" messages. >> I think it's a good idea. There is even more information available >> but this is probably enough. >> >> >> >I like it. >> > >> >Best, >> >George >> > > > I think the suggested change isn't correct as is assumes every socket's pcb is > an inpcb. You are right. I'd need to think a bit more about a possible solution. Thanks for your help. cheers, Hiren > > Navdeep > >> >>New message would look something like: >> >>sonewconn: pcb 0xfffff8001b155760: Listen queue overflow on port >> >>13120: 1 already in queue awaiting acceptance (454 occurrences) >> >> >> >>I've recently ran into a situation at $work where I could not catch >> >>the culprit application via "netstat -A" and had to dive into kgdb to >> >>find the port from pcb where this application was listening to. >> >> >> >>IMO, this change will make debugging easier. >> >> >> >>cheers, >> >>Hiren >> >> >> >>Index: sys/kern/uipc_socket.c >> >>=================================================================== >> >>--- sys/kern/uipc_socket.c (revision 262861) >> >>+++ sys/kern/uipc_socket.c (working copy) >> >>@@ -136,6 +136,7 @@ >> >>#include >> >>#include >> >>#include >> >>+#include >> >> >> >>#include >> >> >> >>@@ -491,8 +492,11 @@ >> >> static int overcount; >> >> >> >> struct socket *so; >> >>+ struct inpcb *inp; >> >> int over; >> >> >> >>+ inp = sotoinpcb(head); >> >>+ >> >> ACCEPT_LOCK(); >> >> over = (head->so_qlen > 3 * head->so_qlimit / 2); >> >> ACCEPT_UNLOCK(); >> >>@@ -504,10 +508,12 @@ >> >> overcount++; >> >> >> >> if (ratecheck(&lastover, &overinterval)) { >> >>- log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow: " >> >>- "%i already in queue awaiting acceptance " >> >>+ log(LOG_DEBUG, "%s: pcb %p: Listen queue overflow on " >> >>+ "port %d: %i already in queue awaiting acceptance " >> >> "(%d occurrences)\n", >> >>- __func__, head->so_pcb, head->so_qlen, overcount); >> >>+ __func__, head->so_pcb, >> >>+ ntohs(inp->inp_inc.inc_lport), head->so_qlen, >> >>+ overcount); >> >> >> >> overcount = 0; >> >> } >> >>_______________________________________________ >> >>freebsd-net@freebsd.org mailing list >> >>http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >_______________________________________________ >> >freebsd-net@freebsd.org mailing list >> >http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 19:51:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B1AC740 for ; Fri, 21 Mar 2014 19:51:22 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 012EDA81 for ; Fri, 21 Mar 2014 19:51:21 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so3329357qcz.19 for ; Fri, 21 Mar 2014 12:51:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nqXkdoKmJwVt8uNRHVtZYu6MeZJhb4lSTd9F7Ju0png=; b=i4Xxus13v1C99R2U/+kAushQigT8uAMuAgJTvT6VPIyPb33rIRx66IO8wJMgLJf5Ew L6SndY0+46u5tbPQQRmnHEZIXH6nNL7jHAqZJSEjGpuxfg1UYylrj14NX15BCs+4XjS9 dWorEvmshkXLutgg8vucrgXK/LaSOn6vhVtUdrIsbVyuYXfnMsvL/AFRbPzFiPP5SoGA Xo+i14B4R4FM0cCobQZjSxW07lQZLW7VS3UcB/jHGjceblkJRRbcJkptoWhyVg6//RSG FYMnRW2U0X44p4T2KEEQAulnCT/mXzuORTJg/yoYdfZoQMXfZ31w0wBGxFi06qI026/l nufQ== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr42459222qga.57.1395431481226; Fri, 21 Mar 2014 12:51:21 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 12:51:21 -0700 (PDT) In-Reply-To: References: <1543350122.637684.1395368002237.JavaMail.root@uoguelph.ca> <34491192-F8EC-45C1-A7C8-61C3EBE5CFBD@hostpoint.ch> <7B586D3A-BD6B-40B1-980E-7F9FD4A49F6A@hostpoint.ch> Date: Fri, 21 Mar 2014 16:51:21 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 19:51:22 -0000 Update: I've noticed a fair number of differences in the ixgbe driver between 9.2 and 10.0-RELEASE, even though they have the same 2.5.15 version. Mostly Netmap integration. I've loaded up a 9.2-STABLE ixgbe driver from Dec 25th as it was handy (I had to hack the source a bit since some #def's had changed), and I immediately notice one difference: netstat -m 21486/1464/22950 mbufs in use (current/cache/total) 4080/168/4248/6127254 mbuf clusters in use (current/cache/total/max) 4080/149 mbuf+clusters out of packet secondary zone in use (current/cache) 0/3/3/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16384/0/16384/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 160987K/714K/161701K bytes allocated to network (current/cache/total) 17721/108/4104 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 2/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile Not a homerun, but definitely better on the jumbo clusters denied. (For reference, I'd normally see: 2/13185/0 requests for jumbo clusters denied (4k/9k/16k) ) It still gives me errors, but you can see it's really not hitting the wall for jumbo clusters on boot. Perhaps those jumbo clusters are being denied as the buffers are being setup? This is after it starts to blow up: netstat -m 21632/12838/34470 mbufs in use (current/cache/total) 4116/4808/8924/6127254 mbuf clusters in use (current/cache/total/max) 4080/4050 mbuf+clusters out of packet secondary zone in use (current/cache) 0/36/36/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16439/121/16560/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 161591K/14058K/175649K bytes allocated to network (current/cache/total) 20581/3188/5880 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 33/84/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile ..after another 5 min of blowups netstat -m 28065/8040/36105 mbufs in use (current/cache/total) 4482/4644/9126/6127254 mbuf clusters in use (current/cache/total/max) 4112/4018 mbuf+clusters out of packet secondary zone in use (current/cache) 0/36/36/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16384/176/16560/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 163436K/13026K/176462K bytes allocated to network (current/cache/total) 22223/3199/5880 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 33/84/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile My next attempt it ixgbe from 10.0-STABLE. I will come back to the 9.2-STABLE driver a bit later. As for what queues are locked from this: CPU0, 8, 16 - Three down, like last time. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 22:22:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2362D677 for ; Fri, 21 Mar 2014 22:22:08 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CE784AB4 for ; Fri, 21 Mar 2014 22:22:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAEO7LFODaFve/2dsb2JhbABZg0FXgwe4IoZpUYEvdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1AIDax7olcXgSmLRIEsAQEbNAeCb4FJBJVyhAmQf4NJITGBBDk X-IronPort-AV: E=Sophos;i="4.97,706,1389762000"; d="scan'208";a="107817844" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 21 Mar 2014 18:21:38 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A002EB40F3; Fri, 21 Mar 2014 18:21:38 -0400 (EDT) Date: Fri, 21 Mar 2014 18:21:38 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <994670630.1147191.1395440498646.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang (packets that exceed 65535bytes in length) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 22:22:08 -0000 Christopher Forgeron wrote: > (Pardon me, for some reason my gmail is sending on my cut-n-pastes if > I cr > down too fast) > > First set of logs: > > Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Ok, so this isn't a TSO segment then, unless I don't understand how the csum flags are used, which is quite possible. Assuming that you printed this out in decimal: 4116->0x1014 Looking in mbuf.h, 0x1014 is CSUM_SCTP_VALID | CSUM_FRAGMENT | CSUM_UDP alternately, if 4116 is hex, then it is: CSUM_TCP_IPV6 | CSUM_IP_CHECKED | CSUM_FRAGMENT | CSUM_UDP either way, it doesn't appear to be a TCP TSO? (But you said that disabling TSO fixed the problem, so colour me confused by this.;-) Sorry, but my rusty networking is confused by this, so maybe someone else can explain it? (I don't think any packet handed to the net interface should exceed 65535. Am I right?) Anyhow, all I can say is that I think these mbuf chains should fail with EFBIG, since they are too big. I have no idea where they come from and I don't know why this would lead to exhaustion of the transmit descriptor entries, which seems to be when things get really wedged. (From what little I can see in the driver sources, these transmit descriptor entries should be released via interrupts, but I've just glanced at it.) Sorry, but I think this will need someone conversant with the networking side to figure out, rick > Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 > Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 > Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 > Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 > Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 > > Here's a few later on. > > Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > > Mar 21 11:23:00 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 > Mar 21 11:23:01 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 > Mar 21 11:23:01 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 > Mar 21 11:23:03 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 > Mar 21 11:23:03 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 > Mar 21 11:23:04 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 > Mar 21 11:23:04 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 > > Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 > Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 > > To be clear, I changed tp->t_tsomax to IP_MAXPACKET at ~ 777 in > sys/netinet/tcp_output.c like so: > > if (len > IP_MAXPACKET - hdrlen) { > len = IP_MAXPACKET - hdrlen; > sendalot = 1; > } > > I notice there is more that is different between 9.1 and 10 for this > file: > http://fxr.watson.org/fxr/diff/netinet/tcp_output.c?v=FREEBSD10;diffval=FREEBSD91;diffvar=v > > I'm going to attempt inserting a 9.1 tcp_output.c and see if that > makes any > difference. > > Otherwise, I wait further ideas from the list. > > Thanks. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 23:06:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A88C74F for ; Fri, 21 Mar 2014 23:06:43 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0D56BE13 for ; Fri, 21 Mar 2014 23:06:42 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so3117281qaj.39 for ; Fri, 21 Mar 2014 16:06:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=wC8A4gQP/2b9JOF6Q3x3zDwsxvOXYVqvD/SUqgn/Vvk=; b=QuJWRGZR1BEM9DdbOz8er1xGSo3i48BbhhVm/LnJxdFhfvgohVBY2joC1UrE45anvF 8L8qDtuM8dgTMBBt7StOJ4oE6ZRHjN1NYtTzEP/t8CKHXfzEZ2mxvNY5wlx+pqkXkWw8 gGtC9zMz7dubSSFPg5WE0cX+PInTl1jg8cnc2myFef/9GluoOwgaSuHZsdg3zsH/0W4S TJbY6PsaaWS1IYsffwobAzm6tGCQrQBxxkot4c5hnMZLYtZhG92UTu5wvPu8TCIu+TUu TfM0CElCrWf3INq+4QTV8F3gvQcQ56f1WPNozcVFJretamXWL/bb+lPxy3w1khnS0rAw v5ow== MIME-Version: 1.0 X-Received: by 10.140.39.212 with SMTP id v78mr57157336qgv.77.1395443202270; Fri, 21 Mar 2014 16:06:42 -0700 (PDT) Received: by 10.96.66.67 with HTTP; Fri, 21 Mar 2014 16:06:42 -0700 (PDT) Date: Sat, 22 Mar 2014 07:06:42 +0800 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang (packets that exceed 65535bytes in length) From: shiu michael To: markus.gebert@hostpoint.ch, csforgeron@gmail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, jfvogel@gmail.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 23:06:43 -0000 >Ok, so this isn't a TSO segment then, unless I don't understand how >the csum flags are used, which is quite possible. >Assuming that you printed this out in decimal: >4116->0x1014 >Looking in mbuf.h, 0x1014 is >CSUM_SCTP_VALID | CSUM_FRAGMENT | CSUM_UDP > >alternately, if 4116 is hex, then it is: >CSUM_TCP_IPV6 | CSUM_IP_CHECKED | CSUM_FRAGMENT | CSUM_UDP > >either way, it doesn't appear to be a TCP TSO? >(But you said that disabling TSO fixed the problem, so colour me >confused by this.;-) Maybe Christopher is printing the csum from the last mbuf chain ?? Just 2 cents, Mike From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 23:13:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9893BBB9 for ; Fri, 21 Mar 2014 23:13:14 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 576A5EEF for ; Fri, 21 Mar 2014 23:13:14 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id i50so9062295qgf.0 for ; Fri, 21 Mar 2014 16:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xaVXVvoFmWwhDtEff1UDfC2zLejkntKZtA0feN0slz8=; b=wTFXT3K9OvQv4d9l+mDo8Alp2i9H6agso/jhikgrI0926u4wd8H1Ww+rRABxPhuqXy Efay9d3SQm3AOreZ/YZDMysG96dNOzDWQQa67jy+xCfuf8j7D99u650SnMARGc8mxeuP j2Yzbtz6o153JcNcQQke9rMzDMeJ6UwB+gbqHmQNyTJz6cTFlRNh5EAbaKgnPUbOwnkC qIaAj5DtOugq9DzdnEDhVeOgEsaPOqckm22pNeDDPCTZ7gax7oyu8CTt7f8iX4ztCngp fHGch1gs4HQWcHlpj/wLlh79xeVPmFQDohIh157PGHdIVffVcVdroEmS1j1RIC2G34fZ +PxA== MIME-Version: 1.0 X-Received: by 10.140.91.228 with SMTP id z91mr57416223qgd.32.1395443593590; Fri, 21 Mar 2014 16:13:13 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 16:13:13 -0700 (PDT) In-Reply-To: <994670630.1147191.1395440498646.JavaMail.root@uoguelph.ca> References: <994670630.1147191.1395440498646.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 20:13:13 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang (packets that exceed 65535bytes in length) From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 23:13:14 -0000 Ah, I appreciate your efforts Rick - If you have any final parting hints, please let me know. I'm opening up access from my IDE so I can look at this with something a bit more advanced than the default vi. For the record, I was printing the flags out as an unsigned long, so that should be decimal. BTW - I feel this is more than just ixgbe - Here's a netstat -m line from a 10.0-STABLE vm running e1000 em0 and vmware's vmx0 0/549/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) Still 549 more denied mbufs than any of my other FreeBSD systems. From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 23:18:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54C9ECAC for ; Fri, 21 Mar 2014 23:18:09 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 14B3BF10 for ; Fri, 21 Mar 2014 23:18:09 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id w8so3178632qac.27 for ; Fri, 21 Mar 2014 16:18:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bYu1FnZVM2XzVr48tOw4Jj5gx9dKrxSsOmr5MRjMP0g=; b=AghYiXs0ljszrLDounUmTGFGnb4+oVMAmoh6Z1gZ9UZ+n1wc1hTZ4IbiPAPtZp46SQ fqfoLvwXKTI76VyoojLiEFkOf2Kwf555weHr9BJe5lVNT85LLTxk1yu6jGLKPgWAs+8T LRvTUK/3OOV3lh+F5K72AekxG+NAO9hQVkWkqPl1q7pblZT4TRp3Nt8YlUdyylLHc7N9 WGl6QmT2RZ8YfTGpl7qybFTUHwsEui/TjFaHRy+0Zr1Tfep85GLyBBUlk3InriKmc++X Vn/vwewxQxF+JW1bGszonfarwfAHXws3hLurxEPALA2lzXd3AFdJWOAnr7BlXsngX7Xi Nkwg== MIME-Version: 1.0 X-Received: by 10.224.122.20 with SMTP id j20mr35854498qar.79.1395443888372; Fri, 21 Mar 2014 16:18:08 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 16:18:08 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Mar 2014 20:18:08 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang (packets that exceed 65535bytes in length) From: Christopher Forgeron To: shiu michael Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 23:18:09 -0000 Good point - I'm printing where Rick asked, in the 'before' printf statement, which comes before the m = m_defrag(*m_headp, M_NOWAIT); command in ixgbe_xmit I'm going to be adding more printf's to the code to see if I can find anything interesting, your suggestions would be welcome. ..and I suppose there is no such thing as Unit Tests for the FreeBSD Kernel? On Fri, Mar 21, 2014 at 8:06 PM, shiu michael wrote: > >Ok, so this isn't a TSO segment then, unless I don't understand how > >the csum flags are used, which is quite possible. > >Assuming that you printed this out in decimal: > >4116->0x1014 > >Looking in mbuf.h, 0x1014 is > >CSUM_SCTP_VALID | CSUM_FRAGMENT | CSUM_UDP > > > >alternately, if 4116 is hex, then it is: > >CSUM_TCP_IPV6 | CSUM_IP_CHECKED | CSUM_FRAGMENT | CSUM_UDP > > > >either way, it doesn't appear to be a TCP TSO? > >(But you said that disabling TSO fixed the problem, so colour me > >confused by this.;-) > > Maybe Christopher is printing the csum from the last mbuf chain ?? > > Just 2 cents, > Mike > From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 23:44:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60FEE378 for ; Fri, 21 Mar 2014 23:44:55 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 13FA31EB for ; Fri, 21 Mar 2014 23:44:54 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqMEAAfOLFODaFve/2dsb2JhbABZg0FXgwe4Ioc6gSx0giUBAQEDAQECIARSBRYOChEZAgQfBy8GE4dlAwkIDaxjmzgNhwwXjFKBZBkbB4JvgUkEkFOFH2qDH4s2hUmDSSGBbg X-IronPort-AV: E=Sophos;i="4.97,706,1389762000"; d="scan'208";a="108072118" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 21 Mar 2014 19:44:53 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9570CB4039; Fri, 21 Mar 2014 19:44:53 -0400 (EDT) Date: Fri, 21 Mar 2014 19:44:53 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1169271_1381755531.1395445493598" X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 23:44:55 -0000 ------=_Part_1169271_1381755531.1395445493598 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > > > > > > > Hello all, > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer > away at the NFS store overnight - But the problem is still there. > > > From what I read, I think the MJUM9BYTES removal is probably good > cleanup (as long as it doesn't trade performance on a lightly memory > loaded system for performance on a heavily memory loaded system). If > I can stabilize my system, I may attempt those benchmarks. > > > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' > netstat > - Until I can boot and see a 'netstat -m' that looks similar to that, > I'm going to have this problem. > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > Turning off TSO works for me, but at a performance hit. > > I'll compile Rick's patch (and extra debugging) this morning and let > you know soon. > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > csforgeron@gmail.com > wrote: > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch that > Jack applied. > > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m > shows: > > 21489/2886/24375 mbufs in use (current/cache/total) > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) > 4080/587 mbuf+clusters out of packet secondary zone in use > (current/cache) > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 79068K/2173K/81241K bytes allocated to network (current/cache/total) > 18831/545/4542 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > Here is an un-patched boot: > > 21550/7400/28950 mbufs in use (current/cache/total) > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2769 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/42/42/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16439/129/16568/907741 9k jumbo clusters in use > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161498K/10699K/172197K bytes allocated to network > (current/cache/total) > 18345/155/4099 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > > See how removing the MJUM9BYTES is just pushing the problem from the > 9k jumbo cluster into the 4k jumbo cluster? > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact > same hardware, revisions, zpool size, etc. Just it's running an > older FreeBSD. > > # uname -a > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > 15:12:14 AST 2013 aatech@FreeBSD-Update > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > root@SAN1:/san1 # uptime > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 > > root@SAN1:/san1 # netstat -m > 37930/15755/53685 mbufs in use (current/cache/total) > 4080/10996/15076/524288 mbuf clusters in use > (current/cache/total/max) > 4080/5775 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/692/692/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > 32773/4257/37030/96000 9k jumbo clusters in use > (current/cache/total/max) > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > 312599K/67011K/379611K bytes allocated to network > (current/cache/total) > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > Lastly, please note this link: > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > Hmm, this mentioned the ethernet header being in the TSO segment. I think I already mentioned my TCP/IP is rusty and I know diddly about TSO. However, at a glance it does appear the driver uses ether_output() for TSO segments and, as such, I think an ethernet header is prepended to the TSO segment. (This makes sense, since how else would the hardware know what ethernet header to use for the TCP segments generated.) I think prepending the ethernet header could push the total length over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over 64K isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so forth. Anyhow, I think the attached patch will reduce if_hw_tsomax, so that the result should fit in 32 clusters and avoid EFBIG for this case, so it might be worth a try? (I still can't think of why the CSUM_TSO bit isn't set for the printf() case, but it seems TSO segments could generate EFBIG errors.) Maybe worth a try, rick > It's so old that I assume the TSO leak that he speaks of has been > patched, but perhaps not. More things to look into tomorrow. > > > > > ------=_Part_1169271_1381755531.1395445493598 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0yMSAxOToyNTo0Ni4wMDAwMDAwMDAg LTA0MDAKQEAgLTI2MTQsNiArMjYxNCw5IEBAIGl4Z2JlX3NldHVwX2ludGVyZmFjZShkZXZpY2Vf dCBkZXYsIHN0cnUKIAlpZnAtPmlmX3NuZC5pZnFfZHJ2X21heGxlbiA9IGFkYXB0ZXItPm51bV90 eF9kZXNjIC0gMjsKIAlJRlFfU0VUX1JFQURZKCZpZnAtPmlmX3NuZCk7CiAjZW5kaWYKKwlpZiAo KGFkYXB0ZXItPm51bV9zZWdzICogTUNMQllURVMgLSBFVEhFUl9IRFJfTEVOKSA8IElQX01BWFBB Q0tFVCkKKwkJaWZwLT5pZl9od190c29tYXggPSBhZGFwdGVyLT5udW1fc2VncyAqIE1DTEJZVEVT IC0KKwkJICAgIEVUSEVSX0hEUl9MRU47CiAKIAlldGhlcl9pZmF0dGFjaChpZnAsIGFkYXB0ZXIt Pmh3Lm1hYy5hZGRyKTsKIAo= ------=_Part_1169271_1381755531.1395445493598-- From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 23:55:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE34B674 for ; Fri, 21 Mar 2014 23:55:56 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B7F92F8 for ; Fri, 21 Mar 2014 23:55:56 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id e9so3680311qcy.15 for ; Fri, 21 Mar 2014 16:55:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CmjmL5mYh/qnLqccVjNmtY90PXioZ8q56IyrYVKZFqo=; b=GPmEqiApvMeGGfuGhmKanHpmWqvRniuvrmTskm4rZXmg4mdQ5hZfN2hcrAfDNasxLT DVWhxRleCiHpBaWmswLJMdPLzfTJ2qrXxpRQJq5ivv0e6I+gxp0jDVXL7dtu319O6TBB 5jQuk0j0dLr3hxD9FHdCrPFDInV6+D45FYilePFvrovwlC+RDalcxBsivHdaYmM7Vhjt Lz4QNd35ntDCPB+c9kgA0GYFKF+sijPO4xL/yQhki6Yhp6QqKL5yuIL1rg66IToHvcuQ FKzf9DnfE1shHTnH8GFvRQa1NnMGMK/AnYk9jn7GNeN4df/+mMmOPpWKhY4FhV79//Y8 W4Lw== MIME-Version: 1.0 X-Received: by 10.140.96.230 with SMTP id k93mr57110575qge.60.1395446155773; Fri, 21 Mar 2014 16:55:55 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 16:55:55 -0700 (PDT) In-Reply-To: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> References: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 20:55:55 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 23:55:56 -0000 Thanks Rick, trying it now. I'm currently working with the 9.2 ixgbe code as a starting point, as I'm curious/encouraged by the lack of jumbo cluster denials in netmap. I'll let you know how it works out. On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > > > > > > > > > > Hello all, > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer > > away at the NFS store overnight - But the problem is still there. > > > > > > From what I read, I think the MJUM9BYTES removal is probably good > > cleanup (as long as it doesn't trade performance on a lightly memory > > loaded system for performance on a heavily memory loaded system). If > > I can stabilize my system, I may attempt those benchmarks. > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' > > netstat > > - Until I can boot and see a 'netstat -m' that looks similar to that, > > I'm going to have this problem. > > > > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > > > > Turning off TSO works for me, but at a performance hit. > > > > I'll compile Rick's patch (and extra debugging) this morning and let > > you know soon. > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch that > > Jack applied. > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m > > shows: > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/587 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 79068K/2173K/81241K bytes allocated to network (current/cache/total) > > 18831/545/4542 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > Here is an un-patched boot: > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16439/129/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161498K/10699K/172197K bytes allocated to network > > (current/cache/total) > > 18345/155/4099 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem from the > > 9k jumbo cluster into the 4k jumbo cluster? > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact > > same hardware, revisions, zpool size, etc. Just it's running an > > older FreeBSD. > > > > # uname -a > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > root@SAN1:/san1 # uptime > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 > > > > root@SAN1:/san1 # netstat -m > > 37930/15755/53685 mbufs in use (current/cache/total) > > 4080/10996/15076/524288 mbuf clusters in use > > (current/cache/total/max) > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/692/692/262144 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 32773/4257/37030/96000 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > 312599K/67011K/379611K bytes allocated to network > > (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > > Lastly, please note this link: > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > Hmm, this mentioned the ethernet header being in the TSO segment. I think > I already mentioned my TCP/IP is rusty and I know diddly about TSO. > However, at a glance it does appear the driver uses ether_output() for > TSO segments and, as such, I think an ethernet header is prepended to the > TSO segment. (This makes sense, since how else would the hardware know > what ethernet header to use for the TCP segments generated.) > > I think prepending the ethernet header could push the total length > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over 64K > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so forth. > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so that > the result should fit in 32 clusters and avoid EFBIG for this case, > so it might be worth a try? > (I still can't think of why the CSUM_TSO bit isn't set for the printf() > case, but it seems TSO segments could generate EFBIG errors.) > > Maybe worth a try, rick > > > It's so old that I assume the TSO leak that he speaks of has been > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 00:15:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41C8AD86 for ; Sat, 22 Mar 2014 00:15:05 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F40416C7 for ; Sat, 22 Mar 2014 00:15:04 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id j107so9197372qga.7 for ; Fri, 21 Mar 2014 17:15:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=fTylW4O+D2aSt4mYhDNuZJanx2DTIFt2eSS9puYA+K0=; b=f6wsUVF6gRwv1ohKRkKz3HSJdCGzJr58v3+qzoqxYF0ZJLf9bkHCaeHddXfAgNgGkN fFhHewsYKVQogtHMrGGTCgfJR7hsaAhlv9lFLYQSAuk2V87KgBDgY21pOY8PKpjHMzTl tgfxQrIc2KEgJJJfHuMfnJtAqqo8k8zraAhLsz+7xWpNyDNFV4XB+IrUkiTi1KdfMiBs ghbgKLfmU/z4NvzQ/4o89nKYAlhzEDpysEKnibXZLk8LhxZRfNrGElHBKE1bo0Lhe8VI fkEfCw1xvIl7Baidp92yL0zxKquqj0RiJXNDaVC0m3Z2Hxssi7E1zdSA/AUML6S5G5bu w8ZA== MIME-Version: 1.0 X-Received: by 10.140.95.65 with SMTP id h59mr57753875qge.2.1395447304201; Fri, 21 Mar 2014 17:15:04 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 17:15:04 -0700 (PDT) In-Reply-To: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> References: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 21:15:04 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 00:15:05 -0000 Well I can tell you that your if statement in that patch is being activated. I added a printf in there, and it showed up on my dmsg. I still see bad things for netstat -m , but I'm starting a load run to see if it makes a difference. Next compile I'll add printouts of what ifp->if_hw_tsomax is once it's been set, and I'll print it out in the trouble spot (ixgbe_xmit) as well. On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > > > > > > > > > > Hello all, > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer > > away at the NFS store overnight - But the problem is still there. > > > > > > From what I read, I think the MJUM9BYTES removal is probably good > > cleanup (as long as it doesn't trade performance on a lightly memory > > loaded system for performance on a heavily memory loaded system). If > > I can stabilize my system, I may attempt those benchmarks. > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' > > netstat > > - Until I can boot and see a 'netstat -m' that looks similar to that, > > I'm going to have this problem. > > > > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > > > > Turning off TSO works for me, but at a performance hit. > > > > I'll compile Rick's patch (and extra debugging) this morning and let > > you know soon. > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch that > > Jack applied. > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m > > shows: > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/587 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 79068K/2173K/81241K bytes allocated to network (current/cache/total) > > 18831/545/4542 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > Here is an un-patched boot: > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16439/129/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161498K/10699K/172197K bytes allocated to network > > (current/cache/total) > > 18345/155/4099 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem from the > > 9k jumbo cluster into the 4k jumbo cluster? > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact > > same hardware, revisions, zpool size, etc. Just it's running an > > older FreeBSD. > > > > # uname -a > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > root@SAN1:/san1 # uptime > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 > > > > root@SAN1:/san1 # netstat -m > > 37930/15755/53685 mbufs in use (current/cache/total) > > 4080/10996/15076/524288 mbuf clusters in use > > (current/cache/total/max) > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/692/692/262144 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 32773/4257/37030/96000 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > 312599K/67011K/379611K bytes allocated to network > > (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > > Lastly, please note this link: > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > Hmm, this mentioned the ethernet header being in the TSO segment. I think > I already mentioned my TCP/IP is rusty and I know diddly about TSO. > However, at a glance it does appear the driver uses ether_output() for > TSO segments and, as such, I think an ethernet header is prepended to the > TSO segment. (This makes sense, since how else would the hardware know > what ethernet header to use for the TCP segments generated.) > > I think prepending the ethernet header could push the total length > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over 64K > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so forth. > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so that > the result should fit in 32 clusters and avoid EFBIG for this case, > so it might be worth a try? > (I still can't think of why the CSUM_TSO bit isn't set for the printf() > case, but it seems TSO segments could generate EFBIG errors.) > > Maybe worth a try, rick > > > It's so old that I assume the TSO leak that he speaks of has been > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 00:25:57 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3973D646; Sat, 22 Mar 2014 00:25:57 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0DAEF840; Sat, 22 Mar 2014 00:25:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2M0Pu6v054426; Sat, 22 Mar 2014 00:25:56 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2M0PuZ9054425; Sat, 22 Mar 2014 00:25:56 GMT (envelope-from linimon) Date: Sat, 22 Mar 2014 00:25:56 GMT Message-Id: <201403220025.s2M0PuZ9054425@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: bin/187835: ngctl(8) strange behavior when adding more than 530 vlan through nethraph X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 00:25:57 -0000 Old Synopsis: Ngctl strange behavior when adding more than 530 vlan through nethraph New Synopsis: ngctl(8) strange behavior when adding more than 530 vlan through nethraph Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sat Mar 22 00:25:29 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=187835 From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 01:25:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6F9E3AF for ; Sat, 22 Mar 2014 01:25:19 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 73477D10 for ; Sat, 22 Mar 2014 01:25:19 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id 63so9353884qgz.6 for ; Fri, 21 Mar 2014 18:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FmdsjPKJTJ59HT3oZ0lg9b2lBHib7SgPMmedWgPelxE=; b=OulSPPcqWlS6wdecfndT39tMO2cgmDG1c43FvP7WpkDqFSvCH/QBO6LPz4SW1UQtQg jFabI0WvCAaYg1WPQrbo+dkLbVqkagbkgfOimkDQ4kQ1UUNy+qV86skygRuUZzBX7x1T zM8zFXYi4qEGlEJeSGpOqWlDZrbLVcJfrtie5aWBnUijmQOkC4tQLbsbLqcKDxujmN6f zd7cEv3UhedR4to21f26Iko8aMlsRD6gojzrVdLV2Wa6jXiIrwNHYH9sEvBEkJeZVD8P bBrko7WXekEhhalu0hEeVNOU/zSgCtICdiivbv02OIJwNp1pwcWNEtlEQ8l7gSbEjpaB 85AA== MIME-Version: 1.0 X-Received: by 10.140.29.139 with SMTP id b11mr58209336qgb.48.1395451518675; Fri, 21 Mar 2014 18:25:18 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 18:25:18 -0700 (PDT) In-Reply-To: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> References: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 22:25:18 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 01:25:19 -0000 It may be a little early, but I think that's it! It's been running without error for nearly an hour - It's very rare it would go this long under this much load. I'm going to let it run longer, then abort and install the kernel with the extra printfs so I can see what value ifp->if_hw_tsomax is before you set it. It still had netstat -m denied entries on boot, but they are not climbing like they did before: $ uptime 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 $ netstat -m 21556/7034/28590 mbufs in use (current/cache/total) 4080/3076/7156/6127254 mbuf clusters in use (current/cache/total/max) 4080/2281 mbuf+clusters out of packet secondary zone in use (current/cache) 0/53/53/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16444/118/16562/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 161545K/9184K/170729K bytes allocated to network (current/cache/total) 17972/2230/4111 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile - Started off bad with the 9k denials, but it's not going up! uptime 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 root@SAN0:/usr/home/aatech # netstat -m 21569/7141/28710 mbufs in use (current/cache/total) 4080/3308/7388/6127254 mbuf clusters in use (current/cache/total/max) 4080/2281 mbuf+clusters out of packet secondary zone in use (current/cache) 0/53/53/3063627 4k (page size) jumbo clusters in use (current/cache/total/max) 16447/121/16568/907741 9k jumbo clusters in use (current/cache/total/max) 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) 161575K/9702K/171277K bytes allocated to network (current/cache/total) 17972/2261/4111 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into the base 10.0 code tomorrow. On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > > > > > > > > > > Hello all, > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer > > away at the NFS store overnight - But the problem is still there. > > > > > > From what I read, I think the MJUM9BYTES removal is probably good > > cleanup (as long as it doesn't trade performance on a lightly memory > > loaded system for performance on a heavily memory loaded system). If > > I can stabilize my system, I may attempt those benchmarks. > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' > > netstat > > - Until I can boot and see a 'netstat -m' that looks similar to that, > > I'm going to have this problem. > > > > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > > > > Turning off TSO works for me, but at a performance hit. > > > > I'll compile Rick's patch (and extra debugging) this morning and let > > you know soon. > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch that > > Jack applied. > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m > > shows: > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/587 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 79068K/2173K/81241K bytes allocated to network (current/cache/total) > > 18831/545/4542 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > Here is an un-patched boot: > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16439/129/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161498K/10699K/172197K bytes allocated to network > > (current/cache/total) > > 18345/155/4099 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem from the > > 9k jumbo cluster into the 4k jumbo cluster? > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact > > same hardware, revisions, zpool size, etc. Just it's running an > > older FreeBSD. > > > > # uname -a > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > root@SAN1:/san1 # uptime > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 > > > > root@SAN1:/san1 # netstat -m > > 37930/15755/53685 mbufs in use (current/cache/total) > > 4080/10996/15076/524288 mbuf clusters in use > > (current/cache/total/max) > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/692/692/262144 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 32773/4257/37030/96000 9k jumbo clusters in use > > (current/cache/total/max) > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > 312599K/67011K/379611K bytes allocated to network > > (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > > Lastly, please note this link: > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > Hmm, this mentioned the ethernet header being in the TSO segment. I think > I already mentioned my TCP/IP is rusty and I know diddly about TSO. > However, at a glance it does appear the driver uses ether_output() for > TSO segments and, as such, I think an ethernet header is prepended to the > TSO segment. (This makes sense, since how else would the hardware know > what ethernet header to use for the TCP segments generated.) > > I think prepending the ethernet header could push the total length > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over 64K > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so forth. > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so that > the result should fit in 32 clusters and avoid EFBIG for this case, > so it might be worth a try? > (I still can't think of why the CSUM_TSO bit isn't set for the printf() > case, but it seems TSO segments could generate EFBIG errors.) > > Maybe worth a try, rick > > > It's so old that I assume the TSO leak that he speaks of has been > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 02:39:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 420BCE4E; Sat, 22 Mar 2014 02:39:39 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id A81882C2; Sat, 22 Mar 2014 02:39:38 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEACD3LFODaFve/2dsb2JhbABZg0FXgwe4JIZpUYEsdIIlAQEBAwEBAQEgBCcgCwUWDgoRGQIEHwYBCSYGCAcEARwEh0QDCQgNrEybMQ2HDBeMUoFHAQEbGRsHgm+BSQSQU4UfaoMfizaFSYNJITGBBDk X-IronPort-AV: E=Sophos;i="4.97,707,1389762000"; d="scan'208";a="108115839" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 21 Mar 2014 22:39:36 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 281AEB4103; Fri, 21 Mar 2014 22:39:36 -0400 (EDT) Date: Fri, 21 Mar 2014 22:39:36 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1613242078.1214156.1395455976156.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1214154_1974091844.1395455976155" X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 02:39:39 -0000 ------=_Part_1214154_1974091844.1395455976155 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > It may be a little early, but I think that's it! > > It's been running without error for nearly an hour - It's very rare > it > would go this long under this much load. > > I'm going to let it run longer, then abort and install the kernel > with the > extra printfs so I can see what value ifp->if_hw_tsomax is before you > set > it. > I think you'll just find it set to 0. Code in if_attach_internal() { in sys/net/if.c } sets it to IP_MAXPACKET (which is 65535) if it is 0. In other words, if the if_attach routine in the driver doesn't set it, this code sets it to the maximum possible value. Here's the snippet: /* Initialize to max value. */ 657 if (ifp->if_hw_tsomax == 0) 658 ifp->if_hw_tsomax = IP_MAXPACKET; Anyhow, this sounds like progress. As far as NFS is concerned, I'd rather set it to a smaller value (maybe 56K) so that m_defrag() doesn't need to be called, but I suspect others wouldn't like this. Hopefully Jack can decide if this patch is ok? Thanks yet again for doing this testing, rick ps: I've attached it again, so Jack (and anyone else who reads this) can look at it. pss: Please report if it keeps working for you. > It still had netstat -m denied entries on boot, but they are not > climbing > like they did before: > > > $ uptime > 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 > $ netstat -m > 21556/7034/28590 mbufs in use (current/cache/total) > 4080/3076/7156/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2281 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/53/53/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16444/118/16562/907741 9k jumbo clusters in use > (current/cache/total/max) > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161545K/9184K/170729K bytes allocated to network > (current/cache/total) > 17972/2230/4111 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > - Started off bad with the 9k denials, but it's not going up! > > uptime > 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 > root@SAN0:/usr/home/aatech # netstat -m > 21569/7141/28710 mbufs in use (current/cache/total) > 4080/3308/7388/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2281 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/53/53/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16447/121/16568/907741 9k jumbo clusters in use > (current/cache/total/max) > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161575K/9702K/171277K bytes allocated to network > (current/cache/total) > 17972/2261/4111 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into the > base > 10.0 code tomorrow. > > > On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem > wrote: > > > Christopher Forgeron wrote: > > > > > > > > > > > > > > > > > > > > > Hello all, > > > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer > > > hammer > > > away at the NFS store overnight - But the problem is still there. > > > > > > > > > From what I read, I think the MJUM9BYTES removal is probably good > > > cleanup (as long as it doesn't trade performance on a lightly > > > memory > > > loaded system for performance on a heavily memory loaded system). > > > If > > > I can stabilize my system, I may attempt those benchmarks. > > > > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a > > > 'clean' > > > netstat > > > - Until I can boot and see a 'netstat -m' that looks similar to > > > that, > > > I'm going to have this problem. > > > > > > > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > > > > > > > Turning off TSO works for me, but at a performance hit. > > > > > > I'll compile Rick's patch (and extra debugging) this morning and > > > let > > > you know soon. > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch > > > that > > > Jack applied. > > > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what > > > netstat -m > > > shows: > > > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > > 4080/626/4706/6127254 mbuf clusters in use > > > (current/cache/total/max) > > > 4080/587 mbuf+clusters out of packet secondary zone in use > > > (current/cache) > > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > > (current/cache/total/max) > > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > 79068K/2173K/81241K bytes allocated to network > > > (current/cache/total) > > > 18831/545/4542 requests for mbufs denied > > > (mbufs/clusters/mbuf+clusters) > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > 0 requests for sfbufs denied > > > 0 requests for sfbufs delayed > > > 0 requests for I/O initiated by sendfile > > > > > > Here is an un-patched boot: > > > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > > 4080/3760/7840/6127254 mbuf clusters in use > > > (current/cache/total/max) > > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > > (current/cache) > > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > > (current/cache/total/max) > > > 16439/129/16568/907741 9k jumbo clusters in use > > > (current/cache/total/max) > > > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > 161498K/10699K/172197K bytes allocated to network > > > (current/cache/total) > > > 18345/155/4099 requests for mbufs denied > > > (mbufs/clusters/mbuf+clusters) > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > 0 requests for sfbufs denied > > > 0 requests for sfbufs delayed > > > 0 requests for I/O initiated by sendfile > > > > > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem from > > > the > > > 9k jumbo cluster into the 4k jumbo cluster? > > > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : > > > Exact > > > same hardware, revisions, zpool size, etc. Just it's running an > > > older FreeBSD. > > > > > > # uname -a > > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > root@SAN1:/san1 # uptime > > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, > > > 0.91 > > > > > > root@SAN1:/san1 # netstat -m > > > 37930/15755/53685 mbufs in use (current/cache/total) > > > 4080/10996/15076/524288 mbuf clusters in use > > > (current/cache/total/max) > > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > > (current/cache) > > > 0/692/692/262144 4k (page size) jumbo clusters in use > > > (current/cache/total/max) > > > 32773/4257/37030/96000 9k jumbo clusters in use > > > (current/cache/total/max) > > > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > > 312599K/67011K/379611K bytes allocated to network > > > (current/cache/total) > > > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > 0/0/0 sfbufs in use (current/peak/max) > > > 0 requests for sfbufs denied > > > 0 requests for sfbufs delayed > > > 0 requests for I/O initiated by sendfile > > > 0 calls to protocol drain routines > > > > > > Lastly, please note this link: > > > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > > > Hmm, this mentioned the ethernet header being in the TSO segment. I > > think > > I already mentioned my TCP/IP is rusty and I know diddly about TSO. > > However, at a glance it does appear the driver uses ether_output() > > for > > TSO segments and, as such, I think an ethernet header is prepended > > to the > > TSO segment. (This makes sense, since how else would the hardware > > know > > what ethernet header to use for the TCP segments generated.) > > > > I think prepending the ethernet header could push the total length > > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over > > 64K > > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so > > forth. > > > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so > > that > > the result should fit in 32 clusters and avoid EFBIG for this case, > > so it might be worth a try? > > (I still can't think of why the CSUM_TSO bit isn't set for the > > printf() > > case, but it seems TSO segments could generate EFBIG errors.) > > > > Maybe worth a try, rick > > > > > It's so old that I assume the TSO leak that he speaks of has been > > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > > > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > ------=_Part_1214154_1974091844.1395455976155 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0yMSAxOToyNTo0Ni4wMDAwMDAwMDAg LTA0MDAKQEAgLTI2MTQsNiArMjYxNCw5IEBAIGl4Z2JlX3NldHVwX2ludGVyZmFjZShkZXZpY2Vf dCBkZXYsIHN0cnUKIAlpZnAtPmlmX3NuZC5pZnFfZHJ2X21heGxlbiA9IGFkYXB0ZXItPm51bV90 eF9kZXNjIC0gMjsKIAlJRlFfU0VUX1JFQURZKCZpZnAtPmlmX3NuZCk7CiAjZW5kaWYKKwlpZiAo KGFkYXB0ZXItPm51bV9zZWdzICogTUNMQllURVMgLSBFVEhFUl9IRFJfTEVOKSA8IElQX01BWFBB Q0tFVCkKKwkJaWZwLT5pZl9od190c29tYXggPSBhZGFwdGVyLT5udW1fc2VncyAqIE1DTEJZVEVT IC0KKwkJICAgIEVUSEVSX0hEUl9MRU47CiAKIAlldGhlcl9pZmF0dGFjaChpZnAsIGFkYXB0ZXIt Pmh3Lm1hYy5hZGRyKTsKIAo= ------=_Part_1214154_1974091844.1395455976155-- From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 02:56:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52453BA for ; Sat, 22 Mar 2014 02:56:07 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0F7085F4 for ; Sat, 22 Mar 2014 02:56:06 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id j15so3315209qaq.30 for ; Fri, 21 Mar 2014 19:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4ji5zztGY/LBEH1iRCgjnnOTaGeBi6REVtN2VfOC0zE=; b=KucC5V3H3D4gStp0FO9/uilAZYZuBfxbcL9clFz1CGhsjoWksXCPZMLF0vSw0eHmbL 15IKR1FdArkVq2Qd5KC5mQdulf3XDsMKUc+tZpbPqqF1dtvsyrZnFYOOI+Lht9sGF3mo cvKoZyUtmP35KThB4BqPbwQLZc2vhf82U5oFF5NOPtxtyWsHN/5vc3gBkKb0UKUAhxhv I/uvQoyUmfRR6TEu0grUHpxoRrvv4fxJhAw+Q6AWo8K1gP8LUVWKW934/fRJvrAFQLiU GyygMVlLsBgLv/ESQSeM5qIEKLUYHeAhFknCyL/J9BMWIjXu6veApzUsSJYhIdlrc87m 1Elw== MIME-Version: 1.0 X-Received: by 10.140.39.212 with SMTP id v78mr57938380qgv.77.1395456966171; Fri, 21 Mar 2014 19:56:06 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 19:56:06 -0700 (PDT) In-Reply-To: References: <649240517.1169273.1395445493600.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 23:56:06 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 02:56:07 -0000 No errors for 1h 46m - That's a record. This is using the 9.2-STABLE ixgbe in a 10.0-RELEASE system, with Rick's suggested code below. I decided this must be it, so I aborted, and modified the ixgbe driver from 10.0-STABLE with Rick's suggestion. Installed and rebooted. Here's the extra values I print out: if ((adapter->num_segs * MCLBYTES - ETHER_HDR_LEN) < IP_MAXPACKET) { printf("CF - Ricks Test! ifp->if_hw_tsomax = %d\n", ifp->if_hw_tsomax); ifp->if_hw_tsomax = adapter->num_segs * MCLBYTES - ETHER_HDR_LEN; printf("CF - After Init, ifp->if_hw_tsomax = %d\n", ifp->if_hw_tsomax); printf("CF - adapter->num_segs=%d, ETHER_HDR_LEN=%d, IP_MAXPACKET=%d\n", adapter->num_segs, ETHER_HDR_LEN, IP_MAXPACKET); } Which shows me: ix0: port 0xfcc0-0xfcdf me m 0xd9000000-0xd93fffff,0xd9bf8000-0xd9bfbfff irq 45 at device 0.0 on pci5 Mar 21 23:00:08 SAN0 kernel: ix0: Using MSIX interrupts with 9 vectors Mar 21 23:00:08 SAN0 kernel: CF - Ricks Test! ifp->if_hw_tsomax = 0 Mar 21 23:00:08 SAN0 kernel: CF - After Init, ifp->if_hw_tsomax = 65522 Mar 21 23:00:08 SAN0 kernel: CF - adapter->num_segs=32, ETHER_HDR_LEN=14, IP_MAXPACKET=65535 ix0: Ethernet address: 00:1b:21:d6:4c:4c I don't see where the TSO max is being set in any other place. I see IXGBE_TSO_SIZE = 262140 in ixgbe.h, and I suppose something similar is happening in ixgbe_tso_setup, setting it to that 262149 default However: This 10.0-STABLE ixgbe has the error. I'm getting it at 25 min of runtime. I don't have the full printf's in this one yet, so I can't tell you more about it. I'm going back to the 9.2-STABLE ixgbe with the above tso modification for a bit longer to confirm that I can run overnight without the error. On Fri, Mar 21, 2014 at 10:25 PM, Christopher Forgeron wrote: > It may be a little early, but I think that's it! > > It's been running without error for nearly an hour - It's very rare it > would go this long under this much load. > > I'm going to let it run longer, then abort and install the kernel with the > extra printfs so I can see what value ifp->if_hw_tsomax is before you set > it. > > It still had netstat -m denied entries on boot, but they are not climbing > like they did before: > > > $ uptime > 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 > $ netstat -m > 21556/7034/28590 mbufs in use (current/cache/total) > 4080/3076/7156/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2281 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/53/53/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16444/118/16562/907741 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161545K/9184K/170729K bytes allocated to network (current/cache/total) > 17972/2230/4111 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > - Started off bad with the 9k denials, but it's not going up! > > uptime > 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 > root@SAN0:/usr/home/aatech # netstat -m > 21569/7141/28710 mbufs in use (current/cache/total) > 4080/3308/7388/6127254 mbuf clusters in use (current/cache/total/max) > 4080/2281 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/53/53/3063627 4k (page size) jumbo clusters in use > (current/cache/total/max) > 16447/121/16568/907741 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > 161575K/9702K/171277K bytes allocated to network (current/cache/total) > 17972/2261/4111 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into the base > 10.0 code tomorrow. > > > On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem wrote: > >> Christopher Forgeron wrote: >> > >> > >> > >> > >> > >> > >> > Hello all, >> > >> > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer hammer >> > away at the NFS store overnight - But the problem is still there. >> > >> > >> > From what I read, I think the MJUM9BYTES removal is probably good >> > cleanup (as long as it doesn't trade performance on a lightly memory >> > loaded system for performance on a heavily memory loaded system). If >> > I can stabilize my system, I may attempt those benchmarks. >> > >> > >> > I think the fix will be obvious at boot for me - My 9.2 has a 'clean' >> > netstat >> > - Until I can boot and see a 'netstat -m' that looks similar to that, >> > I'm going to have this problem. >> > >> > >> > Markus: Do your systems show denied mbufs at boot like mine does? >> > >> > >> > Turning off TSO works for me, but at a performance hit. >> > >> > I'll compile Rick's patch (and extra debugging) this morning and let >> > you know soon. >> > >> > >> > >> > >> > >> > >> > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < >> > csforgeron@gmail.com > wrote: >> > >> > >> > >> > >> > >> > >> > >> > >> > BTW - I think this will end up being a TSO issue, not the patch that >> > Jack applied. >> > >> > When I boot Jack's patch (MJUM9BYTES removal) this is what netstat -m >> > shows: >> > >> > 21489/2886/24375 mbufs in use (current/cache/total) >> > 4080/626/4706/6127254 mbuf clusters in use (current/cache/total/max) >> > 4080/587 mbuf+clusters out of packet secondary zone in use >> > (current/cache) >> > 16384/50/16434/3063627 4k (page size) jumbo clusters in use >> > (current/cache/total/max) >> > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) >> > >> > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) >> > 79068K/2173K/81241K bytes allocated to network (current/cache/total) >> > 18831/545/4542 requests for mbufs denied >> > (mbufs/clusters/mbuf+clusters) >> > >> > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) >> > >> > 0 requests for sfbufs denied >> > 0 requests for sfbufs delayed >> > 0 requests for I/O initiated by sendfile >> > >> > Here is an un-patched boot: >> > >> > 21550/7400/28950 mbufs in use (current/cache/total) >> > 4080/3760/7840/6127254 mbuf clusters in use (current/cache/total/max) >> > 4080/2769 mbuf+clusters out of packet secondary zone in use >> > (current/cache) >> > 0/42/42/3063627 4k (page size) jumbo clusters in use >> > (current/cache/total/max) >> > 16439/129/16568/907741 9k jumbo clusters in use >> > (current/cache/total/max) >> > >> > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) >> > 161498K/10699K/172197K bytes allocated to network >> > (current/cache/total) >> > 18345/155/4099 requests for mbufs denied >> > (mbufs/clusters/mbuf+clusters) >> > >> > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) >> > >> > 0 requests for sfbufs denied >> > 0 requests for sfbufs delayed >> > 0 requests for I/O initiated by sendfile >> > >> > >> > >> > See how removing the MJUM9BYTES is just pushing the problem from the >> > 9k jumbo cluster into the 4k jumbo cluster? >> > >> > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : Exact >> > same hardware, revisions, zpool size, etc. Just it's running an >> > older FreeBSD. >> > >> > # uname -a >> > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 >> > 15:12:14 AST 2013 aatech@FreeBSD-Update >> > Server:/usr/obj/usr/src/sys/GENERIC amd64 >> > >> > root@SAN1:/san1 # uptime >> > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, 0.91 >> > >> > root@SAN1:/san1 # netstat -m >> > 37930/15755/53685 mbufs in use (current/cache/total) >> > 4080/10996/15076/524288 mbuf clusters in use >> > (current/cache/total/max) >> > 4080/5775 mbuf+clusters out of packet secondary zone in use >> > (current/cache) >> > 0/692/692/262144 4k (page size) jumbo clusters in use >> > (current/cache/total/max) >> > 32773/4257/37030/96000 9k jumbo clusters in use >> > (current/cache/total/max) >> > >> > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) >> > 312599K/67011K/379611K bytes allocated to network >> > (current/cache/total) >> > >> > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> > 0/0/0 sfbufs in use (current/peak/max) >> > 0 requests for sfbufs denied >> > 0 requests for sfbufs delayed >> > 0 requests for I/O initiated by sendfile >> > 0 calls to protocol drain routines >> > >> > Lastly, please note this link: >> > >> > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html >> > >> Hmm, this mentioned the ethernet header being in the TSO segment. I think >> I already mentioned my TCP/IP is rusty and I know diddly about TSO. >> However, at a glance it does appear the driver uses ether_output() for >> TSO segments and, as such, I think an ethernet header is prepended to the >> TSO segment. (This makes sense, since how else would the hardware know >> what ethernet header to use for the TCP segments generated.) >> >> I think prepending the ethernet header could push the total length >> over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over 64K >> isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so forth. >> >> Anyhow, I think the attached patch will reduce if_hw_tsomax, so that >> the result should fit in 32 clusters and avoid EFBIG for this case, >> so it might be worth a try? >> (I still can't think of why the CSUM_TSO bit isn't set for the printf() >> case, but it seems TSO segments could generate EFBIG errors.) >> >> Maybe worth a try, rick >> >> > It's so old that I assume the TSO leak that he speaks of has been >> > patched, but perhaps not. More things to look into tomorrow. >> > >> > >> > >> > >> > >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 03:14:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7D094C8; Sat, 22 Mar 2014 03:14:09 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8FB8E873; Sat, 22 Mar 2014 03:14:09 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so3835591qcz.30 for ; Fri, 21 Mar 2014 20:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kyzbe9v0PmUJj6PoGScFiBO3IUzs5psxU8QxuZDEF4M=; b=UfLY+C+1Nug0Aqij6ahXyjwbGZnOr5/MkUhRXiAS8CRQ1Ai9Q8yxg8s9W0MWNYwrF9 ro4L6nOZ28okI7LAg4P1lwxUpYEEmzjCFHT3M6zM0v8b0Tz7+mLSaIG8M+I4yggXVqGt fdP++SQZcfbCnIjLI/QVd1NUZ2pgbqbcVN1aIQPox105CeSbBxX1Rlm7xq7pTj1CY7X8 wqi8o/fv3LZA/+QJWkmUQQ6LwWwMbA5t7a5bcVCSDRhRuL/l9rIrQcU1voOTIYbF88/f zBjekEbXJHOlmXDmg4Fj8z4R+/jMTZ6XyNYHhRbajEkiPXoBa26rzBqQlRxBY+tJeExE sNFw== MIME-Version: 1.0 X-Received: by 10.224.79.200 with SMTP id q8mr60725415qak.7.1395458048151; Fri, 21 Mar 2014 20:14:08 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 20:14:07 -0700 (PDT) In-Reply-To: <1613242078.1214156.1395455976156.JavaMail.root@uoguelph.ca> References: <1613242078.1214156.1395455976156.JavaMail.root@uoguelph.ca> Date: Sat, 22 Mar 2014 00:14:07 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 03:14:10 -0000 Ah yes, I see it now: Line #658 #if defined(INET) || defined(INET6) /* Initialize to max value. */ if (ifp->if_hw_tsomax == 0) ifp->if_hw_tsomax = IP_MAXPACKET; KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && ifp->if_hw_tsomax >= IP_MAXPACKET / 8, ("%s: tsomax outside of range", __func__)); #endif Should this be the location where it's being set rather than in ixgbe? I would assume that other drivers could fall prey to this issue. Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to make sure VLANs fit? Perhaps there is something in the newer network code that is filling up the frames to the point where they are full - thus a TSO = IP_MAXPACKET is just now causing problems. I'm back on the 9.2-STABLE ixgbe with the tso patch for now. I'll make it run overnight while copying a few TB of data to make sure it's stable there before investigating the 10.0-STABLE driver more. ..and there is still the case of the denied jumbo clusters on boot - something else is off someplace. BTW - In all of this, I did not mention that my ix0 uses a MTU of 9000 - I assume others assumed this. On Fri, Mar 21, 2014 at 11:39 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > It may be a little early, but I think that's it! > > > > It's been running without error for nearly an hour - It's very rare > > it > > would go this long under this much load. > > > > I'm going to let it run longer, then abort and install the kernel > > with the > > extra printfs so I can see what value ifp->if_hw_tsomax is before you > > set > > it. > > > I think you'll just find it set to 0. Code in if_attach_internal() > { in sys/net/if.c } sets it to IP_MAXPACKET (which is 65535) if it > is 0. In other words, if the if_attach routine in the driver doesn't > set it, this code sets it to the maximum possible value. > > Here's the snippet: > /* Initialize to max value. */ > 657 if (ifp->if_hw_tsomax == 0) > 658 ifp->if_hw_tsomax = IP_MAXPACKET; > > Anyhow, this sounds like progress. > > As far as NFS is concerned, I'd rather set it to a smaller value > (maybe 56K) so that m_defrag() doesn't need to be called, but I > suspect others wouldn't like this. > > Hopefully Jack can decide if this patch is ok? > > Thanks yet again for doing this testing, rick > ps: I've attached it again, so Jack (and anyone else who reads this) > can look at it. > pss: Please report if it keeps working for you. > > > It still had netstat -m denied entries on boot, but they are not > > climbing > > like they did before: > > > > > > $ uptime > > 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 > > $ netstat -m > > 21556/7034/28590 mbufs in use (current/cache/total) > > 4080/3076/7156/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16444/118/16562/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161545K/9184K/170729K bytes allocated to network > > (current/cache/total) > > 17972/2230/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > - Started off bad with the 9k denials, but it's not going up! > > > > uptime > > 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 > > root@SAN0:/usr/home/aatech # netstat -m > > 21569/7141/28710 mbufs in use (current/cache/total) > > 4080/3308/7388/6127254 mbuf clusters in use (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16447/121/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161575K/9702K/171277K bytes allocated to network > > (current/cache/total) > > 17972/2261/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into the > > base > > 10.0 code tomorrow. > > > > > > On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem > > wrote: > > > > > Christopher Forgeron wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello all, > > > > > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer > > > > hammer > > > > away at the NFS store overnight - But the problem is still there. > > > > > > > > > > > > From what I read, I think the MJUM9BYTES removal is probably good > > > > cleanup (as long as it doesn't trade performance on a lightly > > > > memory > > > > loaded system for performance on a heavily memory loaded system). > > > > If > > > > I can stabilize my system, I may attempt those benchmarks. > > > > > > > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a > > > > 'clean' > > > > netstat > > > > - Until I can boot and see a 'netstat -m' that looks similar to > > > > that, > > > > I'm going to have this problem. > > > > > > > > > > > > Markus: Do your systems show denied mbufs at boot like mine does? > > > > > > > > > > > > Turning off TSO works for me, but at a performance hit. > > > > > > > > I'll compile Rick's patch (and extra debugging) this morning and > > > > let > > > > you know soon. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch > > > > that > > > > Jack applied. > > > > > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what > > > > netstat -m > > > > shows: > > > > > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > > > 4080/626/4706/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/587 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > > 79068K/2173K/81241K bytes allocated to network > > > > (current/cache/total) > > > > 18831/545/4542 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > Here is an un-patched boot: > > > > > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > > > 4080/3760/7840/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 16439/129/16568/907741 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > > > 161498K/10699K/172197K bytes allocated to network > > > > (current/cache/total) > > > > 18345/155/4099 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem from > > > > the > > > > 9k jumbo cluster into the 4k jumbo cluster? > > > > > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : > > > > Exact > > > > same hardware, revisions, zpool size, etc. Just it's running an > > > > older FreeBSD. > > > > > > > > # uname -a > > > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > > > root@SAN1:/san1 # uptime > > > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, > > > > 0.91 > > > > > > > > root@SAN1:/san1 # netstat -m > > > > 37930/15755/53685 mbufs in use (current/cache/total) > > > > 4080/10996/15076/524288 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/692/692/262144 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 32773/4257/37030/96000 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/508538 16k jumbo clusters in use (current/cache/total/max) > > > > 312599K/67011K/379611K bytes allocated to network > > > > (current/cache/total) > > > > > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > Lastly, please note this link: > > > > > > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > > > > > Hmm, this mentioned the ethernet header being in the TSO segment. I > > > think > > > I already mentioned my TCP/IP is rusty and I know diddly about TSO. > > > However, at a glance it does appear the driver uses ether_output() > > > for > > > TSO segments and, as such, I think an ethernet header is prepended > > > to the > > > TSO segment. (This makes sense, since how else would the hardware > > > know > > > what ethernet header to use for the TCP segments generated.) > > > > > > I think prepending the ethernet header could push the total length > > > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over > > > 64K > > > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so > > > forth. > > > > > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so > > > that > > > the result should fit in 32 clusters and avoid EFBIG for this case, > > > so it might be worth a try? > > > (I still can't think of why the CSUM_TSO bit isn't set for the > > > printf() > > > case, but it seems TSO segments could generate EFBIG errors.) > > > > > > Maybe worth a try, rick > > > > > > > It's so old that I assume the TSO leak that he speaks of has been > > > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > "freebsd-net-unsubscribe@freebsd.org" > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 06:29:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89837167 for ; Sat, 22 Mar 2014 06:29:07 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4314C913 for ; Sat, 22 Mar 2014 06:29:06 +0000 (UTC) Received: from julian-mbp3.pixel8networks.com (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s2M6Srep098999 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 21 Mar 2014 23:28:54 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <532D2D9A.10309@freebsd.org> Date: Fri, 21 Mar 2014 23:28:42 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Mark van der Meulen , Zeus Panchenko Subject: Re: vlanXXX vs ifXX.YY notation References: <20140318074246.8519@smtp.new-ukraine.org> <803C5DD5-47C1-44C9-9212-E17894B15168@fivenynes.com> In-Reply-To: <803C5DD5-47C1-44C9-9212-E17894B15168@fivenynes.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 06:29:07 -0000 On 3/18/14, 12:30 AM, Mark van der Meulen wrote: > Hi, > > I have noticed net graph has issues with device that have a period in their name. Either that is the case or something similar. > > I couldn’t enable netflow on lagg0.X devices and I also couldn’t use some of mpd's features(such as LAC) either. I believe there were other examples but those are the two that came to mind and they both use net graph - I couldn’t work out why things would work on em0 or lagg0 but not on em0.X or lagg0.X and as i did some trawling through logs I saw something which suggested an issue with the names so I changed all my sub interfaces into “vlan” interfaces and I was able to get full functionality in MPD and with net graph netflow. > > Now that I have made the change, I would also say it is a much more manageable way of doing things as well. yes netgraph uses '.' for a special meaning.. I'm surprised that we don't translate it into '_' or something. > > Mark > > > On 18 Mar 2014, at 4:42 pm, Zeus Panchenko wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> hi, >> >> is there any advantage of using vlanXXX vs ifXX.YY notation? >> >> I mean >> >>> ifconfig em0 >> vlan777: flags=8843 metric 0 mtu 1500 >> ether 00:1b:b9:8b:ca:33 >> ... >> vlan: 777 parent interface: em0 >> >> vs >> >> em0.555: flags=8843 metric 0 mtu 1500 >> ether 00:1b:b9:8b:ca:33 >> ... >> vlan: 555 parent interface: em0 >> >> - -- >> Zeus V. Panchenko jid:zeus@im.ibs.dn.ua >> IT Dpt., I.B.S. LLC GMT+2 (EET) >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iEYEARECAAYFAlMn3NYACgkQr3jpPg/3oyqFEgCgj5te5rIwlKqZmlzENBBjLbdG >> j5kAoPv8vFLgrsJRVPeIAzhCvpEC+Xxj >> =BUzc >> -----END PGP SIGNATURE----- >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Mark van der Meulen: Fivenynes - Network Engineer Mob: 0459 312 495 > Phone: 1300 661 902 Fax: 1300 855 328 > Address: Unit 4, 23 Leeds St Rhodes NSW 2138 > Web: www.fivenynes.com > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 10:16:34 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E67055F; Sat, 22 Mar 2014 10:16:34 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3AACC04; Sat, 22 Mar 2014 10:16:33 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id F14CE7F4A0; Sat, 22 Mar 2014 14:16:24 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru F14CE7F4A0 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1395483385; bh=74QjzOSVIaxG6qLB3U75z8Zliqd9nkzqkb3a/HYyDKI=; l=3843; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=urBE7eoQM2w5bUPzLA3fuksByFUt/TMJuKzwxdf0kfJWijTBEnOsGvY5l4LWNMqyR NQ0wfMX1UOdo6kbQgH9XFHq3NPXZnHFJ3NZmLPdbaHolWw4uZSU9lLHEmDlGRJLtU2 OzbHj1sNAVpiJ4Db4maZs++WASUXqbonuEaP8n5E= Message-ID: <532D62F8.4080103@oktetlabs.ru> Date: Sat, 22 Mar 2014 14:16:24 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Gleb Smirnoff Subject: Re: [PATCH 2/6] sfxge: limit software Tx queue size References: <532817F5.8010505@oktetlabs.ru> <20140318132440.GG1499@FreeBSD.org> In-Reply-To: <20140318132440.GG1499@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------080407050309040104050306" Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 10:16:34 -0000 This is a multi-part message in MIME format. --------------080407050309040104050306 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Gleb, On 03/18/2014 05:24 PM, Gleb Smirnoff wrote: > Andrew, > > On Tue, Mar 18, 2014 at 01:55:01PM +0400, Andrew Rybchenko wrote: > A> sfxge: limit software Tx queue size > A> > A> Previous implementation limits put queue size only (when Tx lock can't > A> be acquired), > A> but get queue may grow unboundedly which results in mbuf pools > A> exhaustion and > A> latency growth. > A> > A> Submitted-by: Andrew Rybchenko > A> Sponsored by: Solarflare Communications, Inc. > > The interaction between sfxge_tx_qdpl_put() and sfxge_tx_packet_add() > is quite complex and I couldn't resist from suggesting you to > simplify the code. > > Can you please look into attached patch? > > - Inline sfxge_tx_qdpl_put() into sfxge_tx_packet_add(). > - Simplify the 'locked' logic. > - Add your PATCH 1/6, the mbuf leak fix. > - Add your PATCH 2/6, the SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT check. I don't like "locked" flag passed to qdpl_put() function as well. However, I prefer to keep patches granular and avoid mixing of different changes in single patch. If the initial patch is OK, please, submit it to repository. Then, I'll rebase your patch, discuss it locally and come back to you. BTW, I see that many drivers use drbr for software Tx queue. What do you think, would it be beneficial to use it instead of the list implemented here? Please, find initial patch with few minor fixes (TAB after #define and @->" at " in suggested commit message) attached. Thanks a lot, Andrew. --------------080407050309040104050306 Content-Type: text/x-patch; name="2-sfxge-limit.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="2-sfxge-limit.patch" sfxge: limit software Tx queue size Previous implementation limits put queue size only (when Tx lock can't be acquired), but get queue may grow unboundedly which results in mbuf pools exhaustion and latency growth. Submitted-by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r ff9f5d3dbafe -r 7632a3355224 src/driver/freebsd/sfxge_tx.c --- a/head/sys/dev/sfxge/sfxge_tx.c Tue Mar 04 13:15:13 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.c Wed Mar 05 09:06:01 2014 +0400 @@ -461,6 +461,9 @@ sfxge_tx_qdpl_swizzle(txq); + if (stdp->std_count >= SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT) + return ENOBUFS; + *(stdp->std_getp) = mbuf; stdp->std_getp = &mbuf->m_nextpkt; stdp->std_count++; @@ -480,7 +483,7 @@ old_len = mp->m_pkthdr.csum_data; } else old_len = 0; - if (old_len >= SFXGE_TX_MAX_DEFERRED) + if (old_len >= SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT) return ENOBUFS; mbuf->m_pkthdr.csum_data = old_len + 1; mbuf->m_nextpkt = (void *)old; @@ -507,12 +510,9 @@ */ locked = mtx_trylock(&txq->lock); - /* - * Can only fail if we weren't able to get the lock. - */ if (sfxge_tx_qdpl_put(txq, m, locked) != 0) { - KASSERT(!locked, - ("sfxge_tx_qdpl_put() failed locked")); + if (locked) + mtx_unlock(&txq->lock); rc = ENOBUFS; goto fail; } diff -r ff9f5d3dbafe -r 7632a3355224 src/driver/freebsd/sfxge_tx.h --- a/head/sys/dev/sfxge/sfxge_tx.h Tue Mar 04 13:15:13 2014 +0400 +++ b/head/sys/dev/sfxge/sfxge_tx.h Wed Mar 05 09:06:01 2014 +0400 @@ -75,7 +75,8 @@ enum sfxge_tx_buf_flags flags; }; -#define SFXGE_TX_MAX_DEFERRED 64 +#define SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT 64 +#define SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT 64 /* * Deferred packet list. --------------080407050309040104050306-- From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 11:56:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4226687; Sat, 22 Mar 2014 11:56:00 +0000 (UTC) Received: from mail-qa0-x233.google.com (mail-qa0-x233.google.com [IPv6:2607:f8b0:400d:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 90A50693; Sat, 22 Mar 2014 11:56:00 +0000 (UTC) Received: by mail-qa0-f51.google.com with SMTP id j7so3514456qaq.24 for ; Sat, 22 Mar 2014 04:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=87tE4Xs5tv4+FkHisEDQw7UxbDRLOBYAQGIzHKtajxk=; b=qjojPt/dexou9yySJE2fOey/riVN9IXKrTfsxt3PXVu/QzOKBQ66Jz6cErl7cxfJ+M K7Z98yZ1bxdfTd0HpNz2sOQv7B+PRMFiflnvnLr8TdM2n0B+miSUcOxdlg7ESqp+1Zr0 LfimjaFS7wzGiJsiJLc3jcyOh57JpNumaeoTlkNw55MRhtClqtJ5DlElq63XA6WNn4uv OZq3Y1BMHuTJrxide83Sh5FCJkaaq/uASrG2zUhyDktzrgbIJPTXILWP5YGhgMMYwvQl mEtWzN79RddhzC+aASYW4WF8IwXNAZ4wN5evEDgoFaXorL8xx0TxkjMMTEEY6Wp3EgtY 1BHg== MIME-Version: 1.0 X-Received: by 10.140.107.10 with SMTP id g10mr60579003qgf.63.1395489359873; Sat, 22 Mar 2014 04:55:59 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sat, 22 Mar 2014 04:55:59 -0700 (PDT) In-Reply-To: References: <1613242078.1214156.1395455976156.JavaMail.root@uoguelph.ca> Date: Sat, 22 Mar 2014 08:55:59 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 11:56:01 -0000 Status Update: Hopeful, but not done. So the 9.2-STABLE ixgbe with Rick's TSO patch has been running all night while iometer hammered away at it. It's got over 8 hours of test time on it. It's still running, the CPU queues are not clogged, and everything is functional. However, my ping_logger.py did record 23 incidents of "sendto: File too large" over the 8 hour run. That's really nothing compared to what I usually run into - Normally I'd have 23 incidents within a 5 minute span. During those 23 incidents, (ping_logger.py triggers a cpuset ping) I see it's having the same symptoms of clogging on a few CPU cores. That clogging does go away, a symptom that Markus says he sometimes experiences. So I would say the TSO patch makes things remarkably better, but something else is still up. Unfortunately, with the TSO patch in place it's now harder to trigger the error, so testing will be more difficult. Could someone confirm for me where the jumbo clusters denied/mbuf denied counters come from for netstat? Would it be from a m_defrag call that fails? I feel the netstat -m stats on boot are part of this issue - I was able to greatly reduce them during one of my test iterations. I'm going to see if I can repeat that with the TSO patch. Getting this working on the 10-STABLE ixgbe: Mike's contributed some edits (slightly different thread) I want to try on that driver. At the same time, a diff of 9.2 <-> 10.0 may give hints, as the 10.0 driver with TSO patch has issues quickly, and frequently... it's doing something that aggravates this condition. Thanks for all the help, please keep the suggestions or tidbits of info coming. From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 18:29:05 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E63E8C2A for ; Sat, 22 Mar 2014 18:29:05 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5BCBB9AE for ; Sat, 22 Mar 2014 18:29:04 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s2MISrv9032640 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 22 Mar 2014 22:28:53 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s2MISr7l032639; Sat, 22 Mar 2014 22:28:53 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 22 Mar 2014 22:28:53 +0400 From: Gleb Smirnoff To: Andrew Rybchenko Subject: Re: [PATCH 2/6] sfxge: limit software Tx queue size Message-ID: <20140322182852.GW1499@glebius.int.ru> References: <532817F5.8010505@oktetlabs.ru> <20140318132440.GG1499@FreeBSD.org> <532D62F8.4080103@oktetlabs.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <532D62F8.4080103@oktetlabs.ru> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 18:29:06 -0000 Andrew, On Sat, Mar 22, 2014 at 02:16:24PM +0400, Andrew Rybchenko wrote: A> > The interaction between sfxge_tx_qdpl_put() and sfxge_tx_packet_add() A> > is quite complex and I couldn't resist from suggesting you to A> > simplify the code. A> > A> > Can you please look into attached patch? A> > A> > - Inline sfxge_tx_qdpl_put() into sfxge_tx_packet_add(). A> > - Simplify the 'locked' logic. A> > - Add your PATCH 1/6, the mbuf leak fix. A> > - Add your PATCH 2/6, the SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT check. A> I don't like "locked" flag passed to qdpl_put() function as well. A> However, I prefer to keep patches granular and avoid mixing of different A> changes in single patch. If the initial patch is OK, please, submit it A> to repository. Then, I'll rebase your patch, discuss it locally and come A> back to you. As you wish. Committed, thanks. A> BTW, I see that many drivers use drbr for software Tx queue. What do you A> think, would it be beneficial to use it instead of the list implemented A> here? No idea. drbr(9) is definitely better than ifqueue(9), that's all I can confident state. No idea how drbr(4) compares to the hand made queue of sxfge(4). You've got the hardware, you measure :) Btw, there are some opinions that with modern cards any software queing is a bad idea. Driver should simply hold as much as hardware tx ring can hold. There is no yet stable decision about this, just thoughts floating around. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 19:33:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBA01BF9 for ; Sat, 22 Mar 2014 19:33:39 +0000 (UTC) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD145EC2 for ; Sat, 22 Mar 2014 19:33:39 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id jw12so3958199veb.23 for ; Sat, 22 Mar 2014 12:33:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=8xWN3748T0PPDdZG9WzL/QRb26TjiQ/Ddbn5Jj4Iep0=; b=P+V3qZgIzhqGtvHhCPNnOcTCtLEWn33/mRqZIN2pM9obwf6aoPhtiDmRZu2ar7xSGi UzKOSJdQa0zzFVn2pfQO0iH+8rY9TtTpp5gZMA1aew/xsQVtZfpID5sDPpZCMyVqCJp8 jBIHhPfsPeDAyIfon8ee3vSUfctLjsqMcxTtjnFSlRcCqhHHnDTfVMis1F04Pwuzs++y sgkQzNU+v135fRq6UdU90TYi/lSG1wO8kKYtdbmKxwAEsTqjOUPK/WNZ6GC14POoaPf3 hLmlxSIRO9qOuoWIcK9NaGrFD3/RD4gA3t5xjv/dxb0Y8mUMXva6VOhHACkBxCBhotdv k/Yw== MIME-Version: 1.0 X-Received: by 10.58.202.106 with SMTP id kh10mr14818vec.31.1395516818890; Sat, 22 Mar 2014 12:33:38 -0700 (PDT) Received: by 10.221.56.6 with HTTP; Sat, 22 Mar 2014 12:33:38 -0700 (PDT) Date: Sat, 22 Mar 2014 14:33:38 -0500 Message-ID: Subject: relayd ssl failure From: Thomas Johnson To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 19:33:40 -0000 Hello, I've been trying to sort out an issue with relayd, and I'm just not having any luck. I am setting up a new load-balancer using net/relayd (5.4.20131122_2) on 10.0-RELEASE. My configuration is pretty simple; a pair of web servers , sitting behind the relayd host. I have a httpd instance running on the relayd host as a backup "sorry" server. The following configuration snippet from relayd.conf is literally a copy-paste job from the working http (no ssl) check; essentially just s/http/https/ redirect wwws { listen on $web_addr port https interface em0 tag RELAYD forward to check https "/" code 302 forward to check https "/favicon.ico" code 200 timeout 100 } With this configuration, my check always fails with the following error: hce_notify_done: 1.2.3.4 (ssl connect failed) host 1.2.3.4, check http code use ssl (5ms), state down -> down, availability 0.00% Looking at tcpdump, I see the beginning of an SSL handshake, then the connection is terminated by relayd. I have verified that the web servers are working correctly. Unfortunately, relayd doesn't seem to offer debugging to explain WHY the check is failing. I don't know how relevant it is, but I also have a relayd instance running on a 9.1-RELEASE host (same version of relayd). The topology and relayd config is virtually identical; the web servers are identical images. This instance has it's own quirks (one problem at a time), but the https check is working. Comparing traffic dumps, I see that relayd sends a different (shorter) list of available ciphers in the ssl client hello, and a different cipher is selected by the apache instance in each case, on 9.1: TLS_RSA_WITH_RC4_128_SHA (0x0005) on 10.0: TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011) In the latter case, the dump shows the server sending it's certificate, and the relayd client disconnecting immediately thereafter. It looks like a problem with the certificate, except the certificate is valid, and the same as the 9.1 setup. Any thoughts would be much appreciated. Tom From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 20:38:15 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67615740; Sat, 22 Mar 2014 20:38:15 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 17248387; Sat, 22 Mar 2014 20:38:15 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so4311015qcr.22 for ; Sat, 22 Mar 2014 13:38:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ayWtcgYfUXQgprM/iLR+2pBjNa/HWFfra1EIImYVp+0=; b=eQ4d0bBX4I1Facnaqxda1wbRcJ68MM+IIKYuvv/Vwo8Ni9YAp6ClqyJYgBqul/qlDw YbocsmAtXtEKqglkvoRpe6qwvQk6vXh0ogSm49CAzwGBwZQGyLqYfwYLpNOmF2CSakm6 Y72bjU77ZxWc+Q8QtokvV0fnW1NpCAaa4qYWH2BY1cxYcJZdkg/MMbNOzYfL9OzBgVdl FRZVeHKWCpBYIGPtSSXNWmCmNEcyrHB55v7upVm8GBDwh7rNqAZ6edf4EsxJXAaimB1q AEPJu5V7Ojg1auI5H7GVY0ncKwukXcf95PCdioJWASK34URH2s8vfMhJ2EaTroD1Oegn BLjQ== MIME-Version: 1.0 X-Received: by 10.224.80.134 with SMTP id t6mr65675203qak.34.1395520693685; Sat, 22 Mar 2014 13:38:13 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.8.137 with HTTP; Sat, 22 Mar 2014 13:38:13 -0700 (PDT) In-Reply-To: <20140322182852.GW1499@glebius.int.ru> References: <532817F5.8010505@oktetlabs.ru> <20140318132440.GG1499@FreeBSD.org> <532D62F8.4080103@oktetlabs.ru> <20140322182852.GW1499@glebius.int.ru> Date: Sat, 22 Mar 2014 13:38:13 -0700 X-Google-Sender-Auth: JYCjo42TT6ynuee3GuFRy85_PNk Message-ID: Subject: Re: [PATCH 2/6] sfxge: limit software Tx queue size From: Adrian Chadd To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Cc: "net@freebsd.org" , Andrew Rybchenko X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 20:38:15 -0000 > Btw, there are some opinions that with modern cards any software > queing is a bad idea. Driver should simply hold as much as hardware > tx ring can hold. There is no yet stable decision about this, just > thoughts floating around. The drbr queue gives the driver the ability to source packets from non-local cores for each TX queue. Ie, it can place things into each hardware ring without having to grab any locks to do so, and the TX can be initiated from the correct ring if needs be. So, it's not just doing "software queueing." -) -a From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 21:18:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6856BEFA; Sat, 22 Mar 2014 21:18:22 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 196E18AB; Sat, 22 Mar 2014 21:18:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAA/9LVODaFve/2dsb2JhbABZg0FXgwe4LIZpUYErdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAIFBAEcBIdQCA2rYqIaF4EpjQABARs0B4JvgUkElXOECZB/g0khMYEEOQ X-IronPort-AV: E=Sophos;i="4.97,710,1389762000"; d="scan'208";a="108310633" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 22 Mar 2014 17:18:14 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 8CD42B3F4F; Sat, 22 Mar 2014 17:18:14 -0400 (EDT) Date: Sat, 22 Mar 2014 17:18:14 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1055107814.1401328.1395523094565.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 21:18:22 -0000 Christopher Forgeron wrote: > Status Update: Hopeful, but not done. > > So the 9.2-STABLE ixgbe with Rick's TSO patch has been running all > night > while iometer hammered away at it. It's got over 8 hours of test time > on > it. > > It's still running, the CPU queues are not clogged, and everything is > functional. > > However, my ping_logger.py did record 23 incidents of "sendto: File > too > large" over the 8 hour run. > Well, you could try making if_hw_tsomax somewhat smaller. (I can't see how the packet including ethernet header would be more than 64K with the patch, but?? For example, the ether_output() code can call ng_output() and I have no idea if that might grow the data size of the packet?) To be honest, the optimum for NFS would be setting if_hw_tsomax == 56K, since that would avoid the overhead of the m_defrag() calls. However, it is suboptimal for other TCP transfers. One other thing you could do (if you still have them) is scan the logs for the code with my previous printf() patch and see if there is ever a size > 65549 in it. If there is, then if_hw_tsomax needs to be smaller by at least that size - 65549. (65535 + 14 == 65549) If I were you, I'd try setting it to 57344 (56K) instead of "num_segs * MCLBYTES - ETHER_HDR_LEN" ie. replace ifp->if_hw_tsomax = adapter->num_segs * MCLBYTES - ETHER_HDR_LEN; with ifp->if_hw_tsomax = 57344; in the patch. Then see if all the errors go away. (Jack probably won't like making it that small, but it will show if decreasing it a bit will completely fix the problem.) > That's really nothing compared to what I usually run into - Normally > I'd > have 23 incidents within a 5 minute span. > > During those 23 incidents, (ping_logger.py triggers a cpuset ping) I > see > it's having the same symptoms of clogging on a few CPU cores. That > clogging > does go away, a symptom that Markus says he sometimes experiences. > > So I would say the TSO patch makes things remarkably better, but > something > else is still up. Unfortunately, with the TSO patch in place it's now > harder to trigger the error, so testing will be more difficult. > > Could someone confirm for me where the jumbo clusters denied/mbuf > denied > counters come from for netstat? Would it be from a m_defrag call that > fails? > I'm not familiar enough with the mbuf/uma allocators to "confirm" it, but I believe the "denied" refers to cases where m_getjcl() fails to get a jumbo mbuf and returns NULL. If this were to happen in m_defrag(), it would return NULL and the ix driver returns ENOBUFS, so this is not the case for EFBIG errors. I don't know if increasing the limits for the jumbo mbufs via sysctl will help. If you are using the code without Jack's patch, which uses 9K mbufs, then I think it can fragment the address space and result in no 9K contiguous areas to allocate from. (I'm just going by what Garrett and others have said about this.) > I feel the netstat -m stats on boot are part of this issue - I was > able to > greatly reduce them during one of my test iterations. I'm going to > see if I > can repeat that with the TSO patch. > > Getting this working on the 10-STABLE ixgbe: > > Mike's contributed some edits (slightly different thread) I want to > try on > that driver. At the same time, a diff of 9.2 <-> 10.0 may give hints, > as > the 10.0 driver with TSO patch has issues quickly, and frequently... > it's > doing something that aggravates this condition. > > > Thanks for all the help, please keep the suggestions or tidbits of > info > coming. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 21:41:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 613C570D; Sat, 22 Mar 2014 21:41:25 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E3090A2B; Sat, 22 Mar 2014 21:41:24 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAAIDLlODaFve/2dsb2JhbABZg0FXgwe4LIZpUYErdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAiMGAQkmBggHBAEcBIdEAwkIDatmmwANhwwXgSmLNIFGBgEBGwEzB4JvgUkElXNqgx+LNoVJg0khMXwIFyI X-IronPort-AV: E=Sophos;i="4.97,710,1389762000"; d="scan'208";a="108313150" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 22 Mar 2014 17:41:23 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3CB31B3F26; Sat, 22 Mar 2014 17:41:23 -0400 (EDT) Date: Sat, 22 Mar 2014 17:41:23 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1752303953.1405506.1395524483238.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 21:41:25 -0000 Christopher Forgeron wrote: > > > > > > > Ah yes, I see it now: Line #658 > > #if defined(INET) || defined(INET6) > /* Initialize to max value. */ > if (ifp->if_hw_tsomax == 0) > ifp->if_hw_tsomax = IP_MAXPACKET; > KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > ("%s: tsomax outside of range", __func__)); > #endif > > > Should this be the location where it's being set rather than in > ixgbe? I would assume that other drivers could fall prey to this > issue. > All of this should be prepended with "I'm an NFS guy, not a networking guy, so I might be wrong". Other drivers (and ixgbe for the 82598 chip) can handle a packet that is in more than 32 mbufs. (I think the 82598 handles 100, grep for SCATTER in *.h in sys/dev/ixgbe.) Now, since several drivers do have this 32 mbufs limit, I can see an argument for making the default a little smaller to make these work, since the driver can override the default. (About now someone usually jumps in and says something along the lines of "You can't do that until all the drivers that can handle IP_MAXPACKET are fixed to set if_hw_tsomax" and since I can't fix drivers I can't test, that pretty much puts a stop on it.) You see the problem isn't that IP_MAXPACKET is too big, but that the hardware has a limit of 32 non-contiguous chunks (mbufs)/packet and 32 * MCLBYTES = 64K. (Hardware/network drivers that can handle 35 or more chunks (they like to call them transmit segments, although ixgbe uses the term scatter) shouldn't have any problems.) I have an untested patch that adds a tsomaxseg count to use along with tsomax bytes so that a driver could inform tcp_output() it can only handle 32 mbufs and then tcp_output() would limit a TSO segment using both, but I can't test it, so who knows when/if that might happen. I also have a patch that modifies NFS to use pagesize clusters (reducing the mbuf count in the list), but that one causes grief when testing on an i386 (seems to run out of kernel memory to the point where it can't allocate something called "boundary tags" and pretty well wedges the machine at that point.) Since I don't know how to fix this (I thought of making the patch "amd64 only"), I can't really commit this to head, either. As such, I think it's going to be "fix the drivers one at a time" and tell folks to "disable TSO or limit rsize,wsize to 32K" when they run into trouble. (As you might have guessed, I'd rather just be "the NFS guy", but since NFS "triggers the problem" I\m kinda stuck with it;-) > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to > make sure VLANs fit? > No idea. (I wouldn't know a VLAN if it jumped up and tried to bite me on the nose.;-) So, I have no idea what does this, but if it means the total ethernet header size can be > 14bytes, then I'd agree. > Perhaps there is something in the newer network code that is filling > up the frames to the point where they are full - thus a TSO = > IP_MAXPACKET is just now causing problems. > Yea, I have no idea why this didn't bite running 9.1. (Did 9.1 have TSO enabled by default?) > I'm back on the 9.2-STABLE ixgbe with the tso patch for now. I'll > make it run overnight while copying a few TB of data to make sure > it's stable there before investigating the 10.0-STABLE driver more. > I have no idea what needs to be changed to back-port a 10.0 driver to 9.2. Good luck with it and thanks for what you've learned sofar, rick > ..and there is still the case of the denied jumbo clusters on boot - > something else is off someplace. > > BTW - In all of this, I did not mention that my ix0 uses a MTU of > 9000 - I assume others assumed this. > > > > > > > > > On Fri, Mar 21, 2014 at 11:39 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > Christopher Forgeron wrote: > > It may be a little early, but I think that's it! > > > > It's been running without error for nearly an hour - It's very rare > > it > > would go this long under this much load. > > > > I'm going to let it run longer, then abort and install the kernel > > with the > > extra printfs so I can see what value ifp->if_hw_tsomax is before > > you > > set > > it. > > > I think you'll just find it set to 0. Code in if_attach_internal() > { in sys/net/if.c } sets it to IP_MAXPACKET (which is 65535) if it > is 0. In other words, if the if_attach routine in the driver doesn't > set it, this code sets it to the maximum possible value. > > Here's the snippet: > /* Initialize to max value. */ > 657 if (ifp->if_hw_tsomax == 0) > 658 ifp->if_hw_tsomax = IP_MAXPACKET; > > Anyhow, this sounds like progress. > > As far as NFS is concerned, I'd rather set it to a smaller value > (maybe 56K) so that m_defrag() doesn't need to be called, but I > suspect others wouldn't like this. > > Hopefully Jack can decide if this patch is ok? > > Thanks yet again for doing this testing, rick > ps: I've attached it again, so Jack (and anyone else who reads this) > can look at it. > pss: Please report if it keeps working for you. > > > > > It still had netstat -m denied entries on boot, but they are not > > climbing > > like they did before: > > > > > > $ uptime > > 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 > > $ netstat -m > > 21556/7034/28590 mbufs in use (current/cache/total) > > 4080/3076/7156/6127254 mbuf clusters in use > > (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16444/118/16562/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161545K/9184K/170729K bytes allocated to network > > (current/cache/total) > > 17972/2230/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > - Started off bad with the 9k denials, but it's not going up! > > > > uptime > > 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 > > root@SAN0:/usr/home/aatech # netstat -m > > 21569/7141/28710 mbufs in use (current/cache/total) > > 4080/3308/7388/6127254 mbuf clusters in use > > (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16447/121/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161575K/9702K/171277K bytes allocated to network > > (current/cache/total) > > 17972/2261/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into > > the > > base > > 10.0 code tomorrow. > > > > > > On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem < > > rmacklem@uoguelph.ca > > > wrote: > > > > > Christopher Forgeron wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello all, > > > > > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer > > > > hammer > > > > away at the NFS store overnight - But the problem is still > > > > there. > > > > > > > > > > > > From what I read, I think the MJUM9BYTES removal is probably > > > > good > > > > cleanup (as long as it doesn't trade performance on a lightly > > > > memory > > > > loaded system for performance on a heavily memory loaded > > > > system). > > > > If > > > > I can stabilize my system, I may attempt those benchmarks. > > > > > > > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a > > > > 'clean' > > > > netstat > > > > - Until I can boot and see a 'netstat -m' that looks similar to > > > > that, > > > > I'm going to have this problem. > > > > > > > > > > > > Markus: Do your systems show denied mbufs at boot like mine > > > > does? > > > > > > > > > > > > Turning off TSO works for me, but at a performance hit. > > > > > > > > I'll compile Rick's patch (and extra debugging) this morning > > > > and > > > > let > > > > you know soon. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch > > > > that > > > > Jack applied. > > > > > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what > > > > netstat -m > > > > shows: > > > > > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > > > 4080/626/4706/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/587 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 79068K/2173K/81241K bytes allocated to network > > > > (current/cache/total) > > > > 18831/545/4542 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > Here is an un-patched boot: > > > > > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > > > 4080/3760/7840/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 16439/129/16568/907741 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 161498K/10699K/172197K bytes allocated to network > > > > (current/cache/total) > > > > 18345/155/4099 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem > > > > from > > > > the > > > > 9k jumbo cluster into the 4k jumbo cluster? > > > > > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : > > > > Exact > > > > same hardware, revisions, zpool size, etc. Just it's running an > > > > older FreeBSD. > > > > > > > > # uname -a > > > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > > > root@SAN1:/san1 # uptime > > > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, > > > > 0.91 > > > > > > > > root@SAN1:/san1 # netstat -m > > > > 37930/15755/53685 mbufs in use (current/cache/total) > > > > 4080/10996/15076/524288 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/692/692/262144 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 32773/4257/37030/96000 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/508538 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 312599K/67011K/379611K bytes allocated to network > > > > (current/cache/total) > > > > > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > Lastly, please note this link: > > > > > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > > > > > Hmm, this mentioned the ethernet header being in the TSO segment. > > > I > > > think > > > I already mentioned my TCP/IP is rusty and I know diddly about > > > TSO. > > > However, at a glance it does appear the driver uses > > > ether_output() > > > for > > > TSO segments and, as such, I think an ethernet header is > > > prepended > > > to the > > > TSO segment. (This makes sense, since how else would the hardware > > > know > > > what ethernet header to use for the TCP segments generated.) > > > > > > I think prepending the ethernet header could push the total > > > length > > > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over > > > 64K > > > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so > > > forth. > > > > > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so > > > that > > > the result should fit in 32 clusters and avoid EFBIG for this > > > case, > > > so it might be worth a try? > > > (I still can't think of why the CSUM_TSO bit isn't set for the > > > printf() > > > case, but it seems TSO segments could generate EFBIG errors.) > > > > > > Maybe worth a try, rick > > > > > > > It's so old that I assume the TSO leak that he speaks of has > > > > been > > > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > " freebsd-net-unsubscribe@freebsd.org " > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > " freebsd-net-unsubscribe@freebsd.org " > > > > From owner-freebsd-net@FreeBSD.ORG Sat Mar 22 22:44:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AC01763 for ; Sat, 22 Mar 2014 22:44:25 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 32FD4EF6 for ; Sat, 22 Mar 2014 22:44:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s2MMiG1Z069016 for ; Sat, 22 Mar 2014 17:44:17 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sat Mar 22 17:44:17 2014 Message-ID: <532E123B.3060702@denninger.net> Date: Sat, 22 Mar 2014 17:44:11 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Strongswan problem (used to work for client NAT to the Internet, no longer does) Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms090804020106020602070602" X-Antivirus: avast! (VPS 140322-1, 03/22/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 22:44:25 -0000 This is a cryptographically signed message in MIME format. --------------ms090804020106020602070602 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable FreeBSD-STABLE 10 r263037M Configuration has outside IPSEC connections coming in to Strongswan=20 which should then be able to NAT back out to the Internet. The premise=20 here is that "roaming" people may connect to this box and obtain both=20 access to "inside" resources and outside Internet access, since the=20 client points default at the IPSEC'd connection. This used to work on 9.1, but am uncertain whether it has since. It does NOT under 10.0. [root@Gateway /disk/karl]# ipsec status Security Associations (1 up, 0 connecting): XX[3]: ESTABLISHED 5 minutes ago, x.x.x.x[C=3DUS, ST=3Dxx, O=3Dx= xx,=20 CN=3Dthe.dom.ain, E=3Dxxxxx]...172.56.20.235[xxxxx] BB10{3}: INSTALLED, TUNNEL, ESP in UDP SPIs: c90f5d36_i 4160832= e_o BB10{3}: 0.0.0.0/0 =3D=3D=3D 192.168.2.1/32 [root@Gateway /disk/karl]# Ok, so theoretically there's a default route from the device, which is=20 fine. And the device (on 192.168.2.1, which was dynamically assigned=20 from a pool) can see anything internal on this external box (x.x.x.x) I = can successfully mount a samba share, for example, on a server that is=20 inside the firewall and I can also see the DNS resolver on the gateway. However, let's now try to go out to an external location via an IP=20 address. We'll watch it using TCPDUMP on em1, the interface that the=20 packets are going to be emitted from toward the Internet: [root@NewFS /disk/karl]# tcpdump -i em1 host 173.245.6.228 17:07:06.524487 IP 192.168.2.1.14927 > 173.245.6.228.http: Flags [S],=20 seq 150879940, win 65535, options [mss 1188,nop,wscale=20 6,sackOK,nop,nop,nop,nop,TS val 778723856 ecr 0], length 0 17:07:19.741732 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 6,sackOK,nop,nop,nop,nop,TS val 778723883 ecr 0], length 0 17:07:20.797330 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 6,sackOK,nop,nop,nop,nop,TS val 778723884 ecr 0], length 0 17:07:22.706839 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 6,sackOK,nop,nop,nop,nop,TS val 778723888 ecr 0], length 0 Ehhh...... that's no good. natd never gets the packets and never=20 translates them; it sure looks like we're trying to blow a private IP=20 number out the wire despite: 01700 divert 8668 ip4 from any to any via em1 01710 divert 8668 ip4 from 192.168.2.0/24 to any via em1 and ahead of that (to prevent exactly this sort of thing) 01120 deny log ip from any to 192.168.0.0/16 via em1 not ipsec Which by the way does NOT log anything. net.inet.ip.fw.one_pass: 0 net.inet.ip.fw.autoinc_step: 100 net.inet.ip.fw.verbose: 1 net.inet.ip.fw.verbose_limit: 0 net.inet.ip.fw.default_rule: 65535 net.inet.ip.fw.tables_max: 128 net.inet.ip.fw.default_to_accept: 0 net.inet.ip.fw.static_count: 108 net.inet.ip.fw.enable: 1 net.inet.ip.fw.dyn_buckets: 256 net.inet.ip.fw.curr_dyn_buckets: 256 net.inet.ip.fw.dyn_count: 3 net.inet.ip.fw.dyn_max: 4096 net.inet.ip.fw.dyn_ack_lifetime: 300 net.inet.ip.fw.dyn_syn_lifetime: 20 net.inet.ip.fw.dyn_fin_lifetime: 1 net.inet.ip.fw.dyn_rst_lifetime: 1 net.inet.ip.fw.dyn_udp_lifetime: 10 net.inet.ip.fw.dyn_short_lifetime: 5 net.inet.ip.fw.dyn_keepalive: 1 one_pass is off. And there is nothing being blocked either; all the "deny" entries have=20 "log" on them and there's nothing showing up in the logfile (there's=20 plenty of random people trying to port scan me that ARE showing up,=20 however, so I know the log is working properly.) It *looks* like anything coming in through IPSEC and being decoded in=20 there never goes through the ipfw chain at all..... --=20 -- Karl karl@denninger.net --------------ms090804020106020602070602 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMjIyMjQ0MTFaMCMGCSqGSIb3DQEJBDEW BBSPxmgqculdslBLTHfYRIFQAK6NWjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIABQS1SNZBlItkh6SE6a8cLjW7RrEo 12syIF9jbYuy2UPOFVaiNTc4GiUv4QkoD/lkdn0v9Jwlu1x4htTF8ThO2ANFIcgMMTrcZ5z5 uIM7W7To+PktoBzqCWC6KUjc9AFgXN+6jke3fkwoe8NFYESib/j/84F8wAXuhyQba0QybaBL ee4S773W1Qnt7oqYngqHHY8cjFE2yGWMVhY5e8DZDhdp79ut5NcJ74Kh2IwtglGHVWL4AQak nxjK73b6FUGTGpHUxI+GMAYJIssDu62pglE+upkGREDpvQ8d0/LTb6MS0l74KNp5ukj/PxDX jIHSeOFZGEqCQt3jiZF234Pcap7Ci9bshC1Tuqun5ZBWC1z6yFr65nHU5BJ5Qk39I+AxfWaE a+FjqpYZXhobR/rKe/+nVar2CrNYB3EK9jxnknup54wwKVu134fRRzsIQFi3BvG2n+h/HVDw Q1cAmQJJa/AY9YccIc5MBGBFP9dmvKCZWH/zLDeq5t75sQ8Rgrqfy/EbMGP8L33Ez72aFpEk lzeORfnW/KsSJjmwUfQ1OlfuytR6slHLhBr0z2B8NXO1GuM0Ocpm/GPBXCXFfjZke1xc4WbK RJMbtvrPCV+cFzm7oMnAgDhDWZCMuxlAxcGbjvWukq5jZOW4ATo7VRiAv6gMzwp1kxaYsOeP FHCxITcAAAAAAAA= --------------ms090804020106020602070602-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 01:28:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B24BE8 for ; Sun, 23 Mar 2014 01:28:33 +0000 (UTC) Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46]) by mx1.freebsd.org (Postfix) with SMTP id F03BDD74 for ; Sun, 23 Mar 2014 01:28:32 +0000 (UTC) Received: (qmail 24135 invoked by uid 1001); 23 Mar 2014 01:28:25 -0000 Delivered-To: qmda-intercept-freebsd-net@freebsd.org DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; b=zNOzNFN8vdL6ugrQ/DwFBiKLX1991OROYYh8E0EnAedR3QGn3kv2aAGT6xKfZXVO; Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys DomainKey-Trace-MD: h=12; b=20; l=C18R71D32M65F38T27S46R48M17C39C27I49; Comments: QMDA 0.3 Received: (qmail 24128 invoked by uid 1001); 23 Mar 2014 01:28:24 -0000 Date: 23 Mar 2014 01:28:24 +0000 Message-ID: <20140323012824.24127.qmail@f5-external.bushwire.net> From: "Mark Delany" To: freebsd-net@freebsd.org Subject: Re: Minor nits with netmap(4) manpage References: <20140319111638.GN75328@qmda.emu.st> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140319111638.GN75328@qmda.emu.st> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 01:28:33 -0000 > 2) the manpage refers to NR_RING_NIC_SW when I think it means > NR_REG_NIC_SW. Found another. 2a) manpage refers to NR_REG_SW_NIC when the include file has NR_REG_SW To summarize: manpage include NR_REG_ALL_NIC NR_REG_SW_NIC NR_REG_SW NR_RING_NIC_SW NR_REG_NIC_SW NR_REG_ONE_NIC NR_REG_PIPE_MASTER NR_REG_PIPE_SLAVE Mark. From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 02:58:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AEC6E62; Sun, 23 Mar 2014 02:58:19 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 0C440366; Sun, 23 Mar 2014 02:58:18 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEABFNLlODaFve/2dsb2JhbABYg0FXgwe4LIZpUYEpdIIlAQEBAwEBAQEgBCcgCwUWDgoRGQIEHwYBCSYGCAcEARwEh0QDCQgNq1uaYQ2HBReMXYFMAQEbGRsHgm+BSQSQU4UgaoMfizaFSYNJITGBBDk X-IronPort-AV: E=Sophos;i="4.97,711,1389762000"; d="scan'208";a="108063977" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 22 Mar 2014 22:58:11 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 33501B3F19; Sat, 22 Mar 2014 22:58:11 -0400 (EDT) Date: Sat, 22 Mar 2014 22:58:11 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1532337187.1452820.1395543491196.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1452818_1467930094.1395543491194" X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 02:58:19 -0000 ------=_Part_1452818_1467930094.1395543491194 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > > > > > > > Ah yes, I see it now: Line #658 > > #if defined(INET) || defined(INET6) > /* Initialize to max value. */ > if (ifp->if_hw_tsomax == 0) > ifp->if_hw_tsomax = IP_MAXPACKET; > KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > ("%s: tsomax outside of range", __func__)); > #endif > > > Should this be the location where it's being set rather than in > ixgbe? I would assume that other drivers could fall prey to this > issue. > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to > make sure VLANs fit? > I took a look and, yes, this does seem to be needed. It will only be needed for the case where a vlan is in use and hwtagging is disabled, if I read the code correctly. Do you use vlans? I've attached an updated patch. It might be nice to have the printf() patch in the driver too, so we can see how big the ones that are too big are? Good luck with it, rick > Perhaps there is something in the newer network code that is filling > up the frames to the point where they are full - thus a TSO = > IP_MAXPACKET is just now causing problems. > > I'm back on the 9.2-STABLE ixgbe with the tso patch for now. I'll > make it run overnight while copying a few TB of data to make sure > it's stable there before investigating the 10.0-STABLE driver more. > > ..and there is still the case of the denied jumbo clusters on boot - > something else is off someplace. > > BTW - In all of this, I did not mention that my ix0 uses a MTU of > 9000 - I assume others assumed this. > > > > > > > > > On Fri, Mar 21, 2014 at 11:39 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > Christopher Forgeron wrote: > > It may be a little early, but I think that's it! > > > > It's been running without error for nearly an hour - It's very rare > > it > > would go this long under this much load. > > > > I'm going to let it run longer, then abort and install the kernel > > with the > > extra printfs so I can see what value ifp->if_hw_tsomax is before > > you > > set > > it. > > > I think you'll just find it set to 0. Code in if_attach_internal() > { in sys/net/if.c } sets it to IP_MAXPACKET (which is 65535) if it > is 0. In other words, if the if_attach routine in the driver doesn't > set it, this code sets it to the maximum possible value. > > Here's the snippet: > /* Initialize to max value. */ > 657 if (ifp->if_hw_tsomax == 0) > 658 ifp->if_hw_tsomax = IP_MAXPACKET; > > Anyhow, this sounds like progress. > > As far as NFS is concerned, I'd rather set it to a smaller value > (maybe 56K) so that m_defrag() doesn't need to be called, but I > suspect others wouldn't like this. > > Hopefully Jack can decide if this patch is ok? > > Thanks yet again for doing this testing, rick > ps: I've attached it again, so Jack (and anyone else who reads this) > can look at it. > pss: Please report if it keeps working for you. > > > > > It still had netstat -m denied entries on boot, but they are not > > climbing > > like they did before: > > > > > > $ uptime > > 9:32PM up 25 mins, 4 users, load averages: 2.43, 6.15, 4.65 > > $ netstat -m > > 21556/7034/28590 mbufs in use (current/cache/total) > > 4080/3076/7156/6127254 mbuf clusters in use > > (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16444/118/16562/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161545K/9184K/170729K bytes allocated to network > > (current/cache/total) > > 17972/2230/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8909/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > - Started off bad with the 9k denials, but it's not going up! > > > > uptime > > 10:20PM up 1:13, 6 users, load averages: 2.10, 3.15, 3.67 > > root@SAN0:/usr/home/aatech # netstat -m > > 21569/7141/28710 mbufs in use (current/cache/total) > > 4080/3308/7388/6127254 mbuf clusters in use > > (current/cache/total/max) > > 4080/2281 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/53/53/3063627 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 16447/121/16568/907741 9k jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/510604 16k jumbo clusters in use (current/cache/total/max) > > 161575K/9702K/171277K bytes allocated to network > > (current/cache/total) > > 17972/2261/4111 requests for mbufs denied > > (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > 35/8913/0 requests for jumbo clusters denied (4k/9k/16k) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > > > This is the 9.2 ixgbe that I'm patching into 10.0, I'll move into > > the > > base > > 10.0 code tomorrow. > > > > > > On Fri, Mar 21, 2014 at 8:44 PM, Rick Macklem < > > rmacklem@uoguelph.ca > > > wrote: > > > > > Christopher Forgeron wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello all, > > > > > > > > I ran Jack's ixgbe MJUM9BYTES removal patch, and let iometer > > > > hammer > > > > away at the NFS store overnight - But the problem is still > > > > there. > > > > > > > > > > > > From what I read, I think the MJUM9BYTES removal is probably > > > > good > > > > cleanup (as long as it doesn't trade performance on a lightly > > > > memory > > > > loaded system for performance on a heavily memory loaded > > > > system). > > > > If > > > > I can stabilize my system, I may attempt those benchmarks. > > > > > > > > > > > > I think the fix will be obvious at boot for me - My 9.2 has a > > > > 'clean' > > > > netstat > > > > - Until I can boot and see a 'netstat -m' that looks similar to > > > > that, > > > > I'm going to have this problem. > > > > > > > > > > > > Markus: Do your systems show denied mbufs at boot like mine > > > > does? > > > > > > > > > > > > Turning off TSO works for me, but at a performance hit. > > > > > > > > I'll compile Rick's patch (and extra debugging) this morning > > > > and > > > > let > > > > you know soon. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 11:47 PM, Christopher Forgeron < > > > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW - I think this will end up being a TSO issue, not the patch > > > > that > > > > Jack applied. > > > > > > > > When I boot Jack's patch (MJUM9BYTES removal) this is what > > > > netstat -m > > > > shows: > > > > > > > > 21489/2886/24375 mbufs in use (current/cache/total) > > > > 4080/626/4706/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/587 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 16384/50/16434/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 0/0/0/907741 9k jumbo clusters in use (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 79068K/2173K/81241K bytes allocated to network > > > > (current/cache/total) > > > > 18831/545/4542 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 15626/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > Here is an un-patched boot: > > > > > > > > 21550/7400/28950 mbufs in use (current/cache/total) > > > > 4080/3760/7840/6127254 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/2769 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/42/42/3063627 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 16439/129/16568/907741 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/510604 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 161498K/10699K/172197K bytes allocated to network > > > > (current/cache/total) > > > > 18345/155/4099 requests for mbufs denied > > > > (mbufs/clusters/mbuf+clusters) > > > > > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 3/3723/0 requests for jumbo clusters denied (4k/9k/16k) > > > > > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > > > > > > > > > > > > > See how removing the MJUM9BYTES is just pushing the problem > > > > from > > > > the > > > > 9k jumbo cluster into the 4k jumbo cluster? > > > > > > > > Compare this to my FreeBSD 9.2 STABLE machine from ~ Dec 2013 : > > > > Exact > > > > same hardware, revisions, zpool size, etc. Just it's running an > > > > older FreeBSD. > > > > > > > > # uname -a > > > > FreeBSD SAN1.XXXXX 9.2-STABLE FreeBSD 9.2-STABLE #0: Wed Dec 25 > > > > 15:12:14 AST 2013 aatech@FreeBSD-Update > > > > Server:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > > > root@SAN1:/san1 # uptime > > > > 7:44AM up 58 days, 38 mins, 4 users, load averages: 0.42, 0.80, > > > > 0.91 > > > > > > > > root@SAN1:/san1 # netstat -m > > > > 37930/15755/53685 mbufs in use (current/cache/total) > > > > 4080/10996/15076/524288 mbuf clusters in use > > > > (current/cache/total/max) > > > > 4080/5775 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/692/692/262144 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 32773/4257/37030/96000 9k jumbo clusters in use > > > > (current/cache/total/max) > > > > > > > > 0/0/0/508538 16k jumbo clusters in use > > > > (current/cache/total/max) > > > > 312599K/67011K/379611K bytes allocated to network > > > > (current/cache/total) > > > > > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > Lastly, please note this link: > > > > > > > > http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033660.html > > > > > > > Hmm, this mentioned the ethernet header being in the TSO segment. > > > I > > > think > > > I already mentioned my TCP/IP is rusty and I know diddly about > > > TSO. > > > However, at a glance it does appear the driver uses > > > ether_output() > > > for > > > TSO segments and, as such, I think an ethernet header is > > > prepended > > > to the > > > TSO segment. (This makes sense, since how else would the hardware > > > know > > > what ethernet header to use for the TCP segments generated.) > > > > > > I think prepending the ethernet header could push the total > > > length > > > over 64K, given a default if_hw_tsomax == IP_MAXPACKET. And over > > > 64K > > > isn't going to fit in 32 * 2K (mclbytes) clusters, etc and so > > > forth. > > > > > > Anyhow, I think the attached patch will reduce if_hw_tsomax, so > > > that > > > the result should fit in 32 clusters and avoid EFBIG for this > > > case, > > > so it might be worth a try? > > > (I still can't think of why the CSUM_TSO bit isn't set for the > > > printf() > > > case, but it seems TSO segments could generate EFBIG errors.) > > > > > > Maybe worth a try, rick > > > > > > > It's so old that I assume the TSO leak that he speaks of has > > > > been > > > > patched, but perhaps not. More things to look into tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > " freebsd-net-unsubscribe@freebsd.org " > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > " freebsd-net-unsubscribe@freebsd.org " > > > > ------=_Part_1452818_1467930094.1395543491194 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0yMiAyMjo0NDo1My4wMDAwMDAwMDAg LTA0MDAKQEAgLTI2MTQsNiArMjYxNCwxMCBAQCBpeGdiZV9zZXR1cF9pbnRlcmZhY2UoZGV2aWNl X3QgZGV2LCBzdHJ1CiAJaWZwLT5pZl9zbmQuaWZxX2Rydl9tYXhsZW4gPSBhZGFwdGVyLT5udW1f dHhfZGVzYyAtIDI7CiAJSUZRX1NFVF9SRUFEWSgmaWZwLT5pZl9zbmQpOwogI2VuZGlmCisJaWYg KChhZGFwdGVyLT5udW1fc2VncyAqIE1DTEJZVEVTIC0gKEVUSEVSX0hEUl9MRU4gKworCSAgICBF VEhFUl9WTEFOX0VOQ0FQX0xFTikpIDwgSVBfTUFYUEFDS0VUKQorCQlpZnAtPmlmX2h3X3Rzb21h eCA9IGFkYXB0ZXItPm51bV9zZWdzICogTUNMQllURVMgLQorCQkgICAgKEVUSEVSX0hEUl9MRU4g KyBFVEhFUl9WTEFOX0VOQ0FQX0xFTik7CiAKIAlldGhlcl9pZmF0dGFjaChpZnAsIGFkYXB0ZXIt Pmh3Lm1hYy5hZGRyKTsKIAo= ------=_Part_1452818_1467930094.1395543491194-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 05:01:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 910656A3 for ; Sun, 23 Mar 2014 05:01:27 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B9CCE43 for ; Sun, 23 Mar 2014 05:01:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s2N51NfJ003522 for ; Sun, 23 Mar 2014 00:01:23 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sun Mar 23 00:01:23 2014 Message-ID: <532E6A9D.9040609@denninger.net> Date: Sun, 23 Mar 2014 00:01:17 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Strongswan problem (used to work for client NAT to the Internet, no longer does) References: <532E123B.3060702@denninger.net> In-Reply-To: <532E123B.3060702@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms070109060009060403070404" X-Antivirus: avast! (VPS 140322-1, 03/22/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 05:01:27 -0000 This is a cryptographically signed message in MIME format. --------------ms070109060009060403070404 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 3/22/2014 5:44 PM, Karl Denninger wrote: > FreeBSD-STABLE 10 r263037M > > Configuration has outside IPSEC connections coming in to Strongswan=20 > which should then be able to NAT back out to the Internet. The=20 > premise here is that "roaming" people may connect to this box and=20 > obtain both access to "inside" resources and outside Internet access,=20 > since the client points default at the IPSEC'd connection. > > This used to work on 9.1, but am uncertain whether it has since. > > It does NOT under 10.0. > > [root@Gateway /disk/karl]# ipsec status > Security Associations (1 up, 0 connecting): > XX[3]: ESTABLISHED 5 minutes ago, x.x.x.x[C=3DUS, ST=3Dxx, O=3D= xxx,=20 > CN=3Dthe.dom.ain, E=3Dxxxxx]...172.56.20.235[xxxxx] > BB10{3}: INSTALLED, TUNNEL, ESP in UDP SPIs: c90f5d36_i=20 > 4160832e_o > BB10{3}: 0.0.0.0/0 =3D=3D=3D 192.168.2.1/32 > [root@Gateway /disk/karl]# > > Ok, so theoretically there's a default route from the device, which is = > fine. And the device (on 192.168.2.1, which was dynamically assigned=20 > from a pool) can see anything internal on this external box (x.x.x.x) = > I can successfully mount a samba share, for example, on a server that=20 > is inside the firewall and I can also see the DNS resolver on the=20 > gateway. > > However, let's now try to go out to an external location via an IP=20 > address. We'll watch it using TCPDUMP on em1, the interface that the=20 > packets are going to be emitted from toward the Internet: > > [root@NewFS /disk/karl]# tcpdump -i em1 host 173.245.6.228 > > > 17:07:06.524487 IP 192.168.2.1.14927 > 173.245.6.228.http: Flags [S],=20 > seq 150879940, win 65535, options [mss 1188,nop,wscale=20 > 6,sackOK,nop,nop,nop,nop,TS val 778723856 ecr 0], length 0 > 17:07:19.741732 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 > seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 > 6,sackOK,nop,nop,nop,nop,TS val 778723883 ecr 0], length 0 > 17:07:20.797330 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 > seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 > 6,sackOK,nop,nop,nop,nop,TS val 778723884 ecr 0], length 0 > 17:07:22.706839 IP 192.168.2.1.16697 > 173.245.6.228.http: Flags [S],=20 > seq 2513053141, win 65535, options [mss 1188,nop,wscale=20 > 6,sackOK,nop,nop,nop,nop,TS val 778723888 ecr 0], length 0 > > Ehhh...... that's no good. natd never gets the packets and never=20 > translates them; it sure looks like we're trying to blow a private IP=20 > number out the wire despite: > > 01700 divert 8668 ip4 from any to any via em1 > 01710 divert 8668 ip4 from 192.168.2.0/24 to any via em1 > > and ahead of that (to prevent exactly this sort of thing) > > 01120 deny log ip from any to 192.168.0.0/16 via em1 not ipsec > > Which by the way does NOT log anything. > > > net.inet.ip.fw.one_pass: 0 > net.inet.ip.fw.autoinc_step: 100 > net.inet.ip.fw.verbose: 1 > net.inet.ip.fw.verbose_limit: 0 > net.inet.ip.fw.default_rule: 65535 > net.inet.ip.fw.tables_max: 128 > net.inet.ip.fw.default_to_accept: 0 > net.inet.ip.fw.static_count: 108 > net.inet.ip.fw.enable: 1 > net.inet.ip.fw.dyn_buckets: 256 > net.inet.ip.fw.curr_dyn_buckets: 256 > net.inet.ip.fw.dyn_count: 3 > net.inet.ip.fw.dyn_max: 4096 > net.inet.ip.fw.dyn_ack_lifetime: 300 > net.inet.ip.fw.dyn_syn_lifetime: 20 > net.inet.ip.fw.dyn_fin_lifetime: 1 > net.inet.ip.fw.dyn_rst_lifetime: 1 > net.inet.ip.fw.dyn_udp_lifetime: 10 > net.inet.ip.fw.dyn_short_lifetime: 5 > net.inet.ip.fw.dyn_keepalive: 1 > > one_pass is off. > > And there is nothing being blocked either; all the "deny" entries have = > "log" on them and there's nothing showing up in the logfile (there's=20 > plenty of random people trying to port scan me that ARE showing up,=20 > however, so I know the log is working properly.) > > It *looks* like anything coming in through IPSEC and being decoded in=20 > there never goes through the ipfw chain at all..... > This may be addressed by PR185876.... checking. --=20 -- Karl karl@denninger.net --------------ms070109060009060403070404 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMjMwNTAxMTdaMCMGCSqGSIb3DQEJBDEW BBSnF4gpq+QxzBnFZZdMYxzHyLB9rjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAS3ji6QF0FOFdasCNiHcA4YzMiQ4Q xQLV4ftxCXMIN/MHarvO/W3ArN8eaH+WAUpaa6Egq7dkcUYVNEn6JYjqmsD+YxN4L5xVHQ7X FnXghuSXAjCUf0/HtN5xJaYC4mKvpAscW1ss2DiapBZ/YSjUSpHy9xWCzCIAwT2bTU1jX9eb wsjfKeS858+x4tzyX3Z6yn44W9naHYUqPdX8BO5HawsuPrjTNhqx9hJ5rUWmnn3epqfWzYqg qsFQpKkmadSW71RjY+VXXDV8d8pL9CGVPLj1R9ZYgsqf9p/UP1wIxOIgG+sfNsDeO6BqFrgM BLx20o0tOqJm3L94bZjrq33WTMYxrS882cgnXC9/ZWbSIlScDTVNntpYPRuRRzV0X51MK2Dj XW/Ucx5vDERnRbrFXEVzyWWIXA6decWn9zdvoVMGjcorWm5RQCpnL6Y1wklDY2EEjJ7U9C+M gsUDyL3mWv1M7F7hoCPr9+yu5e20Noe9XdjHsk20Tfgc6QExxVfjKzOF8Yg50cLTZTYoiUwK BwYFyIDPyhRBy/wij+8A2x7D31j+3fzsXmCmjwNYV/mjtc3/WvmsRDfaIyXdx8aRHDgKjIcE /5v1kLy9zqwXqT5hcCl5XIahxLfOE8oelbCyWrnN5FW7f1RFJ1YoSRqjeg16VnxU+DmfhLGp G65Qf00AAAAAAAA= --------------ms070109060009060403070404-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 14:54:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 326DC646; Sun, 23 Mar 2014 14:54:56 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3D8A15E; Sun, 23 Mar 2014 14:54:55 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so13448236qgd.1 for ; Sun, 23 Mar 2014 07:54:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=imHw7gs6vO9gS5lHL8nqeqcUdLU9sVBEhPQKR2M3x6c=; b=TMwcmC98hvgxLx4eSy+Zn8jD/yGUyCelxznKPcnEAS8snWKq8xKkmUN1mSDvghNV4y HmtYNDzEACoh+O+YDNUSRbPHZFg5x2xkmn3z5M9yeRkE1DMO2xLW3IHwGY8/pAjFB8He pGPFBFljHiUJCAZi/Q6TjlMwXq3FrbH2PADSChAO62jkuZ9qLSpsf0KbLQMaXewhE2Zp jmrzvWcw8hS5Sgh3QYoJ9eUwF+LMtizGR99t85tLqFDEVzqZM79cIHv/JdUefv0gy+pd enaBOwpji0oYBCv//g3OGMIFTpDyr0Vpji9jms0CmaGBoSaGx8fywkQuJAqoivq6x/gR qgSw== MIME-Version: 1.0 X-Received: by 10.224.147.206 with SMTP id m14mr10354026qav.41.1395586494566; Sun, 23 Mar 2014 07:54:54 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sun, 23 Mar 2014 07:54:54 -0700 (PDT) In-Reply-To: <1055107814.1401328.1395523094565.JavaMail.root@uoguelph.ca> References: <1055107814.1401328.1395523094565.JavaMail.root@uoguelph.ca> Date: Sun, 23 Mar 2014 11:54:54 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 14:54:56 -0000 Hi Rick, very helpful as always. On Sat, Mar 22, 2014 at 6:18 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > Well, you could try making if_hw_tsomax somewhat smaller. (I can't see > how the packet including ethernet header would be more than 64K with the > patch, but?? For example, the ether_output() code can call ng_output() > and I have no idea if that might grow the data size of the packet?) > That's what I was thinking - I was going to drop it down to 32k, which is extreme, but I wanted to see if it cured it or not. Something would have to be very broken to be adding nearly 32k to a packet. > To be honest, the optimum for NFS would be setting if_hw_tsomax == 56K, > since that would avoid the overhead of the m_defrag() calls. However, > it is suboptimal for other TCP transfers. > I'm very interested in NFS performance, so this is interesting to me - Do you have the time to educate me on this? I was going to spend this week hacking out the NFS server cache, as I feel ZFS does a better job, and my cache stats are always terrible, as to be expected when I have such a wide data usage on these sans. > > One other thing you could do (if you still have them) is scan the logs > for the code with my previous printf() patch and see if there is ever > a size > 65549 in it. If there is, then if_hw_tsomax needs to be smaller > by at least that size - 65549. (65535 + 14 == 65549) > There were some 65548's for sure. Interestingly enough, the amount that it ruptures by seems to be increasing slowly. I should possibly let it rupture and run for a long time to see if there is a steadily increasing pattern... perhaps something is accidentally incrementing the packet by say 4 bytes in a heavily loaded error condition. > > I'm not familiar enough with the mbuf/uma allocators to "confirm" it, > but I believe the "denied" refers to cases where m_getjcl() fails to get > a jumbo mbuf and returns NULL. > > If this were to happen in m_defrag(), it would return NULL and the ix > driver returns ENOBUFS, so this is not the case for EFBIG errors. > > BTW, the loop that your original printf code is in, just before the retry: goto label: That's an error loop, and it looks to me that all/most packets traverse it at some time? > I don't know if increasing the limits for the jumbo mbufs via sysctl > will help. If you are using the code without Jack's patch, which uses > 9K mbufs, then I think it can fragment the address space and result > in no 9K contiguous areas to allocate from. (I'm just going by what > Garrett and others have said about this.) > > I never seem to be running out of mbufs - 4k or 9k. Unless it's possible for a starvation to occur without incrementing the counters. Additionally, netstat -m is recording denied mbufs on boot, so on a 96 Gig system that is just starting up, I don't think I am.. but a large increase in the buffers is on my list of desperation things to try. Thanks for the hint on m_getjcl().. I'll dig around and see if I can find what's happening there. I guess it's time for me to learn basic dtrace as well. :-) From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 15:19:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A53ABC1C; Sun, 23 Mar 2014 15:19:26 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 52A8C338; Sun, 23 Mar 2014 15:19:26 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id hw13so4288276qab.4 for ; Sun, 23 Mar 2014 08:19:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PhaSToECLAPC+3AH7DZil2toIj8LX1Ulh3x+r7e/034=; b=hAgZyg1vUrLE08Ahtxut4yCE6XMyXg6W46DqwL+4Mi2BsjUQ2C64TysnQ8UUjV50qj 3dWKDj2LvahX6Trhba1DBMZf6R/IWPFAW+nrXoa40cpoi5SyJeMq+BA2MnYUGyRv9C8u iPkRNXnHRXVZTLYAANInaMRaTH81J+dF7UUgjU1AZBlJtX5G9DKWhxxLzSLLEF4vVcih G4OhweNRcailAh3Icj2F0twtDLZTEApCozOzs9WOlhr/t0lXSaCxz/AT+y33qNUx1m4J G7yVQuMFJ86GAFDQTGSyyAzXimKnmpK4P2Ki+YPCuyibgQBHqouz+JtjPxHhJGHzGutJ fQDA== MIME-Version: 1.0 X-Received: by 10.140.48.199 with SMTP id o65mr31043448qga.16.1395587965557; Sun, 23 Mar 2014 08:19:25 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sun, 23 Mar 2014 08:19:25 -0700 (PDT) In-Reply-To: <1752303953.1405506.1395524483238.JavaMail.root@uoguelph.ca> References: <1752303953.1405506.1395524483238.JavaMail.root@uoguelph.ca> Date: Sun, 23 Mar 2014 12:19:25 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 15:19:26 -0000 On Sat, Mar 22, 2014 at 6:41 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > #if defined(INET) || defined(INET6) > > /* Initialize to max value. */ > > if (ifp->if_hw_tsomax == 0) > > ifp->if_hw_tsomax = IP_MAXPACKET; > > KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > > ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > > ("%s: tsomax outside of range", __func__)); > > #endif > > > > > > Should this be the location where it's being set rather than in > > ixgbe? I would assume that other drivers could fall prey to this > > issue. > > > All of this should be prepended with "I'm an NFS guy, not a networking > guy, so I might be wrong". > > Other drivers (and ixgbe for the 82598 chip) can handle a packet that > is in more than 32 mbufs. (I think the 82598 handles 100, grep for SCATTER > in *.h in sys/dev/ixgbe.) > [...] Yes, I agree we have to be careful about the limitations of other drivers, but I'm thinking setting tso to IP_MAXPACKET is a bad idea, unless all of the header subtractions are happening elsewhere. Then again, perhaps every other driver (and possibly ixgbe.. i need to look more) does a maxtso - various_headers to set a limit for data packets. I'm not familiar with the Freebsd network conventions/styles - I'm just asking questions, something I have a bad habit for, but I'm in charge of code stability issues at my work so it's hard to stop. > > Now, since several drivers do have this 32 mbufs limit, I can see an > argument > for making the default a little smaller to make these work, since the > driver can override the default. (About now someone usually jumps in and > says > something along the lines of "You can't do that until all the drivers that > can handle IP_MAXPACKET are fixed to set if_hw_tsomax" and since I can't > fix > drivers I can't test, that pretty much puts a stop on it.) > > Testing is a problem isn't it? I once again offer my stack of network cards and systems for some sort of testing.. I still have coax and token ring around. :-) > You see the problem isn't that IP_MAXPACKET is too big, but that the > hardware > has a limit of 32 non-contiguous chunks (mbufs)/packet and 32 * MCLBYTES = > 64K. > (Hardware/network drivers that can handle 35 or more chunks (they like to > call > them transmit segments, although ixgbe uses the term scatter) shouldn't > have > any problems.) > > I have an untested patch that adds a tsomaxseg count to use along with > tsomax > bytes so that a driver could inform tcp_output() it can only handle 32 > mbufs > and then tcp_output() would limit a TSO segment using both, but I can't > test > it, so who knows when/if that might happen. > > I think you give that to me in the next email - if not, please send. > I also have a patch that modifies NFS to use pagesize clusters (reducing > the > mbuf count in the list), but that one causes grief when testing on an i386 > (seems to run out of kernel memory to the point where it can't allocate > something > called "boundary tags" and pretty well wedges the machine at that point.) > Since I don't know how to fix this (I thought of making the patch "amd64 > only"), > I can't really commit this to head, either. > > Send me that one too. I love NFS patches. > As such, I think it's going to be "fix the drivers one at a time" and tell > folks to "disable TSO or limit rsize,wsize to 32K" when they run into > trouble. > (As you might have guessed, I'd rather just be "the NFS guy", but since NFS > "triggers the problem" I\m kinda stuck with it;-) > > I know in some circumstances disabling TSO can be a benefit, but in general you'd want it on a modern system with heavy data load. > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to > > make sure VLANs fit? > > > No idea. (I wouldn't know a VLAN if it jumped up and tried to > bite me on the nose.;-) So, I have no idea what does this, but > if it means the total ethernet header size can be > 14bytes, then I'd > agree. > > Yeah, you need another 4 bytes for VLAN header if you're not using hardware that strips it before the TCP stack gets it. I have a mix of hardware and software VLANs running on our backbone, mostly due to a mixed FreeBSD/OpenBSD/Windows environment. > > Perhaps there is something in the newer network code that is filling > > up the frames to the point where they are full - thus a TSO = > > IP_MAXPACKET is just now causing problems. > > > Yea, I have no idea why this didn't bite running 9.1. (Did 9.1 have > TSO enabled by default?) > I believe 9.0 has TSO on by default.. I seem to recall it always being there, but I can't easily confirm it now. My last 9.0-STABLE doesn't have an ixgbe card in it. > > From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 15:26:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFE37173; Sun, 23 Mar 2014 15:26:00 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5E01E5E7; Sun, 23 Mar 2014 15:26:00 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id m20so4846080qcx.21 for ; Sun, 23 Mar 2014 08:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cyuQauoe/J6AkMxKPLTbY4Njijeo4IEvd/3XQyhlgwg=; b=ZZPXJ1jzlNmMIazADJ0oyf24VoA7vw6N95gY+yM9WDxKwZYq3hns1KWX7VwW6FTceu brKvUhVCYn7qCyXzrsNO3bWZxjyxsPZCPawkX6R32jollJhDyE5szfuAWUHsVtR+s4wU 0N5OxZe/elvAc9TyEC54iWJimPQbcFyhhN++/VxSOgW2NkXuiIip7eB9JZkfZRNuvU8Y l7sV1Zd2zNg5AtFyu1gj/bQ2TBT3NoZzq7gcQECSK3t1VkVRuAoLGgdCNdKXuaErrlG2 OPLAaNkswU2SSYLP5YXKwiNsSOfM5xKg8MKOpQpUOruYA5RUrWO7QqP8wCg1V+t5lDaQ BDTw== MIME-Version: 1.0 X-Received: by 10.140.80.209 with SMTP id c75mr2319514qgd.79.1395588359602; Sun, 23 Mar 2014 08:25:59 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sun, 23 Mar 2014 08:25:59 -0700 (PDT) In-Reply-To: <1532337187.1452820.1395543491196.JavaMail.root@uoguelph.ca> References: <1532337187.1452820.1395543491196.JavaMail.root@uoguelph.ca> Date: Sun, 23 Mar 2014 12:25:59 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 15:26:00 -0000 On Sat, Mar 22, 2014 at 11:58 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to > > make sure VLANs fit? > > > I took a look and, yes, this does seem to be needed. It will only be > needed for the case where a vlan is in use and hwtagging is disabled, > if I read the code correctly. > Yes, or in the rare care where you configure your switch to pass the v_lan header through to the NIC. > > Do you use vlans? > (Answered in above email) > > I've attached an updated patch. > > It might be nice to have the printf() patch in the driver too, so > we can see how big the ones that are too big are? > > Yes, I'm going to leave those in until I know we have this fixed.. will probably leave it in a while longer as it should only have a minor performance impact to iter-loop like that, and I'd like to see what the story is a few months down the road. Thanks for the patches, will have to start giving them code-names so we can keep them straight. :-) I guess we have printf, tsomax, and this one. From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 15:57:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ECFCFEA7 for ; Sun, 23 Mar 2014 15:57:36 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B62B28D6 for ; Sun, 23 Mar 2014 15:57:36 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s2NFvYLq036684 for ; Sun, 23 Mar 2014 10:57:34 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sun Mar 23 10:57:34 2014 Message-ID: <532F0469.10202@denninger.net> Date: Sun, 23 Mar 2014 10:57:29 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Strongswan problem (used to work for client NAT to the Internet, no longer does) References: <532E123B.3060702@denninger.net> <532E6A9D.9040609@denninger.net> In-Reply-To: <532E6A9D.9040609@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms060608090702030301050806" X-Antivirus: avast! (VPS 140323-0, 03/23/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 15:57:37 -0000 This is a cryptographically signed message in MIME format. --------------ms060608090702030301050806 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 3/23/2014 12:01 AM, Karl Denninger wrote: > > On 3/22/2014 5:44 PM, Karl Denninger wrote: >> FreeBSD-STABLE 10 r263037M >> >> >> It *looks* like anything coming in through IPSEC and being decoded in = >> there never goes through the ipfw chain at all..... >> > This may be addressed by PR185876.... checking. > Or not.... Now the packets just disappear entirely. Still investigating.... --=20 -- Karl karl@denninger.net --------------ms060608090702030301050806 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMjMxNTU3MjlaMCMGCSqGSIb3DQEJBDEW BBTI+x0v12xnw+WE/55oUOH3+32izDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAYXeXRUT0yPt+/IEVQbI1meXFK3ep ZzvqqUq4HDnIDyGdzeij5xyuqK+Lslz+torkU+EQWxY/W+huxnDknnNGnh26gEswx9FQNNsS QeSB207HGmDQXG2m14DEQFsyo55quKFPK7Toc2QwLBHMMZ5zouueADHcyCbFFi6DOv8dNmmO jT2PRKlMES9xCNKJE2sMeKtUYCpW0bOe2J1vdVWT0ioZcKFyuTxd7cdChjCW9VkP0RTCoFVF qDN1ZUeYDgOEbEUjVUhJYCm6hxBMyoAF93FHt4xLY22AKrelyCfzafR9FXYK+BmSy+DxmoAc Lvs9MUBLQ//rpTOowm69rdKYMspmXkT+WWyptIMeP66xQuee3ccXU0YuxNp4kyLo4GrOpkAR vHDN5Mqgf7XpmQwZ+InuKk4GxvJDDRARQ04ZY5Lg2bMuNdKu+thKMX7p5HH0M7CPCcZsmUPB phiyrIw3xogITmdyoqoB6hLR2YLUqjk1slNqfHHQ/gjVkLTFrb4z3SRh5IXWYqgibMPrcXI1 u28LRG/UalcCmTvEXyaaQCAvIypTPle3MR636cta1qMQdmrGWcy3glHacnO6KVWj2XnK0i2v SID+aUZGhqaI0JZyi1jS2H/3SuxjepFDxW46Gl+tj+objnClT6Bn4h29kloXFh20x/9W9if0 Wxn1hvMAAAAAAAA= --------------ms060608090702030301050806-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 16:23:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AED53A67 for ; Sun, 23 Mar 2014 16:23:13 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 57F51AF3 for ; Sun, 23 Mar 2014 16:23:12 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s2NGNBbF046106 for ; Sun, 23 Mar 2014 11:23:11 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sun Mar 23 11:23:11 2014 Message-ID: <532F0A6A.7040003@denninger.net> Date: Sun, 23 Mar 2014 11:23:06 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Strongswan problem (used to work for client NAT to the Internet, no longer does) [[RESOLVED]] References: <532E123B.3060702@denninger.net> <532E6A9D.9040609@denninger.net> <532F0469.10202@denninger.net> In-Reply-To: <532F0469.10202@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms010202040108040908060702" X-Antivirus: avast! (VPS 140323-0, 03/23/2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 16:23:13 -0000 This is a cryptographically signed message in MIME format. --------------ms010202040108040908060702 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 3/23/2014 10:57 AM, Karl Denninger wrote: > > On 3/23/2014 12:01 AM, Karl Denninger wrote: >> >> On 3/22/2014 5:44 PM, Karl Denninger wrote: >>> FreeBSD-STABLE 10 r263037M >>> >>> >>> It *looks* like anything coming in through IPSEC and being decoded=20 >>> in there never goes through the ipfw chain at all..... >>> >> This may be addressed by PR185876.... checking. >> > Or not.... > > Now the packets just disappear entirely. Still investigating.... > Got it. With the patches you have to be verrrry careful with the nat, and make=20 sure you first explicitly *exclude* NAT processing from IPSEC-related=20 packets (which DO have their tags properly carried forward now) and then = you must also explicitly process NAT *outbound only* for IPSEC-outbound=20 packets that arrive coming inward. In other words, with pr185876 on your system, assuming 192.168.2.0/24 is = your IPSEC pool and the Internet-accessible interface is em1, you need=20 the following fragments if you want NAT to the Internet at-large to work = for IPSEC-connected clients: 01700 divert 8668 ip4 from any to any not ipsec via em1 01705 divert 8668 ip4 from 192.168.2.0/24 to any ipsec xmit em1 To process all NAT-related traffic EXCEPT outbound IPSEC-related, and=20 then to explicitly process *only* outbound IPSEC related packets (and=20 not inbound ones, which are picked up by the first rule already) That works. pr185876's fixes must be in your system, and because they change header=20 definitions you must rebuild world, not just the kernel. --=20 -- Karl karl@denninger.net --------------ms010202040108040908060702 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMjMxNjIzMDZaMCMGCSqGSIb3DQEJBDEW BBQina/zw/CQYPTCwFRZBx7N88or0jBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAiu42Rz4CErXJRXVCNLYWF0zKdd+9 3rHsDf7ATaFJFUG07gcBtClRiT6giq+x937mL79CRXWeeRWHzlcidou7BvKtYA9UQoaQ+sxY fsutw18cltxDH5eHww5vGrB3M8ETPldzGEZLRKyUaZodLW0mp8lLRvBQQtpnVLM1mrSEgGcR aVdmmoXyi8AzHKAUZm9Jrh7PcJuLcW39/y6ECjDKs/lMqjxxg1OD5ZYxShY64/5fRZnlvOw4 vg5GI5mAAzxQtcHL9S6bQAhVXhUbWujXvgmA1Mtsr3ByEpxDYAQXLlN60d5iHsNTtPTS7KVC RnUmzzp0PNVl5dOv33mydLktF5w5Qu83Be9r09m/0PNmUvIRbjo5sFYe0EtuzmyHx49yPo5G LdDjiqkCyAbm1gG1/9XSqGg3HQ7K/0OEUSO/O1BVmnUYJP4Zq1ogH749P3Fm1ZwqxCRQFFQV VFDGVZQFFGwkCw62+wPJiPTK8FBLhgtcOuWspliPdQyLM5lguibORfuNDnvT1eBDrOx6Embn r8tHDYuUjteY+aEHxaCHaFtcF/yLXavMAreXBEk0mSRD4n3lGj6s3xuMJfTSEto3oYPRHpW8 MWx6XKQ0nMGz3ryKpOmhhKADJ12A0zTNQKvskt6hX0RxcZcz6w8r+aJr7FbUiO6aW0hDAcV+ czcDgZ4AAAAAAAA= --------------ms010202040108040908060702-- From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 20:28:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 935164ED; Sun, 23 Mar 2014 20:28:03 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3EB521A8; Sun, 23 Mar 2014 20:28:03 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id m20so5089501qcx.35 for ; Sun, 23 Mar 2014 13:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qbHzEIWIpWCmaxJEPh1IfwuPRbDQCqwWw7GI0EKs0Wc=; b=pR/nows16+sudSch+BsEWHLEofs+C0k845zrPSlUjqtMcJCNEkFeVyF+EDhUAz1jB5 zYxcitkepTuuWBPwLMz1zd05wAoFFObwewjEO/4yFHh/DLDSDIAVJ6BiAZCvzorvP3Hb E8Kg+NpgJuAJY4r/G15r9voT8QEk8RZ25cRIZDlcjFx2knFV/AzEHbrOQv80UZoy32vu pl9/q8yQa73VsJFogP6jCIvyoJOjFcvMjqG5rn6CDSIt/FTyJ3FTTu53+WavbcU8Fk/u WCYs40OQ8bbnOAqZWRliZ/039b05evIdn7tg5+Dy3m83XRl0UtCDZWTj+UkLbZgDPbX/ xG0w== MIME-Version: 1.0 X-Received: by 10.140.90.80 with SMTP id w74mr2664956qgd.96.1395606482497; Sun, 23 Mar 2014 13:28:02 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sun, 23 Mar 2014 13:28:02 -0700 (PDT) In-Reply-To: References: <1532337187.1452820.1395543491196.JavaMail.root@uoguelph.ca> Date: Sun, 23 Mar 2014 17:28:02 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 20:28:03 -0000 Update: For giggles, I set IP_MAXPACKET = 32768. Over a hour of runtime, and no issues. This is better than with the TSO patch and the 9.2 ixgbe, as that was just a drastic reduction in errors. Still have an 'angry' netstat -m on boot, and I'm still incrementing denied netbuf calls, so something else is wrong. I'm going to modify Rick's prinft in ixgbe to also output when we're over 32768. I'm sure it's still happening, but with an extra 32k of space, we're not busting like we did before. I notice a few interesting ip->ip_len changes since 9.2 - Like here, at line 720 http://fxr.watson.org/fxr/diff/netinet/ip_output.c?v=FREEBSD10;im=kwqeqdhhvovqn;diffval=FREEBSD92;diffvar=v Looks like older code didn't byteswap with ntohs - I see that often in tcp_output.c, and in tcp_options.c. I'm also curious about this:Line 524 http://fxr.watson.org/fxr/diff/netinet/ip_options.c?v=FREEBSD10;diffval=FREEBSD92;diffvar=v New 10 code: ip ->ip_len = htons (ntohs (ip ->ip_len) + optlen); Old 9.2 Code: ip ->ip_len += optlen; I wonder if there are any unexpected consequences of these changes, or perhaps a line someplace that doesn't make the change. Is there a dtrace command I could use to watch these functions and compare the new ip_len with ip->ip_len or other variables? On Sun, Mar 23, 2014 at 12:25 PM, Christopher Forgeron wrote: > > On Sat, Mar 22, 2014 at 11:58 PM, Rick Macklem wrote: > >> Christopher Forgeron wrote: >> > >> >> > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to >> > make sure VLANs fit? >> > >> I took a look and, yes, this does seem to be needed. It will only be >> needed for the case where a vlan is in use and hwtagging is disabled, >> if I read the code correctly. >> > > Yes, or in the rare care where you configure your switch to pass the v_lan > header through to the NIC. > >> >> Do you use vlans? >> > > (Answered in above email) > > >> >> I've attached an updated patch. >> >> It might be nice to have the printf() patch in the driver too, so >> we can see how big the ones that are too big are? >> >> > Yes, I'm going to leave those in until I know we have this fixed.. will > probably leave it in a while longer as it should only have a minor > performance impact to iter-loop like that, and I'd like to see what the > story is a few months down the road. > > Thanks for the patches, will have to start giving them code-names so we > can keep them straight. :-) I guess we have printf, tsomax, and this one. > From owner-freebsd-net@FreeBSD.ORG Sun Mar 23 23:57:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EDA4E55; Sun, 23 Mar 2014 23:57:12 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id B1DDB628; Sun, 23 Mar 2014 23:57:10 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAOpzL1ODaFve/2dsb2JhbABZhBiDB79sgSZ0giUBAQEDASMEUhsOChEZAgRHAQ0GE4dxCKtEoWwXjhgKAQYBHBkbB4JvgUkEkFOaKINJIYEsAQgXIg X-IronPort-AV: E=Sophos;i="4.97,716,1389762000"; d="scan'208";a="108238890" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 23 Mar 2014 19:57:09 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 0E324B3FAD; Sun, 23 Mar 2014 19:57:09 -0400 (EDT) Date: Sun, 23 Mar 2014 19:57:09 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1936201708.1670348.1395619029045.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1670344_208375117.1395619029041" X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Mar 2014 23:57:12 -0000 ------=_Part_1670344_208375117.1395619029041 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > > > > > > > On Sat, Mar 22, 2014 at 6:41 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > Christopher Forgeron wrote: > > #if defined(INET) || defined(INET6) > > /* Initialize to max value. */ > > if (ifp->if_hw_tsomax == 0) > > ifp->if_hw_tsomax = IP_MAXPACKET; > > KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > > ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > > ("%s: tsomax outside of range", __func__)); > > #endif > > > > > > Should this be the location where it's being set rather than in > > ixgbe? I would assume that other drivers could fall prey to this > > issue. > > > All of this should be prepended with "I'm an NFS guy, not a > networking > guy, so I might be wrong". > > Other drivers (and ixgbe for the 82598 chip) can handle a packet that > is in more than 32 mbufs. (I think the 82598 handles 100, grep for > SCATTER > in *.h in sys/dev/ixgbe.) > > > [...] > > > Yes, I agree we have to be careful about the limitations of other > drivers, but I'm thinking setting tso to IP_MAXPACKET is a bad idea, > unless all of the header subtractions are happening elsewhere. Then > again, perhaps every other driver (and possibly ixgbe.. i need to > look more) does a maxtso - various_headers to set a limit for data > packets. > > > I'm not familiar with the Freebsd network conventions/styles - I'm > just asking questions, something I have a bad habit for, but I'm in > charge of code stability issues at my work so it's hard to stop. > Well, IP_MAXPACKET is simply the largest # that fits in the 16bit length field of an IP header (65535). This limit is on the TSO segment (which is really just a TCP/IP packet greater than the MTU) and does not include a MAC level (ethernet) header. Beyond that, it is the specific hardware that limits things, such as this case, which is limited to 32 mbufs (which happens to imply 64K total, including ethernet header using 2K mbuf clusters). (The 64K limit is just a quirk caused by the 32mbuf limit and the fact that mbuf clusters hold 2K of data each.) > > > Now, since several drivers do have this 32 mbufs limit, I can see an > argument > for making the default a little smaller to make these work, since the > driver can override the default. (About now someone usually jumps in > and says > something along the lines of "You can't do that until all the drivers > that > can handle IP_MAXPACKET are fixed to set if_hw_tsomax" and since I > can't fix > drivers I can't test, that pretty much puts a stop on it.) > > > > > Testing is a problem isn't it? I once again offer my stack of network > cards and systems for some sort of testing.. I still have coax and > token ring around. :-) > > > > You see the problem isn't that IP_MAXPACKET is too big, but that the > hardware > has a limit of 32 non-contiguous chunks (mbufs)/packet and 32 * > MCLBYTES = 64K. > (Hardware/network drivers that can handle 35 or more chunks (they > like to call > them transmit segments, although ixgbe uses the term scatter) > shouldn't have > any problems.) > > I have an untested patch that adds a tsomaxseg count to use along > with tsomax > bytes so that a driver could inform tcp_output() it can only handle > 32 mbufs > and then tcp_output() would limit a TSO segment using both, but I > can't test > it, so who knows when/if that might happen. > > > > > I think you give that to me in the next email - if not, please send. > > > > I also have a patch that modifies NFS to use pagesize clusters > (reducing the > mbuf count in the list), but that one causes grief when testing on an > i386 > (seems to run out of kernel memory to the point where it can't > allocate something > called "boundary tags" and pretty well wedges the machine at that > point.) > Since I don't know how to fix this (I thought of making the patch > "amd64 only"), > I can't really commit this to head, either. > > > > > Send me that one too. I love NFS patches. > > > > As such, I think it's going to be "fix the drivers one at a time" and > tell > folks to "disable TSO or limit rsize,wsize to 32K" when they run into > trouble. > (As you might have guessed, I'd rather just be "the NFS guy", but > since NFS > "triggers the problem" I\m kinda stuck with it;-) > > > > I know in some circumstances disabling TSO can be a benefit, but in > general you'd want it on a modern system with heavy data load. > > > > > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax > > to > > make sure VLANs fit? > > > No idea. (I wouldn't know a VLAN if it jumped up and tried to > bite me on the nose.;-) So, I have no idea what does this, but > if it means the total ethernet header size can be > 14bytes, then I'd > agree. > > > > Yeah, you need another 4 bytes for VLAN header if you're not using > hardware that strips it before the TCP stack gets it. I have a mix > of hardware and software VLANs running on our backbone, mostly due > to a mixed FreeBSD/OpenBSD/Windows environment. > > > > > > Perhaps there is something in the newer network code that is > > filling > > up the frames to the point where they are full - thus a TSO = > > IP_MAXPACKET is just now causing problems. > > > Yea, I have no idea why this didn't bite running 9.1. (Did 9.1 have > TSO enabled by default?) > > > > I believe 9.0 has TSO on by default.. I seem to recall it always > being there, but I can't easily confirm it now. My last 9.0-STABLE > doesn't have an ixgbe card in it. > > > > Ok, I've attached 3 patches: ixgbe.patch - A slightly updated version of the one that sets if_hw_tsomax, which subtracts out the additional 4bytes for the VLAN header. *** If you can test this, it would be nice to know if this gets rid of all the EFBIG replies, since I think Jack might feel it is ok to commit if it does do so. 4kmcl.patch - This one modifies NFS to use pagesize mbuf clusters for the large RPC messages. It is NOT safe to use on a small i386, but might be ok on a large amd64 box. On a small i386, using a mix of 2K and 4K mbuf clusters seems to fragment kernel memory enough that allocation of "boundary tags" (whatever those are?) fail and this trainwrecks the system. Using pagesize (4K) clusters reduces the mbuf count for an IP_MAXPACKET sized TSO segment to 19, avoiding the 32 limit and any need to call m_defrag() for NFS. *** Only use on a test system, at your own risk. tsomaxseg.patch - This one adds support for if_hw_tsomaxseg, which is a limit on the # of mbufs in an output TSO segment (and defaults to 32). *** This one HAS NOT BEEN TESTED and probably doesn't even work at this point. rick ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0yMiAyMjo0NDo1My4wMDAwMDAwMDAg LTA0MDAKQEAgLTI2MTQsNiArMjYxNCwxMCBAQCBpeGdiZV9zZXR1cF9pbnRlcmZhY2UoZGV2aWNl X3QgZGV2LCBzdHJ1CiAJaWZwLT5pZl9zbmQuaWZxX2Rydl9tYXhsZW4gPSBhZGFwdGVyLT5udW1f dHhfZGVzYyAtIDI7CiAJSUZRX1NFVF9SRUFEWSgmaWZwLT5pZl9zbmQpOwogI2VuZGlmCisJaWYg KChhZGFwdGVyLT5udW1fc2VncyAqIE1DTEJZVEVTIC0gKEVUSEVSX0hEUl9MRU4gKworCSAgICBF VEhFUl9WTEFOX0VOQ0FQX0xFTikpIDwgSVBfTUFYUEFDS0VUKQorCQlpZnAtPmlmX2h3X3Rzb21h eCA9IGFkYXB0ZXItPm51bV9zZWdzICogTUNMQllURVMgLQorCQkgICAgKEVUSEVSX0hEUl9MRU4g KyBFVEhFUl9WTEFOX0VOQ0FQX0xFTik7CiAKIAlldGhlcl9pZmF0dGFjaChpZnAsIGFkYXB0ZXIt Pmh3Lm1hYy5hZGRyKTsKIAo= ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=4kmcl.patch Content-Disposition: attachment; filename=4kmcl.patch Content-Transfer-Encoding: base64 LS0tIGZzL25mc3NlcnZlci9uZnNfbmZzZHBvcnQuYy5zYXYyCTIwMTQtMDEtMjYgMTg6NTQ6Mjku MDAwMDAwMDAwIC0wNTAwCisrKyBmcy9uZnNzZXJ2ZXIvbmZzX25mc2Rwb3J0LmMJMjAxNC0wMy0x NiAyMzoyMjo1Ni4wMDAwMDAwMDAgLTA0MDAKQEAgLTU2Niw4ICs1NjYsNyBAQCBuZnN2bm9fcmVh ZGxpbmsoc3RydWN0IHZub2RlICp2cCwgc3RydWN0CiAJbGVuID0gMDsKIAlpID0gMDsKIAl3aGls ZSAobGVuIDwgTkZTX01BWFBBVEhMRU4pIHsKLQkJTkZTTUdFVChtcCk7Ci0JCU1DTEdFVChtcCwg TV9XQUlUT0spOworCQlORlNNQ0xHRVQobXAsIE1fTk9XQUlUKTsKIAkJbXAtPm1fbGVuID0gTkZT TVNJWihtcCk7CiAJCWlmIChsZW4gPT0gMCkgewogCQkJbXAzID0gbXAyID0gbXA7CkBAIC02MjEs NyArNjIwLDcgQEAgbmZzdm5vX3JlYWQoc3RydWN0IHZub2RlICp2cCwgb2ZmX3Qgb2ZmLAogICAg IHN0cnVjdCB0aHJlYWQgKnAsIHN0cnVjdCBtYnVmICoqbXBwLCBzdHJ1Y3QgbWJ1ZiAqKm1wZW5k cCkKIHsKIAlzdHJ1Y3QgbWJ1ZiAqbTsKLQlpbnQgaTsKKwlpbnQgZG9fcGFnZXNpemUsIGk7CiAJ c3RydWN0IGlvdmVjICppdjsKIAlzdHJ1Y3QgaW92ZWMgKml2MjsKIAlpbnQgZXJyb3IgPSAwLCBs ZW4sIGxlZnQsIHNpeiwgdGxlbiwgaW9mbGFnID0gMDsKQEAgLTYzMCwxNCArNjI5LDMzIEBAIG5m c3Zub19yZWFkKHN0cnVjdCB2bm9kZSAqdnAsIG9mZl90IG9mZiwKIAlzdHJ1Y3QgbmZzaGV1ciAq bmg7CiAKIAlsZW4gPSBsZWZ0ID0gTkZTTV9STkRVUChjbnQpOworCWRvX3BhZ2VzaXplID0gMDsK KyNpZiBNSlVNUEFHRVNJWkUgIT0gTUNMQllURVMKKwlpZiAobGVmdCA+IE1DTEJZVEVTKQorCQlk b19wYWdlc2l6ZSA9IDE7CisjZW5kaWYKIAltMyA9IE5VTEw7CiAJLyoKIAkgKiBHZW5lcmF0ZSB0 aGUgbWJ1ZiBsaXN0IHdpdGggdGhlIHVpb19pb3YgcmVmLiB0byBpdC4KIAkgKi8KIAlpID0gMDsK IAl3aGlsZSAobGVmdCA+IDApIHsKLQkJTkZTTUdFVChtKTsKLQkJTUNMR0VUKG0sIE1fV0FJVE9L KTsKKwkJLyoKKwkJICogRm9yIGxhcmdlIHJlYWRzLCB0cnkgYW5kIGFjcXVpcmUgTUpVTVBBR0VT SVpFIGNsdXN0ZXJzLgorCQkgKiBIb3dldmVyLCBkbyBzbyB3aXRoIE1fTk9XQUlUIHNvIHRoZSB0 aHJlYWQgY2FuJ3QgZ2V0CisJCSAqIHN0dWNrIHNsZWVwaW5nIG9uICJidGFsbG9jIi4KKwkJICog SWYgdGhpcyBmYWlscywgdXNlIE5GU01DTEdFVCguLk1fTk9XQUlUKSwgd2hpY2ggZG9lcyBhbgor CQkgKiBNR0VUKC4uTV9XQUlUT0spIGZvbGxvd2VkIGJ5IGEgTUNMR0VUKC4uTV9OT1dBSVQpLiAg VGhlCisJCSAqIE1DTEdFVCguLk1fTk9XQUlUKSBtYXkgbm90IGdldCBhIGNsdXN0ZXIsIGJ1dCB3 aWxsIGRyYWluCisJCSAqIHRoZSBtYnVmIGNsdXN0ZXIgem9uZSB3aGVuIGl0IGZhaWxzLgorCQkg KiBBcyBzdWNoLCBhbiBtYnVmIHdpbGwgYWx3YXlzIGJlIGFsbG9jYXRlZCBhbmQgbW9zdCBsaWtl bHkKKwkJICogaXQgd2lsbCBoYXZlIGEgY2x1c3Rlci4KKwkJICovCisJCW0gPSBOVUxMOworCQlp ZiAoZG9fcGFnZXNpemUgIT0gMCkKKwkJCW0gPSBtX2dldGpjbChNX05PV0FJVCwgTVRfREFUQSwg MCwgTUpVTVBBR0VTSVpFKTsKKwkJaWYgKG0gPT0gTlVMTCkKKwkJCU5GU01DTEdFVChtLCBNX05P V0FJVCk7CiAJCW0tPm1fbGVuID0gMDsKIAkJc2l6ID0gbWluKE1fVFJBSUxJTkdTUEFDRShtKSwg bGVmdCk7CiAJCWxlZnQgLT0gc2l6OwpAQCAtMTY1MywxMCArMTY3MSwxMCBAQCBhZ2FpbjoKIAlp ZiAoc2l6ID09IDApIHsKIAkJdnB1dCh2cCk7CiAJCWlmIChuZC0+bmRfZmxhZyAmIE5EX05GU1Yy KSB7Ci0JCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgMiAqIE5GU1hfVU5TSUdORUQpOwor CQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVE KTsKIAkJfSBlbHNlIHsKIAkJCW5mc3J2X3Bvc3RvcGF0dHIobmQsIGdldHJldCwgJmF0KTsKLQkJ CU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCA0ICogTkZTWF9VTlNJR05FRCk7CisJCQlORlNN X0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5TSUdORUQpOwogCQkJ dHhkcl9oeXBlcihhdC5uYV9maWxlcmV2LCB0bCk7CiAJCQl0bCArPSAyOwogCQl9CkBAIC0xNzA4 LDcgKzE3MjYsNyBAQCBhZ2FpbjoKIAkgKi8KIAlpZiAobmQtPm5kX2ZsYWcgJiBORF9ORlNWMykg ewogCQluZnNydl9wb3N0b3BhdHRyKG5kLCBnZXRyZXQsICZhdCk7Ci0JCU5GU01fQlVJTEQodGws IHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJR05FRCk7CisJCU5GU01fQlVJTERfUEFHRU1CQ0wo dGwsIHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJR05FRCk7CiAJCXR4ZHJfaHlwZXIoYXQubmFf ZmlsZXJldiwgdGwpOwogCQlkaXJsZW4gPSBORlNYX1YzUE9TVE9QQVRUUiArIE5GU1hfVkVSRiAr IDIgKiBORlNYX1VOU0lHTkVEOwogCX0gZWxzZSB7CkBAIC0xNzM0LDIwICsxNzUyLDI0IEBAIGFn YWluOgogCQkJICogdGhlIGRpcmVudCBlbnRyeS4KIAkJCSAqLwogCQkJaWYgKG5kLT5uZF9mbGFn ICYgTkRfTkZTVjMpIHsKLQkJCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgMyAqIE5GU1hf VU5TSUdORUQpOworCQkJCU5GU01fQlVJTERfUEFHRU1CQ0wodGwsIHVfaW50MzJfdCAqLAorCQkJ CSAgICAzICogTkZTWF9VTlNJR05FRCk7CiAJCQkJKnRsKysgPSBuZXduZnNfdHJ1ZTsKIAkJCQkq dGwrKyA9IDA7CiAJCQl9IGVsc2UgewotCQkJCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAy ICogTkZTWF9VTlNJR05FRCk7CisJCQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90 ICosCisJCQkJICAgIDIgKiBORlNYX1VOU0lHTkVEKTsKIAkJCQkqdGwrKyA9IG5ld25mc190cnVl OwogCQkJfQogCQkJKnRsID0gdHhkcl91bnNpZ25lZChkcC0+ZF9maWxlbm8pOwogCQkJKHZvaWQp IG5mc21fc3RydG9tKG5kLCBkcC0+ZF9uYW1lLCBubGVuKTsKIAkJCWlmIChuZC0+bmRfZmxhZyAm IE5EX05GU1YzKSB7Ci0JCQkJTkZTTV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VO U0lHTkVEKTsKKwkJCQlORlNNX0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwKKwkJCQkg ICAgMiAqIE5GU1hfVU5TSUdORUQpOwogCQkJCSp0bCsrID0gMDsKIAkJCX0gZWxzZQotCQkJCU5G U01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCBORlNYX1VOU0lHTkVEKTsKKwkJCQlORlNNX0JVSUxE X1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwKKwkJCQkgICAgTkZTWF9VTlNJR05FRCk7CiAJCQkq dGwgPSB0eGRyX3Vuc2lnbmVkKCpjb29raWVwKTsKIAkJfQogCQljcG9zICs9IGRwLT5kX3JlY2xl bjsKQEAgLTE3NTcsNyArMTc3OSw3IEBAIGFnYWluOgogCX0KIAlpZiAoY3BvcyA8IGNlbmQpCiAJ CWVvZmZsYWcgPSAwOwotCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJ R05FRCk7CisJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VO U0lHTkVEKTsKIAkqdGwrKyA9IG5ld25mc19mYWxzZTsKIAlpZiAoZW9mZmxhZykKIAkJKnRsID0g bmV3bmZzX3RydWU7CkBAIC0xOTI4LDcgKzE5NTAsNyBAQCBhZ2FpbjoKIAkJdnB1dCh2cCk7CiAJ CWlmIChuZC0+bmRfZmxhZyAmIE5EX05GU1YzKQogCQkJbmZzcnZfcG9zdG9wYXR0cihuZCwgZ2V0 cmV0LCAmYXQpOwotCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5TSUdO RUQpOworCQlORlNNX0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5T SUdORUQpOwogCQl0eGRyX2h5cGVyKGF0Lm5hX2ZpbGVyZXYsIHRsKTsKIAkJdGwgKz0gMjsKIAkJ KnRsKysgPSBuZXduZnNfZmFsc2U7CkBAIC0yMDMxLDcgKzIwNTMsNyBAQCBhZ2FpbjoKIAl9IGVs c2UgewogCQlkaXJsZW4gPSBORlNYX1ZFUkYgKyAyICogTkZTWF9VTlNJR05FRDsKIAl9Ci0JTkZT TV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIE5GU1hfVkVSRik7CisJTkZTTV9CVUlMRF9QQUdFTUJD TCh0bCwgdV9pbnQzMl90ICosIE5GU1hfVkVSRik7CiAJdHhkcl9oeXBlcihhdC5uYV9maWxlcmV2 LCB0bCk7CiAKIAkvKgpAQCAtMjE4NiwxMiArMjIwOCwxNCBAQCBhZ2FpbjoKIAkJCSAqIEJ1aWxk IHRoZSBkaXJlY3RvcnkgcmVjb3JkIHhkcgogCQkJICovCiAJCQlpZiAobmQtPm5kX2ZsYWcgJiBO RF9ORlNWMykgewotCQkJCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAzICogTkZTWF9VTlNJ R05FRCk7CisJCQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosCisJCQkJICAg IDMgKiBORlNYX1VOU0lHTkVEKTsKIAkJCQkqdGwrKyA9IG5ld25mc190cnVlOwogCQkJCSp0bCsr ID0gMDsKIAkJCQkqdGwgPSB0eGRyX3Vuc2lnbmVkKGRwLT5kX2ZpbGVubyk7CiAJCQkJZGlybGVu ICs9IG5mc21fc3RydG9tKG5kLCBkcC0+ZF9uYW1lLCBubGVuKTsKLQkJCQlORlNNX0JVSUxEKHRs LCB1X2ludDMyX3QgKiwgMiAqIE5GU1hfVU5TSUdORUQpOworCQkJCU5GU01fQlVJTERfUEFHRU1C Q0wodGwsIHVfaW50MzJfdCAqLAorCQkJCSAgICAyICogTkZTWF9VTlNJR05FRCk7CiAJCQkJKnRs KysgPSAwOwogCQkJCSp0bCA9IHR4ZHJfdW5zaWduZWQoKmNvb2tpZXApOwogCQkJCW5mc3J2X3Bv c3RvcGF0dHIobmQsIDAsIG52YXApOwpAQCAtMjIwMCw3ICsyMjI0LDggQEAgYWdhaW46CiAJCQkJ aWYgKG52cCAhPSBOVUxMKQogCQkJCQl2cHV0KG52cCk7CiAJCQl9IGVsc2UgewotCQkJCU5GU01f QlVJTEQodGwsIHVfaW50MzJfdCAqLCAzICogTkZTWF9VTlNJR05FRCk7CisJCQkJTkZTTV9CVUlM RF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosCisJCQkJICAgIDMgKiBORlNYX1VOU0lHTkVEKTsK IAkJCQkqdGwrKyA9IG5ld25mc190cnVlOwogCQkJCSp0bCsrID0gMDsKIAkJCQkqdGwgPSB0eGRy X3Vuc2lnbmVkKCpjb29raWVwKTsKQEAgLTIyNjcsNyArMjI5Miw3IEBAIGFnYWluOgogCX0gZWxz ZSBpZiAoY3BvcyA8IGNlbmQpCiAJCWVvZmZsYWcgPSAwOwogCWlmICghbmQtPm5kX3JlcHN0YXQp IHsKLQkJTkZTTV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVEKTsKKwkJ TkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVEKTsK IAkJKnRsKysgPSBuZXduZnNfZmFsc2U7CiAJCWlmIChlb2ZmbGFnKQogCQkJKnRsID0gbmV3bmZz X3RydWU7Ci0tLSBmcy9uZnNjbGllbnQvbmZzX2NsY29tc3Vicy5jLnNhdjIJMjAxNC0wMi0wMSAy MDo0NzowNy4wMDAwMDAwMDAgLTA1MDAKKysrIGZzL25mc2NsaWVudC9uZnNfY2xjb21zdWJzLmMJ MjAxNC0wMy0xNiAyMzoyMjowNi4wMDAwMDAwMDAgLTA0MDAKQEAgLTE1NSw3ICsxNTUsNyBAQCBu ZnNjbF9yZXFzdGFydChzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQgKm5kCiAJICogR2V0IHRoZSBmaXJz dCBtYnVmIGZvciB0aGUgcmVxdWVzdC4KIAkgKi8KIAlpZiAobmZzX2JpZ3JlcXVlc3RbcHJvY251 bV0pCi0JCU5GU01DTEdFVChtYiwgTV9XQUlUT0spOworCQlORlNNQ0xHRVQobWIsIE1fTk9XQUlU KTsKIAllbHNlCiAJCU5GU01HRVQobWIpOwogCW1idWZfc2V0bGVuKG1iLCAwKTsKQEAgLTI2Nyw5 ICsyNjcsMjkgQEAgbmZzbV91aW9tYnVmKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIAogCQl3 aGlsZSAobGVmdCA+IDApIHsKIAkJCW1sZW4gPSBNX1RSQUlMSU5HU1BBQ0UobXApOwogCQkJaWYg KG1sZW4gPT0gMCkgewotCQkJCWlmIChjbGZsZykKLQkJCQkJTkZTTUNMR0VUKG1wLCBNX1dBSVRP Syk7Ci0JCQkJZWxzZQorCQkJCWlmIChjbGZsZyAhPSAwKSB7CisJCQkJCS8qCisJCQkJCSAqIEZv ciBsYXJnZSB3cml0ZXMsIHRyeSBhbmQgYWNxdWlyZQorCQkJCQkgKiBNSlVNUEFHRVNJWkUgY2x1 c3RlcnMuCisJCQkJCSAqIEhvd2V2ZXIsIGRvIHNvIHdpdGggTV9OT1dBSVQgc28gdGhlCisJCQkJ CSAqIHRocmVhZCBjYW4ndCBnZXQgc3R1Y2sgc2xlZXBpbmcgb24KKwkJCQkJICogImJ0YWxsb2Mi LiAgSWYgdGhpcyBmYWlscywgdXNlCisJCQkJCSAqIE5GU01DTEdFVCguLk1fTk9XQUlUKSwgd2hp Y2ggZG9lcyBhbgorCQkJCQkgKiBNR0VUKC4uTV9XQUlUT0spIGZvbGxvd2VkIGJ5IGEKKwkJCQkJ ICogTUNMR0UgVCguLk1fTk9XQUlUKS4gVGhpcyBtYXkgbm90IGdldAorCQkJCQkgKiBhIGNsdXN0 ZXIsIGJ1dCB3aWxsIGRyYWluIHRoZSBtYnVmCisJCQkJCSAqIGNsdXN0ZXIgem9uZSB3aGVuIGl0 IGZhaWxzLgorCQkJCQkgKiBBcyBzdWNoLCBhbiBtYnVmIHdpbGwgYWx3YXlzIGJlCisJCQkJCSAq IGFsbG9jYXRlZCBhbmQgbW9zdCBsaWtlbHkgaXQgd2lsbAorCQkJCQkgKiBoYXZlIGEgY2x1c3Rl ci4KKwkJCQkJICovCisjaWYgTUpVTVBBR0VTSVpFICE9IE1DTEJZVEVTCisJCQkJCW1wID0gbV9n ZXRqY2woTV9OT1dBSVQsIE1UX0RBVEEsIDAsCisJCQkJCSAgICBNSlVNUEFHRVNJWkUpOworCQkJ CQlpZiAobXAgPT0gTlVMTCkKKyNlbmRpZgorCQkJCQkJTkZTTUNMR0VUKG1wLCBNX05PV0FJVCk7 CisJCQkJfSBlbHNlCiAJCQkJCU5GU01HRVQobXApOwogCQkJCW1idWZfc2V0bGVuKG1wLCAwKTsK IAkJCQltYnVmX3NldG5leHQobXAyLCBtcCk7Ci0tLSBmcy9uZnMvbmZzbV9zdWJzLmguc2F2Mgky MDE0LTAyLTAxIDE5OjUxOjEyLjAwMDAwMDAwMCAtMDUwMAorKysgZnMvbmZzL25mc21fc3Vicy5o CTIwMTQtMDMtMTMgMTg6NTQ6MjcuMDAwMDAwMDAwIC0wNDAwCkBAIC04OSw2ICs4OSwzNyBAQCBu ZnNtX2J1aWxkKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIGluCiAKICNkZWZpbmUJTkZTTV9C VUlMRChhLCBjLCBzKQkoKGEpID0gKGMpbmZzbV9idWlsZChuZCwgKHMpKSkKIAorLyoKKyAqIFNh bWUgYXMgYWJvdmUsIGJ1dCBhbGxvY2F0ZXMgTUpVTVBBR0VTSVpFIG1idWYgY2x1c3RlcnMsIGlm IHBvc3NpYmxlLgorICovCitzdGF0aWMgX19pbmxpbmUgdm9pZCAqCituZnNtX2J1aWxkX3BhZ2Vt YmNsKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIGludCBzaXopCit7CisJdm9pZCAqcmV0cDsK KwlzdHJ1Y3QgbWJ1ZiAqbWIyOworCisJaWYgKHNpeiA+IE1fVFJBSUxJTkdTUEFDRShuZC0+bmRf bWIpKSB7CisJCW1iMiA9IE5VTEw7CisjaWYgTUpVTVBBR0VTSVpFICE9IE1DTEJZVEVTCisJCW1i MiA9IG1fZ2V0amNsKE1fTk9XQUlULCBNVF9EQVRBLCAwLCBNSlVNUEFHRVNJWkUpOworI2VuZGlm CisJCWlmIChtYjIgPT0gTlVMTCkKKwkJCU5GU01DTEdFVChtYjIsIE1fTk9XQUlUKTsKKwkJaWYg KHNpeiA+IE1MRU4pCisJCQlwYW5pYygiYnVpbGQgPiBNTEVOIik7CisJCW1idWZfc2V0bGVuKG1i MiwgMCk7CisJCW5kLT5uZF9icG9zID0gTkZTTVRPRChtYjIsIGNhZGRyX3QpOworCQluZC0+bmRf bWItPm1fbmV4dCA9IG1iMjsKKwkJbmQtPm5kX21iID0gbWIyOworCX0KKwlyZXRwID0gKHZvaWQg KikobmQtPm5kX2Jwb3MpOworCW5kLT5uZF9tYi0+bV9sZW4gKz0gc2l6OworCW5kLT5uZF9icG9z ICs9IHNpejsKKwlyZXR1cm4gKHJldHApOworfQorCisjZGVmaW5lCU5GU01fQlVJTERfUEFHRU1C Q0woYSwgYywgcykJKChhKSA9IChjKW5mc21fYnVpbGRfcGFnZW1iY2wobmQsIChzKSkpCisKIHN0 YXRpYyBfX2lubGluZSB2b2lkICoKIG5mc21fZGlzc2VjdChzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQg Km5kLCBpbnQgc2l6KQogewotLS0gZnMvbmZzL25mc3BvcnQuaC5zYXYyCTIwMTQtMDItMTMgMTk6 MDM6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBmcy9uZnMvbmZzcG9ydC5oCTIwMTQtMDItMTMgMTk6 MTQ6MjQuMDAwMDAwMDAwIC0wNTAwCkBAIC0xMzgsNiArMTM4LDggQEAKIAogLyoKICAqIEFsbG9j YXRlIG1idWZzLiBNdXN0IHN1Y2NlZWQgYW5kIG5ldmVyIHNldCB0aGUgbWJ1ZiBwdHIgdG8gTlVM TC4KKyAqIE5vdGUgdGhhdCB3aGVuIE5GU01DTEdFVChtLCBNX05PV0FJVCkgaXMgZG9uZSwgaXQg c3RpbGwgbXVzdCBhbGxvY2F0ZQorICogYW4gbWJ1ZiAoYW5kIGNhbiBzbGVlcCksIGJ1dCBtaWdo dCBub3QgZ2V0IGEgY2x1c3RlciwgaW4gdGhlIHdvcnN0IGNhc2UuCiAgKi8KICNkZWZpbmUJTkZT TUdFVChtKQlkbyB7IAkJCQkJXAogCQlNR0VUKChtKSwgTV9XQUlUT0ssIE1UX0RBVEEpOyAJCQlc Cg== ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=tsomaxseg.patch Content-Disposition: attachment; filename=tsomaxseg.patch Content-Transfer-Encoding: base64 LS0tIGtlcm4vdWlwY19zb2NrYnVmLmMuc2F2CTIwMTQtMDEtMzAgMjA6Mjc6MTcuMDAwMDAwMDAw IC0wNTAwCisrKyBrZXJuL3VpcGNfc29ja2J1Zi5jCTIwMTQtMDEtMzAgMjI6MTI6MDguMDAwMDAw MDAwIC0wNTAwCkBAIC05NjUsNiArOTY1LDM5IEBAIHNic25kcHRyKHN0cnVjdCBzb2NrYnVmICpz YiwgdV9pbnQgb2ZmLCAKIH0KIAogLyoKKyAqIFJldHVybiB0aGUgZmlyc3QgbWJ1ZiBmb3IgdGhl IHByb3ZpZGVkIG9mZnNldC4KKyAqLworc3RydWN0IG1idWYgKgorc2JzbmRtYnVmKHN0cnVjdCBz b2NrYnVmICpzYiwgdV9pbnQgb2ZmLCBsb25nICpmaXJzdF9sZW4pCit7CisJc3RydWN0IG1idWYg Km07CisKKwlLQVNTRVJUKHNiLT5zYl9tYiAhPSBOVUxMLCAoIiVzOiBzYl9tYiBpcyBOVUxMIiwg X19mdW5jX18pKTsKKworCSpmaXJzdF9sZW4gPSAwOworCS8qCisJICogSXMgb2ZmIGJlbG93IHN0 b3JlZCBvZmZzZXQ/IEhhcHBlbnMgb24gcmV0cmFuc21pdHMuCisJICogSWYgc28sIGp1c3QgdXNl IHNiX21iLgorCSAqLworCWlmIChzYi0+c2Jfc25kcHRyID09IE5VTEwgfHwgc2ItPnNiX3NuZHB0 cm9mZiA+IG9mZikKKwkJbSA9IHNiLT5zYl9tYjsKKwllbHNlIHsKKwkJbSA9IHNiLT5zYl9zbmRw dHI7CisJCW9mZiAtPSBzYi0+c2Jfc25kcHRyb2ZmOworCX0KKwl3aGlsZSAob2ZmID4gMCAmJiBt ICE9IE5VTEwpIHsKKwkJaWYgKG9mZiA8IG0tPm1fbGVuKQorCQkJYnJlYWs7CisJCW9mZiAtPSBt LT5tX2xlbjsKKwkJbSA9IG0tPm1fbmV4dDsKKwl9CisJaWYgKG0gIT0gTlVMTCkKKwkJKmZpcnN0 X2xlbiA9IG0tPm1fbGVuIC0gb2ZmOworCisJcmV0dXJuIChtKTsKK30KKworLyoKICAqIERyb3Ag YSByZWNvcmQgb2ZmIHRoZSBmcm9udCBvZiBhIHNvY2tidWYgYW5kIG1vdmUgdGhlIG5leHQgcmVj b3JkIHRvIHRoZQogICogZnJvbnQuCiAgKi8KLS0tIHN5cy9zb2NrYnVmLmguc2F2CTIwMTQtMDEt MzAgMjA6NDI6MjguMDAwMDAwMDAwIC0wNTAwCisrKyBzeXMvc29ja2J1Zi5oCTIwMTQtMDEtMzAg MjI6MDg6NDMuMDAwMDAwMDAwIC0wNTAwCkBAIC0xNTMsNiArMTUzLDggQEAgaW50CXNicmVzZXJ2 ZV9sb2NrZWQoc3RydWN0IHNvY2tidWYgKnNiLAogCSAgICBzdHJ1Y3QgdGhyZWFkICp0ZCk7CiBz dHJ1Y3QgbWJ1ZiAqCiAJc2JzbmRwdHIoc3RydWN0IHNvY2tidWYgKnNiLCB1X2ludCBvZmYsIHVf aW50IGxlbiwgdV9pbnQgKm1vZmYpOworc3RydWN0IG1idWYgKgorCXNic25kbWJ1ZihzdHJ1Y3Qg c29ja2J1ZiAqc2IsIHVfaW50IG9mZiwgbG9uZyAqZmlyc3RfbGVuKTsKIHZvaWQJc2J0b3hzb2Nr YnVmKHN0cnVjdCBzb2NrYnVmICpzYiwgc3RydWN0IHhzb2NrYnVmICp4c2IpOwogaW50CXNid2Fp dChzdHJ1Y3Qgc29ja2J1ZiAqc2IpOwogaW50CXNibG9jayhzdHJ1Y3Qgc29ja2J1ZiAqc2IsIGlu dCBmbGFncyk7Ci0tLSBuZXRpbmV0L3RjcF9pbnB1dC5jLnNhdgkyMDE0LTAxLTMwIDE5OjM3OjUy LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3BfaW5wdXQuYwkyMDE0LTAxLTMwIDE5OjM5 OjA3LjAwMDAwMDAwMCAtMDUwMApAQCAtMzYyNyw2ICszNjI3LDcgQEAgdGNwX21zcyhzdHJ1Y3Qg dGNwY2IgKnRwLCBpbnQgb2ZmZXIpCiAJaWYgKGNhcC5pZmNhcCAmIENTVU1fVFNPKSB7CiAJCXRw LT50X2ZsYWdzIHw9IFRGX1RTTzsKIAkJdHAtPnRfdHNvbWF4ID0gY2FwLnRzb21heDsKKwkJdHAt PnRfdHNvbWF4c2VncyA9IGNhcC50c29tYXhzZWdzOwogCX0KIH0KIAotLS0gbmV0aW5ldC90Y3Bf b3V0cHV0LmMuc2F2CTIwMTQtMDEtMzAgMTg6NTU6MTUuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRp bmV0L3RjcF9vdXRwdXQuYwkyMDE0LTAxLTMwIDIyOjE4OjU2LjAwMDAwMDAwMCAtMDUwMApAQCAt MTY2LDggKzE2Niw4IEBAIGludAogdGNwX291dHB1dChzdHJ1Y3QgdGNwY2IgKnRwKQogewogCXN0 cnVjdCBzb2NrZXQgKnNvID0gdHAtPnRfaW5wY2ItPmlucF9zb2NrZXQ7Ci0JbG9uZyBsZW4sIHJl Y3dpbiwgc2VuZHdpbjsKLQlpbnQgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBp bGVyIGhhcHB5ICovCisJbG9uZyBsZW4sIHJlY3dpbiwgc2VuZHdpbiwgdHNvX3RsZW47CisJaW50 IGNudCwgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBpbGVyIGhhcHB5ICovCiAJ c3RydWN0IG1idWYgKm07CiAJc3RydWN0IGlwICppcCA9IE5VTEw7CiAJc3RydWN0IGlwb3ZseSAq aXBvdiA9IE5VTEw7CkBAIC03ODAsNiArNzgwLDI0IEBAIHNlbmQ6CiAJCQl9CiAKIAkJCS8qCisJ CQkgKiBMaW1pdCB0aGUgbnVtYmVyIG9mIFRTTyB0cmFuc21pdCBzZWdtZW50cyAobWJ1ZnMKKwkJ CSAqIGluIG1idWYgbGlzdCkgdG8gdHAtPnRfdHNvbWF4c2Vncy4KKwkJCSAqLworCQkJY250ID0g MDsKKwkJCW0gPSBzYnNuZG1idWYoJnNvLT5zb19zbmQsIG9mZiwgJnRzb190bGVuKTsKKwkJCXdo aWxlIChtICE9IE5VTEwgJiYgY250IDwgdHAtPnRfdHNvbWF4c2VncyAmJgorCQkJICAgIHRzb190 bGVuIDwgbGVuKSB7CisJCQkJaWYgKGNudCA+IDApCisJCQkJCXRzb190bGVuICs9IG0tPm1fbGVu OworCQkJCWNudCsrOworCQkJCW0gPSBtLT5tX25leHQ7CisJCQl9CisJCQlpZiAobSAhPSBOVUxM ICYmIHRzb190bGVuIDwgbGVuKSB7CisJCQkJbGVuID0gdHNvX3RsZW47CisJCQkJc2VuZGFsb3Qg PSAxOworCQkJfQorCisJCQkvKgogCQkJICogUHJldmVudCB0aGUgbGFzdCBzZWdtZW50IGZyb20g YmVpbmcKIAkJCSAqIGZyYWN0aW9uYWwgdW5sZXNzIHRoZSBzZW5kIHNvY2tidWYgY2FuCiAJCQkg KiBiZSBlbXB0aWVkLgotLS0gbmV0aW5ldC90Y3Bfc3Vici5jLnNhdgkyMDE0LTAxLTMwIDE5OjQ0 OjM1LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3Bfc3Vici5jCTIwMTQtMDEtMzAgMjA6 NTY6MTIuMDAwMDAwMDAwIC0wNTAwCkBAIC0xODAwLDYgKzE4MDAsMTIgQEAgdGNwX21heG10dShz dHJ1Y3QgaW5fY29ubmluZm8gKmluYywgc3RydQogCQkJICAgIGlmcC0+aWZfaHdhc3Npc3QgJiBD U1VNX1RTTykKIAkJCQljYXAtPmlmY2FwIHw9IENTVU1fVFNPOwogCQkJCWNhcC0+dHNvbWF4ID0g aWZwLT5pZl9od190c29tYXg7CisjaWZkZWYgbm90eWV0CisJCQkJY2FwLT50c29tYXhzZWdzID0g aWZwLT5pZl9od190c29tYXhzZWdzOworI2VuZGlmCisJCQkJaWYgKGNhcC0+dHNvbWF4c2VncyA9 PSAwKQorCQkJCQljYXAtPnRzb21heHNlZ3MgPQorCQkJCQkgICAgVENQVFNPX01BWF9UWF9TRUdT X0RFRkFVTFQ7CiAJCX0KIAkJUlRGUkVFKHNyby5yb19ydCk7CiAJfQotLS0gbmV0aW5ldC90Y3Bf dmFyLmguc2F2CTIwMTQtMDEtMzAgMTk6Mzk6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRpbmV0 L3RjcF92YXIuaAkyMDE0LTAxLTMwIDIwOjUyOjU3LjAwMDAwMDAwMCAtMDUwMApAQCAtMjA5LDYg KzIwOSw3IEBAIHN0cnVjdCB0Y3BjYiB7CiAJdV9pbnQJdF9rZWVwY250OwkJLyogbnVtYmVyIG9m IGtlZXBhbGl2ZXMgYmVmb3JlIGNsb3NlICovCiAKIAl1X2ludAl0X3Rzb21heDsJCS8qIHRzbyBi dXJzdCBsZW5ndGggbGltaXQgKi8KKwl1X2ludAl0X3Rzb21heHNlZ3M7CQkvKiB0c28gYnVyc3Qg c2VnbWVudCBsaW1pdCAqLwogCiAJdWludDMyX3QgdF9pc3BhcmVbOF07CQkvKiA1IFVUTywgMyBU QkQgKi8KIAl2b2lkCSp0X3BzcGFyZTJbNF07CQkvKiA0IFRCRCAqLwpAQCAtMjY4LDYgKzI2OSwx MSBAQCBzdHJ1Y3QgdGNwY2IgewogI2RlZmluZQlUQ1BPT0JfSEFWRURBVEEJMHgwMQogI2RlZmlu ZQlUQ1BPT0JfSEFEREFUQQkweDAyCiAKKy8qCisgKiBEZWZhdWx0IHZhbHVlIGZvciBUU08gbWF4 aW11bSBudW1iZXIgb2YgdHJhbnNtaXQgc2VnbWVudHMgKGNvdW50IG9mIG1idWZzKS4KKyAqLwor I2RlZmluZQlUQ1BUU09fTUFYX1RYX1NFR1NfREVGQVVMVAkzMAorCiAjaWZkZWYgVENQX1NJR05B VFVSRQogLyoKICAqIERlZmluZXMgd2hpY2ggYXJlIG5lZWRlZCBieSB0aGUgeGZvcm1fdGNwIG1v ZHVsZSBhbmQgdGNwX1tpbnxvdXRdcHV0CkBAIC0zMzMsNiArMzM5LDcgQEAgc3RydWN0IGhjX21l dHJpY3NfbGl0ZSB7CS8qIG11c3Qgc3RheSBpbgogc3RydWN0IHRjcF9pZmNhcCB7CiAJaW50CWlm Y2FwOwogCXVfaW50CXRzb21heDsKKwl1X2ludAl0c29tYXhzZWdzOwogfTsKIAogI2lmbmRlZiBf TkVUSU5FVF9JTl9QQ0JfSF8K ------=_Part_1670344_208375117.1395619029041-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 00:39:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8382CD9F; Mon, 24 Mar 2014 00:39:26 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 03103993; Mon, 24 Mar 2014 00:39:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAHV+L1ODaFve/2dsb2JhbABZg0FXgwe4MoZpUYEndIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAIFBAEcBIdQCA2rM6FuF4EpjHoGAQEbATMHgm+BSQSVc4QJkH+DSSExewEGAhci X-IronPort-AV: E=Sophos;i="4.97,716,1389762000"; d="scan'208";a="108242799" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 23 Mar 2014 20:38:59 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 500A7B3F13; Sun, 23 Mar 2014 20:38:59 -0400 (EDT) Date: Sun, 23 Mar 2014 20:38:59 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1850411724.1687820.1395621539316.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 00:39:26 -0000 Christopher Forgeron wrote: > Hi Rick, very helpful as always. > > > On Sat, Mar 22, 2014 at 6:18 PM, Rick Macklem > wrote: > > > Christopher Forgeron wrote: > > > > Well, you could try making if_hw_tsomax somewhat smaller. (I can't > > see > > how the packet including ethernet header would be more than 64K > > with the > > patch, but?? For example, the ether_output() code can call > > ng_output() > > and I have no idea if that might grow the data size of the packet?) > > > > That's what I was thinking - I was going to drop it down to 32k, > which is > extreme, but I wanted to see if it cured it or not. Something would > have to > be very broken to be adding nearly 32k to a packet. > > > > To be honest, the optimum for NFS would be setting if_hw_tsomax == > > 56K, > > since that would avoid the overhead of the m_defrag() calls. > > However, > > it is suboptimal for other TCP transfers. > > > Ok, here is the critical code snippet from tcp_output(): /* 774 * Limit a burst to t_tsomax minus IP, 775 * TCP and options length to keep ip->ip_len 776 * from overflowing or exceeding the maximum 777 * length allowed by the network interface. 778 */ 779 if (len > tp->t_tsomax - hdrlen) { 780 len = tp->t_tsomax - hdrlen; 781 sendalot = 1; 782 } 783 784 /* 785 * Prevent the last segment from being 786 * fractional unless the send sockbuf can 787 * be emptied. 788 */ 789 if (sendalot && off + len < so->so_snd.sb_cc) { 790 len -= len % (tp->t_maxopd - optlen); 791 sendalot = 1; 792 } The first "if" at #779 limits the len to if_hw_tsomax - hdrlen. (tp->t_tsomax == if_hw_tsomax and hdrlen == size of TCP/IP header) The second "if" at #789 reduces the len to an exact multiple of the output MTU if it won't empty the send queue. Here's how I think things work: - For a full 64K of read/write data, NFS generates an mbuf list with 32 MCLBYTES clusters of data and two small header packets prepended in front of them (one for the RPC header + one for the NFS args that come before the data). Total data length is a little over 64K (something like 65600bytes). - When the above code processes this, it reduces the length to if_hw_tsomax (65535 by default). { if at #779 } - Second "if" at #789 reduces it further (63000 for a 9000byte MTU). tcp_output() prepends an mbuf with the TCP/IP header in it, resulting is a total data length somewhat less than 64K and passes this to the ixgbe.c driver. - The ixgbe.c driver prepends an ethernet header (14 or maybe 18bytes in length) by calling ether_output() and then hands it (a little less than 64K bytes of data in 35mbufs) to ixgbe_xmit(). ixgbe_xmit() calls bus_dmamap_load_mbuf_sg() which fails, returning EFBIG, because the list has more than 32 mbufs in it. - then it calls m_defrag(), which copies the slightly less than 64K of data to a list of 32 mbuf clusters. - bus_dmamap_load_mbuf_sg() is called again and succeeds this time because the list is only 32 mbufs long. (The call to m_defrag() adds some overhead and does have the potential to fail if mbuf clusters are exhausted, so this works, but isn't ideal.) The problem case happens when the size of the I/O is a little less than the full 64K (hit EOF for read or a smaller than 64K dirty region in a buffer cache block for write. - Now, for example, the total data length for the mbuf chain (including RPC, NFS and TCP/IP headers) could be 65534 (slightly less than 64K). The first "if" doesn't change the "len", since it is less than if_hw_tsomax. The second "if" doesn't change the "len" if there is no additional data in the send queue. --> Now the ixgbe driver prepends an ethernet header, increasing the total data length to 65548 (a little over 64K). - First call to bus_dmamap_load_mbuf_sg() fails with EFBIG because the mbuf list has more than 32 entries. - calls m_defrag(), which copies the data to a list of 33 mbuf clusters. (> 64K requires 33 * 2K clusters) - Second call to bus_dmamap_load_mbuf_sg() fails again with EFBIG, because the list has 33 mbufs in it. --> Returns EFBIG and throws away the TSO segment without sending it. For NFS, the ideal would be to not only never fail with EFBIG, but to not have the overhead of calling m_defrag(). - One way is to use pagesize (4K) clusters, so that the mbuf list only has 19 entries. - Another way is to teach tcp_output() to limit the mbuf list to 32 mbufs as well as 65535 bytes in length. - Yet another is to make if_hw_tsomax small enough that the mbuf list doesn't exceed 32 mbufs. (56K would do this for NFS, but is suboptimal for other traffic.) I am hoping that the only reason you still saw a few EFBIGs with the patch that reduced if_hw_tsomax by ETHER_HDR_LEN was that some had the additional 4bytes for a vlan header. If that is the case, the slightly modified patch should make all EFBIG error returns go away. > I'm very interested in NFS performance, so this is interesting to me > - Do > you have the time to educate me on this? I was going to spend this > week > hacking out the NFS server cache, as I feel ZFS does a better job, > and my > cache stats are always terrible, as to be expected when I have such a > wide > data usage on these sans. > The DRC (duplicate request cache) is not for performance, it is for correctness. It comes from the fact that Sun RPC is "at least once" and can retry non-idempotent operations (ones that modify the file system) resulting in a corrupted file system on the server. The risk is lower when using TCP vs UDP, but is still non-zero. If you had a "perfect network fabric that never lost packets", the hit rate of the DRC is 0 and could safely be disabled. However, each hit implies a case where file system corruption has been avoided, so I think most environments want it, despite the overhead. --> The better your network environment, the lower the hit rate. (Which means you want to see "terrible" cache stats.;-) --> It is always some amount of overhead for the sake of correctness and never improves performance (well technically it does avoid redoing file system ops when there is a hit, but the performance effect is miniscule and not relevant). This was all fixed by NFSv4.1, which uses a machanism called Sessions to provide "exactly once" RPC semantics. As such, the NFSv4.1 server (still in a projects branch on svn) doesn't use the DRC. > > > > One other thing you could do (if you still have them) is scan the > > logs > > for the code with my previous printf() patch and see if there is > > ever > > a size > 65549 in it. If there is, then if_hw_tsomax needs to be > > smaller > > by at least that size - 65549. (65535 + 14 == 65549) > > > > There were some 65548's for sure. Interestingly enough, the amount > that it > ruptures by seems to be increasing slowly. I should possibly let it > rupture > and run for a long time to see if there is a steadily increasing > pattern... > perhaps something is accidentally incrementing the packet by say 4 > bytes in > a heavily loaded error condition. > I doubt it, since people run other network interfaces that don't have the 32mbuf (transmit segment) limitation without difficulties, as far as I know. > > > > > I'm not familiar enough with the mbuf/uma allocators to "confirm" > > it, > > but I believe the "denied" refers to cases where m_getjcl() fails > > to get > > a jumbo mbuf and returns NULL. > > > > If this were to happen in m_defrag(), it would return NULL and the > > ix > > driver returns ENOBUFS, so this is not the case for EFBIG errors. > > > > BTW, the loop that your original printf code is in, just before the > > retry: > goto label: That's an error loop, and it looks to me that all/most > packets > traverse it at some time? > It does a second try at calling bus_dmamap_load_mbuf_sg() after doing the compaction copying of the mbuf list via m_defrag() and then returns EFBIG if the second attempt fails. > > > I don't know if increasing the limits for the jumbo mbufs via > > sysctl > > will help. If you are using the code without Jack's patch, which > > uses > > 9K mbufs, then I think it can fragment the address space and result > > in no 9K contiguous areas to allocate from. (I'm just going by what > > Garrett and others have said about this.) > > > > > I never seem to be running out of mbufs - 4k or 9k. Unless it's > possible > for a starvation to occur without incrementing the counters. > Additionally, > netstat -m is recording denied mbufs on boot, so on a 96 Gig system > that is > just starting up, I don't think I am.. but a large increase in the > buffers > is on my list of desperation things to try. > > Thanks for the hint on m_getjcl().. I'll dig around and see if I can > find > what's happening there. I guess it's time for me to learn basic > dtrace as > well. :-) > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 00:47:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 132F226D; Mon, 24 Mar 2014 00:47:13 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id A659AA52; Mon, 24 Mar 2014 00:47:12 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAGl/L1ODaFve/2dsb2JhbABZg0FXgwe/bIEndIIlAQEBAwEjBFIFFg4KAgINGQIjNgYTh2UDCQgNqzWaXA2HBReBKYs0HIEfEQEcNAeCb4FJBJZdjlWFSYNJIYE1OQ X-IronPort-AV: E=Sophos;i="4.97,716,1389762000"; d="scan'208";a="108512871" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 23 Mar 2014 20:47:06 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2E808B4052; Sun, 23 Mar 2014 20:47:06 -0400 (EDT) Date: Sun, 23 Mar 2014 20:47:06 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 00:47:13 -0000 Christopher Forgeron wrote: > > > > > > > > > Update: > > For giggles, I set IP_MAXPACKET = 32768. > Well, I'm pretty sure you don't want to do that, except for an experiment. You can just set if_hw_tsomax to whatever you want to try, at the place my ixgbe.patch put it (just before the call to ether_ifattach()). > Over a hour of runtime, and no issues. This is better than with the > TSO patch and the 9.2 ixgbe, as that was just a drastic reduction in > errors. > So now the question becomes "how much does if_hw_tsomax need to be reduced from 65535 to get this?". If reducing it by the additional 4bytes for a vlan header is sufficient, then I understand what is going on. If it needs to be reduced by more than that, then there is something going on that I still don't understand. > Still have an 'angry' netstat -m on boot, and I'm still incrementing > denied netbuf calls, so something else is wrong. > > I'm going to modify Rick's prinft in ixgbe to also output when we're > over 32768. I'm sure it's still happening, but with an extra 32k of > space, we're not busting like we did before. > > > I notice a few interesting ip->ip_len changes since 9.2 - Like here, > at line 720 > > http://fxr.watson.org/fxr/diff/netinet/ip_output.c?v=FREEBSD10;im=kwqeqdhhvovqn;diffval=FREEBSD92;diffvar=v > > Looks like older code didn't byteswap with ntohs - I see that often > in tcp_output.c, and in tcp_options.c. > > > I'm also curious about this:Line 524 > http://fxr.watson.org/fxr/diff/netinet/ip_options.c?v=FREEBSD10;diffval=FREEBSD92;diffvar=v > > > New 10 code: > > ip ->ip_len = htons ( ntohs ( ip ->ip_len) + optlen); Old 9.2 Code: > ip ->ip_len += optlen; > Well, TSO segments aren't generated when optlen > 0, so I doubt this matters for our issue (and I would find it hard to believe that this would have been broken?). You can always look at the svn commit logs to see why/how something was changed. > > > I wonder if there are any unexpected consequences of these changes, > or perhaps a line someplace that doesn't make the change. > > Is there a dtrace command I could use to watch these functions and > compare the new ip_len with ip->ip_len or other variables? > > > > > > > > On Sun, Mar 23, 2014 at 12:25 PM, Christopher Forgeron < > csforgeron@gmail.com > wrote: > > > > > > > > On Sat, Mar 22, 2014 at 11:58 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > > Christopher Forgeron wrote: > > > > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax > > to > > make sure VLANs fit? > > > I took a look and, yes, this does seem to be needed. It will only be > needed for the case where a vlan is in use and hwtagging is disabled, > if I read the code correctly. > > > > Yes, or in the rare care where you configure your switch to pass the > v_lan header through to the NIC. > > > > Do you use vlans? > > > (Answered in above email) > > > > > > I've attached an updated patch. > > It might be nice to have the printf() patch in the driver too, so > we can see how big the ones that are too big are? > > > > Yes, I'm going to leave those in until I know we have this fixed.. > will probably leave it in a while longer as it should only have a > minor performance impact to iter-loop like that, and I'd like to see > what the story is a few months down the road. > > > Thanks for the patches, will have to start giving them code-names so > we can keep them straight. :-) I guess we have printf, tsomax, and > this one. > > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 05:14:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 803BAC2; Mon, 24 Mar 2014 05:14:07 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C89B6A; Mon, 24 Mar 2014 05:14:07 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so4825661qaj.7 for ; Sun, 23 Mar 2014 22:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+kJhGR4/N6D/tkhHRIlOX22FJGk1xjnOqeS99FUPD/U=; b=Av2nb97Tdck/hU3KpOvd59dpLt0eqlHIncE2KWuIQwQyAufg3Sc2hBQfvDG5YB8Yaq 2mjOT7RDvC/iX9+oGnoVTAaNRlAwHDhQGtfP7PEdulUYOwLCmu6ftn9QN7BiHx5RIqz0 qjVXKKNbDuHVMciT3hWor1NeEmwev4KHJiDkQvRZ8y5SV12m3rzSeWPgijcHoKGhhb7i FTvE6R3UlVbOa0DFUyOldQhl8wmCoOrUeih0npeObMWZHdz1JrZlLVHElz4xiudS6XmK lUHT0bvgrT85XlsuB2lBvEp5ZDlu+RXP13IQuB2al8Sel1ckQKtl3w5G9SaYg0DOI5dY kaDQ== MIME-Version: 1.0 X-Received: by 10.140.38.199 with SMTP id t65mr5532831qgt.28.1395638046291; Sun, 23 Mar 2014 22:14:06 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Sun, 23 Mar 2014 22:14:06 -0700 (PDT) In-Reply-To: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> Date: Mon, 24 Mar 2014 02:14:06 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 05:14:07 -0000 Hi, I'll follow up more tomorrow, as it's late and I don't have time for detail. The basic TSO patch didn't work, as packets were were still going over 65535 by a fair amount. I thought I wrote that earlier, but I am dumping a lot of info into a few threads, so I apologize if I'm not as concise as I could be. However, setting IP_MAXPACKET did. 4 hours of continuous run-time, no issues. No lost pings, no issues. Of course this isn't a fix - but it helps isolate the problem. I used IP_MAXPACKET = 32k originally, and I'm currently on 65495 bytes now (40 bytes shorter than IP_MAXPACKET). Of course, it's still sometimes going over IP_MAXPACKET, but since there is plenty of space to the real 65535 limit, we're not having any bad effects. Looking at my logs, I currently see the debug printf's showing packets of 65506 (11 bytes larger than IP_MAXPACKET). That's the real error. I should dump these packets to see what's in them.. perhaps wireshark or tcpdump is in order. One item of note is that setting the if's TSO doesn't matter in all output cases. I'll quote TSO code tomorrow that is still set to IP_MAXPACKET, thus setting the if's TSO isn't effective. From looking deeper at the transmit code, I thike I understand that we shouldn't be setting TSO to a value that is subtracted of headers, as the output routine should cover all angles, including TCP Options which can add size. I also booted up a 9.2-RELEASE system to make sure it was clean, and it is. No netstat -m denied stats. I didn't have time to put it under heavy load. The src diff between 10.0 and 9.2 holds the key to this problem. The largest changes I see are the use of htons / ntohs, and I have to wonder if there isn't a problem in there someplace. 9.2 -> 10 seems to have a lot of changes with how we calculate packet length and react to it, and this problem is one of packet length. Either we're not storing packet length properly, or we're not detecting it and fragmenting the packet properly. Lastly, I really should compile a debug kernel of 10-STABLE and check the asserts and the like. I will resume tomorrow as I feel we're maddeningly close. It's been ages since I had to do this much TCP, but I'm starting to get the hang of it again. Will resume tomorrow... On Sun, Mar 23, 2014 at 9:47 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > > > > > > > > > > > > > > > > > Update: > > > > For giggles, I set IP_MAXPACKET = 32768. > > > Well, I'm pretty sure you don't want to do that, except for an experiment. > You can just set if_hw_tsomax to whatever you want to try, at the place > my ixgbe.patch put it (just before the call to ether_ifattach()). > > > Over a hour of runtime, and no issues. This is better than with the > > TSO patch and the 9.2 ixgbe, as that was just a drastic reduction in > > errors. > > > So now the question becomes "how much does if_hw_tsomax need to be > reduced from 65535 to get this?". If reducing it by the additional > 4bytes for a vlan header is sufficient, then I understand what is > going on. If it needs to be reduced by more than that, then there > is something going on that I still don't understand. > > > Still have an 'angry' netstat -m on boot, and I'm still incrementing > > denied netbuf calls, so something else is wrong. > > > > I'm going to modify Rick's prinft in ixgbe to also output when we're > > over 32768. I'm sure it's still happening, but with an extra 32k of > > space, we're not busting like we did before. > > > > > > I notice a few interesting ip->ip_len changes since 9.2 - Like here, > > at line 720 > > > > > http://fxr.watson.org/fxr/diff/netinet/ip_output.c?v=FREEBSD10;im=kwqeqdhhvovqn;diffval=FREEBSD92;diffvar=v > > > > Looks like older code didn't byteswap with ntohs - I see that often > > in tcp_output.c, and in tcp_options.c. > > > > > > I'm also curious about this:Line 524 > > > http://fxr.watson.org/fxr/diff/netinet/ip_options.c?v=FREEBSD10;diffval=FREEBSD92;diffvar=v > > > > > > New 10 code: > > > > ip ->ip_len = htons ( ntohs ( ip ->ip_len) + optlen); Old 9.2 Code: > > ip ->ip_len += optlen; > > > Well, TSO segments aren't generated when optlen > 0, so I doubt this > matters for our issue (and I would find it hard to believe that this > would have been broken?). You can always look at the svn commit logs > to see why/how something was changed. > > > > > > > I wonder if there are any unexpected consequences of these changes, > > or perhaps a line someplace that doesn't make the change. > > > > Is there a dtrace command I could use to watch these functions and > > compare the new ip_len with ip->ip_len or other variables? > > > > > > > > > > > > > > > > On Sun, Mar 23, 2014 at 12:25 PM, Christopher Forgeron < > > csforgeron@gmail.com > wrote: > > > > > > > > > > > > > > > > On Sat, Mar 22, 2014 at 11:58 PM, Rick Macklem < rmacklem@uoguelph.ca > > > wrote: > > > > > > > > > > Christopher Forgeron wrote: > > > > > > > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax > > > to > > > make sure VLANs fit? > > > > > I took a look and, yes, this does seem to be needed. It will only be > > needed for the case where a vlan is in use and hwtagging is disabled, > > if I read the code correctly. > > > > > > > > Yes, or in the rare care where you configure your switch to pass the > > v_lan header through to the NIC. > > > > > > > > Do you use vlans? > > > > > > (Answered in above email) > > > > > > > > > > > > I've attached an updated patch. > > > > It might be nice to have the printf() patch in the driver too, so > > we can see how big the ones that are too big are? > > > > > > > > Yes, I'm going to leave those in until I know we have this fixed.. > > will probably leave it in a while longer as it should only have a > > minor performance impact to iter-loop like that, and I'd like to see > > what the story is a few months down the road. > > > > > > Thanks for the patches, will have to start giving them code-names so > > we can keep them straight. :-) I guess we have printf, tsomax, and > > this one. > > > > > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 06:40:26 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E532F08 for ; Mon, 24 Mar 2014 06:40:26 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2991B99D for ; Mon, 24 Mar 2014 06:40:26 +0000 (UTC) Received: by mail-pd0-f173.google.com with SMTP id z10so4911384pdj.18 for ; Sun, 23 Mar 2014 23:40:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Vdt+5Xdc78LeQ7D5y4Jj5ZdX3FbmjJ0KIIRE9pi9nkU=; b=Zyzws8OAcMaC12RWWRKU4kUeVC6pRzj8aKk5ilg0YJv2sjVl4IIX0bhVAF52E7xFdW eVY++Wm47NMcLG/ZLXHovU8V8PndsLrG+zhbRGOhT6oSauQRS65idjk+ybPE9By47EVy FfD3WJZ7p5PWHEgGUrlsYvIOhsb4Vk9snDwEbJw5ihN2AV1nAVdvdLb/hqFCBzH0q3X6 0/9WHa0EYO8jOKX4q7eydEtxYczlEr80CpLBD6WIUU/iirC813XEBzRVbnC5hcAtQvZo ujcukNSJoYjW5P51MadixGTvHA2zwXV9nqbB0pOtKS/iWbIeoR7bgrOWHFvFAOr/U2rN 0FiA== MIME-Version: 1.0 X-Received: by 10.68.40.138 with SMTP id x10mr70741301pbk.8.1395643225816; Sun, 23 Mar 2014 23:40:25 -0700 (PDT) Received: by 10.70.128.9 with HTTP; Sun, 23 Mar 2014 23:40:25 -0700 (PDT) Date: Mon, 24 Mar 2014 12:10:25 +0530 Message-ID: Subject: Handling Jumbo Packets From: soumya panigrahi To: rizzo.unipi@gmail.com, giuseppe.lettieri73@gmail.com, v.maffione@gmail.com, net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 06:40:26 -0000 Hi, First of all I would like to thank you all for developing an awesome packet capturing framework. I have been using it recently and find it very very fast and efficient. However off late I am trying to capture Jumbo packets on the network for the following scenario's but that doesn't seem to be working. Please suggest in its present form is NetMap capable of handling the below scenario's w.r.t Jumbo frames? If how do we have to some special handling w.r.t to this? 1) Netmap is receiving un-fragmented Jumbo frame. In this case is NetMap capable of passing on the Jumbo packet to the user? 2) Netmap receving fragmented Jumbo frames. In this case is NetMap capable of reassembling the fragmented packets and deliver the assembled packet to the user? I have been stuck with this issue for some time now. A quick response would help a great deal. Thanks, Soumya Panigrahi From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 08:07:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49133A45 for ; Mon, 24 Mar 2014 08:07:30 +0000 (UTC) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D8D3DFC7 for ; Mon, 24 Mar 2014 08:07:29 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id b15so4066256eek.20 for ; Mon, 24 Mar 2014 01:07:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=HKAnViBldTzzim1eZUlVJinemGf4F3G0pIQ/fUBqX1o=; b=i8HXMu8p42Z9r0vMYIQHyCxEKu/KC4cwEHvPaIcMh5XVYxNycrpzeFFtwO4wtdT5hW RHx/jtOSum0VIutomKW3zyAEaX48tSURlcc8DjswOwnB5MHZ5spPca5b1nFH/t1Wcb0E xbWwRuJFU2b5U4a2Coha1uPEZoqV7vZCoQHTokJPiZJGdyLXASU0DgaKQfFoeHY+XW+o 4wfCsT874vdzhQBn8+tzzgD91AbC5NgkWQNSTbNxrRJO/SVRUWohXq3UQpwWcCmTKl8P WFTAxb+GDJIZzetrU/0vCiC+SKrRjOOO30bxDZgLAgdEZWEIZVzYcH3j7X+RUf3BbGYj D5mA== MIME-Version: 1.0 X-Received: by 10.14.220.130 with SMTP id o2mr7687293eep.42.1395648448274; Mon, 24 Mar 2014 01:07:28 -0700 (PDT) Received: by 10.14.67.2 with HTTP; Mon, 24 Mar 2014 01:07:28 -0700 (PDT) Date: Mon, 24 Mar 2014 01:07:28 -0700 Message-ID: Subject: hostcache's effect on initcwnd calculation From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 08:07:30 -0000 Host A:10000 talks to Host B:20000 (10000 and 20000 being ports) and it gets logged in tcp's hostcache. Now if a new connection happens between A:10001 and Host B:20000, the initcwnd (initial congestion window) would not look at the CWND from hostcache but will always use what we've set as tp->snd_cwnd in cc_conn_init() of tcp_input.c - is this a correct assumption? What I want to confirm is, whether hostcache has any effect on initial congestion window of a new connection being setup. >From my reading of the code, cc_conn_init() always happens before we call tcp_hc_get() to get data out of hostcache to update other parameters of a connection. I do not see hc_entry->rmx_cwnd being actually used anywhere in tcp_input.c while connection bring up. Am I missing anything? cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 09:15:34 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CC573EF for ; Mon, 24 Mar 2014 09:15:34 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85A8D7E3 for ; Mon, 24 Mar 2014 09:15:33 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id n15so3447084lbi.13 for ; Mon, 24 Mar 2014 02:15:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=lRGcwyj9VbEOrLSLk0uwwTk8Hj4GEUwDQ4S4RdcUWco=; b=nIdDhZFBZMOE/KSgdL++5cGDwwhqLGA1TDpMkNr+rA3xXWhEX6KVeDDR5/pmqg0P9f kndzka/EoI7r9k2g2LjjeE7UrcLr4U4K0EPycnlPLEaqAEMEqBAtWxeNCD4B4b8DSeHL CReEmN6EzT1lEr3Xv6fp0/WrzMPcuSJidB86XJRZNy+MV9u26ZYwtLKbcZg2MMnR76Om Nyic8Bs76F/HRKzg4rJWece29rorxRlebBPdIUcRAU71MwbOhMi+atJ05v25dHSrxQEF Biw1DC5cfjs+LrDnwyVvXHhwPSjlXP5VuBtbYXjXkaJYTwLKgH4jNGtc5S+JtGpz5nLx jN1Q== MIME-Version: 1.0 X-Received: by 10.112.106.40 with SMTP id gr8mr9790420lbb.0.1395652531550; Mon, 24 Mar 2014 02:15:31 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Mon, 24 Mar 2014 02:15:31 -0700 (PDT) In-Reply-To: References: Date: Mon, 24 Mar 2014 10:15:31 +0100 X-Google-Sender-Auth: kAcsnjRNg7GwfxT1d1lgbsdNEWQ Message-ID: Subject: Re: Handling Jumbo Packets From: Luigi Rizzo To: soumya panigrahi Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Giuseppe Lettieri , Vincenzo Maffione , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 09:15:34 -0000 On Mon, Mar 24, 2014 at 7:40 AM, soumya panigrahi wrote: > Hi, > > First of all I would like to thank you all for developing an awesome > packet capturing framework. > I have been using it recently and find it very very fast and efficient. > > However off late I am trying to capture Jumbo packets on the network for > the following scenario's but that doesn't seem to be working. > the sysctl tree dev.netmap (on FreeBSD) or the filesystem /sys/module/netmap_lin/parameters (on Linux) have a number of parameters that control buffer numbers and sizes. In particular "buf_size" is used to allocate packet buffers and you can bump it up from 2048 to something larger to accommodate jumbo buffers. Depending on the NIC you may need to do something in the initialization of the NIC itself to allow sending/receiving frames larger than 1518 bytes, but I'd just try and see whether it works for you now. cheers luigi > Please suggest in its present form is NetMap capable of handling the below > scenario's w.r.t Jumbo frames? If how do we have to some special handling > w.r.t to this? > > 1) Netmap is receiving un-fragmented Jumbo frame. In this case is NetMap > capable of passing on the Jumbo packet to the user? > 2) Netmap receving fragmented Jumbo frames. In this case is NetMap > capable of reassembling the fragmented packets and deliver the assembled > packet to the user? > > I have been stuck with this issue for some time now. A quick response > would help a great deal. > > Thanks, > Soumya Panigrahi > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 09:47:20 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52C4D9B6 for ; Mon, 24 Mar 2014 09:47:20 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 27199A32 for ; Mon, 24 Mar 2014 09:47:20 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id kq14so5238951pab.9 for ; Mon, 24 Mar 2014 02:47:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xg4e6I5MFW9JtFsT8lzbD1C5DZEPTjKeAamXmjq6VzE=; b=SLMNrCvsek2DF/IPI0QDgoJvsFLt4esl1k2IH/dGS1zm7mgbrDzMkqX7GlT8ulEm2m oNTiwC/1zCxdH/eVZ4cAwxus4Lh6Cj6qp4VmG+/9VpHdPOvyxZzGYOcT69y+NQu1/wVA hmhu9t2RmW8o82iCCr3jTZbL/LSdwTOQAU7rw/aZxd2iq0G696gGfOMMbmrwHZfDb4Ec CyrPqNi7PJea5fDSXf62SwAOdwntpvIJJMZRQTLQe038TvI0NeOuNVYkR6J0aUF5U7cX Q0TtsgZdtfgJYa9vosV6kkNFjgMur7RoCQ0R/MvjQ0Xh3WzZMNtx+KVHRvC3GAA92/HV XWoA== MIME-Version: 1.0 X-Received: by 10.66.12.67 with SMTP id w3mr72012514pab.29.1395654439754; Mon, 24 Mar 2014 02:47:19 -0700 (PDT) Received: by 10.70.128.9 with HTTP; Mon, 24 Mar 2014 02:47:19 -0700 (PDT) In-Reply-To: References: Date: Mon, 24 Mar 2014 15:17:19 +0530 Message-ID: Subject: Re: Handling Jumbo Packets From: soumya panigrahi To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Giuseppe Lettieri , Vincenzo Maffione , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 09:47:20 -0000 On Mon, Mar 24, 2014 at 2:45 PM, Luigi Rizzo wrote: > > > > On Mon, Mar 24, 2014 at 7:40 AM, soumya panigrahi < > soumyapanigrahi@gmail.com> wrote: > >> Hi, >> >> First of all I would like to thank you all for developing an awesome >> packet capturing framework. >> I have been using it recently and find it very very fast and efficient. >> >> However off late I am trying to capture Jumbo packets on the network for >> the following scenario's but that doesn't seem to be working. >> > > the sysctl tree dev.netmap (on FreeBSD) > or the filesystem > /sys/module/netmap_lin/parameters > (on Linux) > have a number of parameters that control buffer numbers and sizes. > In particular "buf_size" is used to allocate packet buffers and > you can bump it up from 2048 to something larger to accommodate > jumbo buffers. > > Depending on the NIC you may need to do something in the initialization > of the NIC itself to allow sending/receiving frames larger than 1518 bytes, > but I'd just try and see whether it works for you now. > > cheers > luigi > Thanks a lot for the quick response. I will also try to see if we can handle Jumbo frames by tweaking the buf_size parameter on Linux. In addition can you advice if NetMap can de-fragment Jumbo packets that were fragmented as IP packets by intermediate IP device because of MTU size limitation? For example a 9000 Bytes single ethernet frame was fragmented into 1516 Bytes multiple ethernet frames at IP layer. Can netmap re-assemble these fragmented ethernet frames into a single Jumbo frame before passing it the application? If so how? Thanks, Soumya > > > >> Please suggest in its present form is NetMap capable of handling the >> below scenario's w.r.t Jumbo frames? If how do we have to some special >> handling w.r.t to this? >> >> 1) Netmap is receiving un-fragmented Jumbo frame. In this case is NetMap >> capable of passing on the Jumbo packet to the user? >> 2) Netmap receving fragmented Jumbo frames. In this case is NetMap >> capable of reassembling the fragmented packets and deliver the assembled >> packet to the user? >> >> I have been stuck with this issue for some time now. A quick response >> would help a great deal. >> >> Thanks, >> Soumya Panigrahi >> > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 09:55:36 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A43DC64 for ; Mon, 24 Mar 2014 09:55:36 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E1DCFACF for ; Mon, 24 Mar 2014 09:55:35 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id s7so3396175lbd.23 for ; Mon, 24 Mar 2014 02:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5RBaAxGCc74OQt5WUQ1IsEm531yEEKsMM9rbXYrPcrc=; b=mgi3zcpfRVpD/l1hkRda9uvD0O6eYgFZ07vgiiMgO1Uphx1Wj5Z1ia1CxHOP4V1S4P ii9bSuz4b1LJCmCZpiirtwkYwuPR0aMpbAWft5Bn/b33Cf315IDwmzi+1pJ6V/1dU1/g M8BfZtI+oKCVZ9P+9UUGEBMvZjR3BoG5auUXWihGnnBR5eA9uQBvAglD22GrGjx+wfiQ 44fQZYBMlA8hcPBd8EZ68wh7Ae1leM3sUpxBrOyJMwPaxkgYEW63rOEzFc0w2E7HE8Ld aMI9Q13kJ4iZJMv2Z1+PMFMtE6fuKytOoBUS2iEFAUlUaO6p+EJBy7S9Jz5Sc/8dAbaj Gphw== MIME-Version: 1.0 X-Received: by 10.152.190.135 with SMTP id gq7mr5258847lac.28.1395654933943; Mon, 24 Mar 2014 02:55:33 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Mon, 24 Mar 2014 02:55:33 -0700 (PDT) In-Reply-To: References: Date: Mon, 24 Mar 2014 10:55:33 +0100 X-Google-Sender-Auth: GKRiDu_kNVjt-RfVuAdSon7FOlo Message-ID: Subject: Re: Handling Jumbo Packets From: Luigi Rizzo To: soumya panigrahi Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Giuseppe Lettieri , Vincenzo Maffione , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 09:55:36 -0000 On Mon, Mar 24, 2014 at 10:47 AM, soumya panigrahi < soumyapanigrahi@gmail.com> wrote: > > On Mon, Mar 24, 2014 at 2:45 PM, Luigi Rizzo wrote: > >> >> >> >> On Mon, Mar 24, 2014 at 7:40 AM, soumya panigrahi < >> soumyapanigrahi@gmail.com> wrote: >> >>> Hi, >>> >>> First of all I would like to thank you all for developing an awesome >>> packet capturing framework. >>> I have been using it recently and find it very very fast and efficient. >>> >>> However off late I am trying to capture Jumbo packets on the network for >>> the following scenario's but that doesn't seem to be working. >>> >> >> the sysctl tree dev.netmap (on FreeBSD) >> or the filesystem >> /sys/module/netmap_lin/parameters >> (on Linux) >> have a number of parameters that control buffer numbers and sizes. >> In particular "buf_size" is used to allocate packet buffers and >> you can bump it up from 2048 to something larger to accommodate >> jumbo buffers. >> >> Depending on the NIC you may need to do something in the initialization >> of the NIC itself to allow sending/receiving frames larger than 1518 >> bytes, >> but I'd just try and see whether it works for you now. >> >> cheers >> luigi >> > > Thanks a lot for the quick response. > I will also try to see if we can handle Jumbo frames by tweaking the > buf_size parameter on Linux. > In addition can you advice if NetMap can de-fragment Jumbo packets that > were fragmented as IP packets by intermediate IP device because of MTU size > limitation? > no, netmap does not do fragmentation or reassembly cheers luigi -- From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44F0B73 for ; Mon, 24 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2F134175 for ; Mon, 24 Mar 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2OB6nOs013927 for ; Mon, 24 Mar 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2OB6mxH013925 for freebsd-net@FreeBSD.org; Mon, 24 Mar 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Mar 2014 11:06:48 GMT Message-Id: <201403241106.s2OB6mxH013925@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 466 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 14:56:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F718BF9; Mon, 24 Mar 2014 14:56:56 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1C0F7EA9; Mon, 24 Mar 2014 14:56:56 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id i8so6019162qcq.17 for ; Mon, 24 Mar 2014 07:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sogmB9J7Efu7vmxTxdGy5sGBfwqNvWaLzRtLEMGCQZM=; b=J6Yh5rh1HAaYn2HMQFKPZiIL2IQxdla8EzEbISLPokgbS+flp8Tb0+V098Cv/5pENd 5vSKIHpbelotegquEThQ4e406Y2TQQ40M991ZE9GdRNS1NJELcM9Q3wOwu4siDE0nU7Y FW2HRAlhDq7p8rsrW9QvnwHfrSdEbeI0U1Of+q9BUxX2ppB4NTd++q4fZjEBrSz58RIw lSxwG9FlwmdCVmlRQBT1zvHZ1ioo/wk9gF1YrBcKfKmE671PB+ShTaN8DIBOq0n/y2+/ hO/+wGoXIdpPooqUS0gE5xDvXtsjbPygUN82h/XsmNHJBIu++ssr1SKLQHRzYwGZe5lS ThLA== MIME-Version: 1.0 X-Received: by 10.140.48.199 with SMTP id o65mr36702147qga.16.1395673015311; Mon, 24 Mar 2014 07:56:55 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Mon, 24 Mar 2014 07:56:55 -0700 (PDT) In-Reply-To: References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> Date: Mon, 24 Mar 2014 11:56:55 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 14:56:56 -0000 I'm going to split this into different posts to focus on each topic. This is about setting IP_MAXPACKET to 65495 Update on Last Night's Run: (Last night's run is a kernel with IP_MAXPACKET = 65495) - Uptime on this run: 10:53AM up 13:21, 5 users, load averages: 1.98, 2.09, 2.13 - Ping logger records no ping errors for the entire run. - At Mar 24th 10:57 I did a grep through the night's log for 'before' (which is the printf logging that Rick suggested a few days ago), and saved it to before_total.txt - With wc -l on before_total.txt I can see that we have 504 lines, thus 504 incidents of the packet being above IP_MAXPACKET during this run. - I did tr -c '[:alnum:]' '[\n*]' < before_total.txt | sort | uniq -c | sort -nr | head -50 to list the most common words. Ignoring the non-pklen output. The relevant output is: 344 65498 (3) 330 65506 (11) 330 65502 (7) - First # being the # of times. (Each pklen is printed twice on the log, thus 2x the total line count). - Last (#) being the byte overrun from 65495 - A fairly even distribution of each type of packet overrun. You will recall that my IP_MAXPACKET is 65495, so each of these packet lengths represents a overshoot. The fact that we have only 3 different types of overrun is good - It suggests a non-random event, more like a broken 'if' statement for a particular case. If IP_MAXPACKET was set to 65535 as it normally is, I would have had 504 incidents of errors, with a chance that any one of them could have blocked the queue for considerable time. Question: Should there be logic that discards packets that are over IP_MAXPACKET to ensure that we don't end up in a blocked queue situation again? Moving forward, I am doing two things: 1) I'm running a longer test with TSO disabled on my ix0 adapter. I want to make sure that over say 4 hours I don't have even 1 packet over 65495. This will at least locate the issue to TSO related code. 2) I have tcpdump running, to see if I can capture the packets over 65495. Here is my command. Any suggestions on additional switches I should include? tcpdump -ennvvXS greater 65495 I'll report in on this again once I have new info. Thanks for reading. On Mon, Mar 24, 2014 at 2:14 AM, Christopher Forgeron wrote: > Hi, > > I'll follow up more tomorrow, as it's late and I don't have time for > detail. > > The basic TSO patch didn't work, as packets were were still going over > 65535 by a fair amount. I thought I wrote that earlier, but I am dumping a > lot of info into a few threads, so I apologize if I'm not as concise as I > could be. > > However, setting IP_MAXPACKET did. 4 hours of continuous run-time, no > issues. No lost pings, no issues. Of course this isn't a fix - but it helps > isolate the problem. > > what the story is a few months down the road. > > > > > > Thanks for the patches, will have to start giving them code-names so > > we can keep them straight. :-) I guess we have printf, tsomax, and > > this one. > > > > > > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 15:21:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A22F874; Mon, 24 Mar 2014 15:21:29 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C53C9167; Mon, 24 Mar 2014 15:21:28 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so6142708qcz.16 for ; Mon, 24 Mar 2014 08:21:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=05A6UjaX2avV4XRZJqQulpScg8ZrhsljB/RGsgcr6Po=; b=Gh0yWzmcN+6jt/QxaT+Rv1i8znbQHXkKWiw2FA8Go8nns+TT4tKaf1pXm/+aqDjXDh A14ynSpHcO3R2NXpjdz69oH7acIc2LCewMsLQ/HI/CsBEHWBwtM9e6NplnWoa4OosLqV re5b43n0nq/VwZtQs5UQZ28f1FNXzGiKMYVd7+pkDC7kylEA2XNqGhLz1UYrwZpR/TSJ 2nltOYWtxcec59AcJVSRRXbuzAnntS6WOFUxmkQx9WJPau9/MpvJWvGXj1jyZ/dHRyQy q+4/xU2pwLVlyFostQkMsaZfwq7Zj/KRLi1jNrqxeKmuvw80E22I9747finkzbjA+GVu MflA== MIME-Version: 1.0 X-Received: by 10.224.69.66 with SMTP id y2mr9605259qai.25.1395674486649; Mon, 24 Mar 2014 08:21:26 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Mon, 24 Mar 2014 08:21:26 -0700 (PDT) In-Reply-To: References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> Date: Mon, 24 Mar 2014 12:21:26 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 15:21:29 -0000 This is regarding the TSO patch that Rick suggested earlier. (With many thanks for his time and suggestion) As I mentioned earlier, it did not fix the issue on a 10.0 system. It did make it less of a problem on 9.2, but either way, I think it's not needed, and shouldn't be considered as a patch for testing/etc. Patching TSO to anything other than a max value (and by default the code gives it IP_MAXPACKET) is confusing the matter, as the packet length ultimately needs to be adjusted for many things on the fly like TCP Options, etc. Using static header sizes won't be a good idea. Additionally, it seems that setting nic TSO will/may be ignored by code like this in sys/netinet/tcp_output.c: 10.0 Code: 780 if (len > tp->t_tsomax - hdrlen) { !! 781 len = tp->t_tsomax - hdrlen; !! 782 sendalot = 1; 783 } I've put debugging here, set the nic's max TSO as per Rick's patch ( set to say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. It's being set someplace else, and thus our attempts to set TSO on the nic may be in vain. It may have mattered more in 9.2, as I see the code doesn't use tp->t_tsomax in some locations, and may actually default to what the nic is set to. The NIC may still win, I didn't walk through the code to confirm, it was enough to suggest to me that setting TSO wouldn't fix this issue. However, this is still a TSO related issue, it's just not one related to the setting of TSO's max size. A 10.0-STABLE system with tso disabled on ix0 doesn't have a single packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer to increase confidence in this assertion, but I don't want to waste time on this when I could be logging problem packets on a system with TSO enabled. Comments are very welcome.. From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 15:45:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E01AA793 for ; Mon, 24 Mar 2014 15:45:31 +0000 (UTC) Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46]) by mx1.freebsd.org (Postfix) with SMTP id 91C263E5 for ; Mon, 24 Mar 2014 15:45:29 +0000 (UTC) Received: (qmail 30672 invoked by uid 1001); 24 Mar 2014 15:45:22 -0000 Delivered-To: qmda-intercept-freebsd-net@freebsd.org DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; b=caS7RxAAMaZaHOsx3da03ZqFVrJdirapuMl0fv7yixU2dudeZYC13yPPk4Th240u; Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys DomainKey-Trace-MD: h=10; b=30; l=C18R71D32M65F38T27S64M17C39C27; Comments: QMDA 0.3 Received: (qmail 30665 invoked by uid 1001); 24 Mar 2014 15:45:22 -0000 Date: 24 Mar 2014 15:45:22 +0000 Message-ID: <20140324154522.30664.qmail@f5-external.bushwire.net> From: "Mark Delany" To: freebsd-net@freebsd.org Subject: How to make netmap NS_FORWARD work with NR_REG_ONE_NIC? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 15:45:31 -0000 (Very recent fbsd10) My application is taking advantage of NR_REG_ONE_NIC to register separate handlers for each h/w ring. (Pro tip, you must re-open /dev/netmap each time, a dup() fd doesn't work). While good for performance, it unfortunately appears that NS_FORWARD does not work in this mode - presumably because NR_REG_ONE_NIC doesn't include the host ring. Now it's possible I simply have a bug and NS_FORWARD should work with NR_REG_ONE_NIC - I have demo code I can share if needed - but assuming for a moment I don't have a bug... What's the right way to implement host forwarding with NR_REG_ONE_NIC? Do I have to register an independent handler with NR_REG_SW_NIC and have the h/w ring handlers synchronize with it? If so, I also presume the NR_REG_SW_NIC handler has to manage transfers in both directions, yes? I'm not keen on that approach if I can avoid it as it introduces synchronization costs between handlers where previously I needed none. I also looked at the kernel module and determined that nr_flags = NR_REG_ONE_NIC | NR_REG_SW_NIC is invalid. Oh well. That would have solved my problem nicely. Mark. From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 16:10:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89E4E14A for ; Mon, 24 Mar 2014 16:10:00 +0000 (UTC) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EFF51862 for ; Mon, 24 Mar 2014 16:09:59 +0000 (UTC) Received: by mail-la0-f41.google.com with SMTP id gl10so3819428lab.14 for ; Mon, 24 Mar 2014 09:09:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=efh/MOoEp+ngiJswcPM0V7+8aEtjTuXpACTBrymy40U=; b=i8vt9vabyEIxbxwZFnRIYRv7kNpRyT0W4iG5DDmwzN/72i1T43IvmDTCXXca9YnI/q lgY+Sr3TqkIMLKbx4mxTIgBtHIA0SbjWUkRF25vIsfmKhOuEsNLKSAqNeGNDo0ad0YjU tnbD9423kUdi7xL/g81z3TjuAQbMjhDTEbsalZV91UrtoblkFQ2fdVNfCC5c+d40PM6p KrzRoiNzvMWl68KwZ1PfMcnoL4Xk2xLc7Mc50XVXvh0Qn6rdSXl239McLPqDaoZgzJ6s u1KvJkqU6AirJpFhbitHe2YmnbmyEKmuQJVdmsO4uIT8o1fzj6pxwDGczvAZ0kFoxq+k F89Q== MIME-Version: 1.0 X-Received: by 10.112.156.165 with SMTP id wf5mr2282687lbb.37.1395677397961; Mon, 24 Mar 2014 09:09:57 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Mon, 24 Mar 2014 09:09:57 -0700 (PDT) In-Reply-To: <20140324154522.30664.qmail@f5-external.bushwire.net> References: <20140324154522.30664.qmail@f5-external.bushwire.net> Date: Mon, 24 Mar 2014 17:09:57 +0100 X-Google-Sender-Auth: B411E2F2MK4wECKTZnFiuQ1vfrc Message-ID: Subject: Re: How to make netmap NS_FORWARD work with NR_REG_ONE_NIC? From: Luigi Rizzo To: Mark Delany Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 16:10:00 -0000 On Mon, Mar 24, 2014 at 4:45 PM, Mark Delany wrote: > (Very recent fbsd10) > > My application is taking advantage of NR_REG_ONE_NIC to register > separate handlers for each h/w ring. (Pro tip, you must re-open > /dev/netmap each time, a dup() fd doesn't work). > > While good for performance, it unfortunately appears that NS_FORWARD > does not work in this mode - presumably because NR_REG_ONE_NIC doesn't > include the host ring. > correct, this is not a supported mode at the moment. If you want to implement it you should do it into netmap_poll() and create another constant NS_REG_ONE_NIC_SW for the new mode. The difficulty is that we do not really have the equivalent of a multiqueue host port in our system. cheers luigi > Now it's possible I simply have a bug and NS_FORWARD should work with > NR_REG_ONE_NIC - I have demo code I can share if needed - but assuming > for a moment I don't have a bug... > > What's the right way to implement host forwarding with NR_REG_ONE_NIC? > > Do I have to register an independent handler with NR_REG_SW_NIC and > have the h/w ring handlers synchronize with it? If so, I also presume > the NR_REG_SW_NIC handler has to manage transfers in both directions, > yes? > > I'm not keen on that approach if I can avoid it as it introduces > synchronization costs between handlers where previously I needed none. > > I also looked at the kernel module and determined that nr_flags = > NR_REG_ONE_NIC | NR_REG_SW_NIC is invalid. Oh well. That would have > solved my problem nicely. > > > Mark. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 16:14:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8701125D; Mon, 24 Mar 2014 16:14:17 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 41C86909; Mon, 24 Mar 2014 16:14:17 +0000 (UTC) Received: from [2001:1620:2013:1:7810:eaed:8406:ff6] (port=49631) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WS7Vt-0005dw-9M; Mon, 24 Mar 2014 17:14:13 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Mon, 24 Mar 2014 17:14:08 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 16:14:17 -0000 On 24.03.2014, at 16:21, Christopher Forgeron = wrote: > This is regarding the TSO patch that Rick suggested earlier. (With = many > thanks for his time and suggestion) >=20 > As I mentioned earlier, it did not fix the issue on a 10.0 system. It = did > make it less of a problem on 9.2, but either way, I think it's not = needed, > and shouldn't be considered as a patch for testing/etc. >=20 > Patching TSO to anything other than a max value (and by default the = code > gives it IP_MAXPACKET) is confusing the matter, as the packet length > ultimately needs to be adjusted for many things on the fly like TCP > Options, etc. Using static header sizes won't be a good idea. >=20 > Additionally, it seems that setting nic TSO will/may be ignored by = code > like this in sys/netinet/tcp_output.c: >=20 > 10.0 Code: >=20 > 780 if (len > tp->t_tsomax - hdrlen) > { !! > 781 len =3D tp->t_tsomax - > hdrlen; !! > 782 sendalot =3D > 1; > 783 } >=20 >=20 > I've put debugging here, set the nic's max TSO as per Rick's patch ( = set to > say 32k), and have seen that tp->t_tsomax =3D=3D IP_MAXPACKET. It's = being set > someplace else, and thus our attempts to set TSO on the nic may be in = vain. >=20 > It may have mattered more in 9.2, as I see the code doesn't use > tp->t_tsomax in some locations, and may actually default to what the = nic is > set to. >=20 > The NIC may still win, I didn't walk through the code to confirm, it = was > enough to suggest to me that setting TSO wouldn't fix this issue. I just applied Rick=92s ixgbe TSO patch and additionally wanted to be = able to easily change the value of hw_tsomax, so I made a sysctl out of = it. While doing that, I asked myself the same question. Where and how will = this value actually be used and how comes that tcp_output() uses that = other value in struct tcpcb. The only place tcpcb->t_tsomax gets set, that I have found so far, is in = tcp_input.c=92s tcp_mss() function. Some subfunctions get called: tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() Then tcp_maxmtu() indeed uses the interface=92s hw_tsomax value: 1746 cap->tsomax =3D ifp->if_hw_tsomax; It get=92s passed back to tcp_mss() where it is set on the connection = level which will be used in tcp_output() later on. tcp_mss() gets called from multiple places, I=92ll look into that later. = I will let you know if I find out more. Markus > However, this is still a TSO related issue, it's just not one related = to > the setting of TSO's max size. >=20 > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single = packet > over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer to > increase confidence in this assertion, but I don't want to waste time = on > this when I could be logging problem packets on a system with TSO = enabled. >=20 > Comments are very welcome.. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 16:23:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19E8F5DA; Mon, 24 Mar 2014 16:23:23 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B79759EA; Mon, 24 Mar 2014 16:23:22 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id j107so17266961qga.7 for ; Mon, 24 Mar 2014 09:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=HZC408ZRAR5ZKn5crL4/up8voxBDnFbZ86OpBmgiICY=; b=azkz133epfpFBehKlZK0Em5m6JWooSK/1uMfSA0x3Um1mbW1oiwNV+Jy8h0+o7PfBY dUCWANRGFXsOdjQA5ll5D0syve9dHENuV5EWb+nZW8Ef97uuDljFxBFHnzjmLGUGP/4L NKnfiBSHNRWlXs7D5nNMP9bXgwlCs1EMzonnvlimofTyuz3HhyXzTvkCYKhaVfMF6eeo /JDEwMoCzIvgPVS23SxIXiAaiWI48ovFhAwInQw6iOi0cJjaBmGwZmBjilmnRB+AZz4S ZiJFvRwG3iYL2wfaXfANcK9rCAwaJuPIipHA03pi8Gd12O7WojMIoXt4b2C/+jAinbxn F/ig== MIME-Version: 1.0 X-Received: by 10.140.29.68 with SMTP id a62mr58308775qga.57.1395678201389; Mon, 24 Mar 2014 09:23:21 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Mon, 24 Mar 2014 09:23:21 -0700 (PDT) In-Reply-To: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> Date: Mon, 24 Mar 2014 13:23:21 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 16:23:23 -0000 I think making hw_tsomax a sysctl would be a good patch to commit - It could enable easy debugging/performance testing for the masses. I'm curious to hear how your environment is working with a tso turned off on your nics. My testbed just hit the 2 hour mark. With TSO off, I don't get a single packet over IP_MAXPACKET. That puts my confidence at around 95% in the statement 'turning off tso negates this issue for me'. I'm now rebooting into a +tso env to see if I can capture the bad packets. I am also sure that the netstat -m mbuf denied is a completely separate issue. I'm going around the lab and powering up different boxes with 10.0-RELEASE, and they all have mbuf/mbuf clusters denied on boot, and that number increases with network traffic. It's probably not helping the IP_MAXPACKET issue. I'll create a separate thread for that one shortly. On Mon, Mar 24, 2014 at 1:14 PM, Markus Gebert wrote: > > On 24.03.2014, at 16:21, Christopher Forgeron > wrote: > > > This is regarding the TSO patch that Rick suggested earlier. (With many > > thanks for his time and suggestion) > > > > As I mentioned earlier, it did not fix the issue on a 10.0 system. It did > > make it less of a problem on 9.2, but either way, I think it's not > needed, > > and shouldn't be considered as a patch for testing/etc. > > > > Patching TSO to anything other than a max value (and by default the code > > gives it IP_MAXPACKET) is confusing the matter, as the packet length > > ultimately needs to be adjusted for many things on the fly like TCP > > Options, etc. Using static header sizes won't be a good idea. > > > > Additionally, it seems that setting nic TSO will/may be ignored by code > > like this in sys/netinet/tcp_output.c: > > > > 10.0 Code: > > > > 780 if (len > tp->t_tsomax - hdrlen) > > { !! > > 781 len = tp->t_tsomax - > > hdrlen; !! > > 782 sendalot = > > 1; > > 783 } > > > > > > I've put debugging here, set the nic's max TSO as per Rick's patch ( set > to > > say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. It's being set > > someplace else, and thus our attempts to set TSO on the nic may be in > vain. > > > > It may have mattered more in 9.2, as I see the code doesn't use > > tp->t_tsomax in some locations, and may actually default to what the nic > is > > set to. > > > > The NIC may still win, I didn't walk through the code to confirm, it was > > enough to suggest to me that setting TSO wouldn't fix this issue. > > > I just applied Rick's ixgbe TSO patch and additionally wanted to be able > to easily change the value of hw_tsomax, so I made a sysctl out of it. > > While doing that, I asked myself the same question. Where and how will > this value actually be used and how comes that tcp_output() uses that other > value in struct tcpcb. > > The only place tcpcb->t_tsomax gets set, that I have found so far, is in > tcp_input.c's tcp_mss() function. Some subfunctions get called: > > tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() > > Then tcp_maxmtu() indeed uses the interface's hw_tsomax value: > > 1746 cap->tsomax = ifp->if_hw_tsomax; > > It get's passed back to tcp_mss() where it is set on the connection level > which will be used in tcp_output() later on. > > tcp_mss() gets called from multiple places, I'll look into that later. I > will let you know if I find out more. > > > Markus > > > > However, this is still a TSO related issue, it's just not one related to > > the setting of TSO's max size. > > > > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > packet > > over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer to > > increase confidence in this assertion, but I don't want to waste time on > > this when I could be logging problem packets on a system with TSO > enabled. > > > > Comments are very welcome.. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 16:36:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 721F4CAC; Mon, 24 Mar 2014 16:36:07 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F255AAF5; Mon, 24 Mar 2014 16:36:06 +0000 (UTC) Received: from [2001:1620:2013:1:7810:eaed:8406:ff6] (port=50246) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WS7r2-0006hT-NS; Mon, 24 Mar 2014 17:36:04 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Mon, 24 Mar 2014 17:36:03 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 16:36:07 -0000 On 24.03.2014, at 17:23, Christopher Forgeron = wrote: > I think making hw_tsomax a sysctl would be a good patch to commit - It > could enable easy debugging/performance testing for the masses. >=20 > I'm curious to hear how your environment is working with a tso turned = off > on your nics. This will take some more time. Only one of the affected systems is = running the test kernel with prinfts, additional sysctl, and Rick=92s = patch. I want to be able to reproduce the problem with that patch, = before changing another variable (like turning TSO off), but that can = take days on one server. I=92ll probably be able to equip some more = servers with that kernel soon, and might run a subgroup without TSO. But = first I have to make sure, the new kernel doesn=92t add any new = problemes, we can=92t afford them on productive servers. > My testbed just hit the 2 hour mark. With TSO off, I don't get a = single > packet over IP_MAXPACKET. That puts my confidence at around 95% in = the > statement 'turning off tso negates this issue for me'. >=20 > I'm now rebooting into a +tso env to see if I can capture the bad = packets. >=20 > I am also sure that the netstat -m mbuf denied is a completely = separate > issue. I'm going around the lab and powering up different boxes with > 10.0-RELEASE, and they all have mbuf/mbuf clusters denied on boot, and = that > number increases with network traffic. It's probably not helping the > IP_MAXPACKET issue. While we have most symptoms in common, I=92ve still not seen any = allocation error in netstat -m. So I tend to agree that this is most = probably a different problem. Markus > I'll create a separate thread for that one shortly. >=20 >=20 > On Mon, Mar 24, 2014 at 1:14 PM, Markus Gebert > wrote: >=20 >>=20 >> On 24.03.2014, at 16:21, Christopher Forgeron >> wrote: >>=20 >>> This is regarding the TSO patch that Rick suggested earlier. (With = many >>> thanks for his time and suggestion) >>>=20 >>> As I mentioned earlier, it did not fix the issue on a 10.0 system. = It did >>> make it less of a problem on 9.2, but either way, I think it's not >> needed, >>> and shouldn't be considered as a patch for testing/etc. >>>=20 >>> Patching TSO to anything other than a max value (and by default the = code >>> gives it IP_MAXPACKET) is confusing the matter, as the packet length >>> ultimately needs to be adjusted for many things on the fly like TCP >>> Options, etc. Using static header sizes won't be a good idea. >>>=20 >>> Additionally, it seems that setting nic TSO will/may be ignored by = code >>> like this in sys/netinet/tcp_output.c: >>>=20 >>> 10.0 Code: >>>=20 >>> 780 if (len > tp->t_tsomax - hdrlen) >>> { !! >>> 781 len =3D tp->t_tsomax - >>> hdrlen; !! >>> 782 sendalot =3D >>> 1; >>> 783 } >>>=20 >>>=20 >>> I've put debugging here, set the nic's max TSO as per Rick's patch ( = set >> to >>> say 32k), and have seen that tp->t_tsomax =3D=3D IP_MAXPACKET. It's = being set >>> someplace else, and thus our attempts to set TSO on the nic may be = in >> vain. >>>=20 >>> It may have mattered more in 9.2, as I see the code doesn't use >>> tp->t_tsomax in some locations, and may actually default to what the = nic >> is >>> set to. >>>=20 >>> The NIC may still win, I didn't walk through the code to confirm, it = was >>> enough to suggest to me that setting TSO wouldn't fix this issue. >>=20 >>=20 >> I just applied Rick's ixgbe TSO patch and additionally wanted to be = able >> to easily change the value of hw_tsomax, so I made a sysctl out of = it. >>=20 >> While doing that, I asked myself the same question. Where and how = will >> this value actually be used and how comes that tcp_output() uses that = other >> value in struct tcpcb. >>=20 >> The only place tcpcb->t_tsomax gets set, that I have found so far, is = in >> tcp_input.c's tcp_mss() function. Some subfunctions get called: >>=20 >> tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() >>=20 >> Then tcp_maxmtu() indeed uses the interface's hw_tsomax value: >>=20 >> 1746 cap->tsomax =3D = ifp->if_hw_tsomax; >>=20 >> It get's passed back to tcp_mss() where it is set on the connection = level >> which will be used in tcp_output() later on. >>=20 >> tcp_mss() gets called from multiple places, I'll look into that = later. I >> will let you know if I find out more. >>=20 >>=20 >> Markus >>=20 >>=20 >>> However, this is still a TSO related issue, it's just not one = related to >>> the setting of TSO's max size. >>>=20 >>> A 10.0-STABLE system with tso disabled on ix0 doesn't have a single >> packet >>> over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer = to >>> increase confidence in this assertion, but I don't want to waste = time on >>> this when I could be logging problem packets on a system with TSO >> enabled. >>>=20 >>> Comments are very welcome.. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 17:04:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45292888; Mon, 24 Mar 2014 17:04:18 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB86FD99; Mon, 24 Mar 2014 17:04:17 +0000 (UTC) Received: from julian-mbp3.pixel8networks.com (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s2OGsCbu008184 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 24 Mar 2014 09:54:13 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5330632E.3050505@freebsd.org> Date: Mon, 24 Mar 2014 09:54:06 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rick Macklem , Christopher Forgeron Subject: Re: 9.2 ixgbe tx queue hang References: <1936201708.1670348.1395619029045.JavaMail.root@uoguelph.ca> In-Reply-To: <1936201708.1670348.1395619029045.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 17:04:18 -0000 On 3/23/14, 4:57 PM, Rick Macklem wrote: > Christopher Forgeron wrote: >> >> >> >> >> >> On Sat, Mar 22, 2014 at 6:41 PM, Rick Macklem < rmacklem@uoguelph.ca >>> wrote: >> >> >> Christopher Forgeron wrote: >>> #if defined(INET) || defined(INET6) >>> /* Initialize to max value. */ >>> if (ifp->if_hw_tsomax == 0) >>> ifp->if_hw_tsomax = IP_MAXPACKET; >>> KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && >>> ifp->if_hw_tsomax >= IP_MAXPACKET / 8, >>> ("%s: tsomax outside of range", __func__)); >>> #endif >>> >>> >>> Should this be the location where it's being set rather than in >>> ixgbe? I would assume that other drivers could fall prey to this >>> issue. >>> >> All of this should be prepended with "I'm an NFS guy, not a >> networking >> guy, so I might be wrong". >> >> Other drivers (and ixgbe for the 82598 chip) can handle a packet that >> is in more than 32 mbufs. (I think the 82598 handles 100, grep for >> SCATTER >> in *.h in sys/dev/ixgbe.) >> the Xen backend can not handle mor ethan 32 segments in some versions of Xen. From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 17:19:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4819F55 for ; Mon, 24 Mar 2014 17:19:47 +0000 (UTC) Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46]) by mx1.freebsd.org (Postfix) with SMTP id 83EA2ECA for ; Mon, 24 Mar 2014 17:19:46 +0000 (UTC) Received: (qmail 31375 invoked by uid 1001); 24 Mar 2014 17:19:42 -0000 Delivered-To: qmda-intercept-freebsd-net@freebsd.org DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; b=hCvzz7J6G903AkMw0PdVqn0yKAd23+mysYMjDk/GvgBfIXUhflH1voQCPgk/Z7pp; Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys DomainKey-Trace-MD: h=13; b=22; l=C18R71D32M65F38T27S68R65?69M17C39C27I81; Comments: QMDA 0.3 Received: (qmail 31368 invoked by uid 1001); 24 Mar 2014 17:19:41 -0000 Date: 24 Mar 2014 17:19:41 +0000 Message-ID: <20140324171941.31367.qmail@f5-external.bushwire.net> From: "Mark Delany" To: freebsd-net@freebsd.org Subject: Re: How to make netmap NS_FORWARD work with NR_REG_ONE_NIC? References: <20140324154522.30664.qmail@f5-external.bushwire.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 17:19:47 -0000 > > While good for performance, it unfortunately appears that NS_FORWARD > > does not work in this mode - presumably because NR_REG_ONE_NIC doesn't > > include the host ring. > > > > correct, this is not a supported mode at the moment. > If you want to implement it you should do it into netmap_poll() > and create another constant NS_REG_ONE_NIC_SW for the new mode. Thanks for the clarification. Unfortunately I don't think my skills are up to hacking features into the netmap module. Is my best user-space alternative much as I described? > > Do I have to register an independent handler with NR_REG_SW_NIC and > > have the h/w ring handlers synchronize with it? If so, I also presume > > the NR_REG_SW_NIC handler has to manage transfers in both directions, > > yes? Or is there a better way in user space? Mark. From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 18:54:02 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90400511 for ; Mon, 24 Mar 2014 18:54:02 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 68DBDBE9 for ; Mon, 24 Mar 2014 18:54:02 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s2OIs1vx041837 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 24 Mar 2014 11:54:01 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s2OIs0Gm041836; Mon, 24 Mar 2014 11:54:00 -0700 (PDT) (envelope-from jmg) Date: Mon, 24 Mar 2014 11:54:00 -0700 From: John-Mark Gurney To: soumya panigrahi Subject: Re: Handling Jumbo Packets Message-ID: <20140324185400.GQ32089@funkthat.com> Mail-Followup-To: soumya panigrahi , Luigi Rizzo , Giuseppe Lettieri , Vincenzo Maffione , "freebsd-net@freebsd.org" References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 24 Mar 2014 11:54:01 -0700 (PDT) Cc: Giuseppe Lettieri , Luigi Rizzo , Vincenzo Maffione , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 18:54:02 -0000 soumya panigrahi wrote this message on Mon, Mar 24, 2014 at 15:17 +0530: > On Mon, Mar 24, 2014 at 2:45 PM, Luigi Rizzo wrote: > > > On Mon, Mar 24, 2014 at 7:40 AM, soumya panigrahi < > > soumyapanigrahi@gmail.com> wrote: > > > >> First of all I would like to thank you all for developing an awesome > >> packet capturing framework. > >> I have been using it recently and find it very very fast and efficient. > >> > >> However off late I am trying to capture Jumbo packets on the network for > >> the following scenario's but that doesn't seem to be working. > > > > the sysctl tree dev.netmap (on FreeBSD) > > or the filesystem > > /sys/module/netmap_lin/parameters > > (on Linux) > > have a number of parameters that control buffer numbers and sizes. > > In particular "buf_size" is used to allocate packet buffers and > > you can bump it up from 2048 to something larger to accommodate > > jumbo buffers. > > > > Depending on the NIC you may need to do something in the initialization > > of the NIC itself to allow sending/receiving frames larger than 1518 bytes, > > but I'd just try and see whether it works for you now. > > Thanks a lot for the quick response. > I will also try to see if we can handle Jumbo frames by tweaking the > buf_size parameter on Linux. > In addition can you advice if NetMap can de-fragment Jumbo packets that > were fragmented as IP packets by intermediate IP device because of MTU size > limitation? > For example a 9000 Bytes single ethernet frame was fragmented into 1516 > Bytes multiple ethernet frames at IP layer. > Can netmap re-assemble these fragmented ethernet frames into a single Jumbo > frame before passing it the application? > If so how? There really isn't a way to tell the difference between a packet that was sent fragemented originally, and one that was fragmented along the route... Now you might be able to tell with some weird situations when the original fragemented packet was refragmented by a router, but this isn't the case you're asking about... look at the Fragmentation section of the IPv4 wikipedia entry for an example: https://en.wikipedia.org/wiki/Ipv4#Fragmentation And as Luigi said, all fragment reassembly has to be done by the netmap client.. that's the point of netmap, is that the kernel doesn't touch the packets... if it did do the reassembly, then the kernel would have to look at every packet, defeating the point of netmap... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 21:52:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F9814E9; Mon, 24 Mar 2014 21:52:46 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A90C0EB2; Mon, 24 Mar 2014 21:52:45 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id z60so18490474qgd.8 for ; Mon, 24 Mar 2014 14:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pKdls6z6/IGm7dNfBlzt5GCQ+aaTwFXFAZbR20pAXtY=; b=jv99KohJ4954flAlm2OzjKkrqVFrsSGl42MHuLRrMIRpFutcVqh82Ai/Q0N0bMhWTq gD/qSepvlO9GuquAxqRcaZ9GnSzGMCD/OnQqL3zTRaFyNeuz/+v8UAlLaK0YApjX6LZ4 QtNRf0fSOG0puMcB85bZodJL35vzJTIeVciJrVJsahBgYl7OHK2JeAIivkQd/TZugabv +BKf/2HGAOZkfY0fj591df0DgzVHjrBOnLmH6nooHd5V78fwQWXiTnHxRRQc7yx3Kcd7 U4jomImaQdIldRLac4dWTal0yg341XWHXtGtrS1rBms0/4AhIfwmkq2AA1oqXjrXZnVC 6bPQ== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr5312290qab.93.1395697964946; Mon, 24 Mar 2014 14:52:44 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Mon, 24 Mar 2014 14:52:44 -0700 (PDT) In-Reply-To: References: <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> Date: Mon, 24 Mar 2014 18:52:44 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 21:52:46 -0000 Well, a few more hours of running, and it's fairly easy to catch the packets with tcpdump, but not as easy to see if there is a pattern to them or what is different about them from the other packets that do pass with normal sizes. I'm using: tcpdump -ennvvvSuxx -i ix0 -s 64 greater 65495 here's some output. 18:41:41.311025 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65502: (tos 0x0, ttl 64, id 37273, offset 0, flags [DF], proto TCP (6), length 65488, bad cksum 0 (->50ee)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [P.], seq 3009729118:3009794554, ack 3477042952, win 28478, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:42:11.284028 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65502: (tos 0x0, ttl 64, id 52388, offset 0, flags [DF], proto TCP (6), length 65488, bad cksum 0 (->15e3)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [.], seq 1533469358:1533534794, ack 478673276, win 29127, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:42:31.385082 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65498: (tos 0x0, ttl 64, id 25808, offset 0, flags [DF], proto TCP (6), length 65484, bad cksum 0 (->7dbb)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [P.], seq 3658906462:3658971894, ack 1460462120, win 29127, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:42:45.200094 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65502: (tos 0x0, ttl 64, id 43985, offset 0, flags [DF], proto TCP (6), length 65488, bad cksum 0 (->36b6)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [P.], seq 805280454:805345890, ack 2122788052, win 29127, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:43:16.601738 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65502: (tos 0x0, ttl 64, id 5657, offset 0, flags [DF], proto TCP (6), length 65488, bad cksum 0 (->cc6e)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [.], seq 3978046962:3978112398, ack 3596907688, win 29127, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:43:37.345685 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65506: (tos 0x0, ttl 64, id 41062, offset 0, flags [DF], proto TCP (6), length 65492, bad cksum 0 (->421d)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [P.], seq 1419570518:1419635958, ack 104148460, win 29127, options [nop,nop,TS[|tcp]> 0x0000: 0050 567d b8ff 001b 21d6 4c4c 0800 4500 18:45:50.266944 00:1b:21:d6:4c:4c > 00:50:56:7d:b8:ff, ethertype IPv4 (0x0800), length 65506: (tos 0x0, ttl 64, id 5853, offset 0, flags [DF], proto TCP (6), length 65492, bad cksum 0 (->cba6)!) 172.16.0.30.2049 > 172.16.0.97.947: Flags [P.], seq 2161102562:2161168002, ack 2086338240, win 29127, options [nop,nop,TS[|tcp]> With the IP_MAXPACKET = 65495, I've had zero problems with networking. On Mon, Mar 24, 2014 at 1:23 PM, Christopher Forgeron wrote: > I think making hw_tsomax a sysctl would be a good patch to commit - It > could enable easy debugging/performance testing for the masses. > > I'm curious to hear how your environment is working with a tso turned off > on your nics. > > My testbed just hit the 2 hour mark. With TSO off, I don't get a single > packet over IP_MAXPACKET. That puts my confidence at around 95% in the > statement 'turning off tso negates this issue for me'. > > I'm now rebooting into a +tso env to see if I can capture the bad packets. > > I am also sure that the netstat -m mbuf denied is a completely separate > issue. I'm going around the lab and powering up different boxes with > 10.0-RELEASE, and they all have mbuf/mbuf clusters denied on boot, and that > number increases with network traffic. It's probably not helping the > IP_MAXPACKET issue. > > > > I'll create a separate thread for that one shortly. > > > On Mon, Mar 24, 2014 at 1:14 PM, Markus Gebert > wrote: > >> >> On 24.03.2014, at 16:21, Christopher Forgeron >> wrote: >> >> > This is regarding the TSO patch that Rick suggested earlier. (With many >> > thanks for his time and suggestion) >> > >> > As I mentioned earlier, it did not fix the issue on a 10.0 system. It >> did >> > make it less of a problem on 9.2, but either way, I think it's not >> needed, >> > and shouldn't be considered as a patch for testing/etc. >> > >> > Patching TSO to anything other than a max value (and by default the code >> > gives it IP_MAXPACKET) is confusing the matter, as the packet length >> > ultimately needs to be adjusted for many things on the fly like TCP >> > Options, etc. Using static header sizes won't be a good idea. >> > >> > Additionally, it seems that setting nic TSO will/may be ignored by code >> > like this in sys/netinet/tcp_output.c: >> > >> > 10.0 Code: >> > >> > 780 if (len > tp->t_tsomax - hdrlen) >> > { !! >> > 781 len = tp->t_tsomax - >> > hdrlen; !! >> > 782 sendalot = >> > 1; >> > 783 } >> > >> > >> > I've put debugging here, set the nic's max TSO as per Rick's patch ( >> set to >> > say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. It's being >> set >> > someplace else, and thus our attempts to set TSO on the nic may be in >> vain. >> > >> > It may have mattered more in 9.2, as I see the code doesn't use >> > tp->t_tsomax in some locations, and may actually default to what the >> nic is >> > set to. >> > >> > The NIC may still win, I didn't walk through the code to confirm, it was >> > enough to suggest to me that setting TSO wouldn't fix this issue. >> >> >> I just applied Rick's ixgbe TSO patch and additionally wanted to be able >> to easily change the value of hw_tsomax, so I made a sysctl out of it. >> >> While doing that, I asked myself the same question. Where and how will >> this value actually be used and how comes that tcp_output() uses that other >> value in struct tcpcb. >> >> The only place tcpcb->t_tsomax gets set, that I have found so far, is in >> tcp_input.c's tcp_mss() function. Some subfunctions get called: >> >> tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() >> >> Then tcp_maxmtu() indeed uses the interface's hw_tsomax value: >> >> 1746 cap->tsomax = ifp->if_hw_tsomax; >> >> It get's passed back to tcp_mss() where it is set on the connection >> level which will be used in tcp_output() later on. >> >> tcp_mss() gets called from multiple places, I'll look into that later. I >> will let you know if I find out more. >> >> >> Markus >> >> >> > However, this is still a TSO related issue, it's just not one related to >> > the setting of TSO's max size. >> > >> > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single >> packet >> > over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer to >> > increase confidence in this assertion, but I don't want to waste time on >> > this when I could be logging problem packets on a system with TSO >> enabled. >> > >> > Comments are very welcome.. >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > >> >> > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 22:29:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0B13477; Mon, 24 Mar 2014 22:29:06 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 87DD325B; Mon, 24 Mar 2014 22:29:06 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAAaxMFODaFve/2dsb2JhbABZg0FXgwe4NIZkUYE3dIIlAQEBBAEBASAEJyALBRYYAgINGQIpAQkmBggHBAEcBIdYDaxTok4XgSmMegYBARs0B4JvgUkElXOECZB/g0khMXwIFyI X-IronPort-AV: E=Sophos;i="4.97,723,1389762000"; d="scan'208";a="108761347" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 24 Mar 2014 18:29:05 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 38FDDB403F; Mon, 24 Mar 2014 18:29:05 -0400 (EDT) Date: Mon, 24 Mar 2014 18:29:05 -0400 (EDT) From: Rick Macklem To: Julian Elischer Message-ID: <349646395.2447442.1395700145224.JavaMail.root@uoguelph.ca> In-Reply-To: <5330632E.3050505@freebsd.org> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Christopher Forgeron , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 22:29:07 -0000 Julian Elischer wrote: > On 3/23/14, 4:57 PM, Rick Macklem wrote: > > Christopher Forgeron wrote: > >> > >> > >> > >> > >> > >> On Sat, Mar 22, 2014 at 6:41 PM, Rick Macklem < > >> rmacklem@uoguelph.ca > >>> wrote: > >> > >> > >> Christopher Forgeron wrote: > >>> #if defined(INET) || defined(INET6) > >>> /* Initialize to max value. */ > >>> if (ifp->if_hw_tsomax == 0) > >>> ifp->if_hw_tsomax = IP_MAXPACKET; > >>> KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > >>> ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > >>> ("%s: tsomax outside of range", __func__)); > >>> #endif > >>> > >>> > >>> Should this be the location where it's being set rather than in > >>> ixgbe? I would assume that other drivers could fall prey to this > >>> issue. > >>> > >> All of this should be prepended with "I'm an NFS guy, not a > >> networking > >> guy, so I might be wrong". > >> > >> Other drivers (and ixgbe for the 82598 chip) can handle a packet > >> that > >> is in more than 32 mbufs. (I think the 82598 handles 100, grep for > >> SCATTER > >> in *.h in sys/dev/ixgbe.) > >> > > the Xen backend can not handle mor ethan 32 segments in some versions > of Xen. > Oops, poorly worded. I should have said "Some other drivers...". Yes, there are several (I once did a find/grep, but didn't keep the output) that have this 32 limit. Also, I have no idea if the limit can easily be increased to 35 for them? (Bryan was able to do that for the virtio network driver.) rick ps: If it was just "ix" I wouldn't care as much about this. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 24 22:47:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7ADB6D3F; Mon, 24 Mar 2014 22:47:48 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 2C1605F3; Mon, 24 Mar 2014 22:47:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEALC1MFODaFve/2dsb2JhbABZg0FXgwe4NYZkUYE4dIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAiMGAQkmBggHBAEcBIdEAwkIDaxVmxANhzIXgSmLNIFMAQEbNAeCb4FJBJVzaoMfizaFSYNJITGBBDk X-IronPort-AV: E=Sophos;i="4.97,723,1389762000"; d="scan'208";a="108513058" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 24 Mar 2014 18:47:13 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 31360B403E; Mon, 24 Mar 2014 18:47:13 -0400 (EDT) Date: Mon, 24 Mar 2014 18:47:13 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1149589960.2454997.1395701233190.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 22:47:48 -0000 Christopher Forgeron wrote: > I'm going to split this into different posts to focus on each topic. > This > is about setting IP_MAXPACKET to 65495 > > Update on Last Night's Run: > > (Last night's run is a kernel with IP_MAXPACKET = 65495) > > - Uptime on this run: 10:53AM up 13:21, 5 users, load averages: > 1.98, > 2.09, 2.13 > - Ping logger records no ping errors for the entire run. > - At Mar 24th 10:57 I did a grep through the night's log for 'before' > (which is the printf logging that Rick suggested a few days ago), and > saved > it to before_total.txt > - With wc -l on before_total.txt I can see that we have 504 lines, > thus 504 > incidents of the packet being above IP_MAXPACKET during this run. > - I did tr -c '[:alnum:]' '[\n*]' < before_total.txt | sort | uniq -c > | > sort -nr | head -50 to list the most common words. Ignoring the > non-pklen > output. The relevant output is: > > 344 65498 (3) > 330 65506 (11) > 330 65502 (7) > This makes sense to me, since tp->t_tsomax is used in tcp_output() for the TCP/IP packet, which does not include the link level (ethernet) header. When that is added, I would expect the length to be up to 14 (or maybe 18 for vlan cases) greater than IP_MAXPACKET. Since none of these are greater than 65509, this looks fine to me. So, unless you get ones greater than (65495 + 18 = 65513), this makes sense and does not indicate a problem. In another post, you indicate that having the driver set if_hw_tsomax didn't set tp->t_tsomax to the same value. --> I believe that is a bug and would mean my ixgbe.patch would not fix the problem, because it is tp->t_tsomax that must be decreased to at least (65536 - 18 = 65518). --> Now, have you tried a case between 65495 and 65518 and seen any EFBIG errors? If so, then I don't understand why 65518 isn't small enough? rick > - First # being the # of times. (Each pklen is printed twice on the > log, > thus 2x the total line count). > - Last (#) being the byte overrun from 65495 > - A fairly even distribution of each type of packet overrun. > > You will recall that my IP_MAXPACKET is 65495, so each of these > packet > lengths represents a overshoot. > > The fact that we have only 3 different types of overrun is good - It > suggests a non-random event, more like a broken 'if' statement for a > particular case. > I think it just means that your load happens to do only 3 sizes of I/O that is a little less than 65536. > If IP_MAXPACKET was set to 65535 as it normally is, I would have had > 504 > incidents of errors, with a chance that any one of them could have > blocked > the queue for considerable time. > If tp->t_tsomax hasn't been set to a smaller value than 65535, the ixgbe.patch didn't do what I thought it would. > Question: Should there be logic that discards packets that are over > IP_MAXPACKET to ensure that we don't end up in a blocked queue > situation > again? > > > Moving forward, I am doing two things: > > 1) I'm running a longer test with TSO disabled on my ix0 adapter. I > want > to make sure that over say 4 hours I don't have even 1 packet over > 65495. > This will at least locate the issue to TSO related code. > > 2) I have tcpdump running, to see if I can capture the packets over > 65495. > Here is my command. Any suggestions on additional switches I should > include? > > tcpdump -ennvvXS greater 65495 > > I'll report in on this again once I have new info. > > Thanks for reading. > > On Mon, Mar 24, 2014 at 2:14 AM, Christopher Forgeron > wrote: > > > Hi, > > > > I'll follow up more tomorrow, as it's late and I don't have time > > for > > detail. > > > > The basic TSO patch didn't work, as packets were were still going > > over > > 65535 by a fair amount. I thought I wrote that earlier, but I am > > dumping a > > lot of info into a few threads, so I apologize if I'm not as > > concise as I > > could be. > > > > However, setting IP_MAXPACKET did. 4 hours of continuous run-time, > > no > > issues. No lost pings, no issues. Of course this isn't a fix - but > > it helps > > isolate the problem. > > > what the story is a few months down the road. > > > > > > > > > Thanks for the patches, will have to start giving them code-names > > > so > > > we can keep them straight. :-) I guess we have printf, tsomax, > > > and > > > this one. > > > > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 01:04:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 619E68F; Tue, 25 Mar 2014 01:04:20 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 1024D15F; Tue, 25 Mar 2014 01:04:19 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAFTVMFODaFve/2dsb2JhbABZg0FXgwe4KoZkUYE4dIIlAQEBAwEBAQEgBCcgCxsYAgINGQIjBgEJJgYIBwQBHAEDh0QDCQgNrEWbEg2HMheBKYs0gUwBARsBMweCb4FJBJVzaoMfizaFSYNJITGBBDk X-IronPort-AV: E=Sophos;i="4.97,724,1389762000"; d="scan'208";a="108794804" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 24 Mar 2014 21:04:18 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D85A0B3F4B; Mon, 24 Mar 2014 21:04:18 -0400 (EDT) Date: Mon, 24 Mar 2014 21:04:18 -0400 (EDT) From: Rick Macklem To: Markus Gebert Message-ID: <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> In-Reply-To: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 01:04:20 -0000 Markus Gebert wrote: >=20 > On 24.03.2014, at 16:21, Christopher Forgeron > wrote: >=20 > > This is regarding the TSO patch that Rick suggested earlier. (With > > many > > thanks for his time and suggestion) > >=20 > > As I mentioned earlier, it did not fix the issue on a 10.0 system. > > It did > > make it less of a problem on 9.2, but either way, I think it's not > > needed, > > and shouldn't be considered as a patch for testing/etc. > >=20 > > Patching TSO to anything other than a max value (and by default the > > code > > gives it IP_MAXPACKET) is confusing the matter, as the packet > > length > > ultimately needs to be adjusted for many things on the fly like TCP > > Options, etc. Using static header sizes won't be a good idea. > >=20 > > Additionally, it seems that setting nic TSO will/may be ignored by > > code > > like this in sys/netinet/tcp_output.c: > >=20 > > 10.0 Code: > >=20 > > 780 if (len > tp->t_tsomax - hdrlen) > > { !! > > 781 len =3D tp->t_tsomax - > > hdrlen; !! > > 782 sendalot =3D > > 1; > > 783 } > >=20 > >=20 > > I've put debugging here, set the nic's max TSO as per Rick's patch > > ( set to > > say 32k), and have seen that tp->t_tsomax =3D=3D IP_MAXPACKET. It's > > being set > > someplace else, and thus our attempts to set TSO on the nic may be > > in vain. > >=20 > > It may have mattered more in 9.2, as I see the code doesn't use > > tp->t_tsomax in some locations, and may actually default to what > > the nic is > > set to. > >=20 > > The NIC may still win, I didn't walk through the code to confirm, > > it was > > enough to suggest to me that setting TSO wouldn't fix this issue. >=20 >=20 > I just applied Rick=E2=80=99s ixgbe TSO patch and additionally wanted to = be > able to easily change the value of hw_tsomax, so I made a sysctl out > of it. >=20 > While doing that, I asked myself the same question. Where and how > will this value actually be used and how comes that tcp_output() > uses that other value in struct tcpcb. >=20 > The only place tcpcb->t_tsomax gets set, that I have found so far, is > in tcp_input.c=E2=80=99s tcp_mss() function. Some subfunctions get called= : >=20 > tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() >=20 > Then tcp_maxmtu() indeed uses the interface=E2=80=99s hw_tsomax value: >=20 > 1746 cap->tsomax =3D ifp->if_hw_tsomax; >=20 > It get=E2=80=99s passed back to tcp_mss() where it is set on the connect= ion > level which will be used in tcp_output() later on. >=20 > tcp_mss() gets called from multiple places, I=E2=80=99ll look into that > later. I will let you know if I find out more. >=20 >=20 > Markus >=20 Well, if tp->t_tsomax isn't set to a value of 65518, then the ixgbe.patch isn't doing what I thought it would. The only explanation I can think of for this is that there might be another net interface driver stacked on top of the ixgbe.c one and that the setting doesn't get propagated up. Does this make any sense? IP_MAXPACKET can't be changed from 65535, but I can see an argument for setting the default value of if_hw_tsomax to a smaller value. For example, in sys/net/if.c change it from: 657 if (ifp->if_hw_tsomax =3D=3D 0) 658 =09ifp->if_hw_tsomax =3D IP_MAXPACKET; to 657 if (ifp->if_hw_tsomax =3D=3D 0) 658 =09ifp->if_hw_tsomax =3D 65536 - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN)= ; This is a slightly smaller default which won't have much impact unless the hardware device can only handle 32 mbuf clusters for transmit of a segment and there are several of those. Christopher, can you do your test run with IP_MAXPACKET set to 65518, which should be the same as the above. If that gets rid of all the EFBIG error replies, then I think the above patch will have the same effect. Thanks, rick >=20 > > However, this is still a TSO related issue, it's just not one > > related to > > the setting of TSO's max size. > >=20 > > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > > packet > > over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer > > to > > increase confidence in this assertion, but I don't want to waste > > time on > > this when I could be logging problem packets on a system with TSO > > enabled. > >=20 > > Comments are very welcome.. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > >=20 >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 01:18:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBEC81FD; Tue, 25 Mar 2014 01:18:09 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 9D2F221C; Tue, 25 Mar 2014 01:18:09 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEABnZMFODaFve/2dsb2JhbABZhBiDB79fgTh0giUBAQEDASMEUhsOCgICDRkCWQYxh1MIrEWiUBeBKYx6IzQHgm+BSQSqe4NJIYEtQQ X-IronPort-AV: E=Sophos;i="4.97,724,1389762000"; d="scan'208";a="108542123" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 24 Mar 2014 21:18:08 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 38C25B409B; Mon, 24 Mar 2014 21:18:08 -0400 (EDT) Date: Mon, 24 Mar 2014 21:18:08 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1973302314.2516695.1395710288222.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 01:18:10 -0000 Christopher Forgeron wrote: > > > > This is regarding the TSO patch that Rick suggested earlier. (With > many thanks for his time and suggestion) > > > As I mentioned earlier, it did not fix the issue on a 10.0 system. It > did make it less of a problem on 9.2, but either way, I think it's > not needed, and shouldn't be considered as a patch for testing/etc. > > > Patching TSO to anything other than a max value (and by default the > code gives it IP_MAXPACKET) is confusing the matter, as the packet > length ultimately needs to be adjusted for many things on the fly > like TCP Options, etc. Using static header sizes won't be a good > idea. > If you look at tcp_output(), you'll notice that it doesn't do TSO if there are any options. That way it knows that the TCP/IP header is just hdrlen. If you don't limit the TSO packet (including TCP/IP and ethernet headers) to 64K, then the "ix" driver can't send them, which is the problem you guys are seeing. There are other ways to fix this problem, but they all may introduce issues that reducing if_hw_tsomax by a small amount does not. For example, m_defrag() could be modified to use 4K pagesize clusters, but this might introduce memory fragmentation problems. (I observed what I think are memory fragmentation problems when I switched NFS to use 4K pagesize clusters for large I/O messages.) If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG error replies), then that is the size that if_hw_tsomax can be set to (just can't change IP_MAXPACKET, but that is defined for other things). (It just happens that IP_MAXPACKET is what if_hw_tsomax defaults to. It has no other effect w.r.t. TSO.) > > Additionally, it seems that setting nic TSO will/may be ignored by > code like this in sys/netinet/tcp_output.c: > Yes, but I don't know why. The only conjecture I can come up with is that another net driver is stacked above "ix" and the setting for if_hw_tsomax doesn't propagate up. (If you look at the commit log message for r251296, the intent of adding if_hw_tsomax was to allow device drivers to set a smaller tsomax than IP_MAXPACKET.) Are you using any of the "stacked" network device drivers like lagg? I don't even know what the others all are? Maybe someone else can list them? rick > > 10.0 Code: > > 780 if (len > tp->t_tsomax - hdrlen) { !! > 781 len = tp->t_tsomax - hdrlen; !! > 782 sendalot = 1; > 783 } > > > > > I've put debugging here, set the nic's max TSO as per Rick's patch ( > set to say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. > It's being set someplace else, and thus our attempts to set TSO on > the nic may be in vain. > > > It may have mattered more in 9.2, as I see the code doesn't use > tp->t_tsomax in some locations, and may actually default to what the > nic is set to. > > The NIC may still win, I didn't walk through the code to confirm, it > was enough to suggest to me that setting TSO wouldn't fix this > issue. > > > However, this is still a TSO related issue, it's just not one related > to the setting of TSO's max size. > > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit > longer to increase confidence in this assertion, but I don't want to > waste time on this when I could be logging problem packets on a > system with TSO enabled. > > > Comments are very welcome.. > > > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 02:00:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E3D1684; Tue, 25 Mar 2014 02:00:59 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id A66E7759; Tue, 25 Mar 2014 02:00:58 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAHHiMFODaFve/2dsb2JhbABZhBiDB79fgTh0giUBAQEEI1YMDxoCDRkCWQaIDKxEok4XgSmMeSQ0B4JvgUkElFwHlhiDSSGBLEI X-IronPort-AV: E=Sophos;i="4.97,724,1389762000"; d="scan'208";a="108551171" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 24 Mar 2014 22:00:57 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A45A7B3F0B; Mon, 24 Mar 2014 22:00:57 -0400 (EDT) Date: Mon, 24 Mar 2014 22:00:57 -0400 (EDT) From: Rick Macklem To: Julian Elischer Message-ID: <510031798.2531808.1395712857661.JavaMail.root@uoguelph.ca> In-Reply-To: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 02:00:59 -0000 Julian Elischer wrote: ----- Original Message ----- > I wrote (and snipped): >> Other drivers (and ixgbe for the 82598 chip) can handle a packet that >> is in more than 32 mbufs. (I think the 82598 handles 100, grep for >> SCATTER >> in *.h in sys/dev/ixgbe.) >> > > the Xen backend can not handle mor ethan 32 segments in some versions > of Xen. Btw, I just did a quick find/grep (so I may have missed some), but here is the list of net devices that appear to support TSO, but limited to 32 transmit segments for at least some supported chips: jme, fxp, age, sge, msk, als, ale, ixgbe/ix, nfe, e1000/em, re Also, several of these call m_collapse() instead of m_defrag() when the run into a transmit mbuf list with > 32 elements. m_collapse() - Isn't likely to squeeze the 35 mbuf 64Kbyte NFS I/O message into 32 mbufs, so I don't think these ones will work at all for NFS with default 64K I/O size and TSO enabled. rick From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 03:56:51 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1679BC5F for ; Tue, 25 Mar 2014 03:56:51 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D5683107 for ; Tue, 25 Mar 2014 03:56:50 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so6340850pab.22 for ; Mon, 24 Mar 2014 20:56:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:message-id:date:to:mime-version; bh=+nRcR/CcsxHODTHP8H4ugIh4e/9SAKLqqTr6LxKUY48=; b=B+OgYSDoY8TZgSploUOlZdST2SzEgMgcXZL5WXvoVnLLJ69eexun+cvYpC0Gy1mhft HGXtlJ2ahQvE0xzPV/mwzLDUx7C79u5K0fbIOAcx9Q1xbs4FrpIgaJbHHeJx01HOUjuR TNDBia7EugjldA1DhiqmiFaGNJ8xFV0Gir5PhQirBRyUcU8vU+d1/heH1P5aIArXJVsF rtj7vumCWIn2hRjCdiYnc42QKG1CHT/y8vLwD9at7KTFyxtn4X8FtSobN82TimnzZJl8 Qn0wgjZv0dA54OTptmOMbqkOBqb+B5txFsFW6/F9vDLA4eXLtxCnZ1pP7On0SjdRcWsU 8lig== X-Received: by 10.67.3.97 with SMTP id bv1mr76543267pad.54.1395719810498; Mon, 24 Mar 2014 20:56:50 -0700 (PDT) Received: from [192.168.1.13] (cpe-76-176-151-248.san.res.rr.com. [76.176.151.248]) by mx.google.com with ESMTPSA id zb2sm38354242pbc.30.2014.03.24.20.56.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 24 Mar 2014 20:56:48 -0700 (PDT) From: Richard Catlin Subject: RE: Running netmap's pkt-get on Debian Message-Id: <08465DA8-8DAA-403F-B692-59FD0E524417@gmail.com> Date: Mon, 24 Mar 2014 20:56:46 -0700 To: net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) X-Mailer: Apple Mail (2.1510) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 03:56:51 -0000 Hi, I'm trying to run the pkt-get example on Debian. I'm not an expert = (yet) on the Linux Network Stack and Drivers, so I need some help. I = did download the "tiny core" bootable Linux image, and pkt-gen runs on = that. I'm just not sure the steps to take to make it run on my VMWare = Debian instance. I did the following: 1. git clone https://code.google.com/p/netmap/ 2. went to the /netmap/LINUX directory and ran "make", and "make apps" 3. /sbin/rmmod e1000 4. /sbin/insmod netmap_lin.ko 5. /sbin/insmod e1000/e1000.ko When I run these, the receiver does not recognize the transmissions.=20 Shell 1 (as root): ./pkt-gen -i vale-foo Shell 2 (as root): ./pkt-gen -i vale-bar -f tx Regards, Richard Catlin Here is a screen shot of them running. From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 04:56:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA95F627 for ; Tue, 25 Mar 2014 04:56:05 +0000 (UTC) Received: from mail-yh0-x232.google.com (mail-yh0-x232.google.com [IPv6:2607:f8b0:4002:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADC6F7BD for ; Tue, 25 Mar 2014 04:56:05 +0000 (UTC) Received: by mail-yh0-f50.google.com with SMTP id c41so6180707yho.23 for ; Mon, 24 Mar 2014 21:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=dV+ikSvS9lYCtetCDXm4+S0H7C+hdJ6eG1ywmwFIC0s=; b=dKPAaozHwue0QcavDP2+SIQXc6aO4XBK7dIdHmD46yiJi/Oob+4eiiljzhcHxTuROj PtD6aL8QXnOg0YQMqk00qF+fxQDF0AFpJZvs1QcL0EKPIhXXBV3UoFgpnLq4ToO+OYIZ hsaclRTE/usUpm4tw4WvO3Imq+metwY//4gyuuWWV/S33Knw/nDNlOpKGEl6ROwM99mz r3l4PLGy6tKNaT9OEaXElOrwaZHRiyPDzR0GbHpe/IbJ8Gxqx6NRWSrgwD8NA66dYzhL 5xgP1ByzsnyIO9QeIhAOaMWlRgehAuPR+oqmxuP1jqQgH6j6BMw2fF+AK9j3HHkQt5tK xapQ== X-Received: by 10.236.126.81 with SMTP id a57mr9434376yhi.95.1395723364928; Mon, 24 Mar 2014 21:56:04 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id g65sm26908779yhm.28.2014.03.24.21.56.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 24 Mar 2014 21:56:04 -0700 (PDT) Message-ID: <53310C57.5070102@gmail.com> Date: Tue, 25 Mar 2014 12:55:51 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: FreeBSD Net Subject: syslogd:sendto: no buffer available on 10-stable Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 04:56:06 -0000 Hi,Lists: I have got lots of "no buffer available" on 10-stable with igb nic. But em and bce works well. And I tried force igb to 4 or 8 queues and set hw.igb.rx_process_limit="-1", but have nothing helped. Regards Simon # uname -a FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r262469: Tue Feb 25 13:27:11 CST 2014 root@sq-l1-n2:/usr/obj/usr/src/sys/stable-10-262458 amd64 # netstat -mb 19126/73289/92415 mbufs in use (current/cache/total) 13289/46841/60130/524288 mbuf clusters in use (current/cache/total/max) 13289/46812 mbuf+clusters out of packet secondary zone in use (current/cache) 5638/22605/28243/262144 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/77672 9k jumbo clusters in use (current/cache/total/max) 0/0/0/43690 16k jumbo clusters in use (current/cache/total/max) 53914K/202424K/256338K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile # netstat -di Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop igb0 1500 00:1b:21:70:5f:80 17212101113 1355809 0 19612978862 0 0 igb1 1500 00:1b:21:70:5f:81 76601294282 81162751 0 74236432686 0 0 lo0 16384 20532742636 0 0 20522475797 0 0 lo0 - your-net localhost 2736994243 - - 20520227166 - - # sysctl hw.igb hw.igb.rxd: 2048 hw.igb.txd: 2048 hw.igb.enable_aim: 1 hw.igb.enable_msix: 1 hw.igb.max_interrupt_rate: 12000 hw.igb.buf_ring_size: 4096 hw.igb.header_split: 0 hw.igb.num_queues: 1 hw.igb.rx_process_limit: 1000 # sysctl dev.igb.1 dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.1.%driver: igb dev.igb.1.%location: slot=0 function=1 dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 subdevice=0xa04c class=0x020000 dev.igb.1.%parent: pci8 dev.igb.1.nvm: -1 dev.igb.1.enable_aim: 1 dev.igb.1.fc: 0 dev.igb.1.rx_processing_limit: 4096 dev.igb.1.link_irq: 3 dev.igb.1.dropped: 0 dev.igb.1.tx_dma_fail: 0 dev.igb.1.rx_overruns: 0 dev.igb.1.watchdog_timeouts: 0 dev.igb.1.device_control: 1086325313 dev.igb.1.rx_control: 67141634 dev.igb.1.interrupt_mask: 4 dev.igb.1.extended_int_mask: 2147483651 dev.igb.1.tx_buf_alloc: 0 dev.igb.1.rx_buf_alloc: 0 dev.igb.1.fc_high_water: 58976 dev.igb.1.fc_low_water: 58960 dev.igb.1.queue0.no_desc_avail: 10874 dev.igb.1.queue0.tx_packets: 74509997338 dev.igb.1.queue0.rx_packets: 76837720630 dev.igb.1.queue0.rx_bytes: 35589607860237 dev.igb.1.queue0.lro_queued: 0 dev.igb.1.queue0.lro_flushed: 0 dev.igb.1.mac_stats.excess_coll: 0 dev.igb.1.mac_stats.single_coll: 0 dev.igb.1.mac_stats.multiple_coll: 0 dev.igb.1.mac_stats.late_coll: 0 dev.igb.1.mac_stats.collision_count: 0 dev.igb.1.mac_stats.symbol_errors: 0 dev.igb.1.mac_stats.sequence_errors: 0 dev.igb.1.mac_stats.defer_count: 0 dev.igb.1.mac_stats.missed_packets: 81162751 dev.igb.1.mac_stats.recv_no_buff: 176691324 dev.igb.1.mac_stats.recv_undersize: 0 dev.igb.1.mac_stats.recv_fragmented: 0 dev.igb.1.mac_stats.recv_oversize: 0 dev.igb.1.mac_stats.recv_jabber: 0 dev.igb.1.mac_stats.recv_errs: 0 dev.igb.1.mac_stats.crc_errs: 0 dev.igb.1.mac_stats.alignment_errs: 0 dev.igb.1.mac_stats.coll_ext_errs: 0 dev.igb.1.mac_stats.xon_recvd: 0 dev.igb.1.mac_stats.xon_txd: 0 dev.igb.1.mac_stats.xoff_recvd: 0 dev.igb.1.mac_stats.xoff_txd: 0 dev.igb.1.mac_stats.total_pkts_recvd: 76925709917 dev.igb.1.mac_stats.good_pkts_recvd: 76837704301 dev.igb.1.mac_stats.bcast_pkts_recvd: 49174716 dev.igb.1.mac_stats.mcast_pkts_recvd: 282670 dev.igb.1.mac_stats.rx_frames_64: 31057121854 dev.igb.1.mac_stats.rx_frames_65_127: 19996324498 dev.igb.1.mac_stats.rx_frames_128_255: 1171960837 dev.igb.1.mac_stats.rx_frames_256_511: 2295894674 dev.igb.1.mac_stats.rx_frames_512_1023: 2026241811 dev.igb.1.mac_stats.rx_frames_1024_1522: 20290160627 dev.igb.1.mac_stats.good_octets_recvd: 36204302378783 dev.igb.1.mac_stats.good_octets_txd: 59038220741656 dev.igb.1.mac_stats.total_pkts_txd: 90973292365 dev.igb.1.mac_stats.good_pkts_txd: 90973292359 dev.igb.1.mac_stats.bcast_pkts_txd: 2408182 dev.igb.1.mac_stats.mcast_pkts_txd: 246782 dev.igb.1.mac_stats.tx_frames_64: 24604769631 dev.igb.1.mac_stats.tx_frames_65_127: 21373976133 dev.igb.1.mac_stats.tx_frames_128_255: 3047554762 dev.igb.1.mac_stats.tx_frames_256_511: 3751820879 dev.igb.1.mac_stats.tx_frames_512_1023: 3481550512 dev.igb.1.mac_stats.tx_frames_1024_1522: 34713620448 dev.igb.1.mac_stats.tso_txd: 9549335102 dev.igb.1.mac_stats.tso_ctx_fail: 0 dev.igb.1.interrupts.asserts: 16929843739 dev.igb.1.interrupts.rx_pkt_timer: 76834839132 dev.igb.1.interrupts.rx_abs_timer: 0 dev.igb.1.interrupts.tx_pkt_timer: 0 dev.igb.1.interrupts.tx_abs_timer: 76835042438 dev.igb.1.interrupts.tx_queue_empty: 90970561829 dev.igb.1.interrupts.tx_queue_min_thresh: 0 dev.igb.1.interrupts.rx_desc_min_thresh: 0 dev.igb.1.interrupts.rx_overrun: 0 dev.igb.1.host.breaker_tx_pkt: 0 dev.igb.1.host.host_tx_pkt_discard: 0 dev.igb.1.host.rx_pkt: 2865171 dev.igb.1.host.breaker_rx_pkts: 0 dev.igb.1.host.breaker_rx_pkt_drop: 0 dev.igb.1.host.tx_good_pkt: 2730536 dev.igb.1.host.breaker_tx_pkt_drop: 0 dev.igb.1.host.rx_good_bytes: 36204307257585 dev.igb.1.host.tx_good_bytes: 59038220743158 dev.igb.1.host.length_errors: 0 dev.igb.1.host.serdes_violation_pkt: 0 dev.igb.1.host.header_redir_missed: 0 # cat /var/run/dmesg.boot |grep igb igb0: mem 0xd5ee0000-0xd5efffff,0xd5800000-0xd5bfffff,0xd5edc000-0xd5edffff irq 18 at device 0.0 on pci8 igb0: Using MSIX interrupts with 2 vectors igb0: Ethernet address: 00:1b:21:70:5f:80 igb1: mem 0xd5ea0000-0xd5ebffff,0xd5400000-0xd57fffff,0xd5ed8000-0xd5edbfff irq 19 at device 0.1 on pci8 igb1: Using MSIX interrupts with 2 vectors igb1: Ethernet address: 00:1b:21:70:5f:81 From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 12:16:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9457FADC; Tue, 25 Mar 2014 12:16:19 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 269D5280; Tue, 25 Mar 2014 12:16:18 +0000 (UTC) Received: from [2001:1620:2013:1:a98e:740e:10d9:e192] (port=51063) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WSQH9-00073G-9f; Tue, 25 Mar 2014 13:16:15 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: <1973302314.2516695.1395710288222.JavaMail.root@uoguelph.ca> Date: Tue, 25 Mar 2014 13:16:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> References: <1973302314.2516695.1395710288222.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 12:16:19 -0000 On 25.03.2014, at 02:18, Rick Macklem wrote: > Christopher Forgeron wrote: >>=20 >>=20 >>=20 >> This is regarding the TSO patch that Rick suggested earlier. (With >> many thanks for his time and suggestion) >>=20 >>=20 >> As I mentioned earlier, it did not fix the issue on a 10.0 system. It >> did make it less of a problem on 9.2, but either way, I think it's >> not needed, and shouldn't be considered as a patch for testing/etc. >>=20 >>=20 >> Patching TSO to anything other than a max value (and by default the >> code gives it IP_MAXPACKET) is confusing the matter, as the packet >> length ultimately needs to be adjusted for many things on the fly >> like TCP Options, etc. Using static header sizes won't be a good >> idea. >>=20 > If you look at tcp_output(), you'll notice that it doesn't do TSO if > there are any options. That way it knows that the TCP/IP header is > just hdrlen. >=20 > If you don't limit the TSO packet (including TCP/IP and ethernet = headers) > to 64K, then the "ix" driver can't send them, which is the problem > you guys are seeing. >=20 > There are other ways to fix this problem, but they all may introduce > issues that reducing if_hw_tsomax by a small amount does not. > For example, m_defrag() could be modified to use 4K pagesize clusters, > but this might introduce memory fragmentation problems. (I observed > what I think are memory fragmentation problems when I switched NFS > to use 4K pagesize clusters for large I/O messages.) >=20 > If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG > error replies), then that is the size that if_hw_tsomax can be set > to (just can't change IP_MAXPACKET, but that is defined for other > things). (It just happens that IP_MAXPACKET is what if_hw_tsomax > defaults to. It has no other effect w.r.t. TSO.) >=20 >>=20 >> Additionally, it seems that setting nic TSO will/may be ignored by >> code like this in sys/netinet/tcp_output.c: >>=20 Is this confirmed or still a =91it seems=92? Have you actually seen a = tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was this = just speculation because the values are stored in different places? = (Sorry, if you already stated this in another email, it=92s currently = hard to keep track of all the information.) Anyway, this dtrace one-liner should be a good test if other values = appear in tp->t_tsomax: # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && = args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: %i\n", = args[0]->t_tsomax); stack(); }' Remember to adjust the value in the condition to whatever you=92re = currently expecting. The value seems to be 0 for new connections, = probably when tcp_mss() has not been called yet. So that=92s seems = normal and I have excluded that case too. This will also print a kernel = stack trace in case it sees an unexpected value. > Yes, but I don't know why. > The only conjecture I can come up with is that another net driver is > stacked above "ix" and the setting for if_hw_tsomax doesn't propagate > up. (If you look at the commit log message for r251296, the intent > of adding if_hw_tsomax was to allow device drivers to set a smaller > tsomax than IP_MAXPACKET.) >=20 > Are you using any of the "stacked" network device drivers like > lagg? I don't even know what the others all are? > Maybe someone else can list them? I guess the most obvious are lagg and vlan (and probably carp on FreeBSD = 9.x or older). On request from Jack, we=92ve eliminated lagg and vlan from the picture, = which gives us plain ixgbe interfaces with no stacked interfaces on top = of it. And we can still reproduce the problem. Markus >=20 > rick >>=20 >> 10.0 Code: >>=20 >> 780 if (len > tp->t_tsomax - hdrlen) { !! >> 781 len =3D tp->t_tsomax - hdrlen; !! >> 782 sendalot =3D 1; >> 783 } >>=20 >>=20 >>=20 >>=20 >> I've put debugging here, set the nic's max TSO as per Rick's patch ( >> set to say 32k), and have seen that tp->t_tsomax =3D=3D IP_MAXPACKET. >> It's being set someplace else, and thus our attempts to set TSO on >> the nic may be in vain. >>=20 >>=20 >> It may have mattered more in 9.2, as I see the code doesn't use >> tp->t_tsomax in some locations, and may actually default to what the >> nic is set to. >>=20 >> The NIC may still win, I didn't walk through the code to confirm, it >> was enough to suggest to me that setting TSO wouldn't fix this >> issue. >>=20 >>=20 >> However, this is still a TSO related issue, it's just not one related >> to the setting of TSO's max size. >>=20 >> A 10.0-STABLE system with tso disabled on ix0 doesn't have a single >> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit >> longer to increase confidence in this assertion, but I don't want to >> waste time on this when I could be logging problem packets on a >> system with TSO enabled. >>=20 >>=20 >> Comments are very welcome.. >>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 12:28:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92D8359F for ; Tue, 25 Mar 2014 12:28:25 +0000 (UTC) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 63C5838A for ; Tue, 25 Mar 2014 12:28:25 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id ld10so369196pab.26 for ; Tue, 25 Mar 2014 05:28:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=PmRHdStjYdE4oW9iaJiHHelhc3okSnly6NFhq+7+lWs=; b=S1mz+oQMC3CdZF9Edh6mn6x8yLoJnfePIjMG+uJZV6gAFO2B3CDf80GMeF+TXDghTz XabId4LKlC7Bhv3QWS/Dc+DSztoCfLqXeeladrVZolUnqQa9dw86hLSlYBLYIRpxCvAG yxA8IE/5K+tOS30MOC4YP1tZOlpKdI8PnvUAmLi/1zFzk1diJet+F/E3ig1bgMsKavB6 kLGFcirApDQKqsCmGOKxLj91YRYcxy8oDTZ9o82cPawjnVsOvwP7IWneiT78Pjvxy1nv qTAia87FZDj9HQoBgLqcYofrAlTpYjIPZxTyDcyN9f5tocBR86cehDwdFQsMcoLsVECk 0MCA== X-Gm-Message-State: ALoCoQlAH5Hw6392Jcp0lUZcutpnBd1qDR+LeTjzAhQahFRWJsbdBZuel8ertoCUr/JUVIHnvfHQ MIME-Version: 1.0 X-Received: by 10.66.252.135 with SMTP id zs7mr79004679pac.13.1395750103933; Tue, 25 Mar 2014 05:21:43 -0700 (PDT) Received: by 10.68.111.37 with HTTP; Tue, 25 Mar 2014 05:21:43 -0700 (PDT) In-Reply-To: <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> References: <1973302314.2516695.1395710288222.JavaMail.root@uoguelph.ca> <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> Date: Tue, 25 Mar 2014 13:21:43 +0100 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Johan Kooijman To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 12:28:25 -0000 Hey guys, I have nothing on your code level to add, but.. while investigating this issue I ran into the guy that originally created the bug ( http://www.freebsd.org/cgi/query-pr.cgi?pr=183390&cat=). In the email exchange that followed he told me that had found a workaround by running a specific -STABLE revision: "Yes, we found a workaround. We upgraded to the -STABLE branch of the 9.2, so we use this currently: [root@storagex ~]# uname -a FreeBSD storagex.lan.granaglia.com 9.2-STABLE FreeBSD 9.2-STABLE #0 r257712: Tue Nov 5 23:02:49 CET 2013 root@storagex.lan.granaglia.com:/usr/obj/usr/src/sys/GENERIC amd64" Maybe this could help you in your quest to hunt this bug down. On Tue, Mar 25, 2014 at 1:16 PM, Markus Gebert wrote: > > On 25.03.2014, at 02:18, Rick Macklem wrote: > > > Christopher Forgeron wrote: > >> > >> > >> > >> This is regarding the TSO patch that Rick suggested earlier. (With > >> many thanks for his time and suggestion) > >> > >> > >> As I mentioned earlier, it did not fix the issue on a 10.0 system. It > >> did make it less of a problem on 9.2, but either way, I think it's > >> not needed, and shouldn't be considered as a patch for testing/etc. > >> > >> > >> Patching TSO to anything other than a max value (and by default the > >> code gives it IP_MAXPACKET) is confusing the matter, as the packet > >> length ultimately needs to be adjusted for many things on the fly > >> like TCP Options, etc. Using static header sizes won't be a good > >> idea. > >> > > If you look at tcp_output(), you'll notice that it doesn't do TSO if > > there are any options. That way it knows that the TCP/IP header is > > just hdrlen. > > > > If you don't limit the TSO packet (including TCP/IP and ethernet headers) > > to 64K, then the "ix" driver can't send them, which is the problem > > you guys are seeing. > > > > There are other ways to fix this problem, but they all may introduce > > issues that reducing if_hw_tsomax by a small amount does not. > > For example, m_defrag() could be modified to use 4K pagesize clusters, > > but this might introduce memory fragmentation problems. (I observed > > what I think are memory fragmentation problems when I switched NFS > > to use 4K pagesize clusters for large I/O messages.) > > > > If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG > > error replies), then that is the size that if_hw_tsomax can be set > > to (just can't change IP_MAXPACKET, but that is defined for other > > things). (It just happens that IP_MAXPACKET is what if_hw_tsomax > > defaults to. It has no other effect w.r.t. TSO.) > > > >> > >> Additionally, it seems that setting nic TSO will/may be ignored by > >> code like this in sys/netinet/tcp_output.c: > >> > > Is this confirmed or still a 'it seems'? Have you actually seen a > tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was this > just speculation because the values are stored in different places? (Sorry, > if you already stated this in another email, it's currently hard to keep > track of all the information.) > > Anyway, this dtrace one-liner should be a good test if other values appear > in tp->t_tsomax: > > # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && > args[0]->t_tsomax != 65518 / { printf("unexpected tp->t_tsomax: %i\n", > args[0]->t_tsomax); stack(); }' > > Remember to adjust the value in the condition to whatever you're currently > expecting. The value seems to be 0 for new connections, probably when > tcp_mss() has not been called yet. So that's seems normal and I have > excluded that case too. This will also print a kernel stack trace in case > it sees an unexpected value. > > > > Yes, but I don't know why. > > The only conjecture I can come up with is that another net driver is > > stacked above "ix" and the setting for if_hw_tsomax doesn't propagate > > up. (If you look at the commit log message for r251296, the intent > > of adding if_hw_tsomax was to allow device drivers to set a smaller > > tsomax than IP_MAXPACKET.) > > > > Are you using any of the "stacked" network device drivers like > > lagg? I don't even know what the others all are? > > Maybe someone else can list them? > > I guess the most obvious are lagg and vlan (and probably carp on FreeBSD > 9.x or older). > > On request from Jack, we've eliminated lagg and vlan from the picture, > which gives us plain ixgbe interfaces with no stacked interfaces on top of > it. And we can still reproduce the problem. > > > Markus > > > > > > rick > >> > >> 10.0 Code: > >> > >> 780 if (len > tp->t_tsomax - hdrlen) { !! > >> 781 len = tp->t_tsomax - hdrlen; !! > >> 782 sendalot = 1; > >> 783 } > >> > >> > >> > >> > >> I've put debugging here, set the nic's max TSO as per Rick's patch ( > >> set to say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. > >> It's being set someplace else, and thus our attempts to set TSO on > >> the nic may be in vain. > >> > >> > >> It may have mattered more in 9.2, as I see the code doesn't use > >> tp->t_tsomax in some locations, and may actually default to what the > >> nic is set to. > >> > >> The NIC may still win, I didn't walk through the code to confirm, it > >> was enough to suggest to me that setting TSO wouldn't fix this > >> issue. > >> > >> > >> However, this is still a TSO related issue, it's just not one related > >> to the setting of TSO's max size. > >> > >> A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > >> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit > >> longer to increase confidence in this assertion, but I don't want to > >> waste time on this when I could be logging problem packets on a > >> system with TSO enabled. > >> > >> > >> Comments are very welcome.. > >> > >> > >> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Met vriendelijke groeten / With kind regards, Johan Kooijman T +31(0) 6 43 44 45 27 F +31(0) 162 82 00 01 E mail@johankooijman.com From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 14:08:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8BB8251 for ; Tue, 25 Mar 2014 14:08:51 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 97719FC4 for ; Tue, 25 Mar 2014 14:08:51 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so1488200qgd.1 for ; Tue, 25 Mar 2014 07:08:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YF+EZx+NYRZCNg03bWawPiPRngfmjh/vy5olODBOx5w=; b=ZTrIUx6I+TCiPhLUgreYcK1an6fjzlABUQZkFVlaIzKZ9yBJUlDfcqvhi63S40lSgG r4lMwIsqT7X7hk06EsCxQoIUBg57G7/zpM9QvnJ0YTHcwigHTMcDM8q3HS8rrmL/iYqd FGjP7atogKLgAtm995y7HlsL6OCXtjloEg2jEoX6P8fE6DpPUkZgmbdTq50h6rrMUDcs OeQD5ehcOq8WNBdZo7n+6CoTGPXDBImERxxetU0o2fgG+r1xAxDja0Y6ue7aXOJVJZKd c/eqMBV2gE2rk8Z9sKntKBk0AIoRYuAZuPVejtjEfYq2360szqIXShymEIOglVzybpsT zhfA== MIME-Version: 1.0 X-Received: by 10.224.57.72 with SMTP id b8mr634489qah.41.1395756530384; Tue, 25 Mar 2014 07:08:50 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 07:08:50 -0700 (PDT) In-Reply-To: <53310C57.5070102@gmail.com> References: <53310C57.5070102@gmail.com> Date: Tue, 25 Mar 2014 11:08:50 -0300 Message-ID: Subject: Re: syslogd:sendto: no buffer available on 10-stable From: Christopher Forgeron To: k simon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 14:08:51 -0000 Hi Simon, Try checking out the "9.2 ixgbe tx queue hang' thread here, and see if it applies to you. On Tue, Mar 25, 2014 at 1:55 AM, k simon wrote: > Hi,Lists: > I have got lots of "no buffer available" on 10-stable with igb nic. > But em and bce works well. And I tried force igb to 4 or 8 queues and > set hw.igb.rx_process_limit="-1", but have nothing helped. > > > Regards > Simon > > > > # uname -a > FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r262469: Tue Feb 25 > 13:27:11 CST 2014 > root@sq-l1-n2:/usr/obj/usr/src/sys/stable-10-262458 amd64 > > > # netstat -mb > 19126/73289/92415 mbufs in use (current/cache/total) > 13289/46841/60130/524288 mbuf clusters in use (current/cache/total/max) > 13289/46812 mbuf+clusters out of packet secondary zone in use > (current/cache) > 5638/22605/28243/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/77672 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/43690 16k jumbo clusters in use (current/cache/total/max) > 53914K/202424K/256338K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > # netstat -di > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll Drop > igb0 1500 00:1b:21:70:5f:80 17212101113 1355809 0 > 19612978862 0 0 > igb1 1500 00:1b:21:70:5f:81 76601294282 81162751 0 > 74236432686 0 0 > lo0 16384 20532742636 0 0 > 20522475797 0 0 > lo0 - your-net localhost 2736994243 - - > 20520227166 - - > > > # sysctl hw.igb > hw.igb.rxd: 2048 > hw.igb.txd: 2048 > hw.igb.enable_aim: 1 > hw.igb.enable_msix: 1 > hw.igb.max_interrupt_rate: 12000 > hw.igb.buf_ring_size: 4096 > hw.igb.header_split: 0 > hw.igb.num_queues: 1 > hw.igb.rx_process_limit: 1000 > > > # sysctl dev.igb.1 > dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 > dev.igb.1.%driver: igb > dev.igb.1.%location: slot=0 function=1 > dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 > subdevice=0xa04c class=0x020000 > dev.igb.1.%parent: pci8 > dev.igb.1.nvm: -1 > dev.igb.1.enable_aim: 1 > dev.igb.1.fc: 0 > dev.igb.1.rx_processing_limit: 4096 > dev.igb.1.link_irq: 3 > dev.igb.1.dropped: 0 > dev.igb.1.tx_dma_fail: 0 > dev.igb.1.rx_overruns: 0 > dev.igb.1.watchdog_timeouts: 0 > dev.igb.1.device_control: 1086325313 > dev.igb.1.rx_control: 67141634 > dev.igb.1.interrupt_mask: 4 > dev.igb.1.extended_int_mask: 2147483651 > dev.igb.1.tx_buf_alloc: 0 > dev.igb.1.rx_buf_alloc: 0 > dev.igb.1.fc_high_water: 58976 > dev.igb.1.fc_low_water: 58960 > dev.igb.1.queue0.no_desc_avail: 10874 > dev.igb.1.queue0.tx_packets: 74509997338 > dev.igb.1.queue0.rx_packets: 76837720630 > dev.igb.1.queue0.rx_bytes: 35589607860237 > dev.igb.1.queue0.lro_queued: 0 > dev.igb.1.queue0.lro_flushed: 0 > dev.igb.1.mac_stats.excess_coll: 0 > dev.igb.1.mac_stats.single_coll: 0 > dev.igb.1.mac_stats.multiple_coll: 0 > dev.igb.1.mac_stats.late_coll: 0 > dev.igb.1.mac_stats.collision_count: 0 > dev.igb.1.mac_stats.symbol_errors: 0 > dev.igb.1.mac_stats.sequence_errors: 0 > dev.igb.1.mac_stats.defer_count: 0 > dev.igb.1.mac_stats.missed_packets: 81162751 > dev.igb.1.mac_stats.recv_no_buff: 176691324 > dev.igb.1.mac_stats.recv_undersize: 0 > dev.igb.1.mac_stats.recv_fragmented: 0 > dev.igb.1.mac_stats.recv_oversize: 0 > dev.igb.1.mac_stats.recv_jabber: 0 > dev.igb.1.mac_stats.recv_errs: 0 > dev.igb.1.mac_stats.crc_errs: 0 > dev.igb.1.mac_stats.alignment_errs: 0 > dev.igb.1.mac_stats.coll_ext_errs: 0 > dev.igb.1.mac_stats.xon_recvd: 0 > dev.igb.1.mac_stats.xon_txd: 0 > dev.igb.1.mac_stats.xoff_recvd: 0 > dev.igb.1.mac_stats.xoff_txd: 0 > dev.igb.1.mac_stats.total_pkts_recvd: 76925709917 > dev.igb.1.mac_stats.good_pkts_recvd: 76837704301 > dev.igb.1.mac_stats.bcast_pkts_recvd: 49174716 > dev.igb.1.mac_stats.mcast_pkts_recvd: 282670 > dev.igb.1.mac_stats.rx_frames_64: 31057121854 > dev.igb.1.mac_stats.rx_frames_65_127: 19996324498 > dev.igb.1.mac_stats.rx_frames_128_255: 1171960837 > dev.igb.1.mac_stats.rx_frames_256_511: 2295894674 > dev.igb.1.mac_stats.rx_frames_512_1023: 2026241811 > dev.igb.1.mac_stats.rx_frames_1024_1522: 20290160627 > dev.igb.1.mac_stats.good_octets_recvd: 36204302378783 > dev.igb.1.mac_stats.good_octets_txd: 59038220741656 > dev.igb.1.mac_stats.total_pkts_txd: 90973292365 > dev.igb.1.mac_stats.good_pkts_txd: 90973292359 > dev.igb.1.mac_stats.bcast_pkts_txd: 2408182 > dev.igb.1.mac_stats.mcast_pkts_txd: 246782 > dev.igb.1.mac_stats.tx_frames_64: 24604769631 > dev.igb.1.mac_stats.tx_frames_65_127: 21373976133 > dev.igb.1.mac_stats.tx_frames_128_255: 3047554762 > dev.igb.1.mac_stats.tx_frames_256_511: 3751820879 > dev.igb.1.mac_stats.tx_frames_512_1023: 3481550512 > dev.igb.1.mac_stats.tx_frames_1024_1522: 34713620448 > dev.igb.1.mac_stats.tso_txd: 9549335102 > dev.igb.1.mac_stats.tso_ctx_fail: 0 > dev.igb.1.interrupts.asserts: 16929843739 > dev.igb.1.interrupts.rx_pkt_timer: 76834839132 > dev.igb.1.interrupts.rx_abs_timer: 0 > dev.igb.1.interrupts.tx_pkt_timer: 0 > dev.igb.1.interrupts.tx_abs_timer: 76835042438 > dev.igb.1.interrupts.tx_queue_empty: 90970561829 > dev.igb.1.interrupts.tx_queue_min_thresh: 0 > dev.igb.1.interrupts.rx_desc_min_thresh: 0 > dev.igb.1.interrupts.rx_overrun: 0 > dev.igb.1.host.breaker_tx_pkt: 0 > dev.igb.1.host.host_tx_pkt_discard: 0 > dev.igb.1.host.rx_pkt: 2865171 > dev.igb.1.host.breaker_rx_pkts: 0 > dev.igb.1.host.breaker_rx_pkt_drop: 0 > dev.igb.1.host.tx_good_pkt: 2730536 > dev.igb.1.host.breaker_tx_pkt_drop: 0 > dev.igb.1.host.rx_good_bytes: 36204307257585 > dev.igb.1.host.tx_good_bytes: 59038220743158 > dev.igb.1.host.length_errors: 0 > dev.igb.1.host.serdes_violation_pkt: 0 > dev.igb.1.host.header_redir_missed: 0 > > > > # cat /var/run/dmesg.boot |grep igb > igb0: mem > 0xd5ee0000-0xd5efffff,0xd5800000-0xd5bfffff,0xd5edc000-0xd5edffff irq 18 > at device 0.0 on pci8 > igb0: Using MSIX interrupts with 2 vectors > igb0: Ethernet address: 00:1b:21:70:5f:80 > igb1: mem > 0xd5ea0000-0xd5ebffff,0xd5400000-0xd57fffff,0xd5ed8000-0xd5edbfff irq 19 > at device 0.1 on pci8 > igb1: Using MSIX interrupts with 2 vectors > igb1: Ethernet address: 00:1b:21:70:5f:81 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 14:16:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1080C3A2; Tue, 25 Mar 2014 14:16:58 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B1054128; Tue, 25 Mar 2014 14:16:57 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id e89so1514905qgf.5 for ; Tue, 25 Mar 2014 07:16:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RQxNn8XLvTW8uv/AeHNi1OEiL60Y2Jbvx6jBvqozXQs=; b=kpHouiaB0X/pm+0ZDES/tbbW9eHlad6rt4vobxqls4nnudSF0WsbAMMhiHe2eq8kF3 Gl0lZMlSr6MS59cDi/CkQCsBb+5ptFZgIvrqGf6jurHAk3sbcm9MOZ7Xbl1LxiHkH/ZY P2HWvKQqTVGWwQg8aKcx8I0N4dPG7eOSAdF0QgW7H5ydCWpruFNJIE8chJvf0ghAKcf8 x0e/Hmdj1XYyk6ihVOthtCMowuVYuRRGilPwwXZtesS1UYA4r0104L0GPnwlOSRGg0nt AiwMRkZ0CblrLlWJ46c5Emz2x4T3MeQ3ZtC8GAecHG9uKrskYN/RlyCAGTHJgStc34YW joeQ== MIME-Version: 1.0 X-Received: by 10.140.20.167 with SMTP id 36mr48316780qgj.54.1395757016878; Tue, 25 Mar 2014 07:16:56 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 07:16:56 -0700 (PDT) In-Reply-To: <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> References: <1973302314.2516695.1395710288222.JavaMail.root@uoguelph.ca> <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> Date: Tue, 25 Mar 2014 11:16:56 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 14:16:58 -0000 Hi guys, I'm in meetings today, so I'll respond to the other emails later. Just wanted to clarify about tp->t_tsomax : I can't make a solid assertion about it's value as I only tracked it briefly. I did see it being != if_hw_tsomax, but that was a short test and should really be checked more carefully. For now we should assume it's a possible, but not confirmed. However, setting if_hw_tsomax as low as 32k did not fix the problem for me. So either setting TSO is not the fix, or not everything is paying attention to if_hw_tsomax. It has to be one or the other. Setting IP_MAXPACKET does fix it for me, but of course that's not a solid fix. On Tue, Mar 25, 2014 at 9:16 AM, Markus Gebert wrote: > > On 25.03.2014, at 02:18, Rick Macklem wrote: > > > Christopher Forgeron wrote: > >> > >> > >> > >> This is regarding the TSO patch that Rick suggested earlier. (With > >> many thanks for his time and suggestion) > >> > >> > >> As I mentioned earlier, it did not fix the issue on a 10.0 system. It > >> did make it less of a problem on 9.2, but either way, I think it's > >> not needed, and shouldn't be considered as a patch for testing/etc. > >> > >> > >> Patching TSO to anything other than a max value (and by default the > >> code gives it IP_MAXPACKET) is confusing the matter, as the packet > >> length ultimately needs to be adjusted for many things on the fly > >> like TCP Options, etc. Using static header sizes won't be a good > >> idea. > >> > > If you look at tcp_output(), you'll notice that it doesn't do TSO if > > there are any options. That way it knows that the TCP/IP header is > > just hdrlen. > > > > If you don't limit the TSO packet (including TCP/IP and ethernet headers) > > to 64K, then the "ix" driver can't send them, which is the problem > > you guys are seeing. > > > > There are other ways to fix this problem, but they all may introduce > > issues that reducing if_hw_tsomax by a small amount does not. > > For example, m_defrag() could be modified to use 4K pagesize clusters, > > but this might introduce memory fragmentation problems. (I observed > > what I think are memory fragmentation problems when I switched NFS > > to use 4K pagesize clusters for large I/O messages.) > > > > If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG > > error replies), then that is the size that if_hw_tsomax can be set > > to (just can't change IP_MAXPACKET, but that is defined for other > > things). (It just happens that IP_MAXPACKET is what if_hw_tsomax > > defaults to. It has no other effect w.r.t. TSO.) > > > >> > >> Additionally, it seems that setting nic TSO will/may be ignored by > >> code like this in sys/netinet/tcp_output.c: > >> > > Is this confirmed or still a 'it seems'? Have you actually seen a > tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was this > just speculation because the values are stored in different places? (Sorry, > if you already stated this in another email, it's currently hard to keep > track of all the information.) > > Anyway, this dtrace one-liner should be a good test if other values appear > in tp->t_tsomax: > > # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && > args[0]->t_tsomax != 65518 / { printf("unexpected tp->t_tsomax: %i\n", > args[0]->t_tsomax); stack(); }' > > Remember to adjust the value in the condition to whatever you're currently > expecting. The value seems to be 0 for new connections, probably when > tcp_mss() has not been called yet. So that's seems normal and I have > excluded that case too. This will also print a kernel stack trace in case > it sees an unexpected value. > > > > Yes, but I don't know why. > > The only conjecture I can come up with is that another net driver is > > stacked above "ix" and the setting for if_hw_tsomax doesn't propagate > > up. (If you look at the commit log message for r251296, the intent > > of adding if_hw_tsomax was to allow device drivers to set a smaller > > tsomax than IP_MAXPACKET.) > > > > Are you using any of the "stacked" network device drivers like > > lagg? I don't even know what the others all are? > > Maybe someone else can list them? > > I guess the most obvious are lagg and vlan (and probably carp on FreeBSD > 9.x or older). > > On request from Jack, we've eliminated lagg and vlan from the picture, > which gives us plain ixgbe interfaces with no stacked interfaces on top of > it. And we can still reproduce the problem. > > > Markus > > > > > > rick > >> > >> 10.0 Code: > >> > >> 780 if (len > tp->t_tsomax - hdrlen) { !! > >> 781 len = tp->t_tsomax - hdrlen; !! > >> 782 sendalot = 1; > >> 783 } > >> > >> > >> > >> > >> I've put debugging here, set the nic's max TSO as per Rick's patch ( > >> set to say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. > >> It's being set someplace else, and thus our attempts to set TSO on > >> the nic may be in vain. > >> > >> > >> It may have mattered more in 9.2, as I see the code doesn't use > >> tp->t_tsomax in some locations, and may actually default to what the > >> nic is set to. > >> > >> The NIC may still win, I didn't walk through the code to confirm, it > >> was enough to suggest to me that setting TSO wouldn't fix this > >> issue. > >> > >> > >> However, this is still a TSO related issue, it's just not one related > >> to the setting of TSO's max size. > >> > >> A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > >> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit > >> longer to increase confidence in this assertion, but I don't want to > >> waste time on this when I could be logging problem packets on a > >> system with TSO enabled. > >> > >> > >> Comments are very welcome.. > >> > >> > >> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 17:09:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B5BA956 for ; Tue, 25 Mar 2014 17:09:50 +0000 (UTC) Received: from mail-pb0-x22d.google.com (mail-pb0-x22d.google.com [IPv6:2607:f8b0:400e:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 04E7481B for ; Tue, 25 Mar 2014 17:09:49 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id uo5so724418pbc.18 for ; Tue, 25 Mar 2014 10:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=//iSyKJpTVah8I5RNmlZ9AXDdaYAXialOiZ0ehmf4j8=; b=No46yPoSOhDDIALS2pFKHy3BmfFzteqvzgtRT2CAemINhR+8NVI/WU2JiOlnWSUYXW OmEBVo8h9TnUX1X7ihthlensTNwgRX/Ei/oPN43RqcbjCH7joziMa1JCU1O22yx9qQAS rvTf7YRovnGU488KbIG4D5isy8QMyo1q3CZ8DooO7Sx6tTV2OmxsWd7mTKTVCyMVE839 e1VhSa1xor68yEV5N+5Hvu947sXp6XY4kx8g4cMM727a2ZLIJaGcPcUVC2MWkeshBl4m XyQrWsU3t/ORJ5PsTRM3qPVXuI/U3YTzwrx3GfQhxc1ya7wPbq/iLBIenCyxnBsDMvOK f5sQ== MIME-Version: 1.0 X-Received: by 10.68.250.3 with SMTP id yy3mr79980149pbc.56.1395767389697; Tue, 25 Mar 2014 10:09:49 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.0.164 with HTTP; Tue, 25 Mar 2014 10:09:49 -0700 (PDT) Date: Tue, 25 Mar 2014 10:09:49 -0700 X-Google-Sender-Auth: h_NhiNdYXGG6pQg8Autxj09ipYM Message-ID: Subject: Server sockets staying in CLOSED for extended periods From: Kevin Oberman To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 17:09:50 -0000 There has been a long thread on stable about sshd processes being hung due to sockets remaining in a CLOSED state for extended periods in 10-stable. This did not seem to be happening with 9.2. (Not sure about 10.0.) Was here a change in the network stack on 10 that would have kept CLOSED sockets around for extended intervals? Should sshd processes hang for extended periods (forever?) unless killed (KILL)? I also wonder if a daemon process should refuse to exit if a closed socket remains. Any suggestions would help. the full thread with a great deal of detail is in stable with the subject "sshd with zombie process on FreeBSD 10.0-STABLE - workaround". -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 18:36:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6E3148D for ; Tue, 25 Mar 2014 18:36:07 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B3ABA2FE for ; Tue, 25 Mar 2014 18:36:07 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id B5DF17321B for ; Tue, 25 Mar 2014 11:36:06 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 82225-02 for ; Tue, 25 Mar 2014 11:36:06 -0700 (PDT) Received: from kruse-47.3.ixsystems.com (kruse-47.3.ixsystems.com [10.2.3.47]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 416A573212 for ; Tue, 25 Mar 2014 11:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1395772566; bh=ljMpTkCdEH6ZAh4HxllCMTga3dGJRxIN3N4qWIfSdyY=; h=From:Subject:Date:To; b=Z/p6h+bhDB2ofQKhBIbkhCdpYA9mgtP/E/10Cotx7BJxMb6FEh/h3jaMcK1GfpFbu m5jTbyQcWzxYOR6ccxnTBpCKZVDp3NVx9Coe+AW6txHp6ZeCUo+spNGwuHshfJi9mz EdkGzXhp/jPOTcNERvE4mb1AqSiv/og8hZpKFrYE= From: Sean Fagan Content-Type: multipart/mixed; boundary="Apple-Mail=_CF84D4FE-C7D8-487C-93B9-541771AA3154" Subject: Non-interrupt packet sending and receiving Message-Id: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> Date: Tue, 25 Mar 2014 11:36:05 -0700 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 18:36:07 -0000 --Apple-Mail=_CF84D4FE-C7D8-487C-93B9-541771AA3154 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii This isn't the same as the polled driver; this is sending and receiving = a single packet at a time. I've gotten (at least to a somewhat workable degree) Apple's KDP ported = to FreeBSD. I've only changed the dev/e1000/if_lem.c driver for now = (that's the one VMWare shows up as :)), but since I'm not particularly = comfortable with device drivers, let alone ethernet drivers, I needed = some feedback. Diffs are attached below. Feedback would be appreciated. (To answer some of the questions I've already gotten: no, i can't use = the DEVICE_POLLING routines, because that still goes through the entire = stack. It's not here, because I am not yet happy with it, but the code = that uses this runs in the kernel debugger, and it needs to be able to = send and receive a single packet at a time -- and it can't let it be = shuffled off through other layers, for reasons that I hope are fairly = clear. Now, one change I would like to make is the mbuf allocation that = it uses; ideally, honestly, it should have its own mbufs -- the protocol = never sends more than 1538 bytes in a UDP packet -- but I would probably = try working on another ethernet driver first, modeling it after this.) (The bulk of the diffs is moving some code out of lem_rxeof into a = function that gets a single packet.) Thanks, Sean. --Apple-Mail=_CF84D4FE-C7D8-487C-93B9-541771AA3154 Content-Disposition: attachment; filename=if_lem-diffs.txt Content-Type: text/plain; name="if_lem-diffs.txt" Content-Transfer-Encoding: quoted-printable diff --git a/dev/e1000/if_lem.c b/dev/e1000/if_lem.c index bfe2c93..90ec8b3 100644 --- a/dev/e1000/if_lem.c +++ b/dev/e1000/if_lem.c @@ -34,6 +34,7 @@ =20 #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_ddb.h" =20 #ifdef HAVE_KERNEL_OPTION_HEADERS #include "opt_device_polling.h" @@ -54,6 +55,7 @@ #include #include #include +#include #include #include =20 @@ -191,7 +193,7 @@ static void lem_free_transmit_structures(struct = adapter *); static void lem_free_receive_structures(struct adapter *); static void lem_update_stats_counters(struct adapter *); static void lem_add_hw_stats(struct adapter *adapter); -static void lem_txeof(struct adapter *); +static void lem_txeof(struct adapter *, int); static void lem_tx_purge(struct adapter *); static int lem_allocate_receive_structures(struct adapter *); static int lem_allocate_transmit_structures(struct adapter *); @@ -246,6 +248,74 @@ static void lem_handle_rxtx(void *context, = int pending); static void lem_handle_link(void *context, int pending); static void lem_add_rx_process_limit(struct adapter *, const char *, const char *, int *, int); +#ifdef DDB +typedef uint32_t (*kdp_link_t)(void); +typedef int (*kdp_mode_t)(int); +extern void *kdp_get_interface(void); +typedef void (*kdp_send_t)(void * pkt, unsigned int pkt_len); +typedef void (*kdp_receive_t)(void * pkt, unsigned int * pkt_len, + unsigned int timeout); +extern void kdp_register_send_receive(kdp_send_t send, kdp_receive_t = receive); +extern void kdp_unregister_send_receive(kdp_send_t send, kdp_receive_t = receive); + +/* + * Function called by kdp. + * timeout is in milliseconds. + * The data is 1538 bytes. + * Return the length in *length; set it to 0 if no packet. + */ +static void lem_kdp_recv_pkt(void *data, unsigned int *length, unsigned = int timeout); + +static void lem_kdp_send_pkt(void *pkt, unsigned int pkt_len); + +/* + * For kdp: + * lex_rxeof() may be usable. However, we'd have to + * change the ifp->if_input() function pointer to be + * something more conducive to our needs. + * + * For transmitting, we'd want to use lem_txeof, but + * we have to figure out how to put data into the + * adapter queue. Something like: + * + * struct mbchain *mbp; + * struct mbuf *m; + * mb_init(mbp); + * mb_put_uint8(mbp, 33); + * mb_put_uint16le(mbp, length); + * m =3D m_copym(mbp->mb_top, 0, M_COPYALL, M_WAIT); + * + * Then we have to get the mbuf chain (m) into the + * device's queue. + * + * Then: + * + * mb_done(mbp); + * + */ +/* + * Tell kdp about functions to query status. + * The link parameter is a pointer to a function + * which returns the status (it only cares about + * IFM_AVALID and IFM_ACTIVE). Note that it has no + * parameters -- so it has to use the kdp_get_interface() + * function to find out the current interface. This should + * probably change. + * + * Similarly, the mode parameter is a function which sets the + * status active (if its parameter is non-zero) or inactive (if + * its parameter is 0). + */ +void kdp_register_link(kdp_link_t link, kdp_mode_t mode); +void kdp_unregister_link(kdp_link_t link, kdp_mode_t mode); + +// This is a bit of a lie: it actually takes a pointer to a = kdp_ether_addr_t structure. +void kdp_set_interface(void *interface, const void *macaddr); + +static uint32_t kdp_media_status(void); +static int kdp_set_media_state(int); + +#endif =20 #ifdef DEVICE_POLLING static poll_handler_t lem_poll; @@ -835,7 +905,7 @@ lem_start_locked(struct ifnet *ifp) * available hits the threshold */ if (adapter->num_tx_desc_avail <=3D EM_TX_CLEANUP_THRESHOLD) { - lem_txeof(adapter); + lem_txeof(adapter, 0); /* Now do we at least have a minimal? */ if (adapter->num_tx_desc_avail <=3D EM_TX_OP_THRESHOLD) = { adapter->no_tx_desc_avail1++; @@ -1212,6 +1282,14 @@ lem_init_locked(struct adapter *adapter) /* AMT based hardware can now take control from firmware */ if (adapter->has_manage && adapter->has_amt) lem_get_hw_control(adapter); + +#ifdef DDB + printf("Setting interface to %p\n", ifp); + kdp_set_interface(ifp, adapter->hw.mac.addr); + kdp_register_link(kdp_media_status, NULL = /*kdp_set_media_state*/); + kdp_register_send_receive(lem_kdp_send_pkt, lem_kdp_recv_pkt); + +#endif } =20 static void @@ -1258,7 +1336,7 @@ lem_poll(struct ifnet *ifp, enum poll_cmd cmd, int = count) lem_rxeof(adapter, count, &rx_done); =20 EM_TX_LOCK(adapter); - lem_txeof(adapter); + lem_txeof(adapter, 0); if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) lem_start_locked(ifp); EM_TX_UNLOCK(adapter); @@ -1309,7 +1387,7 @@ lem_intr(void *arg) lem_rxeof(adapter, -1, NULL); =20 EM_TX_LOCK(adapter); - lem_txeof(adapter); + lem_txeof(adapter, 0); if (ifp->if_drv_flags & IFF_DRV_RUNNING && !IFQ_DRV_IS_EMPTY(&ifp->if_snd)) lem_start_locked(ifp); @@ -1348,7 +1426,7 @@ lem_handle_rxtx(void *context, int pending) if (ifp->if_drv_flags & IFF_DRV_RUNNING) { bool more =3D lem_rxeof(adapter, = adapter->rx_process_limit, NULL); EM_TX_LOCK(adapter); - lem_txeof(adapter); + lem_txeof(adapter, 0); if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) lem_start_locked(ifp); EM_TX_UNLOCK(adapter); @@ -1405,6 +1483,34 @@ lem_irq_fast(void *arg) return FILTER_HANDLED; } =20 +#ifdef DDB +static uint32_t __unused +kdp_media_status(void) +{ + struct ifnet *ifp =3D kdp_get_interface(); + struct ifmediareq ifmr =3D { .ifm_status =3D 0 }; + + if (ifp) { + lem_media_status(ifp, &ifmr); + } + return ifmr.ifm_status; +} + +static int __unused +kdp_set_media_state(int onoroff) +{ + struct ifnet *ifp =3D kdp_get_interface(); + if (ifp) { + if (onoroff) { + lem_start(ifp); + } else { + lem_stop(ifp); + } + } + return 1; +} + +#endif =20 /********************************************************************* * @@ -2975,14 +3081,15 @@ lem_transmit_checksum_setup(struct adapter = *adapter, struct mbuf *mp, * = **********************************************************************/ static void -lem_txeof(struct adapter *adapter) +lem_txeof(struct adapter *adapter, int nolock) { int first, last, done, num_avail; struct em_buffer *tx_buffer; struct e1000_tx_desc *tx_desc, *eop_desc; struct ifnet *ifp =3D adapter->ifp; =20 - EM_TX_LOCK_ASSERT(adapter); + if (nolock =3D=3D 0) + EM_TX_LOCK_ASSERT(adapter); =20 #ifdef DEV_NETMAP if (netmap_tx_irq(ifp, 0 | = (NETMAP_LOCKED_ENTER|NETMAP_LOCKED_EXIT))) @@ -3082,7 +3189,7 @@ lem_tx_purge(struct adapter *adapter) { if ((!adapter->link_active) && (adapter->watchdog_check)) { EM_TX_LOCK(adapter); - lem_txeof(adapter); + lem_txeof(adapter, 0); EM_TX_UNLOCK(adapter); if (adapter->watchdog_check) /* Still outstanding? */ lem_init_locked(adapter); @@ -3426,6 +3533,254 @@ lem_free_receive_structures(struct adapter = *adapter) } } =20 +static __unused struct mbuf * +lem_recv_one_packet(struct adapter *adapter, + int *start_desc_index) +{ + struct mbuf *retval =3D NULL; + struct mbuf *mp; // Used to build up retval + int i =3D *start_desc_index; + struct ifnet *ifp =3D adapter->ifp; + u8 status =3D 0, accept_frame =3D 0, eop =3D 0; + u16 len, desc_len, prev_len_adj; + struct e1000_rx_desc *current_desc =3D NULL; + +// printf("%s(%d)\n", __FUNCTION__, __LINE__); + + + while (eop =3D=3D 0) { + current_desc =3D &adapter->rx_desc_base[i]; + status =3D current_desc->status; + + if ((status & E1000_RXD_STAT_DD) =3D=3D 0) { + break; + } + + mp =3D adapter->rx_buffer_area[i].m_head; + /* + * Can't defer bus_dmamap_sync(9) because TBI_ACCEPT + * needs to access the last received byte in the mbuf. + */ + bus_dmamap_sync(adapter->rxtag, = adapter->rx_buffer_area[i].map, + BUS_DMASYNC_POSTREAD); + accept_frame =3D 1; + prev_len_adj =3D 0; + desc_len =3D le16toh(current_desc->length); + + if (status & E1000_RXD_STAT_EOP) { + eop =3D 1; // Found the end of packet + if (desc_len < ETHER_CRC_LEN) { + len =3D 0; + prev_len_adj =3D ETHER_CRC_LEN - = desc_len; + } else { + len =3D desc_len - ETHER_CRC_LEN; + } + } else { + len =3D desc_len; + } + + if (current_desc->errors & E1000_RXD_ERR_FRAME_ERR_MASK) = { + u8 last_byte; + u32 pkt_len =3D desc_len; + =09 + if (adapter->fmp !=3D NULL) + pkt_len +=3D adapter->fmp->m_pkthdr.len; + =09 + last_byte =3D *(mtod(mp, caddr_t) + desc_len - = 1); =09 + if (TBI_ACCEPT(&adapter->hw, status, + current_desc->errors, pkt_len, = last_byte, + adapter->min_frame_size, = adapter->max_frame_size)) { + = e1000_tbi_adjust_stats_82543(&adapter->hw, + = &adapter->stats, pkt_len, + = adapter->hw.mac.addr, + = adapter->max_frame_size); + if (len > 0) + len--; + } else + accept_frame =3D 0; + } + if (accept_frame) { + if (lem_get_buf(adapter, i) !=3D 0) { + ifp->if_iqdrops++; + goto discard; + } + =09 + /* Assign correct length to the current fragment = */ + mp->m_len =3D len; + =09 + if (adapter->fmp =3D=3D NULL) { + mp->m_pkthdr.len =3D len; + adapter->fmp =3D mp; /* Store the first = mbuf */ + adapter->lmp =3D mp; + } else { + /* Chain mbuf's together */ + mp->m_flags &=3D ~M_PKTHDR; + /* + * Adjust length of previous mbuf in = chain if + * we received less than 4 bytes in the = last + * descriptor. + */ + if (prev_len_adj > 0) { + adapter->lmp->m_len -=3D = prev_len_adj; + adapter->fmp->m_pkthdr.len -=3D + prev_len_adj; + } + adapter->lmp->m_next =3D mp; + adapter->lmp =3D adapter->lmp->m_next; + adapter->fmp->m_pkthdr.len +=3D len; + } + =09 + if (eop) { + adapter->fmp->m_pkthdr.rcvif =3D ifp; + ifp->if_ipackets++; + lem_receive_checksum(adapter, = current_desc, + adapter->fmp); +#ifndef __NO_STRICT_ALIGNMENT + if (adapter->max_frame_size > + (MCLBYTES - ETHER_ALIGN) && + lem_fixup_rx(adapter) !=3D 0) + goto skip; +#endif + if (status & E1000_RXD_STAT_VP) { + = adapter->fmp->m_pkthdr.ether_vtag =3D + = le16toh(current_desc->special); + adapter->fmp->m_flags |=3D = M_VLANTAG; + } +#ifndef __NO_STRICT_ALIGNMENT + skip: +#endif + retval =3D adapter->fmp; + adapter->fmp =3D NULL; + adapter->lmp =3D NULL; + } + } else { + adapter->dropped_pkts++; + discard: + /* Reuse loaded DMA map and just update mbuf = chain */ + mp =3D adapter->rx_buffer_area[i].m_head; + mp->m_len =3D mp->m_pkthdr.len =3D MCLBYTES; + mp->m_data =3D mp->m_ext.ext_buf; + mp->m_next =3D NULL; + if (adapter->max_frame_size <=3D + (MCLBYTES - ETHER_ALIGN)) + m_adj(mp, ETHER_ALIGN); + if (adapter->fmp !=3D NULL) { + m_freem(adapter->fmp); + adapter->fmp =3D NULL; + adapter->lmp =3D NULL; + } + retval =3D NULL; + } + /* Zero out the receive descriptors status. */ + current_desc->status =3D 0; + bus_dmamap_sync(adapter->rxdma.dma_tag, = adapter->rxdma.dma_map, + BUS_DMASYNC_PREREAD | = BUS_DMASYNC_PREWRITE); +=09 + /* Advance our pointers to the next descriptor. */ + if (++i =3D=3D adapter->num_rx_desc) + i =3D 0; + } + + *start_desc_index =3D i; + /* + * At this point, either retval has a packet, or is NULL + */ +// printf("%s(%d): retval =3D %p\n", __FUNCTION__, __LINE__, = retval); + return retval; +} + +static void +lem_kdp_send_pkt(__unused void *pkt, __unused unsigned int pkt_len) +{ + printf("%s\n", __FUNCTION__); + struct mbuf *m_head =3D NULL; + struct ifnet *ifp =3D kdp_get_interface(); + struct adapter *adapter =3D ifp->if_softc; + struct iovec iov; + struct uio uio; + int error; + + printf("%s(%p, %u)\n", __FUNCTION__, pkt, pkt_len); + + iov.iov_base =3D (caddr_t)pkt; + iov.iov_len =3D pkt_len; + uio.uio_iov =3D &iov; + uio.uio_iovcnt =3D 1; + uio.uio_offset =3D 0; + uio.uio_resid =3D (ssize_t)pkt_len; + uio.uio_segflg =3D UIO_SYSSPACE; + uio.uio_rw =3D UIO_WRITE; + uio.uio_td =3D curthread; + + + m_head =3D m_uiotombuf(&uio, M_WAITOK, pkt_len, 0, 0); +=09 + if (m_head =3D=3D NULL) { + printf("%s(%d): m_utiotombuf failed us!\n", = __FUNCTION__, __LINE__); + } else { + lem_txeof(adapter, 1); + printf("%s(%d): calling lem_xmit\n", __FUNCTION__, = __LINE__); + error =3D lem_xmit(adapter, &m_head); + =09 + if (error !=3D 0) { + printf("%s(%d): lem_xmit returned %d\n", = __FUNCTION__, __LINE__, error); + } else { + printf("%s(%d): calling lem_txeof\n", = __FUNCTION__, __LINE__); + lem_txeof(adapter, 1); + printf("\tand done\n"); + } + if (m_head) + m_freem(m_head); + } + return; +} + +static void __unused +lem_kdp_recv_pkt(void *data, unsigned int *length, unsigned int = timeout) +{ + *length =3D 0; + int indx =3D 0; + int uSecs =3D timeout * 1000; // DELAY takes microseconds + struct mbuf *packet =3D NULL; + struct ifnet *ifp =3D kdp_get_interface(); + struct adapter *adapter =3D ifp->if_softc; + struct e1000_rx_desc *current_desc; + + indx =3D adapter->next_rx_desc_to_check; + current_desc =3D &adapter->rx_desc_base[indx]; + + bus_dmamap_sync(adapter->rxdma.dma_tag, + adapter->rxdma.dma_map, + BUS_DMASYNC_POSTREAD); + + if ((current_desc->status & E1000_RXD_STAT_DD) =3D=3D 0) { + goto done; + } + + while (packet =3D=3D NULL && + (ifp->if_drv_flags & IFF_DRV_RUNNING) && + uSecs > 0) { + packet =3D lem_recv_one_packet(adapter, &indx); + + if (packet =3D=3D NULL) { + DELAY(50); + uSecs -=3D 50; + } + } + if (packet !=3D NULL) { + int packet_len =3D packet->m_pkthdr.len; + m_copydata(packet, 0, packet_len, data); + *length =3D packet_len; + adapter->next_rx_desc_to_check =3D indx; + if (indx =3D=3D 0) { + E1000_WRITE_REG(&adapter->hw, E1000_RDT(0), = adapter->num_rx_desc - 1); + } + + } +done: + return; +} + /********************************************************************* * * This routine executes in interrupt context. It replenishes @@ -3441,9 +3796,11 @@ static bool lem_rxeof(struct adapter *adapter, int count, int *done) { struct ifnet *ifp =3D adapter->ifp; - struct mbuf *mp; - u8 status =3D 0, accept_frame =3D 0, eop =3D 0; - u16 len, desc_len, prev_len_adj; + u8 status =3D 0; + u8 __unused accept_frame =3D 0; + u8 __unused eop =3D 0; + u16 __unused len; + u16 __unused desc_len, prev_len_adj; int i, rx_sent =3D 0; struct e1000_rx_desc *current_desc; =20 @@ -3467,6 +3824,14 @@ lem_rxeof(struct adapter *adapter, int count, int = *done) =20 while (count !=3D 0 && ifp->if_drv_flags & IFF_DRV_RUNNING) { struct mbuf *m =3D NULL; +#if 1 + current_desc =3D &adapter->rx_desc_base[i]; + status =3D current_desc->status; + if ((status & E1000_RXD_STAT_DD) =3D=3D 0) + break; + m =3D lem_recv_one_packet(adapter, &i); +#else + struct mbuf *mp =3D NULL; =20 status =3D current_desc->status; if ((status & E1000_RXD_STAT_DD) =3D=3D 0) @@ -3598,6 +3963,8 @@ discard: /* Advance our pointers to the next descriptor. */ if (++i =3D=3D adapter->num_rx_desc) i =3D 0; +#endif + /* Call into the stack */ if (m !=3D NULL) { adapter->next_rx_desc_to_check =3D i; --Apple-Mail=_CF84D4FE-C7D8-487C-93B9-541771AA3154 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_CF84D4FE-C7D8-487C-93B9-541771AA3154-- From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 19:15:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E36B0268 for ; Tue, 25 Mar 2014 19:15:32 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AFD6C936 for ; Tue, 25 Mar 2014 19:15:32 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id uz6so1135253obc.15 for ; Tue, 25 Mar 2014 12:15:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1VloIbCUU/AIw8etmufqcxam1OiOVWjeitPdU8ofuj0=; b=RBI+ayAHQqeuYBRUNxOwcMz8Rj9LUeHTjtj6+UqymG8oBP5SKH+j2VYBK1fPoYLTEh 1HSsNoecrclC2X818VvTrql+Dj+pWkGTyvjjdJFo5mLKzyrfFaVaO7h8BOoH+DDfEPeg EvCEwoRv0ARLbRq0t/dpQaPBYyEW8iEMOy4bLnRdYe7QOuKBpoP46iAYe3zny15R4whq FFUnbpYJqsPVASNIYFtigBoordYDsiHZwsF4HK52U+zuZ/U2iK6bJrAmAcN3HIrvOTF0 2B+fMEQOsx+1BeFNzQoY2fNibtcDB3rk4JZwvjUja8K09fi2G3AqOmpQo1zXkgvGUbuc 0NeQ== MIME-Version: 1.0 X-Received: by 10.60.142.229 with SMTP id rz5mr63604676oeb.1.1395774932084; Tue, 25 Mar 2014 12:15:32 -0700 (PDT) Received: by 10.76.115.129 with HTTP; Tue, 25 Mar 2014 12:15:32 -0700 (PDT) In-Reply-To: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> References: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> Date: Tue, 25 Mar 2014 15:15:32 -0400 Message-ID: Subject: Re: Non-interrupt packet sending and receiving From: Ryan Stone To: Sean Fagan Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 19:15:32 -0000 You might want to take a look at the projects/sv branch, which implement kernel core dumps over the network. We had to solve a similar problem there (in lem, em, igb and ixgbe) and ended up piggybacking on most of the DEVICE_POLLING code to do it. The work ended up stalling over objections over calling into the mbuf allocator (which I guess may be a stumbling block for your work). Unfortunately we weren't able to come up with a clean way to share the existing rx/tx paths in the drivers while separating the mbuf allocations out. From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 19:25:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D53147F8; Tue, 25 Mar 2014 19:25:02 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 81B21A2A; Tue, 25 Mar 2014 19:25:02 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id r5so1293498qcx.18 for ; Tue, 25 Mar 2014 12:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LCuUVuZryU+TxFGWuvR9OkvT/N9f9ThECrmyvj9eat4=; b=wTjH6TJrzLZ0KC+jqFsZ89FzYUrjqHU6hs/dSy54JBy4IPb7JctRvhMJs5cdzxzi0Z +Dv0IKekmvrNvgB+V9lSq9o4tG9Hr8ZoAe2ZP9xROSKZzKjxxC8E7oayo+MGIhGOV8jg T4PMHBMQJ087alMtsJGAwL8xCZTxCPjOo3MwZKBGG+1KUI4clD5nU1tQaV6wKBMq//X/ dTSyJT4yUeoKTU26pEQUV2/ZEZKMu6VXD4WVtcriZZD7OkKpXGj2AkHULrrKo0fqw7Bf JgkYUh9ApHb6F5Dn5O1wd2/+75bml7LdqYpmPJRXsRv3DPP+v9JlT0FeLgj2z6/Vf/Rs nVLw== MIME-Version: 1.0 X-Received: by 10.140.20.167 with SMTP id 36mr50244943qgj.54.1395775501045; Tue, 25 Mar 2014 12:25:01 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 12:25:00 -0700 (PDT) In-Reply-To: <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> Date: Tue, 25 Mar 2014 16:25:00 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 19:25:03 -0000 I'm quite positive that an IP_MAXPACKET = 65518 would fix this, as I've never seen a packet overshoot by more than 11 bytes, although that's just in my case. It's next up on my test list. BTW, to answer the next message: I am expierencing the error with a raw ix or lagg interface. Originally I was on lagg, but have dropped down to a single ix for testing. Thanks for your continued help. On Mon, Mar 24, 2014 at 10:04 PM, Rick Macklem wrote: > Markus Gebert wrote: > > > > On 24.03.2014, at 16:21, Christopher Forgeron > > wrote: > > > > > This is regarding the TSO patch that Rick suggested earlier. (With > > > many > > > thanks for his time and suggestion) > > > > > > As I mentioned earlier, it did not fix the issue on a 10.0 system. > > > It did > > > make it less of a problem on 9.2, but either way, I think it's not > > > needed, > > > and shouldn't be considered as a patch for testing/etc. > > > > > > Patching TSO to anything other than a max value (and by default the > > > code > > > gives it IP_MAXPACKET) is confusing the matter, as the packet > > > length > > > ultimately needs to be adjusted for many things on the fly like TCP > > > Options, etc. Using static header sizes won't be a good idea. > > > > > > Additionally, it seems that setting nic TSO will/may be ignored by > > > code > > > like this in sys/netinet/tcp_output.c: > > > > > > 10.0 Code: > > > > > > 780 if (len > tp->t_tsomax - hdrlen) > > > { !! > > > 781 len = tp->t_tsomax - > > > hdrlen; !! > > > 782 sendalot = > > > 1; > > > 783 } > > > > > > > > > I've put debugging here, set the nic's max TSO as per Rick's patch > > > ( set to > > > say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. It's > > > being set > > > someplace else, and thus our attempts to set TSO on the nic may be > > > in vain. > > > > > > It may have mattered more in 9.2, as I see the code doesn't use > > > tp->t_tsomax in some locations, and may actually default to what > > > the nic is > > > set to. > > > > > > The NIC may still win, I didn't walk through the code to confirm, > > > it was > > > enough to suggest to me that setting TSO wouldn't fix this issue. > > > > > > I just applied Rick's ixgbe TSO patch and additionally wanted to be > > able to easily change the value of hw_tsomax, so I made a sysctl out > > of it. > > > > While doing that, I asked myself the same question. Where and how > > will this value actually be used and how comes that tcp_output() > > uses that other value in struct tcpcb. > > > > The only place tcpcb->t_tsomax gets set, that I have found so far, is > > in tcp_input.c's tcp_mss() function. Some subfunctions get called: > > > > tcp_mss() -> tcp_mss_update() -> tcp_maxmtu() > > > > Then tcp_maxmtu() indeed uses the interface's hw_tsomax value: > > > > 1746 cap->tsomax = ifp->if_hw_tsomax; > > > > It get's passed back to tcp_mss() where it is set on the connection > > level which will be used in tcp_output() later on. > > > > tcp_mss() gets called from multiple places, I'll look into that > > later. I will let you know if I find out more. > > > > > > Markus > > > Well, if tp->t_tsomax isn't set to a value of 65518, then the ixgbe.patch > isn't doing what I thought it would. > > The only explanation I can think of for this is that there might be > another net interface driver stacked on top of the ixgbe.c one and > that the setting doesn't get propagated up. > Does this make any sense? > > IP_MAXPACKET can't be changed from 65535, but I can see an argument > for setting the default value of if_hw_tsomax to a smaller value. > For example, in sys/net/if.c change it from: > 657 if (ifp->if_hw_tsomax == 0) > 658 ifp->if_hw_tsomax = IP_MAXPACKET; > to > 657 if (ifp->if_hw_tsomax == 0) > 658 ifp->if_hw_tsomax = 65536 - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > > This is a slightly smaller default which won't have much impact unless > the hardware device can only handle 32 mbuf clusters for transmit of > a segment and there are several of those. > > Christopher, can you do your test run with IP_MAXPACKET set to 65518, > which should be the same as the above. If that gets rid of all the > EFBIG error replies, then I think the above patch will have the same > effect. > > Thanks, rick > > > > > > However, this is still a TSO related issue, it's just not one > > > related to > > > the setting of TSO's max size. > > > > > > A 10.0-STABLE system with tso disabled on ix0 doesn't have a single > > > packet > > > over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer > > > to > > > increase confidence in this assertion, but I don't want to waste > > > time on > > > this when I could be logging problem packets on a system with TSO > > > enabled. > > > > > > Comments are very welcome.. > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > "freebsd-net-unsubscribe@freebsd.org" > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 19:33:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01777BC7 for ; Tue, 25 Mar 2014 19:33:44 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D45B6AF0 for ; Tue, 25 Mar 2014 19:33:43 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 42E58721AA; Tue, 25 Mar 2014 12:33:43 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 86558-04-4; Tue, 25 Mar 2014 12:33:43 -0700 (PDT) Received: from kruse-47.3.ixsystems.com (kruse-47.3.ixsystems.com [10.2.3.47]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 0937972196; Tue, 25 Mar 2014 12:33:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1395776011; bh=BEup5g2Ltv4yCstgjlfCPjSMSFL+/z0enBaOc0b9EHE=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=ZetbJeYLv+yOpjaZOSJYiwU5KI/Q6qCbBe6G0yS2v/ASMD2GJVXqYv98GX5xpHkbp 1x1NM0JPKBhQ179X6AekYlhsMvZon5ZQewSJ2C9kR8/nwiYxk5Zcu09YG3gKB1twZL axUI7cKUV2BzRePdoQfJMKQyWZv4qJ9q31TCFoeY= Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Non-interrupt packet sending and receiving From: Sean Fagan In-Reply-To: Date: Tue, 25 Mar 2014 12:33:30 -0700 Content-Transfer-Encoding: 7bit Message-Id: References: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> To: Ryan Stone X-Mailer: Apple Mail (2.1874) Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 19:33:44 -0000 On Mar 25, 2014, at 12:15 PM, Ryan Stone wrote: > You might want to take a look at the projects/sv branch, which > implement kernel core dumps over the network. We had to solve a > similar problem there (in lem, em, igb and ixgbe) and ended up > piggybacking on most of the DEVICE_POLLING code to do it. The work > ended up stalling over objections over calling into the mbuf allocator > (which I guess may be a stumbling block for your work). Unfortunately > we weren't able to come up with a clean way to share the existing > rx/tx paths in the drivers while separating the mbuf allocations out. I checked the XNU drivers I could find online, and it turns out that they did mbuf allocations. Since I was using that as my model, I went with it for now. I was aware of the network core dump work, but not until I'd started to make some progress with this. (The kdp protocol theoretically has support for dumping a core over the network, but I haven't implemented that at all.) Looking at if_lem.c, it appears similar to a large degree. Perhaps I should start from scratch... Sean. From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 21:46:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09418D8E; Tue, 25 Mar 2014 21:46:23 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 7C175994; Tue, 25 Mar 2014 21:46:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAOT4MVODaFve/2dsb2JhbABZg0FXgwe4MIZkUYEzdIIlAQEBAwEBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEcAQOHUAgNrUSiJheBKYxjCwUCARsBMweCb4FJBJV2hAmRAINKITF8QQ X-IronPort-AV: E=Sophos;i="4.97,730,1389762000"; d="scan'208";a="108832838" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Mar 2014 17:45:59 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 0A309B4050; Tue, 25 Mar 2014 17:46:00 -0400 (EDT) Date: Tue, 25 Mar 2014 17:46:00 -0400 (EDT) From: Rick Macklem To: Markus Gebert Message-ID: <2042344654.506796.1395783960030.JavaMail.root@uoguelph.ca> In-Reply-To: <906D7DF8-DD6E-4501-B3ED-42EF728241F4@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 21:46:23 -0000 Markus Gebert wrote: >=20 > On 25.03.2014, at 02:18, Rick Macklem wrote: >=20 > > Christopher Forgeron wrote: > >>=20 > >>=20 > >>=20 > >> This is regarding the TSO patch that Rick suggested earlier. (With > >> many thanks for his time and suggestion) > >>=20 > >>=20 > >> As I mentioned earlier, it did not fix the issue on a 10.0 system. > >> It > >> did make it less of a problem on 9.2, but either way, I think it's > >> not needed, and shouldn't be considered as a patch for > >> testing/etc. > >>=20 > >>=20 > >> Patching TSO to anything other than a max value (and by default > >> the > >> code gives it IP_MAXPACKET) is confusing the matter, as the packet > >> length ultimately needs to be adjusted for many things on the fly > >> like TCP Options, etc. Using static header sizes won't be a good > >> idea. > >>=20 > > If you look at tcp_output(), you'll notice that it doesn't do TSO > > if > > there are any options. That way it knows that the TCP/IP header is > > just hdrlen. > >=20 > > If you don't limit the TSO packet (including TCP/IP and ethernet > > headers) > > to 64K, then the "ix" driver can't send them, which is the problem > > you guys are seeing. > >=20 > > There are other ways to fix this problem, but they all may > > introduce > > issues that reducing if_hw_tsomax by a small amount does not. > > For example, m_defrag() could be modified to use 4K pagesize > > clusters, > > but this might introduce memory fragmentation problems. (I observed > > what I think are memory fragmentation problems when I switched NFS > > to use 4K pagesize clusters for large I/O messages.) > >=20 > > If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG > > error replies), then that is the size that if_hw_tsomax can be set > > to (just can't change IP_MAXPACKET, but that is defined for other > > things). (It just happens that IP_MAXPACKET is what if_hw_tsomax > > defaults to. It has no other effect w.r.t. TSO.) > >=20 > >>=20 > >> Additionally, it seems that setting nic TSO will/may be ignored by > >> code like this in sys/netinet/tcp_output.c: > >>=20 >=20 > Is this confirmed or still a =E2=80=98it seems=E2=80=99? Have you actuall= y seen a > tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was > this just speculation because the values are stored in different > places? (Sorry, if you already stated this in another email, it=E2=80=99s > currently hard to keep track of all the information.) >=20 > Anyway, this dtrace one-liner should be a good test if other values > appear in tp->t_tsomax: >=20 > # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && > args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: > %i\n", args[0]->t_tsomax); stack(); }' >=20 > Remember to adjust the value in the condition to whatever you=E2=80=99re > currently expecting. The value seems to be 0 for new connections, > probably when tcp_mss() has not been called yet. So that=E2=80=99s seems > normal and I have excluded that case too. This will also print a > kernel stack trace in case it sees an unexpected value. >=20 >=20 > > Yes, but I don't know why. > > The only conjecture I can come up with is that another net driver > > is > > stacked above "ix" and the setting for if_hw_tsomax doesn't > > propagate > > up. (If you look at the commit log message for r251296, the intent > > of adding if_hw_tsomax was to allow device drivers to set a smaller > > tsomax than IP_MAXPACKET.) > >=20 > > Are you using any of the "stacked" network device drivers like > > lagg? I don't even know what the others all are? > > Maybe someone else can list them? >=20 > I guess the most obvious are lagg and vlan (and probably carp on > FreeBSD 9.x or older). >=20 > On request from Jack, we=E2=80=99ve eliminated lagg and vlan from the > picture, which gives us plain ixgbe interfaces with no stacked > interfaces on top of it. And we can still reproduce the problem. >=20 This was related to the "did if_hw_tsomax set tp->t_tsomax to the same value?" question. Since you reported that my patch that set if_hw_tsomax in the driver didn't fix the problem, that suggests that tp->t_tsomax isn't being set to if_hw_tsomax from the driver, but we don't know why? rick >=20 > Markus >=20 >=20 > >=20 > > rick > >>=20 > >> 10.0 Code: > >>=20 > >> 780 if (len > tp->t_tsomax - hdrlen) { !! > >> 781 len =3D tp->t_tsomax - hdrlen; !! > >> 782 sendalot =3D 1; > >> 783 } > >>=20 > >>=20 > >>=20 > >>=20 > >> I've put debugging here, set the nic's max TSO as per Rick's patch > >> ( > >> set to say 32k), and have seen that tp->t_tsomax =3D=3D IP_MAXPACKET. > >> It's being set someplace else, and thus our attempts to set TSO on > >> the nic may be in vain. > >>=20 > >>=20 > >> It may have mattered more in 9.2, as I see the code doesn't use > >> tp->t_tsomax in some locations, and may actually default to what > >> the > >> nic is set to. > >>=20 > >> The NIC may still win, I didn't walk through the code to confirm, > >> it > >> was enough to suggest to me that setting TSO wouldn't fix this > >> issue. > >>=20 > >>=20 > >> However, this is still a TSO related issue, it's just not one > >> related > >> to the setting of TSO's max size. > >>=20 > >> A 10.0-STABLE system with tso disabled on ix0 doesn't have a > >> single > >> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a > >> bit > >> longer to increase confidence in this assertion, but I don't want > >> to > >> waste time on this when I could be logging problem packets on a > >> system with TSO enabled. > >>=20 > >>=20 > >> Comments are very welcome.. > >>=20 > >>=20 > >>=20 > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > >=20 >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 22:11:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C7566FB; Tue, 25 Mar 2014 22:11:22 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0D758C07; Tue, 25 Mar 2014 22:11:21 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:57080 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WSZZ1-00084F-Iv; Tue, 25 Mar 2014 23:11:19 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: <2042344654.506796.1395783960030.JavaMail.root@uoguelph.ca> Date: Tue, 25 Mar 2014 23:11:17 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1573EFCE-EFCF-4ABF-A1A5-77714B56F9F1@hostpoint.ch> References: <2042344654.506796.1395783960030.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:11:22 -0000 On 25.03.2014, at 22:46, Rick Macklem wrote: > Markus Gebert wrote: >>=20 >> On 25.03.2014, at 02:18, Rick Macklem wrote: >>=20 >>> Christopher Forgeron wrote: >>>>=20 >>>>=20 >>>>=20 >>>> This is regarding the TSO patch that Rick suggested earlier. (With >>>> many thanks for his time and suggestion) >>>>=20 >>>>=20 >>>> As I mentioned earlier, it did not fix the issue on a 10.0 system. >>>> It >>>> did make it less of a problem on 9.2, but either way, I think it's >>>> not needed, and shouldn't be considered as a patch for >>>> testing/etc. >>>>=20 >>>>=20 >>>> Patching TSO to anything other than a max value (and by default >>>> the >>>> code gives it IP_MAXPACKET) is confusing the matter, as the packet >>>> length ultimately needs to be adjusted for many things on the fly >>>> like TCP Options, etc. Using static header sizes won't be a good >>>> idea. >>>>=20 >>> If you look at tcp_output(), you'll notice that it doesn't do TSO >>> if >>> there are any options. That way it knows that the TCP/IP header is >>> just hdrlen. >>>=20 >>> If you don't limit the TSO packet (including TCP/IP and ethernet >>> headers) >>> to 64K, then the "ix" driver can't send them, which is the problem >>> you guys are seeing. >>>=20 >>> There are other ways to fix this problem, but they all may >>> introduce >>> issues that reducing if_hw_tsomax by a small amount does not. >>> For example, m_defrag() could be modified to use 4K pagesize >>> clusters, >>> but this might introduce memory fragmentation problems. (I observed >>> what I think are memory fragmentation problems when I switched NFS >>> to use 4K pagesize clusters for large I/O messages.) >>>=20 >>> If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG >>> error replies), then that is the size that if_hw_tsomax can be set >>> to (just can't change IP_MAXPACKET, but that is defined for other >>> things). (It just happens that IP_MAXPACKET is what if_hw_tsomax >>> defaults to. It has no other effect w.r.t. TSO.) >>>=20 >>>>=20 >>>> Additionally, it seems that setting nic TSO will/may be ignored by >>>> code like this in sys/netinet/tcp_output.c: >>>>=20 >>=20 >> Is this confirmed or still a =91it seems=92? Have you actually seen a >> tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was >> this just speculation because the values are stored in different >> places? (Sorry, if you already stated this in another email, it=92s >> currently hard to keep track of all the information.) >>=20 >> Anyway, this dtrace one-liner should be a good test if other values >> appear in tp->t_tsomax: >>=20 >> # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && >> args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: >> %i\n", args[0]->t_tsomax); stack(); }' >>=20 >> Remember to adjust the value in the condition to whatever you=92re >> currently expecting. The value seems to be 0 for new connections, >> probably when tcp_mss() has not been called yet. So that=92s seems >> normal and I have excluded that case too. This will also print a >> kernel stack trace in case it sees an unexpected value. >>=20 >>=20 >>> Yes, but I don't know why. >>> The only conjecture I can come up with is that another net driver >>> is >>> stacked above "ix" and the setting for if_hw_tsomax doesn't >>> propagate >>> up. (If you look at the commit log message for r251296, the intent >>> of adding if_hw_tsomax was to allow device drivers to set a smaller >>> tsomax than IP_MAXPACKET.) >>>=20 >>> Are you using any of the "stacked" network device drivers like >>> lagg? I don't even know what the others all are? >>> Maybe someone else can list them? >>=20 >> I guess the most obvious are lagg and vlan (and probably carp on >> FreeBSD 9.x or older). >>=20 >> On request from Jack, we=92ve eliminated lagg and vlan from the >> picture, which gives us plain ixgbe interfaces with no stacked >> interfaces on top of it. And we can still reproduce the problem. >>=20 > This was related to the "did if_hw_tsomax set tp->t_tsomax to the > same value?" question. Since you reported that my patch that set > if_hw_tsomax in the driver didn't fix the problem, that suggests > that tp->t_tsomax isn't being set to if_hw_tsomax from the driver, > but we don't know why? Jack asked us to remove lagg/vlans in the very beginning of this thread, = and when had done that, the problem was still there. So my answer was = not related to your recent patch. I wanted to clarify that we have been = testing with ixgbe only for quite some time and that stacked interfaces = could not be a source of problems in our test scenario. We have just started testing your patch that sets if_hw_tsomax = yesterday. So far I have it running on two systems along with some = printfs and the dtrace one-liner that watches over tp->t_tsomax in = tcp_output(). So far we=92ve haven=92t had any problems with these two = servers, and the dtrace probe never fired, so far it looks like = tp->t_tsomax always gets set from if_hw_tsomax. But it=92s too soon to = make a conclusion, it may take days to trigger the problem again. It = might also be fixed with your patch. I=92m booting more systems with the test kernel and I will be watching = all of them with dtrace to see I i find an occurence where tp->t_tsomax = is off. I hope that with more systems, I=92ll have an answer more = quickly. But digging around the code, I still don=92t see a way how tp->tsomax = could not have been set from if_hw_tsomax when there are no stacked = interfaces=85 Markus > rick >=20 >>=20 >> Markus >>=20 >>=20 >>>=20 >>> rick >>>>=20 >>>> 10.0 Code: >>>>=20 >>>> 780 if (len > tp->t_tsomax - hdrlen) { !! >>>> 781 len =3D tp->t_tsomax - hdrlen; !! >>>> 782 sendalot =3D 1; >>>> 783 } >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I've put debugging here, set the nic's max TSO as per Rick's patch >>>> ( >>>> set to say 32k), and have seen that tp->t_tsomax =3D=3D = IP_MAXPACKET. >>>> It's being set someplace else, and thus our attempts to set TSO on >>>> the nic may be in vain. >>>>=20 >>>>=20 >>>> It may have mattered more in 9.2, as I see the code doesn't use >>>> tp->t_tsomax in some locations, and may actually default to what >>>> the >>>> nic is set to. >>>>=20 >>>> The NIC may still win, I didn't walk through the code to confirm, >>>> it >>>> was enough to suggest to me that setting TSO wouldn't fix this >>>> issue. >>>>=20 >>>>=20 >>>> However, this is still a TSO related issue, it's just not one >>>> related >>>> to the setting of TSO's max size. >>>>=20 >>>> A 10.0-STABLE system with tso disabled on ix0 doesn't have a >>>> single >>>> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a >>>> bit >>>> longer to increase confidence in this assertion, but I don't want >>>> to >>>> waste time on this when I could be logging problem packets on a >>>> system with TSO enabled. >>>>=20 >>>>=20 >>>> Comments are very welcome.. >>>>=20 >>>>=20 >>>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to >>> "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 22:21:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51FA3980; Tue, 25 Mar 2014 22:21:59 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id CBF6ACD7; Tue, 25 Mar 2014 22:21:58 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAA8BMlODaFve/2dsb2JhbABZg0FXgwe4MIZkUYEzdIIlAQEBAwEBAQEgBCcgCwUWGAICDRkCKQEJJgYIBwQBHAEDh1AIDa1LoiUXgSmMYwsFAgEbATMHgi5BgUkElXaECZEAg0ohMXxB X-IronPort-AV: E=Sophos;i="4.97,730,1389762000"; d="scan'208";a="109093910" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 25 Mar 2014 18:21:50 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3E048B4059; Tue, 25 Mar 2014 18:21:50 -0400 (EDT) Date: Tue, 25 Mar 2014 18:21:50 -0400 (EDT) From: Rick Macklem To: Markus Gebert Message-ID: <1212254686.521736.1395786110243.JavaMail.root@uoguelph.ca> In-Reply-To: <1573EFCE-EFCF-4ABF-A1A5-77714B56F9F1@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:21:59 -0000 Markus Gebert wrote: >=20 > On 25.03.2014, at 22:46, Rick Macklem wrote: >=20 > > Markus Gebert wrote: > >>=20 > >> On 25.03.2014, at 02:18, Rick Macklem > >> wrote: > >>=20 > >>> Christopher Forgeron wrote: > >>>>=20 > >>>>=20 > >>>>=20 > >>>> This is regarding the TSO patch that Rick suggested earlier. > >>>> (With > >>>> many thanks for his time and suggestion) > >>>>=20 > >>>>=20 > >>>> As I mentioned earlier, it did not fix the issue on a 10.0 > >>>> system. > >>>> It > >>>> did make it less of a problem on 9.2, but either way, I think > >>>> it's > >>>> not needed, and shouldn't be considered as a patch for > >>>> testing/etc. > >>>>=20 > >>>>=20 > >>>> Patching TSO to anything other than a max value (and by default > >>>> the > >>>> code gives it IP_MAXPACKET) is confusing the matter, as the > >>>> packet > >>>> length ultimately needs to be adjusted for many things on the > >>>> fly > >>>> like TCP Options, etc. Using static header sizes won't be a good > >>>> idea. > >>>>=20 > >>> If you look at tcp_output(), you'll notice that it doesn't do TSO > >>> if > >>> there are any options. That way it knows that the TCP/IP header > >>> is > >>> just hdrlen. > >>>=20 > >>> If you don't limit the TSO packet (including TCP/IP and ethernet > >>> headers) > >>> to 64K, then the "ix" driver can't send them, which is the > >>> problem > >>> you guys are seeing. > >>>=20 > >>> There are other ways to fix this problem, but they all may > >>> introduce > >>> issues that reducing if_hw_tsomax by a small amount does not. > >>> For example, m_defrag() could be modified to use 4K pagesize > >>> clusters, > >>> but this might introduce memory fragmentation problems. (I > >>> observed > >>> what I think are memory fragmentation problems when I switched > >>> NFS > >>> to use 4K pagesize clusters for large I/O messages.) > >>>=20 > >>> If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG > >>> error replies), then that is the size that if_hw_tsomax can be > >>> set > >>> to (just can't change IP_MAXPACKET, but that is defined for other > >>> things). (It just happens that IP_MAXPACKET is what if_hw_tsomax > >>> defaults to. It has no other effect w.r.t. TSO.) > >>>=20 > >>>>=20 > >>>> Additionally, it seems that setting nic TSO will/may be ignored > >>>> by > >>>> code like this in sys/netinet/tcp_output.c: > >>>>=20 > >>=20 > >> Is this confirmed or still a =E2=80=98it seems=E2=80=99? Have you actu= ally seen a > >> tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was > >> this just speculation because the values are stored in different > >> places? (Sorry, if you already stated this in another email, it=E2=80= =99s > >> currently hard to keep track of all the information.) > >>=20 > >> Anyway, this dtrace one-liner should be a good test if other > >> values > >> appear in tp->t_tsomax: > >>=20 > >> # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && > >> args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: > >> %i\n", args[0]->t_tsomax); stack(); }' > >>=20 > >> Remember to adjust the value in the condition to whatever you=E2=80=99= re > >> currently expecting. The value seems to be 0 for new connections, > >> probably when tcp_mss() has not been called yet. So that=E2=80=99s see= ms > >> normal and I have excluded that case too. This will also print a > >> kernel stack trace in case it sees an unexpected value. > >>=20 > >>=20 > >>> Yes, but I don't know why. > >>> The only conjecture I can come up with is that another net driver > >>> is > >>> stacked above "ix" and the setting for if_hw_tsomax doesn't > >>> propagate > >>> up. (If you look at the commit log message for r251296, the > >>> intent > >>> of adding if_hw_tsomax was to allow device drivers to set a > >>> smaller > >>> tsomax than IP_MAXPACKET.) > >>>=20 > >>> Are you using any of the "stacked" network device drivers like > >>> lagg? I don't even know what the others all are? > >>> Maybe someone else can list them? > >>=20 > >> I guess the most obvious are lagg and vlan (and probably carp on > >> FreeBSD 9.x or older). > >>=20 > >> On request from Jack, we=E2=80=99ve eliminated lagg and vlan from the > >> picture, which gives us plain ixgbe interfaces with no stacked > >> interfaces on top of it. And we can still reproduce the problem. > >>=20 > > This was related to the "did if_hw_tsomax set tp->t_tsomax to the > > same value?" question. Since you reported that my patch that set > > if_hw_tsomax in the driver didn't fix the problem, that suggests > > that tp->t_tsomax isn't being set to if_hw_tsomax from the driver, > > but we don't know why? >=20 > Jack asked us to remove lagg/vlans in the very beginning of this > thread, and when had done that, the problem was still there. So my > answer was not related to your recent patch. I wanted to clarify > that we have been testing with ixgbe only for quite some time and > that stacked interfaces could not be a source of problems in our > test scenario. >=20 > We have just started testing your patch that sets if_hw_tsomax > yesterday. So far I have it running on two systems along with some > printfs and the dtrace one-liner that watches over tp->t_tsomax in > tcp_output(). So far we=E2=80=99ve haven=E2=80=99t had any problems with = these two > servers, and the dtrace probe never fired, so far it looks like > tp->t_tsomax always gets set from if_hw_tsomax. But it=E2=80=99s too soon= to > make a conclusion, it may take days to trigger the problem again. It > might also be fixed with your patch. >=20 Righto. Setting if_hw_tsomax in the driver is supposed to set tp->t_tsomax and I could see it work in a trivial test (I hacked the code so the assignm= ents are done for the non-tso case and it worked for the non-tso "re" driver I r= un.) { As an aside, one of these assignments does happen for non-tso cases, sinc= e although it is indented, there are no {} for the block. In tcp_subr.c if = I recall. However, doing the assignment for the non-tso case seems harmless= to me. } > I=E2=80=99m booting more systems with the test kernel and I will be watch= ing > all of them with dtrace to see I i find an occurence where > tp->t_tsomax is off. I hope that with more systems, I=E2=80=99ll have an > answer more quickly. >=20 > But digging around the code, I still don=E2=80=99t see a way how tp->tsom= ax > could not have been set from if_hw_tsomax when there are no stacked > interfaces=E2=80=A6 >=20 It seems to happen where you mentioned before. Since it only gets set from cap.tsomax and that gets set from if_hw_tsomax, it would be 0 otherwise. Christopher sees in change when he changes IP_MAXPACET, so the default setting works, but for him setting it in the driver didn't, for some reason? Thanks for doing the testing, rick >=20 > Markus >=20 >=20 > > rick > >=20 > >>=20 > >> Markus > >>=20 > >>=20 > >>>=20 > >>> rick > >>>>=20 > >>>> 10.0 Code: > >>>>=20 > >>>> 780 if (len > tp->t_tsomax - hdrlen) { !! > >>>> 781 len =3D tp->t_tsomax - hdrlen; !! > >>>> 782 sendalot =3D 1; > >>>> 783 } > >>>>=20 > >>>>=20 > >>>>=20 > >>>>=20 > >>>> I've put debugging here, set the nic's max TSO as per Rick's > >>>> patch > >>>> ( > >>>> set to say 32k), and have seen that tp->t_tsomax =3D=3D > >>>> IP_MAXPACKET. > >>>> It's being set someplace else, and thus our attempts to set TSO > >>>> on > >>>> the nic may be in vain. > >>>>=20 > >>>>=20 > >>>> It may have mattered more in 9.2, as I see the code doesn't use > >>>> tp->t_tsomax in some locations, and may actually default to what > >>>> the > >>>> nic is set to. > >>>>=20 > >>>> The NIC may still win, I didn't walk through the code to > >>>> confirm, > >>>> it > >>>> was enough to suggest to me that setting TSO wouldn't fix this > >>>> issue. > >>>>=20 > >>>>=20 > >>>> However, this is still a TSO related issue, it's just not one > >>>> related > >>>> to the setting of TSO's max size. > >>>>=20 > >>>> A 10.0-STABLE system with tso disabled on ix0 doesn't have a > >>>> single > >>>> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a > >>>> bit > >>>> longer to increase confidence in this assertion, but I don't > >>>> want > >>>> to > >>>> waste time on this when I could be logging problem packets on a > >>>> system with TSO enabled. > >>>>=20 > >>>>=20 > >>>> Comments are very welcome.. > >>>>=20 > >>>>=20 > >>>>=20 > >>> _______________________________________________ > >>> freebsd-net@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>> To unsubscribe, send any mail to > >>> "freebsd-net-unsubscribe@freebsd.org" > >>>=20 > >>=20 > >>=20 > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:06:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D59AD1D; Tue, 25 Mar 2014 23:06:44 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23B4112E; Tue, 25 Mar 2014 23:06:44 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so1660938qcr.39 for ; Tue, 25 Mar 2014 16:06:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zoD0KE9aAbgbUW0clh2h+FVgn/h46nQ2aDImti0DVuw=; b=O6mqLGY7yvA6X1lRZd+lvZF1aZAQGQoxq9fPS688PZysbDrltRY7TKbLp1qgtYoPvP NeFa3/NyCirNNviqei5/4TTj4fBzj9x72mvEXZHmMvlBrSRQ0kj34xZ8tgKZXwEcZHDp h7C/eeFTpa26jk0CJmWjGEhJE1xZvwfsf6YvA+zyTtf9sO3kj3xrvf/LYf9AbJ1z7Tld xBxBd9Ui81Ff1K2RmWwBIIRuvns96ViVRPh8dV6nab0sxVyQBLe+Pj95IyLC0ll1eMjx CKBjEVyubO65Sd0qXzHXU14dRxZhvOay5AHh6vYWKlP2OF8g1lmAsiRQsksTIr5g/cDS pZtw== MIME-Version: 1.0 X-Received: by 10.140.50.231 with SMTP id s94mr26017545qga.33.1395788803361; Tue, 25 Mar 2014 16:06:43 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 16:06:43 -0700 (PDT) In-Reply-To: References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> Date: Tue, 25 Mar 2014 20:06:43 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:06:44 -0000 Update: I'm changing my mind, and I believe Rick's TSO patch is fixing things (sorry). In looking at my notes, it's possible I had lagg on for those tests. lagg does seem to negate the TSO patch in my case. kernel.10stable_basicTSO_65535/ - IP_MAXPACKET = 65535; - manually forced (no if statement) ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); - Verified on boot via printf that ifp->if_hw_tsomax = 65517 - Boot in a NON LAGG environment. ix0 only. ixgbe's printf is showing packets up to 65530. Haven't run long enough yet to see if anything will go over 65535 I have this tcpdump running to check packet size. tcpdump -ennvvXS -i ix0 greater 65518 I do expect to get packets over 65518, but I was just curious to see if any of them would go over 65535. Time will tell. In a separate test, If I enable lagg, we have LOTS of oversized packet problems. It looks like tsomax is definitely not making it through in if_lagg.c - Any recommendations there? I will eventually need lagg, as I'm sure will others. With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not be happening? dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); }' 6 31403 tcp_output:entry unexpected tp->t_tsomax: 65535 kernel`tcp_do_segment+0x2c99 kernel`tcp_input+0x11a2 kernel`ip_input+0xa2 kernel`netisr_dispatch_src+0x5e kernel`ether_demux+0x12a kernel`ether_nh_input+0x35f kernel`netisr_dispatch_src+0x5e kernel`bce_intr+0x765 kernel`intr_event_execute_handlers+0xab kernel`ithread_loop+0x96 kernel`fork_exit+0x9a kernel`0xffffffff80c75b2e 3 31403 tcp_output:entry unexpected tp->t_tsomax: 65535 kernel`tcp_do_segment+0x2c99 kernel`tcp_input+0x11a2 kernel`ip_input+0xa2 kernel`netisr_dispatch_src+0x5e kernel`ether_demux+0x12a kernel`ether_nh_input+0x35f kernel`netisr_dispatch_src+0x5e kernel`bce_intr+0x765 kernel`intr_event_execute_handlers+0xab kernel`ithread_loop+0x96 kernel`fork_exit+0x9a kernel`0xffffffff80c75b2e 6 31403 tcp_output:entry unexpected tp->t_tsomax: 65535 kernel`tcp_do_segment+0x2c99 kernel`tcp_input+0x11a2 kernel`ip_input+0xa2 kernel`netisr_dispatch_src+0x5e kernel`ether_demux+0x12a kernel`ether_nh_input+0x35f kernel`netisr_dispatch_src+0x5e kernel`bce_intr+0x765 kernel`intr_event_execute_handlers+0xab kernel`ithread_loop+0x96 kernel`fork_exit+0x9a kernel`0xffffffff80c75b2e 1 31403 tcp_output:entry unexpected tp->t_tsomax: 65535 kernel`tcp_do_segment+0x2c99 kernel`tcp_input+0x11a2 kernel`ip_input+0xa2 kernel`netisr_dispatch_src+0x5e kernel`ether_demux+0x12a kernel`ether_nh_input+0x35f kernel`netisr_dispatch_src+0x5e kernel`bce_intr+0x765 kernel`intr_event_execute_handlers+0xab kernel`ithread_loop+0x96 kernel`fork_exit+0x9a kernel`0xffffffff80c75b2e From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:07:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41F90DAE; Tue, 25 Mar 2014 23:07:23 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BCA65137; Tue, 25 Mar 2014 23:07:22 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:57010 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WSaRE-000Aju-9y; Wed, 26 Mar 2014 00:07:20 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: <1212254686.521736.1395786110243.JavaMail.root@uoguelph.ca> Date: Wed, 26 Mar 2014 00:07:18 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2C869AD2-FC04-4E0B-9CBC-02FFA9AEC0EA@hostpoint.ch> References: <1212254686.521736.1395786110243.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:07:23 -0000 On 25.03.2014, at 23:21, Rick Macklem wrote: > Markus Gebert wrote: >>=20 >> On 25.03.2014, at 22:46, Rick Macklem wrote: >>=20 >>> Markus Gebert wrote: >>>>=20 >>>> On 25.03.2014, at 02:18, Rick Macklem >>>> wrote: >>>>=20 >>>>> Christopher Forgeron wrote: >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> This is regarding the TSO patch that Rick suggested earlier. >>>>>> (With >>>>>> many thanks for his time and suggestion) >>>>>>=20 >>>>>>=20 >>>>>> As I mentioned earlier, it did not fix the issue on a 10.0 >>>>>> system. >>>>>> It >>>>>> did make it less of a problem on 9.2, but either way, I think >>>>>> it's >>>>>> not needed, and shouldn't be considered as a patch for >>>>>> testing/etc. >>>>>>=20 >>>>>>=20 >>>>>> Patching TSO to anything other than a max value (and by default >>>>>> the >>>>>> code gives it IP_MAXPACKET) is confusing the matter, as the >>>>>> packet >>>>>> length ultimately needs to be adjusted for many things on the >>>>>> fly >>>>>> like TCP Options, etc. Using static header sizes won't be a good >>>>>> idea. >>>>>>=20 >>>>> If you look at tcp_output(), you'll notice that it doesn't do TSO >>>>> if >>>>> there are any options. That way it knows that the TCP/IP header >>>>> is >>>>> just hdrlen. >>>>>=20 >>>>> If you don't limit the TSO packet (including TCP/IP and ethernet >>>>> headers) >>>>> to 64K, then the "ix" driver can't send them, which is the >>>>> problem >>>>> you guys are seeing. >>>>>=20 >>>>> There are other ways to fix this problem, but they all may >>>>> introduce >>>>> issues that reducing if_hw_tsomax by a small amount does not. >>>>> For example, m_defrag() could be modified to use 4K pagesize >>>>> clusters, >>>>> but this might introduce memory fragmentation problems. (I >>>>> observed >>>>> what I think are memory fragmentation problems when I switched >>>>> NFS >>>>> to use 4K pagesize clusters for large I/O messages.) >>>>>=20 >>>>> If setting IP_MAXPACKET to 65518 fixes the problem (no more EFBIG >>>>> error replies), then that is the size that if_hw_tsomax can be >>>>> set >>>>> to (just can't change IP_MAXPACKET, but that is defined for other >>>>> things). (It just happens that IP_MAXPACKET is what if_hw_tsomax >>>>> defaults to. It has no other effect w.r.t. TSO.) >>>>>=20 >>>>>>=20 >>>>>> Additionally, it seems that setting nic TSO will/may be ignored >>>>>> by >>>>>> code like this in sys/netinet/tcp_output.c: >>>>>>=20 >>>>=20 >>>> Is this confirmed or still a =91it seems=92? Have you actually seen = a >>>> tp->t_tsomax value in tcp_output() bigger than if_hw_tsomax or was >>>> this just speculation because the values are stored in different >>>> places? (Sorry, if you already stated this in another email, it=92s >>>> currently hard to keep track of all the information.) >>>>=20 >>>> Anyway, this dtrace one-liner should be a good test if other >>>> values >>>> appear in tp->t_tsomax: >>>>=20 >>>> # dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && >>>> args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: >>>> %i\n", args[0]->t_tsomax); stack(); }' >>>>=20 >>>> Remember to adjust the value in the condition to whatever you=92re >>>> currently expecting. The value seems to be 0 for new connections, >>>> probably when tcp_mss() has not been called yet. So that=92s seems >>>> normal and I have excluded that case too. This will also print a >>>> kernel stack trace in case it sees an unexpected value. >>>>=20 >>>>=20 >>>>> Yes, but I don't know why. >>>>> The only conjecture I can come up with is that another net driver >>>>> is >>>>> stacked above "ix" and the setting for if_hw_tsomax doesn't >>>>> propagate >>>>> up. (If you look at the commit log message for r251296, the >>>>> intent >>>>> of adding if_hw_tsomax was to allow device drivers to set a >>>>> smaller >>>>> tsomax than IP_MAXPACKET.) >>>>>=20 >>>>> Are you using any of the "stacked" network device drivers like >>>>> lagg? I don't even know what the others all are? >>>>> Maybe someone else can list them? >>>>=20 >>>> I guess the most obvious are lagg and vlan (and probably carp on >>>> FreeBSD 9.x or older). >>>>=20 >>>> On request from Jack, we=92ve eliminated lagg and vlan from the >>>> picture, which gives us plain ixgbe interfaces with no stacked >>>> interfaces on top of it. And we can still reproduce the problem. >>>>=20 >>> This was related to the "did if_hw_tsomax set tp->t_tsomax to the >>> same value?" question. Since you reported that my patch that set >>> if_hw_tsomax in the driver didn't fix the problem, that suggests >>> that tp->t_tsomax isn't being set to if_hw_tsomax from the driver, >>> but we don't know why? >>=20 >> Jack asked us to remove lagg/vlans in the very beginning of this >> thread, and when had done that, the problem was still there. So my >> answer was not related to your recent patch. I wanted to clarify >> that we have been testing with ixgbe only for quite some time and >> that stacked interfaces could not be a source of problems in our >> test scenario. >>=20 >> We have just started testing your patch that sets if_hw_tsomax >> yesterday. So far I have it running on two systems along with some >> printfs and the dtrace one-liner that watches over tp->t_tsomax in >> tcp_output(). So far we=92ve haven=92t had any problems with these = two >> servers, and the dtrace probe never fired, so far it looks like >> tp->t_tsomax always gets set from if_hw_tsomax. But it=92s too soon = to >> make a conclusion, it may take days to trigger the problem again. It >> might also be fixed with your patch. >>=20 > Righto. Setting if_hw_tsomax in the driver is supposed to set = tp->t_tsomax > and I could see it work in a trivial test (I hacked the code so the = assignments > are done for the non-tso case and it worked for the non-tso "re" = driver I run.) > { As an aside, one of these assignments does happen for non-tso cases, = since > although it is indented, there are no {} for the block. In tcp_subr.c = if I > recall. However, doing the assignment for the non-tso case seems = harmless to me. } >=20 >> I=92m booting more systems with the test kernel and I will be = watching >> all of them with dtrace to see I i find an occurence where >> tp->t_tsomax is off. I hope that with more systems, I=92ll have an >> answer more quickly. >>=20 >> But digging around the code, I still don=92t see a way how tp->tsomax >> could not have been set from if_hw_tsomax when there are no stacked >> interfaces=85 >>=20 > It seems to happen where you mentioned before. Since it only gets set > from cap.tsomax and that gets set from if_hw_tsomax, it would be 0 > otherwise. Sorry, my sentence was probably a bit misleading. What you=92re saying = is what I meant. There=92s the tcp_mss() -> tcp_mss_update() -> = tcp_maxmtu() call chain that ultimately sets tp->t_tsomax from = if_hw_tsomax (via the cap struct). tp->t_tsomax is indeed 0 on fresh = connections when tcp_mss() has not been called yet, I could confirm that = with dtrace. As soon as the connection gets running, it=92s set to = whatever the interface=92s if_hw_tsomax is. What I have _not_ found is another place that alters tp->t_tsomax, so I = really don=92t get how Christopher can see different values for = tp->t_tsomax. > Christopher sees in change when he changes IP_MAXPACET, so > the default setting works, but for him setting it in the driver = didn't, > for some reason? Christopher, can you run your tests again with default IP_MAXPACKET and = just Rick's if_hw_maxtso patch? I think it=92s important to confirm that = tp->t_tsomax is really off in that case, which you can easily test with = my dtrace one-liner. I=92m running this test too, but I will take me = much more time until I can make a statement. > Thanks for doing the testing, rick No problem. Thank you guys! >>=20 >> Markus >>=20 >>=20 >>> rick >>>=20 >>>>=20 >>>> Markus >>>>=20 >>>>=20 >>>>>=20 >>>>> rick >>>>>>=20 >>>>>> 10.0 Code: >>>>>>=20 >>>>>> 780 if (len > tp->t_tsomax - hdrlen) { !! >>>>>> 781 len =3D tp->t_tsomax - hdrlen; !! >>>>>> 782 sendalot =3D 1; >>>>>> 783 } >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> I've put debugging here, set the nic's max TSO as per Rick's >>>>>> patch >>>>>> ( >>>>>> set to say 32k), and have seen that tp->t_tsomax =3D=3D >>>>>> IP_MAXPACKET. >>>>>> It's being set someplace else, and thus our attempts to set TSO >>>>>> on >>>>>> the nic may be in vain. >>>>>>=20 >>>>>>=20 >>>>>> It may have mattered more in 9.2, as I see the code doesn't use >>>>>> tp->t_tsomax in some locations, and may actually default to what >>>>>> the >>>>>> nic is set to. >>>>>>=20 >>>>>> The NIC may still win, I didn't walk through the code to >>>>>> confirm, >>>>>> it >>>>>> was enough to suggest to me that setting TSO wouldn't fix this >>>>>> issue. >>>>>>=20 >>>>>>=20 >>>>>> However, this is still a TSO related issue, it's just not one >>>>>> related >>>>>> to the setting of TSO's max size. >>>>>>=20 >>>>>> A 10.0-STABLE system with tso disabled on ix0 doesn't have a >>>>>> single >>>>>> packet over IP_MAXPACKET in 1 hour of runtime. I'll let it go a >>>>>> bit >>>>>> longer to increase confidence in this assertion, but I don't >>>>>> want >>>>>> to >>>>>> waste time on this when I could be logging problem packets on a >>>>>> system with TSO enabled. >>>>>>=20 >>>>>>=20 >>>>>> Comments are very welcome.. >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to >>>>> "freebsd-net-unsubscribe@freebsd.org" >>>>>=20 >>>>=20 >>>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to >>> "freebsd-net-unsubscribe@freebsd.org" >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:10:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B689AF1C; Tue, 25 Mar 2014 23:10:36 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 447F915D; Tue, 25 Mar 2014 23:10:36 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAKwLMlODaFve/2dsb2JhbABZg0FXgwe+R4EegTN0gk8ERws1Ag0ZAl8BLYdeDa02oiEXgSmMbQEjNIJ2gUkElF8HhRmRAINKIYEsAR8i X-IronPort-AV: E=Sophos;i="4.97,730,1389762000"; d="scan'208";a="109106594" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 25 Mar 2014 19:10:35 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 54716B405A; Tue, 25 Mar 2014 19:10:35 -0400 (EDT) Date: Tue, 25 Mar 2014 19:10:35 -0400 (EDT) From: Rick Macklem To: FreeBSD Filesystems , FreeBSD Net Message-ID: <1609686124.539328.1395789035334.JavaMail.root@uoguelph.ca> Subject: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Alexander Motin , Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:10:36 -0000 Hi, First off, I hope you don't mind that I cross-posted this, but I wanted to make sure both the NFS/iSCSI and networking types say it. If you look in this mailing list thread: http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root you'll see that several people have been working hard at testing and thanks to them, I think I now know what is going on. (This applies to network drivers that support TSO and are limited to 32 transmit segments->32 mbufs in chain.) Doing a quick search I found the following drivers that appear to be affected (I may have missed some): jme, fxp, age, sge, msk, alc, ale, ixgbe/ix, nfe, e1000/em, re Further, of these drivers, the following use m_collapse() and not m_defrag() to try and reduce the # of mbufs in the chain. m_collapse() is not going to get the 35 mbufs down to 32 mbufs, as far as I can see, so these ones are more badly broken: jme, fxp, age, sge, alc, ale, nfe, re The long description is in the above thread, but the short version is: - NFS generates a chain with 35 mbufs in it for (read/readdir replies and write requests) made up of (tcpip header, RPC header, NFS args, 32 clusters of file data) - tcp_output() usually trims the data size down to tp->t_tsomax (65535) and then some more to make it an exact multiple of TCP transmit data size. - the net driver prepends an ethernet header, growing the length by 14 (or sometimes 18 for vlans), but in the first mbuf and not adding one to the chain. - m_defrag() copies this to a chain of 32 mbuf clusters (because the total data length is <= 64K) and it gets sent However, it the data length is a little less than 64K when passed to tcp_output() so that the length including headers is in the range 65519->65535... - tcp_output() doesn't reduce its size. - the net driver adds an ethernet header, making the total data length slightly greater than 64K - m_defrag() copies it to a chain of 33 mbuf clusters, which fails with EFBIG --> trainwrecks NFS performance, because the TSO segment is dropped instead of sent. A tester also stated that the problem could be reproduced using iSCSI. Maybe Edward Napierala might know some details w.r.t. what kind of mbuf chain iSCSI generates? Also, one tester has reported that setting if_hw_tsomax in the driver before the ether_ifattach() call didn't make the value of tp->t_tsomax smaller. However, reducing IP_MAXPACKET (which is what it is set to by default) did reduce it. I have no idea why this happens or how to fix it, but it implies that setting if_hw_tsomax in the driver isn't a solution until this is resolved. So, what to do about this? First, I'd like a simple fix/workaround that can go into 9.3. (which is code freeze in May). The best thing I can think of is setting if_hw_tsomax to a smaller default value. (Line# 658 of sys/net/if.c in head.) Version A: replace ifp->if_hw_tsomax = IP_MAXPACKET; with ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN), IP_MAXPACKET); plus replace m_collapse() with m_defrag() in the drivers listed above. This would only reduce the default from 65535->65518, so it only impacts the uncommon case where the output size (with tcpip header) is within this range. (As such, I don't think it would have a negative impact for drivers that handle more than 32 transmit segments.) >From the testers, it seems that this is sufficient to get rid of the EFBIG errors. (The total data length including ethernet header doesn't exceed 64K, so m_defrag() fits it into 32 mbuf clusters.) The main downside of this is that there will be a lot of m_defrag() calls being done and they do quite a bit of bcopy()'ng. Version B: replace ifp->if_hw_tsomax = IP_MAXPACKET; with ifp->if_hw_tsomax = min(29 * MCLBYTES, IP_MAXPACKET); This one would avoid the m_defrag() calls, but might have a negative impact on TSO performance for drivers that can handle 35 transmit segments, since the maximum TSO segment size is reduced by about 6K. (Because of the second size reduction to an exact multiple of TCP transmit data size, the exact amount varies.) Possible longer term fixes: One longer term fix might be to add something like if_hw_tsomaxseg so that a driver can set a limit on the number of transmit segments (mbufs in chain) and tcp_output() could use that to limit the size of the TSO segment, as required. (I have a first stab at such a patch, but no way to test it, so I can't see that being done by May. Also, it would require changes to a lot of drivers to make it work. I've attached this patch, in case anyone wants to work on it?) Another might be to increase the size of MCLBYTES (I don't see this as practical for 9.3, although the actual change is simple.). I do think that increasing MCLBYTES might be something to consider doing in the future, for reasons beyond fixing this. So, what do others think should be done? rick From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:16:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B2E52C0; Tue, 25 Mar 2014 23:16:42 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DCA3F20B; Tue, 25 Mar 2014 23:16:41 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAAAOMlODaFve/2dsb2JhbABZg0FXgwe4MYZkUYEzdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1AIDa01oiIXgSmMdAEBGzQHgm+BSQSVdoQJkQCDSiExgQQ5 X-IronPort-AV: E=Sophos;i="4.97,730,1389762000"; d="scan'208";a="108851797" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Mar 2014 19:16:29 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id AC2BCB3F17; Tue, 25 Mar 2014 19:16:29 -0400 (EDT) Date: Tue, 25 Mar 2014 19:16:29 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Andre Oppermann , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:16:42 -0000 Christopher Forgeron wrote: > Update: > > I'm changing my mind, and I believe Rick's TSO patch is fixing > things > (sorry). In looking at my notes, it's possible I had lagg on for > those > tests. lagg does seem to negate the TSO patch in my case. > Ok, that's useful information. It implies that r251296 doesn't quite work and needs to be fixed for "stacked" network interface drivers before it can be used. I've cc'd Andre who is the author of that patch, in case he knows how to fix it. Thanks for checking this, rick > kernel.10stable_basicTSO_65535/ > > - IP_MAXPACKET = 65535; > - manually forced (no if statement) ifp->if_hw_tsomax = IP_MAXPACKET > - > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > - Verified on boot via printf that ifp->if_hw_tsomax = 65517 > - Boot in a NON LAGG environment. ix0 only. > > ixgbe's printf is showing packets up to 65530. Haven't run long > enough yet > to see if anything will go over 65535 > > I have this tcpdump running to check packet size. > tcpdump -ennvvXS -i ix0 greater 65518 > > I do expect to get packets over 65518, but I was just curious to see > if any > of them would go over 65535. Time will tell. > > In a separate test, If I enable lagg, we have LOTS of oversized > packet > problems. It looks like tsomax is definitely not making it through in > if_lagg.c - Any recommendations there? I will eventually need lagg, > as I'm > sure will others. > > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not be > happening? > > > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax: > %i\n", > args[0]->t_tsomax); stack(); }' > > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > 65535 > > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > > 3 31403 tcp_output:entry unexpected tp->t_tsomax: > 65535 > > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > 65535 > > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > > 1 31403 tcp_output:entry unexpected tp->t_tsomax: > 65535 > > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:21:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35CF04CD; Tue, 25 Mar 2014 23:21:45 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA0CC23A; Tue, 25 Mar 2014 23:21:44 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:55203 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WSaf6-000BLB-I7; Wed, 26 Mar 2014 00:21:40 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Wed, 26 Mar 2014 00:21:38 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:21:45 -0000 On 26.03.2014, at 00:06, Christopher Forgeron = wrote: > Update: >=20 > I'm changing my mind, and I believe Rick's TSO patch is fixing things > (sorry). In looking at my notes, it's possible I had lagg on for those > tests. lagg does seem to negate the TSO patch in my case. I=92m glad to hear you could check that scenario again. In the other = email I just sent, I just asked you to redo this test. Now it makes = perfect sense why you saw oversized packets despite Rick=92s = if_hw_tsomax patch. > kernel.10stable_basicTSO_65535/ >=20 > - IP_MAXPACKET =3D 65535; > - manually forced (no if statement) ifp->if_hw_tsomax =3D IP_MAXPACKET = - > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > - Verified on boot via printf that ifp->if_hw_tsomax =3D 65517 Is 65517 correct? With Ricks patch, I get this: dev.ix.0.hw_tsomax: 65518 Also the dtrace command you used excludes 65518... > - Boot in a NON LAGG environment. ix0 only. >=20 > ixgbe's printf is showing packets up to 65530. Haven't run long enough = yet > to see if anything will go over 65535 >=20 > I have this tcpdump running to check packet size. > tcpdump -ennvvXS -i ix0 greater 65518 >=20 > I do expect to get packets over 65518, but I was just curious to see = if any > of them would go over 65535. Time will tell. >=20 > In a separate test, If I enable lagg, we have LOTS of oversized packet > problems. It looks like tsomax is definitely not making it through in > if_lagg.c - Any recommendations there? I will eventually need lagg, as = I'm > sure will others. I think somebody has to invent a way to propagate if_hw_maxtso to = interfaces on top of each other. > With dtrace, it's showing t_tsomax >=3D 65518. Shouldn't that not be > happening? Looks like these all come from bce interfaces (bce_intr in the stack = trace), which probably have another value for if_hw_tsomax. Markus > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && > args[0]->t_tsomax >=3D 65518 / { printf("unexpected tp->t_tsomax: = %i\n", > args[0]->t_tsomax); stack(); }' >=20 >=20 > 6 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 3 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 6 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 1 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:05:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94073A33; Wed, 26 Mar 2014 02:05:03 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 18C7F6CF; Wed, 26 Mar 2014 02:05:02 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEALU0MlODaFve/2dsb2JhbABZg0FXgwu4OoZkUYEwdIIlAQEBAwEBAQEgBCcgCxsYAgINGQIjBgEJJgYIBwQBHASHRAMJCA2tNpp1DYcdF4EpiymBOhACARsBMweCb4FJBJV2aoMfizaFSoNKITGBPQ X-IronPort-AV: E=Sophos;i="4.97,732,1389762000"; d="scan'208";a="109152802" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 25 Mar 2014 22:04:55 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 328E6B4052; Tue, 25 Mar 2014 22:04:55 -0400 (EDT) Date: Tue, 25 Mar 2014 22:04:55 -0400 (EDT) From: Rick Macklem To: Markus Gebert Message-ID: <1175880198.604234.1395799495187.JavaMail.root@uoguelph.ca> In-Reply-To: <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:05:03 -0000 Markus Gebert wrote: >=20 > On 26.03.2014, at 00:06, Christopher Forgeron > wrote: >=20 > > Update: > >=20 > > I'm changing my mind, and I believe Rick's TSO patch is fixing > > things > > (sorry). In looking at my notes, it's possible I had lagg on for > > those > > tests. lagg does seem to negate the TSO patch in my case. >=20 > I=E2=80=99m glad to hear you could check that scenario again. In the othe= r > email I just sent, I just asked you to redo this test. Now it makes > perfect sense why you saw oversized packets despite Rick=E2=80=99s > if_hw_tsomax patch. >=20 >=20 > > kernel.10stable_basicTSO_65535/ > >=20 > > - IP_MAXPACKET =3D 65535; > > - manually forced (no if statement) ifp->if_hw_tsomax =3D > > IP_MAXPACKET - > > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > > - Verified on boot via printf that ifp->if_hw_tsomax =3D 65517 >=20 > Is 65517 correct? With Ricks patch, I get this: >=20 > dev.ix.0.hw_tsomax: 65518 >=20 > Also the dtrace command you used excludes 65518... >=20 I am using 32 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN) which is 65518. Although IP_MAXPACKET (maximum IP len, not including ethernet hea= der) is 65535 (largest # that fits in 16bits), the maximum data length (including ethernet header) that will fit in 32 mbuf clusters is 65536. (In practice 65517 or anything <=3D 65518 should fix the problem.) rick > > - Boot in a NON LAGG environment. ix0 only. > >=20 > > ixgbe's printf is showing packets up to 65530. Haven't run long > > enough yet > > to see if anything will go over 65535 > >=20 With the ethernet header length, it can be <=3D 65536, because that is 32 * MCLBYTES. rick > > I have this tcpdump running to check packet size. > > tcpdump -ennvvXS -i ix0 greater 65518 > >=20 > > I do expect to get packets over 65518, but I was just curious to > > see if any > > of them would go over 65535. Time will tell. > >=20 > > In a separate test, If I enable lagg, we have LOTS of oversized > > packet > > problems. It looks like tsomax is definitely not making it through > > in > > if_lagg.c - Any recommendations there? I will eventually need lagg, > > as I'm > > sure will others. >=20 > I think somebody has to invent a way to propagate if_hw_maxtso to > interfaces on top of each other. >=20 >=20 > > With dtrace, it's showing t_tsomax >=3D 65518. Shouldn't that not be > > happening? >=20 > Looks like these all come from bce interfaces (bce_intr in the stack > trace), which probably have another value for if_hw_tsomax. >=20 >=20 > Markus >=20 >=20 > > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && > > args[0]->t_tsomax >=3D 65518 / { printf("unexpected tp->t_tsomax: > > %i\n", > > args[0]->t_tsomax); stack(); }' > >=20 > >=20 > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > >=20 > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > >=20 > > 3 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > >=20 > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > >=20 > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > >=20 > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > >=20 > > 1 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > >=20 > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > >=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:27:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72F25515; Wed, 26 Mar 2014 02:27:34 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0ED4F8EE; Wed, 26 Mar 2014 02:27:33 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so1833163qcz.19 for ; Tue, 25 Mar 2014 19:27:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Y418+WnnelDRCvvlIQ9egXwwWaO30n3S1FZFVBBiYQE=; b=O9sJRO0HnTJsKbnTAXTZIWRQ6mjMTQD2jDw8x4rv/mQyCaY7rU6QomGHbbuGuo8pl5 rpQ9LRnqlv2kkbaKQ5utig9kxdRCtpDeBDXWlXJPbcY7UbXyycL0K98b0uby4z3HJNB0 lzYp0BYC/IwG1cW7Ttx3uda0e+1isiTqnVdvV8Q+OWxuzNjx8i2eG9wU3JBIbOb+1KnU Dl6bmXDPANrRYTqlkvUBpcl09T38v37a7WotvekW7I/n1r6prCAhQLh+WsavUeMyXnA8 XpovWPTY/ClG1cVtiOEXjr7sADG+uRbVpJJNdbMy8V3oeDFSUFD7ffgXFFAp5MeYMfBa T8sA== MIME-Version: 1.0 X-Received: by 10.224.54.209 with SMTP id r17mr14802912qag.34.1395800853257; Tue, 25 Mar 2014 19:27:33 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 19:27:33 -0700 (PDT) In-Reply-To: <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca> References: <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca> Date: Tue, 25 Mar 2014 23:27:33 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Andre Oppermann , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:27:34 -0000 That's interesting. I see here in the r251296 commit Andre says : Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to change the limit. I wonder if we add your same TSO patch to if_lagg.c before line 356's ether_ifattach() will fix it. Ultimately, it will need to load the if_hw_tsomax from the if below it - but then again, if the calculation for ixgbe is good enough for that driver, why wouldn't it be good enough for lagg? Unless people think I'm crazy, I'll compile that in at line 356 in if_lagg.c and give it a test run tomorrow. This may need to go into vlan and carp as well, I'm not sure yet. On Tue, Mar 25, 2014 at 8:16 PM, Rick Macklem wrote: > Christopher Forgeron wrote: > > Update: > > > > I'm changing my mind, and I believe Rick's TSO patch is fixing > > things > > (sorry). In looking at my notes, it's possible I had lagg on for > > those > > tests. lagg does seem to negate the TSO patch in my case. > > > Ok, that's useful information. It implies that r251296 doesn't quite > work and needs to be fixed for "stacked" network interface drivers > before it can be used. I've cc'd Andre who is the author of that > patch, in case he knows how to fix it. > > Thanks for checking this, rick > > > kernel.10stable_basicTSO_65535/ > > > > - IP_MAXPACKET = 65535; > > - manually forced (no if statement) ifp->if_hw_tsomax = IP_MAXPACKET > > - > > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > > - Verified on boot via printf that ifp->if_hw_tsomax = 65517 > > - Boot in a NON LAGG environment. ix0 only. > > > > ixgbe's printf is showing packets up to 65530. Haven't run long > > enough yet > > to see if anything will go over 65535 > > > > I have this tcpdump running to check packet size. > > tcpdump -ennvvXS -i ix0 greater 65518 > > > > I do expect to get packets over 65518, but I was just curious to see > > if any > > of them would go over 65535. Time will tell. > > > > In a separate test, If I enable lagg, we have LOTS of oversized > > packet > > problems. It looks like tsomax is definitely not making it through in > > if_lagg.c - Any recommendations there? I will eventually need lagg, > > as I'm > > sure will others. > > > > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not be > > happening? > > > > > > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && > > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax: > > %i\n", > > args[0]->t_tsomax); stack(); }' > > > > > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > > > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > > > > 3 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > > > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > > > > 6 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > > > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > > > > 1 31403 tcp_output:entry unexpected tp->t_tsomax: > > 65535 > > > > kernel`tcp_do_segment+0x2c99 > > kernel`tcp_input+0x11a2 > > kernel`ip_input+0xa2 > > kernel`netisr_dispatch_src+0x5e > > kernel`ether_demux+0x12a > > kernel`ether_nh_input+0x35f > > kernel`netisr_dispatch_src+0x5e > > kernel`bce_intr+0x765 > > kernel`intr_event_execute_handlers+0xab > > kernel`ithread_loop+0x96 > > kernel`fork_exit+0x9a > > kernel`0xffffffff80c75b2e > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:33:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EA248DE; Wed, 26 Mar 2014 02:33:46 +0000 (UTC) Received: from mail-pb0-x22f.google.com (mail-pb0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 163AF99C; Wed, 26 Mar 2014 02:33:46 +0000 (UTC) Received: by mail-pb0-f47.google.com with SMTP id up15so1300451pbc.20 for ; Tue, 25 Mar 2014 19:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=UtVQt/sQokC8xJXXoqVNS9RGbRJMIZzPazy5/Soq6b0=; b=vjPuER3UWd1fg1maYJ3/vle58zppYGRcgEsFxw39X/m+Ckvh964eKeb3sN/GSEyHJ5 xBw4MEgj4gBrf02Q8Ru05TyzZ5UMP1ljkJzAn0+Ss+Kk9nv/yoQtvPE/uUx9GworBCAA neQ3k3xwguODhp0diYTVFcyKWKowOhhsrCoWbZcFn60DAOwAwk6UCZtWRxmZqeW81lRA n2W9Yy+L/l6CsVIcn6O3qvPEhh5UObElCJNOCgWkqSbpWLwZvPWsiC84P1dhsKQUbux8 8Ro437QUrQLQHLLEsiFB6XTxPCoe8qtBFtNy2/lfeFFnLWjHIWYcyID7ukjFvCj6qNzD N+bQ== X-Received: by 10.68.217.234 with SMTP id pb10mr20431225pbc.142.1395801225703; Tue, 25 Mar 2014 19:33:45 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id iu7sm51231653pbc.60.2014.03.25.19.33.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 25 Mar 2014 19:33:45 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 26 Mar 2014 11:33:34 +0900 From: Yonghyeon PYUN Date: Wed, 26 Mar 2014 11:33:34 +0900 To: Rick Macklem Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem Message-ID: <20140326023334.GB2973@michelle.cdnetworks.com> References: <1609686124.539328.1395789035334.JavaMail.root@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1609686124.539328.1395789035334.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Filesystems , FreeBSD Net , Alexander Motin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:33:46 -0000 On Tue, Mar 25, 2014 at 07:10:35PM -0400, Rick Macklem wrote: > Hi, > > First off, I hope you don't mind that I cross-posted this, > but I wanted to make sure both the NFS/iSCSI and networking > types say it. > If you look in this mailing list thread: > http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > you'll see that several people have been working hard at testing and > thanks to them, I think I now know what is going on. Thanks for your hard work on narrowing down that issue. I'm too busy for $work in these days so I couldn't find time to investigate the issue. > (This applies to network drivers that support TSO and are limited to 32 transmit > segments->32 mbufs in chain.) Doing a quick search I found the following > drivers that appear to be affected (I may have missed some): > jme, fxp, age, sge, msk, alc, ale, ixgbe/ix, nfe, e1000/em, re > The magic number 32 was chosen long time ago when I implemented TSO in non-Intel drivers. I tried to find optimal number to reduce the size kernel stack usage at that time. bus_dma(9) will coalesce with previous segment if possible so I thought the number 32 was not an issue. Not sure current bus_dma(9) also has the same code though. The number 32 is arbitrary one so you can increase it if you want. > Further, of these drivers, the following use m_collapse() and not m_defrag() > to try and reduce the # of mbufs in the chain. m_collapse() is not going to > get the 35 mbufs down to 32 mbufs, as far as I can see, so these ones are > more badly broken: > jme, fxp, age, sge, alc, ale, nfe, re I guess m_defeg(9) is more optimized for non-TSO packets. You don't want to waste CPU cycles to copy the full frame to reduce the number of mbufs in the chain. For TSO packets, m_defrag(9) looks better but if we always have to copy a full TSO packet to make TSO work, driver writers have to invent better scheme rather than blindly relying on m_defrag(9), I guess. > > The long description is in the above thread, but the short version is: > - NFS generates a chain with 35 mbufs in it for (read/readdir replies and write requests) > made up of (tcpip header, RPC header, NFS args, 32 clusters of file data) > - tcp_output() usually trims the data size down to tp->t_tsomax (65535) and > then some more to make it an exact multiple of TCP transmit data size. > - the net driver prepends an ethernet header, growing the length by 14 (or > sometimes 18 for vlans), but in the first mbuf and not adding one to the chain. > - m_defrag() copies this to a chain of 32 mbuf clusters (because the total data > length is <= 64K) and it gets sent > > However, it the data length is a little less than 64K when passed to tcp_output() > so that the length including headers is in the range 65519->65535... > - tcp_output() doesn't reduce its size. > - the net driver adds an ethernet header, making the total data length slightly > greater than 64K > - m_defrag() copies it to a chain of 33 mbuf clusters, which fails with EFBIG > --> trainwrecks NFS performance, because the TSO segment is dropped instead of sent. > > A tester also stated that the problem could be reproduced using iSCSI. Maybe > Edward Napierala might know some details w.r.t. what kind of mbuf chain iSCSI > generates? > > Also, one tester has reported that setting if_hw_tsomax in the driver before > the ether_ifattach() call didn't make the value of tp->t_tsomax smaller. > However, reducing IP_MAXPACKET (which is what it is set to by default) did > reduce it. I have no idea why this happens or how to fix it, but it implies > that setting if_hw_tsomax in the driver isn't a solution until this is resolved. > > So, what to do about this? > First, I'd like a simple fix/workaround that can go into 9.3. (which is code > freeze in May). The best thing I can think of is setting if_hw_tsomax to a > smaller default value. (Line# 658 of sys/net/if.c in head.) > > Version A: > replace > ifp->if_hw_tsomax = IP_MAXPACKET; > with > ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN), > IP_MAXPACKET); > plus > replace m_collapse() with m_defrag() in the drivers listed above. > > This would only reduce the default from 65535->65518, so it only impacts > the uncommon case where the output size (with tcpip header) is within > this range. (As such, I don't think it would have a negative impact for > drivers that handle more than 32 transmit segments.) > From the testers, it seems that this is sufficient to get rid of the EFBIG > errors. (The total data length including ethernet header doesn't exceed 64K, > so m_defrag() fits it into 32 mbuf clusters.) > > The main downside of this is that there will be a lot of m_defrag() calls > being done and they do quite a bit of bcopy()'ng. > > Version B: > replace > ifp->if_hw_tsomax = IP_MAXPACKET; > with > ifp->if_hw_tsomax = min(29 * MCLBYTES, IP_MAXPACKET); > > This one would avoid the m_defrag() calls, but might have a negative > impact on TSO performance for drivers that can handle 35 transmit segments, > since the maximum TSO segment size is reduced by about 6K. (Because of the > second size reduction to an exact multiple of TCP transmit data size, the > exact amount varies.) > > Possible longer term fixes: > One longer term fix might be to add something like if_hw_tsomaxseg so that > a driver can set a limit on the number of transmit segments (mbufs in chain) > and tcp_output() could use that to limit the size of the TSO segment, as > required. (I have a first stab at such a patch, but no way to test it, so > I can't see that being done by May. Also, it would require changes to a lot > of drivers to make it work. I've attached this patch, in case anyone wants > to work on it?) > > Another might be to increase the size of MCLBYTES (I don't see this as > practical for 9.3, although the actual change is simple.). I do think > that increasing MCLBYTES might be something to consider doing in the > future, for reasons beyond fixing this. > > So, what do others think should be done? rick > AFAIK all TSO capable drivers you mentioned above have no limit on the number of TX segments in TSO path. Not sure about Intel controllers though. Increasing the number of segment will consume lots of kernel stack in those drivers. Given that ixgbe, which seems to use 100, didn't show any kernal stack shortage, I think bumping the number of segments would be quick way to address the issue. From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:33:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CBBFD8DC; Wed, 26 Mar 2014 02:33:45 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 778CF99B; Wed, 26 Mar 2014 02:33:45 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id w8so1577432qac.41 for ; Tue, 25 Mar 2014 19:33:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=juyjoZ86NPKVOIjrqUMBmJq1TAe2MaY7et12J+Itkos=; b=DZMtXb+c4cC+B1yukf5juzj0lYK+O/VPflY6rv27Z3T/sBpzdSDUSXgF37Kw2wW27z v2rlqjowRPJINzl8wsjNRlOZMmm1hpnEuAMGLjVDd6Xp/DU4sm94/dt3zn34+0JdpAJ6 8C5H6zywBM1KJBC+KuwzzBjrymZnJcIKIO7YAFSIWCGKVzHhLJlSATsxqDPVpfDzx722 ljUKGFELkjSunBYMnSwrr51iGvIAom1WK8W4+xMUByPWsWc8Aeo8anTZhGWJmKu0Eejs gUFyeDWxLfmtwOVpYe2H9HvfIRIAYr7iNN6mziSqZmEtxIIkVNbH9eaRjbHQru++E0lM i3BA== MIME-Version: 1.0 X-Received: by 10.140.82.175 with SMTP id h44mr3463033qgd.65.1395801224728; Tue, 25 Mar 2014 19:33:44 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Tue, 25 Mar 2014 19:33:44 -0700 (PDT) In-Reply-To: <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> Date: Tue, 25 Mar 2014 23:33:44 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:33:46 -0000 On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert wrote: > > > Is 65517 correct? With Ricks patch, I get this: > > dev.ix.0.hw_tsomax: 65518 > Perhaps a difference between 9.2 and 10 for one of the macros? My code is: ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); printf("CSF - 3 Init, ifp->if_hw_tsomax = %d\n", ifp->if_hw_tsomax); (BTW, you should submit the hw_tsomax sysctl patch, that's useful to others) > Also the dtrace command you used excludes 65518... > Oh, I thought it was giving every packet that is greater than or equal to 65518 - Could you show me the proper command? That's the third time I've used dtrace, so I'm making this up as I go. :-) From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:51:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C42FAC2C for ; Wed, 26 Mar 2014 02:51:38 +0000 (UTC) Received: from mail-yk0-x231.google.com (mail-yk0-x231.google.com [IPv6:2607:f8b0:4002:c07::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 83E8AAF0 for ; Wed, 26 Mar 2014 02:51:38 +0000 (UTC) Received: by mail-yk0-f177.google.com with SMTP id q200so98241ykb.22 for ; Tue, 25 Mar 2014 19:51:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=R8Sj21yHT2kV4iges3HeRaKt9OsTaMm/eWwcbY79tRQ=; b=wgTAhoU9T+COkCovIURukbyRw8XJunB5RfLrBekfzmUIHLmlIz50dyWX2XEWR1Pidm pPRmlQPyoA7nWR9l7ES8UY/ik49v0t2ifQOAcIXWwk3p88EGNn1elsFvG/VSRLQm2D0Z kI0/n3oZsGDgYDAxmV1wIdXMXnJIlcKIJILhic5dCOw64G3asC1w8fMwXC43yT1+Asoh /sU41ymgrUlS7qhWUspTFMYDdEM88auEbhmIptKcCyJIT+K3VYa8HSf8DNGy6r7tie/w z6Kyw18y7jXXX901JcaySbFdkkgGN9EYa0uLue9Fxz8u74nufXTT5+b9lW3vCg8AbogW aS+Q== X-Received: by 10.236.8.68 with SMTP id 44mr79360468yhq.39.1395802297763; Tue, 25 Mar 2014 19:51:37 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id u5sm15566919yhg.25.2014.03.25.19.51.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Mar 2014 19:51:37 -0700 (PDT) Message-ID: <533240B4.50809@gmail.com> Date: Wed, 26 Mar 2014 10:51:32 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Christopher Forgeron Subject: Re: syslogd:sendto: no buffer available on 10-stable References: <53310C57.5070102@gmail.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:51:38 -0000 Thanks, Christopher. But I think my problem may does not related to TSO issue. I have tried disable tso with "ifconfig igb(x) -tso" and ovserved with "netstat -ihw 1", and found "oErrs" does not disappeared. Regards Simon 䚎 14-3-25 22:08, Christopher Forgeron 写道: > Hi Simon, > > Try checking out the "9.2 ixgbe tx queue hang' thread here, and see if > it applies to you. > > > On Tue, Mar 25, 2014 at 1:55 AM, k simon > wrote: > > Hi,Lists: > I have got lots of "no buffer available" on 10-stable with igb nic. > But em and bce works well. And I tried force igb to 4 or 8 queues and > set hw.igb.rx_process_limit="-1", but have nothing helped. > > > Regards > Simon > > > > # uname -a > FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r262469: Tue Feb 25 > 13:27:11 CST 2014 > root@sq-l1-n2:/usr/obj/usr/src/sys/stable-10-262458 amd64 > > > # netstat -mb > 19126/73289/92415 mbufs in use (current/cache/total) > 13289/46841/60130/524288 mbuf clusters in use (current/cache/total/max) > 13289/46812 mbuf+clusters out of packet secondary zone in use > (current/cache) > 5638/22605/28243/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/77672 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/43690 16k jumbo clusters in use (current/cache/total/max) > 53914K/202424K/256338K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > # netstat -di > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll Drop > igb0 1500 00:1b:21:70:5f:80 17212101113 1355809 0 > 19612978862 0 0 > igb1 1500 00:1b:21:70:5f:81 76601294282 81162751 0 > 74236432686 0 0 > lo0 16384 20532742636 0 0 > 20522475797 0 0 > lo0 - your-net localhost 2736994243 - - > 20520227166 - - > > > # sysctl hw.igb > hw.igb.rxd: 2048 > hw.igb.txd: 2048 > hw.igb.enable_aim: 1 > hw.igb.enable_msix: 1 > hw.igb.max_interrupt_rate: 12000 > hw.igb.buf_ring_size: 4096 > hw.igb.header_split: 0 > hw.igb.num_queues: 1 > hw.igb.rx_process_limit: 1000 > > > # sysctl dev.igb.1 > dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 > dev.igb.1.%driver: igb > dev.igb.1.%location: slot=0 function=1 > dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 > subdevice=0xa04c class=0x020000 > dev.igb.1.%parent: pci8 > dev.igb.1.nvm: -1 > dev.igb.1.enable_aim: 1 > dev.igb.1.fc: 0 > dev.igb.1.rx_processing_limit: 4096 > dev.igb.1.link_irq: 3 > dev.igb.1.dropped: 0 > dev.igb.1.tx_dma_fail: 0 > dev.igb.1.rx_overruns: 0 > dev.igb.1.watchdog_timeouts: 0 > dev.igb.1.device_control: 1086325313 > dev.igb.1.rx_control: 67141634 > dev.igb.1.interrupt_mask: 4 > dev.igb.1.extended_int_mask: 2147483651 > dev.igb.1.tx_buf_alloc: 0 > dev.igb.1.rx_buf_alloc: 0 > dev.igb.1.fc_high_water: 58976 > dev.igb.1.fc_low_water: 58960 > dev.igb.1.queue0.no_desc_avail: 10874 > dev.igb.1.queue0.tx_packets: 74509997338 > dev.igb.1.queue0.rx_packets: 76837720630 > dev.igb.1.queue0.rx_bytes: 35589607860237 > dev.igb.1.queue0.lro_queued: 0 > dev.igb.1.queue0.lro_flushed: 0 > dev.igb.1.mac_stats.excess_coll: 0 > dev.igb.1.mac_stats.single_coll: 0 > dev.igb.1.mac_stats.multiple_coll: 0 > dev.igb.1.mac_stats.late_coll: 0 > dev.igb.1.mac_stats.collision_count: 0 > dev.igb.1.mac_stats.symbol_errors: 0 > dev.igb.1.mac_stats.sequence_errors: 0 > dev.igb.1.mac_stats.defer_count: 0 > dev.igb.1.mac_stats.missed_packets: 81162751 > dev.igb.1.mac_stats.recv_no_buff: 176691324 > dev.igb.1.mac_stats.recv_undersize: 0 > dev.igb.1.mac_stats.recv_fragmented: 0 > dev.igb.1.mac_stats.recv_oversize: 0 > dev.igb.1.mac_stats.recv_jabber: 0 > dev.igb.1.mac_stats.recv_errs: 0 > dev.igb.1.mac_stats.crc_errs: 0 > dev.igb.1.mac_stats.alignment_errs: 0 > dev.igb.1.mac_stats.coll_ext_errs: 0 > dev.igb.1.mac_stats.xon_recvd: 0 > dev.igb.1.mac_stats.xon_txd: 0 > dev.igb.1.mac_stats.xoff_recvd: 0 > dev.igb.1.mac_stats.xoff_txd: 0 > dev.igb.1.mac_stats.total_pkts_recvd: 76925709917 > dev.igb.1.mac_stats.good_pkts_recvd: 76837704301 > dev.igb.1.mac_stats.bcast_pkts_recvd: 49174716 > dev.igb.1.mac_stats.mcast_pkts_recvd: 282670 > dev.igb.1.mac_stats.rx_frames_64: 31057121854 > dev.igb.1.mac_stats.rx_frames_65_127: 19996324498 > dev.igb.1.mac_stats.rx_frames_128_255: 1171960837 > dev.igb.1.mac_stats.rx_frames_256_511: 2295894674 > dev.igb.1.mac_stats.rx_frames_512_1023: 2026241811 > dev.igb.1.mac_stats.rx_frames_1024_1522: 20290160627 > dev.igb.1.mac_stats.good_octets_recvd: 36204302378783 > dev.igb.1.mac_stats.good_octets_txd: 59038220741656 > dev.igb.1.mac_stats.total_pkts_txd: 90973292365 > dev.igb.1.mac_stats.good_pkts_txd: 90973292359 > dev.igb.1.mac_stats.bcast_pkts_txd: 2408182 > dev.igb.1.mac_stats.mcast_pkts_txd: 246782 > dev.igb.1.mac_stats.tx_frames_64: 24604769631 > dev.igb.1.mac_stats.tx_frames_65_127: 21373976133 > dev.igb.1.mac_stats.tx_frames_128_255: 3047554762 > dev.igb.1.mac_stats.tx_frames_256_511: 3751820879 > dev.igb.1.mac_stats.tx_frames_512_1023: 3481550512 > dev.igb.1.mac_stats.tx_frames_1024_1522: 34713620448 > dev.igb.1.mac_stats.tso_txd: 9549335102 > dev.igb.1.mac_stats.tso_ctx_fail: 0 > dev.igb.1.interrupts.asserts: 16929843739 > dev.igb.1.interrupts.rx_pkt_timer: 76834839132 > dev.igb.1.interrupts.rx_abs_timer: 0 > dev.igb.1.interrupts.tx_pkt_timer: 0 > dev.igb.1.interrupts.tx_abs_timer: 76835042438 > dev.igb.1.interrupts.tx_queue_empty: 90970561829 > dev.igb.1.interrupts.tx_queue_min_thresh: 0 > dev.igb.1.interrupts.rx_desc_min_thresh: 0 > dev.igb.1.interrupts.rx_overrun: 0 > dev.igb.1.host.breaker_tx_pkt: 0 > dev.igb.1.host.host_tx_pkt_discard: 0 > dev.igb.1.host.rx_pkt: 2865171 > dev.igb.1.host.breaker_rx_pkts: 0 > dev.igb.1.host.breaker_rx_pkt_drop: 0 > dev.igb.1.host.tx_good_pkt: 2730536 > dev.igb.1.host.breaker_tx_pkt_drop: 0 > dev.igb.1.host.rx_good_bytes: 36204307257585 > dev.igb.1.host.tx_good_bytes: 59038220743158 > dev.igb.1.host.length_errors: 0 > dev.igb.1.host.serdes_violation_pkt: 0 > dev.igb.1.host.header_redir_missed: 0 > > > > # cat /var/run/dmesg.boot |grep igb > igb0: mem > 0xd5ee0000-0xd5efffff,0xd5800000-0xd5bfffff,0xd5edc000-0xd5edffff irq 18 > at device 0.0 on pci8 > igb0: Using MSIX interrupts with 2 vectors > igb0: Ethernet address: 00:1b:21:70:5f:80 > igb1: mem > 0xd5ea0000-0xd5ebffff,0xd5400000-0xd57fffff,0xd5ed8000-0xd5edbfff irq 19 > at device 0.1 on pci8 > igb1: Using MSIX interrupts with 2 vectors > igb1: Ethernet address: 00:1b:21:70:5f:81 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org > " > > From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 16:42:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 693CCE46 for ; Wed, 26 Mar 2014 16:42:20 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D8AFB1A for ; Wed, 26 Mar 2014 16:42:20 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id uz6so2821238obc.15 for ; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/8YG+rkmeuUd2RV9dT8F039RXfptjIAp11YNZYCk91I=; b=ijw7SKXGnWxldqdeHtOvDwHgkJ/17fjzrsWZA/OK+fbZBflyA5KsDHBq2cTP4guvel hj/u5qZ/PmdGqtan+D+EvV074UrwRhfg4cX2MitYJgvIywWZSRMH3SidZLFUY8ZCzWpn QOkO5qI7yDWRkqM5kWpr1+yde/m/strHSDP0+HXMOYIrjj5NL1r2OpwCMO8/5LDASIVe 4sEebR7j9S/MUnzJ/vMF+0C7hfpSRtJT9YHUA12tDeMip/HFhWIbPTABMKGhHXDS8KKJ MY4dl39YwFkW9RPteU4UKRgAgSVJ9GO5LraBK+8r3rNV/sljKX6I/Ahte0tPJ3xTWxEd iCpw== MIME-Version: 1.0 X-Received: by 10.182.22.227 with SMTP id h3mr27388189obf.36.1395852139444; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) Received: by 10.76.12.34 with HTTP; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) In-Reply-To: References: <531771C8.1040207@yandex.ru> <20140306145231.Q75313@sola.nimnet.asn.au> <20140307024724.T75313@sola.nimnet.asn.au> Date: Wed, 26 Mar 2014 17:42:19 +0100 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: Ian Smith , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 16:42:20 -0000 ... snip ... >> I'm wondering what's happening on the outbound path, most of your rules >> handle inbound (to kernel) and it seems that rule 65535 deals with most >> outbound, except those specifically acting on both paths. >> > So do I :) > >> >> Maybe try adding to the above: >> ipfw add 63510 count log ip from table(1) to any out recv table(8) >> and >> ipfw add 64510 count log ip from table(1) to any out recv table(8) >> >> which should catch them on the outbound path before the default rule; >> outbound packets have both receive and transmit interfaces available. >> >> They never get that far :( Those rules log nothing. So I see the packet > coming in, triggering the skipto, triggering the count log ... in recv but > not the count log ... out. > > >> I guess you're sure that these packets are actually going out to some >> external address, not a localhost or alias address (which of course are >> received and not sent out anywhere)? >> > Yes, when the go through they go to external address, which is in the > routing table. > >> >> I guess the above new log lines should show the interface (if any) >> these packets are leaving on, after routing (maybe a routing issue?) >> >> Good luck hunting. If in doubt, throw ever more logging at it! :) And >> please let the list know if you solve it - or not! >> >> cheers, Ian >> > > To make it even more interesting, it tested this on another machine, where > I'm unable to reproduce this "problem". I'm thinking that a reboot might > help, but this is kind of fascinating so I would like to actually find the > cause. I will investigate further. > > Best regards > Andreas > And now it happened on another machine, but different type of traffic. Traffic triggering the divert rule work fine, some traffic not triggering the divert rule does not. After everything works as expected. Before reboot I flushed the firewall so that only default rule was in play ( allow all from any to any ), and then *no* traffic for that destination went out of the box. There are something like ~1000 routes in the routing table. Routes are added and removed according to some triggers, so during uptime of the machine there is a bit of messing with the routing table. At this stage I'm at a loss on how to continue investigating this, so I'll just implement the forwarding via fwd rules in ipfw and altogether ignore the routing table. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 17:44:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71ACA5C0; Wed, 26 Mar 2014 17:44:38 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A79A1F8; Wed, 26 Mar 2014 17:44:38 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so2621455qaj.11 for ; Wed, 26 Mar 2014 10:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bsA84drya82gxl/p3iPfMQUCsaPymhQTfIJ/DLpVZYo=; b=AH/eHKOf+rAzxppBLCw61u1316a85NLiFJpSTVobwc8Hk7Xg7J6DeIOaWGcJvQ6HKH 4RCQnxzlt9+FLVq3VXxXDyv87OQxciyjQwu5WajwuLlYzRaw5SocQvL2cy3HMNUuPN0m GXugddwvuV9CYHJu2U1e8JPb3aQenYmGDgPi6rtar+oeW0b1dpam4Abtv5ALwhgKITiJ hDtzMhB/jXokwYY/cIh/pVKoelcWHLy+g1m9kS7n5DVhz3IjABA5WtUFPU+mVRf8muTG npLqINHv30Jo15vIuf7PZGtehxSH8IZgMvpRLaRpF+IGj4swvn8McU7JsXBqkEtyyrPR iAhA== MIME-Version: 1.0 X-Received: by 10.140.48.199 with SMTP id o65mr51466713qga.16.1395855877261; Wed, 26 Mar 2014 10:44:37 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Wed, 26 Mar 2014 10:44:37 -0700 (PDT) In-Reply-To: References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> Date: Wed, 26 Mar 2014 14:44:37 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Markus Gebert Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 17:44:38 -0000 Up for almost 19 hours under load without a single error. I would say the TSO patch does work, now I'm going to run lagg tests. The more I think of it, the more I wonder if setting tsomax in if.c at line 660 isn't the better idea, like below. 660: if (ifp->if_hw_tsomax == 0) 661: ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); I know there are concerns about the impact on various cards, but right now if.c will set if_hw_tssomax to IP_MAXPACKET, which we know is bad for ixgbe, and I believe bad for lagg (tests will show) - If the driver isn't specifically setting it to a different setting, is there harm in limiting all if's to a default of IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN) if not specified otherwise? When is a TSO of 65535 going to be useful? I can confirm that with just the TSO patch in ixgbe, and lagg enabled, the problem still exists. Last night's tests never went above a packet of 65530. Now with lagg enabled, I'm seeing packets of 65543 within 5 minutes, so we're already breaking. On Tue, Mar 25, 2014 at 11:33 PM, Christopher Forgeron wrote: > > > On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert > wrote: > >> >> >> Is 65517 correct? With Ricks patch, I get this: >> >> dev.ix.0.hw_tsomax: 65518 >> > > Perhaps a difference between 9.2 and 10 for one of the macros? My code is: > > > ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > printf("CSF - 3 Init, ifp->if_hw_tsomax = %d\n", > ifp->if_hw_tsomax); > > (BTW, you should submit the hw_tsomax sysctl patch, that's useful to > others) > > >> Also the dtrace command you used excludes 65518... >> > > Oh, I thought it was giving every packet that is greater than or equal to > 65518 - Could you show me the proper command? That's the third time I've > used dtrace, so I'm making this up as I go. :-) > From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 22:20:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 077B05D6; Wed, 26 Mar 2014 22:20:27 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 958B9379; Wed, 26 Mar 2014 22:20:26 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so3364779qcz.5 for ; Wed, 26 Mar 2014 15:20:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=96B24yRasPJDl7ovY7BkgeZLK2eiSfKtSoQLKJck3O0=; b=dJ3ZMoSlt1gWMWk2nYv7WRt06dDCxS2kqrt25DiN/bdw0dAcms47LZx1QIQSmxfMSa Wm3N3FMSqC/ytA7FqG27mZprfobByqDvBGgVLJ+sBi5j7apmgLp9naiSGrcP5g2XD9YE 1CBCKdO9eMzeptna9/97u7CfxpL32RbbAov5TIgXFblRRdJplO1gPu3Z9JcK1/fLNTJv nwB3SmngM4az/w5p2G8i3QOm8c88WldvXnSQp2Uk/pm/D1HDQRVAc83zVvocltQyQpna c672Y4zpRa3FPJgE1Z72lhw4RE4FZPQG3PCiSyKybrN6WgoZGtgUdfPeia1X/EyqLN/e iVwA== MIME-Version: 1.0 X-Received: by 10.224.22.147 with SMTP id n19mr5619227qab.93.1395872425576; Wed, 26 Mar 2014 15:20:25 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Wed, 26 Mar 2014 15:20:25 -0700 (PDT) In-Reply-To: References: <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca> Date: Wed, 26 Mar 2014 19:20:25 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Andre Oppermann , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 22:20:27 -0000 Confirmed that adding this to sys/net/if.c fixes the issue for lagg as well as ixgbe. 660: if (ifp->if_hw_tsomax == 0) 661: ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); Code before (looks to be introduced in 9.2, r251296 as Rick mentions above) just sets ifp->if_hw_tsomax = IP_MAXPACKET , so I really don't see much downside in setting it to ~65518 if it enhances compatibility. This should also fix it for carp, vlan, and others. I'm going to do a few more tests, but in the meantime lets discuss the cons of doing this. On Tue, Mar 25, 2014 at 11:27 PM, Christopher Forgeron wrote: > That's interesting. I see here in the r251296 commit Andre says : > > Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to > change the limit. > > I wonder if we add your same TSO patch to if_lagg.c before line 356's > ether_ifattach() will fix it. > > Ultimately, it will need to load the if_hw_tsomax from the if below it - > but then again, if the calculation for ixgbe is good enough for that > driver, why wouldn't it be good enough for lagg? > > Unless people think I'm crazy, I'll compile that in at line 356 in > if_lagg.c and give it a test run tomorrow. > > This may need to go into vlan and carp as well, I'm not sure yet. > > > On Tue, Mar 25, 2014 at 8:16 PM, Rick Macklem wrote: > >> Christopher Forgeron wrote: >> > Update: >> > >> > I'm changing my mind, and I believe Rick's TSO patch is fixing >> > things >> > (sorry). In looking at my notes, it's possible I had lagg on for >> > those >> > tests. lagg does seem to negate the TSO patch in my case. >> > >> Ok, that's useful information. It implies that r251296 doesn't quite >> work and needs to be fixed for "stacked" network interface drivers >> before it can be used. I've cc'd Andre who is the author of that >> patch, in case he knows how to fix it. >> >> Thanks for checking this, rick >> >> > kernel.10stable_basicTSO_65535/ >> > >> > - IP_MAXPACKET = 65535; >> > - manually forced (no if statement) ifp->if_hw_tsomax = IP_MAXPACKET >> > - >> > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); >> > - Verified on boot via printf that ifp->if_hw_tsomax = 65517 >> > - Boot in a NON LAGG environment. ix0 only. >> > >> > ixgbe's printf is showing packets up to 65530. Haven't run long >> > enough yet >> > to see if anything will go over 65535 >> > >> > I have this tcpdump running to check packet size. >> > tcpdump -ennvvXS -i ix0 greater 65518 >> > >> > I do expect to get packets over 65518, but I was just curious to see >> > if any >> > of them would go over 65535. Time will tell. >> > >> > In a separate test, If I enable lagg, we have LOTS of oversized >> > packet >> > problems. It looks like tsomax is definitely not making it through in >> > if_lagg.c - Any recommendations there? I will eventually need lagg, >> > as I'm >> > sure will others. >> > >> > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not be >> > happening? >> > >> > >> > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && >> > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax: >> > %i\n", >> > args[0]->t_tsomax); stack(); }' >> > >> > >> > 6 31403 tcp_output:entry unexpected tp->t_tsomax: >> > 65535 >> > >> > kernel`tcp_do_segment+0x2c99 >> > kernel`tcp_input+0x11a2 >> > kernel`ip_input+0xa2 >> > kernel`netisr_dispatch_src+0x5e >> > kernel`ether_demux+0x12a >> > kernel`ether_nh_input+0x35f >> > kernel`netisr_dispatch_src+0x5e >> > kernel`bce_intr+0x765 >> > kernel`intr_event_execute_handlers+0xab >> > kernel`ithread_loop+0x96 >> > kernel`fork_exit+0x9a >> > kernel`0xffffffff80c75b2e >> > >> > 3 31403 tcp_output:entry unexpected tp->t_tsomax: >> > 65535 >> > >> > kernel`tcp_do_segment+0x2c99 >> > kernel`tcp_input+0x11a2 >> > kernel`ip_input+0xa2 >> > kernel`netisr_dispatch_src+0x5e >> > kernel`ether_demux+0x12a >> > kernel`ether_nh_input+0x35f >> > kernel`netisr_dispatch_src+0x5e >> > kernel`bce_intr+0x765 >> > kernel`intr_event_execute_handlers+0xab >> > kernel`ithread_loop+0x96 >> > kernel`fork_exit+0x9a >> > kernel`0xffffffff80c75b2e >> > >> > 6 31403 tcp_output:entry unexpected tp->t_tsomax: >> > 65535 >> > >> > kernel`tcp_do_segment+0x2c99 >> > kernel`tcp_input+0x11a2 >> > kernel`ip_input+0xa2 >> > kernel`netisr_dispatch_src+0x5e >> > kernel`ether_demux+0x12a >> > kernel`ether_nh_input+0x35f >> > kernel`netisr_dispatch_src+0x5e >> > kernel`bce_intr+0x765 >> > kernel`intr_event_execute_handlers+0xab >> > kernel`ithread_loop+0x96 >> > kernel`fork_exit+0x9a >> > kernel`0xffffffff80c75b2e >> > >> > 1 31403 tcp_output:entry unexpected tp->t_tsomax: >> > 65535 >> > >> > kernel`tcp_do_segment+0x2c99 >> > kernel`tcp_input+0x11a2 >> > kernel`ip_input+0xa2 >> > kernel`netisr_dispatch_src+0x5e >> > kernel`ether_demux+0x12a >> > kernel`ether_nh_input+0x35f >> > kernel`netisr_dispatch_src+0x5e >> > kernel`bce_intr+0x765 >> > kernel`intr_event_execute_handlers+0xab >> > kernel`ithread_loop+0x96 >> > kernel`fork_exit+0x9a >> > kernel`0xffffffff80c75b2e >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to >> > "freebsd-net-unsubscribe@freebsd.org" >> > >> > > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 00:27:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75F6AF67; Thu, 27 Mar 2014 00:27:56 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DF39D162; Thu, 27 Mar 2014 00:27:55 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqwEAK9vM1ODaFve/2dsb2JhbABZg0FXgwq4PYYdTVGBMHSCJQEBAQMBAQEBIAQnIAsFFhgCAg0ZAiMGAQkmDgcEARoCBIdEAwkIDa5cmwUNh0gXgSmIHIMQgUQBBgEBGzQHgm+BSQSUXweBEGqDIIs2hUqDSiExewEIFyI X-IronPort-AV: E=Sophos;i="4.97,738,1389762000"; d="scan'208";a="109331825" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 26 Mar 2014 20:27:48 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 949A2B3EEF; Wed, 26 Mar 2014 20:27:48 -0400 (EDT) Date: Wed, 26 Mar 2014 20:27:48 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com Message-ID: <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> In-Reply-To: <20140326023334.GB2973@michelle.cdnetworks.com> Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Filesystems , FreeBSD Net , Alexander Motin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 00:27:56 -0000 pyunyh@gmail.com wrote: > On Tue, Mar 25, 2014 at 07:10:35PM -0400, Rick Macklem wrote: > > Hi, > > > > First off, I hope you don't mind that I cross-posted this, > > but I wanted to make sure both the NFS/iSCSI and networking > > types say it. > > If you look in this mailing list thread: > > http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > > you'll see that several people have been working hard at testing > > and > > thanks to them, I think I now know what is going on. > > > Thanks for your hard work on narrowing down that issue. I'm too > busy for $work in these days so I couldn't find time to investigate > the issue. > > > (This applies to network drivers that support TSO and are limited > > to 32 transmit > > segments->32 mbufs in chain.) Doing a quick search I found the > > following > > drivers that appear to be affected (I may have missed some): > > jme, fxp, age, sge, msk, alc, ale, ixgbe/ix, nfe, e1000/em, re > > > > The magic number 32 was chosen long time ago when I implemented TSO > in non-Intel drivers. I tried to find optimal number to reduce the > size kernel stack usage at that time. bus_dma(9) will coalesce > with previous segment if possible so I thought the number 32 was > not an issue. Not sure current bus_dma(9) also has the same code > though. The number 32 is arbitrary one so you can increase > it if you want. > Well, in the case of "ix" Jack Vogel says it is a hardware limitation. I can't change drivers that I can't test and don't know anything about the hardware. Maybe replacing m_collapse() with m_defrag() is an exception, since I know what that is doing and it isn't hardware related, but I would still prefer a review by the driver author/maintainer before making such a change. If there are drivers that you know can be increased from 32->35 please do so, since that will not only avoid the EFBIG failures but also avoid a lot of calls to m_defrag(). > > Further, of these drivers, the following use m_collapse() and not > > m_defrag() > > to try and reduce the # of mbufs in the chain. m_collapse() is not > > going to > > get the 35 mbufs down to 32 mbufs, as far as I can see, so these > > ones are > > more badly broken: > > jme, fxp, age, sge, alc, ale, nfe, re > > I guess m_defeg(9) is more optimized for non-TSO packets. You don't > want to waste CPU cycles to copy the full frame to reduce the > number of mbufs in the chain. For TSO packets, m_defrag(9) looks > better but if we always have to copy a full TSO packet to make TSO > work, driver writers have to invent better scheme rather than > blindly relying on m_defrag(9), I guess. > Yes, avoiding m_defrag() calls would be nice. For this issue, increasing the transmit segment limit from 32->35 does that, if the change can be done easily/safely. Otherwise, all I can think of is my suggestion to add something like if_hw_tsomaxseg which the driver can use to tell tcp_output() the driver's limit for # of mbufs in the chain. > > > > The long description is in the above thread, but the short version > > is: > > - NFS generates a chain with 35 mbufs in it for (read/readdir > > replies and write requests) > > made up of (tcpip header, RPC header, NFS args, 32 clusters of > > file data) > > - tcp_output() usually trims the data size down to tp->t_tsomax > > (65535) and > > then some more to make it an exact multiple of TCP transmit data > > size. > > - the net driver prepends an ethernet header, growing the length > > by 14 (or > > sometimes 18 for vlans), but in the first mbuf and not adding > > one to the chain. > > - m_defrag() copies this to a chain of 32 mbuf clusters (because > > the total data > > length is <= 64K) and it gets sent > > > > However, it the data length is a little less than 64K when passed > > to tcp_output() > > so that the length including headers is in the range > > 65519->65535... > > - tcp_output() doesn't reduce its size. > > - the net driver adds an ethernet header, making the total data > > length slightly > > greater than 64K > > - m_defrag() copies it to a chain of 33 mbuf clusters, which > > fails with EFBIG > > --> trainwrecks NFS performance, because the TSO segment is dropped > > instead of sent. > > > > A tester also stated that the problem could be reproduced using > > iSCSI. Maybe > > Edward Napierala might know some details w.r.t. what kind of mbuf > > chain iSCSI > > generates? > > > > Also, one tester has reported that setting if_hw_tsomax in the > > driver before > > the ether_ifattach() call didn't make the value of tp->t_tsomax > > smaller. > > However, reducing IP_MAXPACKET (which is what it is set to by > > default) did > > reduce it. I have no idea why this happens or how to fix it, but it > > implies > > that setting if_hw_tsomax in the driver isn't a solution until this > > is resolved. > > > > So, what to do about this? > > First, I'd like a simple fix/workaround that can go into 9.3. > > (which is code > > freeze in May). The best thing I can think of is setting > > if_hw_tsomax to a > > smaller default value. (Line# 658 of sys/net/if.c in head.) > > > > Version A: > > replace > > ifp->if_hw_tsomax = IP_MAXPACKET; > > with > > ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + > > ETHER_VLAN_ENCAP_LEN), > > IP_MAXPACKET); > > plus > > replace m_collapse() with m_defrag() in the drivers listed above. > > > > This would only reduce the default from 65535->65518, so it only > > impacts > > the uncommon case where the output size (with tcpip header) is > > within > > this range. (As such, I don't think it would have a negative impact > > for > > drivers that handle more than 32 transmit segments.) > > From the testers, it seems that this is sufficient to get rid of > > the EFBIG > > errors. (The total data length including ethernet header doesn't > > exceed 64K, > > so m_defrag() fits it into 32 mbuf clusters.) > > > > The main downside of this is that there will be a lot of m_defrag() > > calls > > being done and they do quite a bit of bcopy()'ng. > > > > Version B: > > replace > > ifp->if_hw_tsomax = IP_MAXPACKET; > > with > > ifp->if_hw_tsomax = min(29 * MCLBYTES, IP_MAXPACKET); > > > > This one would avoid the m_defrag() calls, but might have a > > negative > > impact on TSO performance for drivers that can handle 35 transmit > > segments, > > since the maximum TSO segment size is reduced by about 6K. (Because > > of the > > second size reduction to an exact multiple of TCP transmit data > > size, the > > exact amount varies.) > > > > Possible longer term fixes: > > One longer term fix might be to add something like if_hw_tsomaxseg > > so that > > a driver can set a limit on the number of transmit segments (mbufs > > in chain) > > and tcp_output() could use that to limit the size of the TSO > > segment, as > > required. (I have a first stab at such a patch, but no way to test > > it, so > > I can't see that being done by May. Also, it would require changes > > to a lot > > of drivers to make it work. I've attached this patch, in case > > anyone wants > > to work on it?) > > > > Another might be to increase the size of MCLBYTES (I don't see this > > as > > practical for 9.3, although the actual change is simple.). I do > > think > > that increasing MCLBYTES might be something to consider doing in > > the > > future, for reasons beyond fixing this. > > > > So, what do others think should be done? rick > > > > AFAIK all TSO capable drivers you mentioned above have no limit on > the number of TX segments in TSO path. Not sure about Intel > controllers though. Increasing the number of segment will consume > lots of kernel stack in those drivers. Given that ixgbe, which > seems to use 100, didn't show any kernal stack shortage, I think > bumping the number of segments would be quick way to address the > issue. > Well, bumping it from 32->35 is all it would take for NFS (can't comment w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 (just so others aren't confused by the above comment). I understand your point was w.r.t. using 100 without blowing the kernel stack, but since the testers have been using "ix" with the 82599 chip which is limited to 32 transmit segments... However, please increase any you know can be safely done from 32->35, rick _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 00:31:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 289F9397; Thu, 27 Mar 2014 00:31:45 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CE23C19E; Thu, 27 Mar 2014 00:31:44 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEANtwM1ODaFve/2dsb2JhbABZhBiDCr94gTB0giUBAQEEIwRSGw4KAgINGQJZBhOHea5roloXgSmNFDQHgm+BSQSrAINKIYFu X-IronPort-AV: E=Sophos;i="4.97,738,1389762000"; d="scan'208";a="109332261" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 26 Mar 2014 20:31:43 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 83BB7B4095; Wed, 26 Mar 2014 20:31:43 -0400 (EDT) Date: Wed, 26 Mar 2014 20:31:43 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1353469098.1239101.1395880303531.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 00:31:45 -0000 Christopher Forgeron wrote: > > > > > > On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert < > markus.gebert@hostpoint.ch > wrote: > > > > > > Is 65517 correct? With Ricks patch, I get this: > > dev.ix.0.hw_tsomax: 65518 > > > > Perhaps a difference between 9.2 and 10 for one of the macros? My > code is: > > ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + > ETHER_VLAN_ENCAP_LEN); > printf("CSF - 3 Init, ifp->if_hw_tsomax = %d\n", ifp->if_hw_tsomax); > The difference is simply that IP_MAXPACKET == 65535, but I've been using 32 * MCLBYTES == 65536 (the latter is the amount of data m_defrag() can squeeze into 32 mbuf clusters). ie. I've suggested: ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN), IP_MAXPACKET); - I put the min() in just so it wouldn't break if MCLBYTES is increased someday. rick > > (BTW, you should submit the hw_tsomax sysctl patch, that's useful to > others) > > > > > > Also the dtrace command you used excludes 65518... > > > > Oh, I thought it was giving every packet that is greater than or > equal to 65518 - Could you show me the proper command? That's the > third time I've used dtrace, so I'm making this up as I go. :-) > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 00:35:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A4B04BF; Thu, 27 Mar 2014 00:35:50 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id F29B1258; Thu, 27 Mar 2014 00:35:49 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAAhyM1ODaFve/2dsb2JhbABZg0FXgwq4PYZqUYEwdIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1AIDa5goloXgSmMcQYBARsBMweCb4FJBJV2hAqRAINKITF8CBci X-IronPort-AV: E=Sophos;i="4.97,738,1389762000"; d="scan'208";a="109332779" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 26 Mar 2014 20:35:48 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9E93AB4067; Wed, 26 Mar 2014 20:35:48 -0400 (EDT) Date: Wed, 26 Mar 2014 20:35:48 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1380107288.1240335.1395880548644.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net , Garrett Wollman , Andre Oppermann , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 00:35:50 -0000 Christopher Forgeron wrote: > That's interesting. I see here in the r251296 commit Andre says : > > Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() > to > change the limit. > > I wonder if we add your same TSO patch to if_lagg.c before line > 356's > ether_ifattach() will fix it. > I think the value(s) for underlying hardware drivers have to somehow be propagated up through lagg. I haven't looked at the code, so I don't know what that would be. Putting the patch for ixgbe.c in lagg wouldn't make sense, since it doesn't know if the underlying devices have the 32 limit. I've suggested in the other thread what you suggested in a recent post...ie. to change the default, at least until the propagation of driver set values is resolved. rick > Ultimately, it will need to load the if_hw_tsomax from the if below > it - > but then again, if the calculation for ixgbe is good enough for that > driver, why wouldn't it be good enough for lagg? > > Unless people think I'm crazy, I'll compile that in at line 356 in > if_lagg.c and give it a test run tomorrow. > > This may need to go into vlan and carp as well, I'm not sure yet. > > > On Tue, Mar 25, 2014 at 8:16 PM, Rick Macklem > wrote: > > > Christopher Forgeron wrote: > > > Update: > > > > > > I'm changing my mind, and I believe Rick's TSO patch is fixing > > > things > > > (sorry). In looking at my notes, it's possible I had lagg on for > > > those > > > tests. lagg does seem to negate the TSO patch in my case. > > > > > Ok, that's useful information. It implies that r251296 doesn't > > quite > > work and needs to be fixed for "stacked" network interface drivers > > before it can be used. I've cc'd Andre who is the author of that > > patch, in case he knows how to fix it. > > > > Thanks for checking this, rick > > > > > kernel.10stable_basicTSO_65535/ > > > > > > - IP_MAXPACKET = 65535; > > > - manually forced (no if statement) ifp->if_hw_tsomax = > > > IP_MAXPACKET > > > - > > > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > > > - Verified on boot via printf that ifp->if_hw_tsomax = 65517 > > > - Boot in a NON LAGG environment. ix0 only. > > > > > > ixgbe's printf is showing packets up to 65530. Haven't run long > > > enough yet > > > to see if anything will go over 65535 > > > > > > I have this tcpdump running to check packet size. > > > tcpdump -ennvvXS -i ix0 greater 65518 > > > > > > I do expect to get packets over 65518, but I was just curious to > > > see > > > if any > > > of them would go over 65535. Time will tell. > > > > > > In a separate test, If I enable lagg, we have LOTS of oversized > > > packet > > > problems. It looks like tsomax is definitely not making it > > > through in > > > if_lagg.c - Any recommendations there? I will eventually need > > > lagg, > > > as I'm > > > sure will others. > > > > > > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not > > > be > > > happening? > > > > > > > > > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && > > > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax: > > > %i\n", > > > args[0]->t_tsomax); stack(); }' > > > > > > > > > 6 31403 tcp_output:entry unexpected > > > tp->t_tsomax: > > > 65535 > > > > > > kernel`tcp_do_segment+0x2c99 > > > kernel`tcp_input+0x11a2 > > > kernel`ip_input+0xa2 > > > kernel`netisr_dispatch_src+0x5e > > > kernel`ether_demux+0x12a > > > kernel`ether_nh_input+0x35f > > > kernel`netisr_dispatch_src+0x5e > > > kernel`bce_intr+0x765 > > > kernel`intr_event_execute_handlers+0xab > > > kernel`ithread_loop+0x96 > > > kernel`fork_exit+0x9a > > > kernel`0xffffffff80c75b2e > > > > > > 3 31403 tcp_output:entry unexpected > > > tp->t_tsomax: > > > 65535 > > > > > > kernel`tcp_do_segment+0x2c99 > > > kernel`tcp_input+0x11a2 > > > kernel`ip_input+0xa2 > > > kernel`netisr_dispatch_src+0x5e > > > kernel`ether_demux+0x12a > > > kernel`ether_nh_input+0x35f > > > kernel`netisr_dispatch_src+0x5e > > > kernel`bce_intr+0x765 > > > kernel`intr_event_execute_handlers+0xab > > > kernel`ithread_loop+0x96 > > > kernel`fork_exit+0x9a > > > kernel`0xffffffff80c75b2e > > > > > > 6 31403 tcp_output:entry unexpected > > > tp->t_tsomax: > > > 65535 > > > > > > kernel`tcp_do_segment+0x2c99 > > > kernel`tcp_input+0x11a2 > > > kernel`ip_input+0xa2 > > > kernel`netisr_dispatch_src+0x5e > > > kernel`ether_demux+0x12a > > > kernel`ether_nh_input+0x35f > > > kernel`netisr_dispatch_src+0x5e > > > kernel`bce_intr+0x765 > > > kernel`intr_event_execute_handlers+0xab > > > kernel`ithread_loop+0x96 > > > kernel`fork_exit+0x9a > > > kernel`0xffffffff80c75b2e > > > > > > 1 31403 tcp_output:entry unexpected > > > tp->t_tsomax: > > > 65535 > > > > > > kernel`tcp_do_segment+0x2c99 > > > kernel`tcp_input+0x11a2 > > > kernel`ip_input+0xa2 > > > kernel`netisr_dispatch_src+0x5e > > > kernel`ether_demux+0x12a > > > kernel`ether_nh_input+0x35f > > > kernel`netisr_dispatch_src+0x5e > > > kernel`bce_intr+0x765 > > > kernel`intr_event_execute_handlers+0xab > > > kernel`ithread_loop+0x96 > > > kernel`fork_exit+0x9a > > > kernel`0xffffffff80c75b2e > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to > > > "freebsd-net-unsubscribe@freebsd.org" > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 02:45:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54089194; Thu, 27 Mar 2014 02:45:01 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9909CF16; Thu, 27 Mar 2014 02:45:00 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id k14so1957774wgh.23 for ; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=WxTzjm+ofTFvUdW3JPNglL8ykzwepfyCuwfZNI/YPEg=; b=EKPr2w+hTR6vs88hZhC6PFPf6I6eM/01VOGK3B14YBHAo2lb+XI+7xXczRU6pf+S7N dTD2nFOv5Y3jelyLhogcN+/u2I1KBDrp/SsrYJYkkhSDXzc/v16wNeAJfvfERYDZYG2u mtnzLsudIFbzRFFeS4p/OX2chVU1JICd7j80pqPuo6zhvZ2JRTBs57OtklKMMCm+5hF9 7O7VzmDZiZg44g3QIrlrwR8/f+yIHPYVai3RfIQpEdtrBVvLy8cT40HTb2WMggzgYuwj LpsdjqS7sm6vKkeaCyf/SDmXMT6aZ7xES+CT6ejjyEg6Z1/fPYD6EDEau8P3pGUetls+ jfqA== MIME-Version: 1.0 X-Received: by 10.180.89.102 with SMTP id bn6mr35999708wib.28.1395888298103; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) Received: by 10.216.190.199 with HTTP; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) In-Reply-To: <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> References: <20140326023334.GB2973@michelle.cdnetworks.com> <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 10:44:58 +0800 Message-ID: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem From: Marcelo Araujo To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 02:45:01 -0000 Hello All, 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > Well, bumping it from 32->35 is all it would take for NFS (can't comment > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 > (just so others aren't confused by the above comment). I understand > your point was w.r.t. using 100 without blowing the kernel stack, but > since the testers have been using "ix" with the 82599 chip which is > limited to 32 transmit segments... > > However, please increase any you know can be safely done from 32->35, rick > > I have plenty of machines using Intel X540 that is based on 82599 chipset. I have applied Rick's patch on ixgbe to check if the packet size is bigger than 65535 or cluster is bigger than 32. So far till now, on FreeBSD 9.1-RELEASE this problem does not happens. Unfortunately all my environment here is based on 9.1-RELEASE, with some merges from 10-RELEASE such like: NFS and IXGBE. Also I have applied the patch that Rick sent in another email with the subject 'NFS patch to use pagesize mbuf clusters'. And we can see some performance boost over 10Gbps Intel. However here at the company, we are still doing benchmarks. If someone wants to have my benchmark result, I can send it later. I'm wondering, if this update on ixgbe from 32->35 could be applied also for versions < 9.2. I'm thinking, that this problem arise only on 9-STABLE and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE doesn't share it. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 12:17:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14BE47BD; Thu, 27 Mar 2014 12:17:04 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B675F966; Thu, 27 Mar 2014 12:17:03 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id k15so3728530qaq.29 for ; Thu, 27 Mar 2014 05:17:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=soOHARjwUgZbz1MH4FhK5NOqAJwLa4NS17mbGOY3Sao=; b=u8KEbeg3jqvExyONwsr+7PsHhsAHXtva2QTCbjlFahaAdbfF4abRxXtrQyLjv42qwW Nccs5V+zwDKaHLH92JeZbYmY/fD/FXBQ4NPRlmKVzygwKIJWPFzr0SMFQYOtd4GGAeTy Ehi6ppuya7VgoM+9KB/LNBV8F2xUBQBwpuhWeVDyGOQRcPAAXBeCpE9fsR4vHDHo5Z+y e2sonHffIHZ3kmRW9ILDPnsNzwIc33lEPO2P9uX+o4IYgIKrXgjFBgiMxM6k3Dxa/kLP 1IQhhotSn6oICLy7shSVgf+OPBsfmFr17TzxCQgPhFRPB0woq9XtNeF50uIVgLdpfDr3 L53A== MIME-Version: 1.0 X-Received: by 10.224.57.72 with SMTP id b8mr1526381qah.41.1395922622731; Thu, 27 Mar 2014 05:17:02 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 27 Mar 2014 05:17:02 -0700 (PDT) In-Reply-To: <1353469098.1239101.1395880303531.JavaMail.root@uoguelph.ca> References: <1353469098.1239101.1395880303531.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 09:17:02 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 12:17:04 -0000 On Wed, Mar 26, 2014 at 9:31 PM, Rick Macklem wrote: > > ie. I've suggested: > ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + > ETHER_VLAN_ENCAP_LEN), > IP_MAXPACKET); > - I put the min() in just so it wouldn't break if MCLBYTES is increased > someday. > I like the added safety for future changes - Good forward thinking. I'll adjust my testing code to reflect this. From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 12:23:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67F548EC; Thu, 27 Mar 2014 12:23:54 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04892A14; Thu, 27 Mar 2014 12:23:53 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so2710231qgd.1 for ; Thu, 27 Mar 2014 05:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qiPMKOKB2htRhy6rthqOebVOOXumpe46T85jaGJwSG0=; b=gPtBiMBDW1WfKuqM3rCXFI0sEMv3roGZoTJbf278AZeJcSLzYW8k5Szt9rY51gEdwQ S482pr7T16ZaFI9UxLHTOeBZ2xQyvwnjtsA79vNgPUNwOeV0LfrEuICt496h2hfDpFnK WEADfCIUe7gukRVjlNRebB9rdoTSU61o6MJFWNpeoX2ZC8a7TwSJmY09k6b72gvwpbbf 9az9+T582RRntKknysJ3CAZcA0TbWS3iMgI62HZhJUNEZKCbgLwzqTacQTcWS9EM0ap6 w1Ki7QXW7e////ywBwsZgXwPKqtJoaW4XezLDjkleHOf17JApeR/TpA9HrvtxAeTslyo uhQQ== MIME-Version: 1.0 X-Received: by 10.140.34.46 with SMTP id k43mr1476914qgk.63.1395923033137; Thu, 27 Mar 2014 05:23:53 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 27 Mar 2014 05:23:52 -0700 (PDT) In-Reply-To: <1380107288.1240335.1395880548644.JavaMail.root@uoguelph.ca> References: <1380107288.1240335.1395880548644.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 09:23:52 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Garrett Wollman , Andre Oppermann , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 12:23:54 -0000 On Wed, Mar 26, 2014 at 9:35 PM, Rick Macklem wrote: > > > I've suggested in the other thread what you suggested in a recent > post...ie. to change the default, at least until the propagation > of driver set values is resolved. > > rick > I wonder if we need to worry about propagating values up from the sub-if's - Setting the default in if.c means this is set for all if's, and it's a simple 1 line code change. If a specific 'if' needs a different value, it can be set before ether_attach() is called. I'm more concerned with the equation we use to calculate if_hw_tsomax - Are we considering the right variables? Are we thinking on the wrong OSI layer for headers? From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 12:42:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49A78477; Thu, 27 Mar 2014 12:42:33 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8A9CBEC; Thu, 27 Mar 2014 12:42:32 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id f51so2699579qge.30 for ; Thu, 27 Mar 2014 05:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gfDL+vv4UPGWHYXuoDe+0WPwvQKOv8dbqRm4AT0K/mo=; b=tpKPaHW+XoyUIemJukrXVnlZSVqJAi3elCEz9L8X+klwfaQCpu/qQLG4a/TvSsGjhK D+yhqJWM+8T5PJESmBgHrlibRBI8/WU4v17qibm1f3vyo64vYA6sRQDxCqVNoYPO/LgN J1hzPbFOvt/deh69wNeXZfWpnxenZV1TSVWVIFuXGaJuPjHcbYCrKEfTEYwB9RiT/Ak8 NcTOCC6HfhKQUowM+jEw4HW/2GFuFnZSDM8bBcQmxTxvBx2QykuVkXrjHynKmecFDy4/ zhWld4SV1KFpVyol4H1Fcw7Rdv5qXBXTsaxrpyLgcrPudQOsVq3mL0zJjjFplVEE6/T6 g5Tw== MIME-Version: 1.0 X-Received: by 10.140.80.209 with SMTP id c75mr1538712qgd.79.1395924151988; Thu, 27 Mar 2014 05:42:31 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 27 Mar 2014 05:42:31 -0700 (PDT) In-Reply-To: References: <20140326023334.GB2973@michelle.cdnetworks.com> <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 09:42:31 -0300 Message-ID: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem From: Christopher Forgeron To: araujo@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , Rick Macklem , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 12:42:33 -0000 I'm quite sure the problem is on 9.2-RELEASE, not 9.1-RELEASE or earlier, as a 9.2-STABLE from last year I have doesn't exhibit the problem. New code in if.c at line 660 looks to be what is starting this, which makes me wonder how TSO was being handled before 9.2. I also like Rick's NFS patch for cluster size. I notice an improvement, but don't have solid numbers yet. I'm still stress testing it as we speak. On Wed, Mar 26, 2014 at 11:44 PM, Marcelo Araujo wrote: > Hello All, > > > 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > > > Well, bumping it from 32->35 is all it would take for NFS (can't comment > > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 > > (just so others aren't confused by the above comment). I understand > > your point was w.r.t. using 100 without blowing the kernel stack, but > > since the testers have been using "ix" with the 82599 chip which is > > limited to 32 transmit segments... > > > > However, please increase any you know can be safely done from 32->35, > rick > > > > > I have plenty of machines using Intel X540 that is based on 82599 chipset. > I have applied Rick's patch on ixgbe to check if the packet size is bigger > than 65535 or cluster is bigger than 32. So far till now, on FreeBSD > 9.1-RELEASE this problem does not happens. > > Unfortunately all my environment here is based on 9.1-RELEASE, with some > merges from 10-RELEASE such like: NFS and IXGBE. > > Also I have applied the patch that Rick sent in another email with the > subject 'NFS patch to use pagesize mbuf clusters'. And we can see some > performance boost over 10Gbps Intel. However here at the company, we are > still doing benchmarks. If someone wants to have my benchmark result, I can > send it later. > > I'm wondering, if this update on ixgbe from 32->35 could be applied also > for versions < 9.2. I'm thinking, that this problem arise only on 9-STABLE > and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE doesn't > share it. > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 14:14:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A214C2FD; Thu, 27 Mar 2014 14:14:06 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C4E169C; Thu, 27 Mar 2014 14:14:06 +0000 (UTC) Received: from [2001:1620:2013:1:2c52:e467:d98f:82e8] (port=56683) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WTB4E-000H5F-2O; Thu, 27 Mar 2014 15:14:02 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Thu, 27 Mar 2014 15:13:57 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5C2A730F-47B6-4E1D-8DA3-276E48DEA810@hostpoint.ch> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 14:14:06 -0000 On 26.03.2014, at 03:33, Christopher Forgeron = wrote: > On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert > wrote: >=20 >>=20 >>=20 >> Is 65517 correct? With Ricks patch, I get this: >>=20 >> dev.ix.0.hw_tsomax: 65518 >>=20 >=20 > Perhaps a difference between 9.2 and 10 for one of the macros? My = code is: >=20 > ifp->if_hw_tsomax =3D IP_MAXPACKET - (ETHER_HDR_LEN + = ETHER_VLAN_ENCAP_LEN); > printf("CSF - 3 Init, ifp->if_hw_tsomax =3D %d\n", = ifp->if_hw_tsomax); Hm, I=92m using Rick=92s patch: if ((adapter->num_segs * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN)) < IP_MAXPACKET) ifp->if_hw_tsomax =3D adapter->num_segs * = MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > (BTW, you should submit the hw_tsomax sysctl patch, that's useful to = others) My patch added a sysctl that is writable, but if I got this right = if_hw_tsomax is not expected to change after the interface is attached. = That=92s why I didn=92t post it. But here=92s a read-only version: --- sys/dev/ixgbe/ixgbe.c 2013-12-19 14:24:10.624279412 +0100 +++ sys/dev/ixgbe/ixgbe.c 2014-03-27 15:00:59.503424634 +0100 @@ -577,6 +582,12 @@ if (ixgbe_setup_interface(dev, adapter) !=3D 0) goto err_late; =20 + /* add interface to hw_tsomax */ + SYSCTL_ADD_INT(device_get_sysctl_ctx(dev), + SYSCTL_CHILDREN(device_get_sysctl_tree(dev)), + OID_AUTO, "hw_tsomax", CTLTYPE_INT|CTLFLAG_RD, + &adapter->ifp->if_hw_tsomax, 1, "hardware TSO limit"); + /* Initialize statistics */ ixgbe_update_stats_counters(adapter); =20 >> Also the dtrace command you used excludes 65518... >>=20 >=20 > Oh, I thought it was giving every packet that is greater than or equal = to > 65518 - Could you show me the proper command? That's the third time = I've > used dtrace, so I'm making this up as I go. :-) No, what looks like a comment (between slashes) are conditions in = dtrace: dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && = args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: %i\n", = args[0]->t_tsomax); stack(); }=92 You have to read the above like this: - fbt::tcp_output:entry -> Add a probe to the beginning of the kernel = function tcp_output() - / args[0]->t_tsomax !=3D 0 && args[0]->t_tsomax !=3D 65518 / -> only = match if t_tsomax is neither 0 nor 65518 (args[0] is struct tcpcb in = case of tcp_output()) - { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); = } -> this is only executed if the probe matched and the condition were = true. It that case a t_tsomax gets printed and a stack trace is = generated In your case, you stated that your if_hw_tsomax is 65517. Since my = version of the dtrace one-liner does _not_ ignore 65517, you should have = seen a lot of output, which you didn=92t mention (you=92ve just posted = dtrace output that was generated from bce interfaces). That=92s why I = thought 65517 was a typo on your part, and I wanted to clarify that. Markus From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 16:48:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 047B639D for ; Thu, 27 Mar 2014 16:48:21 +0000 (UTC) Received: from smtp.domeneshop.no (smtp.domeneshop.no [194.63.252.54]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAE578BA for ; Thu, 27 Mar 2014 16:48:20 +0000 (UTC) Received: from uib-guest.uib.no ([129.177.138.114]:58447 helo=[10.103.254.196]) by smtp.domeneshop.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1WTD5o-0004iA-A5 for freebsd-net@freebsd.org; Thu, 27 Mar 2014 17:23:48 +0100 Message-ID: <5334509B.8010706@sande.im> Date: Thu, 27 Mar 2014 17:23:55 +0100 From: Mikal Sande User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: FreeBSD Net Subject: ipv6 neighbor cache question Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 16:48:21 -0000 Is the IPv6 neighbor cache supposed to not inlcude incomplete entries? When my freebsd box resolves a previously unknown ipv6 address with ndp it does not add anything to the neighbor cache before it gets a reachability confirmation. I have viewed the neighbor cache with ndp -a. The ipv6 addresses in question are local, so I am pretty sure that on-link determination is not interfering. The same thing happens with both link-local and global addresses. When I ping an unused ipv6 address I do not find any corresponding incomplete entry in the neighbor cache afterwards. But, if I ping an unused ipv4 address i do find an incomplete entry in the arp cache. I am curious as to why this behavior occurs. Is it intentional? Is it by design? The reason for my curiosity is that I have not observed this behavior in other OSes such as linux and openbsd. My box is currently running. FreeBSD hostname 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 -- Mikal Sande From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 21:37:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB12B133; Thu, 27 Mar 2014 21:37:44 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3F1CF9BF; Thu, 27 Mar 2014 21:37:43 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqsEAN+YNFODaFve/2dsb2JhbABZg0FXgwq4HYYZTVGBNXSCJQEBAQMBAQEBIAQnIAsbDgoCAg0ZAiMGAQkmBggHBAEcBIdEAwkIDa5ymxoNhysXgSmLNIE7CgYCARs0B4JvgUkElXZqgyCLN4VKg0shMXtC X-IronPort-AV: E=Sophos;i="4.97,745,1389762000"; d="scan'208";a="109827248" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 27 Mar 2014 17:37:36 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 72614B404C; Thu, 27 Mar 2014 17:37:36 -0400 (EDT) Date: Thu, 27 Mar 2014 17:37:36 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <380071870.1832898.1395956256461.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , araujo@freebsd.org, FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 21:37:44 -0000 Christopher Forgeron wrote: > I'm quite sure the problem is on 9.2-RELEASE, not 9.1-RELEASE or > earlier, > as a 9.2-STABLE from last year I have doesn't exhibit the problem. > New > code in if.c at line 660 looks to be what is starting this, which > makes me > wonder how TSO was being handled before 9.2. > > I also like Rick's NFS patch for cluster size. I notice an > improvement, but > don't have solid numbers yet. I'm still stress testing it as we > speak. > Unfortunately, this causes problems for small i386 systems, so I am reluctant to commit it to head. Maybe a variant that is only enabled for amd64 systems with lots of memory would be ok? > > On Wed, Mar 26, 2014 at 11:44 PM, Marcelo Araujo > wrote: > > > Hello All, > > > > > > 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > > > > > Well, bumping it from 32->35 is all it would take for NFS (can't > > > comment > > > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the > > > 82599 > > > (just so others aren't confused by the above comment). I > > > understand > > > your point was w.r.t. using 100 without blowing the kernel stack, > > > but > > > since the testers have been using "ix" with the 82599 chip which > > > is > > > limited to 32 transmit segments... > > > > > > However, please increase any you know can be safely done from > > > 32->35, > > rick > > > > > > > > I have plenty of machines using Intel X540 that is based on 82599 > > chipset. > > I have applied Rick's patch on ixgbe to check if the packet size is > > bigger > > than 65535 or cluster is bigger than 32. So far till now, on > > FreeBSD > > 9.1-RELEASE this problem does not happens. > > > > Unfortunately all my environment here is based on 9.1-RELEASE, with > > some > > merges from 10-RELEASE such like: NFS and IXGBE. > > I can't see why it couldn't happen on 9.1 or earlier, since it just uses IP_MAXPACKET in tcp_output(). However, to make it happen NFS has to do a read reply (server) or write request (client) that is a little under 64K bytes. Normally the default will be a full 64K bytes, so for the server side it would take a read of a file where the EOF is just shy of the 64K boundary. For the client write it would be a write of a partially dirtied block where most of the block has been dirtied. (Software builds generate a lot of patially dirtied blocks, but I don't know what else would. For sequential writing it would be a file that ends just shy of a 64K boundary (similar to the server side) being written. I think it is more likely your NFS file activity and not 9.1 vs 9.2 that avoids the problem. (I suspect there are quite a few folk running NFS 9.2 or later on these ix chips who don't see the problem as well.) Fortunately you (Christopher) were able to reproduce it, so the problem could be isolated. Thanks everyone for your help with this, rick > > Also I have applied the patch that Rick sent in another email with > > the > > subject 'NFS patch to use pagesize mbuf clusters'. And we can see > > some > > performance boost over 10Gbps Intel. However here at the company, > > we are > > still doing benchmarks. If someone wants to have my benchmark > > result, I can > > send it later. > > > > I'm wondering, if this update on ixgbe from 32->35 could be applied > > also > > for versions < 9.2. I'm thinking, that this problem arise only on > > 9-STABLE > > and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE > > doesn't > > share it. > > > > Best Regards, > > -- > > Marcelo Araujo > > araujo@FreeBSD.org > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 22:14:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB692B7F; Thu, 27 Mar 2014 22:14:48 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3F194D5F; Thu, 27 Mar 2014 22:14:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqoEAEKiNFODaFve/2dsb2JhbABZg0FXgwq4HYYZTVGBN3SCJQEBAQMBAQEBIAQnIAsFFhgCAg0ZAikBCSYOBwQBHASHUAgNrn2iVReBKYxvCgYCARs0BxaCWYFJBJV2hAqRAYNLITF7Qg X-IronPort-AV: E=Sophos;i="4.97,745,1389762000"; d="scan'208";a="109838808" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 27 Mar 2014 18:14:47 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1270EB3F0B; Thu, 27 Mar 2014 18:14:47 -0400 (EDT) Date: Thu, 27 Mar 2014 18:14:47 -0400 (EDT) From: Rick Macklem To: araujo@FreeBSD.org Message-ID: <1882337939.1849245.1395958487066.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 22:14:48 -0000 Marcelo Araujo wrote: > Hello All, > > > 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > > > Well, bumping it from 32->35 is all it would take for NFS (can't > > comment > > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the > > 82599 > > (just so others aren't confused by the above comment). I understand > > your point was w.r.t. using 100 without blowing the kernel stack, > > but > > since the testers have been using "ix" with the 82599 chip which is > > limited to 32 transmit segments... > > > > However, please increase any you know can be safely done from > > 32->35, rick > > > > > I have plenty of machines using Intel X540 that is based on 82599 > chipset. > I have applied Rick's patch on ixgbe to check if the packet size is > bigger > than 65535 or cluster is bigger than 32. So far till now, on FreeBSD > 9.1-RELEASE this problem does not happens. > > Unfortunately all my environment here is based on 9.1-RELEASE, with > some > merges from 10-RELEASE such like: NFS and IXGBE. > > Also I have applied the patch that Rick sent in another email with > the > subject 'NFS patch to use pagesize mbuf clusters'. And we can see > some > performance boost over 10Gbps Intel. However here at the company, we > are > still doing benchmarks. If someone wants to have my benchmark result, > I can > send it later. > > I'm wondering, if this update on ixgbe from 32->35 could be applied > also > for versions < 9.2. I'm thinking, that this problem arise only on > 9-STABLE > and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE > doesn't > share it. > My understanding is that the 32 limitation is a hardware one for the 82599. It appears that other drivers than the ixgbe.c can be increased from 32->35, but not ixgbe.c (for the 82599 chips). rick > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 22:44:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F85DB14; Thu, 27 Mar 2014 22:44:02 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 2D901F9; Thu, 27 Mar 2014 22:44:01 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEABmpNFODaFve/2dsb2JhbABZhBiDCr9UgTB0giUBAQEEIwRSGw4KERkCBFUGE4d5rnyiUheOIyMZGweCb4FJBJBVmiyDSyGBLUE X-IronPort-AV: E=Sophos;i="4.97,745,1389762000"; d="scan'208";a="109674004" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 27 Mar 2014 18:44:01 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 0495CB3F4A; Thu, 27 Mar 2014 18:44:01 -0400 (EDT) Date: Thu, 27 Mar 2014 18:44:01 -0400 (EDT) From: Rick Macklem To: Christopher Forgeron Message-ID: <1292881633.1858906.1395960241007.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1858903_1764427006.1395960241003" X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Pyun YongHyeon , Andre Oppermann , FreeBSD Net , Markus Gebert , Jack Vogel , Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 22:44:02 -0000 ------=_Part_1858903_1764427006.1395960241003 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > > > > > > > On Wed, Mar 26, 2014 at 9:35 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > > I've suggested in the other thread what you suggested in a recent > post...ie. to change the default, at least until the propagation > of driver set values is resolved. > > rick > > > > I wonder if we need to worry about propagating values up from the > sub-if's - Setting the default in if.c means this is set for all > if's, and it's a simple 1 line code change. If a specific 'if' needs > a different value, it can be set before ether_attach() is called. > > > I'm more concerned with the equation we use to calculate if_hw_tsomax > - Are we considering the right variables? Are we thinking on the > wrong OSI layer for headers? > Well, I'm pragmatic (which means I mostly care about some fix that works), but it seems to me that: - The problem is that some TSO enabled network drivers/hardware can only handle 32 transmit segments (or 32 mbufs in the chain for the TSO packet to be transmitted, if that is clearer). --> Since the problem is in certain drivers, it seems that those drivers should be where the long term fix goes. --> Since some hardware can't handle more than 32, it seems that the driver should be able to specify that limit, which tcp_output() can then apply. I have an untested patch that does this by adding if_hw_tsomaxseg. (The attachment called tsomaxseg.patch.) Changing if_hw_tsomax or its default value is just a hack that gets tcp_output() to apply a limit that the driver can then fix to 32 mbufs in the chain via m_defrag(). Since if_hw_tsomax (and if_hw_tsomaxseg in the untested patch) aren't propagated up through lagg, that needs to be fixed. (Yet another attached untested patch called lagg.patch.) As I said before, I don't see these patches getting tested/reviewed etc in time for 9.3, so I think reducing the default value of if_hw_tsomax is a reasonable short term hack to work around the problem. (And it sounds like Pyun YongHyeon has volunteered to fix many of the drivers, where the 32 limit isn't a hardware one.) rick ------=_Part_1858903_1764427006.1395960241003 Content-Type: text/x-patch; name=tsomaxseg.patch Content-Disposition: attachment; filename=tsomaxseg.patch Content-Transfer-Encoding: base64 LS0tIGtlcm4vdWlwY19zb2NrYnVmLmMuc2F2CTIwMTQtMDEtMzAgMjA6Mjc6MTcuMDAwMDAwMDAw IC0wNTAwCisrKyBrZXJuL3VpcGNfc29ja2J1Zi5jCTIwMTQtMDEtMzAgMjI6MTI6MDguMDAwMDAw MDAwIC0wNTAwCkBAIC05NjUsNiArOTY1LDM5IEBAIHNic25kcHRyKHN0cnVjdCBzb2NrYnVmICpz YiwgdV9pbnQgb2ZmLCAKIH0KIAogLyoKKyAqIFJldHVybiB0aGUgZmlyc3QgbWJ1ZiBmb3IgdGhl IHByb3ZpZGVkIG9mZnNldC4KKyAqLworc3RydWN0IG1idWYgKgorc2JzbmRtYnVmKHN0cnVjdCBz b2NrYnVmICpzYiwgdV9pbnQgb2ZmLCBsb25nICpmaXJzdF9sZW4pCit7CisJc3RydWN0IG1idWYg Km07CisKKwlLQVNTRVJUKHNiLT5zYl9tYiAhPSBOVUxMLCAoIiVzOiBzYl9tYiBpcyBOVUxMIiwg X19mdW5jX18pKTsKKworCSpmaXJzdF9sZW4gPSAwOworCS8qCisJICogSXMgb2ZmIGJlbG93IHN0 b3JlZCBvZmZzZXQ/IEhhcHBlbnMgb24gcmV0cmFuc21pdHMuCisJICogSWYgc28sIGp1c3QgdXNl IHNiX21iLgorCSAqLworCWlmIChzYi0+c2Jfc25kcHRyID09IE5VTEwgfHwgc2ItPnNiX3NuZHB0 cm9mZiA+IG9mZikKKwkJbSA9IHNiLT5zYl9tYjsKKwllbHNlIHsKKwkJbSA9IHNiLT5zYl9zbmRw dHI7CisJCW9mZiAtPSBzYi0+c2Jfc25kcHRyb2ZmOworCX0KKwl3aGlsZSAob2ZmID4gMCAmJiBt ICE9IE5VTEwpIHsKKwkJaWYgKG9mZiA8IG0tPm1fbGVuKQorCQkJYnJlYWs7CisJCW9mZiAtPSBt LT5tX2xlbjsKKwkJbSA9IG0tPm1fbmV4dDsKKwl9CisJaWYgKG0gIT0gTlVMTCkKKwkJKmZpcnN0 X2xlbiA9IG0tPm1fbGVuIC0gb2ZmOworCisJcmV0dXJuIChtKTsKK30KKworLyoKICAqIERyb3Ag YSByZWNvcmQgb2ZmIHRoZSBmcm9udCBvZiBhIHNvY2tidWYgYW5kIG1vdmUgdGhlIG5leHQgcmVj b3JkIHRvIHRoZQogICogZnJvbnQuCiAgKi8KLS0tIHN5cy9zb2NrYnVmLmguc2F2CTIwMTQtMDEt MzAgMjA6NDI6MjguMDAwMDAwMDAwIC0wNTAwCisrKyBzeXMvc29ja2J1Zi5oCTIwMTQtMDEtMzAg MjI6MDg6NDMuMDAwMDAwMDAwIC0wNTAwCkBAIC0xNTMsNiArMTUzLDggQEAgaW50CXNicmVzZXJ2 ZV9sb2NrZWQoc3RydWN0IHNvY2tidWYgKnNiLAogCSAgICBzdHJ1Y3QgdGhyZWFkICp0ZCk7CiBz dHJ1Y3QgbWJ1ZiAqCiAJc2JzbmRwdHIoc3RydWN0IHNvY2tidWYgKnNiLCB1X2ludCBvZmYsIHVf aW50IGxlbiwgdV9pbnQgKm1vZmYpOworc3RydWN0IG1idWYgKgorCXNic25kbWJ1ZihzdHJ1Y3Qg c29ja2J1ZiAqc2IsIHVfaW50IG9mZiwgbG9uZyAqZmlyc3RfbGVuKTsKIHZvaWQJc2J0b3hzb2Nr YnVmKHN0cnVjdCBzb2NrYnVmICpzYiwgc3RydWN0IHhzb2NrYnVmICp4c2IpOwogaW50CXNid2Fp dChzdHJ1Y3Qgc29ja2J1ZiAqc2IpOwogaW50CXNibG9jayhzdHJ1Y3Qgc29ja2J1ZiAqc2IsIGlu dCBmbGFncyk7Ci0tLSBuZXRpbmV0L3RjcF9pbnB1dC5jLnNhdgkyMDE0LTAxLTMwIDE5OjM3OjUy LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3BfaW5wdXQuYwkyMDE0LTAxLTMwIDE5OjM5 OjA3LjAwMDAwMDAwMCAtMDUwMApAQCAtMzYyNyw2ICszNjI3LDcgQEAgdGNwX21zcyhzdHJ1Y3Qg dGNwY2IgKnRwLCBpbnQgb2ZmZXIpCiAJaWYgKGNhcC5pZmNhcCAmIENTVU1fVFNPKSB7CiAJCXRw LT50X2ZsYWdzIHw9IFRGX1RTTzsKIAkJdHAtPnRfdHNvbWF4ID0gY2FwLnRzb21heDsKKwkJdHAt PnRfdHNvbWF4c2VncyA9IGNhcC50c29tYXhzZWdzOwogCX0KIH0KIAotLS0gbmV0aW5ldC90Y3Bf b3V0cHV0LmMuc2F2CTIwMTQtMDEtMzAgMTg6NTU6MTUuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRp bmV0L3RjcF9vdXRwdXQuYwkyMDE0LTAxLTMwIDIyOjE4OjU2LjAwMDAwMDAwMCAtMDUwMApAQCAt MTY2LDggKzE2Niw4IEBAIGludAogdGNwX291dHB1dChzdHJ1Y3QgdGNwY2IgKnRwKQogewogCXN0 cnVjdCBzb2NrZXQgKnNvID0gdHAtPnRfaW5wY2ItPmlucF9zb2NrZXQ7Ci0JbG9uZyBsZW4sIHJl Y3dpbiwgc2VuZHdpbjsKLQlpbnQgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBp bGVyIGhhcHB5ICovCisJbG9uZyBsZW4sIHJlY3dpbiwgc2VuZHdpbiwgdHNvX3RsZW47CisJaW50 IGNudCwgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBpbGVyIGhhcHB5ICovCiAJ c3RydWN0IG1idWYgKm07CiAJc3RydWN0IGlwICppcCA9IE5VTEw7CiAJc3RydWN0IGlwb3ZseSAq aXBvdiA9IE5VTEw7CkBAIC03ODAsNiArNzgwLDI0IEBAIHNlbmQ6CiAJCQl9CiAKIAkJCS8qCisJ CQkgKiBMaW1pdCB0aGUgbnVtYmVyIG9mIFRTTyB0cmFuc21pdCBzZWdtZW50cyAobWJ1ZnMKKwkJ CSAqIGluIG1idWYgbGlzdCkgdG8gdHAtPnRfdHNvbWF4c2Vncy4KKwkJCSAqLworCQkJY250ID0g MDsKKwkJCW0gPSBzYnNuZG1idWYoJnNvLT5zb19zbmQsIG9mZiwgJnRzb190bGVuKTsKKwkJCXdo aWxlIChtICE9IE5VTEwgJiYgY250IDwgdHAtPnRfdHNvbWF4c2VncyAmJgorCQkJICAgIHRzb190 bGVuIDwgbGVuKSB7CisJCQkJaWYgKGNudCA+IDApCisJCQkJCXRzb190bGVuICs9IG0tPm1fbGVu OworCQkJCWNudCsrOworCQkJCW0gPSBtLT5tX25leHQ7CisJCQl9CisJCQlpZiAobSAhPSBOVUxM ICYmIHRzb190bGVuIDwgbGVuKSB7CisJCQkJbGVuID0gdHNvX3RsZW47CisJCQkJc2VuZGFsb3Qg PSAxOworCQkJfQorCisJCQkvKgogCQkJICogUHJldmVudCB0aGUgbGFzdCBzZWdtZW50IGZyb20g YmVpbmcKIAkJCSAqIGZyYWN0aW9uYWwgdW5sZXNzIHRoZSBzZW5kIHNvY2tidWYgY2FuCiAJCQkg KiBiZSBlbXB0aWVkLgotLS0gbmV0aW5ldC90Y3Bfc3Vici5jLnNhdgkyMDE0LTAxLTMwIDE5OjQ0 OjM1LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3Bfc3Vici5jCTIwMTQtMDEtMzAgMjA6 NTY6MTIuMDAwMDAwMDAwIC0wNTAwCkBAIC0xODAwLDYgKzE4MDAsMTIgQEAgdGNwX21heG10dShz dHJ1Y3QgaW5fY29ubmluZm8gKmluYywgc3RydQogCQkJICAgIGlmcC0+aWZfaHdhc3Npc3QgJiBD U1VNX1RTTykKIAkJCQljYXAtPmlmY2FwIHw9IENTVU1fVFNPOwogCQkJCWNhcC0+dHNvbWF4ID0g aWZwLT5pZl9od190c29tYXg7CisjaWZkZWYgbm90eWV0CisJCQkJY2FwLT50c29tYXhzZWdzID0g aWZwLT5pZl9od190c29tYXhzZWdzOworI2VuZGlmCisJCQkJaWYgKGNhcC0+dHNvbWF4c2VncyA9 PSAwKQorCQkJCQljYXAtPnRzb21heHNlZ3MgPQorCQkJCQkgICAgVENQVFNPX01BWF9UWF9TRUdT X0RFRkFVTFQ7CiAJCX0KIAkJUlRGUkVFKHNyby5yb19ydCk7CiAJfQotLS0gbmV0aW5ldC90Y3Bf dmFyLmguc2F2CTIwMTQtMDEtMzAgMTk6Mzk6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRpbmV0 L3RjcF92YXIuaAkyMDE0LTAxLTMwIDIwOjUyOjU3LjAwMDAwMDAwMCAtMDUwMApAQCAtMjA5LDYg KzIwOSw3IEBAIHN0cnVjdCB0Y3BjYiB7CiAJdV9pbnQJdF9rZWVwY250OwkJLyogbnVtYmVyIG9m IGtlZXBhbGl2ZXMgYmVmb3JlIGNsb3NlICovCiAKIAl1X2ludAl0X3Rzb21heDsJCS8qIHRzbyBi dXJzdCBsZW5ndGggbGltaXQgKi8KKwl1X2ludAl0X3Rzb21heHNlZ3M7CQkvKiB0c28gYnVyc3Qg c2VnbWVudCBsaW1pdCAqLwogCiAJdWludDMyX3QgdF9pc3BhcmVbOF07CQkvKiA1IFVUTywgMyBU QkQgKi8KIAl2b2lkCSp0X3BzcGFyZTJbNF07CQkvKiA0IFRCRCAqLwpAQCAtMjY4LDYgKzI2OSwx MSBAQCBzdHJ1Y3QgdGNwY2IgewogI2RlZmluZQlUQ1BPT0JfSEFWRURBVEEJMHgwMQogI2RlZmlu ZQlUQ1BPT0JfSEFEREFUQQkweDAyCiAKKy8qCisgKiBEZWZhdWx0IHZhbHVlIGZvciBUU08gbWF4 aW11bSBudW1iZXIgb2YgdHJhbnNtaXQgc2VnbWVudHMgKGNvdW50IG9mIG1idWZzKS4KKyAqLwor I2RlZmluZQlUQ1BUU09fTUFYX1RYX1NFR1NfREVGQVVMVAkzMAorCiAjaWZkZWYgVENQX1NJR05B VFVSRQogLyoKICAqIERlZmluZXMgd2hpY2ggYXJlIG5lZWRlZCBieSB0aGUgeGZvcm1fdGNwIG1v ZHVsZSBhbmQgdGNwX1tpbnxvdXRdcHV0CkBAIC0zMzMsNiArMzM5LDcgQEAgc3RydWN0IGhjX21l dHJpY3NfbGl0ZSB7CS8qIG11c3Qgc3RheSBpbgogc3RydWN0IHRjcF9pZmNhcCB7CiAJaW50CWlm Y2FwOwogCXVfaW50CXRzb21heDsKKwl1X2ludAl0c29tYXhzZWdzOwogfTsKIAogI2lmbmRlZiBf TkVUSU5FVF9JTl9QQ0JfSF8K ------=_Part_1858903_1764427006.1395960241003 Content-Type: text/x-patch; name=lagg.patch Content-Disposition: attachment; filename=lagg.patch Content-Transfer-Encoding: base64 LS0tIG5ldC9pZl9sYWdnLmMuc2F2CTIwMTQtMDMtMjYgMjE6MDQ6MzYuMDAwMDAwMDAwIC0wNDAw CisrKyBuZXQvaWZfbGFnZy5jCTIwMTQtMDMtMjYgMjE6MDc6NTcuMDAwMDAwMDAwIC0wNDAwCkBA IC00MjUsNiArNDI1LDcgQEAgbGFnZ19jYXBhYmlsaXRpZXMoc3RydWN0IGxhZ2dfc29mdGMgKnNj KQogCXN0cnVjdCBsYWdnX3BvcnQgKmxwOwogCWludCBjYXAgPSB+MCwgZW5hID0gfjA7CiAJdV9s b25nIGh3YSA9IH4wVUw7CisJdV9pbnQgaHdfdHNvbWF4ID0gSVBfTUFYUEFDS0VUOwogCiAJTEFH R19XTE9DS19BU1NFUlQoc2MpOwogCkBAIC00MzMsNiArNDM0LDggQEAgbGFnZ19jYXBhYmlsaXRp ZXMoc3RydWN0IGxhZ2dfc29mdGMgKnNjKQogCQljYXAgJj0gbHAtPmxwX2lmcC0+aWZfY2FwYWJp bGl0aWVzOwogCQllbmEgJj0gbHAtPmxwX2lmcC0+aWZfY2FwZW5hYmxlOwogCQlod2EgJj0gbHAt PmxwX2lmcC0+aWZfaHdhc3Npc3Q7CisJCWlmIChscC0+bHBfaWZwLT5pZl9od190c29tYXggPCBo d190c29tYXgpCisJCQlod190c29tYXggPSBscC0+bHBfaWZwLT5pZl9od190c29tYXg7CiAJfQog CWNhcCA9IChjYXAgPT0gfjAgPyAwIDogY2FwKTsKIAllbmEgPSAoZW5hID09IH4wID8gMCA6IGVu YSk7CkBAIC00NDAsMTAgKzQ0MywxMiBAQCBsYWdnX2NhcGFiaWxpdGllcyhzdHJ1Y3QgbGFnZ19z b2Z0YyAqc2MpCiAKIAlpZiAoc2MtPnNjX2lmcC0+aWZfY2FwYWJpbGl0aWVzICE9IGNhcCB8fAog CSAgICBzYy0+c2NfaWZwLT5pZl9jYXBlbmFibGUgIT0gZW5hIHx8Ci0JICAgIHNjLT5zY19pZnAt PmlmX2h3YXNzaXN0ICE9IGh3YSkgeworCSAgICBzYy0+c2NfaWZwLT5pZl9od2Fzc2lzdCAhPSBo d2EgfHwKKwkgICAgc2MtPnNjX2lmcC0+aWZfaHdfdHNvbWF4ICE9IGh3X3Rzb21heCkgewogCQlz Yy0+c2NfaWZwLT5pZl9jYXBhYmlsaXRpZXMgPSBjYXA7CiAJCXNjLT5zY19pZnAtPmlmX2NhcGVu YWJsZSA9IGVuYTsKIAkJc2MtPnNjX2lmcC0+aWZfaHdhc3Npc3QgPSBod2E7CisJCXNjLT5zY19p ZnAtPmlmX2h3X3Rzb21heCA9IGh3X3Rzb21heDsKIAkJZ2V0bWljcm90aW1lKCZzYy0+c2NfaWZw LT5pZl9sYXN0Y2hhbmdlKTsKIAogCQlpZiAoc2MtPnNjX2lmZmxhZ3MgJiBJRkZfREVCVUcpCg== ------=_Part_1858903_1764427006.1395960241003-- From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 02:22:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60AAE3E4; Fri, 28 Mar 2014 02:22:59 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1CEA9DC; Fri, 28 Mar 2014 02:22:58 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id k14so3072483wgh.34 for ; Thu, 27 Mar 2014 19:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=NPFp+tus8ErKwhknLJ2A80qU2/J7j8uPgzmKaAojwWg=; b=Dr11GtEjnRUDTfSDjzEGC4qxB1Xv26RlhLjVxSPSlIYIjeZa4dka4uC98KyHP97ciK KC80mqUi/WOUxJjsQj6E2sUFPPOt0o75XgsB9RQq/Z8qSl7U3BM52Ahz29RsNjh50nXp 29Qiz163E4mIYq3aduNJxSi8OIrsi7DRrTX43e0fmN7VTPnpfmnZ76mEiNkCngD3LtfH 37wi2Q/2KohBbne0f90BNP2AOG8/jLJed1VgWW92zaEwgQcLgTleXuTWeaHju4j+P0T5 zGCDWhDG5cH4YFYPS+y+uzbcLVuVheQlrEX4ii9l9LNm8QC7hPZf907WiRjVB3JtBUPL B0pQ== MIME-Version: 1.0 X-Received: by 10.180.94.196 with SMTP id de4mr9118830wib.16.1395973376922; Thu, 27 Mar 2014 19:22:56 -0700 (PDT) Received: by 10.216.190.199 with HTTP; Thu, 27 Mar 2014 19:22:56 -0700 (PDT) In-Reply-To: <380071870.1832898.1395956256461.JavaMail.root@uoguelph.ca> References: <380071870.1832898.1395956256461.JavaMail.root@uoguelph.ca> Date: Fri, 28 Mar 2014 10:22:56 +0800 Message-ID: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem From: Marcelo Araujo To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , FreeBSD Net , Christopher Forgeron X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 02:22:59 -0000 2014-03-28 5:37 GMT+08:00 Rick Macklem : > Christopher Forgeron wrote: > > I'm quite sure the problem is on 9.2-RELEASE, not 9.1-RELEASE or > > earlier, > > as a 9.2-STABLE from last year I have doesn't exhibit the problem. > > New > > code in if.c at line 660 looks to be what is starting this, which > > makes me > > wonder how TSO was being handled before 9.2. > > > > I also like Rick's NFS patch for cluster size. I notice an > > improvement, but > > don't have solid numbers yet. I'm still stress testing it as we > > speak. > > > Unfortunately, this causes problems for small i386 systems, so I > am reluctant to commit it to head. Maybe a variant that is only > enabled for amd64 systems with lots of memory would be ok? > > Rick, Maybe you can create a SYSCTL to enable/disable it by the end user will be more reasonable. Also, of course, it is so far safe if only 64Bits CPU can enable this SYSCTL. Any other option seems not OK, will be hard to judge what is lots of memory and what is not, it will depends what is running onto the system. The SYSCTL will be great, and in case you don't have time to do it, I can give you a hand. I'm gonna do more benchmarks today and will send another report, but in our product here, I'm inclined to use this patch, because 10~20% speed up in read for me is a lot. :-) Thank you so much and best regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 03:27:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A029581; Fri, 28 Mar 2014 03:27:10 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C74A7F2C; Fri, 28 Mar 2014 03:27:09 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s2S3R2fS008218 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Mar 2014 20:27:02 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s2S3R11C008217; Thu, 27 Mar 2014 20:27:01 -0700 (PDT) (envelope-from jmg) Date: Thu, 27 Mar 2014 20:27:01 -0700 From: John-Mark Gurney To: Willy Offermans Subject: Re: sendmail Broken Pipe Error Message-ID: <20140328032701.GG60889@funkthat.com> Mail-Followup-To: Willy Offermans , freebsd-current@freebsd.org, freebsd-net@freebsd.org References: <20140325103934.GH8104@vpn.offrom.nl> <20140325164316.GC32089@funkthat.com> <20140326111748.GJ27307@vpn.offrom.nl> <20140326162034.GI60889@funkthat.com> <20140326172206.GB4119@vpn.offrom.nl> <20140326230427.GR60889@funkthat.com> <20140327144609.GI3611@vpn.offrom.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140327144609.GI3611@vpn.offrom.nl> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 27 Mar 2014 20:27:03 -0700 (PDT) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 03:27:10 -0000 Willy Offermans wrote this message on Thu, Mar 27, 2014 at 15:46 +0100: > Hello John-Mark and FreeBSD friends, > > On Wed, Mar 26, 2014 at 04:04:27PM -0700, John-Mark Gurney wrote: > > Willy Offermans wrote this message on Wed, Mar 26, 2014 at 18:22 +0100: > > > Hello John-Mark and FreeBSD friends, > > > > > > On Wed, Mar 26, 2014 at 09:20:35AM -0700, John-Mark Gurney wrote: > > > > Willy Offermans wrote this message on Wed, Mar 26, 2014 at 12:17 +0100: > > > > > On Tue, Mar 25, 2014 at 09:43:16AM -0700, John-Mark Gurney wrote: > > > > > > Willy Offermans wrote this message on Tue, Mar 25, 2014 at 11:39 +0100: > > > > > > > I'm not an expert in tcpdump. Can anyone make sense out of the messages? > > > > > > > > > > > > If you dumped the contents, using -s 0 -X, and look at that last packet > > > > > > you should see 0d 0a 2e 0d 0a at the end.. which is CR/LF/./CR/LF.. If > > > > > > you don't see that, then for some reason sendmail/FreeBSD isn't telling > > > > > > the server that it's done sending which would prevent the receiving > > > > > > side from ack'ing the email causing the timeout... > > > > > > > > > > I followed your suggestions. However I'm not able to distinguish the last > > > > > packet. Is there a way to find this with help of the Flags? The following > > > > > is the output of tcpdump -r /root/tmp/tcpdump -X | grep Flags > > > > > > > > > > 11:57:56.539788 IP MyServer.com.41115 > Smarthost.com.smtp: Flags [S], seq 1001452351, win 65535, options [mss 1448,nop,wscale 6,sackOK,TS val 407239960 ecr 0], length 0 > > > > > 11:57:56.555262 IP Smarthost.com.smtp > MyServer.com.41115: Flags [S.], seq 1277075046, ack 1001452352, win 8192, options [mss 1452], length 0 > > > > > > > > It should look something like: > > > > 09:18:34.723280 IP jmgmac.funkthat.com.64724 > h2.funkthat.com.ssh: Flags [.], ack 177, win 33280, options [nop,nop,TS val 1854905469 ecr 3482476972], length 0 > > > > 0x0000: 4510 0034 d7ac 4000 4006 e1af c0a8 0003 E..4..@.@....... > > > > 0x0010: c0a8 0004 fcd4 0016 7e48 238e d872 43dc ........~H#..rC. > > > > 0x0020: 8010 8200 7c08 0000 0101 080a 6e8f 9c7d ....|.......n..} > > > > 0x0030: cf92 61ac ..a. > > > > > > > > Notice the hex output... I didn't see any of that in your output... > > > > The last packet I was talking about is the last one that had length > > > > 1448 that your server sent... > > > > > > I sent two e-mails consecutively: the first without an attachment, the > > > second with attachment. I dumped tcp of the NIC for port smtp. I got the > > > following: > > > > > > > > > 12:20:55.988622 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [P.], seq 18943:19104, ack 412, win 65535, length 161 > > > 0x0000: 4500 00c9 eebd 4000 4006 0000 c0a8 0004 E.....@.@....... > > > 0x0010: d54b 3f0d 9147 0019 4ea0 36dd 15a7 38a0 .K?..G..N.6...8. > > > 0x0020: 5018 ffff 4481 0000 2020 2020 2020 2020 P...D........... > > > 0x0030: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > 0x0040: 2020 2020 2020 2020 2020 2020 2020 205c ...............\ > > > 0x0050: 2f20 205c 205e 0d0a 2020 2020 2020 2020 /..\.^.......... > > > 0x0060: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > 0x0070: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > 0x0080: 2020 202e 5c2e 5f2f 5f29 0d0a 0d0a 2020 ....\._/_)...... > > > 0x0090: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > 0x00a0: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > 0x00b0: 2020 2020 2077 7777 2e46 7265 6542 5344 .....www.FreeBSD > > > 0x00c0: 2e6f 7267 0d0a 2e0d 0a .org..... > > > > > > As predicted by John-Mark, the first ended with "0d0a 2e0d 0a". However it > > > was not the last packet with length 1448. I hope that this will not spoil > > > the party. Is the Flag [P.] more indicative? It looks like to me, but I'm > > > just learning. > > > > > > Anyway the second mail ended with: > > > > > > 12:22:17.960896 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [.], seq 35127:36575, ack 638, win 65535, length 1448 > > > 0x0000: 4500 05d0 fe9d 4000 4006 0000 c0a8 0004 E.....@.@....... > > > > > > > > > 0x0560: 5670 6876 4a67 5a5a 5a50 4b2f 4b78 3774 VphvJgZZZPK/Kx7t > > > 0x0570: 382f 4230 594f 6b78 3449 0d0a 4a76 6551 8/B0YOkx4I..JveQ > > > 0x0580: 2b6e 7765 5647 2f33 6e79 6231 6133 496f +nweVG/3nyb1a3Io > > > 0x0590: 5474 554f 4d61 4374 696b 714b 436b 4959 TtUOMaCtikqKCkIY > > > 0x05a0: 704a 7668 3055 416d 6c33 4754 4f4c 6455 pJvh0UAml3GTOLdU > > > 0x05b0: 774b 4145 7151 5741 7841 4141 5a66 7647 wKAEqQWAxAAAZfvG > > > 0x05c0: 706b 6c36 0d0a 7a4e 6234 745a 6633 5a6c pkl6..zNb4tZf3Zl > > > > > > Being packet with length 1448 and sent from my side. The code "0d0a 2e0d > > > 0a" is missing. Immediately thereafter the following packets: > > > > We clearly haven't gotten the last mime-boundary, we are still in the > > base64 encoded data of the attachment... > > > > > 12:22:18.003557 IP Smarthost.com.smtp > MyServer.com.37191: Flags [.], ack 36575, win 65160, length 0 > > > 0x0000: 4500 0028 11fb 4000 7a06 19d0 d54b 3f0d E..(..@.z....K?. > > > 0x0010: c0a8 0004 0019 9147 15a7 3982 4ea0 7bbd .......G..9.N.{. > > > 0x0020: 5010 fe88 315f 0000 0000 0000 0000 P...1_........ > > > > The remote acking that it got your last packet... > > > > > 12:22:37.665889 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [R.], seq 39471, ack 638, win 65535, length 0 > > > 0x0000: 4500 0028 492c 4000 4006 0000 c0a8 0004 E..(I,@.@....... > > > 0x0010: d54b 3f0d 9147 0019 4ea0 870d 15a7 3982 .K?..G..N.....9. > > > 0x0020: 5014 ffff 2494 0000 P...$... > > > > Local host closing down the connection because of time out... > > > > > 12:22:37.680857 IP Smarthost.com.smtp > MyServer.com.37191: Flags [.], ack 36575, win 65160, length 0 > > > 0x0000: 4500 0028 0584 0000 f906 e746 d54b 3f0d E..(.......F.K?. > > > 0x0010: c0a8 0004 0019 9147 15a7 3982 4ea0 7bbd .......G..9.N.{. > > > 0x0020: 5010 fe88 315f 0000 0000 0000 0000 P...1_........ > > > 12:22:37.680920 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [R], seq 1319140285, win 0, length 0 > > > 0x0000: 4500 0028 4935 4000 4006 0000 c0a8 0004 E..(I5@.@....... > > > 0x0010: d54b 3f0d 9147 0019 4ea0 7bbd 0000 0000 .K?..G..N.{..... > > > 0x0020: 5004 0000 7f1d 0000 P....... > > > > > > It looks like Smarthost.com asks for more, but there is not more to sent. > > > The final packet seems to be absent. I cannot look for the closing remark > > > "www.FreeBSD.org", since there is the attachment at the end of the second > > > mail. Is there any connection between the encoded attachment in the second > > > mail and the output of tcpdump? > > > > > > Am I the only one noticing this error? I can hardly believe this. > > > > > > Is there a way to force the insertion of .? > > > > So, this is more looking like a kernel problem where the last packet(s) > > aren't being sent out... Could you possibly catch the output of > > netstat -anfinet of the connection between the last packet and the > > reset? It'll be interesting to see if there is data in the send-q > > for the connection (third column)... If it's zero, that seems to imply > > that the server process hasn't sent all the data necessary... Then > > the next bit of investigation would be to run ktrace on the sendmail > > process and make sure that it writes all the correct data to the > > socket... > > > > Do you know what OS the remote side is running? You could use nmap > > to try to figure it out, as it could be a TCP stack interaction issue.. > > netstat -anfinet gave: > > tcp4 0 33304 MyServerIP.35395 SmarthostIP.25 ESTABLISHED > > So there is still data to be sent, if I interpret correctly. What is next > to be investigated? Yeh, if there is data in the send-q (per above) and you aren't seeing any more packets, someone on -net with some TCP clue should help you debug this... Also, did you figure out what the OS is of the other end? Knowing that will also help them... For -net's reference, the opening syns look like: 11:57:56.539788 IP MyServer.com.41115 > Smarthost.com.smtp: Flags [S], seq 1001452351, win 65535, options [mss 1448,nop,wscale 6,sackOK,TS val 407239960 ecr 0], length 0 11:57:56.555262 IP Smarthost.com.smtp > MyServer.com.41115: Flags [S.], seq 1277075046, ack 1001452352, win 8192, options [mss 1452], length 0 -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 11:13:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0486A445; Fri, 28 Mar 2014 11:13:17 +0000 (UTC) Received: from cpsmtpb-ews06.kpnxchange.com (cpsmtpb-ews06.kpnxchange.com [213.75.39.9]) by mx1.freebsd.org (Postfix) with ESMTP id 5A0C9D90; Fri, 28 Mar 2014 11:13:16 +0000 (UTC) Received: from cpsps-ews14.kpnxchange.com ([10.94.84.181]) by cpsmtpb-ews06.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Fri, 28 Mar 2014 12:12:05 +0100 Received: from CPSMTPM-CMT103.kpnxchange.com ([195.121.3.19]) by cpsps-ews14.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Fri, 28 Mar 2014 12:12:05 +0100 Received: from donald.offrom.nl ([77.170.60.162]) by CPSMTPM-CMT103.kpnxchange.com with Microsoft SMTPSVC(7.0.6002.18264); Fri, 28 Mar 2014 12:11:12 +0100 Received: from squid (squid.vpn.offrom.nl [10.168.0.72]) by donald.offrom.nl (8.14.7/8.14.7) with ESMTP id s2SBBBTA004406; Fri, 28 Mar 2014 12:11:11 +0100 (CET) (envelope-from willy@vpn.offrom.nl) Received: from willy by squid with local (Exim 4.72) (envelope-from ) id 1WTUgk-0006W5-No; Fri, 28 Mar 2014 12:11:06 +0100 Date: Fri, 28 Mar 2014 12:11:06 +0100 From: Willy Offermans To: freebsd-current@freebsd.org, freebsd-net@freebsd.org Subject: Re: sendmail Broken Pipe Error Message-ID: <20140328111106.GG3781@vpn.offrom.nl> References: <20140325103934.GH8104@vpn.offrom.nl> <20140325164316.GC32089@funkthat.com> <20140326111748.GJ27307@vpn.offrom.nl> <20140326162034.GI60889@funkthat.com> <20140326172206.GB4119@vpn.offrom.nl> <20140326230427.GR60889@funkthat.com> <20140327144609.GI3611@vpn.offrom.nl> <20140328032701.GG60889@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140328032701.GG60889@funkthat.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 28 Mar 2014 11:11:12.0900 (UTC) FILETIME=[6DFE1C40:01CF4A76] X-RcptDomain: freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Willy@Offermans.Rompen.nl List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 11:13:17 -0000 Hello John-Mark and FreeBSD friends, On Thu, Mar 27, 2014 at 08:27:01PM -0700, John-Mark Gurney wrote: > Willy Offermans wrote this message on Thu, Mar 27, 2014 at 15:46 +0100: > > Hello John-Mark and FreeBSD friends, > > > > On Wed, Mar 26, 2014 at 04:04:27PM -0700, John-Mark Gurney wrote: > > > Willy Offermans wrote this message on Wed, Mar 26, 2014 at 18:22 +0100: > > > > Hello John-Mark and FreeBSD friends, > > > > > > > > On Wed, Mar 26, 2014 at 09:20:35AM -0700, John-Mark Gurney wrote: > > > > > Willy Offermans wrote this message on Wed, Mar 26, 2014 at 12:17 +0100: > > > > > > On Tue, Mar 25, 2014 at 09:43:16AM -0700, John-Mark Gurney wrote: > > > > > > > Willy Offermans wrote this message on Tue, Mar 25, 2014 at 11:39 +0100: > > > > > > > > I'm not an expert in tcpdump. Can anyone make sense out of the messages? > > > > > > > > > > > > > > If you dumped the contents, using -s 0 -X, and look at that last packet > > > > > > > you should see 0d 0a 2e 0d 0a at the end.. which is CR/LF/./CR/LF.. If > > > > > > > you don't see that, then for some reason sendmail/FreeBSD isn't telling > > > > > > > the server that it's done sending which would prevent the receiving > > > > > > > side from ack'ing the email causing the timeout... > > > > > > > > > > > > I followed your suggestions. However I'm not able to distinguish the last > > > > > > packet. Is there a way to find this with help of the Flags? The following > > > > > > is the output of tcpdump -r /root/tmp/tcpdump -X | grep Flags > > > > > > > > > > > > 11:57:56.539788 IP MyServer.com.41115 > Smarthost.com.smtp: Flags [S], seq 1001452351, win 65535, options [mss 1448,nop,wscale 6,sackOK,TS val 407239960 ecr 0], length 0 > > > > > > 11:57:56.555262 IP Smarthost.com.smtp > MyServer.com.41115: Flags [S.], seq 1277075046, ack 1001452352, win 8192, options [mss 1452], length 0 > > > > > > > > > > It should look something like: > > > > > 09:18:34.723280 IP jmgmac.funkthat.com.64724 > h2.funkthat.com.ssh: Flags [.], ack 177, win 33280, options [nop,nop,TS val 1854905469 ecr 3482476972], length 0 > > > > > 0x0000: 4510 0034 d7ac 4000 4006 e1af c0a8 0003 E..4..@.@....... > > > > > 0x0010: c0a8 0004 fcd4 0016 7e48 238e d872 43dc ........~H#..rC. > > > > > 0x0020: 8010 8200 7c08 0000 0101 080a 6e8f 9c7d ....|.......n..} > > > > > 0x0030: cf92 61ac ..a. > > > > > > > > > > Notice the hex output... I didn't see any of that in your output... > > > > > The last packet I was talking about is the last one that had length > > > > > 1448 that your server sent... > > > > > > > > I sent two e-mails consecutively: the first without an attachment, the > > > > second with attachment. I dumped tcp of the NIC for port smtp. I got the > > > > following: > > > > > > > > > > > > 12:20:55.988622 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [P.], seq 18943:19104, ack 412, win 65535, length 161 > > > > 0x0000: 4500 00c9 eebd 4000 4006 0000 c0a8 0004 E.....@.@....... > > > > 0x0010: d54b 3f0d 9147 0019 4ea0 36dd 15a7 38a0 .K?..G..N.6...8. > > > > 0x0020: 5018 ffff 4481 0000 2020 2020 2020 2020 P...D........... > > > > 0x0030: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > > 0x0040: 2020 2020 2020 2020 2020 2020 2020 205c ...............\ > > > > 0x0050: 2f20 205c 205e 0d0a 2020 2020 2020 2020 /..\.^.......... > > > > 0x0060: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > > 0x0070: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > > 0x0080: 2020 202e 5c2e 5f2f 5f29 0d0a 0d0a 2020 ....\._/_)...... > > > > 0x0090: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > > 0x00a0: 2020 2020 2020 2020 2020 2020 2020 2020 ................ > > > > 0x00b0: 2020 2020 2077 7777 2e46 7265 6542 5344 .....www.FreeBSD > > > > 0x00c0: 2e6f 7267 0d0a 2e0d 0a .org..... > > > > > > > > As predicted by John-Mark, the first ended with "0d0a 2e0d 0a". However it > > > > was not the last packet with length 1448. I hope that this will not spoil > > > > the party. Is the Flag [P.] more indicative? It looks like to me, but I'm > > > > just learning. > > > > > > > > Anyway the second mail ended with: > > > > > > > > 12:22:17.960896 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [.], seq 35127:36575, ack 638, win 65535, length 1448 > > > > 0x0000: 4500 05d0 fe9d 4000 4006 0000 c0a8 0004 E.....@.@....... > > > > > > > > > > > > 0x0560: 5670 6876 4a67 5a5a 5a50 4b2f 4b78 3774 VphvJgZZZPK/Kx7t > > > > 0x0570: 382f 4230 594f 6b78 3449 0d0a 4a76 6551 8/B0YOkx4I..JveQ > > > > 0x0580: 2b6e 7765 5647 2f33 6e79 6231 6133 496f +nweVG/3nyb1a3Io > > > > 0x0590: 5474 554f 4d61 4374 696b 714b 436b 4959 TtUOMaCtikqKCkIY > > > > 0x05a0: 704a 7668 3055 416d 6c33 4754 4f4c 6455 pJvh0UAml3GTOLdU > > > > 0x05b0: 774b 4145 7151 5741 7841 4141 5a66 7647 wKAEqQWAxAAAZfvG > > > > 0x05c0: 706b 6c36 0d0a 7a4e 6234 745a 6633 5a6c pkl6..zNb4tZf3Zl > > > > > > > > Being packet with length 1448 and sent from my side. The code "0d0a 2e0d > > > > 0a" is missing. Immediately thereafter the following packets: > > > > > > We clearly haven't gotten the last mime-boundary, we are still in the > > > base64 encoded data of the attachment... > > > > > > > 12:22:18.003557 IP Smarthost.com.smtp > MyServer.com.37191: Flags [.], ack 36575, win 65160, length 0 > > > > 0x0000: 4500 0028 11fb 4000 7a06 19d0 d54b 3f0d E..(..@.z....K?. > > > > 0x0010: c0a8 0004 0019 9147 15a7 3982 4ea0 7bbd .......G..9.N.{. > > > > 0x0020: 5010 fe88 315f 0000 0000 0000 0000 P...1_........ > > > > > > The remote acking that it got your last packet... > > > > > > > 12:22:37.665889 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [R.], seq 39471, ack 638, win 65535, length 0 > > > > 0x0000: 4500 0028 492c 4000 4006 0000 c0a8 0004 E..(I,@.@....... > > > > 0x0010: d54b 3f0d 9147 0019 4ea0 870d 15a7 3982 .K?..G..N.....9. > > > > 0x0020: 5014 ffff 2494 0000 P...$... > > > > > > Local host closing down the connection because of time out... > > > > > > > 12:22:37.680857 IP Smarthost.com.smtp > MyServer.com.37191: Flags [.], ack 36575, win 65160, length 0 > > > > 0x0000: 4500 0028 0584 0000 f906 e746 d54b 3f0d E..(.......F.K?. > > > > 0x0010: c0a8 0004 0019 9147 15a7 3982 4ea0 7bbd .......G..9.N.{. > > > > 0x0020: 5010 fe88 315f 0000 0000 0000 0000 P...1_........ > > > > 12:22:37.680920 IP MyServer.com.37191 > Smarthost.com.smtp: Flags [R], seq 1319140285, win 0, length 0 > > > > 0x0000: 4500 0028 4935 4000 4006 0000 c0a8 0004 E..(I5@.@....... > > > > 0x0010: d54b 3f0d 9147 0019 4ea0 7bbd 0000 0000 .K?..G..N.{..... > > > > 0x0020: 5004 0000 7f1d 0000 P....... > > > > > > > > It looks like Smarthost.com asks for more, but there is not more to sent. > > > > The final packet seems to be absent. I cannot look for the closing remark > > > > "www.FreeBSD.org", since there is the attachment at the end of the second > > > > mail. Is there any connection between the encoded attachment in the second > > > > mail and the output of tcpdump? > > > > > > > > Am I the only one noticing this error? I can hardly believe this. > > > > > > > > Is there a way to force the insertion of .? > > > > > > So, this is more looking like a kernel problem where the last packet(s) > > > aren't being sent out... Could you possibly catch the output of > > > netstat -anfinet of the connection between the last packet and the > > > reset? It'll be interesting to see if there is data in the send-q > > > for the connection (third column)... If it's zero, that seems to imply > > > that the server process hasn't sent all the data necessary... Then > > > the next bit of investigation would be to run ktrace on the sendmail > > > process and make sure that it writes all the correct data to the > > > socket... > > > > > > Do you know what OS the remote side is running? You could use nmap > > > to try to figure it out, as it could be a TCP stack interaction issue.. > > > > netstat -anfinet gave: > > > > tcp4 0 33304 MyServerIP.35395 SmarthostIP.25 ESTABLISHED > > > > So there is still data to be sent, if I interpret correctly. What is next > > to be investigated? > > Yeh, if there is data in the send-q (per above) and you aren't seeing > any more packets, someone on -net with some TCP clue should help you > debug this... > > Also, did you figure out what the OS is of the other end? Knowing > that will also help them... > > For -net's reference, the opening syns look like: > 11:57:56.539788 IP MyServer.com.41115 > Smarthost.com.smtp: Flags [S], seq 1001452351, win 65535, options [mss 1448,nop,wscale 6,sackOK,TS val 407239960 ecr 0], length 0 > 11:57:56.555262 IP Smarthost.com.smtp > MyServer.com.41115: Flags [S.], seq 1277075046, ack 1001452352, win 8192, options [mss 1452], length 0 > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" I did some more investigations and I found the following: 1) The error depends on the size of the e-mail not the presence or absence of an attachment. I put rfc3028.txt into a test mail and try to sent it. It got stuck in the mail server with eventually the same Broken pipe/time out error as previously discussed. 2) I noticed a substantial increase in CPU usage of natd and dhcpd when the e-mail is being processed by the mail server. >From top: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1235 root 1 93 0 28908K 2144K RUN 0 54:05 71.78% natd 1614 dhcpd 1 4 0 26784K 14936K RUN 0 29:24 38.77% dhcpd The idle cpu load of these services is ca. 0.00%. 3) I experienced extremely slow copy processes with scp not related with e-mail processing. 4) All other processes run smoothly to my notice and that makes it tricky to discover. 5) I tried to figure out the OS on the Smarthost: Nmap scan report for Smarthost.com (SmarthostIP) Host is up (0.023s latency). Not shown: 994 closed ports PORT STATE SERVICE 25/tcp open smtp 135/tcp filtered msrpc 139/tcp filtered netbios-ssn 445/tcp filtered microsoft-ds 1720/tcp filtered H.323/Q.931 5555/tcp filtered freeciv This looks like some kind of MS OS to me. *) So I get the feeling that the encountered problems are somehow hardware related or related to my settings. Can someone suggest a good description of my problem? -- Met vriendelijke groeten, With kind regards, Mit freundlichen Gruessen, De jrus wah, Wiel ************************************* W.K. Offermans e-mail: Willy@Offermans.Rompen.nl Powered by .... (__) \\\'',) \/ \ ^ .\._/_) www.FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 16:20:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15C6AE63 for ; Fri, 28 Mar 2014 16:20:14 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9647F62 for ; Fri, 28 Mar 2014 16:20:13 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id il7so6153840vcb.32 for ; Fri, 28 Mar 2014 09:20:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=kDxBOibfGVMlMBblLHe3QJGu7n6ttbeADWkAberAor0=; b=Rbad6Hw2VH7A/eqnrcepAviuYk0WsXLgmd6GZoxaACWcs617rEkF10xqQ61/4/123/ JPsHLmC+r+xJ7P9Dv9zXgOZXOAatOyx4KzCGdQk4DYjnVh2buQEItsg41zmQ/c48GP/2 gOOgJyJqjKiBMnNraPLHswLSd0dqSeJUB3Sr1+MCUDUmQaxXR54lhdlU3yS1ol2Yg4Sl X45VTFxOIlCu6zmwvA5OZYfW8S7+0E2ILsShGEZ0UVZNopaTBP1Jx0H6bk4LqGRkmk6/ uigzSqobikzq7uSB7/vMlvXyvdempxytdjEK/52NZ5yngtU/ZkDr8u5gg33/gvHOXUKZ Pmdg== MIME-Version: 1.0 X-Received: by 10.58.229.167 with SMTP id sr7mr7692752vec.7.1396023612917; Fri, 28 Mar 2014 09:20:12 -0700 (PDT) Sender: jinmei.tatuya@gmail.com Received: by 10.220.173.133 with HTTP; Fri, 28 Mar 2014 09:20:12 -0700 (PDT) In-Reply-To: <5334509B.8010706@sande.im> References: <5334509B.8010706@sande.im> Date: Fri, 28 Mar 2014 09:20:12 -0700 X-Google-Sender-Auth: Sa5Hs6xxRI9LZp0mdgZFpdYrdhw Message-ID: Subject: Re: ipv6 neighbor cache question From: =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: Mikal Sande Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 16:20:14 -0000 At Thu, 27 Mar 2014 17:23:55 +0100, Mikal Sande wrote: > Is the IPv6 neighbor cache supposed to not inlcude incomplete entries? Wh= en my freebsd box resolves a previously unknown ipv6 address with ndp it do= es not add anything to the neighbor cache before it gets a reachability con= firmation. I have viewed the neighbor cache with ndp -a. > > The ipv6 addresses in question are local, so I am pretty sure that on-lin= k determination is not interfering. The same thing happens with both link-l= ocal and global addresses. > > When I ping an unused ipv6 address I do not find any corresponding incomp= lete entry in the neighbor cache afterwards. But, if I ping an unused ipv4 = address i do find an incomplete entry in the arp cache. I am curious as to = why this behavior occurs. Is it intentional? Is it by design? > > The reason for my curiosity is that I have not observed this behavior in = other OSes such as linux and openbsd. I suspect that's something specific to recent versions of FreeBSD. The very original kernel neighbor cache and ndp implementations of the KAME project should have behaved as you expected above and actually saw with OpenBSD. I don't know if the change was intentional or a kind of defect, though. Hopefully someone more familiar with recent updates in FreeBSD can clarify that. -- JINMEI, Tatuya From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 16:26:20 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29B49E1; Fri, 28 Mar 2014 16:26:20 +0000 (UTC) Received: from cpsmtpb-ews04.kpnxchange.com (cpsmtpb-ews04.kpnxchange.com [213.75.39.7]) by mx1.freebsd.org (Postfix) with ESMTP id 92FEC80; Fri, 28 Mar 2014 16:26:16 +0000 (UTC) Received: from cpsps-ews15.kpnxchange.com ([10.94.84.182]) by cpsmtpb-ews04.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Fri, 28 Mar 2014 17:26:06 +0100 Received: from CPSMTPM-CMT105.kpnxchange.com ([195.121.3.21]) by cpsps-ews15.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Fri, 28 Mar 2014 17:26:06 +0100 Received: from donald.offrom.nl ([77.170.60.162]) by CPSMTPM-CMT105.kpnxchange.com with Microsoft SMTPSVC(7.0.6002.18264); Fri, 28 Mar 2014 17:26:02 +0100 Received: from squid (squid.vpn.offrom.nl [10.168.0.72]) by donald.offrom.nl (8.14.7/8.14.7) with ESMTP id s2SGPxsV005209; Fri, 28 Mar 2014 17:26:00 +0100 (CET) (envelope-from willy@vpn.offrom.nl) Received: from willy by squid with local (Exim 4.72) (envelope-from ) id 1WTZbO-000704-9n; Fri, 28 Mar 2014 17:25:54 +0100 Date: Fri, 28 Mar 2014 17:25:54 +0100 From: Willy Offermans To: freebsd-stable@FreeBSD.ORG, freebsd-net@FreeBSD.ORG Subject: TCP packets remain unsent Message-ID: <20140328162554.GA26748@vpn.offrom.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 28 Mar 2014 16:26:02.0786 (UTC) FILETIME=[693DF820:01CF4AA2] X-RcptDomain: FreeBSD.ORG X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Willy@Offermans.Rompen.nl List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 16:26:20 -0000 Dear FreeBSD friends, I have a problem with my relatively new FreeBSD server. I came across the problem when sending e-mails of larger size and copying files with scp. The e-mails were not sent out because of time-out error and the copying was extremely slow, though successful after a while. I already started a thread on this topic on freebsd-current. See http://docs.freebsd.org/mail/current/freebsd-current.html, topic sendmail Broken Pipe Error. I got some help to narrow down the error: Sending out e-mails of larger size stops at some point. TCP packets were not transferred to the smarthost causing a timeout error. There were still some TCP packets waiting to be sent. My system is a HP ProLiant Gen8 MicroServer with FreeBSD 10.0-STABLE #0 r261266M. The server has two network cards: bge0@pci0:3:0:0: class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme BCM5720 Gigabit Ethernet PCIe' class = network subclass = ethernet bge1@pci0:3:0:1: class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme BCM5720 Gigabit Ethernet PCIe' class = network subclass = ethernet I do not know if there any known issues with these cards (drivers). Before the time out error occurs, the CPU loading of natd and dhcpd is steadily increasing to extreme values to my opinion: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1235 root 1 93 0 28908K 2144K RUN 0 54:05 71.78% natd 1614 dhcpd 1 4 0 26784K 14936K RUN 0 29:24 38.77% dhcpd I followed an advice of another FreeBSD friend to modify tcp_input.c ... You may want to locally apply SVN r258821 to sys/netinet/tcp_input.c, in case it has not been merged back to the FreeBSD version you use: --- sys/netinet/tcp_input.c (revision 258820) +++ sys/netinet/tcp_input.c (revision 258821) @@ -2429,13 +2429,15 @@ hhook_run_tcp_est_in(tp, th, &to); if (SEQ_LEQ(th->th_ack, tp->snd_una)) { - if (tlen == 0 && tiwin == tp->snd_wnd) { + if (tlen == 0 && tiwin == tp->snd_wnd && + !(thflags & TH_FIN)) { TCPSTAT_INC(tcps_rcvdupack); /* * If we have outstanding data (other than * a window probe), this is a completely * duplicate ack (ie, window info didn't - * change), the ack is the biggest we've + * change and FIN isn't set), + * the ack is the biggest we've * seen and we've seen exactly our rexmt * threshhold of them, assume a packet * has been dropped and retransmit it. ... However this did not solve the issue. So my question to you is if someone else encountered a similar problem with NetXtreme BCM5720 Gigabit Ethernet PCIe and if someone is able to take this up and to help me to solve this issue. I would love to run this FreeBSD server as a swiss clock as other beastie servers, I have setup in the past. -- Met vriendelijke groeten, With kind regards, Mit freundlichen Gruessen, De jrus wah, Wiel ************************************* W.K. Offermans e-mail: Willy@Offermans.Rompen.nl Powered by .... (__) \\\'',) \/ \ ^ .\._/_) www.FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 18:01:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C507C11 for ; Fri, 28 Mar 2014 18:01:39 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36CF8E2B for ; Fri, 28 Mar 2014 18:01:38 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id C0316139C8 for ; Fri, 28 Mar 2014 18:04:14 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1396040652; x=1396904653; bh=pTWtJhqTdmuPKLX3VkpDdAXXWh3VHE+CTbU PDUMO6us=; b=ZIhBYWYqjHHrAi9Dtqr7C8SC/lt1y8I5Krbqx8eoKW64/nQNmBV btFoCnNEEyjvawze9C1G1jqbt5WeIZrJJoOJNrAQz0UP64Oi3/ZaWLlRheMR3a2D sqY4slsIKjYL2ztg19naX2qDPBm02zWf7uxiYeVR/I2kuUY2HgdxFfss= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ui52G7Y4AUdO for ; Fri, 28 Mar 2014 18:04:12 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 3B406139C3 for ; Fri, 28 Mar 2014 18:04:12 -0300 (BRT) Message-ID: <5335B8F7.7050001@bsdinfo.com.br> Date: Fri, 28 Mar 2014 15:01:27 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: sshd with zombie process on FreeBSD 10.0-STABLE Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 18:01:39 -0000 Hi all, I have noticed zombie processes on the system after a few lost connections on ssh. # ps afx [...] 8045 - Is 0:00.01 sshd: unknown [priv] (sshd) 8046 - Z 0:00.01 8054 - IW 0:00.00 sshd: unknown [pam] (sshd) 28146 - Is 0:00.01 sshd: unknown [priv] (sshd) 28147 - Z 0:00.01 28155 - IW 0:00.00 sshd: unknown [pam] (sshd) 43320 - Is 0:00.01 sshd: unknown [priv] (sshd) 43321 - Z 0:00.01 43322 - IW 0:00.00 sshd: unknown [pam] (sshd) 73413 - Is 0:00.01 sshd: unknown [priv] (sshd) 73414 - Z 0:00.01 73430 - IW 0:00.00 sshd: unknown [pam] (sshd) [...] More information discussed in freebsd-stable@list: On Tue, Mar 25, 2014 at 3:52 AM, Karl Pielorz > wrote: --On 21 March 2014 22:02 -0700 Kevin Oberman > wrote: Ideally I'd look to try and capture the packets st the end of the session. Can you do something to trigger this reliably? if so "standard" "tcpdump -pw file.bpf host HOST". I seem to recall that these connections are scheduled. If so, you can put the packet capture in a crontab to run at the same time. If you feed this to a tool like wireshark, you should get a good idea of what is happening, if not why. I understand that the timing of this might be very tricky. Ok, fwiw as I have this issue as well - I've done a packet capture (it's below). This box has 59 'CLOSED' sockets from ssh on it, and 60 sshd stuck. Also - initially I thought this was a Xen issue - so there's a couple of posts on that list from a couple of weeks ago, in brief - the sshd processes I have are stuck in 'urdlck' - one of the Xen guys commented "It seems like the process is stuck while trying to acquire a rw mutex in read mode." I did a backtrace of a stuck process - I can post that if you want (or check the FreeBSD-Xen list for 'stuck sshd in urdlck'. Also, if I ssh into this host, 90% of the time (seems to get worse the longer the box is up) I get: " ssh_exchange_identification: Connection closed by remote host " That does not leave a lingering CLOSED socket. In fact, a successful ssh login - and logout, does not in testing appear to leave a lingering CLOSED socket, nor sshd stuck in urdlck - so I'm not entirely sure where they're coming from, or how often they are created. tcpdump from the start of a successful ssh connect: 10:47:04.777765 IP (tos 0x0, ttl 61, id 4058, offset 0, flags [DF], proto TCP (6), length 60) 192.168.0.37.31139 > 192.168.0.138.22: Flags [S], cksum 0x57ce (correct), seq 634709832, win 65535, options [mss 1368,nop,wscale 3,sackOK,TS val 1019391535 ecr 0], length 0 10:47:04.777776 IP (tos 0x0, ttl 64, id 10060, offset 0, flags [DF], proto TCP (6), length 60) 192.168.0.138.22 > 192.168.0.37.31139: Flags [S.], cksum 0x7ef8 (incorrect -> 0x69d8), seq 2316386788, ack 634709833, win 65535, options [mss 1368,nop,wscale 6,sackOK,TS val 805368299 ecr 1019391535], length 0 10:47:04.804218 IP (tos 0x0, ttl 61, id 4059, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x77d3 (correct), ack 1, win 8305, options [nop,nop,TS val 1019391538 ecr 805368299], length 0 10:47:04.809692 IP (tos 0x0, ttl 64, id 10061, offset 0, flags [DF], proto TCP (6), length 99) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f1f (incorrect -> 0x5799), seq 1:48, ack 1, win 1038, options [nop,nop,TS val 805368328 ecr 1019391538], length 47 10:47:04.836110 IP (tos 0x0, ttl 61, id 4060, offset 0, flags [DF], proto TCP (6), length 92) 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x8afa (correct), seq 1:41, ack 48, win 8305, options [nop,nop,TS val 1019391541 ecr 805368328], length 40 10:47:04.836669 IP (tos 0x0, ttl 64, id 10062, offset 0, flags [DF], proto TCP (6), length 1596) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x84f8 (incorrect -> 0x4d42), seq 48:1592, ack 41, win 1038, options [nop,nop,TS val 805368358 ecr 1019391541], length 1544 And the end of the session: 10:47:15.243540 IP (tos 0x10, ttl 61, id 4132, offset 0, flags [DF], proto TCP (6), length 100) 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x6364 (correct), seq 2321:2369, ack 3520, win 8305, options [nop,nop,TS val 1019392582 ecr 805375068], length 48 10:47:15.243736 IP (tos 0x10, ttl 64, id 10125, offset 0, flags [DF], proto TCP (6), length 100) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f20 (incorrect -> 0x3ea8), seq 3520:3568, ack 2369, win 1038, options [nop,nop,TS val 805378765 ecr 1019392582], length 48 10:47:15.243796 IP (tos 0x10, ttl 64, id 10126, offset 0, flags [DF], proto TCP (6), length 100) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f20 (incorrect -> 0xdd22), seq 3568:3616, ack 2369, win 1038, options [nop,nop,TS val 805378765 ecr 1019392582], length 48 10:47:15.244627 IP (tos 0x10, ttl 64, id 10127, offset 0, flags [DF], proto TCP (6), length 84) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f10 (incorrect -> 0x86ed), seq 3616:3648, ack 2369, win 1038, options [nop,nop,TS val 805378765 ecr 1019392582], length 32 10:47:15.244812 IP (tos 0x10, ttl 64, id 10128, offset 0, flags [DF], proto TCP (6), length 212) 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f90 (incorrect -> 0x431c), seq 3648:3808, ack 2369, win 1038, options [nop,nop,TS val 805378765 ecr 1019392582], length 160 10:47:15.271439 IP (tos 0x10, ttl 61, id 4134, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x3381 (correct), ack 3616, win 8299, options [nop,nop,TS val 1019392585 ecr 805378765], length 0 10:47:15.272238 IP (tos 0x10, ttl 61, id 4135, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x32d9 (correct), ack 3808, win 8275, options [nop,nop,TS val 1019392585 ecr 805378765], length 0 10:47:15.273515 IP (tos 0x10, ttl 61, id 4137, offset 0, flags [DF], proto TCP (6), length 84) 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x43cc (correct), seq 2369:2401, ack 3808, win 8305, options [nop,nop,TS val 1019392585 ecr 805378765], length 32 10:47:15.276199 IP (tos 0x10, ttl 61, id 4138, offset 0, flags [DF], proto TCP (6), length 116) 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0xb169 (correct), seq 2401:2465, ack 3808, win 8305, options [nop,nop,TS val 1019392585 ecr 805378765], length 64 10:47:15.276220 IP (tos 0x10, ttl 64, id 10129, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.138.22 > 192.168.0.37.31139: Flags [.], cksum 0x7ef0 (incorrect -> 0x4ea3), ack 2465, win 1037, options [nop,nop,TS val 805378793 ecr 1019392585], length 0 10:47:15.276970 IP (tos 0x10, ttl 61, id 4140, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.37.31139 > 192.168.0.138.22: Flags [F.], cksum 0x325a (correct), seq 2465, ack 3808, win 8305, options [nop,nop,TS val 1019392585 ecr 805378765], length 0 10:47:15.276978 IP (tos 0x10, ttl 64, id 10130, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.138.22 > 192.168.0.37.31139: Flags [.], cksum 0x7ef0 (incorrect -> 0x4e9c), ack 2466, win 1038, options [nop,nop,TS val 805378798 ecr 1019392585], length 0 10:47:15.277212 IP (tos 0x10, ttl 64, id 10131, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.138.22 > 192.168.0.37.31139: Flags [F.], cksum 0x7ef0 (incorrect -> 0x4e9b), seq 3808, ack 2466, win 1038, options [nop,nop,TS val 805378798 ecr 1019392585], length 0 10:47:15.303993 IP (tos 0x10, ttl 61, id 4142, offset 0, flags [DF], proto TCP (6), length 52) 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x3235 (correct), ack 3809, win 8305, options [nop,nop,TS val 1019392588 ecr 805378798], length 0 This box has no services running on it at present, and just sits there idle. I'll periodically check on it and see if the CLOSED socket count, or hung sshd count goes up. It's runing as a PVHVM domU under XenServer 6.2 Since I am retired, I no longer have access to my beloved "TCP/IP Illustrated Vol. 1", but I believe CLOSED status indicates that the socket is no longer in use.As the name says, it is CLOSED. I don't know why sshd should be waiting on a CLOSED socket, nor do I understand why a CLOSED server socket should live on for extended times. That said, I have been monitoring my FreeBSD 10 system and it seems to pick up an occasional case where a socket gets "stuck" in the CLOSED state for a very long time. I can't say forever, but at least many minutes. I don't see this on 9.2 systems. Right now I have two CLOSED sockets, both the the same address on port 80. It appears that the only thing that removes the socket is to kill the owning process (in this case it's Firefox). The owning firefox process does exist just fine when asked and I think sshd should do so as well, but it looks to me like the root problem is that CLOSED socket seem to live on in 10.0 and they don't on older versions. I think it's time to move this over to net@, as I suspect a change made in 10 is triggering this. I'm not sure the changes is wrong or that sshd is wrong in not exiting cleanly when sockets are hanging around in CLOSED state. (Nor am I sure sshd is wrong.) If I am confused about this, please let me know. I'm going ot read the 10.0 release notes to see if they say anything about this. -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 19:57:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 412E9D5 for ; Fri, 28 Mar 2014 19:57:44 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D4B6BEF for ; Fri, 28 Mar 2014 19:57:43 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id A192D139C8 for ; Fri, 28 Mar 2014 20:00:24 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1396047622; x=1396911623; bh=qjYCOI0x1T/3W5tvYFRgS/R8rJAzxP9M5QB VlQ9hKJw=; b=WPglMl3v84bshrku6zT/VV9+wtDMFMgy/ADHWdCt8Zaxs2ChPN8 7ma+j4zqOs9BVF7dbrlLHIEH0QA+6XvgG9+L4sRAtHGKxooFKqD37RRXICOb0TfK 1dih2nlov5SqFQG6iedDuKta1SEnY/MaG/FThulJYzkoTfqQuahNINiU= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GcxOcFhCR27l for ; Fri, 28 Mar 2014 20:00:22 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 4DC42139C3 for ; Fri, 28 Mar 2014 20:00:22 -0300 (BRT) Message-ID: <5335D430.8010108@bsdinfo.com.br> Date: Fri, 28 Mar 2014 16:57:36 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: sshd with zombie process on FreeBSD 10.0-STABLE References: <5335B8F7.7050001@bsdinfo.com.br> In-Reply-To: <5335B8F7.7050001@bsdinfo.com.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 19:57:44 -0000 Em 28/03/14 15:01, Marcelo Gondim escreveu: > Hi all, > > I have noticed zombie processes on the system after a few lost > connections on ssh. > > # ps afx > [...] > 8045 - Is 0:00.01 sshd: unknown [priv] (sshd) > 8046 - Z 0:00.01 > 8054 - IW 0:00.00 sshd: unknown [pam] (sshd) > 28146 - Is 0:00.01 sshd: unknown [priv] (sshd) > 28147 - Z 0:00.01 > 28155 - IW 0:00.00 sshd: unknown [pam] (sshd) > 43320 - Is 0:00.01 sshd: unknown [priv] (sshd) > 43321 - Z 0:00.01 > 43322 - IW 0:00.00 sshd: unknown [pam] (sshd) > 73413 - Is 0:00.01 sshd: unknown [priv] (sshd) > 73414 - Z 0:00.01 > 73430 - IW 0:00.00 sshd: unknown [pam] (sshd) > [...] > > More information discussed in freebsd-stable@list: > > > On Tue, Mar 25, 2014 at 3:52 AM, Karl Pielorz > wrote: > > > > --On 21 March 2014 22:02 -0700 Kevin Oberman > wrote: > > Ideally I'd look to try and capture the packets st the end of > the session. > Can you do something to trigger this reliably? if so "standard" > "tcpdump > -pw file.bpf host HOST". I seem to recall that these > connections are > scheduled. If so, you can put the packet capture in a crontab to > run at > the same time. If you feed this to a tool like wireshark, you > should get > a good idea of what is happening, if not why. I understand that > the > timing of this might be very tricky. > > > Ok, fwiw as I have this issue as well - I've done a packet capture > (it's below). This box has 59 'CLOSED' sockets from ssh on it, and > 60 sshd stuck. > > Also - initially I thought this was a Xen issue - so there's a > couple of posts on that list from a couple of weeks ago, in brief - > the sshd processes I have are stuck in 'urdlck' - one of the Xen > guys commented "It seems like the process is stuck while trying to > acquire a rw mutex in read mode." > > I did a backtrace of a stuck process - I can post that if you want > (or check the FreeBSD-Xen list for 'stuck sshd in urdlck'. > > Also, if I ssh into this host, 90% of the time (seems to get worse > the longer the box is up) I get: > > " > ssh_exchange_identification: Connection closed by remote host > " > > That does not leave a lingering CLOSED socket. In fact, a successful > ssh login - and logout, does not in testing appear to leave a > lingering CLOSED socket, nor sshd stuck in urdlck - so I'm not > entirely sure where they're coming from, or how often they are > created. > > tcpdump from the start of a successful ssh connect: > > 10:47:04.777765 IP (tos 0x0, ttl 61, id 4058, offset 0, flags [DF], > proto TCP (6), length 60) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [S], cksum 0x57ce > (correct), seq 634709832, win 65535, options [mss 1368,nop,wscale > 3,sackOK,TS val 1019391535 ecr 0], length 0 > > 10:47:04.777776 IP (tos 0x0, ttl 64, id 10060, offset 0, flags [DF], > proto TCP (6), length 60) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [S.], cksum 0x7ef8 > (incorrect -> 0x69d8), seq 2316386788, ack 634709833, win 65535, > options [mss 1368,nop,wscale 6,sackOK,TS val 805368299 ecr > 1019391535], length 0 > > 10:47:04.804218 IP (tos 0x0, ttl 61, id 4059, offset 0, flags [DF], > proto TCP (6), length 52) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x77d3 > (correct), ack 1, win 8305, options [nop,nop,TS val 1019391538 ecr > 805368299], length 0 > > 10:47:04.809692 IP (tos 0x0, ttl 64, id 10061, offset 0, flags [DF], > proto TCP (6), length 99) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f1f > (incorrect -> 0x5799), seq 1:48, ack 1, win 1038, options > [nop,nop,TS val 805368328 ecr 1019391538], length 47 > > 10:47:04.836110 IP (tos 0x0, ttl 61, id 4060, offset 0, flags [DF], > proto TCP (6), length 92) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x8afa > (correct), seq 1:41, ack 48, win 8305, options [nop,nop,TS val > 1019391541 ecr 805368328], length 40 > > 10:47:04.836669 IP (tos 0x0, ttl 64, id 10062, offset 0, flags [DF], > proto TCP (6), length 1596) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x84f8 > (incorrect -> 0x4d42), seq 48:1592, ack 41, win 1038, options > [nop,nop,TS val 805368358 ecr 1019391541], length 1544 > > > And the end of the session: > > 10:47:15.243540 IP (tos 0x10, ttl 61, id 4132, offset 0, flags [DF], > proto TCP (6), length 100) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x6364 > (correct), seq 2321:2369, ack 3520, win 8305, options [nop,nop,TS > val 1019392582 ecr 805375068], length 48 > > 10:47:15.243736 IP (tos 0x10, ttl 64, id 10125, offset 0, flags > [DF], proto TCP (6), length 100) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f20 > (incorrect -> 0x3ea8), seq 3520:3568, ack 2369, win 1038, options > [nop,nop,TS val 805378765 ecr 1019392582], length 48 > > 10:47:15.243796 IP (tos 0x10, ttl 64, id 10126, offset 0, flags > [DF], proto TCP (6), length 100) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f20 > (incorrect -> 0xdd22), seq 3568:3616, ack 2369, win 1038, options > [nop,nop,TS val 805378765 ecr 1019392582], length 48 > > 10:47:15.244627 IP (tos 0x10, ttl 64, id 10127, offset 0, flags > [DF], proto TCP (6), length 84) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f10 > (incorrect -> 0x86ed), seq 3616:3648, ack 2369, win 1038, options > [nop,nop,TS val 805378765 ecr 1019392582], length 32 > > 10:47:15.244812 IP (tos 0x10, ttl 64, id 10128, offset 0, flags > [DF], proto TCP (6), length 212) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [P.], cksum 0x7f90 > (incorrect -> 0x431c), seq 3648:3808, ack 2369, win 1038, options > [nop,nop,TS val 805378765 ecr 1019392582], length 160 > > 10:47:15.271439 IP (tos 0x10, ttl 61, id 4134, offset 0, flags [DF], > proto TCP (6), length 52) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x3381 > (correct), ack 3616, win 8299, options [nop,nop,TS val 1019392585 > ecr 805378765], length 0 > > 10:47:15.272238 IP (tos 0x10, ttl 61, id 4135, offset 0, flags [DF], > proto TCP (6), length 52) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x32d9 > (correct), ack 3808, win 8275, options [nop,nop,TS val 1019392585 > ecr 805378765], length 0 > > 10:47:15.273515 IP (tos 0x10, ttl 61, id 4137, offset 0, flags [DF], > proto TCP (6), length 84) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0x43cc > (correct), seq 2369:2401, ack 3808, win 8305, options [nop,nop,TS > val 1019392585 ecr 805378765], length 32 > > 10:47:15.276199 IP (tos 0x10, ttl 61, id 4138, offset 0, flags [DF], > proto TCP (6), length 116) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [P.], cksum 0xb169 > (correct), seq 2401:2465, ack 3808, win 8305, options [nop,nop,TS > val 1019392585 ecr 805378765], length 64 > > 10:47:15.276220 IP (tos 0x10, ttl 64, id 10129, offset 0, flags > [DF], proto TCP (6), length 52) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [.], cksum 0x7ef0 > (incorrect -> 0x4ea3), ack 2465, win 1037, options [nop,nop,TS val > 805378793 ecr 1019392585], length 0 > > 10:47:15.276970 IP (tos 0x10, ttl 61, id 4140, offset 0, flags [DF], > proto TCP (6), length 52) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [F.], cksum 0x325a > (correct), seq 2465, ack 3808, win 8305, options [nop,nop,TS val > 1019392585 ecr 805378765], length 0 > > 10:47:15.276978 IP (tos 0x10, ttl 64, id 10130, offset 0, flags > [DF], proto TCP (6), length 52) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [.], cksum 0x7ef0 > (incorrect -> 0x4e9c), ack 2466, win 1038, options [nop,nop,TS val > 805378798 ecr 1019392585], length 0 > > 10:47:15.277212 IP (tos 0x10, ttl 64, id 10131, offset 0, flags > [DF], proto TCP (6), length 52) > 192.168.0.138.22 > 192.168.0.37.31139: Flags [F.], cksum 0x7ef0 > (incorrect -> 0x4e9b), seq 3808, ack 2466, win 1038, options > [nop,nop,TS val 805378798 ecr 1019392585], length 0 > > 10:47:15.303993 IP (tos 0x10, ttl 61, id 4142, offset 0, flags [DF], > proto TCP (6), length 52) > 192.168.0.37.31139 > 192.168.0.138.22: Flags [.], cksum 0x3235 > (correct), ack 3809, win 8305, options [nop,nop,TS val 1019392588 > ecr 805378798], length 0 > > > This box has no services running on it at present, and just sits > there idle. I'll periodically check on it and see if the CLOSED > socket count, or hung sshd count goes up. > > It's runing as a PVHVM domU under XenServer 6.2 > > > Since I am retired, I no longer have access to my beloved "TCP/IP > Illustrated Vol. 1", but I believe CLOSED status indicates that the > socket is no longer in use.As the name says, it is CLOSED. I don't > know why sshd should be waiting on a CLOSED socket, nor do I > understand why a CLOSED server socket should live on for extended times. > > That said, I have been monitoring my FreeBSD 10 system and it seems to > pick up an occasional case where a socket gets "stuck" in the CLOSED > state for a very long time. I can't say forever, but at least many > minutes. I don't see this on 9.2 systems. Right now I have two CLOSED > sockets, both the the same address on port 80. It appears that the > only thing that removes the socket is to kill the owning process (in > this case it's Firefox). The owning firefox process does exist just > fine when asked and I think sshd should do so as well, but it looks to > me like the root problem is that CLOSED socket seem to live on in 10.0 > and they don't on older versions. > > I think it's time to move this over to net@, as I suspect a change > made in 10 is triggering this. I'm not sure the changes is wrong or > that sshd is wrong in not exiting cleanly when sockets are hanging > around in CLOSED state. (Nor am I sure sshd is wrong.) > > If I am confused about this, please let me know. I'm going ot read the > 10.0 release notes to see if they say anything about this. Another my server with 12 zombie process related with CLOSED connections: last pid: 84965; load averages: 2.74, 2.65, 2.07 up 14+23:43:16 16:10:00 133 processes: 1 running, 120 sleeping, 12 zombie CPU: 8.5% user, 0.0% nice, 39.3% system, 11.8% interrupt, 40.4% idle Mem: 194M Active, 2575M Inact, 828M Wired, 77M Cache, 391M Buf, 3056K Free Swap: 16G Total, 377M Used, 16G Free, 2% Inuse, 8K In # netstat -n |grep CLOSED tcp4 0 0 192.168.254.1.22 10.254.0.12.26792 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.60591 CLOSED tcp4 0 0 192.168.254.1.22 10.254.0.12.26974 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.54034 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.52999 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.65345 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.64099 CLOSED tcp4 0 0 192.168.254.1.22 10.254.0.12.26457 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.54181 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.53717 CLOSED tcp4 0 0 186.xxx.xx.2.22 186.xxx.xx.8.57035 CLOSED tcp4 0 0 192.168.254.1.22 10.254.0.12.26832 CLOSED From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 22:46:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C446044C for ; Fri, 28 Mar 2014 22:46:42 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2827EBB for ; Fri, 28 Mar 2014 22:46:41 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 6A0C071282; Fri, 28 Mar 2014 15:46:40 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 20115-01; Fri, 28 Mar 2014 15:46:40 -0700 (PDT) Received: from kruse-47.3.ixsystems.com (kruse-47.3.ixsystems.com [10.2.3.47]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id D1D8771273; Fri, 28 Mar 2014 15:46:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396046800; bh=J+GCyfZ12VpN6IVzldjgEF6IuKlmPyQ3K/t8dwWt4hM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=ukAIw2/Q/2Rqd7QgdaMG8UKHTt17BvfKRM90BlYGCdEXQS6o4wdJM1PzZPCJz8Gzr xlEvNOVf6emAw3N7+YFuvGPIZiTkd8DqXwRBwrl67UZ+MZUs+GaVgjVJ2/NTaYykG6 Z8JedXJ6YQ/B1zirLMh6nrhp0FroUHugobA7WBTc= Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Non-interrupt packet sending and receiving From: Sean Fagan In-Reply-To: Date: Fri, 28 Mar 2014 15:46:39 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <32118BD7-31D4-44BB-9336-9434AC878455@ixsystems.com> References: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> To: Ryan Stone X-Mailer: Apple Mail (2.1874) Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 22:46:42 -0000 Okay, I've taken the abandoned netdump code, gotten into 10.0 (what I'm = using for development right now), and restructure it to the point where I can = consider using it for my own purposes, while still having it work for dumping a kernel = over ethernet. Anyone want to take a look at the diffs I have so far? Sean. From owner-freebsd-net@FreeBSD.ORG Fri Mar 28 23:22:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E58D3FC for ; Fri, 28 Mar 2014 23:22:00 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 81DD223B for ; Fri, 28 Mar 2014 23:22:00 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 613C620ABF; Fri, 28 Mar 2014 16:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1396048919; bh=jZMQDetHhgj86hjwj93A6TFfI/TNGKYqEthcfHLn82w=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=wJ1lCOvDEcwjspzg0DNifi2f2cattyXfzFFa+AG5iGOtRSZsSrQ8PRy6ZHQfySIXJ cpPIlqOzmBlW70rLkaIoQetKZ3Vsh3PsxLawO4gDvLVmX1MYFIdH1ovXi9ewRzLwTu oB4m0EBHhvHDIceoSNskzmOYT8kuOxEDRy8vPdRs= Message-ID: <53360415.8000102@delphij.net> Date: Fri, 28 Mar 2014 16:21:57 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Sean Fagan , Ryan Stone Subject: Re: Non-interrupt packet sending and receiving References: <27D25BFC-7BB3-400F-8405-43B8D08135D2@ixsystems.com> <32118BD7-31D4-44BB-9336-9434AC878455@ixsystems.com> In-Reply-To: <32118BD7-31D4-44BB-9336-9434AC878455@ixsystems.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 23:22:00 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 03/28/14 15:46, Sean Fagan wrote: > Okay, I've taken the abandoned netdump code, gotten into 10.0 (what > I'm using for development right now), and restructure it to the > point where I can consider using it for my own purposes, while > still having it work for dumping a kernel over ethernet. > > Anyone want to take a look at the diffs I have so far? Cool! Maybe create a branch in svn.freebsd.org or github? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCgAGBQJTNgQVAAoJEJW2GBstM+nszp4P/jr1ddDV4dS9WFs+S287PS2n qa5t79gYdjj8geBcRU43tjy1XOUR+eHlQL5mmNxItEY6RFXs+X8RbS+dXAjuMWq4 AD75fRZUHrcQE3HWvLBlSS2IcO1eVh76+40vL/dVXrtIJJq4DtRyODgWL8IyTSiD yjiFC9DgZpJMwetyk8xaFovpGtbkmFgKTxOaOT+Wsqp0K7+gyOVVEfYeVw02Wkrm FHPfIR5yERFCIo9eM9qxBFbUcFyaBEV7g9CFTOjuNcPPT6tDNEflcsTnqRD7aq3R uX+Lx7xVX138tTZtty1E8OUUjskaWECpkQP1ooAp3QnOqFZ7x69fBit1swD+7xiD mwwAcFJFDp7Bh6rYu7Z/jHfIKLHEZ0dex4kE2f1RD3juHoo4Q/ZXnhyBASeNkL2u 4MYXcc4BepxYdzA0I8s9AHfaeQ7uYvBqb/tTAirj4dd9QO6Ra+xpZbHD5kVRgBCo bhmQriHziRD6HLvp+Mkrt5RAXsNn1pI5FtpDZj9k6xRg43AdeQn9USsKXTjYAChE gaMhGNRrNK0p5FDb58qJxqREs4MfQqmDoA7ykkjadf9cFGcBjaaSN5svTuQ3AMe4 yo+s1918bvj7XTEAx9eC/qEH4r/TqBXf5m/DDMfKaDf/SLMrKyz+RyfsDVwlGQQ5 +fxkdKL1iSGjZ+6HzVeM =0Pgz -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Sat Mar 29 14:02:43 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A16A918; Sat, 29 Mar 2014 14:02:43 +0000 (UTC) Received: from cpsmtpb-ews03.kpnxchange.com (cpsmtpb-ews03.kpnxchange.com [213.75.39.6]) by mx1.freebsd.org (Postfix) with ESMTP id B19953B7; Sat, 29 Mar 2014 14:02:42 +0000 (UTC) Received: from cpsps-ews27.kpnxchange.com ([10.94.84.193]) by cpsmtpb-ews03.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Sat, 29 Mar 2014 15:02:32 +0100 Received: from CPSMTPM-CMT108.kpnxchange.com ([195.121.3.24]) by cpsps-ews27.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Sat, 29 Mar 2014 15:02:32 +0100 Received: from donald.offrom.nl ([77.170.60.162]) by CPSMTPM-CMT108.kpnxchange.com with Microsoft SMTPSVC(7.0.6002.18264); Sat, 29 Mar 2014 15:02:30 +0100 Received: from squid (squid.offrom.nl [192.168.0.72]) by donald.offrom.nl (8.14.7/8.14.7) with ESMTP id s2TE2T8C009227; Sat, 29 Mar 2014 15:02:29 +0100 (CET) (envelope-from willy@vpn.offrom.nl) Received: from willy by squid with local (Exim 4.72) (envelope-from ) id 1WTtq9-0005go-Pj; Sat, 29 Mar 2014 15:02:29 +0100 Date: Sat, 29 Mar 2014 15:02:29 +0100 From: Willy Offermans To: Willy Offermans Subject: Re: TCP packets remain unsent Message-ID: <20140329140229.GE3528@vpn.offrom.nl> References: <20140328162554.GA26748@vpn.offrom.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140328162554.GA26748@vpn.offrom.nl> User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 29 Mar 2014 14:02:30.0639 (UTC) FILETIME=[866A5FF0:01CF4B57] X-RcptDomain: FreeBSD.ORG Cc: freebsd-net@FreeBSD.ORG, freebsd-stable@FreeBSD.ORG X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Willy@Offermans.Rompen.nl List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Mar 2014 14:02:43 -0000 Dear FreeBSD friends, On Fri, Mar 28, 2014 at 05:25:54PM +0100, Willy Offermans wrote: > Dear FreeBSD friends, > > I have a problem with my relatively new FreeBSD server. I came across the > problem when sending e-mails of larger size and copying files with scp. > The e-mails were not sent out because of time-out error and the copying was > extremely slow, though successful after a while. I already started a thread > on this topic on freebsd-current. See > http://docs.freebsd.org/mail/current/freebsd-current.html, topic > sendmail Broken Pipe Error. I got some help to narrow down the > error: Sending out e-mails of larger size stops at some point. TCP packets > were not transferred to the smarthost causing a timeout error. There were > still some TCP packets waiting to be sent. > > My system is a HP ProLiant Gen8 MicroServer with FreeBSD 10.0-STABLE #0 > r261266M. The server has two network cards: > > bge0@pci0:3:0:0: class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetXtreme BCM5720 Gigabit Ethernet PCIe' > class = network > subclass = ethernet > bge1@pci0:3:0:1: class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetXtreme BCM5720 Gigabit Ethernet PCIe' > class = network > subclass = ethernet > > I do not know if there any known issues with these cards (drivers). > > Before the time out error occurs, the CPU loading of natd and dhcpd is > steadily increasing to extreme values to my opinion: > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 1235 root 1 93 0 28908K 2144K RUN 0 54:05 71.78% natd > 1614 dhcpd 1 4 0 26784K 14936K RUN 0 29:24 38.77% dhcpd > > I followed an advice of another FreeBSD friend to modify tcp_input.c > > > ... > You may want to locally apply SVN r258821 to sys/netinet/tcp_input.c, > in case it has not been merged back to the FreeBSD version you use: > > --- sys/netinet/tcp_input.c (revision 258820) > +++ sys/netinet/tcp_input.c (revision 258821) > @@ -2429,13 +2429,15 @@ > hhook_run_tcp_est_in(tp, th, &to); > > if (SEQ_LEQ(th->th_ack, tp->snd_una)) { > - if (tlen == 0 && tiwin == tp->snd_wnd) { > + if (tlen == 0 && tiwin == tp->snd_wnd && > + !(thflags & TH_FIN)) { > TCPSTAT_INC(tcps_rcvdupack); > /* > * If we have outstanding data (other than > * a window probe), this is a completely > * duplicate ack (ie, window info didn't > - * change), the ack is the biggest we've > + * change and FIN isn't set), > + * the ack is the biggest we've > * seen and we've seen exactly our rexmt > * threshhold of them, assume a packet > * has been dropped and retransmit it. > > ... > > > However this did not solve the issue. > > So my question to you is if someone else encountered a similar problem with > NetXtreme BCM5720 Gigabit Ethernet PCIe and if someone is able to take this > up and to help me to solve this issue. I would love to run this FreeBSD > server as a swiss clock as other beastie servers, I have setup in the past. > > -- I could narrow down the cause of the error: If I remove the following line from my firewall rules, I could sent out e-mails without issues. /sbin/ipfw add 50 divert natd all from any to any via bge0 I do not know yet how things are related, but I will dig into it. If someone has a hint, please respond to the list. -- Met vriendelijke groeten, With kind regards, Mit freundlichen Gruessen, De jrus wah, Wiel ************************************* W.K. Offermans e-mail: Willy@Offermans.Rompen.nl Powered by .... (__) \\\'',) \/ \ ^ .\._/_) www.FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Sat Mar 29 17:12:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3BDBF53; Sat, 29 Mar 2014 17:12:47 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40AE0807; Sat, 29 Mar 2014 17:12:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s2THCZB0056749; Sun, 30 Mar 2014 04:12:36 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sun, 30 Mar 2014 04:12:34 +1100 (EST) From: Ian Smith To: Willy Offermans Subject: Re: TCP packets remain unsent In-Reply-To: <20140329140229.GE3528@vpn.offrom.nl> Message-ID: <20140330012659.Y78237@sola.nimnet.asn.au> References: <20140328162554.GA26748@vpn.offrom.nl> <20140329140229.GE3528@vpn.offrom.nl> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Mar 2014 17:12:48 -0000 On Sat, 29 Mar 2014 15:02:29 +0100, Willy Offermans wrote: > Dear FreeBSD friends, > > On Fri, Mar 28, 2014 at 05:25:54PM +0100, Willy Offermans wrote: > > Dear FreeBSD friends, > > > > I have a problem with my relatively new FreeBSD server. I came across the > > problem when sending e-mails of larger size and copying files with scp. > > The e-mails were not sent out because of time-out error and the copying was > > extremely slow, though successful after a while. I already started a thread > > on this topic on freebsd-current. See > > http://docs.freebsd.org/mail/current/freebsd-current.html, topic > > sendmail Broken Pipe Error. I got some help to narrow down the > > error: Sending out e-mails of larger size stops at some point. TCP packets > > were not transferred to the smarthost causing a timeout error. There were > > still some TCP packets waiting to be sent. > > > > My system is a HP ProLiant Gen8 MicroServer with FreeBSD 10.0-STABLE #0 > > r261266M. The server has two network cards: [..] > > Before the time out error occurs, the CPU loading of natd and dhcpd is > > steadily increasing to extreme values to my opinion: > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > > > 1235 root 1 93 0 28908K 2144K RUN 0 54:05 71.78% natd > > 1614 dhcpd 1 4 0 26784K 14936K RUN 0 29:24 38.77% dhcpd [..] > I could narrow down the cause of the error: > > If I remove the following line from my firewall rules, I could sent out > e-mails without issues. > > /sbin/ipfw add 50 divert natd all from any to any via bge0 > > I do not know yet how things are related, but I will dig into it. > > If someone has a hint, please respond to the list. Is your system running IPv6? Sendmail will prefer using ip6 if enabled. You need to use 'ip4' rather than 'all' with divert; natd (and I assume, ipfw nat?) doesn't like ip6 packets being sent its way. Also, ipfw nat and natd both use libalias(3) which doesn't work with TSO; check that's turned off with ifconfig. See ipfw(8) /BUGS section. Just guesswork, Ian From owner-freebsd-net@FreeBSD.ORG Sun Mar 30 17:49:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2483FD2; Sun, 30 Mar 2014 17:49:09 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3CBE1806; Sun, 30 Mar 2014 17:49:08 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s2UHcq2n046999 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 30 Mar 2014 19:38:56 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: melifaro@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s2UHcgbv091234; Mon, 31 Mar 2014 00:38:42 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <533856A2.7030401@grosbein.net> Date: Mon, 31 Mar 2014 00:38:42 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net , "Alexander V. Chernikov" Subject: icmp_error() fails to clear "fragmented" flag Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * 2.8 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-Spam-Level: *** X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 17:49:09 -0000 Hi! Suppose, you have FreeBSD host A behind FreeBSD router R and run "traceroute -I outerhost 1501" command from A. You will see only "stars" for first hop. That's because router R erroneously sends ICMP "time exceeded" packets with "more fragments" flag in the IP header when original packet was fragmented. This flag is copied from original header. I've just tested the following patch, it fixes the problem: http://www.grosbein.net/freebsd/patches/ip_icmp.c.diff --- sys/netinet/ip_icmp.c.orig 2013-10-21 21:07:06.000000000 +0700 +++ sys/netinet/ip_icmp.c 2014-03-31 00:06:48.000000000 +0700 @@ -332,6 +332,7 @@ stdreply: icmpelen = max(8, min(V_icmp_q * reply should bypass as well. */ m->m_flags |= n->m_flags & M_SKIP_FIREWALL; + m->m_flags &= ~(M_FRAG | M_FIRSTFRAG | M_LASTFRAG); m->m_data -= sizeof(struct ip); m->m_len += sizeof(struct ip); m->m_pkthdr.len = m->m_len; @@ -343,6 +344,7 @@ stdreply: icmpelen = max(8, min(V_icmp_q nip->ip_hl = 5; nip->ip_p = IPPROTO_ICMP; nip->ip_tos = 0; + nip->ip_off = 0; icmp_reflect(m); freeit: (I've discovered this while debugging real-world issue concerning problems with UDP fragmented traffic while using L2TP tunnel.) Please review/commit. Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Sun Mar 30 18:00:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02F3F2FB; Sun, 30 Mar 2014 18:00:24 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A88C08C9; Sun, 30 Mar 2014 18:00:23 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so6310961qga.4 for ; Sun, 30 Mar 2014 11:00:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+ybd/i9MDrDL05qhHUoR3s1+Wks2k541cV6rSICaF9g=; b=RwmEwbUvwuQ2LGE/m98p5bJMchNa5NMaHX4Ddah3eIRKkvWYmq+eRyo2IZwecxwuzM HlTZt6/vjzGfj8MuTUp58huCbP8tui8RAJSs6oWDOZdlzxTwLBlHeuYADmA0tNHvaHw2 RURLmjo2ApTLD393ypu/hFABAcO81BoDp9ZbvzrezChj2WSylNrIzlxBYC52nZHKxqkq Iwvin8E4pTsWZKmqo0f8IYNzklK4CI6cpVOzEltX/5fZyFlNvL8x4E6XgVMGiz0smxVp A0cqz75TIaOEe0Q5zgxhOcNDqPZPAOgKXGbtIg3qtkPr7Sm4mhYIsH0uTZI7YJdVodnN TzOQ== MIME-Version: 1.0 X-Received: by 10.224.60.71 with SMTP id o7mr22728934qah.38.1396202422823; Sun, 30 Mar 2014 11:00:22 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.143 with HTTP; Sun, 30 Mar 2014 11:00:22 -0700 (PDT) In-Reply-To: <533856A2.7030401@grosbein.net> References: <533856A2.7030401@grosbein.net> Date: Sun, 30 Mar 2014 11:00:22 -0700 X-Google-Sender-Auth: fVsQMfzBeT5pr-LGieayuyVwQf8 Message-ID: Subject: Re: icmp_error() fails to clear "fragmented" flag From: Adrian Chadd To: Eugene Grosbein Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net , "Alexander V. Chernikov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 18:00:24 -0000 Can you file a PR with exactly this? :-P Thanks! -a On 30 March 2014 10:38, Eugene Grosbein wrote: > Hi! > > Suppose, you have FreeBSD host A behind FreeBSD router R and run > "traceroute -I outerhost 1501" command from A. You will see only "stars" > for first hop. That's because router R erroneously sends ICMP "time exceeded" packets > with "more fragments" flag in the IP header when original packet was fragmented. > This flag is copied from original header. > > I've just tested the following patch, it fixes the problem: > http://www.grosbein.net/freebsd/patches/ip_icmp.c.diff > > --- sys/netinet/ip_icmp.c.orig 2013-10-21 21:07:06.000000000 +0700 > +++ sys/netinet/ip_icmp.c 2014-03-31 00:06:48.000000000 +0700 > @@ -332,6 +332,7 @@ stdreply: icmpelen = max(8, min(V_icmp_q > * reply should bypass as well. > */ > m->m_flags |= n->m_flags & M_SKIP_FIREWALL; > + m->m_flags &= ~(M_FRAG | M_FIRSTFRAG | M_LASTFRAG); > m->m_data -= sizeof(struct ip); > m->m_len += sizeof(struct ip); > m->m_pkthdr.len = m->m_len; > @@ -343,6 +344,7 @@ stdreply: icmpelen = max(8, min(V_icmp_q > nip->ip_hl = 5; > nip->ip_p = IPPROTO_ICMP; > nip->ip_tos = 0; > + nip->ip_off = 0; > icmp_reflect(m); > > freeit: > > > (I've discovered this while debugging real-world issue concerning > problems with UDP fragmented traffic while using L2TP tunnel.) > > Please review/commit. > > Eugene Grosbein > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Mar 30 18:42:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B4FA735; Sun, 30 Mar 2014 18:42:04 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8798AD1B; Sun, 30 Mar 2014 18:42:03 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s2UIfjSk047188 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 30 Mar 2014 20:41:49 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: melifaro@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s2UIfZXa092621; Mon, 31 Mar 2014 01:41:36 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <5338655F.20804@grosbein.net> Date: Mon, 31 Mar 2014 01:41:35 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: icmp_error() fails to clear "fragmented" flag References: <533856A2.7030401@grosbein.net> In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: freebsd-net , "Alexander V. Chernikov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 18:42:04 -0000 On 31.03.2014 01:00, Adrian Chadd wrote: > Can you file a PR with exactly this? :-P Done: http://www.freebsd.org/cgi/query-pr.cgi?pr=188092 From owner-freebsd-net@FreeBSD.ORG Sun Mar 30 20:09:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E5A826E; Sun, 30 Mar 2014 20:09:46 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0057638; Sun, 30 Mar 2014 20:09:44 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s2UKCPl9050235; Sun, 30 Mar 2014 13:12:31 -0700 (PDT) (envelope-from chrish@UltimateDNS.NET) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s2UKCKSe050234; Sun, 30 Mar 2014 13:12:20 -0700 (PDT) (envelope-from chrish@UltimateDNS.NET) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Sun, 30 Mar 2014 13:12:20 -0700 (PDT) Message-ID: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> Date: Sun, 30 Mar 2014 13:12:20 -0700 (PDT) Subject: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: chrish@UltimateDNS.NET To: "freebsd-net" , "freebsd-stable" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 20:09:46 -0000 Greetings, I'm not sure whether this best belonged on net@, or stable@ so I'm using both. :) I'm testing both releng_9, and MB, and I encountered a new message I don't usually see using the nfe(4) driver: miibus0: mii_mediachg: can't handle non-zero PHY instance 1 ... miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Truncated for brevity (31 lines in total; 1-31). I don't know how interpret this. An issue with my version of the driver, or the hardware itself? This occurred with both GENERIC, as well as my custom kernel. # uname -a FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756: Wed Mar 26 11:28:10 PDT 2014 root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 Thank you for all your time, and consideration. --Chris From owner-freebsd-net@FreeBSD.ORG Sun Mar 30 20:45:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 592C8AC3 for ; Sun, 30 Mar 2014 20:45:36 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D896098C for ; Sun, 30 Mar 2014 20:45:35 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so5262467lan.12 for ; Sun, 30 Mar 2014 13:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=qX1lBqnSOsamRfA5xh23dJp1tTFvQF8vf8bnUxPSId0=; b=EvgCRgoOPnDNu4sO/ALPQZHH/dm/YaDZPeEkJ8wiS+StKOXjvOL9/fi5oofaszbaJj SwjV4T05frYtf5FNEawKkk2OPVSWv84yPxx1txbSHfBhPs1Dj6BnA7emiQCle7Q7unTw h+sEoUMQXyV+FtqB6aY/w3f/uzHqeNQwnre6H0oQIR6RNk50NeBsA1lVXJKwGXiQfzzf GhYoBSM6bQBCXNY8uaR4HuyxhxvKXceXR7jGUgsZVlaaKABao/hobaVR5OO/hgpO60PN 5MTrnjkFBllLVxYhuLWrD1iH/CByrvcVu+UoGE53xXitTzLGXoMaTGFxCQ+qnR2EDQqH tm3A== MIME-Version: 1.0 X-Received: by 10.112.137.193 with SMTP id qk1mr15856lbb.53.1396212333880; Sun, 30 Mar 2014 13:45:33 -0700 (PDT) Sender: pkelsey@gmail.com Received: by 10.112.141.196 with HTTP; Sun, 30 Mar 2014 13:45:33 -0700 (PDT) Date: Sun, 30 Mar 2014 16:45:33 -0400 X-Google-Sender-Auth: GZtf3a2QnZtDFADM1yQifnPXCpM Message-ID: Subject: Extraneous code in syncache_socket() From: Patrick Kelsey To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 20:45:36 -0000 Using sys/netinet/tcp_syncache.c in r261594 for reference ( http://svnweb.freebsd.org/base/head/sys/netinet/tcp_syncache.c?view=markup), I believe the following code in syncache_socket() serves no purpose: These two lines beginning at line 780: if (IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_laddr)) inp->in6p_laddr = sc->sc_inc.inc6_laddr; and these two lines beginning at line 820: if (inp->inp_laddr.s_addr == INADDR_ANY) inp->inp_laddr = sc->sc_inc.inc_laddr; In the cases where the above lines are reached, the same assignments have already been performed by the if/else block starting at line 702. The only intervening code path that I see that modifies inp->in6p_laddr or inp->inp_laddr exits the routine, via the goto abort; statement at line 748, before reaching the above lines. Thus, conditionally performing these assignments again serves no apparent purpose. -Patrick From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 02:32:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E32AC1B; Mon, 31 Mar 2014 02:32:56 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8F7693E; Mon, 31 Mar 2014 02:32:55 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id lf10so7588471pab.27 for ; Sun, 30 Mar 2014 19:32:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=E0KhTnbURd8QuFlqldN8V4UJLFnqvrb1S78F+c6Z7mQ=; b=Tk3IrTtDGi/g14mHY/fSECh1HXScD/0PdTxd/GefRksIQfeeP69spImETymKA3HfsV UZ+p/ilENX3ru1MA7EJ9tv7XAL8QkGn043D7VeIZAWhWDAN6P+SUqZAGEyvp+ktfcVLT BY/bNdq4QxZLFi8thAZPotQd/OYz+dWTZ0PyB5jlP+SUpPsaOork9CSSn0uBzao0ybXH Xb2/CoRNaMTv07Hta8zQdd//gO0BkmpMMVZGWoikv1LfX9EjO3PDQeuotMljDpBVq4lj R7TPVaazwTurvqlS0cDJp3IQSbb7oBMMrxdxYS++dK0euNm6/+gvJnRQf2VOQVB6dmk2 HB2A== X-Received: by 10.68.58.34 with SMTP id n2mr4460385pbq.122.1396233175467; Sun, 30 Mar 2014 19:32:55 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id dn1sm39954451pbb.54.2014.03.30.19.32.52 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 30 Mar 2014 19:32:54 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 31 Mar 2014 11:32:53 +0900 From: Yonghyeon PYUN Date: Mon, 31 Mar 2014 11:32:53 +0900 To: Rick Macklem Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem Message-ID: <20140331023253.GC3548@michelle.cdnetworks.com> References: <20140326023334.GB2973@michelle.cdnetworks.com> <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Filesystems , FreeBSD Net , Alexander Motin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 02:32:56 -0000 On Wed, Mar 26, 2014 at 08:27:48PM -0400, Rick Macklem wrote: > pyunyh@gmail.com wrote: > > On Tue, Mar 25, 2014 at 07:10:35PM -0400, Rick Macklem wrote: > > > Hi, > > > > > > First off, I hope you don't mind that I cross-posted this, > > > but I wanted to make sure both the NFS/iSCSI and networking > > > types say it. > > > If you look in this mailing list thread: > > > http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > > > you'll see that several people have been working hard at testing > > > and > > > thanks to them, I think I now know what is going on. > > > > > > Thanks for your hard work on narrowing down that issue. I'm too > > busy for $work in these days so I couldn't find time to investigate > > the issue. > > > > > (This applies to network drivers that support TSO and are limited > > > to 32 transmit > > > segments->32 mbufs in chain.) Doing a quick search I found the > > > following > > > drivers that appear to be affected (I may have missed some): > > > jme, fxp, age, sge, msk, alc, ale, ixgbe/ix, nfe, e1000/em, re > > > > > > > The magic number 32 was chosen long time ago when I implemented TSO > > in non-Intel drivers. I tried to find optimal number to reduce the > > size kernel stack usage at that time. bus_dma(9) will coalesce > > with previous segment if possible so I thought the number 32 was > > not an issue. Not sure current bus_dma(9) also has the same code > > though. The number 32 is arbitrary one so you can increase > > it if you want. > > > Well, in the case of "ix" Jack Vogel says it is a hardware limitation. > I can't change drivers that I can't test and don't know anything about > the hardware. Maybe replacing m_collapse() with m_defrag() is an exception, > since I know what that is doing and it isn't hardware related, but I > would still prefer a review by the driver author/maintainer before making > such a change. > > If there are drivers that you know can be increased from 32->35 please do > so, since that will not only avoid the EFBIG failures but also avoid a > lot of calls to m_defrag(). > > > > Further, of these drivers, the following use m_collapse() and not > > > m_defrag() > > > to try and reduce the # of mbufs in the chain. m_collapse() is not > > > going to > > > get the 35 mbufs down to 32 mbufs, as far as I can see, so these > > > ones are > > > more badly broken: > > > jme, fxp, age, sge, alc, ale, nfe, re > > > > I guess m_defeg(9) is more optimized for non-TSO packets. You don't > > want to waste CPU cycles to copy the full frame to reduce the > > number of mbufs in the chain. For TSO packets, m_defrag(9) looks > > better but if we always have to copy a full TSO packet to make TSO > > work, driver writers have to invent better scheme rather than > > blindly relying on m_defrag(9), I guess. > > > Yes, avoiding m_defrag() calls would be nice. For this issue, increasing > the transmit segment limit from 32->35 does that, if the change can be > done easily/safely. > > Otherwise, all I can think of is my suggestion to add something like > if_hw_tsomaxseg which the driver can use to tell tcp_output() the > driver's limit for # of mbufs in the chain. > > > > > > > The long description is in the above thread, but the short version > > > is: > > > - NFS generates a chain with 35 mbufs in it for (read/readdir > > > replies and write requests) > > > made up of (tcpip header, RPC header, NFS args, 32 clusters of > > > file data) > > > - tcp_output() usually trims the data size down to tp->t_tsomax > > > (65535) and > > > then some more to make it an exact multiple of TCP transmit data > > > size. > > > - the net driver prepends an ethernet header, growing the length > > > by 14 (or > > > sometimes 18 for vlans), but in the first mbuf and not adding > > > one to the chain. > > > - m_defrag() copies this to a chain of 32 mbuf clusters (because > > > the total data > > > length is <= 64K) and it gets sent > > > > > > However, it the data length is a little less than 64K when passed > > > to tcp_output() > > > so that the length including headers is in the range > > > 65519->65535... > > > - tcp_output() doesn't reduce its size. > > > - the net driver adds an ethernet header, making the total data > > > length slightly > > > greater than 64K > > > - m_defrag() copies it to a chain of 33 mbuf clusters, which > > > fails with EFBIG > > > --> trainwrecks NFS performance, because the TSO segment is dropped > > > instead of sent. > > > > > > A tester also stated that the problem could be reproduced using > > > iSCSI. Maybe > > > Edward Napierala might know some details w.r.t. what kind of mbuf > > > chain iSCSI > > > generates? > > > > > > Also, one tester has reported that setting if_hw_tsomax in the > > > driver before > > > the ether_ifattach() call didn't make the value of tp->t_tsomax > > > smaller. > > > However, reducing IP_MAXPACKET (which is what it is set to by > > > default) did > > > reduce it. I have no idea why this happens or how to fix it, but it > > > implies > > > that setting if_hw_tsomax in the driver isn't a solution until this > > > is resolved. > > > > > > So, what to do about this? > > > First, I'd like a simple fix/workaround that can go into 9.3. > > > (which is code > > > freeze in May). The best thing I can think of is setting > > > if_hw_tsomax to a > > > smaller default value. (Line# 658 of sys/net/if.c in head.) > > > > > > Version A: > > > replace > > > ifp->if_hw_tsomax = IP_MAXPACKET; > > > with > > > ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + > > > ETHER_VLAN_ENCAP_LEN), > > > IP_MAXPACKET); > > > plus > > > replace m_collapse() with m_defrag() in the drivers listed above. > > > > > > This would only reduce the default from 65535->65518, so it only > > > impacts > > > the uncommon case where the output size (with tcpip header) is > > > within > > > this range. (As such, I don't think it would have a negative impact > > > for > > > drivers that handle more than 32 transmit segments.) > > > From the testers, it seems that this is sufficient to get rid of > > > the EFBIG > > > errors. (The total data length including ethernet header doesn't > > > exceed 64K, > > > so m_defrag() fits it into 32 mbuf clusters.) > > > > > > The main downside of this is that there will be a lot of m_defrag() > > > calls > > > being done and they do quite a bit of bcopy()'ng. > > > > > > Version B: > > > replace > > > ifp->if_hw_tsomax = IP_MAXPACKET; > > > with > > > ifp->if_hw_tsomax = min(29 * MCLBYTES, IP_MAXPACKET); > > > > > > This one would avoid the m_defrag() calls, but might have a > > > negative > > > impact on TSO performance for drivers that can handle 35 transmit > > > segments, > > > since the maximum TSO segment size is reduced by about 6K. (Because > > > of the > > > second size reduction to an exact multiple of TCP transmit data > > > size, the > > > exact amount varies.) > > > > > > Possible longer term fixes: > > > One longer term fix might be to add something like if_hw_tsomaxseg > > > so that > > > a driver can set a limit on the number of transmit segments (mbufs > > > in chain) > > > and tcp_output() could use that to limit the size of the TSO > > > segment, as > > > required. (I have a first stab at such a patch, but no way to test > > > it, so > > > I can't see that being done by May. Also, it would require changes > > > to a lot > > > of drivers to make it work. I've attached this patch, in case > > > anyone wants > > > to work on it?) > > > > > > Another might be to increase the size of MCLBYTES (I don't see this > > > as > > > practical for 9.3, although the actual change is simple.). I do > > > think > > > that increasing MCLBYTES might be something to consider doing in > > > the > > > future, for reasons beyond fixing this. > > > > > > So, what do others think should be done? rick > > > > > > > AFAIK all TSO capable drivers you mentioned above have no limit on > > the number of TX segments in TSO path. Not sure about Intel > > controllers though. Increasing the number of segment will consume > > lots of kernel stack in those drivers. Given that ixgbe, which > > seems to use 100, didn't show any kernal stack shortage, I think > > bumping the number of segments would be quick way to address the > > issue. > > > Well, bumping it from 32->35 is all it would take for NFS (can't comment > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 > (just so others aren't confused by the above comment). I understand > your point was w.r.t. using 100 without blowing the kernel stack, but > since the testers have been using "ix" with the 82599 chip which is > limited to 32 transmit segments... > > However, please increase any you know can be safely done from 32->35, rick Done in r263957. From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 05:00:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85511646; Mon, 31 Mar 2014 05:00:09 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 538BF80C; Mon, 31 Mar 2014 05:00:09 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id v10so7444783pde.39 for ; Sun, 30 Mar 2014 22:00:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=mTB8l0tBj88GEnj2UHxDqPkNBM0MW3De3Ze8BfSHYZM=; b=JOaG0KLmszO3LUaW821o7+Kwl8nK0H+z4y8WJvxRDHtOKp5i58ea4aLCUi+HtLLda2 8QzY78ShU0itKY2ekLpxTpSgxty/XFA23dTG8ZnwnYTlGsWLos3ZVhAMA8eDPeIFGsml 0/u3F6oDn4VzOIiA2HTMS6yFdkts0VKW32vd0TJ1JbZvw8jGfb8wwLTDxv6WBQbQ81KW NQW0gcKLl2b2crA2FTEL//ox3z1f+xbXF/maXhCCcaRMOGnheW9LQUTj+Hvxx+uaJlYd WQs5tVts4oXSqV72cnaQbENued6HqpTCRFWfNYb6prHiHj45/o8TPU5zZkfGlOgchEhv K6dA== X-Received: by 10.66.226.145 with SMTP id rs17mr1173478pac.144.1396242008835; Sun, 30 Mar 2014 22:00:08 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id av2sm120896pbc.16.2014.03.30.22.00.06 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 30 Mar 2014 22:00:08 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 31 Mar 2014 14:00:02 +0900 From: Yonghyeon PYUN Date: Mon, 31 Mar 2014 14:00:02 +0900 To: chrish@UltimateDNS.NET Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140331050002.GC1359@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 05:00:09 -0000 On Sun, Mar 30, 2014 at 01:12:20PM -0700, chrish@UltimateDNS.NET wrote: > Greetings, > I'm not sure whether this best belonged on net@, or stable@ > so I'm using both. :) > I'm testing both releng_9, and MB, and I encountered a new > message I don't usually see using the nfe(4) driver: > > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > ... > miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > > Truncated for brevity (31 lines in total; 1-31). I don't know > how interpret this. An issue with my version of the driver, or > the hardware itself? This occurred with both GENERIC, as well > as my custom kernel. Would you show me the dmesg output? > > # uname -a > FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756: Wed Mar 26 11:28:10 PDT 2014 > root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 > > Thank you for all your time, and consideration. > > --Chris From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 05:37:12 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2CF5B4B for ; Mon, 31 Mar 2014 05:37:12 +0000 (UTC) Received: from mail.sfc.wide.ad.jp (ns.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:143]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3EEE5AE9 for ; Mon, 31 Mar 2014 05:37:12 +0000 (UTC) Received: from Midoris-MacBook-Air.local (pdf8584bf.tokynt01.ap.so-net.ne.jp [223.133.132.191]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 98005278141; Mon, 31 Mar 2014 14:37:09 +0900 (JST) Message-ID: <5338FF05.1020302@sfc.wide.ad.jp> Date: Mon, 31 Mar 2014 14:37:09 +0900 From: Midori Kato User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: DCTCP implementation Content-Type: multipart/mixed; boundary="------------030401080100030902090403" Cc: "Eggert, Lars" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 05:37:12 -0000 This is a multi-part message in MIME format. --------------030401080100030902090403 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi FreeBSD developpers, I'm Midori Kato. I'm working on the DCTCP implementation in the FreeBSD with Lars Eggert. I mail you because I would like to ask you a code review and testing. The attached patch is not good enough to test our code. Please give me your message. I will send an ECN marking implmenetation in dummynet and test scripts personally to you. A DCTCP paper is published in SIGCOMM 2010. DCTCP is also published as an IETF draft [1]. In our implementation, there are a change to the modular congestion control framework and five changes from the original DCTCP algorithm. I briefly describe each of them as below. <1> A change for the modular congestion control framework DCTCP uses the different ECN processing from RFC3168. We need three functions to do the proper DCTCP ECN processing. a) The kernel decides whether an ECE flag should be set in the outgoing TCP segment. (tcp_input.c) b) The kernel controls congestion if an ECE flag is set in the arriving TCP segment. (tcp_input.c) c) After the outgoing TCP segment is generated, the kernel decides whether an ECT bit should be set in an ECN field of IP header in the outgoing packet. (tcp_output.c) The current framework has no housekeeping functions for (a) and (b). Therefore, I add two functions into the moduler cc framework: ecnpkt_handler() and ect_handler(). - ecnpkt_handler() allows the kernel to do the additional ECN processing by snooping ECN field in IP and TCP headers. As an option, this function check a delayed ACK flag, which tells whether this function is in the delayed ACK. This function returns an integer value. When the return value is set, the kernel force to disable delayed ACK. - ect_handler() allows the kernel to use the different rule from RFC3168 in terms of an ECT marking in the outgoing segment. This function returns an integer value. If the value is set, an ECT bit is set to the outgoing segment. <2> Five changes from the original DCTCP algorithm In order to reflect the DCTCP motivation correctly, the following modifications are included in our patch. First four modifications are prepared for senders and the last modification is prepared for receivers. (1) ECE processing FreeBSD handles ECN as a congestion event but it's not true for DCTCP senders. A DCTCP sender uses ECN as a means to understand the extent of congestions. Therefore, a modified DCTCP sender never enters congestion recovery mode in any situations. (2) selective initial alpha value DCTCP defines alpha as a parameter to see the depth of a congestion. When the alpha value is large, it allows a saw-toothed CWND behavior to a DCTCP sender. A problem is that the alpha value is not reliable during a dozen of RTTs because there is no way to identify the depth of a congestion over a network from the beginning. When considering the alpha reliability, I think the initial alpha should be selective for applications by users. When a user chooses DCTCP for latency-sensitive applications, the initial alpha is preferred. Otherwise, DCTCP senders had better to set the initial alpha value to zero. The default alpha value is set to zero in our implementation. (3) alpha value initialization after an idle period The original DCTCP paper does not define how the sender behaves after idle time. A DCTCP sender resets alpha to the initial value when an idle time happens. The following changes is applied to eliminate a compatibility issue to standard ECN defined in RFC3465. Currently, DCTCP and standard ECN servers have no way to identify which mechanism is working on the peer. Thus, we eliminate the worst situation in a network mixing DCTCP senders/receivers and standard ECN senders/receivers. (4) Emitting CWRs at one-sided senders This change is applied for a situation when a sender uses DCTCP and a reciever uses standard ECN. Under the situation, we find that a DCTCP sender minimizes CWND. Fortunately, the current tcp_input() function complement this change, thus, there is no modification in our patch. (5) delayed ACK at one-sided receivers This change is applied for a situation when a sender uses standard ECN and a reciever uses DCTCP. Under the situation, we find that a standard ECN sender increases smaller CWND than expected when the one-sided DCTCP receiver unsets delayed ACK against a packet with CWR flag. Thus, we always apply delayed ACK only when CWR flag is set in the arriving packet. If you want to understand the detailed background of these modifications, see my thesis [2], especially in section 3 and 4. I'm looking forward to hear from you! Regards, -- Midori [1] http://tools.ietf.org/html/draft-bensley-tcpm-dctcp [2]https://eggert.org/students/kato-thesis.pdf --------------030401080100030902090403 Content-Type: text/plain; charset=UTF-8; name="dctcp.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dctcp.patch" ZGlmZiAtLWdpdCBhL3N5cy9tb2R1bGVzL2NjL01ha2VmaWxlIGIvc3lzL21vZHVsZXMvY2Mv TWFrZWZpbGUKaW5kZXggN2I4NTFmNS4uN2Y0ZTk0ZSAxMDA2NDQKLS0tIGEvc3lzL21vZHVs ZXMvY2MvTWFrZWZpbGUKKysrIGIvc3lzL21vZHVsZXMvY2MvTWFrZWZpbGUKQEAgLTMsNiAr Myw3IEBACiBTVUJESVI9CWNjX2NkZyBcCiAJY2NfY2hkIFwKIAljY19jdWJpYyBcCisJY2Nf ZGN0Y3AgXAogCWNjX2hkIFwKIAljY19odGNwIFwKIAljY192ZWdhcwpkaWZmIC0tZ2l0IGEv c3lzL21vZHVsZXMvY2MvY2NfZGN0Y3AvTWFrZWZpbGUgYi9zeXMvbW9kdWxlcy9jYy9jY19k Y3RjcC9NYWtlZmlsZQpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwLi4zMjkx OWNkCi0tLSAvZGV2L251bGwKKysrIGIvc3lzL21vZHVsZXMvY2MvY2NfZGN0Y3AvTWFrZWZp bGUKQEAgLTAsMCArMSw5IEBACisjICRGcmVlQlNEJAorCisuaW5jbHVkZSA8YnNkLm93bi5t az4KKworLlBBVEg6ICR7LkNVUkRJUn0vLi4vLi4vLi4vbmV0aW5ldC9jYworS01PRD0JY2Nf ZGN0Y3AKK1NSQ1M9CWNjX2RjdGNwLmMKKworLmluY2x1ZGUgPGJzZC5rbW9kLm1rPgpkaWZm IC0tZ2l0IGEvc3lzL25ldGluZXQvY2MuaCBiL3N5cy9uZXRpbmV0L2NjLmgKaW5kZXggMTRi NGE5ZC4uMzgxZjk0ZSAxMDA2NDQKLS0tIGEvc3lzL25ldGluZXQvY2MuaAorKysgYi9zeXMv bmV0aW5ldC9jYy5oCkBAIC0xNDMsNiArMTQzLDEzIEBAIHN0cnVjdCBjY19hbGdvIHsKIAkv KiBDYWxsZWQgd2hlbiBkYXRhIHRyYW5zZmVyIHJlc3VtZXMgYWZ0ZXIgYW4gaWRsZSBwZXJp b2QuICovCiAJdm9pZAkoKmFmdGVyX2lkbGUpKHN0cnVjdCBjY192YXIgKmNjdik7CiAKKwkv KiBDYWxsZWQgZm9yIGFuIGFkZGl0aW9uYWwgRUNOIHByb2Nlc3NpbmcgYXBhcnQgZnJvbSBS RkMzMTY4LiAqLworCWludAkoKmVjbnBrdF9oYW5kbGVyKShzdHJ1Y3QgY2NfdmFyICpjY3Ys IHVpbnQ4X3QgaXB0b3MsIGludCBjd3IsCisJCSAgICBpbnQgaXNfZGVsYXlhY2spOworCisJ LyogQ2FsbGVkIHdoZW4gdGhlIGhvc3QgbWFya3MgRUNOIGNhcGFibGUgdHJhbnNtaXNzaW9u IChFQ1QpLiAqLworCWludAkoKmVjdF9oYW5kbGVyKShzdHJ1Y3QgY2NfdmFyICpjY3YpOwor CiAJU1RBSUxRX0VOVFJZIChjY19hbGdvKSBlbnRyaWVzOwogfTsKIApkaWZmIC0tZ2l0IGEv c3lzL25ldGluZXQvY2MvY2NfZGN0Y3AuYyBiL3N5cy9uZXRpbmV0L2NjL2NjX2RjdGNwLmMK bmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMC4uZDhjZDE2NgotLS0gL2Rldi9u dWxsCisrKyBiL3N5cy9uZXRpbmV0L2NjL2NjX2RjdGNwLmMKQEAgLTAsMCArMSw0NDIgQEAK Ky8qLQorICogQ29weXJpZ2h0IChjKSAyMDA3LTIwMDgKKyAqIAlTd2luYnVybmUgVW5pdmVy c2l0eSBvZiBUZWNobm9sb2d5LCBNZWxib3VybmUsIEF1c3RyYWxpYQorICogQ29weXJpZ2h0 IChjKSAyMDA5LTIwMTAgTGF3cmVuY2UgU3Rld2FydCA8bHN0ZXdhcnRAZnJlZWJzZC5vcmc+ CisgKiBDb3B5cmlnaHQgKGMpIDIwMTQgTWlkb3JpIEthdG8gPGthdG9vbkBzZmMud2lkZS5h ZC5qcD4KKyAqIENvcHlyaWdodCAoYykgMjAxNCBUaGUgRnJlZUJTRCBGb3VuZGF0aW9uCisg KiBBbGwgcmlnaHRzIHJlc2VydmVkLgorICoKKyAqIFJlZGlzdHJpYnV0aW9uIGFuZCB1c2Ug aW4gc291cmNlIGFuZCBiaW5hcnkgZm9ybXMsIHdpdGggb3Igd2l0aG91dAorICogbW9kaWZp Y2F0aW9uLCBhcmUgcGVybWl0dGVkIHByb3ZpZGVkIHRoYXQgdGhlIGZvbGxvd2luZyBjb25k aXRpb25zCisgKiBhcmUgbWV0OgorICogMS4gUmVkaXN0cmlidXRpb25zIG9mIHNvdXJjZSBj b2RlIG11c3QgcmV0YWluIHRoZSBhYm92ZSBjb3B5cmlnaHQKKyAqICAgIG5vdGljZSwgdGhp cyBsaXN0IG9mIGNvbmRpdGlvbnMgYW5kIHRoZSBmb2xsb3dpbmcgZGlzY2xhaW1lci4KKyAq IDIuIFJlZGlzdHJpYnV0aW9ucyBpbiBiaW5hcnkgZm9ybSBtdXN0IHJlcHJvZHVjZSB0aGUg YWJvdmUgY29weXJpZ2h0CisgKiAgICBub3RpY2UsIHRoaXMgbGlzdCBvZiBjb25kaXRpb25z IGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIgaW4gdGhlCisgKiAgICBkb2N1bWVudGF0 aW9uIGFuZC9vciBvdGhlciBtYXRlcmlhbHMgcHJvdmlkZWQgd2l0aCB0aGUgZGlzdHJpYnV0 aW9uLgorICoKKyAqIFRISVMgU09GVFdBUkUgSVMgUFJPVklERUQgQlkgVEhFIEFVVEhPUiBB TkQgQ09OVFJJQlVUT1JTIGBgQVMgSVMnJyBBTkQKKyAqIEFOWSBFWFBSRVNTIE9SIElNUExJ RUQgV0FSUkFOVElFUywgSU5DTFVESU5HLCBCVVQgTk9UIExJTUlURUQgVE8sIFRIRQorICog SU1QTElFRCBXQVJSQU5USUVTIE9GIE1FUkNIQU5UQUJJTElUWSBBTkQgRklUTkVTUyBGT1Ig QSBQQVJUSUNVTEFSIFBVUlBPU0UKKyAqIEFSRSBESVNDTEFJTUVELiBJTiBOTyBFVkVOVCBT SEFMTCBUSEUgQVVUSE9SIE9SIENPTlRSSUJVVE9SUyBCRSBMSUFCTEUKKyAqIEZPUiBBTlkg RElSRUNULCBJTkRJUkVDVCwgSU5DSURFTlRBTCwgU1BFQ0lBTCwgRVhFTVBMQVJZLCBPUiBD T05TRVFVRU5USUFMCisgKiBEQU1BR0VTIChJTkNMVURJTkcsIEJVVCBOT1QgTElNSVRFRCBU TywgUFJPQ1VSRU1FTlQgT0YgU1VCU1RJVFVURSBHT09EUworICogT1IgU0VSVklDRVM7IExP U1MgT0YgVVNFLCBEQVRBLCBPUiBQUk9GSVRTOyBPUiBCVVNJTkVTUyBJTlRFUlJVUFRJT04p CisgKiBIT1dFVkVSIENBVVNFRCBBTkQgT04gQU5ZIFRIRU9SWSBPRiBMSUFCSUxJVFksIFdI RVRIRVIgSU4gQ09OVFJBQ1QsIFNUUklDVAorICogTElBQklMSVRZLCBPUiBUT1JUIChJTkNM VURJTkcgTkVHTElHRU5DRSBPUiBPVEhFUldJU0UpIEFSSVNJTkcgSU4gQU5ZIFdBWQorICog T1VUIE9GIFRIRSBVU0UgT0YgVEhJUyBTT0ZUV0FSRSwgRVZFTiBJRiBBRFZJU0VEIE9GIFRI RSBQT1NTSUJJTElUWSBPRgorICogU1VDSCBEQU1BR0UuCisgKi8KKworLyoKKyAqIEFuIGlt cGxlbWVudGF0aW9uIG9mIHRoZSBEQ1RDUCBhbGdvcml0aG0gZm9yIEZyZWVCU0QsIGJhc2Vk IG9uCisgKiAiRGF0YSBDZW50ZXIgVENQIChEQ1RDUCkiIGJ5IE0uIEFsaXphZGVoLCBBLiBH cmVlbmJlcmcsIEQuIEEuIE1hbHR6LAorICogSi4gUGFkaHllLCBQLiBQYXRlbCwgQi4gUHJh Ymhha2FyLCBTLiBTZW5ndXB0YSwgYW5kIE0uIFNyaWRoYXJhbi4sCisgKiBpbiBBQ00gQ29u ZmVyZW5jZSBvbiBTSUdDT01NIDIwMTAsIE5ldyBZb3JrLCBVU0EsCisgKiBPcmlnaW5hbGx5 IHJlbGVhc2VkIGFzIHRoZSBjb250cmlidXRpb24gb2YgTWljcm9zb2Z0IFJlc2VhcmNoIHBy b2plY3QuCisgKi8KKworI2luY2x1ZGUgPHN5cy9jZGVmcy5oPgorX19GQlNESUQoIiRGcmVl QlNEJCIpOworCisjaW5jbHVkZSA8c3lzL3BhcmFtLmg+CisjaW5jbHVkZSA8c3lzL2tlcm5l bC5oPgorI2luY2x1ZGUgPHN5cy9tYWxsb2MuaD4KKyNpbmNsdWRlIDxzeXMvbW9kdWxlLmg+ CisjaW5jbHVkZSA8c3lzL3NvY2tldC5oPgorI2luY2x1ZGUgPHN5cy9zb2NrZXR2YXIuaD4K KyNpbmNsdWRlIDxzeXMvc3lzY3RsLmg+CisjaW5jbHVkZSA8c3lzL3N5c3RtLmg+CisKKyNp bmNsdWRlIDxuZXQvdm5ldC5oPgorCisjaW5jbHVkZSA8bmV0aW5ldC9pbi5oPgorI2luY2x1 ZGUgPG5ldGluZXQvaXAuaD4KKyNpbmNsdWRlIDxuZXRpbmV0L2NjLmg+CisjaW5jbHVkZSA8 bmV0aW5ldC90Y3Bfc2VxLmg+CisjaW5jbHVkZSA8bmV0aW5ldC90Y3BfdmFyLmg+CisKKyNp bmNsdWRlIDxuZXRpbmV0L2NjL2NjX21vZHVsZS5oPgorCisjZGVmaW5lCUNBU1RfUFRSX0lO VChYKQkoKigoaW50KikoWCkpKQorCitzdGF0aWMgVk5FVF9ERUZJTkUodWludDMyX3QsIGRj dGNwX3NoaWZ0X2cpID0gNDsKK3N0YXRpYyBWTkVUX0RFRklORSh1aW50MzJfdCwgZGN0Y3Bf c2xvd3N0YXJ0KSA9IDA7CisjZGVmaW5lIFZfZGN0Y3Bfc2hpZnRfZwkJVk5FVChkY3RjcF9z aGlmdF9nKQorI2RlZmluZQlWX2RjdGNwX3Nsb3dzdGFydAlWTkVUKGRjdGNwX3Nsb3dzdGFy dCkKKworc3RydWN0IGRjdGNwIHsKKwkvKiAjIG9mIG1hcmtlZCBieXRlcyBkdXJpbmcgYSBS VFQgKi8KKwlpbnQgICAgIGJ5dGVzX2VjbjsKKwkvKiAjIG9mIGFja2VkIGJ5dGVzIGR1cmlu ZyBhIFJUVCAqLworCWludCAgICAgYnl0ZXNfdG90YWw7CisJLyogdGhlIGZyYWN0aW9uIG9m IG1hcmtlZCBieXRlcyAqLworCWludCAgICAgYWxwaGE7CisJLyogQ0Ugc3RhdGUgb2YgdGhl IGxhc3Qgc2VnbWVudCAqLworCWludCAgICAgY2VfcHJldjsKKwkvKiBlbmQgc2VxdWVuY2Ug bnVtYmVyIG9mIHRoZSBjdXJyZW50IHdpbmRvdyAqLworCWludCAgICAgc2F2ZV9zbmRueHQ7 CisJLyogRUNFIGZsYWcgaW4gdGhpcyBzZWdtZW50ICovCisJaW50CWlzX2VjZTsKKwkvKiBF Q0UgZmxhZyBpbiB0aGUgbGFzdCBzZWdtZW50ICovCisJaW50CWVjZV9wcmV2OworCS8qICMg b2YgY29uZ2VzdGlvbiBldmVudHMgKi8KKwl1aW50MzJfdAludW1fY29uZ19ldmVudHM7Cit9 OworCitzdGF0aWMgTUFMTE9DX0RFRklORShNX2RjdGNwLCAiZGN0Y3AgZGF0YSIsCisgICAg IlBlciBjb25uZWN0aW9uIGRhdGEgcmVxdWlyZWQgZm9yIHRoZSBkY3RjcCBhbGdvcml0aG0i KTsKKworc3RhdGljIHZvaWQJZGN0Y3BfYWNrX3JlY2VpdmVkKHN0cnVjdCBjY192YXIgKmNj diwgdWludDE2X3QgdHlwZSk7CitzdGF0aWMgdm9pZAlkY3RjcF9hZnRlcl9pZGxlKHN0cnVj dCBjY192YXIgKmNjdik7CitzdGF0aWMgdm9pZAlkY3RjcF9jYl9kZXN0cm95KHN0cnVjdCBj Y192YXIgKmNjdik7CitzdGF0aWMgaW50CWRjdGNwX2NiX2luaXQoc3RydWN0IGNjX3ZhciAq Y2N2KTsKK3N0YXRpYyB2b2lkCWRjdGNwX2Nvbmdfc2lnbmFsKHN0cnVjdCBjY192YXIgKmNj diwgdWludDMyX3QgdHlwZSk7CitzdGF0aWMgdm9pZAlkY3RjcF9jb25uX2luaXQoc3RydWN0 IGNjX3ZhciAqY2N2KTsKK3N0YXRpYyB2b2lkCWRjdGNwX3Bvc3RfcmVjb3Zlcnkoc3RydWN0 IGNjX3ZhciAqY2N2KTsKK3N0YXRpYyBpbnQJZGN0Y3BfZWNucGt0X2hhbmRsZXIoc3RydWN0 IGNjX3ZhciAqY2N2LCB1aW50OF90IGlwdG9zLCBpbnQgY3dyLAorCQkgICAgaW50IGlzX2Rl bGF5YWNrKTsKK3N0YXRpYyBpbnQJZGN0Y3BfZWN0aGFuZGxlcihzdHJ1Y3QgY2NfdmFyICpj Y3YpOworc3RhdGljIHZvaWQJZGN0Y3BfdXBkYXRlX2FscGhhKHN0cnVjdCBjY192YXIgKmNj dik7CisKK3N0cnVjdCBjY19hbGdvIGRjdGNwX2NjX2FsZ28gPSB7CisJLm5hbWUgPSAiZGN0 Y3AiLAorCS5hY2tfcmVjZWl2ZWQgPSBkY3RjcF9hY2tfcmVjZWl2ZWQsCisJLmNiX2Rlc3Ry b3kgPSBkY3RjcF9jYl9kZXN0cm95LAorCS5jYl9pbml0ID0gZGN0Y3BfY2JfaW5pdCwKKwku Y29uZ19zaWduYWwgPSBkY3RjcF9jb25nX3NpZ25hbCwKKwkuY29ubl9pbml0ID0gZGN0Y3Bf Y29ubl9pbml0LAorCS5wb3N0X3JlY292ZXJ5ID0gZGN0Y3BfcG9zdF9yZWNvdmVyeSwKKwku ZWNucGt0X2hhbmRsZXIgPSBkY3RjcF9lY25wa3RfaGFuZGxlciwKKwkuYWZ0ZXJfaWRsZSA9 IGRjdGNwX2FmdGVyX2lkbGUsCisJLmVjdF9oYW5kbGVyID0gZGN0Y3BfZWN0aGFuZGxlciwK K307CisKK3N0YXRpYyB2b2lkCitkY3RjcF9hY2tfcmVjZWl2ZWQoc3RydWN0IGNjX3ZhciAq Y2N2LCB1aW50MTZfdCB0eXBlKQoreworCXN0cnVjdCBkY3RjcCAqZGN0Y3BfZGF0YTsKKwlp bnQgYnl0ZXNfYWNrZWQgPSAwOworCisJZGN0Y3BfZGF0YSA9IGNjdi0+Y2NfZGF0YTsKKwor CS8qCisJICogRENUQ1AgZG9lc24ndCByZWdhcmQgd2l0aCBFQ04gYXMgYSBjb25nZXN0aW9u LgorCSAqIFRodXMsIERDVENQIGFsd2F5cyBleGVjdXRlcyB0aGUgQUNLIHByb2Nlc3Npbmcg b3V0CisJICogb2YgY29uZ2VzdGlvbiByZWNvdmVyeS4KKwkgKi8KKwlpZiAoSU5fQ09OR1JF Q09WRVJZKENDVihjY3YsIHRfZmxhZ3MpKSkgeworCQlFWElUX0NPTkdSRUNPVkVSWShDQ1Yo Y2N2LCB0X2ZsYWdzKSk7CisJCW5ld3Jlbm9fY2NfYWxnby5hY2tfcmVjZWl2ZWQoY2N2LCB0 eXBlKTsKKwkJRU5URVJfQ09OR1JFQ09WRVJZKENDVihjY3YsIHRfZmxhZ3MpKTsKKwl9IGVs c2UKKwkJbmV3cmVub19jY19hbGdvLmFja19yZWNlaXZlZChjY3YsIHR5cGUpOworCisJLyog VXBkYXRlcyB0aGUgZnJhY3Rpb24gb2YgbWFya2VkIGJ5dGVzLiAqLworCWlmIChDQ1YoY2N2 LCB0X2ZsYWdzKSAmIFRGX0VDTl9QRVJNSVQpIHsKKworCQlpZiAodHlwZSA9PSBDQ19EVVBB Q0spCisJCQlieXRlc19hY2tlZCA9IENDVihjY3YsIHRfbWF4c2VnKTsKKworCQlpZiAodHlw ZSA9PSBDQ19BQ0spCisJCQlieXRlc19hY2tlZCA9IGNjdi0+Ynl0ZXNfdGhpc19hY2s7CisK KwkJLyogVXBkYXRlIHRvdGFsIGJ5dGVzLiAqLworCQlkY3RjcF9kYXRhLT5ieXRlc190b3Rh bCArPSBieXRlc19hY2tlZDsKKworCQkvKiBVcGRhdGUgdG90YWwgbWFya2VkIGJ5dGVzLiAq LworCQlpZiAoZGN0Y3BfZGF0YS0+aXNfZWNlKSB7CisJCQlpZiAoIWRjdGNwX2RhdGEtPmVj ZV9wcmV2CisJCQkgICAgJiYgYnl0ZXNfYWNrZWQgPiBDQ1YoY2N2LCB0X21heHNlZykpIHsK KwkJCQlkY3RjcF9kYXRhLT5ieXRlc19lY24gKz0KKwkJCQkgICAgKGJ5dGVzX2Fja2VkIC0g Q0NWKGNjdiwgdF9tYXhzZWcpKTsKKwkJCX0gZWxzZQorCQkJCWRjdGNwX2RhdGEtPmJ5dGVz X2VjbiArPSBieXRlc19hY2tlZDsKKwkJCWRjdGNwX2RhdGEtPmVjZV9wcmV2ID0gMTsKKwkJ fSBlbHNlIHsKKwkJCWlmIChkY3RjcF9kYXRhLT5lY2VfcHJldgorCQkJICAgICYmIGJ5dGVz X2Fja2VkID4gQ0NWKGNjdiwgdF9tYXhzZWcpKQorCQkJCWRjdGNwX2RhdGEtPmJ5dGVzX2Vj biArPSBDQ1YoY2N2LCB0X21heHNlZyk7CisJCQlkY3RjcF9kYXRhLT5lY2VfcHJldiA9IDA7 CisJCX0KKwkJZGN0Y3BfZGF0YS0+aXNfZWNlID0gMDsKKworCQkvKgorCQkgKiBVcGRhdGUg dGhlIGZyYWN0aW9uIG9mIG1hcmtlZCBieXRlcyBhdCB0aGUgZW5kIG9mCisJCSAqIGN1cnJl bnQgd2luZG93IHNpemUuCisJCSAqLworCQlpZiAoKElOX0ZBU1RSRUNPVkVSWShDQ1YoY2N2 LCB0X2ZsYWdzKSkgJiYKKwkJICAgIFNFUV9HRVEoY2N2LT5jdXJhY2ssIENDVihjY3YsIHNu ZF9yZWNvdmVyKSkpIHx8CisJCSAgICAoIUlOX0ZBU1RSRUNPVkVSWShDQ1YoY2N2LCB0X2Zs YWdzKSkgJiYKKwkJICAgIFNFUV9HVChjY3YtPmN1cmFjaywgZGN0Y3BfZGF0YS0+c2F2ZV9z bmRueHQpKSkKKwkJCWRjdGNwX3VwZGF0ZV9hbHBoYShjY3YpOworCX0KK30KKworc3RhdGlj IHZvaWQKK2RjdGNwX2FmdGVyX2lkbGUoc3RydWN0IGNjX3ZhciAqY2N2KQoreworCXN0cnVj dCBkY3RjcCAqZGN0Y3BfZGF0YTsKKworCWRjdGNwX2RhdGEgPSBjY3YtPmNjX2RhdGE7CisK KwkvKiBJbml0aWFsaXplIGludGVybmFsIHBhcmFtZXRlcnMgYWZ0ZXIgaWRsZSB0aW1lICov CisJZGN0Y3BfZGF0YS0+Ynl0ZXNfZWNuID0gMDsKKwlkY3RjcF9kYXRhLT5ieXRlc190b3Rh bCA9IDA7CisJZGN0Y3BfZGF0YS0+c2F2ZV9zbmRueHQgPSBDQ1YoY2N2LCBzbmRfbnh0KTsK KwlkY3RjcF9kYXRhLT5hbHBoYSA9IDA7CisJZGN0Y3BfZGF0YS0+aXNfZWNlID0gMDsKKwlk Y3RjcF9kYXRhLT5lY2VfcHJldiA9IDA7CisJZGN0Y3BfZGF0YS0+bnVtX2NvbmdfZXZlbnRz ID0gMDsKKworCWRjdGNwX2NjX2FsZ28uYWZ0ZXJfaWRsZSA9IG5ld3Jlbm9fY2NfYWxnby5h ZnRlcl9pZGxlOworfQorCitzdGF0aWMgdm9pZAorZGN0Y3BfY2JfZGVzdHJveShzdHJ1Y3Qg Y2NfdmFyICpjY3YpCit7CisJaWYgKGNjdi0+Y2NfZGF0YSAhPSBOVUxMKQorCQlmcmVlKGNj di0+Y2NfZGF0YSwgTV9kY3RjcCk7Cit9CisKK3N0YXRpYyBpbnQKK2RjdGNwX2NiX2luaXQo c3RydWN0IGNjX3ZhciAqY2N2KQoreworCXN0cnVjdCBkY3RjcCAqZGN0Y3BfZGF0YTsKKwor CWRjdGNwX2RhdGEgPSBtYWxsb2Moc2l6ZW9mKHN0cnVjdCBkY3RjcCksIE1fZGN0Y3AsIE1f Tk9XQUlUfE1fWkVSTyk7CisKKwlpZiAoZGN0Y3BfZGF0YSA9PSBOVUxMKQorCQlyZXR1cm4g KEVOT01FTSk7CisKKwkvKiBJbml0aWFsaXplIHNvbWUga2V5IHZhcmlhYmxlcyB3aXRoIHNl bnNpYmxlIGRlZmF1bHRzLiAqLworCWRjdGNwX2RhdGEtPmJ5dGVzX2VjbiA9IDA7CisJZGN0 Y3BfZGF0YS0+Ynl0ZXNfdG90YWwgPSAwOworCWRjdGNwX2RhdGEtPmFscGhhID0gMDsKKwlk Y3RjcF9kYXRhLT5zYXZlX3NuZG54dCA9IDA7CisJZGN0Y3BfZGF0YS0+Y2VfcHJldiA9IDA7 CisJZGN0Y3BfZGF0YS0+aXNfZWNlID0gMDsKKwlkY3RjcF9kYXRhLT5lY2VfcHJldiA9IDA7 CisJZGN0Y3BfZGF0YS0+bnVtX2NvbmdfZXZlbnRzID0gMDsKKworCWNjdi0+Y2NfZGF0YSA9 IGRjdGNwX2RhdGE7CisJcmV0dXJuICgwKTsKK30KKworLyoKKyAqIFBlcmZvcm0gYW55IG5l Y2Vzc2FyeSB0YXNrcyBiZWZvcmUgd2UgZW50ZXIgY29uZ2VzdGlvbiByZWNvdmVyeS4KKyAq Lworc3RhdGljIHZvaWQKK2RjdGNwX2Nvbmdfc2lnbmFsKHN0cnVjdCBjY192YXIgKmNjdiwg dWludDMyX3QgdHlwZSkKK3sKKwlzdHJ1Y3QgZGN0Y3AgKmRjdGNwX2RhdGE7CisJdV9pbnQg d2luLCBtc3M7CisKKwlkY3RjcF9kYXRhID0gY2N2LT5jY19kYXRhOworCXdpbiA9IENDVihj Y3YsIHNuZF9jd25kKTsKKwltc3MgPSBDQ1YoY2N2LCB0X21heHNlZyk7CisKKwlzd2l0Y2gg KHR5cGUpIHsKKwljYXNlIENDX05EVVBBQ0s6CisJCWlmICghSU5fRkFTVFJFQ09WRVJZKEND VihjY3YsIHRfZmxhZ3MpKSkgeworCQkJaWYgKCFJTl9DT05HUkVDT1ZFUlkoQ0NWKGNjdiwg dF9mbGFncykpKSB7CisJCQkJQ0NWKGNjdiwgc25kX3NzdGhyZXNoKSA9IG1zcyAqCisJCQkJ ICAgIG1heCh3aW4gLyAyIC8gbXNzLCAyKTsKKwkJCQlkY3RjcF9kYXRhLT5udW1fY29uZ19l dmVudHMrKzsKKwkJCX0gZWxzZSB7CisJCQkJLyogY3duZCBoYXMgYWxyZWFkeSB1cGRhdGVk IGFzIGNvbmdlc3Rpb24KKwkJCQkgKiByZWNvdmVyeS4gUmV2ZXJzZSBjd25kIHZhbHVlIHVz aW5nCisJCQkJICogc25kX2N3bmRfcHJldiBhbmQgcmVjYWxjdWxhdGUgc25kX3NzdGhyZXNo CisJCQkJICovCisJCQkJd2luID0gQ0NWKGNjdiwgc25kX2N3bmRfcHJldik7CisJCQkJQ0NW KGNjdiwgc25kX3NzdGhyZXNoKSA9CisJCQkJICAgIG1heCh3aW4gLyAyIC8gbXNzLCAyKSAq IG1zczsKKwkJCX0KKwkJCUVOVEVSX1JFQ09WRVJZKENDVihjY3YsIHRfZmxhZ3MpKTsKKwkJ fQorCQlicmVhazsKKwljYXNlIENDX0VDTjoKKwkJLyoKKwkJICogU2F2ZSBjdXJyZW50IHNu ZF9jd25kIHdoZW4gdGhlIGhvc3QgZW5jb3VudGVycyBib3RoCisJCSAqIGNvbmdlc3Rpb24g cmVjb3ZlcnkgYW5kIGZhc3QgcmVjb3ZlcnkuCisJCSAqLworCQlDQ1YoY2N2LCBzbmRfY3du ZF9wcmV2KSA9IHdpbjsKKwkJaWYgKCFJTl9DT05HUkVDT1ZFUlkoQ0NWKGNjdiwgdF9mbGFn cykpKSB7CisJCQlpZiAoVl9kY3RjcF9zbG93c3RhcnQgJiYKKwkJCSAgICBkY3RjcF9kYXRh LT5udW1fY29uZ19ldmVudHMrKyA9PSAwKSB7CisJCQkJQ0NWKGNjdiwgc25kX3NzdGhyZXNo KSA9CisJCQkJICAgIG1zcyAqIG1heCh3aW4gLyAyIC8gbXNzLCAyKTsKKwkJCQlkY3RjcF9k YXRhLT5hbHBoYSA9IDEwMjQ7CisJCQkJZGN0Y3BfZGF0YS0+Ynl0ZXNfZWNuID0gMDsKKwkJ CQlkY3RjcF9kYXRhLT5ieXRlc190b3RhbCA9IDA7CisJCQkJZGN0Y3BfZGF0YS0+c2F2ZV9z bmRueHQgPSBDQ1YoY2N2LCBzbmRfbnh0KTsKKwkJCX0gZWxzZQorCQkJCUNDVihjY3YsIHNu ZF9zc3RocmVzaCkgPSBtYXgoKHdpbiAtICgod2luICoKKwkJCQkgICAgZGN0Y3BfZGF0YS0+ YWxwaGEpID4+IDExKSkgLyBtc3MsIDIpICogbXNzOworCQkJQ0NWKGNjdiwgc25kX2N3bmQp ID0gQ0NWKGNjdiwgc25kX3NzdGhyZXNoKTsKKwkJCUVOVEVSX0NPTkdSRUNPVkVSWShDQ1Yo Y2N2LCB0X2ZsYWdzKSk7CisJCX0KKwkJZGN0Y3BfZGF0YS0+aXNfZWNlID0gMTsKKwkJYnJl YWs7CisJY2FzZSBDQ19SVE86CisJCWlmIChDQ1YoY2N2LCB0X2ZsYWdzKSAmIFRGX0VDTl9Q RVJNSVQpIHsKKwkJCUNDVihjY3YsIHRfZmxhZ3MpIHw9IFRGX0VDTl9TTkRfQ1dSOworCQkJ ZGN0Y3BfdXBkYXRlX2FscGhhKGNjdik7CisJCQlkY3RjcF9kYXRhLT5zYXZlX3NuZG54dCAr PSBDQ1YoY2N2LCB0X21heHNlZyk7CisJCQlkY3RjcF9kYXRhLT5udW1fY29uZ19ldmVudHMr KzsKKwkJfQorCQlicmVhazsKKwl9Cit9CisKK3N0YXRpYyB2b2lkCitkY3RjcF9jb25uX2lu aXQoc3RydWN0IGNjX3ZhciAqY2N2KQoreworCXN0cnVjdCBkY3RjcCAqZGN0Y3BfZGF0YTsK KworCWRjdGNwX2RhdGEgPSBjY3YtPmNjX2RhdGE7CisKKwlpZiAoQ0NWKGNjdiwgdF9mbGFn cykgJiBURl9FQ05fUEVSTUlUKQorCQlkY3RjcF9kYXRhLT5zYXZlX3NuZG54dCA9IENDVihj Y3YsIHNuZF9ueHQpOworfQorCisvKgorICogUGVyZm9ybSBhbnkgbmVjZXNzYXJ5IHRhc2tz IGJlZm9yZSB3ZSBleGl0IGNvbmdlc3Rpb24gcmVjb3ZlcnkuCisgKi8KK3N0YXRpYyB2b2lk CitkY3RjcF9wb3N0X3JlY292ZXJ5KHN0cnVjdCBjY192YXIgKmNjdikKK3sKKwlkY3RjcF9j Y19hbGdvLnBvc3RfcmVjb3ZlcnkgPSBuZXdyZW5vX2NjX2FsZ28ucG9zdF9yZWNvdmVyeTsK KworCWlmIChDQ1YoY2N2LCB0X2ZsYWdzKSAmIFRGX0VDTl9QRVJNSVQpCisJCWRjdGNwX3Vw ZGF0ZV9hbHBoYShjY3YpOworfQorCitzdGF0aWMgaW50CitkY3RjcF9lY25wa3RfaGFuZGxl cihzdHJ1Y3QgY2NfdmFyICpjY3YsIHVpbnQ4X3QgaXB0b3MsIGludCBjd3IsIGludCBpc19k ZWxheWFjaykKK3sKKwlzdHJ1Y3QgZGN0Y3AgKmRjdGNwX2RhdGE7CisJaW50IHJldCA9IDA7 CisKKwlkY3RjcF9kYXRhID0gY2N2LT5jY19kYXRhOworCS8qCisJICogRENUQ1AgcmVzcG9u c2VzIGFuIEFDSyBpbW1lZGlhdGVseQorCSAqIC0gd2hlbiB0aGUgQ0Ugc3RhdGUgaW4gYmV0 d2VlbiB0aGlzIHNlZ21lbnQKKwkgKiAgIGFuZCB0aGUgbGFzdCBzZWdtZW50IGlzIG5vdCBz YW1lCisJICogLSB3aGVuIHRoaXMgc2VnbWVudCBzZXRzIHRoZSBDV1IgZmxhZworCSAqLwor CXN3aXRjaCAoaXB0b3MgJiBJUFRPU19FQ05fTUFTSykgeworCWNhc2UgSVBUT1NfRUNOX0NF OgorCQlpZiAoIWRjdGNwX2RhdGEtPmNlX3ByZXYgJiYgaXNfZGVsYXlhY2spCisJCQlyZXQg PSAxOworCQlkY3RjcF9kYXRhLT5jZV9wcmV2ID0gMTsKKwkJQ0NWKGNjdiwgdF9mbGFncykg fD0gVEZfRUNOX1NORF9FQ0U7CisJCWJyZWFrOworCWNhc2UgSVBUT1NfRUNOX0VDVDA6CisJ CWlmIChkY3RjcF9kYXRhLT5jZV9wcmV2ICYmIGlzX2RlbGF5YWNrKQorCQkJcmV0ID0gMTsK KwkJQ0NWKGNjdiwgdF9mbGFncykgJj0gflRGX0VDTl9TTkRfRUNFOworCQlkY3RjcF9kYXRh LT5jZV9wcmV2ID0gMDsKKwkJYnJlYWs7CisJY2FzZSBJUFRPU19FQ05fRUNUMToKKwkJaWYg KGRjdGNwX2RhdGEtPmNlX3ByZXYgJiYgaXNfZGVsYXlhY2spCisJCQlyZXQgPSAxOworCQlD Q1YoY2N2LCB0X2ZsYWdzKSAmPSB+VEZfRUNOX1NORF9FQ0U7CisJCWRjdGNwX2RhdGEtPmNl X3ByZXYgPSAwOworCQlicmVhazsKKwl9CisJaWYgKGN3ciAmJiBpc19kZWxheWFjaykKKwkJ cmV0ID0gMDsKKworCXJldHVybiAocmV0KTsKK30KKworc3RhdGljIGludAorZGN0Y3BfZWN0 aGFuZGxlcihzdHJ1Y3QgY2NfdmFyICpjY3YpCit7CisJLyogRENUQ1AgYWx3YXlzIG1hcmtz IEVDVCAqLworCXJldHVybiAoMSk7Cit9CisKKy8qCisgKiBVcGRhdGUgdGhlIGZyYWN0aW9u IG9mIG1hcmtlZCBieXRlcyBuYW1lZCBhbHBoYS4gVGhlbiwgaW5pdGlhbGl6ZQorICogc2V2 ZXJhbCBpbnRlcm5hbCBwYXJhbWV0ZXJzIGF0IHRoZSBlbmQgb2YgdGhpcyBmdW5jdGlvbi4K KyAqLworc3RhdGljIHZvaWQKK2RjdGNwX3VwZGF0ZV9hbHBoYShzdHJ1Y3QgY2NfdmFyICpj Y3YpCit7CisJc3RydWN0IGRjdGNwICpkY3RjcF9kYXRhOworCWludCBhbHBoYV9wcmV2Owor CisJZGN0Y3BfZGF0YSA9IGNjdi0+Y2NfZGF0YTsKKworCWFscGhhX3ByZXYgPSBkY3RjcF9k YXRhLT5hbHBoYTsKKworCWRjdGNwX2RhdGEtPmJ5dGVzX3RvdGFsID0gbWF4KGRjdGNwX2Rh dGEtPmJ5dGVzX3RvdGFsLCAxKTsKKworCS8qCisJICogVXBkYXRlIGFscGhhOiBhbHBoYSA9 ICgxIC0gZykgKiBhbHBoYSArIGcgKiBGLgorCSAqIEFscGhhIG11c3QgYmUgcm91bmQgdG8g MCAtIDEwMjQuCisJICogWFhYTUlET1JJIElzIG1vcmUgZmluZS1ncmFpbmVkIGFscGhhIG5l Y2Vzc2FyeT8KKwkgKi8KKwlkY3RjcF9kYXRhLT5hbHBoYSA9IG1pbihhbHBoYV9wcmV2IC0g KGFscGhhX3ByZXYgPj4gVl9kY3RjcF9zaGlmdF9nKSArCisJICAgIChkY3RjcF9kYXRhLT5i eXRlc19lY24gPDwgKDEwIC0gVl9kY3RjcF9zaGlmdF9nKSkgLworCSAgICBkY3RjcF9kYXRh LT5ieXRlc190b3RhbCwgMTAyNCk7CisKKwkvKiBJbml0aWFsaXplIGludGVybmFsIHBhcmFt ZXRlcnMgZm9yIG5leHQgYWxwaGEgY2FsY3VsYXRpb24gKi8KKwlkY3RjcF9kYXRhLT5ieXRl c19lY24gPSAwOworCWRjdGNwX2RhdGEtPmJ5dGVzX3RvdGFsID0gMDsKKwlkY3RjcF9kYXRh LT5zYXZlX3NuZG54dCA9IENDVihjY3YsIHNuZF9ueHQpOworfQorCitzdGF0aWMgaW50Citk Y3RjcF9zaGlmdF9nX2hhbmRsZXIoU1lTQ1RMX0hBTkRMRVJfQVJHUykKK3sKKwlpbnQgZXJy b3I7CisJdWludDMyX3QgbmV3OworCisJbmV3ID0gVl9kY3RjcF9zaGlmdF9nIDsKKwllcnJv ciA9IHN5c2N0bF9oYW5kbGVfaW50KG9pZHAsICZuZXcsIDAsIHJlcSk7CisJaWYgKGVycm9y ID09IDAgJiYgcmVxLT5uZXdwdHIgIT0gTlVMTCkgeworCQlpZiAoQ0FTVF9QVFJfSU5UKHJl cS0+bmV3cHRyKSA+IDEpCisJCQllcnJvciA9IEVJTlZBTDsKKwkJZWxzZQorCQkJVl9kY3Rj cF9zaGlmdF9nID0gbmV3OworCX0KKworCXJldHVybiAoZXJyb3IpOworfQorCitzdGF0aWMg aW50CitkY3RjcF9zbG93c3RhcnRfaGFuZGxlcihTWVNDVExfSEFORExFUl9BUkdTKQorewor CWludCBlcnJvcjsKKwl1aW50MzJfdCBuZXc7CisKKwluZXcgPSBWX2RjdGNwX3Nsb3dzdGFy dDsKKwllcnJvciA9IHN5c2N0bF9oYW5kbGVfaW50KG9pZHAsICZuZXcsIDAsIHJlcSk7CisJ aWYgKGVycm9yID09IDAgJiYgcmVxLT5uZXdwdHIgIT0gTlVMTCkgeworCQlpZiAoQ0FTVF9Q VFJfSU5UKHJlcS0+bmV3cHRyKSA+IDEpCisJCQllcnJvciA9IEVJTlZBTDsKKwkJZWxzZQor CQkJVl9kY3RjcF9zbG93c3RhcnQgPSBuZXc7CisJfQorCisJcmV0dXJuIChlcnJvcik7Cit9 CisKK1NZU0NUTF9ERUNMKF9uZXRfaW5ldF90Y3BfY2NfZGN0Y3ApOworU1lTQ1RMX05PREUo X25ldF9pbmV0X3RjcF9jYywgT0lEX0FVVE8sIGRjdGNwLCBDVExGTEFHX1JXLCBOVUxMLAor ICAgICJkY3RjcCBjb25nZXN0aW9uIGNvbnRyb2wgcmVsYXRlZCBzZXR0aW5ncyIpOworCitT WVNDVExfVk5FVF9QUk9DKF9uZXRfaW5ldF90Y3BfY2NfZGN0Y3AsIE9JRF9BVVRPLCBzaGlm dF9nLAorICAgIENUTFRZUEVfVUlOVHxDVExGTEFHX1JXLCAmVk5FVF9OQU1FKGRjdGNwX3No aWZ0X2cpLCA0LAorICAgICZkY3RjcF9zaGlmdF9nX2hhbmRsZXIsCisgICAgIklVIiwgImRj dGNwIHNoaWZ0IHBhcmFtZXRlciIpOworCitTWVNDVExfVk5FVF9QUk9DKF9uZXRfaW5ldF90 Y3BfY2NfZGN0Y3AsIE9JRF9BVVRPLCBzbG93c3RhcnQsCisgICAgQ1RMVFlQRV9VSU5UfENU TEZMQUdfUlcsICZWTkVUX05BTUUoZGN0Y3Bfc2xvd3N0YXJ0KSwgMCwKKyAgICAmZGN0Y3Bf c2xvd3N0YXJ0X2hhbmRsZXIsCisgICAgIklVIiwgImhhbGYgQ1dORCByZWR1Y3Rpb24gYWZ0 ZXIgdGhlIGZpcnN0IHNsb3cgc3RhcnQiKTsKKworREVDTEFSRV9DQ19NT0RVTEUoZGN0Y3As ICZkY3RjcF9jY19hbGdvKTsKZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9pbnB1dC5j IGIvc3lzL25ldGluZXQvdGNwX2lucHV0LmMKaW5kZXggMjBjMjJlZC4uMjgyMjI0OCAxMDA2 NDQKLS0tIGEvc3lzL25ldGluZXQvdGNwX2lucHV0LmMKKysrIGIvc3lzL25ldGluZXQvdGNw X2lucHV0LmMKQEAgLTQ1NSw2ICs0NTUsMzIgQEAgY2NfcG9zdF9yZWNvdmVyeShzdHJ1Y3Qg dGNwY2IgKnRwLCBzdHJ1Y3QgdGNwaGRyICp0aCkKIAl0cC0+dF9ieXRlc19hY2tlZCA9IDA7 CiB9CiAKKy8qCisgKiBJbmRpY2F0ZSB3aGV0aGVyIHRoaXMgYWNrIHNob3VsZCBiZSBkZWxh eWVkLiAgV2UgY2FuIGRlbGF5IHRoZSBhY2sgaWYKKyAqCS0gdGhlcmUgaXMgbm8gZGVsYXll ZCBhY2sgdGltZXIgaW4gcHJvZ3Jlc3MgYW5kCisgKgktIG91ciBsYXN0IGFjayB3YXNuJ3Qg YSAwLXNpemVkIHdpbmRvdy4gIFdlIG5ldmVyIHdhbnQgdG8gZGVsYXkKKyAqCSAgdGhlIGFj ayB0aGF0IG9wZW5zIHVwIGEgMC1zaXplZCB3aW5kb3cgYW5kCisgKgkJLSBkZWxheWVkIGFj a3MgYXJlIGVuYWJsZWQgb3IKKyAqCQktIHRoaXMgaXMgYSBoYWxmLXN5bmNocm9uaXplZCBU L1RDUCBjb25uZWN0aW9uLgorICovCisjZGVmaW5lIERFTEFZX0FDSyh0cCkJCQkJCQkJXAor CSgoIXRjcF90aW1lcl9hY3RpdmUodHAsIFRUX0RFTEFDSykgJiYJCQkJXAorCSAgICAodHAt PnRfZmxhZ3MgJiBURl9SWFdJTjBTRU5UKSA9PSAwKSAmJgkJCVwKKwkgICAgKFZfdGNwX2Rl bGFja19lbmFibGVkIHx8ICh0cC0+dF9mbGFncyAmIFRGX05FRURTWU4pKSkKKworc3RhdGlj IHZvaWQgaW5saW5lCitjY19lY25wa3RfaGFuZGxlcihzdHJ1Y3QgdGNwY2IgKnRwLCBzdHJ1 Y3QgdGNwaGRyICp0aCwgdWludDhfdCBpcHRvcykKK3sKKwlJTlBfV0xPQ0tfQVNTRVJUKHRw LT50X2lucGNiKTsKKworCWlmIChDQ19BTEdPKHRwKS0+ZWNucGt0X2hhbmRsZXIgIT0gTlVM TCkgeworCQlpZiAoQ0NfQUxHTyh0cCktPmVjbnBrdF9oYW5kbGVyKHRwLT5jY3YsIGlwdG9z LAorCQkgICAgKHRoLT50aF9mbGFncyAmIFRIX0NXUiksIERFTEFZX0FDSyh0cCkpKSB7CisJ CQl0Y3BfdGltZXJfYWN0aXZhdGUodHAsIFRUX0RFTEFDSywgdGNwX2RlbGFja3RpbWUpOwor CQl9CisJfQorfQorCiBzdGF0aWMgaW5saW5lIHZvaWQKIHRjcF9maWVsZHNfdG9faG9zdChz dHJ1Y3QgdGNwaGRyICp0aCkKIHsKQEAgLTUwMiwxOSArNTI4LDYgQEAgZG8geyBcCiAjZW5k aWYKIAogLyoKLSAqIEluZGljYXRlIHdoZXRoZXIgdGhpcyBhY2sgc2hvdWxkIGJlIGRlbGF5 ZWQuICBXZSBjYW4gZGVsYXkgdGhlIGFjayBpZgotICoJLSB0aGVyZSBpcyBubyBkZWxheWVk IGFjayB0aW1lciBpbiBwcm9ncmVzcyBhbmQKLSAqCS0gb3VyIGxhc3QgYWNrIHdhc24ndCBh IDAtc2l6ZWQgd2luZG93LiAgV2UgbmV2ZXIgd2FudCB0byBkZWxheQotICoJICB0aGUgYWNr IHRoYXQgb3BlbnMgdXAgYSAwLXNpemVkIHdpbmRvdyBhbmQKLSAqCQktIGRlbGF5ZWQgYWNr cyBhcmUgZW5hYmxlZCBvcgotICoJCS0gdGhpcyBpcyBhIGhhbGYtc3luY2hyb25pemVkIFQv VENQIGNvbm5lY3Rpb24uCi0gKi8KLSNkZWZpbmUgREVMQVlfQUNLKHRwKQkJCQkJCQlcCi0J KCghdGNwX3RpbWVyX2FjdGl2ZSh0cCwgVFRfREVMQUNLKSAmJgkJCQlcCi0JICAgICh0cC0+ dF9mbGFncyAmIFRGX1JYV0lOMFNFTlQpID09IDApICYmCQkJXAotCSAgICAoVl90Y3BfZGVs YWNrX2VuYWJsZWQgfHwgKHRwLT50X2ZsYWdzICYgVEZfTkVFRFNZTikpKQotCi0vKgogICog VENQIGlucHV0IGhhbmRsaW5nIGlzIHNwbGl0IGludG8gbXVsdGlwbGUgcGFydHM6CiAgKiAg IHRjcDZfaW5wdXQgaXMgYSB0aGluIHdyYXBwZXIgYXJvdW5kIHRjcF9pbnB1dCBmb3IgdGhl IGV4dGVuZGVkCiAgKglpcDZfcHJvdG94W10gY2FsbCBmb3JtYXQgaW4gaXA2X2lucHV0CkBA IC0xNTM5LDYgKzE1NTIsMTAgQEAgdGNwX2RvX3NlZ21lbnQoc3RydWN0IG1idWYgKm0sIHN0 cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1Y3Qgc29ja2V0ICpzbywKIAkJCVRDUFNUQVRfSU5DKHRj cHNfZWNuX2VjdDEpOwogCQkJYnJlYWs7CiAJCX0KKworCQkvKiBQcm9jZXNzIGEgcGFja2V0 IGRpZmZlcmVudGx5IGZyb20gUkZDMzE2OC4gKi8KKwkJY2NfZWNucGt0X2hhbmRsZXIodHAs IHRoLCBpcHRvcyk7CisKIAkJLyogQ29uZ2VzdGlvbiBleHBlcmllbmNlZC4gKi8KIAkJaWYg KHRoZmxhZ3MgJiBUSF9FQ0UpIHsKIAkJCWNjX2Nvbmdfc2lnbmFsKHRwLCB0aCwgQ0NfRUNO KTsKZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9vdXRwdXQuYyBiL3N5cy9uZXRpbmV0 L3RjcF9vdXRwdXQuYwppbmRleCAwMGQ1NDE1Li4zMGU5YjE5IDEwMDY0NAotLS0gYS9zeXMv bmV0aW5ldC90Y3Bfb3V0cHV0LmMKKysrIGIvc3lzL25ldGluZXQvdGNwX291dHB1dC5jCkBA IC0xNjIsNiArMTYyLDE4IEBAIGNjX2FmdGVyX2lkbGUoc3RydWN0IHRjcGNiICp0cCkKIAkJ Q0NfQUxHTyh0cCktPmFmdGVyX2lkbGUodHAtPmNjdik7CiB9CiAKK3N0YXRpYyBpbnQgaW5s aW5lCitjY19lY3RfaGFuZGxlcihzdHJ1Y3QgdGNwY2IgKnRwKQoreworCUlOUF9XTE9DS19B U1NFUlQodHAtPnRfaW5wY2IpOworCisJaWYgKENDX0FMR08odHApLT5lY3RfaGFuZGxlciAh PSBOVUxMKSB7CisJCWlmIChDQ19BTEdPKHRwKS0+ZWN0X2hhbmRsZXIodHAtPmNjdikpCisJ CQlyZXR1cm4gKDEpOworCX0KKwlyZXR1cm4gKDApOworfQorCiAvKgogICogVGNwIG91dHB1 dCByb3V0aW5lOiBmaWd1cmUgb3V0IHdoYXQgc2hvdWxkIGJlIHNlbnQgYW5kIHNlbmQgaXQu CiAgKi8KQEAgLTk2Niw5ICs5NzgsMTUgQEAgc2VuZDoKIAkJICogSWYgdGhlIHBlZXIgaGFz IEVDTiwgbWFyayBkYXRhIHBhY2tldHMgd2l0aAogCQkgKiBFQ04gY2FwYWJsZSB0cmFuc21p c3Npb24gKEVDVCkuCiAJCSAqIElnbm9yZSBwdXJlIGFjayBwYWNrZXRzLCByZXRyYW5zbWlz c2lvbnMgYW5kIHdpbmRvdyBwcm9iZXMuCisJCSAqIE1hcmsgZGF0YSBwYWNrZXQgd2l0aCBF Q04gY2FwYWJsZSB0cmFuc21pc3Npb24gKEVDVCkKKwkJICogd2hlbiBDQ19BTEdPIG1lZXRz IHNwZWNpZmljIGNvbmRpdGlvbi4KKwkJICogT3IsIGlmIHRoZSBwZWVyIGhhcyBFQ04sIG1h cmsgZGF0YSBwYWNrZXRzIHdpdGggRUNUCisJCSAqIChSRkMgMzE2OCkuIElnbm9yZSBwdXJl IGFjayBwYWNrZXRzLCByZXRyYW5zbWlzc2lvbnMKKwkJICogYW5kIHdpbmRvdyBwcm9iZXMu CiAJCSAqLwotCQlpZiAobGVuID4gMCAmJiBTRVFfR0VRKHRwLT5zbmRfbnh0LCB0cC0+c25k X21heCkgJiYKLQkJICAgICEoKHRwLT50X2ZsYWdzICYgVEZfRk9SQ0VEQVRBKSAmJiBsZW4g PT0gMSkpIHsKKwkJaW50IG1hcmtfZWN0ID0gY2NfZWN0X2hhbmRsZXIodHApOworCQlpZiAo bWFya19lY3QgfHwgKGxlbiA+IDAgJiYgU0VRX0dFUSh0cC0+c25kX254dCwgdHAtPnNuZF9t YXgpCisJCSAgICAmJiAhKCh0cC0+dF9mbGFncyAmIFRGX0ZPUkNFREFUQSkgJiYgbGVuID09 IDEpKSkgewogI2lmZGVmIElORVQ2CiAJCQlpZiAoaXNpcHY2KQogCQkJCWlwNi0+aXA2X2Zs b3cgfD0gaHRvbmwoSVBUT1NfRUNOX0VDVDAgPDwgMjApOwo= --------------030401080100030902090403-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 11:06:48 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 538B7A53 for ; Mon, 31 Mar 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FB32BA3 for ; Mon, 31 Mar 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2VB6mfU058763 for ; Mon, 31 Mar 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2VB6lp2058761 for freebsd-net@FreeBSD.org; Mon, 31 Mar 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 31 Mar 2014 11:06:47 GMT Message-Id: <201403311106.s2VB6lp2058761@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 466 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 13:32:07 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF08B4FD for ; Mon, 31 Mar 2014 13:32:07 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A5147DAC for ; Mon, 31 Mar 2014 13:32:07 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=NotTgrhJ c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=qZLpgtK1PmoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=b_2RNOecpp4LzuSYSBIA:9 a=QEXdDO2ut3YA:10 a=vH3ey5Qa1pYA:10 a=XyGgOm23cikA:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp01.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:2333] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id 3F/28-15227-65E69335; Mon, 31 Mar 2014 09:32:06 -0400 Message-ID: <53396E4B.4050102@rcn.com> Date: Mon, 31 Mar 2014 09:31:55 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: questions about (system) dhcp Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 13:32:07 -0000 Hello: Is this the correct place to ask? Respectfully, Robert Huff From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 14:54:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 553CCF99; Mon, 31 Mar 2014 14:54:50 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 32BEB9B9; Mon, 31 Mar 2014 14:54:49 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s2VEvXJr097997; Mon, 31 Mar 2014 07:57:39 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s2VEvQ5E097993; Mon, 31 Mar 2014 07:57:26 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Mon, 31 Mar 2014 07:57:28 -0700 (PDT) Message-ID: In-Reply-To: <20140331050002.GC1359@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> Date: Mon, 31 Mar 2014 07:57:28 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 14:54:50 -0000 > On Sun, Mar 30, 2014 at 01:12:20PM -0700, chrish@UltimateDNS.NET wrote: >> Greetings, >> I'm not sure whether this best belonged on net@, or stable@ >> so I'm using both. :) >> I'm testing both releng_9, and MB, and I encountered a new >> message I don't usually see using the nfe(4) driver: >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> ... >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >> >> Truncated for brevity (31 lines in total; 1-31). I don't know >> how interpret this. An issue with my version of the driver, or >> the hardware itself? This occurred with both GENERIC, as well >> as my custom kernel. > > Would you show me the dmesg output? Happily: Calibrating TSC clock ... TSC clock: 3231132841 Hz CPU: AMD Sempron(tm) 140 Processor (3231.13-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f62 Family = 0x10 Model = 0x6 Stepping = 2 Features=0x78bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x37fd TSC: P-state invariant L1 2MB data TLB: 48 entries, fully associative L1 2MB instruction TLB: 16 entries, fully associative L1 4KB data TLB: 48 entries, fully associative L1 4KB instruction TLB: 32 entries, fully associative L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L2 2MB data TLB: 128 entries, 2-way associative L2 2MB instruction TLB: 0 entries, 2-way associative L2 4KB data TLB: 512 entries, 4-way associative L2 4KB instruction TLB: 512 entries, 4-way associative L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative real memory = 2147483648 (2048 MB) Physical memory chunk(s): 0x0000000000010000 - 0x000000000009bfff, 573440 bytes (140 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x0000000001dcc000 - 0x000000006cab2fff, 1791913984 bytes (437479 pages) avail memory = 1777266688 (1694 MB) INTR: Adding local APIC 0 as a target Event timer "LAPIC" quality 400 ACPI APIC Table: <7309MS A7309200> x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 x86bios: SSEG 0x098000-0x098fff at 0xffffff8000226000 x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 APIC: CPU 0 has ACPI ID 1 ULE: setup cpu 0 ACPI: RSDP 0xfaab0 00014 (v00 ACPIAM) ACPI: RSDT 0x6ffa0000 00048 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: FACP 0x6ffa0200 00084 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: DSDT 0x6ffa04b0 0503B (v01 A7309 A7309200 00000200 INTL 20051117) ACPI: FACS 0x6ffae000 00040 ACPI: APIC 0x6ffa0390 00090 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: MCFG 0x6ffa0420 0003C (v01 7309MS OEMMCFG 20101122 MSFT 00000097) ACPI: WDRT 0x6ffa0460 00047 (v01 7309MS NV-WDRT 20101122 MSFT 00000097) ACPI: OEMB 0x6ffae040 00072 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: SRAT 0x6ffaa4b0 00090 (v03 AMD FAM_F_10 00000002 AMD 00000001) ACPI: MSCT 0x6ffaa540 0004E (v01 A M I OEMBOARD 00000001 AMD 00000001) ACPI: HPET 0x6ffaa590 00038 (v01 7309MS OEMHPET0 20101122 MSFT 00000097) ACPI: SSDT 0x6ffaa5d0 0023E (v01 A M I POWERNOW 00000001 AMD 00000001) MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level MADT: Interrupt override: source 14, irq 14 MADT: Interrupt override: source 15, irq 15 ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 wlan: <802.11 Link Layer> kbd: new array size 4 kbd1 at kbdmux0 nfslock: pseudo-device mem: null: random: cpuctl: access to MSR registers/cpuid info. VESA: INT 0x10 vector 0xc000:0x0fd8 VESA: information block 0000 56 45 53 41 00 03 00 01 00 99 01 00 00 00 22 00 0010 00 99 00 10 61 05 07 01 00 99 1b 01 00 99 2c 01 0020 00 99 00 01 01 01 02 01 03 01 04 01 05 01 06 01 0030 07 01 0e 01 0f 01 11 01 12 01 14 01 15 01 17 01 0040 18 01 1a 01 1b 01 30 01 31 01 32 01 33 01 34 01 0050 35 01 36 01 3d 01 3e 01 45 01 46 01 47 01 48 01 0060 52 01 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 4e 56 49 44 49 41 00 42 75 69 6c 64 20 20 20 20 0110 30 36 31 30 31 30 2e 33 0d 0a 00 4d 43 50 36 31 0120 20 2d 20 6d 63 70 36 31 2d 38 30 00 43 68 69 70 0130 20 52 65 76 20 20 20 00 00 00 00 00 00 00 00 00 0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 VESA: 32 mode(s) found VESA: v3.0, 262144k memory, flags:0x1, mode table:0xffffff800025e022 (99000022) VESA: NVIDIA VESA: Build 061010.3 MCP61 - mcp61-80 Chip Rev io: acpi0: <7309MS A7309200> on motherboard PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 ACPI: Executed 1 blocks of module-level executable AML code acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 7ff00000 (3) failed cpu0: on acpi0 cpu0: switching to generic Cx mode ACPI: Processor \134_PR_.P002 (ACPI ID 2) ignored ACPI: Processor \134_PR_.P003 (ACPI ID 3) ignored ACPI: Processor \134_PR_.P004 (ACPI ID 4) ignored ACPI: Processor \134_PR_.P005 (ACPI ID 5) ignored ACPI: Processor \134_PR_.P006 (ACPI ID 6) ignored attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfeff0000-0xfeff0fff on acpi0 hpet0: vendor 0x10de, rev 0x1, 25000000Hz, 3 timers, legacy route hpet0: t0: irqs 0x00ff3efa (31), periodic hpet0: t1: irqs 0x00ff3efa (31) hpet0: t2: irqs 0x00ff3efa (31) Timecounter "HPET" frequency 25000000 Hz quality 950 ACPI timer: 1/1 1/2 1/2 1/2 1/2 1/2 1/2 1/2 1/2 1/2 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link8: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link9: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link10: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link11: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link12: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link13: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link14: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link15: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link16: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link17: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: decoding 3 range 0xa0000-0xbffff pcib0: decoding 3 range 0xd0000-0xdffff pcib0: decoding 3 range 0x80000000-0xdfffffff pcib0: decoding 3 range 0xf0000000-0xfed44fff pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x10de, dev=0x03ea, revid=0xa1 domain=0, bus=0, slot=0, func=0 class=05-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03e0, revid=0xa2 domain=0, bus=0, slot=1, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) map[10]: type I/O Port, range 32, base 0x4f00, size 8, enabled pcib0: allocated type 4 (0x4f00-0x4fff) for rid 10 of pci0:0:1:0 found-> vendor=0x10de, dev=0x03eb, revid=0xa2 domain=0, bus=0, slot=1, func=1 class=0c-05-00, hdrtype=0x00, mfdev=1 cmdreg=0x0001, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0x4900, size 6, enabled pcib0: allocated type 4 (0x4900-0x493f) for rid 10 of pci0:0:1:1 map[20]: type I/O Port, range 32, base 0x4d00, size 6, enabled pcib0: allocated type 4 (0x4d00-0x4d3f) for rid 20 of pci0:0:1:1 map[24]: type I/O Port, range 32, base 0x4e00, size 6, enabled pcib0: allocated type 4 (0x4e00-0x4e3f) for rid 24 of pci0:0:1:1 pcib0: matched entry for 0.1.INTA (src \134_SB_.LSMB:0) pci_link13: Picked IRQ 20 with weight 0 pcib0: slot 1 INTA routed to irq 20 via \134_SB_.LSMB found-> vendor=0x10de, dev=0x03f5, revid=0xa2 domain=0, bus=0, slot=1, func=2 class=05-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0400, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03f1, revid=0xa3 domain=0, bus=0, slot=2, func=0 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7f000, size 12, enabled pcib0: allocated type 3 (0xdff7f000-0xdff7ffff) for rid 10 of pci0:0:2:0 pcib0: matched entry for 0.2.INTA (src \134_SB_.LUB0:0) pci_link8: Picked IRQ 21 with weight 0 pcib0: slot 2 INTA routed to irq 21 via \134_SB_.LUB0 ohci early: SMM active, request owner change found-> vendor=0x10de, dev=0x03f2, revid=0xa3 domain=0, bus=0, slot=2, func=1 class=0c-03-20, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7ec00, size 8, enabled pcib0: allocated type 3 (0xdff7ec00-0xdff7ecff) for rid 10 of pci0:0:2:1 pcib0: matched entry for 0.2.INTB (src \134_SB_.LUB2:0) pci_link9: Picked IRQ 22 with weight 0 pcib0: slot 2 INTB routed to irq 22 via \134_SB_.LUB2 found-> vendor=0x10de, dev=0x03f3, revid=0xa1 domain=0, bus=0, slot=4, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x02 (500 ns) found-> vendor=0x10de, dev=0x03f0, revid=0xa2 domain=0, bus=0, slot=5, func=0 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x05 (1250 ns) intpin=b, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff78000, size 14, enabled pcib0: allocated type 3 (0xdff78000-0xdff7bfff) for rid 10 of pci0:0:5:0 pcib0: matched entry for 0.5.INTB (src \134_SB_.LAZA:0) pci_link11: Picked IRQ 23 with weight 0 pcib0: slot 5 INTB routed to irq 23 via \134_SB_.LAZA found-> vendor=0x10de, dev=0x03ec, revid=0xa2 domain=0, bus=0, slot=6, func=0 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) powerspec 2 supports D0 D3 current D0 pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:6:0 pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:6:0 pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:6:0 pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:6:0 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled pcib0: allocated type 4 (0xffa0-0xffaf) for rid 20 of pci0:0:6:0 found-> vendor=0x10de, dev=0x03ef, revid=0xa2 domain=0, bus=0, slot=7, func=0 class=06-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x01 (250 ns), maxlat=0x14 (5000 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 8 messages, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff7d000, size 12, enabled pcib0: allocated type 3 (0xdff7d000-0xdff7dfff) for rid 10 of pci0:0:7:0 map[14]: type I/O Port, range 32, base 0xe480, size 3, enabled pcib0: allocated type 4 (0xe480-0xe487) for rid 14 of pci0:0:7:0 pcib0: matched entry for 0.7.INTA (src \134_SB_.LMAC:0) pci_link10: Picked IRQ 20 with weight 1 pcib0: slot 7 INTA routed to irq 20 via \134_SB_.LMAC found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=0 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xe400, size 3, enabled pcib0: allocated type 4 (0xe400-0xe407) for rid 10 of pci0:0:8:0 map[14]: type I/O Port, range 32, base 0xe080, size 2, enabled pcib0: allocated type 4 (0xe080-0xe083) for rid 14 of pci0:0:8:0 map[18]: type I/O Port, range 32, base 0xe000, size 3, enabled pcib0: allocated type 4 (0xe000-0xe007) for rid 18 of pci0:0:8:0 map[1c]: type I/O Port, range 32, base 0xdc00, size 2, enabled pcib0: allocated type 4 (0xdc00-0xdc03) for rid 1c of pci0:0:8:0 map[20]: type I/O Port, range 32, base 0xd880, size 4, enabled pcib0: allocated type 4 (0xd880-0xd88f) for rid 20 of pci0:0:8:0 map[24]: type Memory, range 32, base 0xdff7c000, size 12, enabled pcib0: allocated type 3 (0xdff7c000-0xdff7cfff) for rid 24 of pci0:0:8:0 pcib0: matched entry for 0.8.INTA (src \134_SB_.LSA0:0) pci_link15: Picked IRQ 21 with weight 1 pcib0: slot 8 INTA routed to irq 21 via \134_SB_.LSA0 found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=1 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xd800, size 3, enabled pcib0: allocated type 4 (0xd800-0xd807) for rid 10 of pci0:0:8:1 map[14]: type I/O Port, range 32, base 0xd480, size 2, enabled pcib0: allocated type 4 (0xd480-0xd483) for rid 14 of pci0:0:8:1 map[18]: type I/O Port, range 32, base 0xd400, size 3, enabled pcib0: allocated type 4 (0xd400-0xd407) for rid 18 of pci0:0:8:1 map[1c]: type I/O Port, range 32, base 0xd080, size 2, enabled pcib0: allocated type 4 (0xd080-0xd083) for rid 1c of pci0:0:8:1 map[20]: type I/O Port, range 32, base 0xd000, size 4, enabled pcib0: allocated type 4 (0xd000-0xd00f) for rid 20 of pci0:0:8:1 map[24]: type Memory, range 32, base 0xdff77000, size 12, enabled pcib0: allocated type 3 (0xdff77000-0xdff77fff) for rid 24 of pci0:0:8:1 pcib0: matched entry for 0.8.INTB (src \134_SB_.LSA1:0) pci_link16: Picked IRQ 22 with weight 1 pcib0: slot 8 INTB routed to irq 22 via \134_SB_.LSA1 found-> vendor=0x10de, dev=0x03e8, revid=0xa2 domain=0, bus=0, slot=9, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=11, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=12, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03d0, revid=0xa2 domain=0, bus=0, slot=13, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xde000000, size 24, enabled pcib0: allocated type 3 (0xde000000-0xdeffffff) for rid 10 of pci0:0:13:0 map[14]: type Prefetchable Memory, range 64, base 0xc0000000, size 28, enabled pcib0: allocated type 3 (0xc0000000-0xcfffffff) for rid 14 of pci0:0:13:0 map[1c]: type Memory, range 64, base 0xdd000000, size 24, enabled pcib0: allocated type 3 (0xdd000000-0xddffffff) for rid 1c of pci0:0:13:0 pcib0: matched entry for 0.13.INTA (src \134_SB_.LMC9:0) pci_link12: Picked IRQ 23 with weight 1 pcib0: slot 13 INTA routed to irq 23 via \134_SB_.LMC9 found-> vendor=0x1022, dev=0x1200, revid=0x00 domain=0, bus=0, slot=24, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1201, revid=0x00 domain=0, bus=0, slot=24, func=1 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1202, revid=0x00 domain=0, bus=0, slot=24, func=2 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1203, revid=0x00 domain=0, bus=0, slot=24, func=3 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1204, revid=0x00 domain=0, bus=0, slot=24, func=4 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) pci0: at device 0.0 (no driver attached) isab0: port 0x4f00-0x4fff at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) pci0: at device 1.2 (no driver attached) ohci0: mem 0xdff7f000-0xdff7ffff irq 21 at device 2.0 on pci0 ioapic0: routing intpin 21 (PCI IRQ 21) to lapic 0 vector 51 usbus0 on ohci0 usbus0: bpf attached ohci0: usbpf: Attached ehci0: mem 0xdff7ec00-0xdff7ecff irq 22 at device 2.1 on pci0 ioapic0: routing intpin 22 (PCI IRQ 22) to lapic 0 vector 52 ehci0: Doorbell workaround enabled usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: bpf attached ehci0: usbpf: Attached pcib1: at device 4.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: special decode subtractive pci1: on pcib1 pci1: domain=0, physical bus=1 hdac0: mem 0xdff78000-0xdff7bfff irq 23 at device 5.0 on pci0 hdac0: PCI card vendor: 0x0888, device: 0x10ec hdac0: HDA Driver Revision: 20120126_0002 hdac0: Config options: on=0x00000000 off=0x00000000 hdac0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 53 hdac0: using IRQ 256 for MSI hdac0: Caps: OSS 4, ISS 4, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 6.0 on pci0 ata0: at channel 0 on atapci0 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 54 ata1: at channel 1 on atapci0 ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 55 nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff irq 20 at device 7.0 on pci0 nfe0: attempting to allocate 8 MSI vectors (8 supported) msi: routing MSI IRQ 257 to local APIC 0 vector 56 msi: routing MSI IRQ 258 to local APIC 0 vector 57 msi: routing MSI IRQ 259 to local APIC 0 vector 58 msi: routing MSI IRQ 260 to local APIC 0 vector 59 msi: routing MSI IRQ 261 to local APIC 0 vector 60 msi: routing MSI IRQ 262 to local APIC 0 vector 61 msi: routing MSI IRQ 263 to local APIC 0 vector 62 msi: routing MSI IRQ 264 to local APIC 0 vector 63 nfe0: using IRQs 257-264 for MSI nfe0: Using 8 MSI messages miibus0: on nfe0 rlphy0: PHY 0 on miibus0 rlphy0: OUI 0x000004, model 0x0020, rev. 1 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy1: PHY 1 on miibus0 rlphy1: OUI 0x000004, model 0x0020, rev. 1 rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy2: PHY 2 on miibus0 rlphy2: OUI 0x000004, model 0x0020, rev. 1 rlphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy3: PHY 3 on miibus0 rlphy3: OUI 0x000004, model 0x0020, rev. 1 rlphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy4: PHY 4 on miibus0 rlphy4: OUI 0x000004, model 0x0020, rev. 1 rlphy4: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy5: PHY 5 on miibus0 rlphy5: OUI 0x000004, model 0x0020, rev. 1 rlphy5: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy6: PHY 6 on miibus0 rlphy6: OUI 0x000004, model 0x0020, rev. 1 rlphy6: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy7: PHY 7 on miibus0 rlphy7: OUI 0x000004, model 0x0020, rev. 1 rlphy7: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy8: PHY 8 on miibus0 rlphy8: OUI 0x000004, model 0x0020, rev. 1 rlphy8: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy9: PHY 9 on miibus0 rlphy9: OUI 0x000004, model 0x0020, rev. 1 rlphy9: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy10: PHY 10 on miibus0 rlphy10: OUI 0x000004, model 0x0020, rev. 1 rlphy10: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy11: PHY 11 on miibus0 rlphy11: OUI 0x000004, model 0x0020, rev. 1 rlphy11: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy12: PHY 12 on miibus0 rlphy12: OUI 0x000004, model 0x0020, rev. 1 rlphy12: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy13: PHY 13 on miibus0 rlphy13: OUI 0x000004, model 0x0020, rev. 1 rlphy13: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy14: PHY 14 on miibus0 rlphy14: OUI 0x000004, model 0x0020, rev. 1 rlphy14: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy15: PHY 15 on miibus0 rlphy15: OUI 0x000004, model 0x0020, rev. 1 rlphy15: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy16: PHY 16 on miibus0 rlphy16: OUI 0x000004, model 0x0020, rev. 1 rlphy16: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy17: PHY 17 on miibus0 rlphy17: OUI 0x000004, model 0x0020, rev. 1 rlphy17: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy18: PHY 18 on miibus0 rlphy18: OUI 0x000004, model 0x0020, rev. 1 rlphy18: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy19: PHY 19 on miibus0 rlphy19: OUI 0x000004, model 0x0020, rev. 1 rlphy19: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy20: PHY 20 on miibus0 rlphy20: OUI 0x000004, model 0x0020, rev. 1 rlphy20: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy21: PHY 21 on miibus0 rlphy21: OUI 0x000004, model 0x0020, rev. 1 rlphy21: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy22: PHY 22 on miibus0 rlphy22: OUI 0x000004, model 0x0020, rev. 1 rlphy22: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy23: PHY 23 on miibus0 rlphy23: OUI 0x000004, model 0x0020, rev. 1 rlphy23: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy24: PHY 24 on miibus0 rlphy24: OUI 0x000004, model 0x0020, rev. 1 rlphy24: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy25: PHY 25 on miibus0 rlphy25: OUI 0x000004, model 0x0020, rev. 1 rlphy25: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy26: PHY 26 on miibus0 rlphy26: OUI 0x000004, model 0x0020, rev. 1 rlphy26: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy27: PHY 27 on miibus0 rlphy27: OUI 0x000004, model 0x0020, rev. 1 rlphy27: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy28: PHY 28 on miibus0 rlphy28: OUI 0x000004, model 0x0020, rev. 1 rlphy28: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy29: PHY 29 on miibus0 rlphy29: OUI 0x000004, model 0x0020, rev. 1 rlphy29: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy30: PHY 30 on miibus0 rlphy30: OUI 0x000004, model 0x0020, rev. 1 rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy31: PHY 31 on miibus0 rlphy31: OUI 0x000004, model 0x0020, rev. 1 rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow nfe0: bpf attached nfe0: Ethernet address: 40:61:86:cd:44:97 atapci1: port 0xe400-0xe407,0xe080-0xe083,0xe000-0xe007,0xdc00-0xdc03,0xd880-0xd88f mem 0xdff7c000-0xdff7cfff irq 21 at device 8.0 on pci0 ata2: at channel 0 on atapci1 ata3: at channel 1 on atapci1 atapci2: port 0xd800-0xd807,0xd480-0xd483,0xd400-0xd407,0xd080-0xd083,0xd000-0xd00f mem 0xdff77000-0xdff77fff irq 22 at device 8.1 on pci0 ata4: at channel 0 on atapci2 ata5: at channel 1 on atapci2 pcib2: at device 9.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: at device 11.0 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pci3: on pcib3 pci3: domain=0, physical bus=3 pcib4: at device 12.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pci4: on pcib4 pci4: domain=0, physical bus=4 vgapci0: mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff,0xdd000000-0xddffffff irq 23 at device 13.0 on pci0 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 64 vgapci0: Boot video device amdtemp0: on hostb3 amdtemp0: Found 1 cores and 1 sensors. acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 65 uart0: fast interrupt ppc0: using extended I/O port range ppc0: SPP ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ioapic0: routing intpin 7 (ISA IRQ 7) to lapic 0 vector 66 ppbus0: on ppc0 plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x83ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 67 atkbd0: [GIANT-LOCKED] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 68 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons psm0: config:00000000, flags:00000008, packet size:4 psm0: syncmask:08, syncbits:00 acpi0: wakeup code va 0xffffff80003d3000 pa 0x90000 pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0 pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0 pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0 pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0 pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0000-0xd07ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd0800-0xd0fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd1000-0xd17ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd1800-0xd1fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd2000-0xd27ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd2800-0xd2fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd3000-0xd37ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd3800-0xd3fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd4000-0xd47ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd4800-0xd4fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd5000-0xd57ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd5800-0xd5fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6000-0xd67ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6800-0xd6fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7000-0xd77ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7800-0xd7fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8000-0xd87ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8800-0xd8fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9000-0xd97ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9800-0xd9fff) for rid 1 of orm0 pcib0: allocated type 3 (0xda000-0xda7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xda800-0xdafff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb000-0xdb7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb800-0xdbfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc000-0xdc7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc800-0xdcfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd000-0xdd7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd800-0xddfff) for rid 1 of orm0 pcib0: allocated type 3 (0xde000-0xde7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xde800-0xdefff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf000-0xdf7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf800-0xdffff) for rid 1 of orm0 isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it ppc: ppc0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0 pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0 fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1 uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices hwpstate0: on cpu0 Device configuration finished. procfs registered lapic: Divisor 2, Frequency 119671611 Hz Timecounters tick every 1.000 msec vlan: initialized, using hash tables with chaining Linux ELF exec handler installed lo0: bpf attached hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 hdaa0: Subsystem ID: 0x14627309 hdaa0: NumGPIO=2 NumGPO=0 NumGPI=0 GPIWake=1 GPIUnsol=1 hdaa0: GPIO0: disabled hdaa0: GPIO1: disabled hdaa0: Original pins configuration: hdaa0: nid 0x as seq device conn jack loc color misc hdaa0: 17 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: 18 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: 20 01014410 1 0 Line-out Jack 1/8 Rear Green 4 hdaa0: 21 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: 22 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: 23 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: 24 01a19c40 4 0 Mic Jack 1/8 Rear Pink 12 hdaa0: 25 02a19c50 5 0 Mic Jack 1/8 Front Pink 12 hdaa0: 26 0181344f 4 15 Line-in Jack 1/8 Rear Blue 4 hdaa0: 27 02214c20 2 0 Headphones Jack 1/8 Front Green 12 hdaa0: 28 593301f0 15 0 CD None ATAPI Onboard Unknown 1 hdaa0: 29 4005c603 0 3 Line-out None Optical 0x00 Res.C 6 hdaa0: 30 014b1130 3 0 SPDIF-out Jack Combo Rear Black 1 hdaa0: 31 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa0: Patching widget caps nid=29 0x00400400 -> 0x00700400 hdaa0: Patched pins configuration: hdaa0: nid 0x as seq device conn jack loc color misc hdaa0: 17 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 18 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 20 01014410 1 0 Line-out Jack 1/8 Rear Green 4 hdaa0: 21 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 22 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 23 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 24 01a19c40 4 0 Mic Jack 1/8 Rear Pink 12 hdaa0: 25 02a19c50 5 0 Mic Jack 1/8 Front Pink 12 hdaa0: 26 0181344f 4 15 Line-in Jack 1/8 Rear Blue 4 hdaa0: 27 02214c20 2 0 Headphones Jack 1/8 Front Green 12 hdaa0: 28 593301f0 15 0 CD None ATAPI Onboard Unknown 1 DISA hdaa0: 30 014b1130 3 0 SPDIF-out Jack Combo Rear Black 1 hdaa0: 31 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa0: 5 associations found: hdaa0: Association 0 (1) out: hdaa0: Pin nid=20 seq=0 hdaa0: Association 1 (2) out: hdaa0: Pin nid=27 seq=0 hdaa0: Association 2 (3) out: hdaa0: Pin nid=30 seq=0 hdaa0: Association 3 (4) in: hdaa0: Pin nid=24 seq=0 hdaa0: Pin nid=26 seq=15 hdaa0: Association 4 (5) in: hdaa0: Pin nid=25 seq=0 hdaa0: Tracing association 0 (1) hdaa0: Pin 20 traced to DAC 2 hdaa0: Association 0 (1) trace succeeded hdaa0: Tracing association 1 (2) hdaa0: Pin 27 traced to DAC 3 hdaa0: Association 1 (2) trace succeeded hdaa0: Tracing association 2 (3) hdaa0: Pin 30 traced to DAC 6 hdaa0: Association 2 (3) trace succeeded hdaa0: Tracing association 3 (4) hdaa0: Pin 24 traced to ADC 8 hdaa0: Pin 26 traced to ADC 8 hdaa0: Association 3 (4) trace succeeded hdaa0: Tracing association 4 (5) hdaa0: Pin 25 traced to ADC 9 hdaa0: Association 4 (5) trace succeeded hdaa0: Looking for additional DAC for association 0 (1) hdaa0: Looking for additional DAC for association 1 (2) hdaa0: Looking for additional DAC for association 2 (3) hdaa0: Looking for additional ADC for association 3 (4) hdaa0: Looking for additional ADC for association 4 (5) hdaa0: Tracing input monitor hdaa0: Tracing nid 11 to out hdaa0: nid 11 is input monitor hdaa0: Tracing nid 34 to out hdaa0: Tracing nid 35 to out hdaa0: Tracing other input monitors hdaa0: Tracing nid 24 to out hdaa0: Tracing nid 25 to out hdaa0: Tracing nid 26 to out hdaa0: Tracing beeper hdaa0: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref hdaa0: hdaa0: +-------------------+ hdaa0: | DUMPING HDA NODES | hdaa0: +-------------------+ hdaa0: hdaa0: Default Parameter hdaa0: ----------------- hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: IN amp: 0x00000000 hdaa0: OUT amp: 0x00000000 hdaa0: hdaa0: nid: 2 hdaa0: Name: audio output hdaa0: Widget cap: 0x00000411 hdaa0: PWR STEREO hdaa0: Association: 0 (0x00000001) hdaa0: OSS: pcm (pcm) hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: hdaa0: nid: 3 hdaa0: Name: audio output hdaa0: Widget cap: 0x00000411 hdaa0: PWR STEREO hdaa0: Association: 1 (0x00000001) hdaa0: OSS: pcm (pcm) hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: hdaa0: nid: 4 [DISABLED] hdaa0: Name: audio output hdaa0: Widget cap: 0x00000411 hdaa0: PWR STEREO hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: hdaa0: nid: 5 [DISABLED] hdaa0: Name: audio output hdaa0: Widget cap: 0x00000411 hdaa0: PWR STEREO hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: hdaa0: nid: 6 hdaa0: Name: audio output hdaa0: Widget cap: 0x00000611 hdaa0: PWR DIGITAL STEREO hdaa0: Association: 2 (0x00000001) hdaa0: OSS: pcm (pcm) hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e05e0 hdaa0: 16 20 24 bits, 44 48 88 96 192 KHz hdaa0: hdaa0: nid: 7 [DISABLED] hdaa0: Name: vendor widget hdaa0: Widget cap: 0x00f00000 hdaa0: hdaa0: nid: 8 hdaa0: Name: audio input hdaa0: Widget cap: 0x0010051b hdaa0: PWR STEREO hdaa0: Association: 3 (0x00008001) hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: Input amp: 0x80051f0b hdaa0: mute=1 step=31 size=5 offset=11 hdaa0: connections: 1 hdaa0: | hdaa0: + <- nid=35 [audio mixer] hdaa0: hdaa0: nid: 9 hdaa0: Name: audio input hdaa0: Widget cap: 0x0010051b hdaa0: PWR STEREO hdaa0: Association: 4 (0x00000001) hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: Input amp: 0x80051f0b hdaa0: mute=1 step=31 size=5 offset=11 hdaa0: connections: 1 hdaa0: | hdaa0: + <- nid=34 [audio mixer] hdaa0: hdaa0: nid: 10 [DISABLED] hdaa0: Name: audio input hdaa0: Widget cap: 0x00100711 hdaa0: PWR DIGITAL STEREO hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: connections: 1 hdaa0: | hdaa0: + [DISABLED] <- nid=31 [pin: Speaker (None)] [DISABLED] hdaa0: hdaa0: nid: 11 hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010b hdaa0: STEREO hdaa0: Association: 3 (0x00008001) hdaa0: OSS: mix (mix) hdaa0: Input amp: 0x80051f17 hdaa0: mute=1 step=31 size=5 offset=23 hdaa0: connections: 10 hdaa0: | hdaa0: + <- nid=24 [pin: Mic (Pink Jack)] hdaa0: + [DISABLED] <- nid=25 [pin: Mic (Pink Jack)] hdaa0: + <- nid=26 [pin: Line-in (Blue Jack)] hdaa0: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdaa0: + [DISABLED] <- nid=28 [pin: CD (None)] [DISABLED] hdaa0: + <- nid=29 [beep widget] hdaa0: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdaa0: + [DISABLED] <- nid=21 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=22 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=23 [pin: Speaker (None)] [DISABLED] hdaa0: hdaa0: nid: 12 hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010f hdaa0: STEREO hdaa0: Association: 0 (0x00000001) hdaa0: OSS: pcm, mix hdaa0: Output amp: 0x00051f1f hdaa0: mute=0 step=31 size=5 offset=31 hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 2 hdaa0: | hdaa0: + <- nid=2 [audio output] hdaa0: + <- nid=11 [audio mixer] hdaa0: hdaa0: nid: 13 hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010f hdaa0: STEREO hdaa0: Association: 1 (0x00000001) hdaa0: OSS: pcm, mix hdaa0: Output amp: 0x00051f1f hdaa0: mute=0 step=31 size=5 offset=31 hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 2 hdaa0: | hdaa0: + <- nid=3 [audio output] hdaa0: + <- nid=11 [audio mixer] hdaa0: hdaa0: nid: 14 [DISABLED] hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010f hdaa0: STEREO hdaa0: Association: -2 (0x00000000) hdaa0: Output amp: 0x00051f1f hdaa0: mute=0 step=31 size=5 offset=31 hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 2 hdaa0: | hdaa0: + [DISABLED] <- nid=4 [audio output] [DISABLED] hdaa0: + [DISABLED] <- nid=11 [audio mixer] hdaa0: hdaa0: nid: 15 [DISABLED] hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010f hdaa0: STEREO hdaa0: Association: -2 (0x00000000) hdaa0: Output amp: 0x00051f1f hdaa0: mute=0 step=31 size=5 offset=31 hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 2 hdaa0: | hdaa0: + [DISABLED] <- nid=5 [audio output] [DISABLED] hdaa0: + [DISABLED] <- nid=11 [audio mixer] hdaa0: hdaa0: nid: 16 [DISABLED] hdaa0: Name: audio output hdaa0: Widget cap: 0x00000611 hdaa0: PWR DIGITAL STEREO hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e05e0 hdaa0: 16 20 24 bits, 44 48 88 96 192 KHz hdaa0: hdaa0: nid: 17 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x00400780 hdaa0: PWR DIGITAL UNSOL hdaa0: Pin cap: 0x00000014 hdaa0: PDC OUT hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: connections: 1 hdaa0: | hdaa0: + <- nid=16 [audio output] [DISABLED] hdaa0: hdaa0: nid: 18 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x00400401 hdaa0: PWR STEREO hdaa0: Pin cap: 0x00000020 hdaa0: IN hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: hdaa0: nid: 19 [DISABLED] hdaa0: Name: vendor widget hdaa0: Widget cap: 0x00f00000 hdaa0: hdaa0: nid: 20 hdaa0: Name: pin: Line-out (Green Jack) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Association: 0 (0x00000001) hdaa0: Pin cap: 0x0001003e hdaa0: TRQD PDC HP OUT IN EAPD hdaa0: Pin config: 0x01014410 hdaa0: Pin control: 0x00000040 OUT hdaa0: EAPD: 0x00000002 hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + <- nid=12 [audio mixer] (selected) hdaa0: + [DISABLED] <- nid=13 [audio mixer] hdaa0: + [DISABLED] <- nid=14 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=15 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 21 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Pin cap: 0x0001003e hdaa0: TRQD PDC HP OUT IN EAPD hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: EAPD: 0x00000002 hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + <- nid=13 [audio mixer] hdaa0: + <- nid=14 [audio mixer] [DISABLED] hdaa0: + <- nid=15 [audio mixer] [DISABLED] hdaa0: + <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 22 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Pin cap: 0x00000036 hdaa0: TRQD PDC OUT IN hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + <- nid=13 [audio mixer] hdaa0: + <- nid=14 [audio mixer] [DISABLED] hdaa0: + <- nid=15 [audio mixer] [DISABLED] hdaa0: + <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 23 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Pin cap: 0x00000036 hdaa0: TRQD PDC OUT IN hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + <- nid=13 [audio mixer] hdaa0: + <- nid=14 [audio mixer] [DISABLED] hdaa0: + <- nid=15 [audio mixer] [DISABLED] hdaa0: + <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 24 hdaa0: Name: pin: Mic (Pink Jack) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Association: 3 (0x00000001) hdaa0: OSS: mic (mic) hdaa0: Pin cap: 0x0000373e hdaa0: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdaa0: Pin config: 0x01a19c40 hdaa0: Pin control: 0x00000025 IN VREFs hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + [DISABLED] <- nid=13 [audio mixer] hdaa0: + [DISABLED] <- nid=14 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=15 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 25 hdaa0: Name: pin: Mic (Pink Jack) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Association: 4 (0x00000001) hdaa0: OSS: monitor (monitor) hdaa0: Pin cap: 0x0000373e hdaa0: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdaa0: Pin config: 0x02a19c50 hdaa0: Pin control: 0x00000025 IN VREFs hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + [DISABLED] <- nid=13 [audio mixer] hdaa0: + [DISABLED] <- nid=14 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=15 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 26 hdaa0: Name: pin: Line-in (Blue Jack) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Association: 3 (0x00008000) hdaa0: OSS: line (line) hdaa0: Pin cap: 0x0000373e hdaa0: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdaa0: Pin config: 0x0181344f hdaa0: Pin control: 0x00000025 IN VREFs hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] (selected) hdaa0: + [DISABLED] <- nid=13 [audio mixer] hdaa0: + [DISABLED] <- nid=14 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=15 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 27 hdaa0: Name: pin: Headphones (Green Jack) hdaa0: Widget cap: 0x0040058f hdaa0: PWR UNSOL STEREO hdaa0: Association: 1 (0x00000001) hdaa0: Pin cap: 0x0000373e hdaa0: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdaa0: Pin config: 0x02214c20 hdaa0: Pin control: 0x000000c0 HP OUT hdaa0: Output amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: Input amp: 0x00270300 hdaa0: mute=0 step=3 size=39 offset=0 hdaa0: connections: 5 hdaa0: | hdaa0: + [DISABLED] <- nid=12 [audio mixer] hdaa0: + <- nid=13 [audio mixer] (selected) hdaa0: + [DISABLED] <- nid=14 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=15 [audio mixer] [DISABLED] hdaa0: + [DISABLED] <- nid=38 [audio mixer] [DISABLED] hdaa0: hdaa0: nid: 28 [DISABLED] hdaa0: Name: pin: CD (None) hdaa0: Widget cap: 0x00400481 hdaa0: PWR UNSOL STEREO hdaa0: Pin cap: 0x00000024 hdaa0: PDC IN hdaa0: Pin config: 0x593301f0 hdaa0: Pin control: 0x00000000 hdaa0: hdaa0: nid: 29 hdaa0: Name: beep widget hdaa0: Widget cap: 0x00700400 hdaa0: PWR hdaa0: Association: -2 (0x00000000) hdaa0: OSS: speaker (speaker) hdaa0: Pin cap: 0x00000020 hdaa0: IN hdaa0: Pin config: 0x4005c603 hdaa0: Pin control: 0x00000020 IN hdaa0: hdaa0: nid: 30 hdaa0: Name: pin: SPDIF-out (Black Jack) hdaa0: Widget cap: 0x00400780 hdaa0: PWR DIGITAL UNSOL hdaa0: Association: 2 (0x00000001) hdaa0: Pin cap: 0x00000014 hdaa0: PDC OUT hdaa0: Pin config: 0x014b1130 hdaa0: Pin control: 0x00000040 OUT hdaa0: connections: 1 hdaa0: | hdaa0: + <- nid=6 [audio output] hdaa0: hdaa0: nid: 31 [DISABLED] hdaa0: Name: pin: Speaker (None) hdaa0: Widget cap: 0x00400680 hdaa0: PWR DIGITAL UNSOL hdaa0: Pin cap: 0x00000024 hdaa0: PDC IN hdaa0: Pin config: 0x411111f0 hdaa0: Pin control: 0x00000000 hdaa0: hdaa0: nid: 32 [DISABLED] hdaa0: Name: vendor widget hdaa0: Widget cap: 0x00f00040 hdaa0: PROC hdaa0: hdaa0: nid: 33 [DISABLED] hdaa0: Name: vendor widget hdaa0: Widget cap: 0x00f00000 hdaa0: hdaa0: nid: 34 hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010b hdaa0: STEREO hdaa0: Association: 4 (0x00000001) hdaa0: OSS: speaker, monitor hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 12 hdaa0: | hdaa0: + [DISABLED] <- nid=24 [pin: Mic (Pink Jack)] hdaa0: + <- nid=25 [pin: Mic (Pink Jack)] hdaa0: + [DISABLED] <- nid=26 [pin: Line-in (Blue Jack)] hdaa0: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdaa0: + [DISABLED] <- nid=28 [pin: CD (None)] [DISABLED] hdaa0: + <- nid=29 [beep widget] hdaa0: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdaa0: + [DISABLED] <- nid=21 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=22 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=23 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=11 [audio mixer] hdaa0: + [DISABLED] <- nid=18 [pin: Speaker (None)] [DISABLED] hdaa0: hdaa0: nid: 35 hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010b hdaa0: STEREO hdaa0: Association: 3 (0x00008001) hdaa0: OSS: speaker, line, mic, mix hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 11 hdaa0: | hdaa0: + <- nid=24 [pin: Mic (Pink Jack)] hdaa0: + [DISABLED] <- nid=25 [pin: Mic (Pink Jack)] hdaa0: + <- nid=26 [pin: Line-in (Blue Jack)] hdaa0: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdaa0: + [DISABLED] <- nid=28 [pin: CD (None)] [DISABLED] hdaa0: + <- nid=29 [beep widget] hdaa0: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdaa0: + [DISABLED] <- nid=21 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=22 [pin: Speaker (None)] [DISABLED] hdaa0: + [DISABLED] <- nid=23 [pin: Speaker (None)] [DISABLED] hdaa0: + <- nid=11 [audio mixer] hdaa0: hdaa0: nid: 36 [DISABLED] hdaa0: Name: vendor widget hdaa0: Widget cap: 0x00f00000 hdaa0: hdaa0: nid: 37 [DISABLED] hdaa0: Name: audio output hdaa0: Widget cap: 0x00000411 hdaa0: PWR STEREO hdaa0: Stream cap: 0x00000001 hdaa0: PCM hdaa0: PCM cap: 0x000e0560 hdaa0: 16 20 24 bits, 44 48 96 192 KHz hdaa0: hdaa0: nid: 38 [DISABLED] hdaa0: Name: audio mixer hdaa0: Widget cap: 0x0020010f hdaa0: STEREO hdaa0: Association: -2 (0x00000000) hdaa0: Output amp: 0x00051f1f hdaa0: mute=0 step=31 size=5 offset=31 hdaa0: Input amp: 0x80000000 hdaa0: mute=1 step=0 size=0 offset=0 hdaa0: connections: 2 hdaa0: | hdaa0: + [DISABLED] <- nid=37 [audio output] [DISABLED] hdaa0: + [DISABLED] <- nid=11 [audio mixer] hdaa0: pcm0: at nid 20 and 24,26 on hdaa0 pcm0: +--------------------------------------+ pcm0: | DUMPING PCM Playback/Record Channels | pcm0: +--------------------------------------+ pcm0: pcm0: Playback: pcm0: pcm0: Stream cap: 0x00000001 pcm0: PCM pcm0: PCM cap: 0x000e0560 pcm0: 16 20 24 bits, 44 48 96 192 KHz pcm0: DAC: 2 pcm0: pcm0: Record: pcm0: pcm0: Stream cap: 0x00000001 pcm0: PCM pcm0: PCM cap: 0x000e0560 pcm0: 16 20 24 bits, 44 48 96 192 KHz pcm0: DAC: 8 pcm0: pcm0: +-------------------------------+ pcm0: | DUMPING Playback/Record Paths | pcm0: +-------------------------------+ pcm0: pcm0: Playback: pcm0: pcm0: nid=20 [pin: Line-out (Green Jack)] pcm0: | pcm0: + <- nid=12 [audio mixer] [src: pcm, mix] pcm0: | pcm0: + <- nid=2 [audio output] [src: pcm] pcm0: + <- nid=11 [audio mixer] [src: mix] pcm0: pcm0: Record: pcm0: pcm0: nid=8 [audio input] pcm0: | pcm0: + <- nid=35 [audio mixer] [src: speaker, line, mic, mix] pcm0: | pcm0: + <- nid=24 [pin: Mic (Pink Jack)] [src: mic] pcm0: + <- nid=26 [pin: Line-in (Blue Jack)] [src: line] pcm0: + <- nid=29 [beep widget] [src: speaker] pcm0: + <- nid=11 [audio mixer] [src: mix] pcm0: pcm0: Input Mix: pcm0: pcm0: nid=11 [audio mixer] pcm0: | pcm0: + <- nid=24 [pin: Mic (Pink Jack)] [src: mic] pcm0: + <- nid=26 [pin: Line-in (Blue Jack)] [src: line] pcm0: + <- nid=29 [beep widget] [src: speaker] pcm0: pcm0: +-------------------------+ pcm0: | DUMPING Volume Controls | pcm0: +-------------------------+ pcm0: pcm0: Master Volume (OSS: vol): -46/0dB pcm0: | pcm0: +- ctl 13 (nid 12 out): -46/0dB (32 steps) pcm0: +- ctl 14 (nid 12 in 0): mute pcm0: +- ctl 15 (nid 12 in 1): mute pcm0: +- ctl 25 (nid 20 in ): mute pcm0: pcm0: PCM Volume (OSS: pcm): 0/0dB pcm0: | pcm0: +- ctl 14 (nid 12 in 0): mute pcm0: pcm0: Microphone Volume (OSS: mic): 0/30dB pcm0: | pcm0: +- ctl 3 (nid 11 in 0): -34/12dB (32 steps) + mute pcm0: +- ctl 34 (nid 24 out): 0/30dB (4 steps) pcm0: +- ctl 53 (nid 35 in 0): mute pcm0: pcm0: Line-in Volume (OSS: line): 0/30dB pcm0: | pcm0: +- ctl 5 (nid 11 in 2): -34/12dB (32 steps) + mute pcm0: +- ctl 38 (nid 26 out): 0/30dB (4 steps) pcm0: +- ctl 55 (nid 35 in 2): mute pcm0: pcm0: Speaker/Beep Volume (OSS: speaker): -34/12dB pcm0: | pcm0: +- ctl 8 (nid 11 in 5): -34/12dB (32 steps) + mute pcm0: +- ctl 58 (nid 35 in 5): mute pcm0: pcm0: Recording Level (OSS: rec): -16/30dB pcm0: | pcm0: +- ctl 1 (nid 8 in 0): -16/30dB (32 steps) + mute pcm0: +- ctl 53 (nid 35 in 0): mute pcm0: +- ctl 55 (nid 35 in 2): mute pcm0: +- ctl 58 (nid 35 in 5): mute pcm0: +- ctl 63 (nid 35 in 10): mute pcm0: pcm0: Input Mix Level (OSS: mix): -34/12dB pcm0: | pcm0: +- ctl 3 (nid 11 in 0): -34/12dB (32 steps) + mute pcm0: +- ctl 5 (nid 11 in 2): -34/12dB (32 steps) + mute pcm0: +- ctl 8 (nid 11 in 5): -34/12dB (32 steps) + mute pcm0: +- ctl 15 (nid 12 in 1): mute pcm0: +- ctl 63 (nid 35 in 10): mute pcm0: pcm0: Input Monitoring Level (OSS: igain): 0/0dB pcm0: | pcm0: +- ctl 15 (nid 12 in 1): mute pcm0: pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Mixer "speaker": pcm0: Mixer "line": pcm0: Mixer "mic": pcm0: Mixer "mix": pcm0: Mixer "rec": pcm0: Mixer "igain": pcm0: Mixer "ogain": pcm0: Soft PCM mixer ENABLED pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 3440000, 10000; 0xffffff806bb1c000 -> 3440000 pcm0: sndbuf_setmap 27c0000, 10000; 0xffffff806bb5c000 -> 27c0000 pcm1: at nid 27 and 25 on hdaa0 pcm1: +--------------------------------------+ pcm1: | DUMPING PCM Playback/Record Channels | pcm1: +--------------------------------------+ pcm1: pcm1: Playback: pcm1: pcm1: Stream cap: 0x00000001 pcm1: PCM pcm1: PCM cap: 0x000e0560 pcm1: 16 20 24 bits, 44 48 96 192 KHz pcm1: DAC: 3 pcm1: pcm1: Record: pcm1: pcm1: Stream cap: 0x00000001 pcm1: PCM pcm1: PCM cap: 0x000e0560 pcm1: 16 20 24 bits, 44 48 96 192 KHz pcm1: DAC: 9 pcm1: pcm1: +-------------------------------+ pcm1: | DUMPING Playback/Record Paths | pcm1: +-------------------------------+ pcm1: pcm1: Playback: pcm1: pcm1: nid=27 [pin: Headphones (Green Jack)] pcm1: | pcm1: + <- nid=13 [audio mixer] [src: pcm, mix] pcm1: | pcm1: + <- nid=3 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: Record: pcm1: pcm1: nid=9 [audio input] pcm1: | pcm1: + <- nid=34 [audio mixer] [src: speaker, monitor] pcm1: | pcm1: + <- nid=25 [pin: Mic (Pink Jack)] [src: monitor] pcm1: + <- nid=29 [beep widget] [src: speaker] pcm1: pcm1: +-------------------------+ pcm1: | DUMPING Volume Controls | pcm1: +-------------------------+ pcm1: pcm1: Master Volume (OSS: vol): -46/0dB pcm1: | pcm1: +- ctl 16 (nid 13 out): -46/0dB (32 steps) pcm1: +- ctl 17 (nid 13 in 0): mute pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: +- ctl 39 (nid 27 in ): mute pcm1: pcm1: PCM Volume (OSS: pcm): 0/0dB pcm1: | pcm1: +- ctl 17 (nid 13 in 0): mute pcm1: pcm1: Microphone2 Volume (OSS: monitor): 0/30dB pcm1: | pcm1: +- ctl 36 (nid 25 out): 0/30dB (4 steps) pcm1: +- ctl 42 (nid 34 in 1): mute pcm1: pcm1: Speaker/Beep Volume (OSS: speaker) pcm1: | pcm1: +- ctl 46 (nid 34 in 5): mute pcm1: pcm1: Recording Level (OSS: rec): -16/30dB pcm1: | pcm1: +- ctl 2 (nid 9 in 0): -16/30dB (32 steps) + mute pcm1: +- ctl 36 (nid 25 out): 0/30dB (4 steps) pcm1: +- ctl 42 (nid 34 in 1): mute pcm1: +- ctl 46 (nid 34 in 5): mute pcm1: pcm1: Input Mix Level (OSS: mix) pcm1: | pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: pcm1: Input Monitoring Level (OSS: igain): 0/0dB pcm1: | pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: pcm1: Mixer "vol": pcm1: Mixer "pcm": pcm1: Mixer "rec": pcm1: Mixer "igain": pcm1: Mixer "monitor": pcm1: Soft PCM mixer ENABLED pcm1: clone manager: deadline=750ms flags=0x8000001e pcm1: sndbuf_setmap 3480000, 10000; 0xffffff806bb9c000 -> 3480000 pcm1: sndbuf_setmap 34c0000, 10000; 0xffffff808473c000 -> 34c0000 pcm2: at nid 30 on hdaa0 pcm2: +--------------------------------------+ pcm2: | DUMPING PCM Playback/Record Channels | pcm2: +--------------------------------------+ pcm2: pcm2: Playback: pcm2: pcm2: Stream cap: 0x00000005 pcm2: AC3 PCM pcm2: PCM cap: 0x000e05e0 pcm2: 16 20 24 bits, 44 48 88 96 192 KHz pcm2: DAC: 6 pcm2: pcm2: +-------------------------------+ pcm2: | DUMPING Playback/Record Paths | pcm2: +-------------------------------+ pcm2: pcm2: Playback: pcm2: pcm2: nid=30 [pin: SPDIF-out (Black Jack)] pcm2: | pcm2: + <- nid=6 [audio output] [src: pcm] pcm2: pcm2: +-------------------------+ pcm2: | DUMPING Volume Controls | pcm2: +-------------------------+ pcm2: pcm2: Mixer "vol" -> "none": child=0x00000010 pcm2: Mixer "pcm": parent="vol" pcm2: Soft PCM mixer ENABLED pcm2: clone manager: deadline=750ms flags=0x8000001e pcm2: sndbuf_setmap 3500000, 10000; 0xffffff808477c000 -> 3500000 usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 ata0: reset tp1 mask=03 ostat0=50 ostat1=50 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=50 devices=0x3 ata1: reset tp1 mask=00 ostat0=ff ostat1=ff (aprobe0:ata0:0:1:0): Spinning up device (aprobe0:ata0:0:1:0): Spin-up done uhub0: 10 ports with 10 removable, self powered ata2: SATA connect timeout status=00000000 ata3: SATA connect timeout status=00000000 ata4: SATA connect timeout status=00000000 ata5: SATA connect timeout status=00000000 pass0 at ata0 bus 0 scbus0 target 0 lun 0 pass0: ATA-6 device pass0: Serial Number M1000000 pass0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) pass1 at ata0 bus 0 scbus0 target 1 lun 0 pass1: ATA-5 device pass1: Serial Number VNP214B2T59A5E pass1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0 at ata0 bus 0 scbus0 target 0 lun 0 ada0: ATA-6 device ada0: Serial Number M1000000 ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0: 19541MB (40020624 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ata0 bus 0 scbus0 target 1 lun 0 ada1: ATA-5 device ada1: Serial Number VNP214B2T59A5E ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada1: 39266MB (80418240 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad1 GEOM: new disk ada0 GEOM: new disk ada1 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1615566420 Hz quality 1000 uhub1: 10 ports with 10 removable, self powered Trying to mount root from ufs:/dev/ada0s1a [rw]... start_init: trying /sbin/init linprocfs registered miibus0: mii_mediachg: can't handle non-zero PHY instance 31 miibus0: mii_mediachg: can't handle non-zero PHY instance 30 miibus0: mii_mediachg: can't handle non-zero PHY instance 29 miibus0: mii_mediachg: can't handle non-zero PHY instance 28 miibus0: mii_mediachg: can't handle non-zero PHY instance 27 miibus0: mii_mediachg: can't handle non-zero PHY instance 26 miibus0: mii_mediachg: can't handle non-zero PHY instance 25 miibus0: mii_mediachg: can't handle non-zero PHY instance 24 miibus0: mii_mediachg: can't handle non-zero PHY instance 23 miibus0: mii_mediachg: can't handle non-zero PHY instance 22 miibus0: mii_mediachg: can't handle non-zero PHY instance 21 miibus0: mii_mediachg: can't handle non-zero PHY instance 20 miibus0: mii_mediachg: can't handle non-zero PHY instance 19 miibus0: mii_mediachg: can't handle non-zero PHY instance 18 miibus0: mii_mediachg: can't handle non-zero PHY instance 17 miibus0: mii_mediachg: can't handle non-zero PHY instance 16 miibus0: mii_mediachg: can't handle non-zero PHY instance 15 miibus0: mii_mediachg: can't handle non-zero PHY instance 14 miibus0: mii_mediachg: can't handle non-zero PHY instance 13 miibus0: mii_mediachg: can't handle non-zero PHY instance 12 miibus0: mii_mediachg: can't handle non-zero PHY instance 11 miibus0: mii_mediachg: can't handle non-zero PHY instance 10 miibus0: mii_mediachg: can't handle non-zero PHY instance 9 miibus0: mii_mediachg: can't handle non-zero PHY instance 8 miibus0: mii_mediachg: can't handle non-zero PHY instance 7 miibus0: mii_mediachg: can't handle non-zero PHY instance 6 miibus0: mii_mediachg: can't handle non-zero PHY instance 5 miibus0: mii_mediachg: can't handle non-zero PHY instance 4 miibus0: mii_mediachg: can't handle non-zero PHY instance 3 miibus0: mii_mediachg: can't handle non-zero PHY instance 2 miibus0: mii_mediachg: can't handle non-zero PHY instance 1 Too much? Thank you very much, for all your time, and consideration. --Chris > >> >> # uname -a >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756: Wed Mar 26 11:28:10 PDT 2014 >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 >> >> Thank you for all your time, and consideration. >> >> --Chris > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 16:03:39 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8A849E8 for ; Mon, 31 Mar 2014 16:03:39 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 550B11C1 for ; Mon, 31 Mar 2014 16:03:39 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id l18so6031411wgh.28 for ; Mon, 31 Mar 2014 09:03:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SkaWteWZo4CqvVOkMnqQTRHKf8+MDh3XkXkpe2TTR40=; b=BfmQLvueQ+O0qSDw7cp7mPEQtZXyj2JK5BO0iPxIi2VPpulgJ1+xeCX5h8SBD0lcAS oxYOVsBB9j/oUj5H+qxnWYZQVLdDphu6WK9heqEik1gJVkWZiaXOoN4iOneqGlpwT1u6 yYDsMvgeDQkdkVMlF6U+ICcGr4t9/CEAG+5HU45o9ARY1b8ltSc8ifP1iKU2ERJsLDvc pTc1xE1AEou6O3/hAb8/TO23ELo2FLQ8pw6Tl1bX9Umhq08WTxHzWnT+VA8rYG2t8cs3 liorUYIp1rZe2aGgNcfdf0Ar6vS/Vq8Rqux41XeQeVZV/cbZXmOevflkEhoyvONZksJ5 MPxw== MIME-Version: 1.0 X-Received: by 10.194.119.168 with SMTP id kv8mr14475544wjb.41.1396281817661; Mon, 31 Mar 2014 09:03:37 -0700 (PDT) Received: by 10.14.211.134 with HTTP; Mon, 31 Mar 2014 09:03:37 -0700 (PDT) In-Reply-To: <53396E4B.4050102@rcn.com> References: <53396E4B.4050102@rcn.com> Date: Mon, 31 Mar 2014 09:03:37 -0700 Message-ID: Subject: Re: questions about (system) dhcp From: hiren panchasara To: Robert Huff Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 16:03:39 -0000 On Mon, Mar 31, 2014 at 6:31 AM, Robert Huff wrote: > Hello: > Is this the correct place to ask? Yes. Cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 18:43:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A06E7FE for ; Mon, 31 Mar 2014 18:43:11 +0000 (UTC) Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E68BB692 for ; Mon, 31 Mar 2014 18:43:09 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id q5so3864131wiv.16 for ; Mon, 31 Mar 2014 11:43:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:content-transfer-encoding :subject:message-id:date:to:mime-version; bh=ZNGmr3JSCNbV3VWCnWAHz0/IvR4J6wiYDWkUCvxAWZc=; b=dOLk9peqB/fEje6DbZb4BmX0jnOUy4xYtKS19Tnpx+Yht2LdFvEugsJpz4SsMAvtZo xGGB9DNmrMNl8s6oNkGSG1TUlEyTEp47lAGPdFM4JCX6QMJwAmcjEi7FkwKZZIRY/oJj xR6S0u3cr1zKV2DM17Ufi5KBSPCTigrV1vmElDPCmTaIvcXSnTHbWZKnMx5NaEJsJMXa iz6aQzYkxfdD1gPj3TDHbh1MB7i4TE4ZThGeUGiVeepHFpRItD+2BPhZY8HMHDTpvd5H dRW55x6CxkDt8d6EG/KG6FUe1GCDWd4n1vGFA002kZTxGeYXupAj2AZuS8s1bGRSFhn2 iXCQ== X-Gm-Message-State: ALoCoQlyscGA4JbiQ4jsTFaC/m7maTO9lr69d/4EqoRKFSxiX4h59DgRIkYhs6fd79ueB7sCKpBV X-Received: by 10.180.101.134 with SMTP id fg6mr14422795wib.6.1396291388171; Mon, 31 Mar 2014 11:43:08 -0700 (PDT) Received: from grey.home.unixconn.com (h-74-23.a183.priv.bahnhof.se. [46.59.74.23]) by mx.google.com with ESMTPSA id a42sm34858454ees.10.2014.03.31.11.43.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 31 Mar 2014 11:43:07 -0700 (PDT) From: mxb Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: FreeBSD 10-STABLE and CARP states Message-Id: Date: Mon, 31 Mar 2014 20:42:39 +0200 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 18:43:11 -0000 Hi list, hopefully this is the right place to have my question regarding CARP on = 10-STABLE. I have two nodes with following setup(node1): lagg0: flags=3D8943 = metric 0 mtu 9000 = options=3D8407bb ether 00:25:90:e3:71:f2 inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 nd6 options=3D29 media: Ethernet autoselect status: active carp: BACKUP vhid 201 advbase 1 advskew 1 carp: BACKUP vhid 202 advbase 5 advskew 100 laggproto lacp lagghash l2,l3,l4 laggport: ix1 flags=3D1c laggport: ix0 flags=3D1c net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: pfsync0: flags=3D41 metric 0 mtu 1500 pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as of = setup. Am I hitting a bug or doing something wrong? I also have noted that after the pfsync bulk update the demotion counter = never setts to 0, but stays on 480, thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? Regards, mxb From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 19:40:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB77EB8F for ; Mon, 31 Mar 2014 19:40:01 +0000 (UTC) Received: from mail.ignoranthack.me (ujvl.x.rootbsd.net [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A54DCBD9 for ; Mon, 31 Mar 2014 19:40:01 +0000 (UTC) Received: from [10.73.160.242] (nat-dip7.cfw-a-gci.corp.yahoo.com [209.131.62.116]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 6FBBC1929C9; Mon, 31 Mar 2014 19:40:00 +0000 (UTC) Subject: Re: FreeBSD 10-STABLE and CARP states From: Sean Bruno To: mxb In-Reply-To: References: Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-sLRjzQhSV3oJnYE25L+7" Date: Mon, 31 Mar 2014 12:39:56 -0700 Message-ID: <1396294796.1663.29.camel@powernoodle.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 19:40:02 -0000 --=-sLRjzQhSV3oJnYE25L+7 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Mon, 2014-03-31 at 20:42 +0200, mxb wrote: > Hi list, >=20 > hopefully this is the right place to have my question regarding CARP on 1= 0-STABLE. >=20 > I have two nodes with following setup(node1): >=20 > lagg0: flags=3D8943 metri= c 0 mtu 9000 > options=3D8407bb > ether 00:25:90:e3:71:f2 > inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 > inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 > inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid 201 > inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid 202 > nd6 options=3D29 > media: Ethernet autoselect > status: active > carp: BACKUP vhid 201 advbase 1 advskew 1 > carp: BACKUP vhid 202 advbase 5 advskew 100 > laggproto lacp lagghash l2,l3,l4 > laggport: ix1 flags=3D1c > laggport: ix0 flags=3D1c Took a look at the freebsd cluster routers(running -current), we seem to not have the carp info on lagg0, but on seperate vlans and we set the vlan parent to lagg0 instead. e.g. lagg0: flags=3D8943 metric 0 mtu 1500 options=3D4219b ether 00:25:90:c2:85:32 inet6 fe80::225:90ff:fec2:8532%lagg0 prefixlen 64 tentative scopeid 0x6=20 nd6 options=3D29 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em1 flags=3D1c laggport: em0 flags=3D1c ---- pflog0: flags=3D141 metric 0 mtu 33160 pfsync0: flags=3D41 metric 0 mtu 1500 pfsync: syncdev: vlan33 syncpeer: 10.0.16.2 maxupd: 128 defer: off ---- vlan33: flags=3D8843 metric 0 mtu 1500 options=3D103 ether 00:25:90:c2:85:32 inet6 fe80::225:90ff:fec2:8532%vlan33 prefixlen 64 scopeid 0x9=20 inet 10.0.16.1 netmask 0xffffff00 broadcast 10.0.16.255=20 nd6 options=3D29 media: Ethernet autoselect status: active vlan: 33 parent interface: lagg0 sean --=-sLRjzQhSV3oJnYE25L+7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAABAgAGBQJTOcSMAAoJEBkJRdwI6BaHi04H/1y6br4WlgcUCVPCHT0DQaSK 3Flx//o0GDQSjXepNOVQ9gUjXXzrNpdkb7ie+W2CwshRi+XyGy6ArV3E9Fu5Zy/I uk5weEEb4oSqgrF/WCpAAsYL6JcmSOisdMGcK24wVzWH7OqoSchfa33t/cVhEDif W8gsxoLa63dEaQZF7kYX+0m4UoiPebX0Xv2l2MX4pLULDa4Egm0f6+tg5qXF3sg1 wK55HsoQC9PyS9ofDfk85NW2O8N2zYcfm16K358GtBDn7yHnBuhzX2n3i2WBfKdt kA85Ah8f2+jvQmuhyS+cmhqJVXSRXAN/H/UpxCaX+qIDSFk3u4ppBgJb9ruFEkE= =oxfr -----END PGP SIGNATURE----- --=-sLRjzQhSV3oJnYE25L+7-- From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:03:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 161F45BE; Mon, 31 Mar 2014 20:03:37 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82056E58; Mon, 31 Mar 2014 20:03:36 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id t61so5365897wes.31 for ; Mon, 31 Mar 2014 13:03:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=cYQ0GiXbp7Gc/EOv/4W9drX62iuI0fOuFzNFtbNmLn4=; b=lg2Rswb1zROIgWl6G1rlftTsSHXVTJxY9Cm1IHNsNwgEcuUQ2s5RwwDXMi1f7CwLcL UDmupDQtAU5Ti2mWshB+tK77H4z5ykjExq7e9vLHXJyTmpJPZjOUDmo8SG/IXRNocMfj 942gxhbS0368zmX83wdBl1jEJf+I4qsfTOSkipwDVYyz8QLEALp4/sZWgV4ZB38H0ioQ tWKAfVxhOwYVE0/KYUFXYHAYwZJchj9pJNPhAp37hc2H5PdE6cfkm/p5pqGJG+51gOKO 6gysSkCNLhPI2cF7AOO4djg3P1cuh1PFRxzG/oKbS2WypE8lvqWCm7HLrbQIidpy9YL2 HbCA== MIME-Version: 1.0 X-Received: by 10.194.190.42 with SMTP id gn10mr17327003wjc.9.1396296213631; Mon, 31 Mar 2014 13:03:33 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Mon, 31 Mar 2014 13:03:33 -0700 (PDT) Date: Mon, 31 Mar 2014 14:03:33 -0600 X-Google-Sender-Auth: LzBoT9LjFSy0zlaVDvbfbnJrjxQ Message-ID: Subject: netstat -i[d] violates PoLS From: Alan Somers To: FreeBSD Net , attilio@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:03:37 -0000 "netstat -i" prints dropped output packets iff you also use "-d". Starting with r199803 on 2009-11-25, "netstat -i" prints dropped input packets regardless of the "-d" flags. That is a PoLS violation, IMHO. I think that the "-d" flag should control printing of dropped input packets as well as dropped output packets. OTOH, this behavior has been around for more than 4 years, and some scripts may rely on it. At the very least, the man page should be updated to reflect r199803. What do you think? Does the likelihood of hardcoded scripts preclude fixing this bug? -Alan From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:06:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7329862 for ; Mon, 31 Mar 2014 20:06:42 +0000 (UTC) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 38167EB2 for ; Mon, 31 Mar 2014 20:06:41 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so6190225lbj.17 for ; Mon, 31 Mar 2014 13:06:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=FE4gG3B+uU0QPZ41Lml4OJ0G35CFeVH3q4uAhs5WRWg=; b=I1QEviQaqbGiXyEH61w3jt/iy/AIbzg6Zv6J/L9v/h1aofw2ZnEhvgfFcdUlOBsb4s yi7DfuyyFXZL9EhLtWtO2eBwC3o7/0dIQ7qvBFtG4yf+C2L/t9Ah3VUOwSEPwsHOxAIc HH2473RFrLaEUMgv4/WmsBDRutHGO6hHf4MotezU9uUoQrn49AlG5KHnlAdWYh7METsM eWdZfBlVjLIZrr9C+vSGS5gl2daRot/bSagldvtH31Skord4q2jikYeJsI48ShsS37CX u0uljeyAE83GS3XdstrLqCm/+f4L57nXDwx9tt7XkjEvpfFzaBlD3qweJBbGQPcn5w5Z r1LQ== X-Gm-Message-State: ALoCoQkYJumbCoVOkfDGdJc0fMFxEWs2oxiMG/wbdk08DsB6HHgGgVJDhy00jPYBlH2gtOImsrbv X-Received: by 10.112.24.9 with SMTP id q9mr18855583lbf.23.1396296393651; Mon, 31 Mar 2014 13:06:33 -0700 (PDT) Received: from grey.home.unixconn.com (h-74-23.a183.priv.bahnhof.se. [46.59.74.23]) by mx.google.com with ESMTPSA id jy5sm14884657lac.9.2014.03.31.13.06.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 31 Mar 2014 13:06:32 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10-STABLE and CARP states From: mxb In-Reply-To: <1396294796.1663.29.camel@powernoodle.corp.yahoo.com> Date: Mon, 31 Mar 2014 22:06:37 +0200 Message-Id: References: <1396294796.1663.29.camel@powernoodle.corp.yahoo.com> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:06:42 -0000 I don=92t see any carp out of the info you sent. e.g. on vlan33 On 31 mar 2014, at 21:39, Sean Bruno wrote: > but on seperate vlans and we set the > vlan parent to lagg0 instead. From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:17:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A852D68; Mon, 31 Mar 2014 20:17:28 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4A80FBE; Mon, 31 Mar 2014 20:17:27 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id d1so4008544wiv.15 for ; Mon, 31 Mar 2014 13:17:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=f6+1gM4mnr42a+XWAS3IxQ9tHwP0oHuQJ/ry33JIv5Q=; b=BSydAuMQMCi1Fu3V91+2spWoAX7l/mTgHgY/8V1Y0lDutIcdn1FV/VN/L8EYT3UD+9 loOBel0YKPhwGzCOT+k5zfR0NiNiOD4Uc6Uq+RfwDUh4MjSq1lbJ4ghAO8oxU61nfhx3 Z7FX4+41ZEMsiZ751O9GUPxh4sG7fBDfBS89KYY5UFEnfDTXHDjaL05H3hrd5mFTVlQp g9q+187Rz/8SoEkTuMtqF7E6o8V/TkRP0ky2gzh6yqoW6DBfPq/46qvkGnTquJWsyCQM tVvKPE3eSute87y270Fr5c0dKb2MQGoX3EWEU92rcMBcFQFhqdT/6Zzh1bnEBsMku40Y ac8w== MIME-Version: 1.0 X-Received: by 10.194.190.42 with SMTP id gn10mr17399472wjc.9.1396297045841; Mon, 31 Mar 2014 13:17:25 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.217.61.196 with HTTP; Mon, 31 Mar 2014 13:17:25 -0700 (PDT) In-Reply-To: References: Date: Mon, 31 Mar 2014 22:17:25 +0200 X-Google-Sender-Auth: 3Egbo1-ZT-NCj_ZSyf3NGFHY73I Message-ID: Subject: Re: netstat -i[d] violates PoLS From: Attilio Rao To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:17:28 -0000 On Mon, Mar 31, 2014 at 10:03 PM, Alan Somers wrote: > "netstat -i" prints dropped output packets iff you also use "-d". > Starting with r199803 on 2009-11-25, "netstat -i" prints dropped input > packets regardless of the "-d" flags. That is a PoLS violation, IMHO. > I think that the "-d" flag should control printing of dropped input > packets as well as dropped output packets. I don't have any time to look into this, so please fix it as you prefer for the time being. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:21:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92BA0F67 for ; Mon, 31 Mar 2014 20:21:52 +0000 (UTC) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com [209.85.215.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17899F3 for ; Mon, 31 Mar 2014 20:21:51 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id hr17so6228675lab.18 for ; Mon, 31 Mar 2014 13:21:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=RKjMNah3z95z8Qx8hTk88iXV0GAaDm5zoDdEkjD0kbg=; b=eHzsAfocyJ+qbyIiiCd9jDT92jshLaAhsJ0bSTg8tjcrYLNlkFsEFuiHpIFrT6jg2Z l7rlCnINwqEF5zX6bhTYH66+wzi/cPfbtWKD7xw7SSGvqtr2WS5nAipKb6wIIxaBlTp7 i4SlS4btCfi/ar7Q0cvTq7kdI/oE6MxLgNJ3tDkp4Pdf0vMWY6dT7uxWmSXslca5Nbfl H83rRgTPLgwWjQ1Y5+90uh4234fEA2I2ElcK7XGF4FzJ0idF0PjfNRsJ9couRpL4DnHd fGy4yLaG3sfjnnjed/28Up2XCiXPsMKj2Ydks0QkZuYm18l2jyTaMT8fpWu53jep5YW1 /WWg== X-Gm-Message-State: ALoCoQlZ0b0QhLJS3MVG6zvd+ipCQAtEmW5Js1QGa1j2KWemYqK7qRgJJjLGnygzPs2oNXhbjZLH X-Received: by 10.152.1.8 with SMTP id 8mr19944367lai.1.1396297303787; Mon, 31 Mar 2014 13:21:43 -0700 (PDT) Received: from grey.home.unixconn.com (h-74-23.a183.priv.bahnhof.se. [46.59.74.23]) by mx.google.com with ESMTPSA id jm3sm10571215lbc.29.2014.03.31.13.21.42 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 31 Mar 2014 13:21:43 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10-STABLE and CARP states From: mxb In-Reply-To: Date: Mon, 31 Mar 2014 22:21:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> References: To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:21:52 -0000 Manually setting net.inet.carp.demotion brought BOTH VHIDs in desired = state. pfsync bulk update seems to not put everything back as it should. lagg0: flags=3D8943 = metric 0 mtu 9000 = options=3D8407bb ether 00:25:90:e3:71:f2 inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 nd6 options=3D29 media: Ethernet autoselect status: active carp: MASTER vhid 201 advbase 1 advskew 1 carp: BACKUP vhid 202 advbase 5 advskew 100 laggproto lacp lagghash l2,l3,l4 laggport: ix1 flags=3D1c laggport: ix0 flags=3D1c On 31 mar 2014, at 20:42, mxb wrote: >=20 > Hi list, >=20 > hopefully this is the right place to have my question regarding CARP = on 10-STABLE. >=20 > I have two nodes with following setup(node1): >=20 > lagg0: flags=3D8943 = metric 0 mtu 9000 > = options=3D8407bb > ether 00:25:90:e3:71:f2 > inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 > inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 > inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 > inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 > nd6 options=3D29 > media: Ethernet autoselect > status: active > carp: BACKUP vhid 201 advbase 1 advskew 1 > carp: BACKUP vhid 202 advbase 5 advskew 100 > laggproto lacp lagghash l2,l3,l4 > laggport: ix1 flags=3D1c > laggport: ix0 flags=3D1c >=20 > net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: >=20 > pfsync0: flags=3D41 metric 0 mtu 1500 > pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off >=20 > The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. > Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as of = setup. >=20 > Am I hitting a bug or doing something wrong? >=20 > I also have noted that after the pfsync bulk update the demotion = counter never setts to 0, but stays on 480, > thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? >=20 > Regards, > mxb >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:31:52 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C788436F for ; Mon, 31 Mar 2014 20:31:52 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8D2921E2 for ; Mon, 31 Mar 2014 20:31:52 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=buTO9Tmi c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=qZLpgtK1PmoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=xA2UYjx5SB6WBWL1eZAA:9 a=QEXdDO2ut3YA:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:4982] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id A5/B3-59476-6B0D9335; Mon, 31 Mar 2014 16:31:51 -0400 Message-ID: <5339D0A0.1030903@rcn.com> Date: Mon, 31 Mar 2014 16:31:28 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: questions about (system) dhclient Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Huff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:31:52 -0000 [Please keep me CC'd as I am not subscribed. Thanks.] I have a system, running r263263, where dhclient is misbehaving. (Yes - this is CURRENT, but I have no reason to believe this inherently a version-specific issue. I am also on current@, and nothing like this has been reported.) The system has an Intel Pro/1000 ethernet card, "em0", connected to a Arris CM820 cable modem. This is the relevant portion of dhclient.conf: http://users.rcn.com/dhclient.conf Upon execution, I get this: http://users.rcn.com/roberthuff/dhcp_offer but then: http://users.rcn.com/roberthuff/dhc_ifconfig "netstat -r" also shows no gateway. However, when I look at dhclient.leases: http://users.rcn.com/roberthuff/dhclient.leases.em0 I get the machine working by configuring the relevant data manually. I have no idea where to start trying to figure this out; this have "just worked" for so long. Help please? Respectfully, Robert Huff From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 20:38:51 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5CAF48F for ; Mon, 31 Mar 2014 20:38:51 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 514A5221 for ; Mon, 31 Mar 2014 20:38:51 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id bs8so4024462wib.11 for ; Mon, 31 Mar 2014 13:38:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hWHxwjdZGsrlqr4it62netxBJssKZTv2TIxWDttAERQ=; b=AzItVXJ0A2Z3ywpv7/C8XiOU3JdIdw7pxqniAk0k/zp7O2/BUfFBcxg5TGXmO16kci bF7kalUaG7FKmlbsiW8MPDiW/TXL4rHL60TflIA2crRu8fk9xVMeaoFHT871GYLkKgsY rQ8D6Oaxnam+3+hq0G8Wzj+vKKP3+NnCayN8Lmk1QBbslF4d6a9LHMXhnxK674jlvyww MYbb23lGvAWVEwYWJ9VgNDNTrgY4bXi5VZbwpjXT6JZr7Eid+ej17QBjQHI/igVc0mQx 7Q5lmGJZ2vDK93FkIIxDFXVBLr8Kz0SaqQVMq5KqN8tSUNb5KLKajX/BpJj4wk8oz4o1 couw== MIME-Version: 1.0 X-Received: by 10.180.106.167 with SMTP id gv7mr15163968wib.40.1396298329615; Mon, 31 Mar 2014 13:38:49 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Mon, 31 Mar 2014 13:38:49 -0700 (PDT) In-Reply-To: <5339D0A0.1030903@rcn.com> References: <5339D0A0.1030903@rcn.com> Date: Mon, 31 Mar 2014 14:38:49 -0600 X-Google-Sender-Auth: uujzNe5p6vjVR7R3px3tFtzuHQc Message-ID: Subject: Re: questions about (system) dhclient From: Alan Somers To: Robert Huff Content-Type: text/plain; charset=ISO-8859-1 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 20:38:51 -0000 On Mon, Mar 31, 2014 at 2:31 PM, Robert Huff wrote: > [Please keep me CC'd as I am not subscribed. Thanks.] > > I have a system, running r263263, where dhclient is misbehaving. > (Yes - this is CURRENT, but I have no reason to believe this inherently a > version-specific issue. I am also on current@, and nothing like this has > been reported.) > The system has an Intel Pro/1000 ethernet card, "em0", connected to > a Arris CM820 cable modem. > This is the relevant portion of dhclient.conf: > > http://users.rcn.com/dhclient.conf File not found > > Upon execution, I get this: > > http://users.rcn.com/roberthuff/dhcp_offer You're getting on offer on a completely different network than that where the dhcp server resides. That's weird. Do you expect that to happen? > > but then: > > http://users.rcn.com/roberthuff/dhc_ifconfig > > "netstat -r" also shows no gateway. > > However, when I look at dhclient.leases: > > http://users.rcn.com/roberthuff/dhclient.leases.em0 This file is empty. Is it supposed to be? > > I get the machine working by configuring the relevant data manually. > > I have no idea where to start trying to figure this out; this have > "just worked" for so long. > Help please? Try running "service netif stop em0" to stop the background dhclient process. Then run "dhclient -d" to force it to remain in the foreground, and see if it prints any useful debugging info. It might also be useful to run "tcpdump -i em0 host 10.23.192.1" in another terminal. > > Respectfully, > > Robert Huff > -Alan From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 21:10:37 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1CECEB8 for ; Mon, 31 Mar 2014 21:10:37 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 876677A9 for ; Mon, 31 Mar 2014 21:10:37 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=buTO9Tmi c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=phkJtM7cgHoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=slE553JrixPIgnkwCocA:9 a=QEXdDO2ut3YA:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:1323] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id EB/AA-59476-BC9D9335; Mon, 31 Mar 2014 17:10:36 -0400 Message-ID: <5339D9AD.4070307@rcn.com> Date: Mon, 31 Mar 2014 17:10:05 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: Re: questions about (system) dhclient Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Huff , Alan Somers X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 21:10:37 -0000 >> This is the relevant portion of dhclient.conf: >> >> http://users.rcn.com/dhclient.conf > > File not found ooops - try "http://users.rcn.com/roberthuff/dhclient.conf" >> Upon execution, I get this: > >> http://users.rcn.com/roberthuff/dhcp_offer > >You're getting on offer on a completely different network than that >where the dhcp server resides. That's weird. Do you expect that to >happen? Yes. (Sometimes it's the same network, sometimes not.) > However, when I look at dhclient.leases: > > http://users.rcn.com/roberthuff/dhclient.leases.em0 > >This file is empty. Is it supposed to be? No. It should be there now. > Try running "service netif stop em0" to stop the background dhclient > process. Manually killed all dhclient processes. > Then run "dhclient -d" to force it to remain in the > foreground, and see if it prints any useful debugging info. Just the offer, ack, and bound-to/renewal lines. > It might > also be useful to run "tcpdump -i em0 host > 10.23.192.1" in another terminal. It may take a while before I can do that. Thanks for helping. Robert Huff From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 22:30:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D4B61B3 for ; Mon, 31 Mar 2014 22:30:06 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F0A02D7A for ; Mon, 31 Mar 2014 22:30:05 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 370C3139C4 for ; Mon, 31 Mar 2014 22:32:43 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1396315961; x=1397179962; bh=H6uvnACDFbfttB8va8izVWsmmpLyNTiyHky jl3aUs7U=; b=BaO1bu4j8Oxv1ldlHapzW7RCdWYJ9jE9XFcfZFayFUyL914hS8e ljXMCRfUmJe7K4edJR4GbJnIo1562FKViF3DY0oTE115IjF6kQPT+plaU4x4saf2 rPuqAIgSL9UjsKOd9l2ew7v+D0YVKlMuRHJcdEFxBSKBbVngd4hCVQOE= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LU9Ok2zEzWAv for ; Mon, 31 Mar 2014 22:32:41 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id F09A5139C3; Mon, 31 Mar 2014 22:32:38 -0300 (BRT) Message-ID: <5339EC5D.3030408@bsdinfo.com.br> Date: Mon, 31 Mar 2014 19:29:49 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: Process handlers, and zombies, or preap(1) References: <20140331211147.GA52184@anubis.morrow.me.uk> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 22:30:06 -0000 Em 31/03/14 18:39, Chris H escreveu: >> Quoth "Chris H" : >>> I'm evaluating/experimenting on releng_9. The install, and now >>> custom kernel have noting exotic, or anything out of the ordinary. >>> top(1), and ps(1) indicate a (1) zombie, or process. On >>> my releng_8 systems, when I occasionally encounter one of these, >>> they soon disappear (are reaped) from the process table. While I >>> have not investigated this far enough on both versions to determine >>> whether the parent process reaped the child on the releng_8 systems, >>> and the parent on releng_9 is simply an irresponsible parent, eg; >>> a different parent. >> What is the parent? > Sorry, that /should/ have been clearer. :) > Meaning; the processes (parents) that are reaping the zombies on releng_8 > are different that those I'm seeing on releng_9. > In other words; On releng_8, I see a zombie, then seconds later, it's > gone. On releng_9, I see a zombie, and it never leaves. > Is the "parent" of the dead "child" on releng_9, different than that of > the parent on releng_8. I couldn't possibly expect you to know. But not > having been able to catch the parent process reaping the defunct child > on releng_8, before it has reaped it. I cannot know. Which led me to ask; > Is there anything different on releng_9, that might cause zombies > terminally within the process table? > A bit wordy, perhaps. But makes the point. No? :) > >>> Before I do, I was wondering if there was any >>> specific difference between the 2 versions that might cause better >>> handling of such situations. While I recognize that resource >>> starvation is HIGHLY unlikely, except by perhaps a rouge parent >> A rouge parent? :) > Yes. An unfit parent, that will not watch after it's child(ren). We > have agencies in the US that seek to end such delinquencies. Maybe > FreeBSD could employ such tactics. :) > >>> spawning multitudes of zombies. I thought it might be useful for >>> "housekeeping" to 1) provide a process table housekeeper (zombie >>> reaper), >> That's called init(8). When the parent exits, init will wait for the >> zombie. >> >>> or 2) create a system utility/command like SunOS/OpenSolaris >>> has; preap(1). >> That seems like a bad idea, to me. Generally speaking I would expect it >> to be safer to kill and restart the parent, allowing init to do its job. > Maybe. Maybe not. I think it depends on the parent process, and what impact > HUPing it, will have on the system. Tho this should not be an excuse for > not fixing the problem parent. But rather, a stop-gap, until a suitable > fix is created/obtained (for the parent). > > > Thanks for taking the time to respond, Ben. > > --Chris > >> Ben This could be related with this problem that I'm having with zombie processes? [...] 40945 - Is 0:00.01 sshd: luciele [priv] (sshd) 40946 - Z 0:00.01 40947 - IW 0:00.00 sshd: luciele [pam] (sshd) 44376 - Is 0:00.01 sshd: unknown [priv] (sshd) 44377 - Z 0:00.01 44378 - IW 0:00.00 sshd: unknown [pam] (sshd) 58892 - IW 0:00.00 /usr/local/sbin/httpd -DNOHTTPACCEPT 58978 - IW 0:00.00 /usr/local/sbin/httpd -DNOHTTPACCEPT 61361 - IW 0:00.00 /usr/local/sbin/httpd -DNOHTTPACCEPT 61684 - Is 0:00.01 sshd: unknown [priv] (sshd) 61685 - Z 0:00.01 61692 - IW 0:00.00 sshd: unknown [pam] (sshd) 78346 - Is 0:00.01 sshd: unknown [priv] (sshd) 78347 - Z 0:00.01 78351 - IW 0:00.00 sshd: unknown [pam] (sshd) [...] # procstat -f 40945 PID COMM FD T V FLAGS REF OFFSET PRO NAME 40945 sshd text v r r------- - - - /usr/sbin/sshd 40945 sshd cwd v d r------- - - - / 40945 sshd root v d r------- - - - / 40945 sshd 0 v c rw------ 6 0 - /dev/null 40945 sshd 1 v c rw------ 6 0 - /dev/null 40945 sshd 2 v c rw------ 6 0 - /dev/null 40945 sshd 3 s - rw---n-- 2 0 TCP 186.xxx.xx.10:4321 186.xxx.xx.8:64762 40945 sshd 4 s - rw------ 1 0 UDS - 40945 sshd 5 p - rw------ 2 0 - - 40945 sshd 6 s - rw------ 2 0 UDS - # procstat -f 44376 PID COMM FD T V FLAGS REF OFFSET PRO NAME 44376 sshd text v r r------- - - - /usr/sbin/sshd 44376 sshd cwd v d r------- - - - / 44376 sshd root v d r------- - - - / 44376 sshd 0 v c rw------ 6 0 - /dev/null 44376 sshd 1 v c rw------ 6 0 - /dev/null 44376 sshd 2 v c rw------ 6 0 - /dev/null 44376 sshd 3 s - rw---n-- 2 0 TCP 186.xxx.xx.10:4321 186.xxx.xx.8:64368 44376 sshd 4 s - rw------ 1 0 UDS - 44376 sshd 5 p - rw------ 2 0 - - 44376 sshd 6 s - rw------ 2 0 UDS - # procstat -f 61684 PID COMM FD T V FLAGS REF OFFSET PRO NAME 61684 sshd text v r r------- - - - /usr/sbin/sshd 61684 sshd cwd v d r------- - - - / 61684 sshd root v d r------- - - - / 61684 sshd 0 v c rw------ 6 0 - /dev/null 61684 sshd 1 v c rw------ 6 0 - /dev/null 61684 sshd 2 v c rw------ 6 0 - /dev/null 61684 sshd 3 s - rw---n-- 2 0 TCP 186.xxx.xx.10:4321 186.xxx.xx.8:61415 61684 sshd 4 s - rw------ 1 0 UDS - 61684 sshd 5 p - rw------ 2 0 - - 61684 sshd 6 s - rw------ 2 0 UDS - # procstat -f 78346 PID COMM FD T V FLAGS REF OFFSET PRO NAME 78346 sshd text v r r------- - - - /usr/sbin/sshd 78346 sshd cwd v d r------- - - - / 78346 sshd root v d r------- - - - / 78346 sshd 0 v c rw------ 6 0 - /dev/null 78346 sshd 1 v c rw------ 6 0 - /dev/null 78346 sshd 2 v c rw------ 6 0 - /dev/null 78346 sshd 3 s - rw---n-- 2 0 TCP 186.xxx.xx.10:4321 186.xxx.xx.8:50994 78346 sshd 4 s - rw------ 1 0 UDS - 78346 sshd 5 p - rw------ 2 0 - - 78346 sshd 6 s - rw------ 2 0 UDS - # netstat -n | grep CLOSED tcp4 0 0 186.xxx.xx.10.4321 186.xxx.xx.8.64368 CLOSED tcp4 0 0 186.xxx.xx.10.4321 186.xxx.xx.8.61415 CLOSED tcp4 0 0 186.xxx.xx.10.4321 186.xxx.xx.8.50994 CLOSED tcp4 0 0 186.xxx.xx.10.4321 186.xxx.xx.8.64762 CLOSED # uname -a FreeBSD xxxxx.xxxxx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #6 r263882: Fri Mar 28 20:28:40 BRT 2014 root@xxxxx.xxxxx.xxx.xx:/usr/obj/usr/src/sys/GONDIM10 amd64 Cheers, Gondim From owner-freebsd-net@FreeBSD.ORG Mon Mar 31 23:09:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5E699D2; Mon, 31 Mar 2014 23:09:32 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 338DB106; Mon, 31 Mar 2014 23:09:31 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArsEAOP0OVODaFve/2dsb2JhbABZg0FXgwq/DIEegTV0giUBAQEDASMERwsbGAICDRkCIzYZG4dKAwkIDa52mwsNh0sXgSmIJ4MTgUQBIzQHgm+BSQSUYAeBeoMgiziFSoNMIYEsAR8i X-IronPort-AV: E=Sophos;i="4.97,768,1389762000"; d="scan'208";a="110685846" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 31 Mar 2014 19:09:24 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 94508B3F46; Mon, 31 Mar 2014 19:09:24 -0400 (EDT) Date: Mon, 31 Mar 2014 19:09:24 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com Message-ID: <779330717.3747229.1396307364595.JavaMail.root@uoguelph.ca> In-Reply-To: <20140331023253.GC3548@michelle.cdnetworks.com> Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Filesystems , FreeBSD Net , Alexander Motin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 23:09:32 -0000 Yonghyeon Pyun wrote: > On Wed, Mar 26, 2014 at 08:27:48PM -0400, Rick Macklem wrote: > > pyunyh@gmail.com wrote: > > > On Tue, Mar 25, 2014 at 07:10:35PM -0400, Rick Macklem wrote: > > > > Hi, > > > > > > > > First off, I hope you don't mind that I cross-posted this, > > > > but I wanted to make sure both the NFS/iSCSI and networking > > > > types say it. > > > > If you look in this mailing list thread: > > > > http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > > > > you'll see that several people have been working hard at > > > > testing > > > > and > > > > thanks to them, I think I now know what is going on. > > > > > > > > > Thanks for your hard work on narrowing down that issue. I'm too > > > busy for $work in these days so I couldn't find time to > > > investigate > > > the issue. > > > > > > > (This applies to network drivers that support TSO and are > > > > limited > > > > to 32 transmit > > > > segments->32 mbufs in chain.) Doing a quick search I found the > > > > following > > > > drivers that appear to be affected (I may have missed some): > > > > jme, fxp, age, sge, msk, alc, ale, ixgbe/ix, nfe, e1000/em, re > > > > > > > > > > The magic number 32 was chosen long time ago when I implemented > > > TSO > > > in non-Intel drivers. I tried to find optimal number to reduce > > > the > > > size kernel stack usage at that time. bus_dma(9) will coalesce > > > with previous segment if possible so I thought the number 32 was > > > not an issue. Not sure current bus_dma(9) also has the same code > > > though. The number 32 is arbitrary one so you can increase > > > it if you want. > > > > > Well, in the case of "ix" Jack Vogel says it is a hardware > > limitation. > > I can't change drivers that I can't test and don't know anything > > about > > the hardware. Maybe replacing m_collapse() with m_defrag() is an > > exception, > > since I know what that is doing and it isn't hardware related, but > > I > > would still prefer a review by the driver author/maintainer before > > making > > such a change. > > > > If there are drivers that you know can be increased from 32->35 > > please do > > so, since that will not only avoid the EFBIG failures but also > > avoid a > > lot of calls to m_defrag(). > > > > > > Further, of these drivers, the following use m_collapse() and > > > > not > > > > m_defrag() > > > > to try and reduce the # of mbufs in the chain. m_collapse() is > > > > not > > > > going to > > > > get the 35 mbufs down to 32 mbufs, as far as I can see, so > > > > these > > > > ones are > > > > more badly broken: > > > > jme, fxp, age, sge, alc, ale, nfe, re > > > > > > I guess m_defeg(9) is more optimized for non-TSO packets. You > > > don't > > > want to waste CPU cycles to copy the full frame to reduce the > > > number of mbufs in the chain. For TSO packets, m_defrag(9) looks > > > better but if we always have to copy a full TSO packet to make > > > TSO > > > work, driver writers have to invent better scheme rather than > > > blindly relying on m_defrag(9), I guess. > > > > > Yes, avoiding m_defrag() calls would be nice. For this issue, > > increasing > > the transmit segment limit from 32->35 does that, if the change can > > be > > done easily/safely. > > > > Otherwise, all I can think of is my suggestion to add something > > like > > if_hw_tsomaxseg which the driver can use to tell tcp_output() the > > driver's limit for # of mbufs in the chain. > > > > > > > > > > The long description is in the above thread, but the short > > > > version > > > > is: > > > > - NFS generates a chain with 35 mbufs in it for (read/readdir > > > > replies and write requests) > > > > made up of (tcpip header, RPC header, NFS args, 32 clusters > > > > of > > > > file data) > > > > - tcp_output() usually trims the data size down to tp->t_tsomax > > > > (65535) and > > > > then some more to make it an exact multiple of TCP transmit > > > > data > > > > size. > > > > - the net driver prepends an ethernet header, growing the > > > > length > > > > by 14 (or > > > > sometimes 18 for vlans), but in the first mbuf and not > > > > adding > > > > one to the chain. > > > > - m_defrag() copies this to a chain of 32 mbuf clusters > > > > (because > > > > the total data > > > > length is <= 64K) and it gets sent > > > > > > > > However, it the data length is a little less than 64K when > > > > passed > > > > to tcp_output() > > > > so that the length including headers is in the range > > > > 65519->65535... > > > > - tcp_output() doesn't reduce its size. > > > > - the net driver adds an ethernet header, making the total > > > > data > > > > length slightly > > > > greater than 64K > > > > - m_defrag() copies it to a chain of 33 mbuf clusters, which > > > > fails with EFBIG > > > > --> trainwrecks NFS performance, because the TSO segment is > > > > dropped > > > > instead of sent. > > > > > > > > A tester also stated that the problem could be reproduced using > > > > iSCSI. Maybe > > > > Edward Napierala might know some details w.r.t. what kind of > > > > mbuf > > > > chain iSCSI > > > > generates? > > > > > > > > Also, one tester has reported that setting if_hw_tsomax in the > > > > driver before > > > > the ether_ifattach() call didn't make the value of tp->t_tsomax > > > > smaller. > > > > However, reducing IP_MAXPACKET (which is what it is set to by > > > > default) did > > > > reduce it. I have no idea why this happens or how to fix it, > > > > but it > > > > implies > > > > that setting if_hw_tsomax in the driver isn't a solution until > > > > this > > > > is resolved. > > > > > > > > So, what to do about this? > > > > First, I'd like a simple fix/workaround that can go into 9.3. > > > > (which is code > > > > freeze in May). The best thing I can think of is setting > > > > if_hw_tsomax to a > > > > smaller default value. (Line# 658 of sys/net/if.c in head.) > > > > > > > > Version A: > > > > replace > > > > ifp->if_hw_tsomax = IP_MAXPACKET; > > > > with > > > > ifp->if_hw_tsomax = min(32 * MCLBYTES - (ETHER_HDR_LEN + > > > > ETHER_VLAN_ENCAP_LEN), > > > > IP_MAXPACKET); > > > > plus > > > > replace m_collapse() with m_defrag() in the drivers listed > > > > above. > > > > > > > > This would only reduce the default from 65535->65518, so it > > > > only > > > > impacts > > > > the uncommon case where the output size (with tcpip header) is > > > > within > > > > this range. (As such, I don't think it would have a negative > > > > impact > > > > for > > > > drivers that handle more than 32 transmit segments.) > > > > From the testers, it seems that this is sufficient to get rid > > > > of > > > > the EFBIG > > > > errors. (The total data length including ethernet header > > > > doesn't > > > > exceed 64K, > > > > so m_defrag() fits it into 32 mbuf clusters.) > > > > > > > > The main downside of this is that there will be a lot of > > > > m_defrag() > > > > calls > > > > being done and they do quite a bit of bcopy()'ng. > > > > > > > > Version B: > > > > replace > > > > ifp->if_hw_tsomax = IP_MAXPACKET; > > > > with > > > > ifp->if_hw_tsomax = min(29 * MCLBYTES, IP_MAXPACKET); > > > > > > > > This one would avoid the m_defrag() calls, but might have a > > > > negative > > > > impact on TSO performance for drivers that can handle 35 > > > > transmit > > > > segments, > > > > since the maximum TSO segment size is reduced by about 6K. > > > > (Because > > > > of the > > > > second size reduction to an exact multiple of TCP transmit data > > > > size, the > > > > exact amount varies.) > > > > > > > > Possible longer term fixes: > > > > One longer term fix might be to add something like > > > > if_hw_tsomaxseg > > > > so that > > > > a driver can set a limit on the number of transmit segments > > > > (mbufs > > > > in chain) > > > > and tcp_output() could use that to limit the size of the TSO > > > > segment, as > > > > required. (I have a first stab at such a patch, but no way to > > > > test > > > > it, so > > > > I can't see that being done by May. Also, it would require > > > > changes > > > > to a lot > > > > of drivers to make it work. I've attached this patch, in case > > > > anyone wants > > > > to work on it?) > > > > > > > > Another might be to increase the size of MCLBYTES (I don't see > > > > this > > > > as > > > > practical for 9.3, although the actual change is simple.). I do > > > > think > > > > that increasing MCLBYTES might be something to consider doing > > > > in > > > > the > > > > future, for reasons beyond fixing this. > > > > > > > > So, what do others think should be done? rick > > > > > > > > > > AFAIK all TSO capable drivers you mentioned above have no limit > > > on > > > the number of TX segments in TSO path. Not sure about Intel > > > controllers though. Increasing the number of segment will > > > consume > > > lots of kernel stack in those drivers. Given that ixgbe, which > > > seems to use 100, didn't show any kernal stack shortage, I think > > > bumping the number of segments would be quick way to address the > > > issue. > > > > > Well, bumping it from 32->35 is all it would take for NFS (can't > > comment > > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the > > 82599 > > (just so others aren't confused by the above comment). I understand > > your point was w.r.t. using 100 without blowing the kernel stack, > > but > > since the testers have been using "ix" with the 82599 chip which is > > limited to 32 transmit segments... > > > > However, please increase any you know can be safely done from > > 32->35, rick > > Done in r263957. > Thanks, rick ps: I've pinged the guys who seem to be maintaining bge, bce, bxe, since they all have the problem, too. From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 02:52:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2983B61; Tue, 1 Apr 2014 02:52:20 +0000 (UTC) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 9D6CC873; Tue, 1 Apr 2014 02:52:20 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id A36511A2373; Tue, 1 Apr 2014 13:52:11 +1100 (EST) Date: Tue, 1 Apr 2014 13:52:10 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alan Somers Subject: Re: netstat -i[d] violates PoLS In-Reply-To: Message-ID: <20140401114356.H878@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=U6SrU4bu c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=Jgj6Na71uFkA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=pH_1NX_M65PnUlo8CmYA:9 a=mL9aW2WkYNHipCf2:21 a=abbXOg4-yxIcHPkX:21 a=CjuIK1q_8ugA:10 Cc: attilio@freebsd.org, FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 02:52:21 -0000 On Mon, 31 Mar 2014, Alan Somers wrote: > "netstat -i" prints dropped output packets iff you also use "-d". > Starting with r199803 on 2009-11-25, "netstat -i" prints dropped input > packets regardless of the "-d" flags. That is a PoLS violation, IMHO. > I think that the "-d" flag should control printing of dropped input > packets as well as dropped output packets. > > OTOH, this behavior has been around for more than 4 years, and some > scripts may rely on it. At the very least, the man page should be > updated to reflect r199803. This also destroyed the output formatting. Please fix other destructions of the output formatting in netstat too. FreeBSD-11 netstat -i: %%% Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll igb0 1500 68:b5:99:b5:2a:02 4189424443 2 0 2499213512 0 0 igb0 - 8.8.178.128/2 freefall 271628427 - - 248798734 - - igb0 - fe80::6ab5:99 fe80::6ab5:99ff:f 182226 - - 182602 - - ... %%% The Idrop column uses space that is not available. Despite using too many columns, the fields are not wide enough to line up. E.g., only 8 columns are available for Ipkts, but 10 are used. The Network and Address fields are also not wide enough. They don't use more columns than are available, but are blindly truncated. FreeBSD-5 netstat -i: %%% Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge0 1500 00:04:76:f3:ac:ad 0 0 5 0 0 ... rl0 1500 122.106.144/2 c122-106-147-133. 674 - 529 - - %%% This gives an example of address truncation even in FreeBSD-5. FreeBSD-11 netstat -id (header only): Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop At least 4 more bugs are visible in this alone: 1. "Drop" is not spelled with an "s". Neither is "Coll". This is to save space. The unbroken format is 79 columns wide. Not a single column is available to consistantly pluralize these, so none were used. This is special to the non-I case. Plurals are used for -I. See below. Using more columns than are available to print Idrops turned this careful formatting into garbage. 2. "Idrop" is not spelled with an "s". This is inconsistent too, but there is more reason for it -- all the short fields have width 5, and keeping them all the same width makes the output easier to read. This leaves no space for pluralization. 3. "Drop" is not spelled with an "O". This together with consistently omitting "s" for IDrop and Drop leaves 1 fewer column under the header available for the numeric value for Drop than for Idrop, so the short fields can't actually all have width 5. 3a. The header only allows 4 columns for "Coll", by 5 are used. This doesn't completely break the formatting since it overlaps the 2-column gap between "Oerrs" and "Coll" in the header. This gap is really too small. It makes it look like "Coll" is associated with output. There is space for pluralization of "Coll" be shrinking the gap further. 3b. The header only allows 4 columns for "Drop". Actually, only 3 were used (preceded by a space). Now, none are used -- "Drop" is not printed at all, and there is an XXXGL comment reminding that they should be printed. Printing the column header without even printing 0's or '-'s under it is negatively useful. Extraction of fields using cut -c doesn't work due to the inconsistent formatting. "Drop" is normally the last field, so omitting its numeric value is not such a large problem. 3c. The above output shows strange printing of numeric values of 0 -- sometimes "0" is printed and sometimes "-" is printed. "-" is harder to post-process. 4. "Drop" is added at the end. If it were actually useful, then it would belong with the output fields, unlike "Coll". Note that what used to be under "Drop" is actually for input, and this was moved to be together with the other input fields. So if there were space for it, then it would not be a bug to print it unconditionally there. If this is fixed by printing it conditionally at the end again, then it needs an "I" in its name, and so would output "Drops" if these were actually counted. FreeBSD-5 netstat -id (header only): Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll Drop There was no space available for "Drop" here too. Perhaps it was intentionally left out. FreeBSD-11 netstat -r: %%% Destination Gateway Flags Netif Expire default router.v108.ysv.fr UGS igb0 8.8.178.128/26 link#1 U igb0 ... Internet6: Destination Gateway Flags Netif Expire ... fe80::6ab5:99ff:fe link#1 UHS lo0 %%% OK, except names are unnecessarily truncated because the fixed format is unnecessarily narrow. FreeBSD-11 netstat -rn: %%% Internet: Destination Gateway Flags Netif Expire default 8.8.178.129 UGS igb0 ... Internet6: Destination Gateway Flags Netif Expire ... 2001:1900:2254:206c::/64 link#1 U igb0 ... ff01::/32 fe80::6ab5:99ff:feb5:2a02 U igb0 %%% Broken. The fixed format is unnecessarily wide for all (?) cases and causes wrap for the "Expire" field. Most "Expire" values are 0, so they don't cause line wrap on every line. FreeBSD-11 netstat -I igb0 1: %%% input igb0 output packets errs idrops bytes packets errs bytes colls 8 0 0 1926 8 0 2345 0 9 0 0 1915 7 0 1921 0 %%% FreeBSD-5 netstat -I bge0 1: %%% input (bge0) output packets errs bytes packets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 %%% Note that everything is pluralized here. Capitalization is inconsistent with that for netstat -i, and worse. The source code has to use separate strings for the field names so as to handle different pluralization and other differences like expanding Ipkts to "input" on 1 line and "packets" on another line. This shows the following regressions: - lost parentheses around the interface name - the interface name and "output" were not moved to the right to adjust for the extra input field - "i" in "idrops" is more inconsistent than for netstat -i, since now it is the only i/o field name with an "i" or an "o" - the extra "i" is not compensated for in the numeric formatting. The numeric values are supposed to be right justified below with their description in the header, but are now off by 1 starting with "idrops". It was very unclear which fields the "input" and "output" headers are over. Now it is even less clear. The interface name used to be centered in the gap between the input and output fields. Now it is over the last input field. This could be improved by not using a separate header for "input" and "output". The abbreviation "I" used for netstat -i is much more readable. Adding -i to netstat -I... doesn't change anything. Adding -d extends the mess only slightly. There is still plenty of space for all the fields. Numeric values for the "drops" field are not available and not printed, as above. Related documentation bugs: - the new Idrops and idrops and the old Drop and Drops are not documented (neither are other field names or field formatting) - -d is still described as being for "dropped packets". It actually gives only the available info for dropped output packets, and that info is null. It used to give the available info for dropped input packets, and that info is not null. Bruce From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 05:26:49 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92EEE1E5D; Tue, 1 Apr 2014 05:26:49 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2607:f2f8:a528::3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id 7C654CC8; Tue, 1 Apr 2014 05:26:49 +0000 (UTC) Received: from [10.0.1.3] (c-24-6-115-18.hsd1.ca.comcast.net [24.6.115.18]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 0F73039827; Mon, 31 Mar 2014 22:26:47 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH 4/6] sfxge: add counter for Tx errors returned from if_transmit From: Rui Paulo In-Reply-To: <53292ED6.5020102@oktetlabs.ru> Date: Mon, 31 Mar 2014 22:26:47 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <98882622-E57D-48C1-8C6E-5B9282C89963@FreeBSD.org> References: <532818D0.4080006@oktetlabs.ru> <20140318125921.GF1499@FreeBSD.org> <53292ED6.5020102@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) Cc: Boris Misenov , net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 05:26:49 -0000 On Mar 18, 2014, at 22:44, Andrew Rybchenko = wrote: > Is the any best practice how to send patches to mailing list? I believe this is not documented anywhere, but it's best to send patches = as attachments. -- Rui Paulo From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 06:54:02 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68787792; Tue, 1 Apr 2014 06:54:02 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D5B907F7; Tue, 1 Apr 2014 06:54:01 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id u57so5572859wes.8 for ; Mon, 31 Mar 2014 23:54:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dCfDsGesrJZQzeMm6LyfjkKLux7KNunXIoRvkC0ycSQ=; b=gNXlhAisOr+0Glk+nS7L9phi2XSMFzNebAejeYVw7ZxZs7SJFYrGdHH6TyW8kfNFZX qzWSLfysqecabgvy3EAe9xdYjRBqU9rV7hQcZwDJrrhe4XcMo8cb4ZfnMGghIutZpHLI 3UHP5KZ4pWX2HAcZwEapLT94UhV8KJ35bpHHG/lyue7SQuMIgD3lFdRv0Mz7oLJ5q4Ds rZhQIJnsvQxGexVhGu0GMK4azMAluxG0mmaDs6mLG3gLc8BJezPHjPvav72feBm87yEx vjWdZFuBUe3LIX2k4OYyu4q5v7/hAWAFw9UWEOThVaZxvpym5STwvNMlPFR8FRLRBG3k 7Krg== MIME-Version: 1.0 X-Received: by 10.180.99.225 with SMTP id et1mr17742820wib.13.1396335240167; Mon, 31 Mar 2014 23:54:00 -0700 (PDT) Received: by 10.14.211.134 with HTTP; Mon, 31 Mar 2014 23:54:00 -0700 (PDT) In-Reply-To: <5338FF05.1020302@sfc.wide.ad.jp> References: <5338FF05.1020302@sfc.wide.ad.jp> Date: Mon, 31 Mar 2014 23:54:00 -0700 Message-ID: Subject: Re: DCTCP implementation From: hiren panchasara To: Midori Kato Content-Type: text/plain; charset=UTF-8 Cc: George Neville-Neil , "Eggert, Lars" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 06:54:02 -0000 On Sun, Mar 30, 2014 at 10:37 PM, Midori Kato wrote: > Hi FreeBSD developpers, > > I'm Midori Kato. I'm working on the DCTCP implementation in the FreeBSD with > Lars Eggert. I mail you because I would like to ask you a code review and > testing. The attached patch is not good enough to test our code. Please give > me your message. I will send an ECN marking implmenetation in dummynet and > test scripts personally to you. Hi Midori, Thank you for your work and I'd like to test this. Please let me know how you setup the dummynet testing cluster and I'll try it. cheers, Hiren ccing gnn@ as he was also asking for the same (testing details). From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 06:58:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB555848; Tue, 1 Apr 2014 06:58:48 +0000 (UTC) Received: from mail-pd0-x236.google.com (mail-pd0-x236.google.com [IPv6:2607:f8b0:400e:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97680819; Tue, 1 Apr 2014 06:58:48 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id y10so9131033pdj.13 for ; Mon, 31 Mar 2014 23:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=qdn/KYoACp8kucYs+CTZ9d+Amk1W4QuXsORwpAfqyOQ=; b=iQhTRHdlnw7E/1oj2J/La2GUCUusfi3buQG3qbkfgy6s0tnVqQxHiUAmvUOgY6iZhd 90gyI7NBZcoEO3MauYXmDp7X0JZW4YJb34BOunUe5o7gSwX36OAn9Ipp0I2XlLP5y/7n iKvVBEh4mgYJnLOIxvP1OHUgVyhjjuRxze3/7BoPGKB4SF9SRBOJ4mFbV/oD5o/TRx8n d+SoQ97uwcPCbx+IqreYW/arAycupsw1PnusmLu9TptPS1+g/RELZ+S7W1lxV89lWkCB Z9SUhSVtrj46/iBx3RHmZ7vtGbok/yDP9zb0dd3BOpyJD0VP552LxhhPBd0uz5H+r4/D hNTw== X-Received: by 10.68.212.10 with SMTP id ng10mr29242884pbc.95.1396335528243; Mon, 31 Mar 2014 23:58:48 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id vg1sm47752672pbc.44.2014.03.31.23.58.45 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 31 Mar 2014 23:58:47 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 01 Apr 2014 15:58:42 +0900 From: Yonghyeon PYUN Date: Tue, 1 Apr 2014 15:58:42 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140401065842.GA1364@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 06:58:48 -0000 On Mon, Mar 31, 2014 at 07:57:28AM -0700, Chris H wrote: > > On Sun, Mar 30, 2014 at 01:12:20PM -0700, chrish@UltimateDNS.NET wrote: > >> Greetings, > >> I'm not sure whether this best belonged on net@, or stable@ > >> so I'm using both. :) > >> I'm testing both releng_9, and MB, and I encountered a new > >> message I don't usually see using the nfe(4) driver: > >> > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> ... > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > >> > >> Truncated for brevity (31 lines in total; 1-31). I don't know > >> how interpret this. An issue with my version of the driver, or > >> the hardware itself? This occurred with both GENERIC, as well > >> as my custom kernel. > > > > Would you show me the dmesg output? > Happily: > > Calibrating TSC clock ... TSC clock: 3231132841 Hz > CPU: AMD Sempron(tm) 140 Processor (3231.13-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0x100f62 Family = 0x10 Model = 0x6 Stepping = 2 > Features=0x78bfbff > Features2=0x802009 > AMD Features=0xee500800 > AMD Features2=0x37fd [...] > nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff > irq 20 at device 7.0 on pci0 > nfe0: attempting to allocate 8 MSI vectors (8 supported) > msi: routing MSI IRQ 257 to local APIC 0 vector 56 > msi: routing MSI IRQ 258 to local APIC 0 vector 57 > msi: routing MSI IRQ 259 to local APIC 0 vector 58 > msi: routing MSI IRQ 260 to local APIC 0 vector 59 > msi: routing MSI IRQ 261 to local APIC 0 vector 60 > msi: routing MSI IRQ 262 to local APIC 0 vector 61 > msi: routing MSI IRQ 263 to local APIC 0 vector 62 > msi: routing MSI IRQ 264 to local APIC 0 vector 63 > nfe0: using IRQs 257-264 for MSI > nfe0: Using 8 MSI messages > miibus0: on nfe0 > rlphy0: PHY 0 on miibus0 > rlphy0: OUI 0x000004, model 0x0020, rev. 1 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy1: PHY 1 on miibus0 > rlphy1: OUI 0x000004, model 0x0020, rev. 1 > rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow [...] > rlphy30: PHY 30 on miibus0 > rlphy30: OUI 0x000004, model 0x0020, rev. 1 > rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy31: PHY 31 on miibus0 > rlphy31: OUI 0x000004, model 0x0020, rev. 1 > rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > nfe0: bpf attached > nfe0: Ethernet address: 40:61:86:cd:44:97 mii(4) thinks it has 32 PHYs and this is the reason why mii(4) complains. Due to unknown reason, accessing PHY registers in device probe stage got valid response which in turn makes the driver think there are 32 PHYs. Did you ever see this this kind of message on old FreeBSD release? Or could you try cold-boot and see whether it makes any difference? From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 07:41:10 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53F115AA for ; Tue, 1 Apr 2014 07:41:10 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F61DC01 for ; Tue, 1 Apr 2014 07:41:09 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.97,770,1389772800"; d="asc'?scan'208";a="113754521" Received: from vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) by mx11-out.netapp.com with ESMTP; 01 Apr 2014 00:40:57 -0700 Received: from SACEXCMBX06-PRD.hq.netapp.com ([169.254.9.198]) by vmwexceht06-prd.hq.netapp.com ([10.106.77.104]) with mapi id 14.03.0123.003; Tue, 1 Apr 2014 00:40:57 -0700 From: "Eggert, Lars" To: Midori Kato Subject: Re: DCTCP implementation Thread-Topic: DCTCP implementation Thread-Index: AQHPTKNLEqFXqRnrckWlFAAW6AoVmJr814AA Date: Tue, 1 Apr 2014 07:40:56 +0000 Message-ID: <8A3033DD-7742-4388-AF04-3E0DB82E0D1A@netapp.com> References: <5338FF05.1020302@sfc.wide.ad.jp> In-Reply-To: <5338FF05.1020302@sfc.wide.ad.jp> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.28] Content-Type: multipart/signed; boundary="Apple-Mail=_286ECCB8-2E2E-4DF7-8F20-E4E10F158E3A"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 07:41:10 -0000 --Apple-Mail=_286ECCB8-2E2E-4DF7-8F20-E4E10F158E3A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, On 2014-3-31, at 7:37, Midori Kato wrote: > I will send an ECN marking implmenetation in dummynet and test = scripts personally to you. I think you can send the dummynet ECN patch also to the list. I know = Luigi is reviewing it for a merge, but that lets people play with it = already. Lars --Apple-Mail=_286ECCB8-2E2E-4DF7-8F20-E4E10F158E3A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBUzptiNZcnpRveo1xAQLFWwP/aXOeHAIpjYbCw3ZiWwWplB1jPQwf42tq hSo4aWfA0zStuw2uSJWccBB8RF20W1fnxiA0Ki57NfP3Z/q4YU1pRubSrQefoht1 qrAe0yULYOSVLC9Z54snJxbdysBRau/a69x1YEbyF0ps/H1mmx52ZfK6yP/A9zx2 7YMOYrA61hg= =uIdJ -----END PGP SIGNATURE----- --Apple-Mail=_286ECCB8-2E2E-4DF7-8F20-E4E10F158E3A-- From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 08:34:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E298975F for ; Tue, 1 Apr 2014 08:34:28 +0000 (UTC) Received: from mail-lb0-f181.google.com (mail-lb0-f181.google.com [209.85.217.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63E66135 for ; Tue, 1 Apr 2014 08:34:27 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id c11so6576839lbj.26 for ; Tue, 01 Apr 2014 01:34:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=qYu7Oq82inL31sS9BMW8NxgIidseezZkCY6/4GaiD0E=; b=lNBhJZcagsZ01feSR9XmwXTCIURYA19x1s1BTKCsyqbk7m33/yLo1xLl2Y+nAYkGwJ rOXh6O6VkuOlQMqMQaWGntPa+tH3oR4RkAKsL66LMhfYhfLzJJXAVa7pmZHX6QSOYnF0 V61eibHjsee4NNrenFK3HWcACHW8TE0/jREdh8bBterWrl9Jn1bvDO1n71Ijx5vDfa6I qsFdZ7TzJGzC6iUdotnOSa0xR2yS2a7mxw2Kl/0BheZeBxh+798LOl26q3xmWTQfHMr6 XLhFA4ePfwBnclHYIOTVvjcJSVBxklhcdxYQnnp295YJr1pEKgxg1dOh+R7re6RrTUVE G7KQ== X-Gm-Message-State: ALoCoQkLmtxJwAYkJQM1uYHJzVSf50f8HVEGkvLxfWUe/fXyWEqXTHsKoq+di1sRn6TrUL7Jz8AK X-Received: by 10.152.2.99 with SMTP id 3mr29912lat.51.1396341259923; Tue, 01 Apr 2014 01:34:19 -0700 (PDT) Received: from grey.office.se.prisjakt.nu ([212.16.170.194]) by mx.google.com with ESMTPSA id x4sm11658111lbn.2.2014.04.01.01.34.19 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 01:34:19 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10-STABLE and CARP states From: mxb In-Reply-To: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> Date: Tue, 1 Apr 2014 10:34:21 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 08:34:29 -0000 PING On 31 mar 2014, at 22:21, mxb wrote: >=20 > Manually setting net.inet.carp.demotion brought BOTH VHIDs in desired = state. > pfsync bulk update seems to not put everything back as it should. >=20 > lagg0: flags=3D8943 = metric 0 mtu 9000 > = options=3D8407bb > ether 00:25:90:e3:71:f2 > inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 > inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 > inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 > inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 > nd6 options=3D29 > media: Ethernet autoselect > status: active > carp: MASTER vhid 201 advbase 1 advskew 1 > carp: BACKUP vhid 202 advbase 5 advskew 100 > laggproto lacp lagghash l2,l3,l4 > laggport: ix1 flags=3D1c > laggport: ix0 flags=3D1c >=20 >=20 > On 31 mar 2014, at 20:42, mxb wrote: >=20 >>=20 >> Hi list, >>=20 >> hopefully this is the right place to have my question regarding CARP = on 10-STABLE. >>=20 >> I have two nodes with following setup(node1): >>=20 >> lagg0: flags=3D8943 = metric 0 mtu 9000 >> = options=3D8407bb >> ether 00:25:90:e3:71:f2 >> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >> nd6 options=3D29 >> media: Ethernet autoselect >> status: active >> carp: BACKUP vhid 201 advbase 1 advskew 1 >> carp: BACKUP vhid 202 advbase 5 advskew 100 >> laggproto lacp lagghash l2,l3,l4 >> laggport: ix1 flags=3D1c >> laggport: ix0 flags=3D1c >>=20 >> net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: >>=20 >> pfsync0: flags=3D41 metric 0 mtu 1500 >> pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off >>=20 >> The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. >> Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as = of setup. >>=20 >> Am I hitting a bug or doing something wrong? >>=20 >> I also have noted that after the pfsync bulk update the demotion = counter never setts to 0, but stays on 480, >> thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? >>=20 >> Regards, >> mxb >>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 12:48:22 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AD73501 for ; Tue, 1 Apr 2014 12:48:22 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59100D96 for ; Tue, 1 Apr 2014 12:48:22 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id y20so9154968ier.27 for ; Tue, 01 Apr 2014 05:48:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=M4NR0s98PeKjoiuB33u8mxd617o2rDlVdbBJ81Te8Og=; b=Sf3bDo+y4cj2OBCO17bBY6VligR8NI26A61cH7mtRyUCyDXPMCb6UKhT6XOglHHSPa h1e8D5uVZzlC0EpowaegObrZi8h6/dDfF9V4SGA6gzliagLv2VQrRq6TGPrp5DXDtoGh YSmzVjsiB/Jg7600ZQIXu+dF16eCuvDen5qT4Bbdzn1bPehGYoAMiGIJC/CSmkZc0+Ih s2K1DEnepIMEeOtrfTNUull/EgX6Ig27At0IJMu710Jp4b4WG7Jsfqu+IxSx/Gmw7mYO VGXuoh2IPCDEpA2Q68WYx2L9ZYMkEiYl+8+Zo9tfHhnmfjN6ioJoUCDhNx8LI50wUZdf GpLg== MIME-Version: 1.0 X-Received: by 10.50.111.135 with SMTP id ii7mr1708696igb.35.1396356501760; Tue, 01 Apr 2014 05:48:21 -0700 (PDT) Sender: vrwmiller@gmail.com Received: by 10.64.235.212 with HTTP; Tue, 1 Apr 2014 05:48:21 -0700 (PDT) In-Reply-To: <5339D0A0.1030903@rcn.com> References: <5339D0A0.1030903@rcn.com> Date: Tue, 1 Apr 2014 08:48:21 -0400 X-Google-Sender-Auth: pbuC-SXa2wRJI35Q2zFiihasOEw Message-ID: Subject: Re: questions about (system) dhclient From: Rick Miller To: Robert Huff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 12:48:22 -0000 On Mon, Mar 31, 2014 at 4:31 PM, Robert Huff wrote: > [Please keep me CC'd as I am not subscribed. Thanks.] > > I have a system, running r263263, where dhclient is misbehaving. > (Yes - this is CURRENT, but I have no reason to believe this inherently a > version-specific issue. I am also on current@, and nothing like this has > been reported.) > The system has an Intel Pro/1000 ethernet card, "em0", connected > to a Arris CM820 cable modem. > This is the relevant portion of dhclient.conf: > > http://users.rcn.com/dhclient.conf > > Upon execution, I get this: > > http://users.rcn.com/roberthuff/dhcp_offer The conversation between a DHCP server and client consists of the initial DHCP DISCOVER request from the client broadcasted to the network to which a DHCP ACK is expected in reply from an available DHCP server. Upon receipt of DHCP ACK, the client sends another DHCP DISCOVER to the server asking for configuration information necessary to initialize networking. The server's response is a DHCP OFFER containing all the information requested by the client. I illustrate this in a sequence diagram describing a PXE workflow for FreeBSD installation at http://hostileadmin.com/images/FreeBSD_PXE_Install_Workflow.gif. The first four steps in the sequence is the workflow's first DHCP conversation. This output suggests only half of the conversation was completed. The client doesn't appear to have sent the second DHCP DISCOVER and thusly did not receive configuration information from the server. Have you ran packet captures to determine whether or not the whole conversation is occurring? -- Take care Rick Miller From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 13:05:07 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED52FB51 for ; Tue, 1 Apr 2014 13:05:06 +0000 (UTC) Received: from cpsmtpb-ews02.kpnxchange.com (cpsmtpb-ews02.kpnxchange.com [213.75.39.5]) by mx1.freebsd.org (Postfix) with ESMTP id 5FE9CF1E for ; Tue, 1 Apr 2014 13:05:05 +0000 (UTC) Received: from cpsps-ews19.kpnxchange.com ([10.94.84.185]) by cpsmtpb-ews02.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Tue, 1 Apr 2014 15:03:52 +0200 Received: from CPSMTPM-CMT105.kpnxchange.com ([195.121.3.21]) by cpsps-ews19.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Tue, 1 Apr 2014 15:03:52 +0200 Received: from donald.offrom.nl ([77.170.60.162]) by CPSMTPM-CMT105.kpnxchange.com with Microsoft SMTPSVC(7.0.6002.18264); Tue, 1 Apr 2014 15:03:52 +0200 Received: from squid (squid.vpn.offrom.nl [10.168.0.72]) by donald.offrom.nl (8.14.7/8.14.7) with ESMTP id s31D3qiE020158; Tue, 1 Apr 2014 15:03:52 +0200 (CEST) (envelope-from willy@vpn.offrom.nl) Received: from willy by squid with local (Exim 4.72) (envelope-from ) id 1WUyLy-0007WI-64; Tue, 01 Apr 2014 15:03:46 +0200 Date: Tue, 1 Apr 2014 15:03:46 +0200 From: Willy Offermans To: Rick Miller Subject: Re: questions about (system) dhclient Message-ID: <20140401130346.GM3604@vpn.offrom.nl> References: <5339D0A0.1030903@rcn.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 01 Apr 2014 13:03:52.0643 (UTC) FILETIME=[D4C40930:01CF4DAA] X-RcptDomain: freebsd.org Cc: Robert Huff , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Willy@Offermans.Rompen.nl List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 13:05:07 -0000 Hello Robert, Rick and FreeBSD friends, On Tue, Apr 01, 2014 at 08:48:21AM -0400, Rick Miller wrote: > On Mon, Mar 31, 2014 at 4:31 PM, Robert Huff wrote: > > > [Please keep me CC'd as I am not subscribed. Thanks.] > > > > I have a system, running r263263, where dhclient is misbehaving. > > (Yes - this is CURRENT, but I have no reason to believe this inherently a > > version-specific issue. I am also on current@, and nothing like this has > > been reported.) > > The system has an Intel Pro/1000 ethernet card, "em0", connected > > to a Arris CM820 cable modem. > > This is the relevant portion of dhclient.conf: > > > > http://users.rcn.com/dhclient.conf > > > > Upon execution, I get this: > > > > http://users.rcn.com/roberthuff/dhcp_offer > > > The conversation between a DHCP server and client consists of the initial > DHCP DISCOVER request from the client broadcasted to the network to which a > DHCP ACK is expected in reply from an available DHCP server. Upon receipt > of DHCP ACK, the client sends another DHCP DISCOVER to the server asking > for configuration information necessary to initialize networking. The > server's response is a DHCP OFFER containing all the information requested > by the client. > > I illustrate this in a sequence diagram describing a PXE workflow for > FreeBSD installation at > http://hostileadmin.com/images/FreeBSD_PXE_Install_Workflow.gif. The first > four steps in the sequence is the workflow's first DHCP conversation. > > This output suggests only half of the conversation was completed. The > client doesn't appear to have sent the second DHCP DISCOVER and thusly did > not receive configuration information from the server. Have you ran packet > captures to determine whether or not the whole conversation is occurring? > > -- > Take care > Rick Miller > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" Does em0 supports TSO? and do you have ipfw running at the same time? run ifconfig em0 and look for TSO! I had some issues with bge0 and TSO. I noticed that both, natd and, most importantly for your problem, dhcpd were increasingly using CPU load when an error occurred by sending an e-mail above a particular size. I figured out via the help of freebsd friends that ipfw and TSO are not compatible. Disabling TSO4 on bge0 solved the issue with sendmail and the CPU load of natd and dhcpd. Maybe you are facing a similar problem. -- Met vriendelijke groeten, With kind regards, Mit freundlichen Gruessen, De jrus wah, Wiel ************************************* W.K. Offermans e-mail: Willy@Offermans.Rompen.nl Powered by .... (__) \\\'',) \/ \ ^ .\._/_) www.FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 14:48:07 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79CB0B59 for ; Tue, 1 Apr 2014 14:48:07 +0000 (UTC) Received: from os-mail-3.tamu.edu (os-mail-3.tamu.edu [165.91.23.217]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.tamu.edu", Issuer "InCommon Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18EDBC92 for ; Tue, 1 Apr 2014 14:48:06 +0000 (UTC) X-TAMU-Auth-ID: daved X-TAMU-SenderIP: 128.194.60.151 X-HAT: SG None, P $RELAY, L incoming_auth X-SRBS: None X-EXTLoop1: 128.194.60.151 X-IronPort-AV: E=Sophos;i="4.97,773,1389765600"; d="scan'208";a="437556213" Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: questions about (system) dhclient From: Dave Duchscher In-Reply-To: Date: Tue, 1 Apr 2014 09:46:58 -0500 Content-Transfer-Encoding: 7bit Message-Id: <628086E4-3832-4221-B33E-9AC9234F3076@tamu.edu> References: <5339D0A0.1030903@rcn.com> To: Rick Miller X-Mailer: Apple Mail (2.1510) Cc: Robert Huff , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 14:48:07 -0000 On Apr 1, 2014, at 7:48 AM, Rick Miller wrote: > On Mon, Mar 31, 2014 at 4:31 PM, Robert Huff wrote: > >> [Please keep me CC'd as I am not subscribed. Thanks.] >> >> I have a system, running r263263, where dhclient is misbehaving. >> (Yes - this is CURRENT, but I have no reason to believe this inherently a >> version-specific issue. I am also on current@, and nothing like this has >> been reported.) >> The system has an Intel Pro/1000 ethernet card, "em0", connected >> to a Arris CM820 cable modem. >> This is the relevant portion of dhclient.conf: >> >> http://users.rcn.com/dhclient.conf >> >> Upon execution, I get this: >> >> http://users.rcn.com/roberthuff/dhcp_offer > > > The conversation between a DHCP server and client consists of the initial > DHCP DISCOVER request from the client broadcasted to the network to which a > DHCP ACK is expected in reply from an available DHCP server. Upon receipt > of DHCP ACK, the client sends another DHCP DISCOVER to the server asking > for configuration information necessary to initialize networking. The > server's response is a DHCP OFFER containing all the information requested > by the client. > > I illustrate this in a sequence diagram describing a PXE workflow for > FreeBSD installation at > http://hostileadmin.com/images/FreeBSD_PXE_Install_Workflow.gif. The first > four steps in the sequence is the workflow's first DHCP conversation. The description of the DHCP packet flow is incorrect. DHCP packet flow is the following for clients requesting a new lease. Diagram below copied from RFC 2131. For clients renewing a lease, only the REQUEST and ACK steps are done. Server Client Server (not selected) (selected) v v v | | | | Begins initialization | | | | | _____________/|\____________ | |/DHCPDISCOVER | DHCPDISCOVER \| | | | Determines | Determines configuration | configuration | | | |\ | ____________/ | | \________ | /DHCPOFFER | | DHCPOFFER\ |/ | | \ | | | Collects replies | | \| | | Selects configuration | | | | | _____________/|\____________ | |/ DHCPREQUEST | DHCPREQUEST\ | | | | | | Commits configuration | | | | | _____________/| | |/ DHCPACK | | | | | Initialization complete | | | | . . . . . . | | | | Graceful shutdown | | | | | |\ ____________ | | | DHCPRELEASE \| | | | | | Discards lease | | | v v v Figure 3: Timeline diagram of messages exchanged between DHCP client and servers when allocating a new network address -- DaveD From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 15:14:08 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85A221EB for ; Tue, 1 Apr 2014 15:14:08 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A5D117C for ; Tue, 1 Apr 2014 15:14:07 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=NotTgrhJ c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=phkJtM7cgHoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=ApzlQX1J-xwAxxfRhQYA:9 a=QEXdDO2ut3YA:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp01.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:3257] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id 72/18-15227-EB7DA335; Tue, 01 Apr 2014 11:14:06 -0400 Message-ID: <533AD7B3.8040208@rcn.com> Date: Tue, 01 Apr 2014 11:13:55 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: Re: questions about (system) dhclient Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Rick Miller , Willy Offermans , Robert Huff X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 15:14:08 -0000 Willy Offermans > Does em0 supports TSO? No: see "http://users.rcn.com/roberthuff/dhc_ifconfig". Or at least it's not enabled. > and do you have ipfw running at the same time? Yes ... but this setup has been running with the same ipfw rules for years. The only change is the version of the system. (Which _might_ make this a problem for current@, but I want to eliminate other possibilities first.) > I figured out via the help of freebsd friends that ipfw and TSO are > not compatible. Disabling TSO4 on bge0 solved the issue with > sendmail Um - what issue with sendmail? I did a packet-dump; greping for "dhcp" produces "http://users.rcn.com/roberthuff/tdump2.txt". (The full dump is 1.7 mbytes, and I don't have a place to make it public.) If there's something else I should be looking for, please let me know. Respectfully Robert Huff From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 15:29:17 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 507CB89E for ; Tue, 1 Apr 2014 15:29:17 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1648E373 for ; Tue, 1 Apr 2014 15:29:16 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=NotTgrhJ c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=phkJtM7cgHoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=HkxQiX07BVne618EYl0A:9 a=QEXdDO2ut3YA:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp01.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:3358] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id A5/CB-15227-B4BDA335; Tue, 01 Apr 2014 11:29:15 -0400 Message-ID: <533ADB41.8000002@rcn.com> Date: Tue, 01 Apr 2014 11:29:05 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org Subject: Re: questions about (system) dhclient Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 15:29:17 -0000 Dave Duchscher writes: > The description of the DHCP packet flow is incorrect. DHCP packet > flow is the following for clients requesting a new lease. Diagram > below copied from RFC 2131. For clients renewing a lease, only > the REQUEST and ACK steps are done. > > Server Client Server > (not selected) (selected) > > v v v > | | | > | Begins initialization | > | | | > | _____________/|\____________ | > |/DHCPDISCOVER | DHCPDISCOVER \| > | | | > Determines | Determines > configuration | configuration > | | | > |\ | ____________/ | > | \________ | /DHCPOFFER | > | DHCPOFFER\ |/ | > | \ | | > | Collects replies | > | \| | > | Selects configuration | > | | | > | _____________/|\____________ | > |/ DHCPREQUEST | DHCPREQUEST\ | > | | | > | | Commits configuration > | | | > | | _____________/| > | |/ DHCPACK | > | | | > | Initialization complete | > | | | > . . . > . . . > | | | > | Graceful shutdown | > | | | > | |\ ____________ | > | | DHCPRELEASE \| > | | | > | | Discards lease > | | | > v v v Synopsis of my (apparent) problem: DISCOVER, OFFER, REQUEST, and ACKNOWLEDGEMENT all happen correctly ... but the information doesn't make it to ifconfig or the routing table. Robert Huff From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 17:22:20 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5498724F for ; Tue, 1 Apr 2014 17:22:20 +0000 (UTC) Received: from mail-ig0-x22a.google.com (mail-ig0-x22a.google.com [IPv6:2607:f8b0:4001:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 207AE1B4 for ; Tue, 1 Apr 2014 17:22:20 +0000 (UTC) Received: by mail-ig0-f170.google.com with SMTP id uq10so4856694igb.3 for ; Tue, 01 Apr 2014 10:22:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=b12urGDlgcCGIcOIz6syj3pfexGU+PG5hA0ZKlcMo0Y=; b=gCGuk5r53qom1U7oo1FksWMVkQZBQlUfPtYML4DlY8jtAooX+IJRmTTHAYpFJ/Mv6l lykCEPLxN1+Ri8JO9xeFWB6loHIj5sga/5vCLb3RBJMyHiFM4VButCl+ZqgN5UtPD82r 3Cbbk5zOHaRGtL0jNlUSg0RN8MPVY8BPc5POyod6itLYoN7wyojwqvGYwqjwQc4gJpdK 0SSHrna52nLB74yl3kuKPsiwvyZWxD63XPSjBVU0aqM6lKB877zSDIXoIfpRnKqTK/p1 XMzOgfH7qYyR36bBOkhWrcx6oY62d9GqF1mtSp4LVXgy2MiXEdTS62F8gwn42PRO4rfG CzTQ== MIME-Version: 1.0 X-Received: by 10.42.139.136 with SMTP id g8mr1680922icu.96.1396372938967; Tue, 01 Apr 2014 10:22:18 -0700 (PDT) Sender: vrwmiller@gmail.com Received: by 10.64.235.212 with HTTP; Tue, 1 Apr 2014 10:22:18 -0700 (PDT) In-Reply-To: <628086E4-3832-4221-B33E-9AC9234F3076@tamu.edu> References: <5339D0A0.1030903@rcn.com> <628086E4-3832-4221-B33E-9AC9234F3076@tamu.edu> Date: Tue, 1 Apr 2014 13:22:18 -0400 X-Google-Sender-Auth: 4_lhz6lMzXnjzpPKefxYqxu8GF8 Message-ID: Subject: Re: questions about (system) dhclient From: Rick Miller To: Dave Duchscher Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Robert Huff , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 17:22:20 -0000 On Tue, Apr 1, 2014 at 10:46 AM, Dave Duchscher wrote: > On Apr 1, 2014, at 7:48 AM, Rick Miller wrote: > > > > > > The conversation between a DHCP server and client consists of the initial > > DHCP DISCOVER request from the client broadcasted to the network to > which a > > DHCP ACK is expected in reply from an available DHCP server. Upon > receipt > > of DHCP ACK, the client sends another DHCP DISCOVER to the server asking > > for configuration information necessary to initialize networking. The > > server's response is a DHCP OFFER containing all the information > requested > > by the client. > > > > I illustrate this in a sequence diagram describing a PXE workflow for > > FreeBSD installation at > > http://hostileadmin.com/images/FreeBSD_PXE_Install_Workflow.gif. The > first > > four steps in the sequence is the workflow's first DHCP conversation. > > > The description of the DHCP packet flow is incorrect. DHCP packet flow > is the following for clients requesting a new lease. Diagram below > copied from RFC 2131. For clients renewing a lease, only the REQUEST and > ACK steps are done. > Thanks, Dave...I did mix up the terminology. -- Take care Rick Miller From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 20:16:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 98C4A38C; Tue, 1 Apr 2014 20:16:28 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9C3E6D2; Tue, 1 Apr 2014 20:16:27 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s31KJHNv037788; Tue, 1 Apr 2014 13:19:23 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s31KJCUY037784; Tue, 1 Apr 2014 13:19:12 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Tue, 1 Apr 2014 13:19:12 -0700 (PDT) Message-ID: In-Reply-To: <20140401065842.GA1364@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> Date: Tue, 1 Apr 2014 13:19:12 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 20:16:28 -0000 > On Mon, Mar 31, 2014 at 07:57:28AM -0700, Chris H wrote: >> > On Sun, Mar 30, 2014 at 01:12:20PM -0700, chrish@UltimateDNS.NET wrote: >> >> Greetings, >> >> I'm not sure whether this best belonged on net@, or stable@ >> >> so I'm using both. :) >> >> I'm testing both releng_9, and MB, and I encountered a new >> >> message I don't usually see using the nfe(4) driver: >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> ... >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >> >> >> >> Truncated for brevity (31 lines in total; 1-31). I don't know >> >> how interpret this. An issue with my version of the driver, or >> >> the hardware itself? This occurred with both GENERIC, as well >> >> as my custom kernel. >> > >> > Would you show me the dmesg output? >> Happily: >> >> Calibrating TSC clock ... TSC clock: 3231132841 Hz >> CPU: AMD Sempron(tm) 140 Processor (3231.13-MHz K8-class CPU) >> Origin = "AuthenticAMD" Id = 0x100f62 Family = 0x10 Model = 0x6 Stepping = 2 >> Features=0x78bfbff >> Features2=0x802009 >> AMD Features=0xee500800 >> AMD Features2=0x37fd > > [...] > >> nfe0: port 0xe480-0xe487 mem >> 0xdff7d000-0xdff7dfff >> irq 20 at device 7.0 on pci0 >> nfe0: attempting to allocate 8 MSI vectors (8 supported) >> msi: routing MSI IRQ 257 to local APIC 0 vector 56 >> msi: routing MSI IRQ 258 to local APIC 0 vector 57 >> msi: routing MSI IRQ 259 to local APIC 0 vector 58 >> msi: routing MSI IRQ 260 to local APIC 0 vector 59 >> msi: routing MSI IRQ 261 to local APIC 0 vector 60 >> msi: routing MSI IRQ 262 to local APIC 0 vector 61 >> msi: routing MSI IRQ 263 to local APIC 0 vector 62 >> msi: routing MSI IRQ 264 to local APIC 0 vector 63 >> nfe0: using IRQs 257-264 for MSI >> nfe0: Using 8 MSI messages >> miibus0: on nfe0 >> rlphy0: PHY 0 on miibus0 >> rlphy0: OUI 0x000004, model 0x0020, rev. 1 >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> rlphy1: PHY 1 on miibus0 >> rlphy1: OUI 0x000004, model 0x0020, rev. 1 >> rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > > [...] > >> rlphy30: PHY 30 on miibus0 >> rlphy30: OUI 0x000004, model 0x0020, rev. 1 >> rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> rlphy31: PHY 31 on miibus0 >> rlphy31: OUI 0x000004, model 0x0020, rev. 1 >> rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> nfe0: bpf attached >> nfe0: Ethernet address: 40:61:86:cd:44:97 > > mii(4) thinks it has 32 PHYs and this is the reason why mii(4) > complains. Due to unknown reason, accessing PHY registers in > device probe stage got valid response which in turn makes the > driver think there are 32 PHYs. Did you ever see this this kind of > message on old FreeBSD release? Or could you try cold-boot and see > whether it makes any difference? Greetings, and thank you for taking the time to respond. I downloaded, and booted to the 8.4-memstick. Droped to FIXIT, and captured the dmesg(8) output. I don't know how much of it you need, so I am including it all: Table 'FACP' at 0x6ffa0200 Table 'APIC' at 0x6ffa0390 APIC: Found table at 0x6ffa0390 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 1: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 129 ACPI ID 2: disabled MADT: Found CPU APIC ID 130 ACPI ID 3: disabled MADT: Found CPU APIC ID 131 ACPI ID 4: disabled MADT: Found CPU APIC ID 132 ACPI ID 5: disabled MADT: Found CPU APIC ID 133 ACPI ID 6: disabled Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.4-RELEASE #0 r251259: Sun Jun 2 21:26:57 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 gcc version 4.2.1 20070831 patched [FreeBSD] Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff8146d000. Preloaded mfs_root "/boot/mfsroot" at 0xffffffff8146d200. Timecounter "i8254" frequency 1193182 Hz quality 0 Calibrating TSC clock ... TSC clock: 3231096596 Hz CPU: AMD Sempron(tm) 140 Processor (3231.10-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f62 Family = 10 Model = 6 Stepping = 2 Features=0x78bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x37fd TSC: P-state invariant L1 2MB data TLB: 48 entries, fully associative L1 2MB instruction TLB: 16 entries, fully associative L1 4KB data TLB: 48 entries, fully associative L1 4KB instruction TLB: 32 entries, fully associative L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L2 2MB data TLB: 128 entries, 2-way associative L2 2MB instruction TLB: 0 entries, 2-way associative L2 4KB data TLB: 512 entries, 4-way associative L2 4KB instruction TLB: 512 entries, 4-way associative L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative real memory = 2147483648 (2048 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) 0x000000000149a000 - 0x000000006cac6fff, 1801637888 bytes (439853 pages) avail memory = 1787707392 (1704 MB) ACPI APIC Table: <7309MS A7309200> x86bios: IVT 0x000000-0x0004ff at 0xffffff0000000000 x86bios: SSEG 0x010000-0x01ffff at 0xffffff8000006000 x86bios: EBDA 0x09f000-0x09ffff at 0xffffff000009f000 x86bios: ROM 0x0a0000-0x0effff at 0xffffff00000a0000 APIC: CPU 0 has ACPI ID 1 ULE: setup cpu 0 ACPI: RSDP 0xfaab0 00014 (v00 ACPIAM) ACPI: RSDT 0x6ffa0000 00048 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: FACP 0x6ffa0200 00084 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: DSDT 0x6ffa04b0 0503B (v01 A7309 A7309200 00000200 INTL 20051117) ACPI: FACS 0x6ffae000 00040 ACPI: APIC 0x6ffa0390 00090 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: MCFG 0x6ffa0420 0003C (v01 7309MS OEMMCFG 20101122 MSFT 00000097) ACPI: WDRT 0x6ffa0460 00047 (v01 7309MS NV-WDRT 20101122 MSFT 00000097) ACPI: OEMB 0x6ffae040 00072 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: SRAT 0x6ffaa4b0 00090 (v03 AMD FAM_F_10 00000002 AMD 00000001) ACPI: MSCT 0x6ffaa540 0004E (v01 A M I OEMBOARD 00000001 AMD 00000001) ACPI: HPET 0x6ffaa590 00038 (v01 7309MS OEMHPET0 20101122 MSFT 00000097) ACPI: SSDT 0x6ffaa5d0 0023E (v01 A M I POWERNOW 00000001 AMD 00000001) MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level MADT: Interrupt override: source 14, irq 14 MADT: Interrupt override: source 15, irq 15 ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 wlan: <802.11 Link Layer> random: nfslock: pseudo-device kbd: new array size 4 kbd1 at kbdmux0 mem: io: null: hpt27xx: RocketRAID 27xx controller driver v1.0 hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2 acpi0: <7309MS A7309200> on motherboard PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 acpi0: [MPSAFE] acpi0: [ITHREAD] ACPI: Executed 1 blocks of module-level executable AML code acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 7ff00000 (3) failed ACPI timer: 0/3 1/2 0/3 1/2 1/2 1/2 0/3 1/2 1/2 0/3 -> 6 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 cpu0: switching to generic Cx mode ACPI: Processor \\_PR_.P002 (ACPI ID 2) ignored ACPI: Processor \\_PR_.P003 (ACPI ID 3) ignored ACPI: Processor \\_PR_.P004 (ACPI ID 4) ignored ACPI: Processor \\_PR_.P005 (ACPI ID 5) ignored ACPI: Processor \\_PR_.P006 (ACPI ID 6) ignored pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link8: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link9: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link10: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link11: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link12: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link13: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link14: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link15: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link16: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link17: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x10de, dev=0x03ea, revid=0xa1 domain=0, bus=0, slot=0, func=0 class=05-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03e0, revid=0xa2 domain=0, bus=0, slot=1, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) map[10]: type I/O Port, range 32, base 0x4f00, size 8, enabled found-> vendor=0x10de, dev=0x03eb, revid=0xa2 domain=0, bus=0, slot=1, func=1 class=0c-05-00, hdrtype=0x00, mfdev=1 cmdreg=0x0001, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0x4900, size 6, enabled map[20]: type I/O Port, range 32, base 0x4d00, size 6, enabled map[24]: type I/O Port, range 32, base 0x4e00, size 6, enabled pcib0: matched entry for 0.1.INTA (src \\_SB_.LSMB:0) pci_link13: Picked IRQ 20 with weight 0 pcib0: slot 1 INTA routed to irq 20 via \\_SB_.LSMB found-> vendor=0x10de, dev=0x03f5, revid=0xa2 domain=0, bus=0, slot=1, func=2 class=05-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0400, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03f1, revid=0xa3 domain=0, bus=0, slot=2, func=0 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7f000, size 12, enabled pcib0: matched entry for 0.2.INTA (src \\_SB_.LUB0:0) pci_link8: Picked IRQ 21 with weight 0 pcib0: slot 2 INTA routed to irq 21 via \\_SB_.LUB0 unknown: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xdff7f000 ohci early: SMM active, request owner change found-> vendor=0x10de, dev=0x03f2, revid=0xa3 domain=0, bus=0, slot=2, func=1 class=0c-03-20, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7ec00, size 8, enabled pcib0: matched entry for 0.2.INTB (src \\_SB_.LUB2:0) pci_link9: Picked IRQ 22 with weight 0 pcib0: slot 2 INTB routed to irq 22 via \\_SB_.LUB2 unknown: Reserved 0x100 bytes for rid 0x10 type 3 at 0xdff7ec00 found-> vendor=0x10de, dev=0x03f3, revid=0xa1 domain=0, bus=0, slot=4, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x02 (500 ns) found-> vendor=0x10de, dev=0x03f0, revid=0xa2 domain=0, bus=0, slot=5, func=0 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x05 (1250 ns) intpin=b, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff78000, size 14, enabled pcib0: matched entry for 0.5.INTB (src \\_SB_.LAZA:0) pci_link11: Picked IRQ 23 with weight 0 pcib0: slot 5 INTB routed to irq 23 via \\_SB_.LAZA found-> vendor=0x10de, dev=0x03ec, revid=0xa2 domain=0, bus=0, slot=6, func=0 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) powerspec 2 supports D0 D3 current D0 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled found-> vendor=0x10de, dev=0x03ef, revid=0xa2 domain=0, bus=0, slot=7, func=0 class=06-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x01 (250 ns), maxlat=0x14 (5000 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 8 messages, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff7d000, size 12, enabled map[14]: type I/O Port, range 32, base 0xe480, size 3, enabled pcib0: matched entry for 0.7.INTA (src \\_SB_.LMAC:0) pci_link10: Picked IRQ 20 with weight 1 pcib0: slot 7 INTA routed to irq 20 via \\_SB_.LMAC found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=0 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xe400, size 3, enabled map[14]: type I/O Port, range 32, base 0xe080, size 2, enabled map[18]: type I/O Port, range 32, base 0xe000, size 3, enabled map[1c]: type I/O Port, range 32, base 0xdc00, size 2, enabled map[20]: type I/O Port, range 32, base 0xd880, size 4, enabled map[24]: type Memory, range 32, base 0xdff7c000, size 12, enabled pcib0: matched entry for 0.8.INTA (src \\_SB_.LSA0:0) pci_link15: Picked IRQ 21 with weight 1 pcib0: slot 8 INTA routed to irq 21 via \\_SB_.LSA0 found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=1 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xd800, size 3, enabled map[14]: type I/O Port, range 32, base 0xd480, size 2, enabled map[18]: type I/O Port, range 32, base 0xd400, size 3, enabled map[1c]: type I/O Port, range 32, base 0xd080, size 2, enabled map[20]: type I/O Port, range 32, base 0xd000, size 4, enabled map[24]: type Memory, range 32, base 0xdff77000, size 12, enabled pcib0: matched entry for 0.8.INTB (src \\_SB_.LSA1:0) pci_link16: Picked IRQ 22 with weight 1 pcib0: slot 8 INTB routed to irq 22 via \\_SB_.LSA1 found-> vendor=0x10de, dev=0x03e8, revid=0xa2 domain=0, bus=0, slot=9, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=11, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=12, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03d0, revid=0xa2 domain=0, bus=0, slot=13, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xde000000, size 24, enabled map[14]: type Prefetchable Memory, range 64, base 0xc0000000, size 28, enabled map[1c]: type Memory, range 64, base 0xdd000000, size 24, enabled pcib0: matched entry for 0.13.INTA (src \\_SB_.LMC9:0) pci_link12: Picked IRQ 23 with weight 1 pcib0: slot 13 INTA routed to irq 23 via \\_SB_.LMC9 found-> vendor=0x1022, dev=0x1200, revid=0x00 domain=0, bus=0, slot=24, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1201, revid=0x00 domain=0, bus=0, slot=24, func=1 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1202, revid=0x00 domain=0, bus=0, slot=24, func=2 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1203, revid=0x00 domain=0, bus=0, slot=24, func=3 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1204, revid=0x00 domain=0, bus=0, slot=24, func=4 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) pci0: at device 0.0 (no driver attached) isab0: port 0x4f00-0x4fff at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) pci0: at device 1.2 (no driver attached) ohci0: mem 0xdff7f000-0xdff7ffff irq 21 at device 2.0 on pci0 ioapic0: routing intpin 21 (PCI IRQ 21) to lapic 0 vector 49 ohci0: [MPSAFE] ohci0: [ITHREAD] usbus0 on ohci0 usbus0: bpf attached ohci0: usbpf: Attached ehci0: mem 0xdff7ec00-0xdff7ecff irq 22 at device 2.1 on pci0 ioapic0: routing intpin 22 (PCI IRQ 22) to lapic 0 vector 50 ehci0: [MPSAFE] ehci0: [ITHREAD] ehci0: Doorbell workaround enabled usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: bpf attached ehci0: usbpf: Attached pcib1: at device 4.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: no prefetched decode pcib1: Subtractively decoded bridge. pci1: on pcib1 pci1: domain=0, physical bus=1 pci0: at device 5.0 (no driver attached) atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 6.0 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xffa0 ata0: at channel 0 on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0 atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6 ata0: reset tp1 mask=03 ostat0=50 ostat1=50 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=50 devices=0x3 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 51 ata0: [MPSAFE] ata0: [ITHREAD] ata1: at channel 1 on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170 atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376 ata1: reset tp1 mask=00 ostat0=ff ostat1=ff ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 52 ata1: [MPSAFE] ata1: [ITHREAD] nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff irq 20 at device 7.0 on pci0 nfe0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xdff7d000 nfe0: attempting to allocate 8 MSI vectors (8 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 56 msi: routing MSI IRQ 257 to local APIC 0 vector 57 msi: routing MSI IRQ 258 to local APIC 0 vector 58 msi: routing MSI IRQ 259 to local APIC 0 vector 59 msi: routing MSI IRQ 260 to local APIC 0 vector 60 msi: routing MSI IRQ 261 to local APIC 0 vector 61 msi: routing MSI IRQ 262 to local APIC 0 vector 62 msi: routing MSI IRQ 263 to local APIC 0 vector 63 nfe0: using IRQs 256-263 for MSI nfe0: Using 8 MSI messages miibus0: on nfe0 rlphy0: PHY 0 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy1: PHY 1 on miibus0 rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy2: PHY 2 on miibus0 rlphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy3: PHY 3 on miibus0 rlphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy4: PHY 4 on miibus0 rlphy4: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy5: PHY 5 on miibus0 rlphy5: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy6: PHY 6 on miibus0 rlphy6: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy7: PHY 7 on miibus0 rlphy7: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy8: PHY 8 on miibus0 rlphy8: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy9: PHY 9 on miibus0 rlphy9: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy10: PHY 10 on miibus0 rlphy10: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy11: PHY 11 on miibus0 rlphy11: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy12: PHY 12 on miibus0 rlphy12: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy13: PHY 13 on miibus0 rlphy13: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy14: PHY 14 on miibus0 rlphy14: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy15: PHY 15 on miibus0 rlphy15: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy16: PHY 16 on miibus0 rlphy16: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy17: PHY 17 on miibus0 rlphy17: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy18: PHY 18 on miibus0 rlphy18: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy19: PHY 19 on miibus0 rlphy19: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy20: PHY 20 on miibus0 rlphy20: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy21: PHY 21 on miibus0 rlphy21: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy22: PHY 22 on miibus0 rlphy22: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy23: PHY 23 on miibus0 rlphy23: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy24: PHY 24 on miibus0 rlphy24: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy25: PHY 25 on miibus0 rlphy25: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy26: PHY 26 on miibus0 rlphy26: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy27: PHY 27 on miibus0 rlphy27: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy28: PHY 28 on miibus0 rlphy28: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy29: PHY 29 on miibus0 rlphy29: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy30: PHY 30 on miibus0 rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow rlphy31: PHY 31 on miibus0 rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow nfe0: bpf attached nfe0: Ethernet address: 40:61:86:cd:44:97 nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] nfe0: [MPSAFE] nfe0: [FILTER] atapci1: port 0xe400-0xe407,0xe080-0xe083,0xe000-0xe007,0xdc00-0xdc03,0xd880-0xd88f mem 0xdff7c000-0xdff7cfff irq 21 at device 8.0 on pci0 atapci1: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd880 atapci1: [MPSAFE] atapci1: [ITHREAD] atapci1: Reserved 0x1000 bytes for rid 0x24 type 3 at 0xdff7c000 ata2: at channel 0 on atapci1 atapci1: Reserved 0x8 bytes for rid 0x10 type 4 at 0xe400 atapci1: Reserved 0x4 bytes for rid 0x14 type 4 at 0xe080 ata2: SATA connect timeout status=00000000 ata2: [MPSAFE] ata2: [ITHREAD] ata3: at channel 1 on atapci1 atapci1: Reserved 0x8 bytes for rid 0x18 type 4 at 0xe000 atapci1: Reserved 0x4 bytes for rid 0x1c type 4 at 0xdc00 ata3: SATA connect timeout status=00000000 ata3: [MPSAFE] ata3: [ITHREAD] atapci2: port 0xd800-0xd807,0xd480-0xd483,0xd400-0xd407,0xd080-0xd083,0xd000-0xd00f mem 0xdff77000-0xdff77fff irq 22 at device 8.1 on pci0 atapci2: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd000 atapci2: [MPSAFE] atapci2: [ITHREAD] atapci2: Reserved 0x1000 bytes for rid 0x24 type 3 at 0xdff77000 ata4: at channel 0 on atapci2 atapci2: Reserved 0x8 bytes for rid 0x10 type 4 at 0xd800 atapci2: Reserved 0x4 bytes for rid 0x14 type 4 at 0xd480 ata4: SATA connect timeout status=00000000 ata4: [MPSAFE] ata4: [ITHREAD] ata5: at channel 1 on atapci2 atapci2: Reserved 0x8 bytes for rid 0x18 type 4 at 0xd400 atapci2: Reserved 0x4 bytes for rid 0x1c type 4 at 0xd080 ata5: SATA connect timeout status=00000000 ata5: [MPSAFE] ata5: [ITHREAD] pcib2: at device 9.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pcib2: no prefetched decode pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: at device 11.0 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pcib3: no prefetched decode pci3: on pcib3 pci3: domain=0, physical bus=3 pcib4: at device 12.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: no prefetched decode pci4: on pcib4 pci4: domain=0, physical bus=4 vgapci0: mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff,0xdd000000-0xddffffff irq 23 at device 13.0 on pci0 acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us) uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 53 uart0: [FILTER] uart0: fast interrupt ppc0: using extended I/O port range ppc0: SPP ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ioapic0: routing intpin 7 (ISA IRQ 7) to lapic 0 vector 54 ppc0: [MPSAFE] ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: bpf attached plip0: [MPSAFE] plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [MPSAFE] lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 acpi_hpet0: iomem 0xfeff0000-0xfeff0fff on acpi0 acpi_hpet0: vend: 0x10de rev: 0x1 num: 3 hz: 25000000 opts: legacy_route Timecounter "HPET" frequency 25000000 Hz quality 900 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x83ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 55 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 64 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons psm0: config:00000000, flags:00000008, packet size:4 psm0: syncmask:08, syncbits:00 acpi0: wakeup code va 0xffffff80001b1000 pa 0x4000 ex_isa_identify() ahc_isa_probe 0: ioport 0xc00 alloc failed ahc_isa_probe 1: ioport 0x1c00 alloc failed ahc_isa_probe 2: ioport 0x2c00 alloc failed ahc_isa_probe 3: ioport 0x3c00 alloc failed ahc_isa_probe 4: ioport 0x4c00 alloc failed ahc_isa_probe 5: ioport 0x5c00 alloc failed ahc_isa_probe 6: ioport 0x6c00 alloc failed ahc_isa_probe 7: ioport 0x7c00 alloc failed ahc_isa_probe 8: ioport 0x8c00 alloc failed ahc_isa_probe 9: ioport 0x9c00 alloc failed ahc_isa_probe 10: ioport 0xac00 alloc failed ahc_isa_probe 11: ioport 0xbc00 alloc failed ahc_isa_probe 12: ioport 0xcc00 alloc failed ahc_isa_probe 13: ioport 0xdc00 alloc failed ahc_isa_probe 14: ioport 0xec00 alloc failed isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it ppc: ppc0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices hwpstate0: on cpu0 Device configuration finished. procfs registered lapic: Divisor 2, Frequency 119670248 Hz Timecounter "TSC" frequency 3231096596 Hz quality 800 Timecounters tick every 1.000 msec vlan: initialized, using hash tables with chaining lo0: bpf attached hpt27xx: no controller detected. hptrr: no controller detected. ata0: Identifying devices: 00000003 ata0: New devices: 00000003 md0: Preloaded image 4194304 bytes at 0xffffffff8106b700 usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 ata0-slave: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire ad0: setting UDMA100 ad0: 19541MB at ata0-master UDMA100 ad0: 40020624 sectors [39703C/16H/63S] 16 sectors/interrupt 1 depth queue GEOM: new disk ad0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 uhub0: 10 ports with 10 removable, self powered ad0: nVidia check1 failed ad0: Adaptec check1 failed ad0: LSI (v3) check1 failed ad0: LSI (v2) check1 failed ad0: FreeBSD check1 failed ad1: setting UDMA100 ad1: 39266MB at ata0-slave UDMA100 ad1: 80418240 sectors [79780C/16H/63S] 16 sectors/interrupt 1 depth queue ad1: nVidia check1 failed ad1: Adaptec check1 failed ad1: LSI (v3) check1 failed ad1: LSI (v2) check1 failed ad1: FreeBSD check1 failed ata1: Identifying devices: 00000000 ata1: New devices: 00000000 ata2: Identifying devices: 00000000 ata2: New devices: 00000000 ata3: Identifying devices: 00000000 ata3: New devices: 00000000 ata4: Identifying devices: 00000000 ata4: New devices: 00000000 ata5: Identifying devices: 00000000 ata5: New devices: 00000000 ATA PseudoRAID loaded GEOM: new disk ad1 Root mount waiting for: usbus1 Root mount waiting for: usbus1 Root mount waiting for: usbus1 uhub1: 10 ports with 10 removable, self powered Root mount waiting for: usbus1 Root mount waiting for: usbus1 ugen1.2: at usbus1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:0:0:-1: Attached to scbus0 Trying to mount root from ufs:/dev/md0 pass0 at umass-sim0 bus 0 scbus0 target 0 lun 0 pass0: Removable Direct Access SCSI-4 device pass0: Serial Number AA28014000000354 pass0: 40.000MB/s transfers GEOM: new disk da0 start_init: trying /sbin/initda0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: Removable Direct Access SCSI-4 device da0: Serial Number AA28014000000354 da0: 40.000MB/s transfers da0: 15600MB (31950720 512 byte sectors: 255H 63S/T 1988C) start_init: trying /sbin/oinit start_init: trying /sbin/init.bak start_init: trying /rescue/init start_init: trying /stand/sysinstall GEOM: da0: media size does not match label. miibus0: mii_mediachg: can't handle non-zero PHY instance 31 miibus0: mii_mediachg: can't handle non-zero PHY instance 30 miibus0: mii_mediachg: can't handle non-zero PHY instance 29 miibus0: mii_mediachg: can't handle non-zero PHY instance 28 miibus0: mii_mediachg: can't handle non-zero PHY instance 27 miibus0: mii_mediachg: can't handle non-zero PHY instance 26 miibus0: mii_mediachg: can't handle non-zero PHY instance 25 miibus0: mii_mediachg: can't handle non-zero PHY instance 24 miibus0: mii_mediachg: can't handle non-zero PHY instance 23 miibus0: mii_mediachg: can't handle non-zero PHY instance 22 miibus0: mii_mediachg: can't handle non-zero PHY instance 21 miibus0: mii_mediachg: can't handle non-zero PHY instance 20 miibus0: mii_mediachg: can't handle non-zero PHY instance 19 miibus0: mii_mediachg: can't handle non-zero PHY instance 18 miibus0: mii_mediachg: can't handle non-zero PHY instance 17 miibus0: mii_mediachg: can't handle non-zero PHY instance 16 miibus0: mii_mediachg: can't handle non-zero PHY instance 15 miibus0: mii_mediachg: can't handle non-zero PHY instance 14 miibus0: mii_mediachg: can't handle non-zero PHY instance 13 miibus0: mii_mediachg: can't handle non-zero PHY instance 12 miibus0: mii_mediachg: can't handle non-zero PHY instance 11 miibus0: mii_mediachg: can't handle non-zero PHY instance 10 miibus0: mii_mediachg: can't handle non-zero PHY instance 9 miibus0: mii_mediachg: can't handle non-zero PHY instance 8 miibus0: mii_mediachg: can't handle non-zero PHY instance 7 miibus0: mii_mediachg: can't handle non-zero PHY instance 6 miibus0: mii_mediachg: can't handle non-zero PHY instance 5 miibus0: mii_mediachg: can't handle non-zero PHY instance 4 miibus0: mii_mediachg: can't handle non-zero PHY instance 3 miibus0: mii_mediachg: can't handle non-zero PHY instance 2 miibus0: mii_mediachg: can't handle non-zero PHY instance 1 As you can see, it looks much the same. I have no idea what I should do to better inform the driver/kernel how to better handle it. Or is it the driver, itself? Thank you again, for your thoughtful response. --Chris > From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 20:29:37 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 804F4B6C; Tue, 1 Apr 2014 20:29:37 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E63287B; Tue, 1 Apr 2014 20:29:37 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WV5JK-000Bid-3K; Tue, 01 Apr 2014 20:29:30 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s31KTR6o084889; Tue, 1 Apr 2014 14:29:27 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+1Rzk96lbXP759T3jbeyTp Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: Ian Lepore To: Chris H In-Reply-To: References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> Content-Type: text/plain; charset="us-ascii" Date: Tue, 01 Apr 2014 14:29:27 -0600 Message-ID: <1396384167.81853.210.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 20:29:37 -0000 On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > [...] > miibus0: on nfe0 > rlphy0: PHY 0 on miibus0 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy1: PHY 1 on miibus0 > rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy2: PHY 2 on miibus0 > rlphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy3: PHY 3 on miibus0 > rlphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy4: PHY 4 on miibus0 > rlphy4: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy5: PHY 5 on miibus0 > rlphy5: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy6: PHY 6 on miibus0 > rlphy6: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy7: PHY 7 on miibus0 > rlphy7: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy8: PHY 8 on miibus0 > rlphy8: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy9: PHY 9 on miibus0 > rlphy9: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy10: PHY 10 on miibus0 > rlphy10: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy11: PHY 11 on miibus0 > rlphy11: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy12: PHY 12 on miibus0 > rlphy12: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy13: PHY 13 on miibus0 > rlphy13: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy14: PHY 14 on miibus0 > rlphy14: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy15: PHY 15 on miibus0 > rlphy15: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy16: PHY 16 on miibus0 > rlphy16: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy17: PHY 17 on miibus0 > rlphy17: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy18: PHY 18 on miibus0 > rlphy18: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy19: PHY 19 on miibus0 > rlphy19: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy20: PHY 20 on miibus0 > rlphy20: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy21: PHY 21 on miibus0 > rlphy21: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy22: PHY 22 on miibus0 > rlphy22: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy23: PHY 23 on miibus0 > rlphy23: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy24: PHY 24 on miibus0 > rlphy24: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy25: PHY 25 on miibus0 > rlphy25: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy26: PHY 26 on miibus0 > rlphy26: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy27: PHY 27 on miibus0 > rlphy27: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy28: PHY 28 on miibus0 > rlphy28: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy29: PHY 29 on miibus0 > rlphy29: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy30: PHY 30 on miibus0 > rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > rlphy31: PHY 31 on miibus0 > rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow [...] > miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > miibus0: mii_mediachg: can't handle non-zero PHY instance 30 > miibus0: mii_mediachg: can't handle non-zero PHY instance 29 > miibus0: mii_mediachg: can't handle non-zero PHY instance 28 > miibus0: mii_mediachg: can't handle non-zero PHY instance 27 > miibus0: mii_mediachg: can't handle non-zero PHY instance 26 > miibus0: mii_mediachg: can't handle non-zero PHY instance 25 > miibus0: mii_mediachg: can't handle non-zero PHY instance 24 > miibus0: mii_mediachg: can't handle non-zero PHY instance 23 > miibus0: mii_mediachg: can't handle non-zero PHY instance 22 > miibus0: mii_mediachg: can't handle non-zero PHY instance 21 > miibus0: mii_mediachg: can't handle non-zero PHY instance 20 > miibus0: mii_mediachg: can't handle non-zero PHY instance 19 > miibus0: mii_mediachg: can't handle non-zero PHY instance 18 > miibus0: mii_mediachg: can't handle non-zero PHY instance 17 > miibus0: mii_mediachg: can't handle non-zero PHY instance 16 > miibus0: mii_mediachg: can't handle non-zero PHY instance 15 > miibus0: mii_mediachg: can't handle non-zero PHY instance 14 > miibus0: mii_mediachg: can't handle non-zero PHY instance 13 > miibus0: mii_mediachg: can't handle non-zero PHY instance 12 > miibus0: mii_mediachg: can't handle non-zero PHY instance 11 > miibus0: mii_mediachg: can't handle non-zero PHY instance 10 > miibus0: mii_mediachg: can't handle non-zero PHY instance 9 > miibus0: mii_mediachg: can't handle non-zero PHY instance 8 > miibus0: mii_mediachg: can't handle non-zero PHY instance 7 > miibus0: mii_mediachg: can't handle non-zero PHY instance 6 > miibus0: mii_mediachg: can't handle non-zero PHY instance 5 > miibus0: mii_mediachg: can't handle non-zero PHY instance 4 > miibus0: mii_mediachg: can't handle non-zero PHY instance 3 > miibus0: mii_mediachg: can't handle non-zero PHY instance 2 > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > > As you can see, it looks much the same. I have no idea what > I should do to better inform the driver/kernel how to better > handle it. Or is it the driver, itself? > > Thank you again, for your thoughtful response. > > --Chris > I think the way to fix a phy that responds at all addresses is to set a hint in loader.conf masking out the ones that aren't real, like so: hint.miibus.0.phymask="1" You might be able to set ="0x00000001" to make it more clear it's a bitmask, but I'm not sure of that. -- Ian From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 20:38:14 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4F34D95; Tue, 1 Apr 2014 20:38:14 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A951894F; Tue, 1 Apr 2014 20:38:14 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s31Kf4uF040870; Tue, 1 Apr 2014 13:41:10 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s31KewPi040866; Tue, 1 Apr 2014 13:40:58 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Tue, 1 Apr 2014 13:40:58 -0700 (PDT) Message-ID: <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> In-Reply-To: <1396384167.81853.210.camel@revolution.hippie.lan> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> Date: Tue, 1 Apr 2014 13:40:58 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: "Ian Lepore" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: pyunyh@gmail.com, freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 20:38:15 -0000 > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> [...] >> miibus0: on nfe0 >> rlphy0: PHY 0 on miibus0 >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> rlphy1: PHY 1 on miibus0 > [...]---big-snip--8<--- >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> As you can see, it looks much the same. I have no idea what >> I should do to better inform the driver/kernel how to better >> handle it. Or is it the driver, itself? >> >> Thank you again, for your thoughtful response. >> >> --Chris >> > > I think the way to fix a phy that responds at all addresses is to set a > hint in loader.conf masking out the ones that aren't real, like so: > > hint.miibus.0.phymask="1" > > You might be able to set ="0x00000001" to make it more clear it's a > bitmask, but I'm not sure of that. Thank you very much for the hint. I'll give it a shot. Any idea why this is happening? I have 4 other MB's using the Nvidia chipset, and the nfe(4) driver. But they don't respond this way. Thank you again, for your helpful reply. I'll report back with my findings. --Chris > > -- Ian > > > From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 20:56:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 537B050D; Tue, 1 Apr 2014 20:56:32 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2041B17; Tue, 1 Apr 2014 20:56:31 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s31KxL4W043508; Tue, 1 Apr 2014 13:59:27 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s31KxEWf043504; Tue, 1 Apr 2014 13:59:14 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Tue, 1 Apr 2014 13:59:15 -0700 (PDT) Message-ID: In-Reply-To: <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> Date: Tue, 1 Apr 2014 13:59:15 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: "Ian Lepore" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: pyunyh@gmail.com, freebsd-net , freebsd-stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 20:56:32 -0000 >> On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >>> [...] >>> miibus0: on nfe0 >>> rlphy0: PHY 0 on miibus0 >>> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >>> rlphy1: PHY 1 on miibus0 >> [...]---big-snip--8<--- >>> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >>> >>> As you can see, it looks much the same. I have no idea what >>> I should do to better inform the driver/kernel how to better >>> handle it. Or is it the driver, itself? >>> >>> Thank you again, for your thoughtful response. >>> >>> --Chris >>> >> >> I think the way to fix a phy that responds at all addresses is to set a >> hint in loader.conf masking out the ones that aren't real, like so: >> >> hint.miibus.0.phymask="1" >> >> You might be able to set ="0x00000001" to make it more clear it's a >> bitmask, but I'm not sure of that. > > Thank you very much for the hint. I'll give it a shot. > Any idea why this is happening? I have 4 other MB's using the Nvidia > chipset, and the nfe(4) driver. But they don't respond this way. > > Thank you again, for your helpful reply. > I'll report back with my findings. > > --Chris OK. As promised. Here's the results. For the record, I used the following in loader.conf(5): hint.miibus.0.phymask="0x00000001" Here's the output (minus the sound card stuff): Syncing disks, vnodes remaining...8 8 4 4 1 0 1 1 1 1 0 0 done All buffers synced. Table 'FACP' at 0x6ffa0200 Table 'APIC' at 0x6ffa0390 APIC: Found table at 0x6ffa0390 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 1: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 129 ACPI ID 2: disabled MADT: Found CPU APIC ID 130 ACPI ID 3: disabled MADT: Found CPU APIC ID 131 ACPI ID 4: disabled MADT: Found CPU APIC ID 132 ACPI ID 5: disabled MADT: Found CPU APIC ID 133 ACPI ID 6: disabled Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.2-STABLE #0 r263756: Wed Mar 26 11:28:10 PDT 2014 root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 gcc version 4.2.1 20070831 patched [FreeBSD] Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff81d9d000. Preloaded elf obj module "/boot/kernel/linux.ko" at 0xffffffff81d9d200. Preloaded elf obj module "/boot/modules/nvidia.ko" at 0xffffffff81d9db28. Calibrating TSC clock ... TSC clock: 3231148565 Hz CPU: AMD Sempron(tm) 140 Processor (3231.15-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f62 Family = 0x10 Model = 0x6 Stepping = 2 Features=0x78bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x37fd TSC: P-state invariant L1 2MB data TLB: 48 entries, fully associative L1 2MB instruction TLB: 16 entries, fully associative L1 4KB data TLB: 48 entries, fully associative L1 4KB instruction TLB: 32 entries, fully associative L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative L2 2MB data TLB: 128 entries, 2-way associative L2 2MB instruction TLB: 0 entries, 2-way associative L2 4KB data TLB: 512 entries, 4-way associative L2 4KB instruction TLB: 512 entries, 4-way associative L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative real memory = 2147483648 (2048 MB) Physical memory chunk(s): 0x0000000000010000 - 0x000000000009bfff, 573440 bytes (140 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x0000000001dcc000 - 0x000000006cab2fff, 1791913984 bytes (437479 pages) avail memory = 1777266688 (1694 MB) INTR: Adding local APIC 0 as a target Event timer "LAPIC" quality 400 ACPI APIC Table: <7309MS A7309200> x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 x86bios: SSEG 0x098000-0x098fff at 0xffffff8000226000 x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 APIC: CPU 0 has ACPI ID 1 ULE: setup cpu 0 ACPI: RSDP 0xfaab0 00014 (v00 ACPIAM) ACPI: RSDT 0x6ffa0000 00048 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: FACP 0x6ffa0200 00084 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: DSDT 0x6ffa04b0 0503B (v01 A7309 A7309200 00000200 INTL 20051117) ACPI: FACS 0x6ffae000 00040 ACPI: APIC 0x6ffa0390 00090 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: MCFG 0x6ffa0420 0003C (v01 7309MS OEMMCFG 20101122 MSFT 00000097) ACPI: WDRT 0x6ffa0460 00047 (v01 7309MS NV-WDRT 20101122 MSFT 00000097) ACPI: OEMB 0x6ffae040 00072 (v01 7309MS A7309200 20101122 MSFT 00000097) ACPI: SRAT 0x6ffaa4b0 00090 (v03 AMD FAM_F_10 00000002 AMD 00000001) ACPI: MSCT 0x6ffaa540 0004E (v01 A M I OEMBOARD 00000001 AMD 00000001) ACPI: HPET 0x6ffaa590 00038 (v01 7309MS OEMHPET0 20101122 MSFT 00000097) ACPI: SSDT 0x6ffaa5d0 0023E (v01 A M I POWERNOW 00000001 AMD 00000001) MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level MADT: Interrupt override: source 14, irq 14 MADT: Interrupt override: source 15, irq 15 ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 wlan: <802.11 Link Layer> kbd: new array size 4 kbd1 at kbdmux0 nfslock: pseudo-device mem: null: random: cpuctl: access to MSR registers/cpuid info. VESA: INT 0x10 vector 0xc000:0x0fd8 VESA: information block 0000 56 45 53 41 00 03 00 01 00 99 01 00 00 00 22 00 0010 00 99 00 10 61 05 07 01 00 99 1b 01 00 99 2c 01 0020 00 99 00 01 01 01 02 01 03 01 04 01 05 01 06 01 0030 07 01 0e 01 0f 01 11 01 12 01 14 01 15 01 17 01 0040 18 01 1a 01 1b 01 30 01 31 01 32 01 33 01 34 01 0050 35 01 36 01 3d 01 3e 01 45 01 46 01 47 01 48 01 0060 52 01 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 4e 56 49 44 49 41 00 42 75 69 6c 64 20 20 20 20 0110 30 36 31 30 31 30 2e 33 0d 0a 00 4d 43 50 36 31 0120 20 2d 20 6d 63 70 36 31 2d 38 30 00 43 68 69 70 0130 20 52 65 76 20 20 20 00 00 00 00 00 00 00 00 00 0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 VESA: 32 mode(s) found VESA: v3.0, 262144k memory, flags:0x1, mode table:0xffffff800025e022 (99000022) VESA: NVIDIA VESA: Build 061010.3 MCP61 - mcp61-80 Chip Rev io: acpi0: <7309MS A7309200> on motherboard PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 ACPI: Executed 1 blocks of module-level executable AML code acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 7ff00000 (3) failed cpu0: on acpi0 cpu0: switching to generic Cx mode ACPI: Processor \134_PR_.P002 (ACPI ID 2) ignored ACPI: Processor \134_PR_.P003 (ACPI ID 3) ignored ACPI: Processor \134_PR_.P004 (ACPI ID 4) ignored ACPI: Processor \134_PR_.P005 (ACPI ID 5) ignored ACPI: Processor \134_PR_.P006 (ACPI ID 6) ignored attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfeff0000-0xfeff0fff on acpi0 hpet0: vendor 0x10de, rev 0x1, 25000000Hz, 3 timers, legacy route hpet0: t0: irqs 0x00ff3efa (31), periodic hpet0: t1: irqs 0x00ff3efa (31) hpet0: t2: irqs 0x00ff3efa (31) Timecounter "HPET" frequency 25000000 Hz quality 950 ACPI timer: 1/2 1/1 1/2 1/2 1/2 1/2 1/2 1/2 1/2 1/3 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 16 17 18 19 Validation 0 255 N 0 16 17 18 19 After Disable 0 255 N 0 16 17 18 19 pci_link8: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link9: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link10: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link11: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link12: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link13: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link14: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link15: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link16: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pci_link17: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 20 21 22 23 Validation 0 255 N 0 20 21 22 23 After Disable 0 255 N 0 20 21 22 23 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: decoding 3 range 0xa0000-0xbffff pcib0: decoding 3 range 0xd0000-0xdffff pcib0: decoding 3 range 0x80000000-0xdfffffff pcib0: decoding 3 range 0xf0000000-0xfed44fff pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x10de, dev=0x03ea, revid=0xa1 domain=0, bus=0, slot=0, func=0 class=05-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03e0, revid=0xa2 domain=0, bus=0, slot=1, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x000f, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) map[10]: type I/O Port, range 32, base 0x4f00, size 8, enabled pcib0: allocated type 4 (0x4f00-0x4fff) for rid 10 of pci0:0:1:0 found-> vendor=0x10de, dev=0x03eb, revid=0xa2 domain=0, bus=0, slot=1, func=1 class=0c-05-00, hdrtype=0x00, mfdev=1 cmdreg=0x0001, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0x4900, size 6, enabled pcib0: allocated type 4 (0x4900-0x493f) for rid 10 of pci0:0:1:1 map[20]: type I/O Port, range 32, base 0x4d00, size 6, enabled pcib0: allocated type 4 (0x4d00-0x4d3f) for rid 20 of pci0:0:1:1 map[24]: type I/O Port, range 32, base 0x4e00, size 6, enabled pcib0: allocated type 4 (0x4e00-0x4e3f) for rid 24 of pci0:0:1:1 pcib0: matched entry for 0.1.INTA (src \134_SB_.LSMB:0) pci_link13: Picked IRQ 20 with weight 0 pcib0: slot 1 INTA routed to irq 20 via \134_SB_.LSMB found-> vendor=0x10de, dev=0x03f5, revid=0xa2 domain=0, bus=0, slot=1, func=2 class=05-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0400, statreg=0x00a0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x10de, dev=0x03f1, revid=0xa3 domain=0, bus=0, slot=2, func=0 class=0c-03-10, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7f000, size 12, enabled pcib0: allocated type 3 (0xdff7f000-0xdff7ffff) for rid 10 of pci0:0:2:0 pcib0: matched entry for 0.2.INTA (src \134_SB_.LUB0:0) pci_link8: Picked IRQ 21 with weight 0 pcib0: slot 2 INTA routed to irq 21 via \134_SB_.LUB0 ohci early: SMM active, request owner change found-> vendor=0x10de, dev=0x03f2, revid=0xa3 domain=0, bus=0, slot=2, func=1 class=0c-03-20, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xdff7ec00, size 8, enabled pcib0: allocated type 3 (0xdff7ec00-0xdff7ecff) for rid 10 of pci0:0:2:1 pcib0: matched entry for 0.2.INTB (src \134_SB_.LUB2:0) pci_link9: Picked IRQ 22 with weight 0 pcib0: slot 2 INTB routed to irq 22 via \134_SB_.LUB2 found-> vendor=0x10de, dev=0x03f3, revid=0xa1 domain=0, bus=0, slot=4, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x02 (500 ns) found-> vendor=0x10de, dev=0x03f0, revid=0xa2 domain=0, bus=0, slot=5, func=0 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x05 (1250 ns) intpin=b, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff78000, size 14, enabled pcib0: allocated type 3 (0xdff78000-0xdff7bfff) for rid 10 of pci0:0:5:0 pcib0: matched entry for 0.5.INTB (src \134_SB_.LAZA:0) pci_link11: Picked IRQ 23 with weight 0 pcib0: slot 5 INTB routed to irq 23 via \134_SB_.LAZA found-> vendor=0x10de, dev=0x03ec, revid=0xa2 domain=0, bus=0, slot=6, func=0 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) powerspec 2 supports D0 D3 current D0 pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:6:0 pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:6:0 pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:6:0 pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:6:0 map[20]: type I/O Port, range 32, base 0xffa0, size 4, enabled pcib0: allocated type 4 (0xffa0-0xffaf) for rid 20 of pci0:0:6:0 found-> vendor=0x10de, dev=0x03ef, revid=0xa2 domain=0, bus=0, slot=7, func=0 class=06-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x01 (250 ns), maxlat=0x14 (5000 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 MSI supports 8 messages, 64 bit, vector masks map[10]: type Memory, range 32, base 0xdff7d000, size 12, enabled pcib0: allocated type 3 (0xdff7d000-0xdff7dfff) for rid 10 of pci0:0:7:0 map[14]: type I/O Port, range 32, base 0xe480, size 3, enabled pcib0: allocated type 4 (0xe480-0xe487) for rid 14 of pci0:0:7:0 pcib0: matched entry for 0.7.INTA (src \134_SB_.LMAC:0) pci_link10: Picked IRQ 20 with weight 1 pcib0: slot 7 INTA routed to irq 20 via \134_SB_.LMAC found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=0 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xe400, size 3, enabled pcib0: allocated type 4 (0xe400-0xe407) for rid 10 of pci0:0:8:0 map[14]: type I/O Port, range 32, base 0xe080, size 2, enabled pcib0: allocated type 4 (0xe080-0xe083) for rid 14 of pci0:0:8:0 map[18]: type I/O Port, range 32, base 0xe000, size 3, enabled pcib0: allocated type 4 (0xe000-0xe007) for rid 18 of pci0:0:8:0 map[1c]: type I/O Port, range 32, base 0xdc00, size 2, enabled pcib0: allocated type 4 (0xdc00-0xdc03) for rid 1c of pci0:0:8:0 map[20]: type I/O Port, range 32, base 0xd880, size 4, enabled pcib0: allocated type 4 (0xd880-0xd88f) for rid 20 of pci0:0:8:0 map[24]: type Memory, range 32, base 0xdff7c000, size 12, enabled pcib0: allocated type 3 (0xdff7c000-0xdff7cfff) for rid 24 of pci0:0:8:0 pcib0: matched entry for 0.8.INTA (src \134_SB_.LSA0:0) pci_link15: Picked IRQ 21 with weight 1 pcib0: slot 8 INTA routed to irq 21 via \134_SB_.LSA0 found-> vendor=0x10de, dev=0x03f6, revid=0xa2 domain=0, bus=0, slot=8, func=1 class=01-01-85, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x03 (750 ns), maxlat=0x01 (250 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 4 messages, 64 bit map[10]: type I/O Port, range 32, base 0xd800, size 3, enabled pcib0: allocated type 4 (0xd800-0xd807) for rid 10 of pci0:0:8:1 map[14]: type I/O Port, range 32, base 0xd480, size 2, enabled pcib0: allocated type 4 (0xd480-0xd483) for rid 14 of pci0:0:8:1 map[18]: type I/O Port, range 32, base 0xd400, size 3, enabled pcib0: allocated type 4 (0xd400-0xd407) for rid 18 of pci0:0:8:1 map[1c]: type I/O Port, range 32, base 0xd080, size 2, enabled pcib0: allocated type 4 (0xd080-0xd083) for rid 1c of pci0:0:8:1 map[20]: type I/O Port, range 32, base 0xd000, size 4, enabled pcib0: allocated type 4 (0xd000-0xd00f) for rid 20 of pci0:0:8:1 map[24]: type Memory, range 32, base 0xdff77000, size 12, enabled pcib0: allocated type 3 (0xdff77000-0xdff77fff) for rid 24 of pci0:0:8:1 pcib0: matched entry for 0.8.INTB (src \134_SB_.LSA1:0) pci_link16: Picked IRQ 22 with weight 1 pcib0: slot 8 INTB routed to irq 22 via \134_SB_.LSA1 found-> vendor=0x10de, dev=0x03e8, revid=0xa2 domain=0, bus=0, slot=9, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=11, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03e9, revid=0xa2 domain=0, bus=0, slot=12, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0004, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 MSI supports 2 messages, 64 bit found-> vendor=0x10de, dev=0x03d0, revid=0xa2 domain=0, bus=0, slot=13, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xde000000, size 24, enabled pcib0: allocated type 3 (0xde000000-0xdeffffff) for rid 10 of pci0:0:13:0 map[14]: type Prefetchable Memory, range 64, base 0xc0000000, size 28, enabled pcib0: allocated type 3 (0xc0000000-0xcfffffff) for rid 14 of pci0:0:13:0 map[1c]: type Memory, range 64, base 0xdd000000, size 24, enabled pcib0: allocated type 3 (0xdd000000-0xddffffff) for rid 1c of pci0:0:13:0 pcib0: matched entry for 0.13.INTA (src \134_SB_.LMC9:0) pci_link12: Picked IRQ 23 with weight 1 pcib0: slot 13 INTA routed to irq 23 via \134_SB_.LMC9 found-> vendor=0x1022, dev=0x1200, revid=0x00 domain=0, bus=0, slot=24, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1201, revid=0x00 domain=0, bus=0, slot=24, func=1 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1202, revid=0x00 domain=0, bus=0, slot=24, func=2 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1203, revid=0x00 domain=0, bus=0, slot=24, func=3 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1022, dev=0x1204, revid=0x00 domain=0, bus=0, slot=24, func=4 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) pci0: at device 0.0 (no driver attached) isab0: port 0x4f00-0x4fff at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) pci0: at device 1.2 (no driver attached) ohci0: mem 0xdff7f000-0xdff7ffff irq 21 at device 2.0 on pci0 ioapic0: routing intpin 21 (PCI IRQ 21) to lapic 0 vector 51 usbus0 on ohci0 usbus0: bpf attached ohci0: usbpf: Attached ehci0: mem 0xdff7ec00-0xdff7ecff irq 22 at device 2.1 on pci0 ioapic0: routing intpin 22 (PCI IRQ 22) to lapic 0 vector 52 ehci0: Doorbell workaround enabled usbus1: EHCI version 1.0 usbus1 on ehci0 usbus1: bpf attached ehci0: usbpf: Attached pcib1: at device 4.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: special decode subtractive pci1: on pcib1 pci1: domain=0, physical bus=1 hdac0: mem 0xdff78000-0xdff7bfff irq 23 at device 5.0 on pci0 hdac0: PCI card vendor: 0x0888, device: 0x10ec hdac0: HDA Driver Revision: 20120126_0002 hdac0: Config options: on=0x00000000 off=0x00000000 hdac0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 53 hdac0: using IRQ 256 for MSI hdac0: Caps: OSS 4, ISS 4, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 6.0 on pci0 ata0: at channel 0 on atapci0 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 54 ata1: at channel 1 on atapci0 ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 55 nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff irq 20 at device 7.0 on pci0 nfe0: attempting to allocate 8 MSI vectors (8 supported) msi: routing MSI IRQ 257 to local APIC 0 vector 56 msi: routing MSI IRQ 258 to local APIC 0 vector 57 msi: routing MSI IRQ 259 to local APIC 0 vector 58 msi: routing MSI IRQ 260 to local APIC 0 vector 59 msi: routing MSI IRQ 261 to local APIC 0 vector 60 msi: routing MSI IRQ 262 to local APIC 0 vector 61 msi: routing MSI IRQ 263 to local APIC 0 vector 62 msi: routing MSI IRQ 264 to local APIC 0 vector 63 nfe0: using IRQs 257-264 for MSI nfe0: Using 8 MSI messages miibus0: on nfe0 rlphy0: PHY 0 on miibus0 rlphy0: OUI 0x000004, model 0x0020, rev. 1 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow nfe0: bpf attached nfe0: Ethernet address: 40:61:86:cd:44:97 atapci1: port 0xe400-0xe407,0xe080-0xe083,0xe000-0xe007,0xdc00-0xdc03,0xd880-0xd88f mem 0xdff7c000-0xdff7cfff irq 21 at device 8.0 on pci0 ata2: at channel 0 on atapci1 ata3: at channel 1 on atapci1 atapci2: port 0xd800-0xd807,0xd480-0xd483,0xd400-0xd407,0xd080-0xd083,0xd000-0xd00f mem 0xdff77000-0xdff77fff irq 22 at device 8.1 on pci0 ata4: at channel 0 on atapci2 ata5: at channel 1 on atapci2 pcib2: at device 9.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: at device 11.0 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pci3: on pcib3 pci3: domain=0, physical bus=3 pcib4: at device 12.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pci4: on pcib4 pci4: domain=0, physical bus=4 vgapci0: mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff,0xdd000000-0xddffffff irq 23 at device 13.0 on pci0 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 64 vgapci0: Boot video device amdtemp0: on hostb3 amdtemp0: Found 1 cores and 1 sensors. acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 65 uart0: fast interrupt ppc0: using extended I/O port range ppc0: SPP ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ioapic0: routing intpin 7 (ISA IRQ 7) to lapic 0 vector 66 ppbus0: on ppc0 plip0: on ppbus0 plip0: bpf attached lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x83ab (2) kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 67 atkbd0: [GIANT-LOCKED] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0065 psm0: irq 12 on atkbdc0 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 68 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons psm0: config:00000000, flags:00000008, packet size:4 psm0: syncmask:08, syncbits:00 acpi0: wakeup code va 0xffffff80003d3000 pa 0x90000 pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0 pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0 pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0 pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0 pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0000-0xd07ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd0800-0xd0fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd1000-0xd17ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd1800-0xd1fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd2000-0xd27ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd2800-0xd2fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd3000-0xd37ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd3800-0xd3fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd4000-0xd47ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd4800-0xd4fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd5000-0xd57ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd5800-0xd5fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6000-0xd67ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6800-0xd6fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7000-0xd77ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7800-0xd7fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8000-0xd87ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8800-0xd8fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9000-0xd97ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9800-0xd9fff) for rid 1 of orm0 pcib0: allocated type 3 (0xda000-0xda7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xda800-0xdafff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb000-0xdb7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb800-0xdbfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc000-0xdc7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc800-0xdcfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd000-0xdd7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd800-0xddfff) for rid 1 of orm0 pcib0: allocated type 3 (0xde000-0xde7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xde800-0xdefff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf000-0xdf7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf800-0xdffff) for rid 1 of orm0 isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it ppc: ppc0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xcefff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0 pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0 fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1 uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices hwpstate0: on cpu0 Device configuration finished. procfs registered lapic: Divisor 2, Frequency 119672206 Hz Timecounters tick every 1.000 msec vlan: initialized, using hash tables with chaining Linux ELF exec handler installed lo0: bpf attached hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 hdaa0: Subsystem ID: 0x14627309 hdaa0: NumGPIO=2 NumGPO=0 NumGPI=0 GPIWake=1 GPIUnsol=1 [...][---8<---] hdaa0: pcm0: at nid 20 and 24,26 on hdaa0 [...][---8<---] pcm0: sndbuf_setmap 3480000, 10000; 0xffffff806bb5c000 -> 3480000 pcm1: at nid 27 and 25 on hdaa0 [...][---8<---] pcm1: sndbuf_setmap 2780000, 10000; 0xffffff808473c000 -> 2780000 pcm2: at nid 30 on hdaa0 [...][--8<---] pcm2: sndbuf_setmap 27c0000, 10000; 0xffffff808477c000 -> 27c0000 usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 ata0: reset tp1 mask=03 ostat0=50 ostat1=50 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: stat1=0x50 err=0x01 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=50 stat1=50 devices=0x3 ata1: reset tp1 mask=00 ostat0=ff ostat1=ff (aprobe0:ata0:0:1:0): Spinning up device (aprobe0:ata0:0:1:0): Spin-up done uhub0: 10 ports with 10 removable, self powered ata2: SATA connect timeout status=00000000 ata3: SATA connect timeout status=00000000 ata4: SATA connect timeout status=00000000 ata5: SATA connect timeout status=00000000 pass0 at ata0 bus 0 scbus0 target 0 lun 0 pass0: ATA-6 device pass0: Serial Number M1000000 pass0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) pass1 at ata0 bus 0 scbus0 target 1 lun 0 pass1: ATA-5 device pass1: Serial Number VNP214B2T59A5E pass1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0 at ata0 bus 0 scbus0 target 0 lun 0 ada0: ATA-6 device ada0: Serial Number M1000000 ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0: 19541MB (40020624 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ata0 bus 0 scbus0 target 1 lun 0 ada1: ATA-5 device ada1: Serial Number VNP214B2T59A5E ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada1: 39266MB (80418240 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad1 GEOM: new disk ada0 GEOM: new disk ada1 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1615574282 Hz quality 1000 uhub1: 10 ports with 10 removable, self powered Trying to mount root from ufs:/dev/ada0s1a [rw]... start_init: trying /sbin/init linprocfs registered Thanks Ian! As you can see, I am no longer spammed with the 30+ extraneous lines. Oh, and yes, the network still functions. :) Be interesting to know why the difference. I've been using nfe(4) for years, and never ran into anything similar. FWIW, the MB is an MSI K9N6PGM2-V2. Not very new. But something I had handy for testing newer versions of FreeBSD on, and experimenting with different CPU's. Thank you again, for providing the (apparent) solution. --Chris > >> >> -- Ian >> >> >> > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Apr 1 22:36:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59ECC65D for ; Tue, 1 Apr 2014 22:36:42 +0000 (UTC) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28E9365A for ; Tue, 1 Apr 2014 22:36:41 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id kq14so10517710pab.37 for ; Tue, 01 Apr 2014 15:36:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=LmWVpAF4vinXLom9A/4rd+m4q0W/6LAmgGEIHKXD3Ig=; b=lLlP3X0imZpC+uQnfdZO+fM5btNj64c4UWKCVYYHJdXl/3Oo1O8BlMOfNAmEt9t6F+ J1GrtyboYxxxCZz3f35IM+t09ztxJnvrtyspGtFgwE0ei/U3WmbU3861DoWGZyLUsc5M 6HYgyTehHP/1H3DHpQZ8dkwp1JTqvYgICNlcwI0RJRjU9SbP32rjO6tAZdT9RqMa1iK+ XeULtAZx4S3q817pIP80wxxoRS2rFnLrGh1+DvxgYabfhvhMChaikwPLqeoQbBaIMszn NYWP54oygO9yKXVctxk0vqqp8R9ZKRjckczDG47mJcWIxlVlTiMlW3j2vxPBi7hsfces h26Q== X-Gm-Message-State: ALoCoQlA6NfMcRQqH8U7RCZbVfJHQsPguyTH8/qNVkqheK8h07szTUCvOnUdRaDmxi0Tmz4GltPB X-Received: by 10.68.4.232 with SMTP id n8mr17726230pbn.114.1396391801191; Tue, 01 Apr 2014 15:36:41 -0700 (PDT) Received: from [10.64.24.154] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ha11sm37111pbd.17.2014.04.01.15.36.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 15:36:40 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: Warner Losh In-Reply-To: <20140401065842.GA1364@michelle.cdnetworks.com> Date: Tue, 1 Apr 2014 16:36:37 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: freebsd-net , freebsd-stable , Chris H X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 22:36:42 -0000 On Apr 1, 2014, at 12:58 AM, Yonghyeon PYUN wrote: > On Mon, Mar 31, 2014 at 07:57:28AM -0700, Chris H wrote: >>> On Sun, Mar 30, 2014 at 01:12:20PM -0700, chrish@UltimateDNS.NET = wrote: >>>> Greetings, >>>> I'm not sure whether this best belonged on net@, or stable@ >>>> so I'm using both. :) >>>> I'm testing both releng_9, and MB, and I encountered a new >>>> message I don't usually see using the nfe(4) driver: >>>>=20 >>>> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >>>> ... >>>> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >>>>=20 >>>> Truncated for brevity (31 lines in total; 1-31). I don't know >>>> how interpret this. An issue with my version of the driver, or >>>> the hardware itself? This occurred with both GENERIC, as well >>>> as my custom kernel. >>>=20 >>> Would you show me the dmesg output? >> Happily: >>=20 >> Calibrating TSC clock ... TSC clock: 3231132841 Hz >> CPU: AMD Sempron(tm) 140 Processor (3231.13-MHz K8-class CPU) >> Origin =3D "AuthenticAMD" Id =3D 0x100f62 Family =3D 0x10 Model =3D= 0x6 Stepping =3D 2 >> = Features=3D0x78bfbff >> Features2=3D0x802009 >> AMD = Features=3D0xee500800 >> AMD = Features2=3D0x37fd >=20 > [...] >=20 >> nfe0: port 0xe480-0xe487 mem = 0xdff7d000-0xdff7dfff >> irq 20 at device 7.0 on pci0 >> nfe0: attempting to allocate 8 MSI vectors (8 supported) >> msi: routing MSI IRQ 257 to local APIC 0 vector 56 >> msi: routing MSI IRQ 258 to local APIC 0 vector 57 >> msi: routing MSI IRQ 259 to local APIC 0 vector 58 >> msi: routing MSI IRQ 260 to local APIC 0 vector 59 >> msi: routing MSI IRQ 261 to local APIC 0 vector 60 >> msi: routing MSI IRQ 262 to local APIC 0 vector 61 >> msi: routing MSI IRQ 263 to local APIC 0 vector 62 >> msi: routing MSI IRQ 264 to local APIC 0 vector 63 >> nfe0: using IRQs 257-264 for MSI >> nfe0: Using 8 MSI messages >> miibus0: on nfe0 >> rlphy0: PHY 0 on miibus0 >> rlphy0: OUI 0x000004, model 0x0020, rev. 1 >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, = auto-flow >> rlphy1: PHY 1 on miibus0 >> rlphy1: OUI 0x000004, model 0x0020, rev. 1 >> rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, = auto-flow >=20 > [...] >=20 >> rlphy30: PHY 30 on miibus0 >> rlphy30: OUI 0x000004, model 0x0020, rev. 1 >> rlphy30: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, = auto-flow >> rlphy31: PHY 31 on miibus0 >> rlphy31: OUI 0x000004, model 0x0020, rev. 1 >> rlphy31: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, = auto-flow >> nfe0: bpf attached >> nfe0: Ethernet address: 40:61:86:cd:44:97 >=20 > mii(4) thinks it has 32 PHYs and this is the reason why mii(4) > complains. Due to unknown reason, accessing PHY registers in > device probe stage got valid response which in turn makes the > driver think there are 32 PHYs. Did you ever see this this kind of > message on old FreeBSD release? Or could you try cold-boot and see > whether it makes any difference? I=92ve seen this a few times when the resources were AFU on CardBus = cards.. But never this coherently=85 I=92ve also seen it during early = bring up when I got the MII address wrong, but you should be well past = that with the rl driver :) The voices in Bill Paul=92s head from that = are almost old enough to collect retirement... Warner From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 00:39:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FD6AD6A; Wed, 2 Apr 2014 00:39:20 +0000 (UTC) Received: from mail-pb0-x231.google.com (mail-pb0-x231.google.com [IPv6:2607:f8b0:400e:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7973FA8; Wed, 2 Apr 2014 00:39:19 +0000 (UTC) Received: by mail-pb0-f49.google.com with SMTP id jt11so10511052pbb.8 for ; Tue, 01 Apr 2014 17:39:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Gu+q8mteaHNquYTMkSgM9ak+Dqxi8ImabSIudIvom70=; b=JqCmpZ4evoeZB1X7ZNz58s43kQmpBrIGVoEgW9rnsX/UwVe+R/WLnzRjfNJK8sqUZu JhI1mZYPQ0MkFKK5IKLCKk6TukrrXWSORxtCnxrYQxthwCUeoWuJi6O5/rvzjnc9FbpI JjguSa8rW8kxR5Lbs4bR9aqq2LceKtdlKF5TiGlbq7VWKFP6kezqdRYisKfeemDvBdZJ n8cTyaVPbaW4czmu3GQNIeZP8/doGfqu976j9x+KOip0ooQbDR1ZhvACofl8WPM+DAyb k+EOVGFGJbpW+sVR5f93SPOnB94Mh/TOd81nzoMNLRJKc699TQA5YI0DgAaAFX0YWO3Y Peqw== X-Received: by 10.66.164.36 with SMTP id yn4mr34243512pab.25.1396399159559; Tue, 01 Apr 2014 17:39:19 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id nx12sm1281627pab.6.2014.04.01.17.39.16 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 01 Apr 2014 17:39:18 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 02 Apr 2014 09:39:13 +0900 From: Yonghyeon PYUN Date: Wed, 2 Apr 2014 09:39:12 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140402003912.GA2938@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 00:39:20 -0000 On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: > > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > >> [...] > >> miibus0: on nfe0 > >> rlphy0: PHY 0 on miibus0 > >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > >> rlphy1: PHY 1 on miibus0 > > [...]---big-snip--8<--- > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> > >> As you can see, it looks much the same. I have no idea what > >> I should do to better inform the driver/kernel how to better > >> handle it. Or is it the driver, itself? > >> > >> Thank you again, for your thoughtful response. > >> > >> --Chris > >> > > > > I think the way to fix a phy that responds at all addresses is to set a > > hint in loader.conf masking out the ones that aren't real, like so: > > > > hint.miibus.0.phymask="1" > > > > You might be able to set ="0x00000001" to make it more clear it's a > > bitmask, but I'm not sure of that. > > Thank you very much for the hint. I'll give it a shot. > Any idea why this is happening? I have 4 other MB's using the Nvidia > chipset, and the nfe(4) driver. But they don't respond this way. > If some nfe(4) variants badly behave in probing stage, this should be handled by driver. We already have too many hints and tunables and I don't think most users know that. In addition, adding additional NIC may change miibus instance number. Could you show me the output of 'kenv | grep smbios'? From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 00:51:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B743348; Wed, 2 Apr 2014 00:51:06 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A90BC180; Wed, 2 Apr 2014 00:51:06 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s320ru2v084277; Tue, 1 Apr 2014 17:54:02 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s320rpp2084270; Tue, 1 Apr 2014 17:53:51 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Tue, 1 Apr 2014 17:53:51 -0700 (PDT) Message-ID: <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> In-Reply-To: <20140402003912.GA2938@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> Date: Tue, 1 Apr 2014 17:53:51 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 00:51:07 -0000 > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> >> [...] >> >> miibus0: on nfe0 >> >> rlphy0: PHY 0 on miibus0 >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> >> rlphy1: PHY 1 on miibus0 >> > [...]---big-snip--8<--- >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> As you can see, it looks much the same. I have no idea what >> >> I should do to better inform the driver/kernel how to better >> >> handle it. Or is it the driver, itself? >> >> >> >> Thank you again, for your thoughtful response. >> >> >> >> --Chris >> >> >> > >> > I think the way to fix a phy that responds at all addresses is to set a >> > hint in loader.conf masking out the ones that aren't real, like so: >> > >> > hint.miibus.0.phymask="1" >> > >> > You might be able to set ="0x00000001" to make it more clear it's a >> > bitmask, but I'm not sure of that. >> >> Thank you very much for the hint. I'll give it a shot. >> Any idea why this is happening? I have 4 other MB's using the Nvidia >> chipset, and the nfe(4) driver. But they don't respond this way. >> > > If some nfe(4) variants badly behave in probing stage, this should > be handled by driver. We already have too many hints and tunables > and I don't think most users know that. In addition, adding > additional NIC may change miibus instance number. > > Could you show me the output of 'kenv | grep smbios'? Yes, of course. Here it is: smbios.bios.reldate="11/22/2010" smbios.bios.vendor="American Megatrends Inc." smbios.bios.version="V2.7" smbios.chassis.maker="MSI" smbios.chassis.serial="To Be Filled By O.E.M." smbios.chassis.tag="To Be Filled By O.E.M." smbios.chassis.version="2.0" smbios.memory.enabled="2097152" smbios.planar.maker="MSI" smbios.planar.product="K9N6PGM2-V2 (MS-7309)" smbios.planar.serial="To be filled by O.E.M." smbios.planar.version="2.0" smbios.socket.enabled="1" smbios.socket.populated="1" smbios.system.maker="MSI" smbios.system.product="MS-7309" smbios.system.serial="To Be Filled By O.E.M." smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" smbios.system.version="2.0" smbios.version="2.6" Hope this helps, and thank you for all your time, and trouble. --Chris > From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 02:04:12 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAD7CC88 for ; Wed, 2 Apr 2014 02:04:12 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F7DE89C for ; Wed, 2 Apr 2014 02:04:12 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s322492E017623 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 1 Apr 2014 20:04:10 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s32249LB017620 for ; Tue, 1 Apr 2014 20:04:09 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 1 Apr 2014 20:04:09 -0600 (MDT) From: Warren Block To: freebsd-net@FreeBSD.org Subject: Gigabyte BIOS/UEFI and WOL Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 01 Apr 2014 20:04:10 -0600 (MDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 02:04:12 -0000 So far I've tried and failed to get a Gigabyte GA-Z68A-D3H-B3 to wake from the network. It had the most recent BIOS (F13), which did not have a WOL option. A UEFI BIOS is available, so I've installed that, and still failed to get it to wake up. There are a bewildering number of undocumented options, none of which mentions WOL. Adding an Intel PCI card made no difference. The card LEDs are on when the system is off, but it still doesn't wake up. Once manually started, the system works fine, and ifconfig shows WOL_MAGIC. Any suggestions on things to try? I'm open to going back to a normal BIOS... if it will let me. It runs fine either way. From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 02:08:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26CF4E3A; Wed, 2 Apr 2014 02:08:11 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA0498D8; Wed, 2 Apr 2014 02:08:10 +0000 (UTC) Received: by mail-pd0-f173.google.com with SMTP id z10so10382381pdj.18 for ; Tue, 01 Apr 2014 19:08:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=SBBU8Rj+fg116gL3X+AuGs9c6HAsTyPqAjaepIgPkfc=; b=R97ELOilBRIOX4ZpAOjA/0o6ECRcMLawmmkaxY6LYs6WK7IrqLkX8GoBVQyDh6SAtN w0UMxe6ELDs9sR2D1mrnUbvf3Q3cQ2iK1TojLQe+Mh7tqK4RFMD92yLbffzvrIaRph6i NSCqVMSmeMDqzul2hau9W4Hi8/8ldFKAirH5TDaiiVo/aW4mRfzGYMv7vJ3gEx5eYBxL VhaLwWKxH/EsJfdILkcYniSwuR81BpbyeRtqWD3LZzvNOBmu376e+uzdKFF60bgNCCAZ l5zZt7PWKFq4MOIokWLna+OkzGZPvycUd/Syyc/r/2vLz7VEU59donNW1pDuZQf8WN67 ypmQ== X-Received: by 10.68.211.164 with SMTP id nd4mr34549976pbc.44.1396404490451; Tue, 01 Apr 2014 19:08:10 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id yw3sm700596pbc.69.2014.04.01.19.08.07 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 01 Apr 2014 19:08:09 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 02 Apr 2014 11:08:03 +0900 From: Yonghyeon PYUN Date: Wed, 2 Apr 2014 11:08:03 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140402020803.GB2938@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="azLHFNyN32YCQGCU" Content-Disposition: inline In-Reply-To: <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 02:08:11 -0000 --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: > > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: > >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > >> >> [...] > >> >> miibus0: on nfe0 > >> >> rlphy0: PHY 0 on miibus0 > >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > >> >> rlphy1: PHY 1 on miibus0 > >> > [...]---big-snip--8<--- > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> >> > >> >> As you can see, it looks much the same. I have no idea what > >> >> I should do to better inform the driver/kernel how to better > >> >> handle it. Or is it the driver, itself? > >> >> > >> >> Thank you again, for your thoughtful response. > >> >> > >> >> --Chris > >> >> > >> > > >> > I think the way to fix a phy that responds at all addresses is to set a > >> > hint in loader.conf masking out the ones that aren't real, like so: > >> > > >> > hint.miibus.0.phymask="1" > >> > > >> > You might be able to set ="0x00000001" to make it more clear it's a > >> > bitmask, but I'm not sure of that. > >> > >> Thank you very much for the hint. I'll give it a shot. > >> Any idea why this is happening? I have 4 other MB's using the Nvidia > >> chipset, and the nfe(4) driver. But they don't respond this way. > >> > > > > If some nfe(4) variants badly behave in probing stage, this should > > be handled by driver. We already have too many hints and tunables > > and I don't think most users know that. In addition, adding > > additional NIC may change miibus instance number. > > > > Could you show me the output of 'kenv | grep smbios'? > Yes, of course. > > Here it is: > > smbios.bios.reldate="11/22/2010" > smbios.bios.vendor="American Megatrends Inc." > smbios.bios.version="V2.7" > smbios.chassis.maker="MSI" > smbios.chassis.serial="To Be Filled By O.E.M." > smbios.chassis.tag="To Be Filled By O.E.M." > smbios.chassis.version="2.0" > smbios.memory.enabled="2097152" > smbios.planar.maker="MSI" > smbios.planar.product="K9N6PGM2-V2 (MS-7309)" > smbios.planar.serial="To be filled by O.E.M." > smbios.planar.version="2.0" > smbios.socket.enabled="1" > smbios.socket.populated="1" > smbios.system.maker="MSI" > smbios.system.product="MS-7309" > smbios.system.serial="To Be Filled By O.E.M." > smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" > smbios.system.version="2.0" > smbios.version="2.6" > > Hope this helps, and thank you for all your time, and trouble. > Thanks for the info. Try attached patch and let me know how it works. Make sure to remove the hint(hint.miibus.0.phymask="1") set in loader.conf before testing it. > --Chris > > > > --azLHFNyN32YCQGCU Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="nfe.msi.diff" Index: sys/dev/nfe/if_nfe.c =================================================================== --- sys/dev/nfe/if_nfe.c (revision 264031) +++ sys/dev/nfe/if_nfe.c (working copy) @@ -79,6 +79,7 @@ static int nfe_suspend(device_t); static int nfe_resume(device_t); static int nfe_shutdown(device_t); static int nfe_can_use_msix(struct nfe_softc *); +static int nfe_detect_msik9(struct nfe_softc *); static void nfe_power(struct nfe_softc *); static int nfe_miibus_readreg(device_t, int, int); static int nfe_miibus_writereg(device_t, int, int, int); @@ -334,13 +335,38 @@ nfe_alloc_msix(struct nfe_softc *sc, int count) } } + static int +nfe_detect_msik9(struct nfe_softc *sc) +{ + static char *maker = "MSI"; + static char *product = "K9N6PGM2-V2 (MS-7309)"; + char *m, *p; + int found; + + found = 0; + m = getenv("smbios.planar.maker"); + p = getenv("smbios.planar.product"); + if (m != NULL && p != NULL) { + if (strcmp(m, maker) == 0 && strcmp(p, product) == 0) + found = 1; + } + if (m != NULL) + freeenv(m); + if (p != NULL) + freeenv(p); + + return (found); +} + + +static int nfe_attach(device_t dev) { struct nfe_softc *sc; struct ifnet *ifp; bus_addr_t dma_addr_max; - int error = 0, i, msic, reg, rid; + int error = 0, i, msic, phyloc, reg, rid; sc = device_get_softc(dev); sc->nfe_dev = dev; @@ -608,8 +634,13 @@ nfe_attach(device_t dev) #endif /* Do MII setup */ + phyloc = MII_PHY_ANY; + if (sc->nfe_devid == PCI_PRODUCT_NVIDIA_MCP61_LAN1) { + if (nfe_detect_msik9(sc) != 0) + phyloc = 0; + } error = mii_attach(dev, &sc->nfe_miibus, ifp, nfe_ifmedia_upd, - nfe_ifmedia_sts, BMSR_DEFCAPMASK, MII_PHY_ANY, MII_OFFSET_ANY, + nfe_ifmedia_sts, BMSR_DEFCAPMASK, phyloc, MII_OFFSET_ANY, MIIF_DOPAUSE); if (error != 0) { device_printf(dev, "attaching PHYs failed\n"); --azLHFNyN32YCQGCU-- From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 02:52:55 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DB375E5 for ; Wed, 2 Apr 2014 02:52:55 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A320BFB for ; Wed, 2 Apr 2014 02:52:54 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s322qr7G017835 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 1 Apr 2014 20:52:53 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s322qrXM017832 for ; Tue, 1 Apr 2014 20:52:53 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 1 Apr 2014 20:52:53 -0600 (MDT) From: Warren Block To: freebsd-net@FreeBSD.org Subject: Re: Gigabyte BIOS/UEFI and WOL In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 01 Apr 2014 20:52:53 -0600 (MDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 02:52:55 -0000 On Tue, 1 Apr 2014, Warren Block wrote: > So far I've tried and failed to get a Gigabyte GA-Z68A-D3H-B3 to wake from > the network. It had the most recent BIOS (F13), which did not have a WOL > option. > > A UEFI BIOS is available, so I've installed that, and still failed to get it > to wake up. There are a bewildering number of undocumented options, none of > which mentions WOL. > > Adding an Intel PCI card made no difference. The card LEDs are on when the > system is off, but it still doesn't wake up. > > Once manually started, the system works fine, and ifconfig shows WOL_MAGIC. > > Any suggestions on things to try? I'm open to going back to a normal BIOS... > if it will let me. It runs fine either way. And of course I tried it one more time out of desperation and it worked. I'll document the settings ...if they work again. From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 03:26:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C921A9C for ; Wed, 2 Apr 2014 03:26:56 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 505C8EE8 for ; Wed, 2 Apr 2014 03:26:56 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id md12so10796186pbc.21 for ; Tue, 01 Apr 2014 20:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=uNOtmipsvrn2iqVmPNSGfn7mBKDQNnKL4AOXb/jxOx0=; b=T4Ulyr1TTQLQu2brymTpOUow5mBIfHYqOMnM7xQ4raXxt83YHwIUd0qU9QGzuoI35x GtkMWXQ8nG+0ZJiHCK3cIvdD4xLbqtafQt9AFspIAQY1HSmtTvS7TUFVT4gDEb3sRRTS dpN8XfZ08yCrgnfhaEAlV131z9GNjYxifEzJ6ukG7wUwKIzc8L9pEVtYmGuFembDBBwm mWThJKJ8a6mcpguEDWEhkIwq5XQnzA4b10ncq5UQTCsuTNBgTfGKpOfx2gdhu5cszj29 kC04Wd1oZY0BHDKbzwlGB9+p1wyMmYKJ/DIVmLHP4aEelXGCvsJkWLlSc06u56oRepqg Ry4w== X-Received: by 10.68.170.66 with SMTP id ak2mr35035012pbc.5.1396409215428; Tue, 01 Apr 2014 20:26:55 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.121]) by mx.google.com with ESMTPSA id op3sm1024873pbc.40.2014.04.01.20.26.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 20:26:54 -0700 (PDT) Message-ID: <533B8377.9060209@gmail.com> Date: Wed, 02 Apr 2014 11:26:47 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Rick Macklem , "freebsd-net@freebsd.org >> FreeBSD Net" Subject: Re: 9.2 ixgbe tx queue hang References: <1292881633.1858906.1395960241007.JavaMail.root@uoguelph.ca> In-Reply-To: <1292881633.1858906.1395960241007.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 03:26:56 -0000 Hi, Rick, Does these patches will commit to the stable soon, or I had to patch it manually? Regards Simon 䚎 14-3-28 6:44, Rick Macklem 写道: > Christopher Forgeron wrote: >> >> >> >> >> >> >> On Wed, Mar 26, 2014 at 9:35 PM, Rick Macklem < rmacklem@uoguelph.ca >>> wrote: >> >> >> >> >> I've suggested in the other thread what you suggested in a recent >> post...ie. to change the default, at least until the propagation >> of driver set values is resolved. >> >> rick >> >> >> >> I wonder if we need to worry about propagating values up from the >> sub-if's - Setting the default in if.c means this is set for all >> if's, and it's a simple 1 line code change. If a specific 'if' needs >> a different value, it can be set before ether_attach() is called. >> >> >> I'm more concerned with the equation we use to calculate if_hw_tsomax >> - Are we considering the right variables? Are we thinking on the >> wrong OSI layer for headers? >> > Well, I'm pragmatic (which means I mostly care about some fix that works), > but it seems to me that: > - The problem is that some TSO enabled network drivers/hardware can only > handle 32 transmit segments (or 32 mbufs in the chain for the TSO packet > to be transmitted, if that is clearer). > --> Since the problem is in certain drivers, it seems that those drivers > should be where the long term fix goes. > --> Since some hardware can't handle more than 32, it seems that the > driver should be able to specify that limit, which tcp_output() can > then apply. > > I have an untested patch that does this by adding if_hw_tsomaxseg. > (The attachment called tsomaxseg.patch.) > > Changing if_hw_tsomax or its default value is just a hack that gets tcp_output() > to apply a limit that the driver can then fix to 32 mbufs in the chain via > m_defrag(). > > Since if_hw_tsomax (and if_hw_tsomaxseg in the untested patch) aren't > propagated up through lagg, that needs to be fixed. > (Yet another attached untested patch called lagg.patch.) > > As I said before, I don't see these patches getting tested/reviewed etc > in time for 9.3, so I think reducing the default value of if_hw_tsomax > is a reasonable short term hack to work around the problem. > (And it sounds like Pyun YongHyeon has volunteered to fix many of the > drivers, where the 32 limit isn't a hardware one.) > > rick > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 07:35:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09724D69 for ; Wed, 2 Apr 2014 07:35:58 +0000 (UTC) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com [209.85.215.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81FF268E for ; Wed, 2 Apr 2014 07:35:56 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id pv20so3522345lab.23 for ; Wed, 02 Apr 2014 00:35:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=vxwURSWygoP2NpefqDSb9bSBSUNEW4waD7GxOVSF6C8=; b=MR3UqSXroZiFGCb28hcIL6wX8CfXbfoAjpQXRUN2xd3hQuuoRHcs5PL2Y/4TKVUnhW cfAq2JFUn8Ar5fh0LskcSyFqo/XLRj50diRihQnSP1zNKS6VfBQ5d/MLmPYI7k9N8UYR awV894/2MlA7AVlW7MhCN6CzcXz1yzLvGkb/LCUTHlO5QAw2gUnKS7zPLMGKQ9+SuD9W yFo0tN/cvqgNAGQykqv3olepsi3Z6E2JeCdXBjd4cN8F2b7OC4t/LHgMHhC+2rTxOXHs sYX5OCnVlceU38gghNZbFrU2GEu5M+xu30AWZQuRObx2jzTUvFkLU1ZR+QkD9Zw9RWvz V9vA== X-Gm-Message-State: ALoCoQnopm14YzP3VADRnKgPHPEbfJBCjLw64JBcwg2+1M1GuUy48A5E5QfRsp3QWNVajU8vc8/3 X-Received: by 10.152.29.8 with SMTP id f8mr26252190lah.11.1396424154430; Wed, 02 Apr 2014 00:35:54 -0700 (PDT) Received: from grey.office.se.prisjakt.nu ([212.16.170.194]) by mx.google.com with ESMTPSA id r5sm810261lbb.7.2014.04.02.00.35.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Apr 2014 00:35:53 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10-STABLE and CARP states From: mxb In-Reply-To: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> Date: Wed, 2 Apr 2014 09:35:54 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1874) Cc: freebsd-pf@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 07:35:58 -0000 Moving this to freebsd-pf. On 31 mar 2014, at 22:21, mxb wrote: >=20 > Manually setting net.inet.carp.demotion brought BOTH VHIDs in desired = state. > pfsync bulk update seems to not put everything back as it should. >=20 > lagg0: flags=3D8943 = metric 0 mtu 9000 > = options=3D8407bb > ether 00:25:90:e3:71:f2 > inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 > inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 > inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 > inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 > nd6 options=3D29 > media: Ethernet autoselect > status: active > carp: MASTER vhid 201 advbase 1 advskew 1 > carp: BACKUP vhid 202 advbase 5 advskew 100 > laggproto lacp lagghash l2,l3,l4 > laggport: ix1 flags=3D1c > laggport: ix0 flags=3D1c >=20 >=20 > On 31 mar 2014, at 20:42, mxb wrote: >=20 >>=20 >> Hi list, >>=20 >> hopefully this is the right place to have my question regarding CARP = on 10-STABLE. >>=20 >> I have two nodes with following setup(node1): >>=20 >> lagg0: flags=3D8943 = metric 0 mtu 9000 >> = options=3D8407bb >> ether 00:25:90:e3:71:f2 >> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >> nd6 options=3D29 >> media: Ethernet autoselect >> status: active >> carp: BACKUP vhid 201 advbase 1 advskew 1 >> carp: BACKUP vhid 202 advbase 5 advskew 100 >> laggproto lacp lagghash l2,l3,l4 >> laggport: ix1 flags=3D1c >> laggport: ix0 flags=3D1c >>=20 >> net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: >>=20 >> pfsync0: flags=3D41 metric 0 mtu 1500 >> pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off >>=20 >> The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. >> Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as = of setup. >>=20 >> Am I hitting a bug or doing something wrong? >>=20 >> I also have noted that after the pfsync bulk update the demotion = counter never setts to 0, but stays on 480, >> thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? >>=20 >> Regards, >> mxb >>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 11:48:20 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3A59193 for ; Wed, 2 Apr 2014 11:48:20 +0000 (UTC) Received: from mail.sfc.wide.ad.jp (ns.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:143]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 961032D6 for ; Wed, 2 Apr 2014 11:48:20 +0000 (UTC) Received: from Midoris-MacBook-Air.local (pa9b06e.tokynt01.ap.so-net.ne.jp [182.169.176.110]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 657FF278203; Wed, 2 Apr 2014 20:48:18 +0900 (JST) Message-ID: <533BF902.7010506@sfc.wide.ad.jp> Date: Wed, 02 Apr 2014 20:48:18 +0900 From: Midori Kato User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@freebsd.org, "Eggert, Lars" Subject: ECN marking implenetation for dummynet Content-Type: multipart/mixed; boundary="------------050400010400020600020602" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 11:48:20 -0000 This is a multi-part message in MIME format. --------------050400010400020600020602 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Hi FreeBSD developers, I'm Midori Kato. I was working with Lars Eggert about DCTCP. I would like to share our patch for an ECN marking mechanism on dummynet, which I used for DCTCP testing. My implementation allows to set ECN with RED as an AQM scheme. The following command is an example: $ ipfw pipe 9999 config red 1/10/10/0.0 ecn Our implementation includes both DCTCP and RFC 3168 ECN marking methodology. If you are interested in our ECN implemention, I'm very happy to receive your review! (I have already submitted my patch to Luigi and hope he will merge ours in near future.) Regards, -- Midori --------------050400010400020600020602 Content-Type: text/plain; charset=UTF-8; name="dctcpecn-dummy.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dctcpecn-dummy.patch" ZGlmZiAtLWdpdCBhL3NiaW4vaXBmdy9kdW1teW5ldC5jIGIvc2Jpbi9pcGZ3L2R1bW15bmV0 LmMKaW5kZXggMjhkYzJjNy4uY2I2Mjg1MyAxMDA2NDQKLS0tIGEvc2Jpbi9pcGZ3L2R1bW15 bmV0LmMKKysrIGIvc2Jpbi9pcGZ3L2R1bW15bmV0LmMKQEAgLTU2LDYgKzU2LDcgQEAgc3Rh dGljIHN0cnVjdCBfc194IGR1bW15bmV0X3BhcmFtc1tdID0gewogCXsgInNjaGVkX21hc2si LAkJVE9LX1NDSEVEX01BU0sgfSwKIAl7ICJmbG93X21hc2siLAkJVE9LX0ZMT1dfTUFTSyB9 LAogCXsgImRyb3B0YWlsIiwJCVRPS19EUk9QVEFJTCB9LAorCXsgImVjbiIsCQlUT0tfRUNO IH0sCiAJeyAicmVkIiwJCVRPS19SRUQgfSwKIAl7ICJncmVkIiwJCVRPS19HUkVEIH0sCiAJ eyAiYnciLAkJCVRPS19CVyB9LApAQCAtMjM5LDcgKzI0MCw3IEBAIHByaW50X2Zsb3dzZXRf cGFybXMoc3RydWN0IGRuX2ZzICpmcywgY2hhciAqcHJlZml4KQogCWVsc2UKIAkJcGxyWzBd ID0gJ1wwJzsKIAotCWlmIChmcy0+ZmxhZ3MgJiBETl9JU19SRUQpCS8qIFJFRCBwYXJhbWV0 ZXJzICovCisJaWYgKGZzLT5mbGFncyAmIEROX0lTX1JFRCkgewkvKiBSRUQgcGFyYW1ldGVy cyAqLwogCQlzcHJpbnRmKHJlZCwKIAkJICAgICJcblx0ICVjUkVEIHdfcSAlZiBtaW5fdGgg JWQgbWF4X3RoICVkIG1heF9wICVmIiwKIAkJICAgIChmcy0+ZmxhZ3MgJiBETl9JU19HRU5U TEVfUkVEKSA/ICdHJyA6ICcgJywKQEAgLTI0Nyw3ICsyNDgsOSBAQCBwcmludF9mbG93c2V0 X3Bhcm1zKHN0cnVjdCBkbl9mcyAqZnMsIGNoYXIgKnByZWZpeCkKIAkJICAgIGZzLT5taW5f dGgsCiAJCSAgICBmcy0+bWF4X3RoLAogCQkgICAgMS4wICogZnMtPm1heF9wIC8gKGRvdWJs ZSkoMSA8PCBTQ0FMRV9SRUQpKTsKLQllbHNlCisJCWlmIChmcy0+ZmxhZ3MgJiBETl9JU19F Q04pCisJCQlzdHJuY2F0KHJlZCwgIiAoZWNuKSIsIDYpOworCX0gZWxzZQogCQlzcHJpbnRm KHJlZCwgImRyb3B0YWlsIik7CiAKIAlpZiAocHJlZml4WzBdKSB7CkBAIC0xMDQ2LDEzICsx MDQ5LDE3IEBAIGVuZF9tYXNrOgogCQkJfQogCQkJaWYgKChlbmQgPSBzdHJzZXAoJmF2WzBd LCAiLyIpKSkgewogCQkJICAgIGRvdWJsZSBtYXhfcCA9IHN0cnRvZChlbmQsIE5VTEwpOwot CQkJICAgIGlmIChtYXhfcCA+IDEgfHwgbWF4X3AgPD0gMCkKLQkJCQllcnJ4KEVYX0RBVEFF UlIsICIwIDwgbWF4X3AgPD0gMSIpOworCQkJICAgIGlmIChtYXhfcCA+IDEgfHwgbWF4X3Ag PCAwKQorCQkJCWVycngoRVhfREFUQUVSUiwgIjAgPD0gbWF4X3AgPD0gMSIpOwogCQkJICAg IGZzLT5tYXhfcCA9IChpbnQpKG1heF9wICogKDEgPDwgU0NBTEVfUkVEKSk7CiAJCQl9CiAJ CQlhYy0tOyBhdisrOwogCQkJYnJlYWs7CiAKKwkJY2FzZSBUT0tfRUNOOgorCQkJZnMtPmZs YWdzIHw9IEROX0lTX0VDTjsKKwkJCWJyZWFrOworCiAJCWNhc2UgVE9LX0RST1BUQUlMOgog CQkJTkVFRChmcywgImRyb3B0YWlsIGlzIG9ubHkgZm9yIGZsb3dzZXRzIik7CiAJCQlmcy0+ ZmxhZ3MgJj0gfihETl9JU19SRUR8RE5fSVNfR0VOVExFX1JFRCk7CkBAIC0xMTc1LDEzICsx MTgyLDIwIEBAIGVuZF9tYXNrOgogCQkJZXJyeChFWF9EQVRBRVJSLCAiMiA8PSBxdWV1ZSBz aXplIDw9ICVsZCIsIGxpbWl0KTsKIAkgICAgfQogCisJICAgIGlmICgoZnMtPmZsYWdzICYg RE5fSVNfRUNOKSAmJiAhKGZzLT5mbGFncyAmIEROX0lTX1JFRCkpCisJCWVycngoRVhfVVNB R0UsICJlbmFibGUgcmVkL2dyZWQgZm9yIEVDTiIpOworCiAJICAgIGlmIChmcy0+ZmxhZ3Mg JiBETl9JU19SRUQpIHsKIAkJc2l6ZV90IGxlbjsKIAkJaW50IGxvb2t1cF9kZXB0aCwgYXZn X3BrdF9zaXplOwogCi0JCWlmIChmcy0+bWluX3RoID49IGZzLT5tYXhfdGgpCisJCWlmICgh KGZzLT5mbGFncyAmIEROX0lTX0VDTikgJiYgKGZzLT5taW5fdGggPj0gZnMtPm1heF90aCkp CiAJCSAgICBlcnJ4KEVYX0RBVEFFUlIsICJtaW5fdGggJWQgbXVzdCBiZSA8IHRoYW4gbWF4 X3RoICVkIiwKIAkJCWZzLT5taW5fdGgsIGZzLT5tYXhfdGgpOworCQllbHNlIGlmICgoZnMt PmZsYWdzICYgRE5fSVNfRUNOKSAmJiAoZnMtPm1pbl90aCA+IGZzLT5tYXhfdGgpKQorCQkg ICAgZXJyeChFWF9EQVRBRVJSLCAibWluX3RoICVkIG11c3QgYmUgPTwgdGhhbiBtYXhfdGgg JWQiLAorCQkJZnMtPm1pbl90aCwgZnMtPm1heF90aCk7CisKIAkJaWYgKGZzLT5tYXhfdGgg PT0gMCkKIAkJICAgIGVycngoRVhfREFUQUVSUiwgIm1heF90aCBtdXN0IGJlID4gMCIpOwog CmRpZmYgLS1naXQgYS9zYmluL2lwZncvaXBmdzIuaCBiL3NiaW4vaXBmdy9pcGZ3Mi5oCmlu ZGV4IGQ1OTI5MzAuLmI1MDM2MWYgMTAwNjQ0Ci0tLSBhL3NiaW4vaXBmdy9pcGZ3Mi5oCisr KyBiL3NiaW4vaXBmdy9pcGZ3Mi5oCkBAIC0xNjUsNiArMTY1LDcgQEAgZW51bSB0b2tlbnMg ewogCVRPS19CVVJTVCwKIAlUT0tfUkVELAogCVRPS19HUkVELAorCVRPS19FQ04sCiAJVE9L X0RST1BUQUlMLAogCVRPS19QUk9UTywKIAkvKiBkdW1teW5ldCB0b2tlbnMgKi8KZGlmZiAt LWdpdCBhL3N5cy9uZXRpbmV0L2lwX2R1bW15bmV0LmggYi9zeXMvbmV0aW5ldC9pcF9kdW1t eW5ldC5oCmluZGV4IDFjMDkxOTcuLjBiMzdlNzEgMTAwNjQ0Ci0tLSBhL3N5cy9uZXRpbmV0 L2lwX2R1bW15bmV0LmgKKysrIGIvc3lzL25ldGluZXQvaXBfZHVtbXluZXQuaApAQCAtMTAy LDYgKzEwMiw3IEBAIGVudW0gewkvKiB1c2VyIGZsYWdzICovCiAJRE5fUUhUX0hBU0gJPSAw eDAwMDQsCS8qIHFodCBpcyBhIGhhc2ggdGFibGUgKi8KIAlETl9RU0laRV9CWVRFUwk9IDB4 MDAwOCwJLyogcXVldWUgc2l6ZSBpcyBpbiBieXRlcyAqLwogCUROX0hBU19QUk9GSUxFCT0g MHgwMDEwLAkvKiBhIGxpbmsgaGFzIGEgcHJvZmlsZSAqLworCUROX0lTX0VDTgk9IDB4MDA4 MCwKIAlETl9JU19SRUQJPSAweDAwMjAsCiAJRE5fSVNfR0VOVExFX1JFRD0gMHgwMDQwLAog CUROX1BJUEVfQ01ECT0gMHgxMDAwLAkvKiBwaXBlIGNvbmZpZy4uLiAqLwpkaWZmIC0tZ2l0 IGEvc3lzL25ldHBmaWwvaXBmdy9pcF9kbl9nbHVlLmMgYi9zeXMvbmV0cGZpbC9pcGZ3L2lw X2RuX2dsdWUuYwppbmRleCA3ZDdlNjk1Li4wOTU3NThmIDEwMDY0NAotLS0gYS9zeXMvbmV0 cGZpbC9pcGZ3L2lwX2RuX2dsdWUuYworKysgYi9zeXMvbmV0cGZpbC9pcGZ3L2lwX2RuX2ds dWUuYwpAQCAtODMsNiArODMsNyBAQCBzdHJ1Y3QgZG5fZmxvd19zZXQgewogI2RlZmluZSBE Tk9MRF9RU0laRV9JU19CWVRFUyAgIDB4MDAwOCAgLyogcXVldWUgc2l6ZSBpcyBtZWFzdXJl ZCBpbiBieXRlcyAqLwogI2RlZmluZSBETk9MRF9OT0VSUk9SICAgICAgMHgwMDEwICAvKiBk byBub3QgcmVwb3J0IEVOT0JVRlMgb24gZHJvcHMgICovCiAjZGVmaW5lIEROT0xEX0hBU19Q Uk9GSUxFICAgICAgMHgwMDIwICAvKiB0aGUgcGlwZSBoYXMgYSBkZWxheSBwcm9maWxlLiAq LworI2RlZmluZSBETk9MRF9JU19FQ04gICAgICAgMHgwMDQwCiAjZGVmaW5lIEROT0xEX0lT X1BJUEUgICAgICAweDQwMDAKICNkZWZpbmUgRE5PTERfSVNfUVVFVUUgICAgIDB4ODAwMAog CkBAIC0zMzgsNiArMzM5LDggQEAgY29udmVydGZsYWdzMm5ldyhpbnQgc3JjKQogCQlkc3Qg fD0gRE5fSVNfUkVEOwogCWlmIChzcmMgJiBETk9MRF9JU19HRU5UTEVfUkVEKQogCQlkc3Qg fD0gRE5fSVNfR0VOVExFX1JFRDsKKwlpZiAoc3JjICYgRE5PTERfSVNfRUNOKQorCQlkc3Qg fD0gRE5fSVNfRUNOOwogCWlmIChzcmMgJiBETk9MRF9IQVNfUFJPRklMRSkKIAkJZHN0IHw9 IEROX0hBU19QUk9GSUxFOwogCmRpZmYgLS1naXQgYS9zeXMvbmV0cGZpbC9pcGZ3L2lwX2Ru X2lvLmMgYi9zeXMvbmV0cGZpbC9pcGZ3L2lwX2RuX2lvLmMKaW5kZXggOWE0YjQ4Ni4uNDQ2 MjUzYiAxMDA2NDQKLS0tIGEvc3lzL25ldHBmaWwvaXBmdy9pcF9kbl9pby5jCisrKyBiL3N5 cy9uZXRwZmlsL2lwZncvaXBfZG5faW8uYwpAQCAtMzAyLDYgKzMwMiw3IEBAIHJlZF9kcm9w cyAoc3RydWN0IGRuX3F1ZXVlICpxLCBpbnQgbGVuKQogCSAqLwogCiAJc3RydWN0IGRuX2Zz ayAqZnMgPSBxLT5mczsKKwlzdHJ1Y3QgZG5fZnMgKmYgPSAmKHEtPmZzLT5mcyk7CiAJaW50 NjRfdCBwX2IgPSAwOwogCiAJLyogUXVldWUgaW4gYnl0ZXMgb3IgcGFja2V0cz8gKi8KQEAg LTMzNyw2ICszMzgsOCBAQCByZWRfZHJvcHMgKHN0cnVjdCBkbl9xdWV1ZSAqcSwgaW50IGxl bikKIAkJcmV0dXJuICgwKTsJLyogYWNjZXB0IHBhY2tldCAqLwogCX0KIAlpZiAocS0+YXZn ID49IGZzLT5tYXhfdGgpIHsJLyogYXZlcmFnZSBxdWV1ZSA+PSAgbWF4IHRocmVzaG9sZCAq LworCQlpZiAoZnMtPmZzLmZsYWdzICYgRE5fSVNfRUNOKQorCQkJcmV0dXJuICgxKTsKIAkJ aWYgKGZzLT5mcy5mbGFncyAmIEROX0lTX0dFTlRMRV9SRUQpIHsKIAkJCS8qCiAJCQkgKiBB Y2NvcmRpbmcgdG8gR2VudGxlLVJFRCwgaWYgYXZnIGlzIGdyZWF0ZXIgdGhhbgpAQCAtMzUy LDYgKzM1NSw4IEBAIHJlZF9kcm9wcyAoc3RydWN0IGRuX3F1ZXVlICpxLCBpbnQgbGVuKQog CQkJcmV0dXJuICgxKTsKIAkJfQogCX0gZWxzZSBpZiAocS0+YXZnID4gZnMtPm1pbl90aCkg eworCQlpZiAoZnMtPmZzLmZsYWdzICYgRE5fSVNfRUNOKQorCQkJcmV0dXJuICgxKTsKIAkJ LyoKIAkJICogV2UgY29tcHV0ZSBwX2IgdXNpbmcgdGhlIGxpbmVhciBkcm9wcGluZyBmdW5j dGlvbgogCQkgKgkgcF9iID0gY18xICogYXZnIC0gY18yCkBAIC0zODQsNiArMzg5LDcxIEBA IHJlZF9kcm9wcyAoc3RydWN0IGRuX3F1ZXVlICpxLCBpbnQgbGVuKQogfQogCiAvKgorICog RUNOIFByb2Nlc3NpbmcKKyAqIFRoZSBwYXJ0IG9mIHRoaXMgZnVuY3Rpb24gaXMgYWRvcHRl ZCBmcm9tIGFsdHEKKyAqLworc3RhdGljIGludAorZWNuX21hcmsoc3RydWN0IG1idWYqIG0p Cit7CisJc3RydWN0IGlwICppcDsKKwlpcCA9IG10b2QobSwgc3RydWN0IGlwICopOworCisJ c3dpdGNoIChpcC0+aXBfdikgeworCWNhc2UgSVBWRVJTSU9OOgorCXsKKwkJdV9pbnQ4X3Qg b3RvczsKKwkJaW50IHN1bTsKKworCQlpZiAoKGlwLT5pcF90b3MgJiBJUFRPU19FQ05fTUFT SykgPT0gSVBUT1NfRUNOX05PVEVDVCkKKwkJCXJldHVybiAoMCk7CS8qIG5vdC1FQ1QgKi8K KwkJaWYgKChpcC0+aXBfdG9zICYgSVBUT1NfRUNOX01BU0spID09IElQVE9TX0VDTl9DRSkK KwkJCXJldHVybiAoMSk7CS8qIGFscmVhZHkgbWFya2VkICovCisKKwkJLyoKKwkJICogZWNu LWNhcGFibGUgYnV0IG5vdCBtYXJrZWQsCisJCSAqIG1hcmsgQ0UgYW5kIHVwZGF0ZSBjaGVj a3N1bQorCQkgKi8KKwkJb3RvcyA9IGlwLT5pcF90b3M7CisJCWlwLT5pcF90b3MgfD0gSVBU T1NfRUNOX0NFOworCQkvKgorCQkgKiB1cGRhdGUgY2hlY2tzdW0gKGZyb20gUkZDMTYyNCkK KwkJICoJICAgSEMnID0gfih+SEMgKyB+bSArIG0nKQorCQkgKi8KKwkJc3VtID0gfm50b2hz KGlwLT5pcF9zdW0pICYgMHhmZmZmOworCQlzdW0gKz0gKH5vdG9zICYgMHhmZmZmKSArIGlw LT5pcF90b3M7CisJCXN1bSA9IChzdW0gPj4gMTYpICsgKHN1bSAmIDB4ZmZmZik7CisJCXN1 bSArPSAoc3VtID4+IDE2KTsgIC8qIGFkZCBjYXJyeSAqLworCQlpcC0+aXBfc3VtID0gaHRv bnMofnN1bSAmIDB4ZmZmZik7CisJCXJldHVybiAoMSk7CisJfQorI2lmZGVmIElORVQ2CisJ Y2FzZSAoSVBWNl9WRVJTSU9OID4+IDQpOgorCXsKKwkJc3RydWN0IGlwNl9oZHIgKmlwNiA9 IG10b2QobSwgc3RydWN0IGlwNl9oZHIgKik7CisJCXVfaW50MzJfdCBmbG93bGFiZWw7CisK KwkJZmxvd2xhYmVsID0gbnRvaGwoaXA2LT5pcDZfZmxvdyk7CisJCWlmICgoZmxvd2xhYmVs ID4+IDI4KSAhPSA2KQorCQkJcmV0dXJuICgwKTsJLyogdmVyc2lvbiBtaXNtYXRjaCEgKi8K KwkJaWYgKChmbG93bGFiZWwgJiAoSVBUT1NfRUNOX01BU0sgPDwgMjApKSA9PQorCQkgICAg KElQVE9TX0VDTl9OT1RFQ1QgPDwgMjApKQorCQkJcmV0dXJuICgwKTsJLyogbm90LUVDVCAq LworCQlpZiAoKGZsb3dsYWJlbCAmIChJUFRPU19FQ05fTUFTSyA8PCAyMCkpID09CisJCSAg ICAoSVBUT1NfRUNOX0NFIDw8IDIwKSkKKwkJCXJldHVybiAoMSk7CS8qIGFscmVhZHkgbWFy a2VkICovCisJCS8qCisJCSAqIGVjbi1jYXBhYmxlIGJ1dCBub3QgbWFya2VkLCBtYXJrIENF CisJCSAqLworCQlmbG93bGFiZWwgfD0gKElQVE9TX0VDTl9DRSA8PCAyMCk7CisJCWlwNi0+ aXA2X2Zsb3cgPSBodG9ubChmbG93bGFiZWwpOworCQlyZXR1cm4gKDEpOworCX0KKyNlbmRp ZgorCX0KKwlyZXR1cm4gKDApOworfQorCisvKgogICogRW5xdWV1ZSBhIHBhY2tldCBpbiBx LCBzdWJqZWN0IHRvIHNwYWNlIGFuZCBxdWV1ZSBtYW5hZ2VtZW50IHBvbGljeQogICogKHdo b3NlIHBhcmFtZXRlcnMgYXJlIGluIHEtPmZzKS4KICAqIFVwZGF0ZSBzdGF0cyBmb3IgdGhl IHF1ZXVlIGFuZCB0aGUgc2NoZWR1bGVyLgpAQCAtNDE0LDggKzQ4NCwxMyBAQCBkbl9lbnF1 ZXVlKHN0cnVjdCBkbl9xdWV1ZSAqcSwgc3RydWN0IG1idWYqIG0sIGludCBkcm9wKQogCQln b3RvIGRyb3A7CiAJaWYgKGYtPnBsciAmJiByYW5kb20oKSA8IGYtPnBscikKIAkJZ290byBk cm9wOwotCWlmIChmLT5mbGFncyAmIEROX0lTX1JFRCAmJiByZWRfZHJvcHMocSwgbS0+bV9w a3RoZHIubGVuKSkKLQkJZ290byBkcm9wOworCWlmIChmLT5mbGFncyAmIEROX0lTX1JFRCAm JiByZWRfZHJvcHMocSwgbS0+bV9wa3RoZHIubGVuKSkgeworCQlpZiAoZi0+ZmxhZ3MgJiBE Tl9JU19FQ04pIHsKKwkJCWlmICghZWNuX21hcmsobSkpCisJCQkJZ290byBkcm9wOworCQl9 IGVsc2UKKwkJCWdvdG8gZHJvcDsKKwl9CiAJaWYgKGYtPmZsYWdzICYgRE5fUVNJWkVfQllU RVMpIHsKIAkJaWYgKHEtPm5pLmxlbl9ieXRlcyA+IGYtPnFzaXplKQogCQkJZ290byBkcm9w OwpkaWZmIC0tZ2l0IGEvc3lzL25ldHBmaWwvaXBmdy9pcF9kdW1teW5ldC5jIGIvc3lzL25l dHBmaWwvaXBmdy9pcF9kdW1teW5ldC5jCmluZGV4IDRkZTIxNTYuLmFjY2JiMDcgMTAwNjQ0 Ci0tLSBhL3N5cy9uZXRwZmlsL2lwZncvaXBfZHVtbXluZXQuYworKysgYi9zeXMvbmV0cGZp bC9pcGZ3L2lwX2R1bW15bmV0LmMKQEAgLTEwNzAsNyArMTA3MCw3IEBAIGNvbmZpZ19yZWQo c3RydWN0IGRuX2ZzayAqZnMpCiAJZnMtPm1pbl90aCA9IFNDQUxFKGZzLT5mcy5taW5fdGgp OwogCWZzLT5tYXhfdGggPSBTQ0FMRShmcy0+ZnMubWF4X3RoKTsKIAotCWZzLT5jXzEgPSBm cy0+bWF4X3AgLyAoZnMtPmZzLm1heF90aCAtIGZzLT5mcy5taW5fdGgpOworCWZzLT5jXzEg PSBmcy0+bWF4X3AgLyBtYXgoZnMtPmZzLm1heF90aCAtIGZzLT5mcy5taW5fdGgsIDEpOwog CWZzLT5jXzIgPSBTQ0FMRV9NVUwoZnMtPmNfMSwgU0NBTEUoZnMtPmZzLm1pbl90aCkpOwog CiAJaWYgKGZzLT5mcy5mbGFncyAmIEROX0lTX0dFTlRMRV9SRUQpIHsK --------------050400010400020600020602-- From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 13:20:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2873612 for ; Wed, 2 Apr 2014 13:20:58 +0000 (UTC) Received: from mail-lb0-f181.google.com (mail-lb0-f181.google.com [209.85.217.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7790FED7 for ; Wed, 2 Apr 2014 13:20:57 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id c11so147611lbj.26 for ; Wed, 02 Apr 2014 06:20:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=IlnkqK0rw0UbYlajJ8GBFCzXDY9B+z6xxp/15OxdNJA=; b=c875ihT2WMxlpR2sEr+Z2LtAeuL6eFSo1wIYmlvBpP1jRtp+29p4UQ/k52asPjSLWA KQqOgmhKAXtP+7QEn3vUyoVactfU9jyhJzY7ITggWIYgPB806XcM6nUdKzzpt9c0pWgu baZvF4pM6LUsbIpbbF81arh8U+1nCbaI4+b4LTYKaTS/qkl9MmA0xvv9DslQyrlduvDr AV7W2kM8cq3v8c9V5zMePFFaPhrAyGSNfaNb3sd+7weDy9d7fCSJmwSKUxbL1zTXXXiB ZmdPSSgQAt/tkgKgo/bAMHgYRBv6izK4nZfD8FVe172n9P7NQqb2xO/J8l1QN/sC9f5A lzGw== X-Gm-Message-State: ALoCoQme6A4F8n9MP/VTk/zkV8xw2xir5QZy5LdfIFcQ32FXGZ9ipI27nf1/4OorIw1WZRGIOH8d X-Received: by 10.112.198.164 with SMTP id jd4mr18355lbc.92.1396444394380; Wed, 02 Apr 2014 06:13:14 -0700 (PDT) Received: from grey.office.se.prisjakt.nu ([212.16.170.194]) by mx.google.com with ESMTPSA id f9sm1919046laa.8.2014.04.02.06.13.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Apr 2014 06:13:13 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10-STABLE and CARP states From: mxb In-Reply-To: Date: Wed, 2 Apr 2014 15:13:21 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1E20234E-4F81-4BA1-BA57-FADD80F782F4@alumni.chalmers.se> References: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1874) Cc: freebsd-pf@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 13:20:59 -0000 OK, thanks everyone whom replayed. E.g. NONE. The problem seems to be related to LACP trunking. Disabling LACP and configuring trunk in =91loadbalance=92 mode puts all = in desired state (even after reboot). lagg0: flags=3D8943 = metric 0 mtu 9000 = options=3D8407bb ether 00:25:90:e3:71:f2 inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 nd6 options=3D29 media: Ethernet autoselect status: active carp: MASTER vhid 201 advbase 1 advskew 1 carp: BACKUP vhid 202 advbase 5 advskew 100 laggproto loadbalance lagghash l2,l3,l4 laggport: ix1 flags=3D4 laggport: ix0 flags=3D4 vlan2: flags=3D8943 = metric 0 mtu 9000 options=3D303 ether 00:25:90:e3:71:f2 inet 10.11.11.201 netmask 0xffffff00 broadcast 10.11.11.255 inet6 fe80::225:90ff:fee3:71f2%vlan2 prefixlen 64 scopeid 0x6 inet 10.11.12.203 netmask 0xffffff00 broadcast 10.11.12.255 vhid = 12 nd6 options=3D29 media: Ethernet autoselect status: active vlan: 2 parent interface: lagg0 carp: BACKUP vhid 12 advbase 1 advskew 100 //mxb =20 On 2 apr 2014, at 09:35, mxb wrote: >=20 > Moving this to freebsd-pf. >=20 > On 31 mar 2014, at 22:21, mxb wrote: >=20 >>=20 >> Manually setting net.inet.carp.demotion brought BOTH VHIDs in desired = state. >> pfsync bulk update seems to not put everything back as it should. >>=20 >> lagg0: flags=3D8943 = metric 0 mtu 9000 >> = options=3D8407bb >> ether 00:25:90:e3:71:f2 >> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >> nd6 options=3D29 >> media: Ethernet autoselect >> status: active >> carp: MASTER vhid 201 advbase 1 advskew 1 >> carp: BACKUP vhid 202 advbase 5 advskew 100 >> laggproto lacp lagghash l2,l3,l4 >> laggport: ix1 flags=3D1c >> laggport: ix0 flags=3D1c >>=20 >>=20 >> On 31 mar 2014, at 20:42, mxb wrote: >>=20 >>>=20 >>> Hi list, >>>=20 >>> hopefully this is the right place to have my question regarding CARP = on 10-STABLE. >>>=20 >>> I have two nodes with following setup(node1): >>>=20 >>> lagg0: flags=3D8943 = metric 0 mtu 9000 >>> = options=3D8407bb >>> ether 00:25:90:e3:71:f2 >>> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >>> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >>> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >>> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >>> nd6 options=3D29 >>> media: Ethernet autoselect >>> status: active >>> carp: BACKUP vhid 201 advbase 1 advskew 1 >>> carp: BACKUP vhid 202 advbase 5 advskew 100 >>> laggproto lacp lagghash l2,l3,l4 >>> laggport: ix1 flags=3D1c >>> laggport: ix0 flags=3D1c >>>=20 >>> net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: >>>=20 >>> pfsync0: flags=3D41 metric 0 mtu 1500 >>> pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off >>>=20 >>> The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. >>> Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as = of setup. >>>=20 >>> Am I hitting a bug or doing something wrong? >>>=20 >>> I also have noted that after the pfsync bulk update the demotion = counter never setts to 0, but stays on 480, >>> thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? >>>=20 >>> Regards, >>> mxb >>>=20 >>>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Wed Apr 2 16:45:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 603A6A78 for ; Wed, 2 Apr 2014 16:45:46 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D8BC0A1A for ; Wed, 2 Apr 2014 16:45:45 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s32GjAJ8099262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 2 Apr 2014 20:45:10 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s32GitcW099258; Wed, 2 Apr 2014 20:44:55 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 2 Apr 2014 20:44:55 +0400 From: Gleb Smirnoff To: mxb Subject: Re: FreeBSD 10-STABLE and CARP states Message-ID: <20140402164455.GI44326@glebius.int.ru> References: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 16:45:46 -0000 mxb, On Mon, Mar 31, 2014 at 10:21:48PM +0200, mxb wrote: m> Manually setting net.inet.carp.demotion brought BOTH VHIDs in desired state. m> pfsync bulk update seems to not put everything back as it should. m> m> lagg0: flags=8943 metric 0 mtu 9000 m> options=8407bb m> ether 00:25:90:e3:71:f2 m> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 m> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 m> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid 201 m> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid 202 m> nd6 options=29 m> media: Ethernet autoselect m> status: active m> carp: MASTER vhid 201 advbase 1 advskew 1 m> carp: BACKUP vhid 202 advbase 5 advskew 100 m> laggproto lacp lagghash l2,l3,l4 m> laggport: ix1 flags=1c m> laggport: ix0 flags=1c Can you please set net.inet.carp.log to 2 and reproduce the bug again? Then send me last several lines from /var/log/messages or dmesg output. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 00:13:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3FAE5515 for ; Thu, 3 Apr 2014 00:13:42 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 04F86DBB for ; Thu, 3 Apr 2014 00:13:41 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAKKmPFODaFve/2dsb2JhbABZg0FXgwq5KIZkUYEydIIlAQEBAwEBAQEgBCcgCwUUAhgCAg0ZAikBCSYGCAcEARwEh1AIDZEYnBiiDReBKYxwBgEBGzQHgm+BSQSWAYQKkQaDTCExfAgXIg X-IronPort-AV: E=Sophos;i="4.97,783,1389762000"; d="scan'208";a="111210871" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 02 Apr 2014 20:13:34 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1DAB6B4085; Wed, 2 Apr 2014 20:13:35 -0400 (EDT) Date: Wed, 2 Apr 2014 20:13:35 -0400 (EDT) From: Rick Macklem To: k simon Message-ID: <106305249.5158231.1396484015110.JavaMail.root@uoguelph.ca> In-Reply-To: <533B8377.9060209@gmail.com> Subject: Re: 9.2 ixgbe tx queue hang MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: "freebsd-net@freebsd.org >> FreeBSD Net" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 00:13:42 -0000 K Simon wrote: > Hi, Rick, > Does these patches will commit to the stable soon, or I had to > patch > it manually? >=20 Yonghyeon Pyun has already committed the changes for the drivers to head (making them handle 35 mbufs in the chain instead of 32). I'll assume those will be in stable in a couple of weeks. I will be able to commit the one line change that reduces the default setting for if_hw_tsomax in a couple of weeks, so it should be in stable in about 1month. rick > Regards > Simon >=20 > =E4=BA=8E 14-3-28 6:44, Rick Macklem =E5=86=99=E9=81=93: > > Christopher Forgeron wrote: > >> > >> > >> > >> > >> > >> > >> On Wed, Mar 26, 2014 at 9:35 PM, Rick Macklem < > >> rmacklem@uoguelph.ca > >>> wrote: > >> > >> > >> > >> > >> I've suggested in the other thread what you suggested in a recent > >> post...ie. to change the default, at least until the propagation > >> of driver set values is resolved. > >> > >> rick > >> > >> > >> > >> I wonder if we need to worry about propagating values up from the > >> sub-if's - Setting the default in if.c means this is set for all > >> if's, and it's a simple 1 line code change. If a specific 'if' > >> needs > >> a different value, it can be set before ether_attach() is called. > >> > >> > >> I'm more concerned with the equation we use to calculate > >> if_hw_tsomax > >> - Are we considering the right variables? Are we thinking on the > >> wrong OSI layer for headers? > >> > > Well, I'm pragmatic (which means I mostly care about some fix that > > works), > > but it seems to me that: > > - The problem is that some TSO enabled network drivers/hardware can > > only > > handle 32 transmit segments (or 32 mbufs in the chain for the > > TSO packet > > to be transmitted, if that is clearer). > > --> Since the problem is in certain drivers, it seems that those > > drivers > > should be where the long term fix goes. > > --> Since some hardware can't handle more than 32, it seems that > > the > > driver should be able to specify that limit, which > > tcp_output() can > > then apply. > > > > I have an untested patch that does this by adding if_hw_tsomaxseg. > > (The attachment called tsomaxseg.patch.) > > > > Changing if_hw_tsomax or its default value is just a hack that gets > > tcp_output() > > to apply a limit that the driver can then fix to 32 mbufs in the > > chain via > > m_defrag(). > > > > Since if_hw_tsomax (and if_hw_tsomaxseg in the untested patch) > > aren't > > propagated up through lagg, that needs to be fixed. > > (Yet another attached untested patch called lagg.patch.) > > > > As I said before, I don't see these patches getting tested/reviewed > > etc > > in time for 9.3, so I think reducing the default value of > > if_hw_tsomax > > is a reasonable short term hack to work around the problem. > > (And it sounds like Pyun YongHyeon has volunteered to fix many of > > the > > drivers, where the 32 limit isn't a hardware one.) > > > > rick > > > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 01:24:28 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9DF04163 for ; Thu, 3 Apr 2014 01:24:28 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A00133F for ; Thu, 3 Apr 2014 01:24:28 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id q5so1556647wiv.13 for ; Wed, 02 Apr 2014 18:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=AE9yD79/CDES9UQ4Rb5Xo/w+kRJGJcMpHeOY9uT6Bsw=; b=IytboadpEe8Nz/ehYiIST9oozEFl7MLXQwZ/77shKtSi9qcHRE7J/u+j+AkbKweMhI xcG8PnMAahQFX9wTQ+0Czw8imgQjPGdoMSR6GLL67xhAEEV+9a+GbQSODxDWdJodNHpr avSNTM2Zzmbg76klzm7YGhNcNC7kak7uiQR6EjlEVD6GVo+JGMclNoTwRMcESlyJ2lNs 8BJbiMxQlGxuHC9jdyJu6u6ovrXECCNE7JAl9AuNV2XdplUE6q4wuiOoHpEf4W+NyvaG MhM0LuKII0hOx1ptglcIUPPuwcQRy3GA9b1py+KeNEew/52Y9E3OC+rJ8Jrb/u4XQdWE zpGg== MIME-Version: 1.0 X-Received: by 10.194.238.231 with SMTP id vn7mr5240824wjc.76.1396488265661; Wed, 02 Apr 2014 18:24:25 -0700 (PDT) Received: by 10.14.211.134 with HTTP; Wed, 2 Apr 2014 18:24:25 -0700 (PDT) In-Reply-To: <5338FF05.1020302@sfc.wide.ad.jp> References: <5338FF05.1020302@sfc.wide.ad.jp> Date: Wed, 2 Apr 2014 18:24:25 -0700 Message-ID: Subject: Re: DCTCP implementation From: hiren panchasara To: Midori Kato Content-Type: text/plain; charset=UTF-8 Cc: "Eggert, Lars" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 01:24:28 -0000 On Sun, Mar 30, 2014 at 10:37 PM, Midori Kato wrote: > Hi FreeBSD developpers, > > I'm Midori Kato. I'm working on the DCTCP implementation in the FreeBSD with > Lars Eggert. I mail you because I would like to ask you a code review and > testing. The attached patch is not good enough to test our code. Please give > me your message. I will send an ECN marking implmenetation in dummynet and > test scripts personally to you. Midori, First thing I noticed in your dctcp.patch is that you are dropping r256920 which adds a new argument tlen to DELAY_ACK() macro. I also see you removed and re-added that macro definition without any changes to it. Is that intentional? If not, can you please fix that and resend the patch? It is usually a good idea to work on -HEAD for such things so patching/committing is easier. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 11:27:31 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64E2DB0B for ; Thu, 3 Apr 2014 11:27:31 +0000 (UTC) Received: from mail.sfc.wide.ad.jp (ns.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:143]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34188F1C for ; Thu, 3 Apr 2014 11:27:30 +0000 (UTC) Received: from Midoris-MacBook-Air.local (pa9b06e.tokynt01.ap.so-net.ne.jp [182.169.176.110]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 0F44D27820D; Thu, 3 Apr 2014 20:27:28 +0900 (JST) Message-ID: <533D459F.8000403@sfc.wide.ad.jp> Date: Thu, 03 Apr 2014 20:27:27 +0900 From: Midori Kato User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: hiren panchasara Subject: Re: DCTCP implementation References: <5338FF05.1020302@sfc.wide.ad.jp> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Eggert, Lars" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 11:27:31 -0000 Hi Hiren, Yes, I intentionally replace the location of DELAY_ACK() macro. A DCTCP receiver sends an immediate ACK when incoming packets sets CE and non-CE bit by turns. To implement this processing, I prepare the cc_ecnpkt_handler() function. This function calls the DEKAY_ACK() macro to check if this packet is in delayed ack or not. This is why I replace the macro position. Let me know if my explanation is not good enough. Regards, -- Midori (4/3/14, 10:24 AM), hiren panchasara wrote: > On Sun, Mar 30, 2014 at 10:37 PM, Midori Kato wrote: >> Hi FreeBSD developpers, >> >> I'm Midori Kato. I'm working on the DCTCP implementation in the FreeBSD with >> Lars Eggert. I mail you because I would like to ask you a code review and >> testing. The attached patch is not good enough to test our code. Please give >> me your message. I will send an ECN marking implmenetation in dummynet and >> test scripts personally to you. > Midori, > > First thing I noticed in your dctcp.patch is that you are dropping > r256920 which adds a new argument tlen to DELAY_ACK() macro. I also > see you removed and re-added that macro definition without any changes > to it. > > Is that intentional? If not, can you please fix that and resend the patch? > > It is usually a good idea to work on -HEAD for such things so > patching/committing is easier. > > cheers, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 16:00:19 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 090EBEB4 for ; Thu, 3 Apr 2014 16:00:19 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98144DDD for ; Thu, 3 Apr 2014 16:00:18 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id q5so9507781wiv.10 for ; Thu, 03 Apr 2014 09:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JzpuD/pM8S9OicTXakalhNGGrAoWcmq39ZhbwmCyG24=; b=ni3Tn93zdF/58w+fFOR1lD+1aJhgeHnMNy7sjVmuXke5zE+AAvizUkD6QXxlEs76j/ F7D6hRhurJZ6ylHC5sOEObylEKjekJdMmPPuXJ8RHLrW9hRw1sv0nmt+otmnoH3dJd9/ 6VS2YMZYbNpL6kDv1TtRzlLfAC2Sf6RqKd9ZOjm7ZqN+LRBSC2557a+38hkTaF5hwrZb 0lyC9nW+/yR5z8c4BSPiSeD0xzSzMac2JFrLXDUeUHlyn8ozH7noKpbgwBiDHmLoB4kP gUn76lJ3TEJ/gSbjwpVGq/vnVBneGC+AqUxk/Sva4jRxOCPF8MZm5/CNRBYQGl27BqcM 3UFg== MIME-Version: 1.0 X-Received: by 10.194.119.168 with SMTP id kv8mr11674670wjb.41.1396540816951; Thu, 03 Apr 2014 09:00:16 -0700 (PDT) Received: by 10.14.211.134 with HTTP; Thu, 3 Apr 2014 09:00:16 -0700 (PDT) In-Reply-To: <533D459F.8000403@sfc.wide.ad.jp> References: <5338FF05.1020302@sfc.wide.ad.jp> <533D459F.8000403@sfc.wide.ad.jp> Date: Thu, 3 Apr 2014 09:00:16 -0700 Message-ID: Subject: Re: DCTCP implementation From: hiren panchasara To: Midori Kato Content-Type: text/plain; charset=UTF-8 Cc: "Eggert, Lars" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 16:00:19 -0000 On Thu, Apr 3, 2014 at 4:27 AM, Midori Kato wrote: > Hi Hiren, > > Yes, I intentionally replace the location of DELAY_ACK() macro. > A DCTCP receiver sends an immediate ACK when incoming packets sets CE and > non-CE bit by turns. To implement this processing, I prepare the > cc_ecnpkt_handler() function. This function calls the DEKAY_ACK() macro to > check if this packet is in delayed ack or not. This is why I replace the > macro position. That is fine but macro definition is also changed. This is how it looks on -HEAD right now: #define DELAY_ACK(tp, tlen) \ ((!tcp_timer_active(tp, TT_DELACK) && \ (tp->t_flags & TF_RXWIN0SENT) == 0) && \ (tlen <= tp->t_maxopd) && \ (V_tcp_delack_enabled || (tp->t_flags & TF_NEEDSYN))) We need to pass this new argument "tlen" when calling DELAY_ACK() from cc_ecnpkt_handler() function. We can probably pass that to cc_encpkg_handler() while calling it from tcp_do_segment(). I'll make that change and see how it goes. cheers, Hiren > > Regards, > -- Midori > > > (4/3/14, 10:24 AM), hiren panchasara wrote: >> >> On Sun, Mar 30, 2014 at 10:37 PM, Midori Kato >> wrote: >>> >>> Hi FreeBSD developpers, >>> >>> I'm Midori Kato. I'm working on the DCTCP implementation in the FreeBSD >>> with >>> Lars Eggert. I mail you because I would like to ask you a code review and >>> testing. The attached patch is not good enough to test our code. Please >>> give >>> me your message. I will send an ECN marking implmenetation in dummynet >>> and >>> test scripts personally to you. >> >> Midori, >> >> First thing I noticed in your dctcp.patch is that you are dropping >> r256920 which adds a new argument tlen to DELAY_ACK() macro. I also >> see you removed and re-added that macro definition without any changes >> to it. >> >> Is that intentional? If not, can you please fix that and resend the patch? >> >> It is usually a good idea to work on -HEAD for such things so >> patching/committing is easier. >> >> cheers, >> Hiren >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 19:21:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC5102E9 for ; Thu, 3 Apr 2014 19:21:46 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8920E36C for ; Thu, 3 Apr 2014 19:21:45 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 86C5472CF6 for ; Thu, 3 Apr 2014 12:21:45 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 16614-01 for ; Thu, 3 Apr 2014 12:21:45 -0700 (PDT) Received: from [10.0.1.7] (base.kithrup.com [173.164.180.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id E334072CEE for ; Thu, 3 Apr 2014 12:21:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396552905; bh=mLJFcZCUv1y2yj9jgDsePBPvL/qFHyhGPSDxoImbSbg=; h=From:Subject:Date:To; b=rGb4Ku0gMILAKvKtQFN8LKPlFCMEg51y+QK052rWFZMj9h/9xdpaa38DpwN+jT1dO Lmedi5VZ99n+3YGcfJT//O2E8c2+Krie06LBOAF+SdIZq//PUJ62mxBIz1TNY22dED mRI4qOPyVRvlGgLBlkLCBngoiZwLx3N+E0HX+Pz4= From: Sean Fagan Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: More on the whole interruptless networking Message-Id: Date: Thu, 3 Apr 2014 12:21:41 -0700 To: freebsd-net Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 19:21:46 -0000 I've made a series of diffs. I've only tested them in vmware fusion for = now, so that's if_lem that gets involved. I need to track down a system = with igb and try that; I'm much less sure of those changes than I am the = if_em / if_lem changes. Note that, for various reasons, I used git for my source control. One = of those reasons was "I really need to get some experience with git, so = this seems like a good excuse." So pardon me if I produced them = incorrectly. I'm attaching two of the sets for perusal; the others are over on one of = my machines. First up: netdump-10-diffs.txt, attached to this message. This gets the old netdump branch compiling in 10.0. Second, also attached to this message, kernel-netdump-diffs.txt, which = then takes that, and splits out some of the code, so that I could use it = in the real purpose for doing all of this. Third, which is at = http://earth.kithrup.com/~sef/netdump/kernel-kdp-diffs.txt, is an = in-progress port (hence all the output it creates) of the KDP protocol = Apple uses for kernel debugging over ethernet. It's not entirely = functional yet, but it's hard to tell how much of that is this side, and = how much is the gdb side. Which leads me to fourth, at = http://earth.kithrup.com/~sef/netdump/gdb-diffs.txt, which are changes = to gdb (also in-progress, hence it can be very verbose) to attach to the = KDP code in the third patch. Despite my splitting them out like that, I haven't tried doing them = separately. See earlier comment about git for why that is; if anyone = really wants, I can poke around and try to get the diffs from each = stage. Obviously, however, they really are all fairly interconnected. Sean. From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 19:23:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE1A73BE for ; Thu, 3 Apr 2014 19:23:10 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C5F537D for ; Thu, 3 Apr 2014 19:23:09 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 671B872D97 for ; Thu, 3 Apr 2014 12:23:09 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 16614-03 for ; Thu, 3 Apr 2014 12:23:09 -0700 (PDT) Received: from [10.0.1.7] (base.kithrup.com [173.164.180.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 6BD1D72D86 for ; Thu, 3 Apr 2014 12:23:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396552988; bh=e4TCrbhZMKVervkXzfGL1RM8XQEP3PUhWzxU/+qG1yk=; h=From:Subject:Date:To; b=bgyFinXDh2T9l47rZPkcc/yZ8HREzB8zNs6nslDv8wuzdSisbEtX7kyY7PBgaRySL NtCxuNnCZY8xg8gXHzQoIRdNVgPm3AQzBrxnXu2xMyrcZPRh+Mrq7/msSF1uQTybyh jvQ69fIhyQg1UrYBqAK0RsEjhvwRJDbPNGbjPOfQ= From: Sean Fagan Content-Type: multipart/mixed; boundary="Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3" Subject: Re: More on the whole interruptless networking Message-Id: <16E0C844-B302-4B61-A250-CA12CABDE476@ixsystems.com> Date: Thu, 3 Apr 2014 12:23:05 -0700 To: freebsd-net Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 19:23:10 -0000 --Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii No, that icon is not attach, it's send. Grrr. Attached diffs that I promised and didn't attach before hitting the = wrong button. --Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3 Content-Disposition: attachment; filename=netdump-10-diffs.txt Content-Type: text/plain; name="netdump-10-diffs.txt" Content-Transfer-Encoding: quoted-printable diff --git a/etc/defaults/rc.conf b/etc/defaults/rc.conf index a34f0c1..e455200 100644 --- a/etc/defaults/rc.conf +++ b/etc/defaults/rc.conf @@ -274,6 +274,14 @@ ctld_enable=3D"NO" # CAM Target Layer / = iSCSI target daemon. local_unbound_enable=3D"NO" # local caching resolver =20 # +# netdumpsrv configuration. +# +netdumpsrv_enable=3D"NO" # Run netdumpsrv. +netdumpsrv_program=3D"/usr/sbin/netdumpsrv" # Path to netdumpsrv. +netdumpsrv_pidfile=3D"/var/run/netdumpsrv.pid" # Path to pidfile for = netdumpsrv. +#netdumpsrv_flags=3D"" # Use this for flags -a, -D, -d, -i + +# # kerberos. Do not run the admin daemons on slave servers # kerberos5_server_enable=3D"NO" # Run a kerberos 5 master server (or = NO). diff --git a/etc/rc.d/netdumpsrv b/etc/rc.d/netdumpsrv new file mode 100644 index 0000000..3ae47e6 --- /dev/null +++ b/etc/rc.d/netdumpsrv @@ -0,0 +1,34 @@ +#!/bin/sh +# +# PROVIDE: netdumpsrv +# BEFORE: LOGIN +# KEYWORD: shutdown + +. /etc/rc.subr + +name=3D"netdumpsrv" +rcvar=3D`set_rcvar` + +command=3D"/usr/sbin/${name}" +pidfile=3D"/var/run/${name}.pid" + +load_rc_config $name + +run_rc_command "$1" +#!/bin/sh +# +# PROVIDE: netdumpsrv +# BEFORE: LOGIN +# KEYWORD: shutdown + +. /etc/rc.subr + +name=3D"netdumpsrv" +rcvar=3D`set_rcvar` + +command=3D"/usr/sbin/${name}" +pidfile=3D"/var/run/${name}.pid" + +load_rc_config $name + +run_rc_command "$1" diff --git a/sys/amd64/amd64/minidump_machdep.c = b/sys/amd64/amd64/minidump_machdep.c index 0ee8bcf..1e94bfc 100644 --- a/sys/amd64/amd64/minidump_machdep.c +++ b/sys/amd64/amd64/minidump_machdep.c @@ -319,13 +319,20 @@ minidumpsys(struct dumperinfo *di) } dumpsize +=3D PAGE_SIZE; =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D E2BIG; - goto fail; + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D E2BIG; + goto fail; + } + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; } - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; progress =3D dumpsize; =20 /* Initialize mdhdr */ diff --git a/sys/amd64/conf/GENERIC b/sys/amd64/conf/GENERIC index 3b48f0f..e1aabb4 100644 --- a/sys/amd64/conf/GENERIC +++ b/sys/amd64/conf/GENERIC @@ -76,7 +76,8 @@ options INCLUDE_CONFIG_FILE # Include this = file in kernel # Debugging support. Always need this: options KDB # Enable kernel debugger = support. options KDB_TRACE # Print a stack trace for a = panic. - +options DDB # Support DDB. +options NETDUMP_CLIENT # Network core-dump # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor = Kernel =20 diff --git a/sys/arm/arm/dump_machdep.c b/sys/arm/arm/dump_machdep.c index e8ba5768..0a88114 100644 --- a/sys/arm/arm/dump_machdep.c +++ b/sys/arm/arm/dump_machdep.c @@ -311,13 +311,20 @@ dumpsys(struct dumperinfo *di) dumpsize +=3D fileofs; hdrgap =3D fileofs - DEV_ALIGN(hdrsz); =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D ENOSPC; - goto fail; + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D ENOSPC; + goto fail; + } + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; } - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; =20 mkdumpheader(&kdh, KERNELDUMPMAGIC, KERNELDUMP_ARM_VERSION, = dumpsize, di->blocksize); =20 diff --git a/sys/arm/arm/minidump_machdep.c = b/sys/arm/arm/minidump_machdep.c index a858709..4779a54 100644 --- a/sys/arm/arm/minidump_machdep.c +++ b/sys/arm/arm/minidump_machdep.c @@ -284,14 +284,21 @@ minidumpsys(struct dumperinfo *di) =20 dumpsize +=3D PAGE_SIZE; =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D ENOSPC; - goto fail; - } + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D ENOSPC; + goto fail; + } =20 - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; + } progress =3D dumpsize; =20 /* Initialize mdhdr */ diff --git a/sys/conf/files b/sys/conf/files index 27d9c7e..46d0d27 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -3261,6 +3261,7 @@ netinet/ip_ipsec.c optional inet = ipsec netinet/ip_mroute.c optional mrouting inet netinet/ip_options.c optional inet netinet/ip_output.c optional inet +netinet/netdump_client.c optional inet netdump_client netinet/raw_ip.c optional inet | inet6 netinet/cc/cc.c optional inet | inet6 netinet/cc/cc_newreno.c optional inet | inet6 diff --git a/sys/conf/options b/sys/conf/options index a4c785e..5cc03db 100644 --- a/sys/conf/options +++ b/sys/conf/options @@ -295,6 +295,10 @@ NFS_ROOT opt_nfsroot.h # SMB/CIFS requester NETSMB opt_netsmb.h =20 +# Netdump client kernel support +NETDUMP_CLIENT opt_netdump.h +NETDUMP_CLIENT_DEBUG opt_netdump.h + # Options used only in subr_param.c. HZ opt_param.h MAXFILES opt_param.h diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c index 16e1d6f..b375388 100644 --- a/sys/dev/e1000/if_em.c +++ b/sys/dev/e1000/if_em.c @@ -75,6 +75,9 @@ #include #include #include +#ifdef NETDUMP_CLIENT +#include +#endif #include #include =20 @@ -87,6 +90,27 @@ #include "e1000_82571.h" #include "if_em.h" =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) + +#define EM_CORE_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_CORE_LOCK(adapter); = \ +} while (0) +#define EM_CORE_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_CORE_UNLOCK(adapter); = \ +} while (0) +#define EM_TX_LOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_TX_LOCK(txr); = \ +} while (0) +#define EM_TX_UNLOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_TX_UNLOCK(txr); = \ +} while (0) + +#endif + /********************************************************************* * Set this to one to display debug statistics *********************************************************************/ @@ -300,14 +324,32 @@ static int = em_sysctl_eee(SYSCTL_HANDLER_ARGS); =20 static __inline void em_rx_discard(struct rx_ring *, int); =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int _em_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, + int count, int locking); static poll_handler_t em_poll; -#endif /* POLLING */ +#endif +#ifdef NETDUMP_CLIENT +static poll_handler_t em_poll_unlocked; +static ndumplock_handler_t em_ndump_disable_intr; +static ndumplock_handler_t em_ndump_enable_intr; +#endif =20 /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ =20 +#ifdef NETDUMP_CLIENT + +static struct netdump_methods em_ndump_methods =3D { + .ne_poll_locked =3D em_poll, + .ne_poll_unlocked =3D em_poll_unlocked, + .ne_disable_intr =3D em_ndump_disable_intr, + .ne_enable_intr =3D em_ndump_enable_intr +}; + +#endif + static device_method_t em_methods[] =3D { /* Device interface */ DEVMETHOD(device_probe, em_probe), @@ -1416,14 +1458,14 @@ em_init(void *arg) } =20 =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) /********************************************************************* * * Legacy polling routine: note this only works with single queue * *********************************************************************/ static int -em_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +_em_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, int count, int = locking) { struct adapter *adapter =3D ifp->if_softc; struct tx_ring *txr =3D adapter->tx_rings; @@ -1431,9 +1473,9 @@ em_poll(struct ifnet *ifp, enum poll_cmd cmd, int = count) u32 reg_icr; int rx_done; =20 - EM_CORE_LOCK(adapter); + EM_CORE_LOCK_COND(adapter, locking); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0) { - EM_CORE_UNLOCK(adapter); + EM_CORE_UNLOCK_COND(adapter, locking); return (0); } =20 @@ -1447,11 +1489,11 @@ em_poll(struct ifnet *ifp, enum poll_cmd cmd, = int count) em_local_timer, adapter); } } - EM_CORE_UNLOCK(adapter); + EM_CORE_UNLOCK_COND(adapter, locking); =20 em_rxeof(rxr, count, &rx_done); =20 - EM_TX_LOCK(txr); + EM_TX_LOCK_COND(txr, locking); em_txeof(txr); #ifdef EM_MULTIQUEUE if (!drbr_empty(ifp, txr->br)) @@ -1460,12 +1502,49 @@ em_poll(struct ifnet *ifp, enum poll_cmd cmd, = int count) if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) em_start_locked(ifp, txr); #endif - EM_TX_UNLOCK(txr); + EM_TX_UNLOCK_COND(txr, locking); =20 return (rx_done); } -#endif /* DEVICE_POLLING */ =20 +static int +em_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_em_poll_generic(ifp, cmd, count, 1)); +} +#endif /* !DEVICE_POLLING && !NETDUMP_CLIENT */ + +#ifdef NETDUMP_CLIENT +static int +em_poll_unlocked(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_em_poll_generic(ifp, cmd, count, 0)); +} + +static void +em_ndump_disable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + EM_CORE_LOCK(adapter); + em_disable_intr(adapter); + EM_CORE_UNLOCK(adapter); +} + +static void +em_ndump_enable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + EM_CORE_LOCK(adapter); + em_enable_intr(adapter); + EM_CORE_UNLOCK(adapter); +} +#endif /* !NETDUMP_CLIENT */ =20 /********************************************************************* * @@ -2989,6 +3068,9 @@ em_setup_interface(device_t dev, struct adapter = *adapter) ifp->if_qflush =3D em_qflush; #else ifp->if_start =3D em_start; +#ifdef NETDUMP_CLIENT + ifp->if_ndumpfuncs =3D &em_ndump_methods; +#endif IFQ_SET_MAXLEN(&ifp->if_snd, adapter->num_tx_desc - 1); ifp->if_snd.ifq_drv_maxlen =3D adapter->num_tx_desc - 1; IFQ_SET_READY(&ifp->if_snd); diff --git a/sys/dev/e1000/if_igb.c b/sys/dev/e1000/if_igb.c index 07a9d3e..93d1add 100644 --- a/sys/dev/e1000/if_igb.c +++ b/sys/dev/e1000/if_igb.c @@ -39,6 +39,7 @@ #ifdef HAVE_KERNEL_OPTION_HEADERS #include "opt_device_polling.h" #include "opt_altq.h" +#include "opt_netdump.h" #endif =20 #include @@ -80,6 +81,9 @@ #include #include #include +#ifdef NETDUMP_CLIENT +#include +#endif #include #include #include @@ -93,6 +97,35 @@ #include "e1000_82575.h" #include "if_igb.h" =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) + +#define IGB_CORE_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_CORE_LOCK(adapter); = \ +} while (0) +#define IGB_CORE_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_CORE_UNLOCK(adapter); = \ +} while (0) +#define IGB_RX_LOCK_COND(rxr, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_RX_LOCK(rxr); = \ +} while (0) +#define IGB_RX_UNLOCK_COND(rxr, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_RX_UNLOCK(rxr); = \ +} while (0) +#define IGB_TX_LOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_TX_LOCK(txr); = \ +} while (0) +#define IGB_TX_UNLOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + IGB_TX_UNLOCK(txr); = \ +} while (0) + +#endif + /********************************************************************* * Set this to one to display debug statistics *********************************************************************/ @@ -240,7 +273,8 @@ static __inline void igb_rx_discard(struct = rx_ring *, int); static __inline void igb_rx_input(struct rx_ring *, struct ifnet *, struct mbuf *, u32); =20 -static bool igb_rxeof(struct igb_queue *, int, int *); +static bool _igb_rxeof_generic(struct igb_queue *, int, int *, int); +#define igb_rxeof(q, c, d) _igb_rxeof_generic(q, c, d, 1) static void igb_rx_checksum(u32, struct mbuf *, u32); static int igb_tx_ctx_setup(struct tx_ring *, struct mbuf *, u32 *, u32 *); @@ -289,14 +323,32 @@ static int = igb_set_flowcntl(SYSCTL_HANDLER_ARGS); static int igb_sysctl_dmac(SYSCTL_HANDLER_ARGS); static int igb_sysctl_eee(SYSCTL_HANDLER_ARGS); =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int _igb_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, + int count, int locking); static poll_handler_t igb_poll; -#endif /* POLLING */ +#endif +#ifdef NETDUMP_CLIENT +static poll_handler_t igb_poll_unlocked; +static ndumplock_handler_t igb_ndump_disable_intr; +static ndumplock_handler_t igb_ndump_enable_intr; +#endif =20 /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ =20 +#ifdef NETDUMP_CLIENT + +static struct netdump_methods igb_ndump_methods =3D { + .ne_poll_locked =3D igb_poll, + .ne_poll_unlocked =3D igb_poll_unlocked, + .ne_disable_intr =3D igb_ndump_disable_intr, + .ne_enable_intr =3D igb_ndump_enable_intr +}; + +#endif + static device_method_t igb_methods[] =3D { /* Device interface */ DEVMETHOD(device_probe, igb_probe), @@ -1524,7 +1576,7 @@ igb_irq_fast(void *arg) return FILTER_HANDLED; } =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) #if __FreeBSD_version >=3D 800000 #define POLL_RETURN_COUNT(a) (a) static int @@ -1532,7 +1584,7 @@ static int #define POLL_RETURN_COUNT(a) static void #endif -igb_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +_igb_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, int count, int = locking) { struct adapter *adapter =3D ifp->if_softc; struct igb_queue *que; @@ -1541,9 +1593,9 @@ igb_poll(struct ifnet *ifp, enum poll_cmd cmd, int = count) u32 loop =3D IGB_MAX_LOOP; bool more; =20 - IGB_CORE_LOCK(adapter); + IGB_CORE_LOCK_COND(adapter, locking); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0) { - IGB_CORE_UNLOCK(adapter); + IGB_CORE_UNLOCK_COND(adapter, locking); return POLL_RETURN_COUNT(rx_done); } =20 @@ -1556,15 +1608,15 @@ igb_poll(struct ifnet *ifp, enum poll_cmd cmd, = int count) if (reg_icr & E1000_ICR_RXO) adapter->rx_overruns++; } - IGB_CORE_UNLOCK(adapter); + IGB_CORE_UNLOCK_COND(adapter, locking); =20 for (int i =3D 0; i < adapter->num_queues; i++) { que =3D &adapter->queues[i]; txr =3D que->txr; =20 - igb_rxeof(que, count, &rx_done); + _igb_rxeof_generic(que, count, &rx_done, locking); =20 - IGB_TX_LOCK(txr); + IGB_TX_LOCK_COND(txr, locking); do { more =3D igb_txeof(txr); } while (loop-- && more); @@ -1575,12 +1627,50 @@ igb_poll(struct ifnet *ifp, enum poll_cmd cmd, = int count) if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) igb_start_locked(txr, ifp); #endif - IGB_TX_UNLOCK(txr); + IGB_TX_UNLOCK_COND(txr, locking); } =20 return POLL_RETURN_COUNT(rx_done); } -#endif /* DEVICE_POLLING */ +#endif /* DEVICE_POLLING || NETDUMP_CLIENT */ + +static int +igb_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_igb_poll_generic(ifp, cmd, count, 1)); +} + +#ifdef NETDUMP_CLIENT +static int +igb_poll_unlocked(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_igb_poll_generic(ifp, cmd, count, 0)); +} + +static void +igb_ndump_disable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IGB_CORE_LOCK(adapter); + igb_disable_intr(adapter); + IGB_CORE_UNLOCK(adapter); +} + +static void +igb_ndump_enable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IGB_CORE_LOCK(adapter); + igb_enable_intr(adapter); + IGB_CORE_UNLOCK(adapter); +} +#endif /* !NETDUMP_CLIENT */ =20 /********************************************************************* * @@ -3123,6 +3213,10 @@ igb_setup_interface(device_t dev, struct adapter = *adapter) IFQ_SET_READY(&ifp->if_snd); #endif =20 +#ifdef NETDUMP_CLIENT + ifp->if_ndumpfuncs =3D &igb_ndump_methods; +#endif + ether_ifattach(ifp, adapter->hw.mac.addr); =20 ifp->if_capabilities =3D ifp->if_capenable =3D 0; @@ -4811,7 +4905,7 @@ igb_rx_input(struct rx_ring *rxr, struct ifnet = *ifp, struct mbuf *m, u32 ptype) * Return TRUE if more to clean, FALSE otherwise *********************************************************************/ static bool -igb_rxeof(struct igb_queue *que, int count, int *done) +_igb_rxeof_generic(struct igb_queue *que, int count, int *done, int = locking) { struct adapter *adapter =3D que->adapter; struct rx_ring *rxr =3D que->rxr; @@ -4822,7 +4916,7 @@ igb_rxeof(struct igb_queue *que, int count, int = *done) u32 ptype, staterr =3D 0; union e1000_adv_rx_desc *cur; =20 - IGB_RX_LOCK(rxr); + IGB_RX_LOCK_COND(rxr, locking); /* Sync the ring. */ bus_dmamap_sync(rxr->rxdma.dma_tag, rxr->rxdma.dma_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); @@ -5007,7 +5101,7 @@ next_desc: if (done !=3D NULL) *done +=3D rxdone; =20 - IGB_RX_UNLOCK(rxr); + IGB_RX_UNLOCK_COND(rxr, locking); return ((staterr & E1000_RXD_STAT_DD) ? TRUE : FALSE); } =20 diff --git a/sys/dev/e1000/if_lem.c b/sys/dev/e1000/if_lem.c index 57e88a4..d05dd00 100644 --- a/sys/dev/e1000/if_lem.c +++ b/sys/dev/e1000/if_lem.c @@ -34,6 +34,7 @@ =20 #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_netdump.h" =20 #ifdef HAVE_KERNEL_OPTION_HEADERS #include "opt_device_polling.h" @@ -72,6 +73,7 @@ #include #include #include +#include #include #include =20 @@ -83,6 +85,35 @@ #include "e1000_api.h" #include "if_lem.h" =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) + +#define EM_CORE_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_CORE_LOCK(adapter); = \ +} while (0) +#define EM_CORE_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_CORE_UNLOCK(adapter); = \ +} while (0) +#define EM_RX_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_RX_LOCK(adapter); = \ +} while (0) +#define EM_RX_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_RX_UNLOCK(adapter); = \ +} while (0) +#define EM_TX_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_TX_LOCK(adapter); = \ +} while (0) +#define EM_TX_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + EM_TX_UNLOCK(adapter); = \ +} while (0) + +#endif + /********************************************************************* * Legacy Em Driver version: *********************************************************************/ @@ -195,7 +226,8 @@ static void lem_txeof(struct adapter *); static void lem_tx_purge(struct adapter *); static int lem_allocate_receive_structures(struct adapter *); static int lem_allocate_transmit_structures(struct adapter *); -static bool lem_rxeof(struct adapter *, int, int *); +static bool _lem_rxeof_generic(struct adapter *, int, int *, int); +#define lem_rxeof(a, c, d) _lem_rxeof_generic(a, c, d, 1) #ifndef __NO_STRICT_ALIGNMENT static int lem_fixup_rx(struct adapter *); #endif @@ -247,14 +279,32 @@ static void lem_handle_link(void *context, = int pending); static void lem_add_rx_process_limit(struct adapter *, const char *, const char *, int *, int); =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int _lem_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, + int count, int locking); static poll_handler_t lem_poll; -#endif /* POLLING */ +#endif +#ifdef NETDUMP_CLIENT +static poll_handler_t lem_poll_unlocked; +static ndumplock_handler_t lem_ndump_disable_intr; +static ndumplock_handler_t lem_ndump_enable_intr; +#endif =20 /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ =20 +#ifdef NETDUMP_CLIENT + +static struct netdump_methods lem_ndump_methods =3D { + .ne_poll_locked =3D lem_poll, + .ne_poll_unlocked =3D lem_poll_unlocked, + .ne_disable_intr =3D lem_ndump_disable_intr, + .ne_enable_intr =3D lem_ndump_enable_intr +}; + +#endif + static device_method_t lem_methods[] =3D { /* Device interface */ DEVMETHOD(device_probe, lem_probe), @@ -1225,21 +1275,21 @@ lem_init(void *arg) } =20 =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) /********************************************************************* * * Legacy polling routine =20 * *********************************************************************/ static int -lem_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +_lem_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, int count, int = locking) { struct adapter *adapter =3D ifp->if_softc; u32 reg_icr, rx_done =3D 0; =20 - EM_CORE_LOCK(adapter); + EM_CORE_LOCK_COND(adapter, locking); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0) { - EM_CORE_UNLOCK(adapter); + EM_CORE_UNLOCK_COND(adapter, locking); return (rx_done); } =20 @@ -1253,18 +1303,56 @@ lem_poll(struct ifnet *ifp, enum poll_cmd cmd, = int count) lem_local_timer, adapter); } } - EM_CORE_UNLOCK(adapter); + EM_CORE_UNLOCK_COND(adapter, locking); =20 - lem_rxeof(adapter, count, &rx_done); + _lem_rxeof_generic(adapter, count, &rx_done, locking); =20 - EM_TX_LOCK(adapter); + EM_TX_LOCK_COND(adapter, locking); lem_txeof(adapter); if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) lem_start_locked(ifp); - EM_TX_UNLOCK(adapter); + EM_TX_UNLOCK_COND(adapter, locking); return (rx_done); } -#endif /* DEVICE_POLLING */ + +static int +lem_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_lem_poll_generic(ifp, cmd, count, 1)); +} +#endif /* !DEVICE_POLLING && !NETDUMP_CLIENT */ + +#ifdef NETDUMP_CLIENT +static int +lem_poll_unlocked(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_lem_poll_generic(ifp, cmd, count, 0)); +} + +static void +lem_ndump_disable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + EM_CORE_LOCK(adapter); + lem_disable_intr(adapter); + EM_CORE_UNLOCK(adapter); +} + +static void +lem_ndump_enable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + EM_CORE_LOCK(adapter); + lem_enable_intr(adapter); + EM_CORE_UNLOCK(adapter); +} +#endif /* !NETDUMP_CLIENT */ =20 /********************************************************************* * @@ -2363,6 +2451,9 @@ lem_setup_interface(device_t dev, struct adapter = *adapter) ifp->if_flags =3D IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; ifp->if_ioctl =3D lem_ioctl; ifp->if_start =3D lem_start; +#ifdef NETDUMP_CLIENT + ifp->if_ndumpfuncs =3D &lem_ndump_methods; +#endif IFQ_SET_MAXLEN(&ifp->if_snd, adapter->num_tx_desc - 1); ifp->if_snd.ifq_drv_maxlen =3D adapter->num_tx_desc - 1; IFQ_SET_READY(&ifp->if_snd); @@ -3438,7 +3529,7 @@ lem_free_receive_structures(struct adapter = *adapter) * For polling we also now return the number of cleaned packets *********************************************************************/ static bool -lem_rxeof(struct adapter *adapter, int count, int *done) +_lem_rxeof_generic(struct adapter *adapter, int count, int *done, int = locking) { struct ifnet *ifp =3D adapter->ifp; struct mbuf *mp; @@ -3447,7 +3538,7 @@ lem_rxeof(struct adapter *adapter, int count, int = *done) int i, rx_sent =3D 0; struct e1000_rx_desc *current_desc; =20 - EM_RX_LOCK(adapter); + EM_RX_LOCK_COND(adapter, locking); i =3D adapter->next_rx_desc_to_check; current_desc =3D &adapter->rx_desc_base[i]; bus_dmamap_sync(adapter->rxdma.dma_tag, adapter->rxdma.dma_map, @@ -3461,7 +3552,7 @@ lem_rxeof(struct adapter *adapter, int count, int = *done) if (!((current_desc->status) & E1000_RXD_STAT_DD)) { if (done !=3D NULL) *done =3D rx_sent; - EM_RX_UNLOCK(adapter); + EM_RX_UNLOCK_COND(adapter, locking); return (FALSE); } =20 @@ -3601,9 +3692,9 @@ discard: /* Call into the stack */ if (m !=3D NULL) { adapter->next_rx_desc_to_check =3D i; - EM_RX_UNLOCK(adapter); + EM_RX_UNLOCK_COND(adapter, locking); (*ifp->if_input)(ifp, m); - EM_RX_LOCK(adapter); + EM_RX_LOCK_COND(adapter, locking); rx_sent++; i =3D adapter->next_rx_desc_to_check; } @@ -3617,7 +3708,7 @@ discard: E1000_WRITE_REG(&adapter->hw, E1000_RDT(0), i); if (done !=3D NULL) *done =3D rx_sent; - EM_RX_UNLOCK(adapter); + EM_RX_UNLOCK_COND(adapter, locking); return ((status & E1000_RXD_STAT_DD) ? TRUE : FALSE); } =20 diff --git a/sys/dev/ixgb/if_ixgb.c b/sys/dev/ixgb/if_ixgb.c index 2e62c1c..32b154a 100644 --- a/sys/dev/ixgb/if_ixgb.c +++ b/sys/dev/ixgb/if_ixgb.c @@ -35,10 +35,24 @@ POSSIBILITY OF SUCH DAMAGE. =20 #ifdef HAVE_KERNEL_OPTION_HEADERS #include "opt_device_polling.h" +#include "opt_netdump.h" #endif =20 #include =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) + +#define IXGB_LOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGB_LOCK(adapter); = \ +} while (0) +#define IXGB_UNLOCK_COND(adapter, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGB_UNLOCK(adapter); = \ +} while (0) + +#endif + /********************************************************************* * Set this to one to display debug statistics *********************************************************************/ @@ -145,14 +159,32 @@ static int ixgb_dma_malloc(struct adapter *, bus_size_t, struct ixgb_dma_alloc *, int); static void ixgb_dma_free(struct adapter *, struct ixgb_dma_alloc = *); -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int _ixgb_poll_generic(struct ifnet *ifp, enum poll_cmd = cmd, + int count, int locking); static poll_handler_t ixgb_poll; #endif +#ifdef NETDUMP_CLIENT +static poll_handler_t ixgb_poll_unlocked; +static ndumplock_handler_t ixgb_ndump_disable_intr; +static ndumplock_handler_t ixgb_ndump_enable_intr; +#endif =20 /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ =20 +#ifdef NETDUMP_CLIENT + +static struct netdump_methods ixgb_ndump_methods =3D { + .ne_poll_locked =3D ixgb_poll, + .ne_poll_unlocked =3D ixgb_poll_unlocked, + .ne_disable_intr =3D ixgb_ndump_disable_intr, + .ne_enable_intr =3D ixgb_ndump_enable_intr +}; + +#endif + static device_method_t ixgb_methods[] =3D { /* Device interface */ DEVMETHOD(device_probe, ixgb_probe), @@ -751,7 +783,7 @@ ixgb_init(void *arg) return; } =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) static int ixgb_poll_locked(struct ifnet * ifp, enum poll_cmd cmd, int count) { @@ -777,18 +809,57 @@ ixgb_poll_locked(struct ifnet * ifp, enum poll_cmd = cmd, int count) } =20 static int -ixgb_poll(struct ifnet * ifp, enum poll_cmd cmd, int count) +_ixgb_poll_generic(struct ifnet * ifp, enum poll_cmd cmd, int count, + int locking) { struct adapter *adapter =3D ifp->if_softc; int rx_npkts =3D 0; =20 - IXGB_LOCK(adapter); + IXGB_LOCK_COND(adapter, locking); if (ifp->if_drv_flags & IFF_DRV_RUNNING) rx_npkts =3D ixgb_poll_locked(ifp, cmd, count); - IXGB_UNLOCK(adapter); + IXGB_UNLOCK_COND(adapter, locking); return (rx_npkts); } -#endif /* DEVICE_POLLING */ + +static int +ixgb_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_ixgb_poll_generic(ifp, cmd, count, 1)); +} +#endif /* !DEVICE_POLLING && !NETDUMP_CLIENT */ + +#ifdef NETDUMP_CLIENT +static int +ixgb_poll_unlocked(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_ixgb_poll_generic(ifp, cmd, count, 0)); +} + +static void +ixgb_ndump_disable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IXGB_LOCK(adapter); + ixgb_disable_intr(adapter); + IXGB_UNLOCK(adapter); +} + +static void +ixgb_ndump_enable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IXGB_LOCK(adapter); + ixgb_enable_intr(adapter); + IXGB_UNLOCK(adapter); +} +#endif /* !NETDUMP_CLIENT */ =20 /********************************************************************* * @@ -1356,6 +1427,9 @@ ixgb_setup_interface(device_t dev, struct adapter = * adapter) ifp->if_ioctl =3D ixgb_ioctl; ifp->if_start =3D ixgb_start; ifp->if_snd.ifq_maxlen =3D adapter->num_tx_desc - 1; +#ifdef NETDUMP_CLIENT + ifp->if_ndumpfuncs =3D &ixgb_ndump_methods; +#endif =20 #if __FreeBSD_version < 500000 ether_ifattach(ifp, ETHER_BPF_SUPPORTED); diff --git a/sys/dev/ixgb/if_ixgb.h b/sys/dev/ixgb/if_ixgb.h index 4e88db7..a75b7b7 100644 --- a/sys/dev/ixgb/if_ixgb.h +++ b/sys/dev/ixgb/if_ixgb.h @@ -60,6 +60,9 @@ POSSIBILITY OF SUCH DAMAGE. #include #include #include +#ifdef NETDUMP_CLIENT +#include +#endif #include #include =20 diff --git a/sys/dev/ixgbe/ixgbe.c b/sys/dev/ixgbe/ixgbe.c index 581dcc6..c1e88c7 100644 --- a/sys/dev/ixgbe/ixgbe.c +++ b/sys/dev/ixgbe/ixgbe.c @@ -35,8 +35,34 @@ =20 #include "opt_inet.h" #include "opt_inet6.h" +#ifdef HAVE_KERNEL_OPTION_HEADERS +#include "opt_device_polling.h" +#include "opt_netdump.h" +#endif + #include "ixgbe.h" =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) + +#define IXGBE_RX_LOCK_COND(rxr, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGBE_RX_LOCK(rxr); = \ +} while (0) +#define IXGBE_RX_UNLOCK_COND(rxr, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGBE_RX_UNLOCK(rxr); = \ +} while (0) +#define IXGBE_TX_LOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGBE_TX_LOCK(txr); = \ +} while (0) +#define IXGBE_TX_UNLOCK_COND(txr, locking) do { = \ + if ((locking) !=3D 0) = \ + IXGBE_TX_UNLOCK(txr); = \ +} while (0) + +#endif + /********************************************************************* * Set this to one to display debug statistics *********************************************************************/ @@ -147,8 +173,9 @@ static void ixgbe_setup_hw_rsc(struct rx_ring *); static void ixgbe_enable_intr(struct adapter *); static void ixgbe_disable_intr(struct adapter *); static void ixgbe_update_stats_counters(struct adapter *); -static void ixgbe_txeof(struct tx_ring *); -static bool ixgbe_rxeof(struct ix_queue *); +static bool ixgbe_txeof(struct tx_ring *); +static bool _ixgbe_rxeof_generic(struct ix_queue *, int, int *, = int); +#define ixgbe_rxeof(a) _ixgbe_rxeof_generic(a, = (a)->rxr->process_limit, NULL, 1) static void ixgbe_rx_checksum(u32, struct mbuf *, u32); static void ixgbe_set_promisc(struct adapter *); static void ixgbe_set_multi(struct adapter *); @@ -207,10 +234,32 @@ static void ixgbe_reinit_fdir(void *, int); /* Missing shared code prototype */ extern void ixgbe_stop_mac_link_on_d3_82599(struct ixgbe_hw *hw); =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int _ixgbe_poll_generic(struct ifnet *ifp, enum poll_cmd = cmd, + int count, int locking); +static poll_handler_t ixgbe_poll; +#endif +#ifdef NETDUMP_CLIENT +static poll_handler_t ixgbe_poll_unlocked; +static ndumplock_handler_t ixgbe_ndump_disable_intr; +static ndumplock_handler_t ixgbe_ndump_enable_intr; +#endif + /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ =20 +#ifdef NETDUMP_CLIENT + +static struct netdump_methods ixgbe_ndump_methods =3D { + .ne_poll_locked =3D ixgbe_poll, + .ne_poll_unlocked =3D ixgbe_poll_unlocked, + .ne_disable_intr =3D ixgbe_ndump_disable_intr, + .ne_enable_intr =3D ixgbe_ndump_enable_intr +}; + +#endif + static device_method_t ixgbe_methods[] =3D { /* Device interface */ DEVMETHOD(device_probe, ixgbe_probe), @@ -667,6 +716,11 @@ ixgbe_detach(device_t dev) return (EBUSY); } =20 +#ifdef DEVICE_POLLING + if ((adapter->ifp->if_capenable & IFCAP_POLLING) !=3D 0) + ether_poll_deregister(adapter->ifp); +#endif + IXGBE_CORE_LOCK(adapter); ixgbe_stop(adapter); IXGBE_CORE_UNLOCK(adapter); @@ -1026,6 +1080,25 @@ ixgbe_ioctl(struct ifnet * ifp, u_long command, = caddr_t data) { int mask =3D ifr->ifr_reqcap ^ ifp->if_capenable; IOCTL_DEBUGOUT("ioctl: SIOCSIFCAP (Set Capabilities)"); +#ifdef DEVICE_POLLING + if ((mask & IFCAP_POLLING) !=3D 0) { + if ((ifr->ifr_reqcap & IFCAP_POLLING) !=3D 0) { + error =3D = ether_poll_register(ixgbe_poll, ifp); + if (error !=3D 0) + return (error); + IXGBE_CORE_LOCK(adapter); + ixgbe_disable_intr(adapter); + ifp->if_capenable |=3D IFCAP_POLLING; + IXGBE_CORE_UNLOCK(adapter); + } else { + error =3D ether_poll_deregister(ifp); + IXGBE_CORE_LOCK(adapter); + ixgbe_enable_intr(adapter); + ifp->if_capenable &=3D ~IFCAP_POLLING; + IXGBE_CORE_UNLOCK(adapter); + } + } +#endif /* !DEVICE_POLLING */ if (mask & IFCAP_HWCSUM) ifp->if_capenable ^=3D IFCAP_HWCSUM; if (mask & IFCAP_TSO4) @@ -1339,8 +1412,13 @@ ixgbe_init_locked(struct adapter *adapter) /* Set up VLAN support and filter */ ixgbe_setup_vlan_hw_support(adapter); =20 - /* And now turn on interrupts */ - ixgbe_enable_intr(adapter); +#ifdef DEVICE_POLLING + /* Disable interrupts if polling is on, enable otherwise. */ + if ((ifp->if_capenable & IFCAP_POLLING) !=3D 0) + ixgbe_disable_intr(adapter); + else +#endif + ixgbe_enable_intr(adapter); =20 /* Now inform the stack we're ready */ ifp->if_drv_flags |=3D IFF_DRV_RUNNING; @@ -1437,6 +1515,10 @@ ixgbe_handle_que(void *context, int pending) return; } =20 +#ifdef DEVICE_POLLING + if ((adapter->ifp->if_capenable & IFCAP_POLLING) !=3D 0) + return; +#endif =20 /********************************************************************* * @@ -2646,6 +2728,9 @@ ixgbe_setup_interface(device_t dev, struct adapter = *adapter) ifp->if_snd.ifq_drv_maxlen =3D adapter->num_tx_desc - 2; IFQ_SET_READY(&ifp->if_snd); #endif +#ifdef NETDUMP_CLIENT + ifp->if_ndumpfuncs =3D &ixgbe_ndump_methods; +#endif =20 ether_ifattach(ifp, adapter->hw.mac.addr); =20 @@ -2665,6 +2750,9 @@ ixgbe_setup_interface(device_t dev, struct adapter = *adapter) | IFCAP_VLAN_MTU | IFCAP_HWSTATS; ifp->if_capenable =3D ifp->if_capabilities; +#ifdef DEVICE_POLLING + ifp->if_capabilities |=3D IFCAP_POLLING; +#endif =20 /* ** Don't turn this on by default, if vlans are @@ -3582,6 +3670,91 @@ ixgbe_atr(struct tx_ring *txr, struct mbuf *mp) } #endif /* IXGBE_FDIR */ =20 +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) +static int +_ixgbe_poll_generic(struct ifnet *ifp, enum poll_cmd cmd, int count, + int locking) +{ + struct adapter *adapter; + struct tx_ring *txr; + struct ix_queue *que; + struct ixgbe_hw *hw; + u32 loop, reg_eicr; + int rx_npkts; + bool more_tx; + + adapter =3D ifp->if_softc; + txr =3D adapter->tx_rings; + que =3D adapter->queues; + hw =3D &adapter->hw; + loop =3D MAX_LOOP; + rx_npkts =3D 0; + + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0) + return (rx_npkts); + + if (cmd =3D=3D POLL_AND_CHECK_STATUS) { + reg_eicr =3D IXGBE_READ_REG(hw, IXGBE_EICR); + + /* Link status change */ + if ((reg_eicr & IXGBE_EICR_LSC) !=3D 0) + taskqueue_enqueue(adapter->tq, = &adapter->link_task); + } + _ixgbe_rxeof_generic(que, count, &rx_npkts, locking); + IXGBE_TX_LOCK_COND(txr, locking); + do { + more_tx =3D ixgbe_txeof(txr); + } while (loop-- && more_tx); +#if __FreeBSD_version >=3D 800000 + if (!drbr_empty(ifp, txr->br)) + ixgbe_mq_start_locked(ifp, txr); +#else + if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) + ixgbe_start_locked(txr, ifp); +#endif + IXGBE_TX_UNLOCK_COND(txr, locking); + return (rx_npkts); +} + +static int +ixgbe_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_ixgbe_poll_generic(ifp, cmd, count, 1)); +} +#endif /* !DEVICE_POLLING && !NETDUMP_CLIENT */ + +#ifdef NETDUMP_CLIENT +static int +ixgbe_poll_unlocked(struct ifnet *ifp, enum poll_cmd cmd, int count) +{ + + return (_ixgbe_poll_generic(ifp, cmd, count, 0)); +} + +static void +ixgbe_ndump_disable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IXGBE_CORE_LOCK(adapter); + ixgbe_disable_intr(adapter); + IXGBE_CORE_UNLOCK(adapter); +} + +static void +ixgbe_ndump_enable_intr(struct ifnet *ifp) +{ + struct adapter *adapter; + + adapter =3D ifp->if_softc; + IXGBE_CORE_LOCK(adapter); + ixgbe_enable_intr(adapter); + IXGBE_CORE_UNLOCK(adapter); +} +#endif /* !NETDUMP_CLIENT */ + /********************************************************************** * * Examine each tx_buffer in the used queue. If the hardware is done @@ -3589,7 +3762,7 @@ ixgbe_atr(struct tx_ring *txr, struct mbuf *mp) * tx_buffer is put back on the free queue. * = **********************************************************************/ -static void +static bool ixgbe_txeof(struct tx_ring *txr) { struct adapter *adapter =3D txr->adapter; @@ -3632,13 +3805,13 @@ ixgbe_txeof(struct tx_ring *txr) netmap_tx_irq(ifp, txr->me | (NETMAP_LOCKED_ENTER|NETMAP_LOCKED_EXIT)); } - return; + return FALSE; } #endif /* DEV_NETMAP */ =20 if (txr->tx_avail =3D=3D txr->num_desc) { txr->queue_status =3D IXGBE_QUEUE_IDLE; - return; + return FALSE; } =20 /* Get work starting point */ @@ -3735,7 +3908,7 @@ ixgbe_txeof(struct tx_ring *txr) if (txr->tx_avail =3D=3D txr->num_desc) txr->queue_status =3D IXGBE_QUEUE_IDLE; =20 - return; + return TRUE; } =20 /********************************************************************* @@ -4405,20 +4578,20 @@ ixgbe_rx_discard(struct rx_ring *rxr, int i) * Return TRUE for more work, FALSE for all clean. *********************************************************************/ static bool -ixgbe_rxeof(struct ix_queue *que) +_ixgbe_rxeof_generic(struct ix_queue *que, int count, int *rx_npktsp, + int locking) { struct adapter *adapter =3D que->adapter; struct rx_ring *rxr =3D que->rxr; struct ifnet *ifp =3D adapter->ifp; struct lro_ctrl *lro =3D &rxr->lro; struct lro_entry *queued; - int i, nextp, processed =3D 0; + int i, nextp, processed =3D 0, rx_npkts =3D = 0; u32 staterr =3D 0; - u16 count =3D rxr->process_limit; union ixgbe_adv_rx_desc *cur; struct ixgbe_rx_buf *rbuf, *nbuf; =20 - IXGBE_RX_LOCK(rxr); + IXGBE_RX_LOCK_COND(rxr, locking); =20 #ifdef DEV_NETMAP /* Same as the txeof routine: wakeup clients on intr. */ @@ -4588,6 +4761,7 @@ next_desc: if (sendmp !=3D NULL) { rxr->next_to_check =3D i; ixgbe_rx_input(rxr, ifp, sendmp, ptype); + rx_npkts++; i =3D rxr->next_to_check; } =20 @@ -4612,15 +4786,18 @@ next_desc: tcp_lro_flush(lro, queued); } =20 - IXGBE_RX_UNLOCK(rxr); + IXGBE_RX_UNLOCK_COND(rxr, locking); =20 + if (rx_npktsp !=3D NULL) + *rx_npktsp =3D rx_npkts; /* ** Still have cleaning to do? */ - if ((staterr & IXGBE_RXD_STAT_DD) !=3D 0) + if ((staterr & IXGBE_RXD_STAT_DD) !=3D 0) { return (TRUE); - else + } else { return (FALSE); + } } =20 =20 diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index 77b72ed..f7278b4 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -66,6 +66,9 @@ #include #include #include +#ifdef NETDUMP_CLIENT +#include +#endif #include #include #include diff --git a/sys/i386/i386/minidump_machdep.c = b/sys/i386/i386/minidump_machdep.c index e0cd1ff..417d9a9 100644 --- a/sys/i386/i386/minidump_machdep.c +++ b/sys/i386/i386/minidump_machdep.c @@ -248,13 +248,19 @@ minidumpsys(struct dumperinfo *di) } dumpsize +=3D PAGE_SIZE; =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D ENOSPC; - goto fail; + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D ENOSPC; + goto fail; + } + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; } - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; progress =3D dumpsize; =20 /* Initialize mdhdr */ diff --git a/sys/ia64/ia64/dump_machdep.c b/sys/ia64/ia64/dump_machdep.c index 6fa8608..716c111 100644 --- a/sys/ia64/ia64/dump_machdep.c +++ b/sys/ia64/ia64/dump_machdep.c @@ -245,13 +245,20 @@ dumpsys(struct dumperinfo *di) dumpsize +=3D fileofs; hdrgap =3D fileofs - DEV_ALIGN(hdrsz); =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D ENOSPC; - goto fail; + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D ENOSPC; + goto fail; + } + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; } - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; =20 mkdumpheader(&kdh, KERNELDUMPMAGIC, KERNELDUMP_IA64_VERSION, = dumpsize, di->blocksize); =20 diff --git a/sys/kern/kern_shutdown.c b/sys/kern/kern_shutdown.c index b120263..df9dd69 100644 --- a/sys/kern/kern_shutdown.c +++ b/sys/kern/kern_shutdown.c @@ -39,6 +39,7 @@ __FBSDID("$FreeBSD$"); =20 #include "opt_ddb.h" #include "opt_kdb.h" +#include "opt_netdump.h" #include "opt_panic.h" #include "opt_sched.h" #include "opt_watchdog.h" @@ -84,6 +85,15 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef NETDUMP_CLIENT +#include + +#include +#include + +#include +#endif + #include =20 #ifndef PANIC_REBOOT_WAIT_TIME @@ -754,6 +764,17 @@ vpanic(const char *fmt, va_list ap) kern_reboot(bootopt); } =20 +#ifdef DDB +DB_COMMAND(netdump, ddb_force_netdump) +{ + + if (nd_enable) + netdumpsys(); + else + db_printf("netdump is not enabled\n"); +} +#endif + /* * Support for poweroff delay. * @@ -861,7 +882,9 @@ dump_write(struct dumperinfo *di, void *virtual, = vm_offset_t physical, off_t offset, size_t length) { =20 - if (length !=3D 0 && (offset < di->mediaoffset || + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) !=3D 0 && length !=3D 0 && + (offset < di->mediaoffset || offset - di->mediaoffset + length > di->mediasize)) { printf("Attempt to write outside dump device = boundaries.\n" "offset(%jd), mediaoffset(%jd), length(%ju), = mediasize(%jd).\n", diff --git a/sys/net/if_var.h b/sys/net/if_var.h index 3288a4f..38fb118 100644 --- a/sys/net/if_var.h +++ b/sys/net/if_var.h @@ -73,6 +73,7 @@ struct carp_softc; struct ifvlantrunk; struct route; struct vnet; +struct netdump_methods; #endif =20 #include /* get TAILQ macros */ @@ -203,7 +204,7 @@ struct ifnet { char *if_description; /* interface description */ u_int if_fib; /* interface FIB */ u_char if_alloctype; /* if_type at time of allocation = */ - + struct netdump_methods *if_ndumpfuncs; /* netdump virtual = methods */ u_int if_hw_tsomax; /* tso burst length limit, the = minimum * is (IP_MAXPACKET / 8). * XXXAO: Have to find a better = place @@ -957,13 +958,13 @@ void if_deregister_com_alloc(u_char type); #define IF_LLADDR(ifp) = \ LLADDR((struct sockaddr_dl *)((ifp)->if_addr->ifa_addr)) =20 -#ifdef DEVICE_POLLING +#if defined(DEVICE_POLLING) || defined(NETDUMP_CLIENT) enum poll_cmd { POLL_ONLY, POLL_AND_CHECK_STATUS }; =20 typedef int poll_handler_t(struct ifnet *ifp, enum poll_cmd cmd, = int count); int ether_poll_register(poll_handler_t *h, struct ifnet *ifp); int ether_poll_deregister(struct ifnet *ifp); -#endif /* DEVICE_POLLING */ +#endif /* DEVICE_POLLING || NETDUMP_CLIENT */ =20 #endif /* _KERNEL */ =20 diff --git a/sys/netinet/netdump.h b/sys/netinet/netdump.h new file mode 100644 index 0000000..fb0510b --- /dev/null +++ b/sys/netinet/netdump.h @@ -0,0 +1,158 @@ +/* + * Copyright (c) 2005-2011 Sandvine Incorporated + * Copyright (c) 2000 Darrell Anderson + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in = the + * documentation and/or other materials provided with the = distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _NETINET_NETDUMP_H_ +#define _NETINET_NETDUMP_H_ + +#include + +#define NETDUMP_PORT 20023 /* Server udp port = number for data. */ +#define NETDUMP_ACKPORT 20024 /* Client udp port = number for acks. */ + +#define NETDUMP_HERALD 1 /* Broadcast before = starting a dump. */ +#define NETDUMP_FINISHED 2 /* Send after finishing = a dump. */ +#define NETDUMP_VMCORE 3 /* Contains dump datas. = */ +#define NETDUMP_KDH 4 /* Contains kernel dump = header. */ + +#define NETDUMP_DATASIZE 8192 /* Packets payload. */ + +struct netdump_msg_hdr { + uint32_t mh_type; /* NETDUMP_HERALD, _FINISHED, _VMCORE, = _KDH. */ + uint32_t mh_seqno; /* Match acks with msgs. */ + uint64_t mh_offset; /* vmcore offset (bytes). */ + uint32_t mh_len; /* Attached data (bytes). */ + uint32_t mh__pad; /* Pad space matching 32- and 64-bits = archs. */ +}; + +struct netdump_ack { + uint32_t na_seqno; /* Match acks with msgs. */ +}; + +struct netdump_msg { + struct netdump_msg_hdr nm_hdr; + uint8_t nm_data[NETDUMP_DATASIZE]; +}; + +#ifdef _KERNEL + +typedef void ndumplock_handler_t(struct ifnet *); + +struct netdump_methods { + poll_handler_t *ne_poll_locked; + poll_handler_t *ne_poll_unlocked; + ndumplock_handler_t *ne_disable_intr; + ndumplock_handler_t *ne_enable_intr; +}; + +int netdumpsys(void); + +extern int nd_enable; + +#endif + +#endif /* !_NETINET_NETDUMP_H_ */ +/* + * Copyright (c) 2005-2011 Sandvine Incorporated + * Copyright (c) 2000 Darrell Anderson + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in = the + * documentation and/or other materials provided with the = distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _NETINET_NETDUMP_H_ +#define _NETINET_NETDUMP_H_ + +#include + +#define NETDUMP_PORT 20023 /* Server udp port = number for data. */ +#define NETDUMP_ACKPORT 20024 /* Client udp port = number for acks. */ + +#define NETDUMP_HERALD 1 /* Broadcast before = starting a dump. */ +#define NETDUMP_FINISHED 2 /* Send after finishing = a dump. */ +#define NETDUMP_VMCORE 3 /* Contains dump datas. = */ +#define NETDUMP_KDH 4 /* Contains kernel dump = header. */ + +#define NETDUMP_DATASIZE 8192 /* Packets payload. */ + +struct netdump_msg_hdr { + uint32_t mh_type; /* NETDUMP_HERALD, _FINISHED, _VMCORE, = _KDH. */ + uint32_t mh_seqno; /* Match acks with msgs. */ + uint64_t mh_offset; /* vmcore offset (bytes). */ + uint32_t mh_len; /* Attached data (bytes). */ + uint32_t mh__pad; /* Pad space matching 32- and 64-bits = archs. */ +}; + +struct netdump_ack { + uint32_t na_seqno; /* Match acks with msgs. */ +}; + +struct netdump_msg { + struct netdump_msg_hdr nm_hdr; + uint8_t nm_data[NETDUMP_DATASIZE]; +}; + +#ifdef _KERNEL + +typedef void ndumplock_handler_t(struct ifnet *); + +struct netdump_methods { + poll_handler_t *ne_poll_locked; + poll_handler_t *ne_poll_unlocked; + ndumplock_handler_t *ne_disable_intr; + ndumplock_handler_t *ne_enable_intr; +}; + +int netdumpsys(void); + +extern int nd_enable; + +#endif + +#endif /* !_NETINET_NETDUMP_H_ */ diff --git a/sys/netinet/netdump_client.c b/sys/netinet/netdump_client.c new file mode 100644 index 0000000..1b73e1b --- /dev/null +++ b/sys/netinet/netdump_client.c @@ -0,0 +1,1201 @@ +/*- + * Copyright (c) 2005-2011 Sandvine Incorporated. All rights reserved. + * Copyright (c) 2000 Darrell Anderson + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in = the + * documentation and/or other materials provided with the = distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF + * SUCH DAMAGE. + */ + +/* + * netdump_client.c + * FreeBSD subsystem supporting netdump network dumps. + * A dedicated server must be running to accept client dumps. + * XXX: This should be split into machdep and non-machdep parts + * +*/ + +#include "opt_ddb.h" +#include "opt_kdb.h" +#include "opt_netdump.h" + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#ifdef DDB +#include +#endif + +#ifdef NETDUMP_CLIENT_DEBUG +#define NETDDEBUG(f, ...) printf((f), ## = __VA_ARGS__) +#define NETDDEBUG_IF(i, f, ...) if_printf((i), (f), ## = __VA_ARGS__) +#if NETDUMP_CLIENT_DEBUG > 1 +#define NETDDEBUGV(f, ...) printf((f), ## = __VA_ARGS__) +#define NETDDEBUGV_IF(i, f, ...) if_printf((i), (f), ## = __VA_ARGS__) +#else +#define NETDDEBUGV(f, ...) +#define NETDDEBUGV_IF(i, f, ...) +#endif +#else +#define NETDDEBUG(f, ...) +#define NETDDEBUG_IF(i, f, ...) +#define NETDDEBUGV(f, ...) +#define NETDDEBUGV_IF(i, f, ...) +#endif + +static void nd_handle_arp(struct mbuf **mb); +static void nd_handle_ip(struct mbuf **mb); +static int netdump_arp_server(void); +static void netdump_config_defaults(void *dummy __unused); +static int netdump_dumper(void *priv __unused, void *virtual, + vm_offset_t physical __unused, off_t offset, size_t = length); +static int netdump_ether_output(struct mbuf *m, struct ifnet *ifp,=20= + struct ether_addr dst, u_short etype); +static int netdump_mbuf_nop(struct mbuf *mp __unused, void *ptr = __unused, void *opt_args __unused); +static void netdump_network_poll(void); +static void netdump_pkt_in(struct ifnet *ifp, struct mbuf *m); +static int netdump_send(uint32_t type, off_t offset, unsigned char = *data, + uint32_t datalen); +static int netdump_send_arp(void); +static int netdump_udp_output(struct mbuf *m); + +static int sysctl_handle_ifxname(SYSCTL_HANDLER_ARGS); +static int sysctl_handle_inaddr(SYSCTL_HANDLER_ARGS); + +/* Must be at least as big as the chunks dumpsys() gives us. */ +static unsigned char buf[MAXDUMPPGS * PAGE_SIZE]; +static uint64_t rcvd_acks; +static uint32_t nd_seqno =3D 1; +static int dump_failed, have_server_mac; + +/* + * Times to poll the NIC (0.5ms each poll) before assuming packetloss + * occurred (default to 5s). + */ +static int nd_polls =3D 10000; + +/* Times to retransmit lost packets. */ +static int nd_retries =3D 10; + +/* General dynamic settings. */ +static char nd_ifp_str[IFNAMSIZ]; +static struct ether_addr nd_gw_mac; +static struct in_addr nd_server =3D {INADDR_ANY}; +static struct in_addr nd_client =3D {INADDR_ANY}; +static struct in_addr nd_gw =3D {INADDR_ANY}; +struct ifnet *nd_ifp; +int nd_enable =3D 0; +static uint16_t nd_server_port =3D NETDUMP_PORT; + +/* Tunables storages. */ +static char nd_server_tun[INET_ADDRSTRLEN]; +static char nd_client_tun[INET_ADDRSTRLEN]; +static char nd_gw_tun[INET_ADDRSTRLEN]; +static char nd_nic_tun[IFNAMSIZ]; + +/* + * Checks for netdump support on a network interface + * + * Parameters: + * ifp The network interface that is being tested for support + * + * Returns: + * int 1 if the interface is supported, 0 if not + */ +static __inline int +netdump_supported_nic(struct ifnet *ifp) +{ + + return (ifp->if_ndumpfuncs !=3D NULL); +} + +/*- + * Sysctls specific code. + */ + +/* + * Sysctl handler converting a string sysctl to/from an ifaddr name. + * + * Parameters: + * SYSCTL_HANDLER_ARGS + * - arg1 is a pointer to the if structure name + * - arg2 is unused + * + * Returns: + * int see errno.h, 0 for success + */ +static int +sysctl_handle_ifxname(SYSCTL_HANDLER_ARGS) +{ + char buf[IFNAMSIZ]; + struct ifnet *ifp; + int error, found; + + strlcpy(buf, arg1, sizeof(buf)); + error =3D sysctl_handle_string(oidp, buf, sizeof(buf), req); + if (error !=3D 0 || req->newptr =3D=3D NULL) + return (error); + found =3D 0; + IFNET_RLOCK_NOSLEEP(); + TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if (!strncmp(ifp->if_xname, buf, strlen(ifp->if_xname)) = && + netdump_supported_nic(ifp)) { + found =3D 1; + break; + } + } + IFNET_RUNLOCK_NOSLEEP(); + if (found =3D=3D 0) + return (EINVAL); + strlcpy(arg1, buf, strlen(buf) + 1); + return (error); +} + +/* + * Sysctl handler converting a string sysctl to/from an in_addr. + * + * Parameters: + * SYSCTL_HANDLER_ARGS + * - arg1 is a pointer to the struct in_addr holding the IP + * - arg2 is unused + * + * Returns: + * int see errno.h, 0 for success + */ +static int +sysctl_handle_inaddr(SYSCTL_HANDLER_ARGS) +{ + struct in_addr addr; + char buf[INET_ADDRSTRLEN]; + int error; + + inet_ntoa_r(*(struct in_addr *)arg1, buf); + error =3D sysctl_handle_string(oidp, buf, sizeof(buf), req); + if (error =3D=3D 0) { + if (!inet_aton(buf, &addr)) + error =3D EINVAL; + else + *(struct in_addr *)arg1 =3D addr; + } + return (error); +} + +SYSCTL_NODE(_net, OID_AUTO, dump, CTLFLAG_RW | CTLFLAG_MPSAFE, 0, = "netdump"); +SYSCTL_PROC(_net_dump, OID_AUTO, server, CTLTYPE_STRING | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_server, 0, sysctl_handle_inaddr, "A", "dump = server"); +SYSCTL_PROC(_net_dump, OID_AUTO, client, CTLTYPE_STRING |CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_client, 0, sysctl_handle_inaddr, "A", "dump = client"); +SYSCTL_PROC(_net_dump, OID_AUTO, gateway, CTLTYPE_STRING | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_gw, 0, sysctl_handle_inaddr, "A", + "dump default gateway"); +SYSCTL_PROC(_net_dump, OID_AUTO, nic, CTLTYPE_STRING | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_ifp_str, 0, sysctl_handle_ifxname, "A", + "dumping interface name"); +SYSCTL_INT(_net_dump, OID_AUTO, polls, CTLTYPE_INT | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_polls, 0, "times to poll NIC per retry"); +SYSCTL_INT(_net_dump, OID_AUTO, retries, CTLTYPE_INT | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_retries, 0, "times to retransmit lost = packets"); +SYSCTL_INT(_net_dump, OID_AUTO, enable, CTLTYPE_INT | CTLFLAG_RW | + CTLFLAG_MPSAFE, &nd_enable, 0, "enable network dump"); + +TUNABLE_STR("net.dump.server", nd_server_tun, sizeof(nd_server_tun)); +TUNABLE_STR("net.dump.client", nd_client_tun, sizeof(nd_client_tun)); +TUNABLE_STR("net.dump.gateway", nd_gw_tun, sizeof(nd_gw_tun)); +TUNABLE_STR("net.dump.nic", nd_nic_tun, sizeof(nd_nic_tun)); +TUNABLE_INT("net.dump.enable", &nd_enable); + +/*- + * Network specific primitives. + * Following down the code they are divided ordered as: + * - Output primitives + * - Input primitives + * - Polling primitives + */ + +/* + * Netdump wraps external mbufs around address ranges. unlike most = sane + * counterparts, netdump uses a stop-and-wait approach to flow control = and + * retransmission, so the ack obviates the need for mbuf reference + * counting. We still need to tell other mbuf handlers not to do = anything + * special with our mbufs, so specify this nop handler. + * + * Parameters: + * ptr data to free (ignored) + * opt_args callback pointer (ignored) + * + * Returns: + * void + */ +static int +netdump_mbuf_nop(struct mbuf *mp __unused, void *ptr __unused, void = *opt_args __unused) +{ + return EXT_FREE_OK; +} + +/* + * Handles creation of the ethernet header, then places outgoing = packets into + * the tx buffer for the NIC + * + * Parameters: + * m The mbuf containing the packet to be sent (will be freed = by + * this function or the NIC driver) + * ifp The interface to send on + * dst The destination ethernet address (source address will be = looked + * up using ifp) + * etype The ETHERTYPE_* value for the protocol that is being = sent + * + * Returns: + * int see errno.h, 0 for success + */ +static int +netdump_ether_output(struct mbuf *m, struct ifnet *ifp, struct = ether_addr dst, + u_short etype) +{ + struct ether_header *eh; + + /* Fill in the ethernet header. */ + M_PREPEND(m, ETHER_HDR_LEN, M_NOWAIT); + if (m =3D=3D NULL) { + printf("netdump_ether_output: Out of mbufs\n"); + return (ENOBUFS); + } + eh =3D mtod(m, struct ether_header *); + memcpy(eh->ether_shost, IF_LLADDR(ifp), ETHER_ADDR_LEN); + memcpy(eh->ether_dhost, dst.octet, ETHER_ADDR_LEN); + eh->ether_type =3D htons(etype); + + if (((ifp->if_flags & (IFF_MONITOR | IFF_UP)) !=3D IFF_UP) || + (ifp->if_drv_flags & IFF_DRV_RUNNING) !=3D IFF_DRV_RUNNING) = { + if_printf(ifp, "netdump_ether_output: Interface isn't = up\n"); + m_freem(m); + return (ENETDOWN); + } + return ((ifp->if_transmit)(ifp, m)); +} + +/* + * Unreliable transmission of an mbuf chain to the netdump server + * Note: can't handle fragmentation; fails if the packet is larger than + * nd_ifp->if_mtu after adding the UDP/IP headers + * + * Parameters: + * m mbuf chain + * + * Returns: + * int see errno.h, 0 for success + */ +static int +netdump_udp_output(struct mbuf *m) +{ + struct udpiphdr *ui; + struct ip *ip; + + MPASS(nd_ifp !=3D NULL); + + M_PREPEND(m, sizeof(struct udpiphdr), M_NOWAIT); + if (m =3D=3D NULL) { + printf("netdump_udp_output: Out of mbufs\n"); + return (ENOBUFS); + } + ui =3D mtod(m, struct udpiphdr *); + bzero(ui->ui_x1, sizeof(ui->ui_x1)); + ui->ui_pr =3D IPPROTO_UDP; + ui->ui_len =3D htons(m->m_pkthdr.len - sizeof(struct ip)); + ui->ui_ulen =3D ui->ui_len; + ui->ui_src =3D nd_client; + ui->ui_dst =3D nd_server; + /* Use this src port so that the server can connect() the socket = */ + ui->ui_sport =3D htons(NETDUMP_ACKPORT); + ui->ui_dport =3D htons(nd_server_port); + ui->ui_sum =3D 0; + if ((ui->ui_sum =3D in_cksum(m, m->m_pkthdr.len)) =3D=3D 0) + ui->ui_sum =3D 0xffff; + + ip =3D mtod(m, struct ip *); + ip->ip_v =3D IPVERSION; + ip->ip_hl =3D sizeof(struct ip) >> 2; + ip->ip_tos =3D 0; + ip->ip_len =3D htons(m->m_pkthdr.len); + ip->ip_id =3D 0; + ip->ip_off =3D htons(IP_DF); + ip->ip_ttl =3D 32; + ip->ip_sum =3D 0; + ip->ip_sum =3D in_cksum(m, sizeof(struct ip)); + + if (m->m_pkthdr.len > nd_ifp->if_mtu) { + printf("netdump_udp_output: Packet is too big: %u > MTU = %lu\n", + m->m_pkthdr.len, (unsigned long)nd_ifp->if_mtu); + m_freem(m); + return (ENOBUFS); + } + return (netdump_ether_output(m, nd_ifp, nd_gw_mac, = ETHERTYPE_IP)); +} + +/* + * Builds and sends a single ARP request to locate the server + * + * Parameters: + * void + * + * Return value: + * 0 on success + * errno on error + */ +static int +netdump_send_arp() +{ + struct ether_addr bcast; + struct mbuf *m; + struct arphdr *ah; + int pktlen; + + MPASS(nd_ifp !=3D NULL); + + /* Fill-up a broadcast address. */ + memset(&bcast, 0xFF, ETHER_ADDR_LEN); + MGETHDR(m, M_NOWAIT, MT_DATA); + if (m =3D=3D NULL) { + printf("netdump_send_arp: Out of mbufs"); + return (ENOBUFS); + } + pktlen =3D arphdr_len2(ETHER_ADDR_LEN, sizeof(struct in_addr)); + m->m_len =3D pktlen; + m->m_pkthdr.len =3D pktlen; + MH_ALIGN(m, pktlen); + ah =3D mtod(m, struct arphdr *); + ah->ar_hrd =3D htons(ARPHRD_ETHER); + ah->ar_pro =3D htons(ETHERTYPE_IP); + ah->ar_hln =3D ETHER_ADDR_LEN; + ah->ar_pln =3D sizeof(struct in_addr); + ah->ar_op =3D htons(ARPOP_REQUEST); + memcpy(ar_sha(ah), IF_LLADDR(nd_ifp), ETHER_ADDR_LEN); + memcpy(ar_spa(ah), &nd_client.s_addr, sizeof(nd_client.s_addr)); + bzero(ar_tha(ah), ETHER_ADDR_LEN); + memcpy(ar_tpa(ah), &nd_gw.s_addr, sizeof(nd_gw.s_addr)); + + return (netdump_ether_output(m, nd_ifp, bcast, ETHERTYPE_ARP)); +} + +/* + * Sends ARP requests to locate the server and waits for a response + * + * Parameters: + * void + * + * Return value: + * 0 on success + * errno on error + */ +static int +netdump_arp_server() +{ + int err, polls, retries; + + for (retries =3D 0; retries < nd_retries && have_server_mac =3D=3D= 0; + retries++) { + err =3D netdump_send_arp(); + if (err !=3D 0) + return (err); + for (polls =3D 0; polls < nd_polls && have_server_mac =3D=3D= 0; + polls++) { + netdump_network_poll(); + DELAY(500); + } + if (have_server_mac =3D=3D 0) + printf("(ARP retry)"); + } + if (have_server_mac !=3D 0) + return (0); + + printf("\nARP timed out.\n"); + return (ETIMEDOUT); +} + +/* + * Construct and reliably send a netdump packet. May fail from a = resource + * shortage or extreme number of unacknowledged retransmissions. Wait = for + * an acknowledgement before returning. Splits packets into chunks = small + * enough to be sent without fragmentation (looks up the interface MTU) + * + * Parameters: + * type netdump packet type (HERALD, FINISHED, or VMCORE) + * offset vmcore data offset (bytes) + * data vmcore data + * datalen vmcore data size (bytes) + * + * Returns: + * int see errno.h, 0 for success + */ +static int +netdump_send(uint32_t type, off_t offset, unsigned char *data, uint32_t = datalen) +{ + uint64_t want_acks; + struct netdump_msg_hdr *nd_msg_hdr; + struct mbuf *m, *m2; + uint32_t i, pktlen, sent_so_far; + int retries, polls, error; + + want_acks =3D 0; + rcvd_acks =3D 0; + retries =3D 0; + + MPASS(nd_ifp !=3D NULL); + +retransmit: + + /* Chunks can be too big to fit in packets. */ + for (i =3D sent_so_far =3D 0; sent_so_far < + datalen || (i =3D=3D 0 && datalen =3D=3D 0); i++) { + pktlen =3D datalen - sent_so_far; + + /* First bound: the packet structure. */ + pktlen =3D min(pktlen, NETDUMP_DATASIZE); + + /* Second bound: the interface MTU (assume no IP = options). */ + pktlen =3D min(pktlen, nd_ifp->if_mtu - sizeof(struct = udpiphdr) - + sizeof(struct netdump_msg_hdr)); + + /* + * Check if it is retransmitting and this has been ACKed + * already. + */ + if ((rcvd_acks & (1 << i)) !=3D 0) { + sent_so_far +=3D pktlen; + continue; + } + + /* + * Get and fill a header mbuf, then chain data as an = extended + * mbuf. + */ + MGETHDR(m, M_NOWAIT, MT_DATA); + if (m =3D=3D NULL) { + printf("netdump_send: Out of mbufs!\n"); + return (ENOBUFS); + } + m->m_len =3D sizeof(struct netdump_msg_hdr); + m->m_pkthdr.len =3D sizeof(struct netdump_msg_hdr); + MH_ALIGN(m, sizeof(struct netdump_msg_hdr)); + nd_msg_hdr =3D mtod(m, struct netdump_msg_hdr *); + nd_msg_hdr->mh_seqno =3D htonl(nd_seqno+i); + nd_msg_hdr->mh_type =3D htonl(type); + nd_msg_hdr->mh_offset =3D htobe64(offset + sent_so_far); + nd_msg_hdr->mh_len =3D htonl(pktlen); + nd_msg_hdr->mh__pad =3D 0; + + if (pktlen !=3D 0) { + m2 =3D m_get(M_NOWAIT, MT_DATA); + if (m2 =3D=3D NULL) { + m_freem(m); + printf("netdump_send: Out of mbufs!\n"); + return (ENOBUFS); + } + MEXTADD(m2, data+sent_so_far, pktlen, = netdump_mbuf_nop, + NULL, NULL, M_RDONLY, EXT_MOD_TYPE); + m2->m_len =3D pktlen; + m->m_next =3D m2; + m->m_pkthdr.len +=3D m2->m_len; + } + error =3D netdump_udp_output(m); + if (error !=3D 0) + return (error); + + /* Note that we're waiting for this packet in the = bitfield. */ + want_acks |=3D 1 << i; + sent_so_far +=3D pktlen; + } + if (i >=3D sizeof(want_acks) * 8) + printf("Warning: Sent more than %zd packets (%d). " + "Acknowledgements will fail unless the size of " + "rcvd_acks/want_acks is increased.\n", + sizeof(want_acks) * 8, i); + + /* + * Wait for acks. A *real* window would speed things up = considerably. + */ + polls =3D 0; + while (rcvd_acks !=3D want_acks) { =09 + if (polls++ > nd_polls) { + if (retries++ > nd_retries) + return (ETIMEDOUT); + printf(". "); + goto retransmit; + } + netdump_network_poll(); + DELAY(500); + } + nd_seqno +=3D i; + return (0); +} + +/* + * Handler for IP packets: checks their sanity and then processes any = netdump + * ACK packets it finds. + * + * It needs to replicate partially the behaviour of ip_input() and + * udp_input(). + * + * Parameters: + * mb a pointer to an mbuf * containing the packet received + * Updates *mb if m_pullup et al change the pointer + * Assumes the calling function will take care of freeing = the mbuf + * + * Return value: + * void + */ +static void +nd_handle_ip(struct mbuf **mb) +{ + struct ip *ip; + struct udpiphdr *udp; + struct netdump_ack *nd_ack; + struct mbuf *m; + int rcv_ackno; + unsigned short hlen; + + /* IP processing. */ + m =3D *mb; + if (m->m_pkthdr.len < sizeof(struct ip)) { + NETDDEBUG("nd_handle_ip: dropping packet too small for = IP " + "header\n"); + return; + } + if (m->m_len < sizeof(struct ip)) { + m =3D m_pullup(m, sizeof(struct ip)); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + return; + } + } + ip =3D mtod(m, struct ip *); + + /* IP version. */ + if (ip->ip_v !=3D IPVERSION) { + NETDDEBUG("nd_handle_ip: Bad IP version %d\n", = ip->ip_v); + return; + } + + /* Header length. */ + hlen =3D ip->ip_hl << 2; + if (hlen < sizeof(struct ip)) { + NETDDEBUG("nd_handle_ip: Bad IP header length (%hu)\n", = hlen); + return; + } + if (hlen > m->m_len) { + m =3D m_pullup(m, hlen); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + return; + } + ip =3D mtod(m, struct ip *); + } + +#ifdef INVARIANTS + if (((ntohl(ip->ip_dst.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET || + (ntohl(ip->ip_src.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET) && + (m->m_pkthdr.rcvif->if_flags & IFF_LOOPBACK) =3D=3D 0) { + NETDDEBUG("nd_handle_ip: Bad IP header (RFC1122)\n"); + return; + } +#endif + + /* Checksum. */ + if ((m->m_pkthdr.csum_flags & CSUM_IP_CHECKED) !=3D 0) { + if ((m->m_pkthdr.csum_flags & CSUM_IP_VALID) =3D=3D 0) { + NETDDEBUG("nd_handle_ip: Bad IP checksum\n"); + return; + } + } else + NETDDEBUG("nd_handle_ip: HW didn't check IP cksum\n"); + + /* Convert fields to host byte order. */ + ip->ip_len =3D ntohs(ip->ip_len); + if (ip->ip_len < hlen) { + NETDDEBUG("nd_handle_ip: IP packet smaller (%hu) than header = (%hu)\n", + ip->ip_len, hlen); + return; + } + ip->ip_off =3D ntohs(ip->ip_off); + + if (m->m_pkthdr.len < ip->ip_len) { +NETDDEBUG("nd_handle_ip: IP packet bigger (%hu) than ethernet packet = (%hu)\n", + ip->ip_len, m->m_pkthdr.len); + return; + } + if (m->m_pkthdr.len > ip->ip_len) { + + /* Truncate the packet to the IP length. */ + if (m->m_len =3D=3D m->m_pkthdr.len) { + m->m_len =3D ip->ip_len; + m->m_pkthdr.len =3D ip->ip_len; + } else + m_adj(m, ip->ip_len - m->m_pkthdr.len); + } + + /* Ignore packets with IP options. */ + if (hlen > sizeof(struct ip)) { + NETDDEBUG("nd_handle_ip: Drop packet with IP = options\n"); + return; + } + + /* Check that the source is the server's IP. */ + if (ip->ip_src.s_addr !=3D nd_server.s_addr) { + NETDDEBUG("nd_handle_ip: Drop packet not from = server\n"); + return; + } + + /* Check if the destination IP is ours. */ + if (ip->ip_dst.s_addr !=3D nd_client.s_addr) { + NETDDEBUGV("nd_handle_ip: Drop packet not to our IP\n"); + return; + } + + if (ip->ip_p !=3D IPPROTO_UDP) { + NETDDEBUG("nd_handle_ip: Drop non-UDP packet\n"); + return; + } + + /* Do not deal with fragments. */ + if ((ip->ip_off & (IP_MF | IP_OFFMASK)) !=3D 0) { + NETDDEBUG("nd_handle_ip: Drop fragmented packet\n"); + return; + } + + /* UDP custom is to have packet length not include IP header. */ + ip->ip_len -=3D hlen; + + /* UDP processing. */ + + /* Get IP and UDP headers together, along with the netdump = packet. */ + if (m->m_pkthdr.len < + sizeof(struct udpiphdr) + sizeof(struct netdump_ack)) { + NETDDEBUG("nd_handle_ip: Ignoring small packet\n"); + return; + } + if (m->m_len < sizeof(struct udpiphdr) + sizeof(struct = netdump_ack)) { + m =3D m_pullup(m, sizeof(struct udpiphdr) + + sizeof(struct netdump_ack)); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + return; + } + } + udp =3D mtod(m, struct udpiphdr *); + + if (ntohs(udp->ui_u.uh_dport) !=3D NETDUMP_ACKPORT) { + NETDDEBUG("not on the netdump port.\n"); + return; + } + + /* Netdump processing. */ + + /* + * Packet is meant for us. Extract the ack sequence number and = the + * port number if necessary. + */ + nd_ack =3D (struct netdump_ack *)(mtod(m, caddr_t) + + sizeof(struct udpiphdr)); + rcv_ackno =3D ntohl(nd_ack->na_seqno); + if (nd_server_port =3D=3D NETDUMP_PORT) + nd_server_port =3D ntohs(udp->ui_u.uh_sport); + if (rcv_ackno >=3D nd_seqno + 64) + printf("nd_handle_ip: ACK %d too far in future!\n", = rcv_ackno); + else if (rcv_ackno >=3D nd_seqno) { + + /* We're interested in this ack. Record it. */ + rcvd_acks |=3D 1 << (rcv_ackno-nd_seqno); + } +} + +/* + * Handler for ARP packets: checks their sanity and then + * 1. If the ARP is a request for our IP, respond with our MAC address + * 2. If the ARP is a response from our server, record its MAC address + * + * It needs to replicate partially the behaviour of arpintr() and + * in_arpinput(). + * + * Parameters: + * mb a pointer to an mbuf * containing the packet received + * Updates *mb if m_pullup et al change the pointer + * Assumes the calling function will take care of freeing = the mbuf + * + * Return value: + * void + */ +static void +nd_handle_arp(struct mbuf **mb) +{ + char buf[INET_ADDRSTRLEN]; + struct in_addr isaddr, itaddr, myaddr; + struct ether_addr dst; + struct mbuf *m; + struct arphdr *ah; + struct ifnet *ifp; + uint8_t *enaddr; + int req_len, op; + + m =3D *mb; + ifp =3D m->m_pkthdr.rcvif; + if (m->m_len < sizeof(struct arphdr)) { + m =3D m_pullup(m, sizeof(struct arphdr)); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("nd_handle_arp: runt packet: m_pullup = failed\n"); + return; + } + } + ah =3D mtod(m, struct arphdr *); + + if (ntohs(ah->ar_hrd) !=3D ARPHRD_ETHER) { + NETDDEBUG("nd_handle_arp: unknown hardware address = 0x%2D)\n", + (unsigned char *)&ah->ar_hrd, ""); + return; + } + if (ntohs(ah->ar_pro) !=3D ETHERTYPE_IP) { + NETDDEBUG("nd_handle_arp: Drop ARP for unknown protocol = %d\n", + ntohs(ah->ar_pro)); + return; + } + req_len =3D arphdr_len2(ifp->if_addrlen, sizeof(struct = in_addr)); + if (m->m_len < req_len) { + m =3D m_pullup(m, req_len); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("nd_handle_arp: runt packet: m_pullup = failed\n"); + return; + } + } + ah =3D mtod(m, struct arphdr *); + + op =3D ntohs(ah->ar_op); + memcpy(&isaddr, ar_spa(ah), sizeof(isaddr)); + memcpy(&itaddr, ar_tpa(ah), sizeof(itaddr)); + enaddr =3D (uint8_t *)IF_LLADDR(ifp); + myaddr =3D nd_client; + + if (!bcmp(ar_sha(ah), enaddr, ifp->if_addrlen)) { + NETDDEBUG("nd_handle_arp: ignoring ARP from myself\n"); + return; + } + + if (isaddr.s_addr =3D=3D nd_client.s_addr) { + printf("nd_handle_arp: %*D is using my IP address = %s!\n", + ifp->if_addrlen, (u_char *)ar_sha(ah), ":", + inet_ntoa(isaddr)); + return; + } + + if (!bcmp(ar_sha(ah), ifp->if_broadcastaddr, ifp->if_addrlen)) { + NETDDEBUG("nd_handle_arp: ignoring ARP from broadcast = address\n"); + return; + } + + if (op =3D=3D ARPOP_REPLY) { + if (isaddr.s_addr !=3D nd_gw.s_addr) { + inet_ntoa_r(isaddr, buf); +NETDDEBUG("nd_handle_arp: ignoring ARP reply from %s (not netdump = server)\n", + buf); + return; + } + memcpy(nd_gw_mac.octet, ar_sha(ah), + min(ah->ar_hln, ETHER_ADDR_LEN)); + have_server_mac =3D 1; + NETDDEBUG("\nnd_handle_arp: Got server MAC address = %6D\n", + nd_gw_mac.octet, ":"); + return; + } + + if (op !=3D ARPOP_REQUEST) { + NETDDEBUG("nd_handle_arp: Ignoring ARP = non-request/reply\n"); + return; + } + + if (itaddr.s_addr !=3D nd_client.s_addr) { + NETDDEBUG("nd_handle_arp: ignoring ARP not to our = IP\n"); + return; + } + + memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); + memcpy(ar_sha(ah), enaddr, ah->ar_hln); + memcpy(ar_tpa(ah), ar_spa(ah), ah->ar_pln); + memcpy(ar_spa(ah), &itaddr, ah->ar_pln); + ah->ar_op =3D htons(ARPOP_REPLY); + ah->ar_pro =3D htons(ETHERTYPE_IP); + m->m_flags &=3D ~(M_BCAST|M_MCAST); + m->m_len =3D sizeof(*ah) + (2 * ah->ar_pln) + (2 * ah->ar_hln); + m->m_pkthdr.len =3D m->m_len; + + memcpy(dst.octet, ar_tha(ah), ETHER_ADDR_LEN); + netdump_ether_output(m, ifp, dst, ETHERTYPE_ARP); + *mb =3D NULL; +} + +/* + * Handler for incoming packets directly from the network adapter + * Identifies the packet type (IP or ARP) and passes it along to one of = the + * helper functions nd_handle_ip or nd_handle_arp. + * + * It needs to replicate partially the behaviour of ether_input() and + * ether_demux(). + * + * Parameters: + * ifp the interface the packet came from (should be nd_ifp) + * m an mbuf containing the packet received + * + * Return value: + * void + */ +static void +netdump_pkt_in(struct ifnet *ifp, struct mbuf *m) +{ + struct ether_header *eh; + u_short etype; + + /* Ethernet processing. */ + if ((m->m_flags & M_PKTHDR) =3D=3D 0) { + NETDDEBUG_IF(ifp, + "netdump_pkt_in: Discard frame without packet = header\n"); + goto done; + } + if (m->m_len < ETHER_HDR_LEN) { + NETDDEBUG_IF(ifp, +"netdump_pkt_in: Discard frame without leading eth header (len %u = pktlen %u)\n", + m->m_len, m->m_pkthdr.len); + goto done; + } + if ((m->m_flags & M_HASFCS) !=3D 0) { + m_adj(m, -ETHER_CRC_LEN); + m->m_flags &=3D ~M_HASFCS; + } + eh =3D mtod(m, struct ether_header *); + m->m_pkthdr.PH_loc.ptr =3D eh; + etype =3D ntohs(eh->ether_type); + if ((m->m_flags & M_VLANTAG) !=3D 0 || etype =3D=3D = ETHERTYPE_VLAN) { + NETDDEBUG_IF(ifp, "netdump_pkt_in: Ignoring vlan = packets\n"); + goto done; + } + + /* XXX: Probably must also check if we're the recipient MAC = address. */ + + /* Done ethernet processing. Strip off the ethernet header. */ + m_adj(m, ETHER_HDR_LEN); + switch (etype) { + case ETHERTYPE_ARP: + nd_handle_arp(&m); + break; + case ETHERTYPE_IP: + nd_handle_ip(&m); + break; + default: + NETDDEBUG_IF(ifp, + "netdump_pkt_in: Dropping unknown ethertype = %hu\n", + etype); + break; + } +done: + if (m !=3D NULL) + m_freem(m); +} + +/* + * After trapping, instead of assuming that most of the network stack = is sane + * just poll the driver directly for packets. + * + * Parameters: + * void + * + * Returns: + * void + */ +static void +netdump_network_poll() +{ + + MPASS(nd_ifp !=3D NULL); + +#if defined(KDB) && !defined(KDB_UNATTENDED) + if (panicstr !=3D NULL) + nd_ifp->if_ndumpfuncs->ne_poll_unlocked(nd_ifp, + POLL_AND_CHECK_STATUS, 1000); + else +#endif + nd_ifp->if_ndumpfuncs->ne_poll_locked(nd_ifp, + POLL_AND_CHECK_STATUS, 1000); +} + +/*- + * Dumping specific primitives. + */ + +/* + * Callback from dumpsys() to dump a chunk of memory. + * Copies it out to our static buffer then sends it across the network. + * Detects the initial KDH and makes sure it is given a special packet = type. + * + * Parameters: + * priv Unused. Optional private pointer. + * virtual Virtual address (where to read the data from) + * physical Unused. Physical memory address. + * offset Offset from start of core file + * length Data length + * + * Return value: + * 0 on success + * errno on error + */ +static int +netdump_dumper(void *priv __unused, void *virtual, + vm_offset_t physical __unused, off_t offset, size_t length) +{ + int err, msgtype; + + NETDDEBUGV("netdump_dumper(NULL, %p, NULL, %ju, %zu)\n", + virtual, (uintmax_t)offset, length); + + if (length > sizeof(buf)) + return (ENOSPC); + + /* + * The first write (at offset 0) is the kernel dump header. = Flag it + * for the server to treat specially. + * XXX: This doesn't strip out the footer KDH, although it + * should not hurt anything. + */ + msgtype =3D NETDUMP_VMCORE; + if (offset =3D=3D 0 && length > 0) + msgtype =3D NETDUMP_KDH; + else if (offset > 0) + offset -=3D sizeof(struct kerneldumpheader); + memcpy(buf, virtual, length); + err =3D netdump_send(msgtype, offset, buf, length); + if (err !=3D 0) { + dump_failed =3D 1; + return (err); + } + return (0); +} + +/* + * Dumper routine, specular to dumpsys(). + * + * Parameters: + * void + * + * Returns: + * int see errno.h, 0 for success + */ +int +netdumpsys() +{ + struct dumperinfo dumper; + void (*old_if_input)(struct ifnet *, struct mbuf *); + int error, found, must_lock, nd_gw_unset; + + old_if_input =3D NULL; + error =3D 0; + found =3D 0; + nd_gw_unset =3D 0; + must_lock =3D 1; +#if defined(KDB) && !defined(KDB_UNATTENDED) + if (panicstr !=3D NULL) + must_lock =3D 0; +#endif + + /* Check if the dumping is allowed to continue. */ + if (nd_enable =3D=3D 0) + return (EINVAL); + + /* Lookup the right if device to be used in the dump. */ + if (must_lock !=3D 0) + IFNET_RLOCK_NOSLEEP(); + TAILQ_FOREACH(nd_ifp, &V_ifnet, if_link) { + if (!strncmp(nd_ifp->if_xname, nd_ifp_str, + strlen(nd_ifp->if_xname)) && + netdump_supported_nic(nd_ifp)) { + found =3D 1; + break; + } + } + if (must_lock !=3D 0) + IFNET_RUNLOCK_NOSLEEP(); + if (found =3D=3D 0) { + printf("netdumpsys: Can't netdump: no valid NIC = given\n"); + return (EINVAL); + } + + MPASS(nd_ifp !=3D NULL); + + if (nd_server.s_addr =3D=3D INADDR_ANY) { + printf("netdumpsys: Can't netdump; no server IP = given\n"); + return (EINVAL); + } + if (nd_client.s_addr =3D=3D INADDR_ANY) { + printf("netdumpsys: Can't netdump; no client IP = given\n"); + return (EINVAL); + } + + /* + * nd_server_port could have switched after the first ack the + * first time it gets called. Adjust it accordingly. + */ + nd_server_port =3D NETDUMP_PORT; + if ((nd_ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && must_lock = !=3D 0) + nd_ifp->if_ndumpfuncs->ne_disable_intr(nd_ifp); + + /* Make the card use *our* receive callback. */ + old_if_input =3D nd_ifp->if_input; + nd_ifp->if_input =3D netdump_pkt_in; + + if (nd_gw.s_addr =3D=3D INADDR_ANY) { + nd_gw.s_addr =3D nd_server.s_addr; + nd_gw_unset =3D 1; + } + printf("\n-----------------------------------\n"); + printf("netdump in progress. searching for server.. "); + if (netdump_arp_server()) { + printf("Failed to locate server MAC address\n"); + error =3D EINVAL; + goto trig_abort; + } + if (netdump_send(NETDUMP_HERALD, 0, NULL, 0) !=3D 0) { + printf("Failed to contact netdump server\n"); + error =3D EINVAL; + goto trig_abort; + } + printf("dumping to %s (%6D)\n", inet_ntoa(nd_server), = nd_gw_mac.octet, + ":"); + printf("-----------------------------------\n"); + + /* Call the dumping routine. */ + dumper.dumper =3D netdump_dumper; + dumper.priv =3D NULL; + dumper.blocksize =3D NETDUMP_DATASIZE; + dumper.mediasize =3D 0; + dumper.mediaoffset =3D 0; + dumpsys(&dumper); + if (dump_failed !=3D 0) { + printf("Failed to dump the actual raw datas\n"); + error =3D EINVAL; + goto trig_abort; + } + if (netdump_send(NETDUMP_FINISHED, 0, NULL, 0) !=3D 0) { + printf("Failed to close the transaction\n"); + error =3D EINVAL; + goto trig_abort; + } + printf("\nnetdump finished.\n"); +trig_abort: + if (nd_gw_unset !=3D 0) + nd_gw.s_addr =3D INADDR_ANY; + if (old_if_input) + nd_ifp->if_input =3D old_if_input; + if ((nd_ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && must_lock = !=3D 0) + nd_ifp->if_ndumpfuncs->ne_enable_intr(nd_ifp); + return (error); +} + +/*- + * KLD specific code. + */ + +/* + * Called upon system init. Initializes the sysctl variables to sane = defaults + * (locates the first available NIC and uses the first IPv4 IP on that = card as + * the client IP). Leaves the server IP unconfigured. + * + * Parameters: + * void *, unused + * + * Returns: + * void + */ +static void +netdump_config_defaults(void *dummy __unused) +{ + struct ifnet *ifp; + int found; + + nd_ifp =3D NULL; + nd_server.s_addr =3D INADDR_ANY; + nd_client.s_addr =3D INADDR_ANY; + nd_gw.s_addr =3D INADDR_ANY; + + if (nd_server_tun[0] !=3D '\0') + inet_aton(nd_server_tun, &nd_server); + if (nd_client_tun[0] !=3D '\0') + inet_aton(nd_client_tun, &nd_client); + if (nd_gw_tun[0] !=3D '\0') + inet_aton(nd_gw_tun, &nd_gw); + if (nd_nic_tun[0] !=3D '\0') { + found =3D 0; + IFNET_RLOCK_NOSLEEP(); + TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if (!strncmp(ifp->if_xname, nd_nic_tun, + strlen(ifp->if_xname))) { + found =3D 1; + break; + } + } + IFNET_RUNLOCK_NOSLEEP(); + if (found !=3D 0 && netdump_supported_nic(ifp)) + nd_ifp =3D ifp; + } +} +SYSINIT(netdump, SI_SUB_KLD, SI_ORDER_ANY, netdump_config_defaults, = NULL); diff --git a/sys/sparc64/sparc64/dump_machdep.c = b/sys/sparc64/sparc64/dump_machdep.c index d5409ac..5d0ad9e 100644 --- a/sys/sparc64/sparc64/dump_machdep.c +++ b/sys/sparc64/sparc64/dump_machdep.c @@ -159,17 +159,22 @@ dumpsys(struct dumperinfo *di) DEV_BSIZE); size +=3D hdrsize; =20 - totsize =3D size + 2 * sizeof(kdh); - if (totsize > di->mediasize) { - printf("Insufficient space on device (need %ld, have = %ld), " - "refusing to dump.\n", (long)totsize, - (long)di->mediasize); - error =3D ENOSPC; - goto fail; - } + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + totsize =3D size + 2 * sizeof(kdh); + if (totsize > di->mediasize) { + printf("Insufficient space on device (need %ld, = " + "have %ld), refusing to dump.\n", = (long)totsize, + (long)di->mediasize); + error =3D ENOSPC; + goto fail; + } =20 - /* Determine dump offset on device. */ - dumplo =3D di->mediaoffset + di->mediasize - totsize; + /* Determine dump offset on device. */ + dumplo =3D di->mediaoffset + di->mediasize - totsize; + } =20 mkdumpheader(&kdh, KERNELDUMPMAGIC, KERNELDUMP_SPARC64_VERSION, = size, di->blocksize); diff --git a/sys/x86/x86/dump_machdep.c b/sys/x86/x86/dump_machdep.c index 5c874f4..c75b1ff 100644 --- a/sys/x86/x86/dump_machdep.c +++ b/sys/x86/x86/dump_machdep.c @@ -311,13 +311,20 @@ dumpsys(struct dumperinfo *di) dumpsize +=3D fileofs; hdrgap =3D fileofs - DEV_ALIGN(hdrsz); =20 - /* Determine dump offset on device. */ - if (di->mediasize < SIZEOF_METADATA + dumpsize + sizeof(kdh) * = 2) { - error =3D ENOSPC; - goto fail; + /* If the upper bound is 0, dumper likely will not use disks. */ + if ((di->mediaoffset + di->mediasize) =3D=3D 0) + dumplo =3D 0; + else { + + /* Determine dump offset on device. */ + if (di->mediasize < + SIZEOF_METADATA + dumpsize + sizeof(kdh) * 2) { + error =3D ENOSPC; + goto fail; + } + dumplo =3D di->mediaoffset + di->mediasize - dumpsize; + dumplo -=3D sizeof(kdh) * 2; } - dumplo =3D di->mediaoffset + di->mediasize - dumpsize; - dumplo -=3D sizeof(kdh) * 2; =20 mkdumpheader(&kdh, KERNELDUMPMAGIC, KERNELDUMP_VERSION, = dumpsize, di->blocksize); diff --git a/usr.sbin/netdumpsrv/Makefile b/usr.sbin/netdumpsrv/Makefile new file mode 100644 index 0000000..08ea0ae --- /dev/null +++ b/usr.sbin/netdumpsrv/Makefile @@ -0,0 +1,13 @@ +# $FreeBSD$ + +PROG=3D netdumpsrv +MAN=3D netdumpsrv.8 + +CFLAGS+=3D -DNDEBUG -I${.CURDIR} + +WARNS?=3D 6 + +DPADD=3D ${LIBUTIL} +LDADD=3D -lutil + +.include diff --git a/usr.sbin/netdumpsrv/netdumpsrv.8 = b/usr.sbin/netdumpsrv/netdumpsrv.8 new file mode 100644 index 0000000..767f087 --- /dev/null +++ b/usr.sbin/netdumpsrv/netdumpsrv.8 @@ -0,0 +1,76 @@ +.\" Copyright (c) 2011 Sandvine Incorporated. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above = copyright +.\" notice, this list of conditions and the following disclaimer in = the +.\" documentation and/or other materials provided with the = distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd August 23, 2010 +.Dt NETDUMPSRV 8 +.Os +.Sh NAME +.Nm netdumpsrv +.Nd receive kernel core dumps over the network +.Sh SYNOPSIS +.Nm +.Op Fl a Ar addr +.Op Fl D +.Op Fl d Ar dumpdir +.Op Fl i Ar script +.Sh DESCRIPTION +The +.Nm +utility listens on a UDP socket for incoming connections +from a +.Fx +kernel core dumping over the network. +.Pp +The following options are available: +.Bl -tag -width indent +.It Fl a +Bind the daemon to the given address +.Dq Pa addr . +.It Fl D +Run the utility in debugging mode. +The daemon version is not entered while the output +is printed entirely on the console. +.It Fl d +Save the core dumps to the specified +.Dq Pa dumpdir +directory. +.It Fl i +Execute the script +.Dq Pa script +after each dump received. +The script accepts the following strings as parameters: +its name, +reasons for invokation, +the client address, +the client hostname, +the info file name and the core dump file name. +.El +.Sh SEE ALSO +.Xr dumpon 8 +.Sh HISTORY +The +.Nm +utility appeared in +.Fx 9.0 . diff --git a/usr.sbin/netdumpsrv/netdumpsrv.c = b/usr.sbin/netdumpsrv/netdumpsrv.c new file mode 100644 index 0000000..af6ac6d --- /dev/null +++ b/usr.sbin/netdumpsrv/netdumpsrv.c @@ -0,0 +1,911 @@ +/*- + * Copyright (c) 2005-2011 Sandvine Incorporated. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in = the + * documentation and/or other materials provided with the = distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#define MAX_DUMPS 256 /* Dumps per IP before to be = cleaned out. */ +#define CLIENT_TIMEOUT 600 /* Clients timeout (secs). */ +#define CLIENT_TPASS 10 /* Clients timeout pass (secs). = */ + +#define PFLAGS_ABIND 0x01 +#define PFLAGS_DDIR 0x02 +#define PFLAGS_DEBUG 0x04 +#define PFLAGS_SCRIPT 0x08 + +#define LOGERR(m, ...) = \ + (*phook)(LOG_ERR | LOG_DAEMON, (m), ## __VA_ARGS__) +#define LOGERR_PERROR(m) = \ + (*phook)(LOG_ERR | LOG_DAEMON, "%s: %s\n", m, strerror(errno)) +#define LOGINFO(m, ...) = \ + (*phook)(LOG_INFO | LOG_DAEMON, (m), ## __VA_ARGS__) +#define LOGWARN(m, ...) = \ + (*phook)(LOG_WARNING | LOG_DAEMON, (m), ## __VA_ARGS__) + +#define client_ntoa(cl) = \ + inet_ntoa((cl)->ip) +#define client_pinfo(cl, f, ...) = \ + fprintf((cl)->infofile, (f), ## __VA_ARGS__) + +struct netdump_client { + char infofilename[MAXPATHLEN]; + char corefilename[MAXPATHLEN]; + char hostname[NI_MAXHOST]; + time_t last_msg; + SLIST_ENTRY(netdump_client) iter; + struct in_addr ip; + FILE *infofile; + int corefd; + int sock; + unsigned short printed_port_warning: 1; + unsigned short any_data_rcvd: 1; +}; + +/* Clients list. */ +static SLIST_HEAD(, netdump_client) clients =3D = SLIST_HEAD_INITIALIZER(clients); + +/* Program arguments handlers. */ +static uint32_t pflags; +static char dumpdir[MAXPATHLEN]; +static char *handler_script; +static struct in_addr bindip; + +/* Miscellaneous handlers. */ +static struct pidfh *pfh; +static time_t now; +static time_t last_timeout_check; +static int do_shutdown; +static int sock; + +/* Daemon print functions hook. */ +static void (*phook)(int, const char *, ...); + +static struct netdump_client *alloc_client(struct sockaddr_in *sip); +static void eventloop(void); +static void exec_handler(struct netdump_client *client, + const char *reason); +static void free_client(struct netdump_client *client); +static void handle_finish(struct netdump_client *client, + struct netdump_msg *msg); +static void handle_herald(struct sockaddr_in *from, + struct netdump_client *client, + struct netdump_msg *msg); +static void handle_kdh(struct netdump_client *client, + struct netdump_msg *msg); +static void handle_packet(struct netdump_client *client, + struct sockaddr_in *from, const char = *fromstr, + struct netdump_msg *msg); +static void handle_timeout(struct netdump_client *client); +static void handle_vmcore(struct netdump_client *client, + struct netdump_msg *msg); +static void phook_printf(int priority, const char *message, = ...); +static int receive_message(int isock, struct sockaddr_in = *from, + char *fromstr, size_t fromstrlen, + struct netdump_msg *msg); +static void send_ack(struct netdump_client *client, + struct netdump_msg *msg); +static void signal_shutdown(int sig __unused); +static void timeout_clients(void); +static void usage(const char *cmd); + +static void +usage(const char *cmd) +{ + + fprintf(stderr, "Usage: %s [-a bind_addr] [-d dump_dir] [-i = script]\n", + cmd); +} + +static void +phook_printf(int priority, const char *message, ...) +{ + va_list ap; + + va_start(ap, message); + if ((priority & LOG_INFO) !=3D 0) { + assert((priority & (LOG_WARNING | LOG_ERR)) =3D=3D 0); + vprintf(message, ap); + } else + vfprintf(stderr, message, ap); +} + +static struct netdump_client * +alloc_client(struct sockaddr_in *sip) +{ + struct sockaddr_in saddr; + struct netdump_client *client; + struct in_addr *ip; + char *firstdot; + int i, ecode, fd, bufsz; + + assert(sip !=3D NULL); + + client =3D calloc(1, sizeof(*client)); + if (client =3D=3D NULL) { + LOGERR_PERROR("calloc()"); + return (NULL); + } + ip =3D &sip->sin_addr; + bcopy(ip, &client->ip, sizeof(*ip)); + client->corefd =3D -1; + client->sock =3D -1; + client->last_msg =3D now; + + ecode =3D getnameinfo((struct sockaddr *)sip, sip->sin_len, + client->hostname, sizeof(client->hostname), NULL, 0, = NI_NAMEREQD); + if (ecode !=3D 0) { + + /* Can't resolve, try with a numeric IP. */ + ecode =3D getnameinfo((struct sockaddr *)sip, = sip->sin_len, + client->hostname, sizeof(client->hostname), NULL, 0, = 0); + if (ecode !=3D 0) { + LOGERR("getnameinfo(): %s\n", = gai_strerror(ecode)); + free(client); + return (NULL); + } + } else { + + /* Strip off the domain name */ + firstdot =3D strchr(client->hostname, '.'); + if (firstdot) + *firstdot =3D '\0'; + } + + client->sock =3D socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP); + if (client->sock =3D=3D -1) { + LOGERR_PERROR("socket()"); + free(client); + return (NULL); + } + if (fcntl(client->sock, F_SETFL, O_NONBLOCK) =3D=3D -1) { + LOGERR_PERROR("fcntl()"); + close(client->sock); + free(client); + return (NULL); + } + bzero(&saddr, sizeof(saddr)); + saddr.sin_len =3D sizeof(saddr); + saddr.sin_family =3D AF_INET; + saddr.sin_addr.s_addr =3D bindip.s_addr; + saddr.sin_port =3D htons(0); + if (bind(client->sock, (struct sockaddr *)&saddr, = sizeof(saddr))) { + LOGERR_PERROR("bind()"); + close(client->sock); + free(client); + return (NULL); + } + bzero(&saddr, sizeof(saddr)); + saddr.sin_len =3D sizeof(saddr); + saddr.sin_family =3D AF_INET; + saddr.sin_addr.s_addr =3D ip->s_addr; + saddr.sin_port =3D htons(NETDUMP_ACKPORT); + if (connect(client->sock, (struct sockaddr *)&saddr, = sizeof(saddr))) { + LOGERR_PERROR("connect()"); + close(client->sock); + free(client); + return (NULL); + } + + /* It should be enough to hold approximatively twize the chunk = size. */ + bufsz =3D 131072; + if (setsockopt(client->sock, SOL_SOCKET, SO_RCVBUF, &bufsz, + sizeof(bufsz))) { + LOGERR_PERROR("setsockopt()"); + LOGWARN("May drop packets from %s due to small receive = buffer\n", + client->hostname); + } + + /* Try info.host.0 through info.host.255 in sequence. */ + for (i =3D 0; i < MAX_DUMPS; i++) { + snprintf(client->infofilename, = sizeof(client->infofilename), + "%s/info.%s.%d", dumpdir, client->hostname, i); + snprintf(client->corefilename, = sizeof(client->corefilename), + "%s/vmcore.%s.%d", dumpdir, client->hostname, i); + + /* Try the info file first. */ + fd =3D open(client->infofilename, O_WRONLY | O_CREAT | = O_EXCL, + DEFFILEMODE); + if (fd =3D=3D -1) { + if (errno !=3D EEXIST) + LOGERR("open(\"%s\"): %s\n", + client->infofilename, = strerror(errno)); + continue; + } + client->infofile =3D fdopen(fd, "w"); + if (client->infofile =3D=3D NULL) { + LOGERR_PERROR("fdopen()"); + close(fd); + unlink(client->infofilename); + continue; + } + + /* Next make the core file. */ + fd =3D open(client->corefilename, O_RDWR | O_CREAT | = O_EXCL, + DEFFILEMODE); + if (fd =3D=3D -1) { + + /* Failed. Keep the numbers in sync. */ + fclose(client->infofile); + unlink(client->infofilename); + client->infofile =3D NULL; + if (errno !=3D EEXIST) + LOGERR("open(\"%s\"): %s\n", + client->corefilename, = strerror(errno)); + continue; + } + client->corefd =3D fd; + break; + } + + if (client->infofile =3D=3D NULL || client->corefd =3D=3D -1) { + LOGERR("Can't create output files for new client %s = [%s]\n", + client->hostname, client_ntoa(client)); + if (client->infofile) + fclose(client->infofile); + if (client->corefd !=3D -1) + close(client->corefd); + if (client->sock !=3D -1) + close(client->sock); + free(client); + return (NULL); + } + SLIST_INSERT_HEAD(&clients, client, iter); + return (client); +} + +static void +free_client(struct netdump_client *client) +{ + + assert(client !=3D NULL); + + /* Remove from the list. Ignore errors from close() routines. = */ + SLIST_REMOVE(&clients, client, netdump_client, iter); + fclose(client->infofile); + close(client->corefd); + close(client->sock); + free(client); +} + +static void +exec_handler(struct netdump_client *client, const char *reason) +{ + int pid; + + assert(client !=3D NULL); + + /* If no script is specified this is a no-op. */ + if ((pflags & PFLAGS_SCRIPT) =3D=3D 0) + return; + + pid =3D fork(); + + /* + * The function is invoked in critical conditions, thus just = exiting + * without reporting errors is fine. + */ + if (pid =3D=3D -1) { + LOGERR_PERROR("fork()"); + return; + } else if (pid !=3D 0) { + close(sock); + pidfile_close(pfh); + if (execl(handler_script, handler_script, reason, + client_ntoa(client), client->hostname, + client->infofilename, client->corefilename, NULL) =3D=3D= -1) { + LOGERR_PERROR("fork()"); + _exit(1); + } + } +} + +static void +handle_timeout(struct netdump_client *client) +{ + + assert(client !=3D NULL); + + LOGINFO("Client %s timed out\n", client_ntoa(client)); + client_pinfo(client, "Dump incomplete: client timed out\n"); + exec_handler(client, "timeout"); + free_client(client); +} + +static void +timeout_clients(void) +{ + struct netdump_client *client, *tmp; + =20 + /* Only time out clients every 10 seconds. */ + if (now - last_timeout_check < CLIENT_TPASS) + return; + last_timeout_check =3D now; + + /* Traverse the list looking for stale clients. */ + SLIST_FOREACH_SAFE(client, &clients, iter, tmp) { + if (client->last_msg + CLIENT_TIMEOUT < now) { + LOGINFO("Timingout with such values: %jd + %jd < = %jd\n", + (intmax_t)client->last_msg, + (intmax_t)CLIENT_TIMEOUT, (intmax_t)now); + handle_timeout(client); + } + } +} + +static void +send_ack(struct netdump_client *client, struct netdump_msg *msg) +{ + struct netdump_ack ack; + int tryagain; + =20 + assert(client !=3D NULL && msg !=3D NULL); + + bzero(&ack, sizeof(ack)); + ack.na_seqno =3D htonl(msg->nm_hdr.mh_seqno); + do { + tryagain =3D 0; + if (send(client->sock, &ack, sizeof(ack), 0) =3D=3D -1) = { + if (errno =3D=3D EINTR) { + tryagain =3D 1; + continue; + } + + /* + * XXX: On EAGAIN, we should probably queue the = packet + * to be sent when the socket is writable but + * that is too much effort, since it is mostly + * harmless to wait for the client to = retransmit. + */ + LOGERR_PERROR("send()"); + } + } while (tryagain); +} + +static void +handle_herald(struct sockaddr_in *from, struct netdump_client *client, + struct netdump_msg *msg) +{ + + assert(from !=3D NULL && msg !=3D NULL); + + if (client !=3D NULL) { + if (client->any_data_rcvd =3D=3D 0) { + + /* Must be a retransmit of the herald packet. */ + send_ack(client, msg); + return; + } + + /* An old connection must have timed out. Clean it up = first. */ + handle_timeout(client); + } + + client =3D alloc_client(from); + if (client =3D=3D NULL) { + LOGERR("handle_herald(): new client allocation = failure\n"); + return; + } + client_pinfo(client, "Dump from %s [%s]\n", client->hostname, + client_ntoa(client)); + LOGINFO("New dump from client %s [%s] (to %s)\n", = client->hostname, + client_ntoa(client), client->corefilename); + send_ack(client, msg); +} + +static void +handle_kdh(struct netdump_client *client, struct netdump_msg *msg) +{ + time_t t; + uint64_t dumplen; + struct kerneldumpheader *h; + int parity_check; + + assert(msg !=3D NULL); + + if (client =3D=3D NULL) + return; + + client->any_data_rcvd =3D 1; + h =3D (struct kerneldumpheader *)msg->nm_data; + if (msg->nm_hdr.mh_len < sizeof(struct kerneldumpheader)) { + LOGERR("Bad KDH from %s [%s]: packet too small\n", + client->hostname, client_ntoa(client)); + client_pinfo(client, "Bad KDH: packet too small\n"); + fflush(client->infofile); + send_ack(client, msg); + return; + } + parity_check =3D kerneldump_parity(h); + + /* Make sure all the strings are null-terminated. */ + h->architecture[sizeof(h->architecture) - 1] =3D '\0'; + h->hostname[sizeof(h->hostname) - 1] =3D '\0'; + h->versionstring[sizeof(h->versionstring) - 1] =3D '\0'; + h->panicstring[sizeof(h->panicstring) - 1] =3D '\0'; + + client_pinfo(client, " Architecture: %s\n", h->architecture); + client_pinfo(client, " Architecture version: %d\n", + dtoh32(h->architectureversion)); + dumplen =3D dtoh64(h->dumplength); + client_pinfo(client, " Dump length: %lldB (%lld MB)\n", + (long long)dumplen, (long long)(dumplen >> 20)); + client_pinfo(client, " blocksize: %d\n", dtoh32(h->blocksize)); + t =3D dtoh64(h->dumptime); + client_pinfo(client, " Dumptime: %s", ctime(&t)); + client_pinfo(client, " Hostname: %s\n", h->hostname); + client_pinfo(client, " Versionstring: %s", h->versionstring); + client_pinfo(client, " Panicstring: %s\n", h->panicstring); + client_pinfo(client, " Header parity check: %s\n", + parity_check ? "Fail" : "Pass"); + fflush(client->infofile); + + LOGINFO("(KDH from %s [%s])", client->hostname, = client_ntoa(client)); + send_ack(client, msg); +} + +static void +handle_vmcore(struct netdump_client *client, struct netdump_msg *msg) +{ + + assert(msg !=3D NULL); + + if (client =3D=3D NULL) + return; + + client->any_data_rcvd =3D 1; + if (msg->nm_hdr.mh_seqno % 11523 =3D=3D 0) { + + /* Approximately every 16MB with MTU of 1500 */ + LOGINFO("."); + } + if (pwrite(client->corefd, msg->nm_data, msg->nm_hdr.mh_len, + msg->nm_hdr.mh_offset) =3D=3D -1) { + LOGERR("pwrite (for client %s [%s]): %s\n", = client->hostname, + client_ntoa(client), strerror(errno)); + client_pinfo(client, + "Dump unsuccessful: write error @ offset = %08"PRIx64": %s\n", + msg->nm_hdr.mh_offset, strerror(errno)); + exec_handler(client, "error"); + free_client(client); + return; + } + send_ack(client, msg); +} + +static void +handle_finish(struct netdump_client *client, struct netdump_msg *msg) +{ + + assert(msg !=3D NULL); + + if (client =3D=3D NULL) + return; + + LOGINFO("\nCompleted dump from client %s [%s]\n", = client->hostname, + client_ntoa(client)); + client_pinfo(client, "Dump complete\n"); + send_ack(client, msg); + exec_handler(client, "success"); + free_client(client); +} + + +static int +receive_message(int isock, struct sockaddr_in *from, char *fromstr, + size_t fromstrlen, struct netdump_msg *msg) +{ + socklen_t fromlen; + ssize_t len; + + assert(from !=3D NULL && fromstr !=3D NULL && msg !=3D NULL); + + bzero(from, sizeof(*from)); + from->sin_family =3D AF_INET; + from->sin_len =3D fromlen =3D sizeof(*from); + from->sin_port =3D 0; + from->sin_addr.s_addr =3D INADDR_ANY; + + len =3D recvfrom(isock, msg, sizeof(*msg), 0, (struct sockaddr = *)from, + &fromlen); + if (len =3D=3D -1) { + + /* + * As long as some callers may discard the errors = printing + * in defined circumstances, leave them the choice and = avoid + * any error reporting. + */ + return (-1); + } + + snprintf(fromstr, fromstrlen, "%s:%hu", = inet_ntoa(from->sin_addr), + ntohs(from->sin_port)); + if ((size_t)len < sizeof(struct netdump_msg_hdr)) { + LOGERR("Ignoring runt packet from %s (got %zu)\n", = fromstr, + (size_t)len); + return (0); + } + + /* Convert byte order. */ + msg->nm_hdr.mh_type =3D ntohl(msg->nm_hdr.mh_type); + msg->nm_hdr.mh_seqno =3D ntohl(msg->nm_hdr.mh_seqno); + msg->nm_hdr.mh_offset =3D be64toh(msg->nm_hdr.mh_offset); + msg->nm_hdr.mh_len =3D ntohl(msg->nm_hdr.mh_len); + + if ((size_t)len < sizeof(struct netdump_msg_hdr) + = msg->nm_hdr.mh_len) { + LOGERR("Packet too small from %s (got %zu, expected = %zu)\n", + fromstr, (size_t)len, + sizeof(struct netdump_msg_hdr) + = msg->nm_hdr.mh_len); + return (0); + } + return (len); +} + +static void +handle_packet(struct netdump_client *client, struct sockaddr_in *from, + const char *fromstr, struct netdump_msg *msg) +{ + + assert(from !=3D NULL && fromstr !=3D NULL && msg !=3D NULL); + + if (client !=3D NULL) + client->last_msg =3D time(NULL); + + switch (msg->nm_hdr.mh_type) { + case NETDUMP_HERALD: + handle_herald(from, client, msg); + break; + case NETDUMP_KDH: + handle_kdh(client, msg); + break; + case NETDUMP_VMCORE: + handle_vmcore(client, msg); + break; + case NETDUMP_FINISHED: + handle_finish(client, msg); + break; + default: + LOGERR("Received unknown message type %d from %s\n", + msg->nm_hdr.mh_type, fromstr); + } +} + +static void +eventloop(void) +{ + struct netdump_msg msg; + char fromstr[INET_ADDRSTRLEN + 6]; + fd_set readfds; + struct sockaddr_in from; + struct timeval tv; + struct netdump_client *client, *tmp; + int len, maxfd; + + while (do_shutdown =3D=3D 0) { + maxfd =3D sock + 1; + FD_ZERO(&readfds); + FD_SET(sock, &readfds); + SLIST_FOREACH(client, &clients, iter) { + FD_SET(client->sock, &readfds); + if (maxfd <=3D client->sock) + maxfd =3D client->sock+1; + } + + /* So that we time out clients regularly. */ + tv.tv_sec =3D CLIENT_TPASS; + tv.tv_usec =3D 0; + if (select(maxfd, &readfds, NULL, NULL, &tv) =3D=3D -1) = { + if (errno =3D=3D EINTR) + continue; + LOGERR_PERROR("select()"); + + /* + * Errors with select() probably will not go = away + * with simple retrying. + */ + pidfile_remove(pfh); + exit(1); + } + now =3D time(NULL); + if (FD_ISSET(sock, &readfds)) { + len =3D receive_message(sock, &from, fromstr, + sizeof(fromstr), &msg); + if (len =3D=3D -1) { + if (errno =3D=3D EINTR) + continue; + if (errno !=3D EAGAIN) { + pidfile_remove(pfh); + LOGERR_PERROR("recvfrom()"); + exit(1); + } + } else if (len !=3D 0) { + + /* + * With len =3D=3D 0 the packet was = rejected + * (probably because it was too small) = so just + * ignore this case. + */ + + /* Check if they are on the clients = list. */ + SLIST_FOREACH(client, &clients, iter) + if (client->ip.s_addr =3D=3D + from.sin_addr.s_addr) + break; + + /* + * Technically, clients should not be + * responding on the server port, so = client + * should be NULL, however, if they = insist on + * doing so, it's not really going to = hurt + * anything (except maybe fill up the = server + * socket's receive buffer), so still + * accept it. The only possibly = legitimate case + * is if there's a new dump starting and = the + * previous one didn't finish cleanly. = Handle + * this by suppressing the error on = HERALD + * packets. + */ + if (client !=3D NULL && + msg.nm_hdr.mh_type !=3D = NETDUMP_HERALD && + client->printed_port_warning =3D=3D = 0) { + LOGWARN("Client %s responding on server = port\n", + client->hostname); + client->printed_port_warning =3D = 1; + } + handle_packet(client, &from, fromstr, = &msg); + } + } + + /* + * handle_packet() and handle_timeout() may free the = client, + * handle stale pointers. + */ + SLIST_FOREACH_SAFE(client, &clients, iter, tmp) { + if (FD_ISSET(client->sock, &readfds)) { + len =3D receive_message(client->sock, = &from, + fromstr, sizeof(fromstr), &msg); + if (len =3D=3D -1) { + if (errno =3D=3D EINTR || errno = =3D=3D EAGAIN) + continue; + LOGERR_PERROR("recvfrom()"); + + /* + * Client socket is broken for + * some reason. + */ + handle_timeout(client); + } else if (len !=3D 0) { + + /* + * With len =3D=3D 0 the packet = was + * rejected (probably because it = was + * too small) so just ignore = this case. + */ + + FD_CLR(client->sock, &readfds); + handle_packet(client, &from, = fromstr, + &msg); + } + } + } + timeout_clients(); + } + LOGINFO("Shutting down..."); + + /* + * Clients is the head of the list, so clients !=3D NULL iff the = list + * is not empty. Call it a timeout so that the scripts get run. + */ + while (!SLIST_EMPTY(&clients)) + handle_timeout(SLIST_FIRST(&clients)); +} + +static void +signal_shutdown(int sig __unused) +{ + + do_shutdown =3D 1; +} + +int +main(int argc, char **argv) +{ + struct stat statbuf; + struct sockaddr_in bindaddr; + struct sigaction sa; + int ch; + + pfh =3D pidfile_open(NULL, 0600, NULL); + if (pfh =3D=3D NULL) { + if (errno =3D=3D EEXIST) + printf("Instance of netdump already running\n"); + else + printf("Impossible to open the pid file\n"); + exit(1); + } + + while ((ch =3D getopt(argc, argv, "a:Dd:i:")) !=3D -1) { + switch (ch) { + case 'a': + pflags |=3D PFLAGS_ABIND; + if (!inet_aton(optarg, &bindip)) { + pidfile_remove(pfh); + fprintf(stderr, "Invalid bind IP = specified\n"); + exit(1); + } + printf("Listening on IP %s\n", optarg); + break; + case 'D': + pflags |=3D PFLAGS_DEBUG; + break; + case 'd': + pflags |=3D PFLAGS_DDIR; + assert(dumpdir[0] =3D=3D '\0'); + strncpy(dumpdir, optarg, sizeof(dumpdir) - 1); + break; + case 'i': + pflags |=3D PFLAGS_SCRIPT; + + /* + * When suddenly closing the process for an = error, + * it is unuseful to take care of handler_script + * deallocation as long as the process will = _exit(2) + * anyway. + */ + handler_script =3D strdup(optarg); + if (handler_script =3D=3D NULL) { + pidfile_remove(pfh); + perror("strdup()"); + fprintf(stderr, "Unable to set script = file\n"); + exit(1); + } + if (access(handler_script, F_OK | X_OK)) { + pidfile_remove(pfh); + perror("access()"); + fprintf(stderr, + "Unable to access script file\n"); + exit(1); + } + break; + default: + pidfile_remove(pfh); + usage(argv[0]); + exit(1); + } + } + if ((pflags & PFLAGS_ABIND) =3D=3D 0) { + bindip.s_addr =3D INADDR_ANY; + printf("Default: listening on all interfaces\n"); + } + if ((pflags & PFLAGS_DDIR) =3D=3D 0) { + strcpy(dumpdir, "/var/crash"); + printf("Default: dumping on /var/crash/\n"); + } + if ((pflags & PFLAGS_DEBUG) =3D=3D 0) + phook =3D syslog; + else + phook =3D phook_printf; + + /* Further sanity checks on dump location. */ + if (stat(dumpdir, &statbuf)) { + pidfile_remove(pfh); + perror("stat()"); + fprintf(stderr, "Invalid dump location specified\n"); + exit(1); + } + if ((statbuf.st_mode & S_IFMT) !=3D S_IFDIR) { + pidfile_remove(pfh); + fprintf(stderr, "Dump location is not a directory\n"); + exit(1); + } + if (access(dumpdir, F_OK | W_OK)) { + fprintf(stderr, + "Warning: May be unable to write into dump location: = %s\n", + strerror(errno)); + } + + if ((pflags & PFLAGS_DEBUG) =3D=3D 0 && daemon(0, 0) =3D=3D -1) = { + pidfile_remove(pfh); + perror("daemon()"); + fprintf(stderr, "Impossible to demonize the process\n"); + exit(1); + } + pidfile_write(pfh); + + /* Set up the server socket. */ + sock =3D socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP); + if (sock =3D=3D -1) { + pidfile_remove(pfh); + LOGERR_PERROR("socket()"); + exit(1); + } + bzero(&bindaddr, sizeof(bindaddr)); + bindaddr.sin_len =3D sizeof(bindaddr); + bindaddr.sin_family =3D AF_INET; + bindaddr.sin_addr.s_addr =3D bindip.s_addr; + bindaddr.sin_port =3D htons(NETDUMP_PORT); + if (bind(sock, (struct sockaddr *)&bindaddr, sizeof(bindaddr))) = { + pidfile_remove(pfh); + close(sock); + LOGERR_PERROR("bind()"); + exit(1); + } + if (fcntl(sock, F_SETFL, O_NONBLOCK) =3D=3D -1) { + pidfile_remove(pfh); + close(sock); + LOGERR_PERROR("fcntl()"); + exit(1); + } + + /* Override some signal handlers. */ + bzero(&sa, sizeof(sa)); + sa.sa_handler =3D signal_shutdown; + if (sigaction(SIGINT, &sa, NULL) || sigaction(SIGTERM, &sa, = NULL)) { + pidfile_remove(pfh); + close(sock); + LOGERR_PERROR("sigaction(SIGINT | SIGTERM)"); + exit(1); + } + bzero(&sa, sizeof(sa)); + sa.sa_handler =3D SIG_IGN; + sa.sa_flags =3D SA_NOCLDWAIT; + if (sigaction(SIGCHLD, &sa, NULL)) { + pidfile_remove(pfh); + LOGERR_PERROR("sigaction(SIGCHLD)"); + close(sock); + exit(1); + } + + LOGINFO("Waiting for clients.\n"); + eventloop(); + + pidfile_remove(pfh); + return (0); +} --Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3 Content-Disposition: attachment; filename=kernel-netdump-diffs.txt Content-Type: text/plain; name="kernel-netdump-diffs.txt" Content-Transfer-Encoding: quoted-printable diff --git a/sys/conf/files b/sys/conf/files index 46d0d27..b910b1a 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -3262,6 +3264,7 @@ netinet/ip_mroute.c optional = mrouting inet netinet/ip_options.c optional inet netinet/ip_output.c optional inet netinet/netdump_client.c optional inet netdump_client +netinet/network_debug.c optional inet netdump_client netinet/raw_ip.c optional inet | inet6 netinet/cc/cc.c optional inet | inet6 netinet/cc/cc_newreno.c optional inet | inet6 diff --git a/sys/netinet/netdump_client.c b/sys/netinet/netdump_client.c index 1b73e1b..ad4c347 100644 --- a/sys/netinet/netdump_client.c +++ b/sys/netinet/netdump_client.c @@ -52,6 +52,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include =20 @@ -69,6 +70,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include =20 @@ -96,20 +98,15 @@ __FBSDID("$FreeBSD$"); #define NETDDEBUGV_IF(i, f, ...) #endif =20 -static void nd_handle_arp(struct mbuf **mb); -static void nd_handle_ip(struct mbuf **mb); +static void nd_handle_ip(struct network_debug *, struct mbuf **mb); +static int nd_packet_wanted(struct network_debug *, struct mbuf *); static int netdump_arp_server(void); static void netdump_config_defaults(void *dummy __unused); static int netdump_dumper(void *priv __unused, void *virtual, vm_offset_t physical __unused, off_t offset, size_t = length); -static int netdump_ether_output(struct mbuf *m, struct ifnet *ifp,=20= - struct ether_addr dst, u_short etype); -static int netdump_mbuf_nop(struct mbuf *mp __unused, void *ptr = __unused, void *opt_args __unused); -static void netdump_network_poll(void); static void netdump_pkt_in(struct ifnet *ifp, struct mbuf *m); static int netdump_send(uint32_t type, off_t offset, unsigned char = *data, uint32_t datalen); -static int netdump_send_arp(void); static int netdump_udp_output(struct mbuf *m); =20 static int sysctl_handle_ifxname(SYSCTL_HANDLER_ARGS); @@ -119,49 +116,30 @@ static int = sysctl_handle_inaddr(SYSCTL_HANDLER_ARGS); static unsigned char buf[MAXDUMPPGS * PAGE_SIZE]; static uint64_t rcvd_acks; static uint32_t nd_seqno =3D 1; -static int dump_failed, have_server_mac; - -/* - * Times to poll the NIC (0.5ms each poll) before assuming packetloss - * occurred (default to 5s). - */ -static int nd_polls =3D 10000; - -/* Times to retransmit lost packets. */ -static int nd_retries =3D 10; - -/* General dynamic settings. */ -static char nd_ifp_str[IFNAMSIZ]; -static struct ether_addr nd_gw_mac; -static struct in_addr nd_server =3D {INADDR_ANY}; -static struct in_addr nd_client =3D {INADDR_ANY}; -static struct in_addr nd_gw =3D {INADDR_ANY}; -struct ifnet *nd_ifp; -int nd_enable =3D 0; -static uint16_t nd_server_port =3D NETDUMP_PORT; +static int dump_failed; + +static struct network_debug __unused +nd_networking =3D { + .poll_time =3D 10000, + .retries =3D 10, + .if_name =3D { 0 }, + .ifp =3D NULL, + .remote =3D {INADDR_ANY}, + .local =3D {INADDR_ANY}, + .gw =3D {INADDR_ANY}, + .ip_handler =3D nd_handle_ip, + .wanted =3D nd_packet_wanted, +}; + +int nd_enable =3D 0; // Used in kern_shutdown, so can't be static +static uint16_t nd_remote_port =3D NETDUMP_PORT; =20 /* Tunables storages. */ -static char nd_server_tun[INET_ADDRSTRLEN]; -static char nd_client_tun[INET_ADDRSTRLEN]; +static char nd_remote_tun[INET_ADDRSTRLEN]; +static char nd_local_tun[INET_ADDRSTRLEN]; static char nd_gw_tun[INET_ADDRSTRLEN]; static char nd_nic_tun[IFNAMSIZ]; =20 -/* - * Checks for netdump support on a network interface - * - * Parameters: - * ifp The network interface that is being tested for support - * - * Returns: - * int 1 if the interface is supported, 0 if not - */ -static __inline int -netdump_supported_nic(struct ifnet *ifp) -{ - - return (ifp->if_ndumpfuncs !=3D NULL); -} - /*- * Sysctls specific code. */ @@ -192,7 +170,7 @@ sysctl_handle_ifxname(SYSCTL_HANDLER_ARGS) IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { if (!strncmp(ifp->if_xname, buf, strlen(ifp->if_xname)) = && - netdump_supported_nic(ifp)) { + network_debug_supported_nic(ifp)) { found =3D 1; break; } @@ -235,24 +213,24 @@ sysctl_handle_inaddr(SYSCTL_HANDLER_ARGS) =20 SYSCTL_NODE(_net, OID_AUTO, dump, CTLFLAG_RW | CTLFLAG_MPSAFE, 0, = "netdump"); SYSCTL_PROC(_net_dump, OID_AUTO, server, CTLTYPE_STRING | CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_server, 0, sysctl_handle_inaddr, "A", "dump = server"); + CTLFLAG_MPSAFE, &nd_networking.remote, 0, sysctl_handle_inaddr, = "A", "dump server"); SYSCTL_PROC(_net_dump, OID_AUTO, client, CTLTYPE_STRING |CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_client, 0, sysctl_handle_inaddr, "A", "dump = client"); + CTLFLAG_MPSAFE, &nd_networking.local, 0, sysctl_handle_inaddr, "A", = "dump client"); SYSCTL_PROC(_net_dump, OID_AUTO, gateway, CTLTYPE_STRING | CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_gw, 0, sysctl_handle_inaddr, "A", + CTLFLAG_MPSAFE, &nd_networking.gw, 0, sysctl_handle_inaddr, "A", "dump default gateway"); SYSCTL_PROC(_net_dump, OID_AUTO, nic, CTLTYPE_STRING | CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_ifp_str, 0, sysctl_handle_ifxname, "A", + CTLFLAG_MPSAFE, &nd_networking.if_name, 0, sysctl_handle_ifxname, = "A", "dumping interface name"); SYSCTL_INT(_net_dump, OID_AUTO, polls, CTLTYPE_INT | CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_polls, 0, "times to poll NIC per retry"); + CTLFLAG_MPSAFE, &nd_networking.poll_time, 0, "times to poll NIC per = retry"); SYSCTL_INT(_net_dump, OID_AUTO, retries, CTLTYPE_INT | CTLFLAG_RW | - CTLFLAG_MPSAFE, &nd_retries, 0, "times to retransmit lost = packets"); + CTLFLAG_MPSAFE, &nd_networking.retries, 0, "times to retransmit = lost packets"); SYSCTL_INT(_net_dump, OID_AUTO, enable, CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, &nd_enable, 0, "enable network dump"); =20 -TUNABLE_STR("net.dump.server", nd_server_tun, sizeof(nd_server_tun)); -TUNABLE_STR("net.dump.client", nd_client_tun, sizeof(nd_client_tun)); +TUNABLE_STR("net.dump.server", nd_remote_tun, sizeof(nd_remote_tun)); +TUNABLE_STR("net.dump.client", nd_local_tun, sizeof(nd_local_tun)); TUNABLE_STR("net.dump.gateway", nd_gw_tun, sizeof(nd_gw_tun)); TUNABLE_STR("net.dump.nic", nd_nic_tun, sizeof(nd_nic_tun)); TUNABLE_INT("net.dump.enable", &nd_enable); @@ -265,66 +243,7 @@ TUNABLE_INT("net.dump.enable", &nd_enable); * - Polling primitives */ =20 -/* - * Netdump wraps external mbufs around address ranges. unlike most = sane - * counterparts, netdump uses a stop-and-wait approach to flow control = and - * retransmission, so the ack obviates the need for mbuf reference - * counting. We still need to tell other mbuf handlers not to do = anything - * special with our mbufs, so specify this nop handler. - * - * Parameters: - * ptr data to free (ignored) - * opt_args callback pointer (ignored) - * - * Returns: - * void - */ -static int -netdump_mbuf_nop(struct mbuf *mp __unused, void *ptr __unused, void = *opt_args __unused) -{ - return EXT_FREE_OK; -} - -/* - * Handles creation of the ethernet header, then places outgoing = packets into - * the tx buffer for the NIC - * - * Parameters: - * m The mbuf containing the packet to be sent (will be freed = by - * this function or the NIC driver) - * ifp The interface to send on - * dst The destination ethernet address (source address will be = looked - * up using ifp) - * etype The ETHERTYPE_* value for the protocol that is being = sent - * - * Returns: - * int see errno.h, 0 for success - */ -static int -netdump_ether_output(struct mbuf *m, struct ifnet *ifp, struct = ether_addr dst, - u_short etype) -{ - struct ether_header *eh; =20 - /* Fill in the ethernet header. */ - M_PREPEND(m, ETHER_HDR_LEN, M_NOWAIT); - if (m =3D=3D NULL) { - printf("netdump_ether_output: Out of mbufs\n"); - return (ENOBUFS); - } - eh =3D mtod(m, struct ether_header *); - memcpy(eh->ether_shost, IF_LLADDR(ifp), ETHER_ADDR_LEN); - memcpy(eh->ether_dhost, dst.octet, ETHER_ADDR_LEN); - eh->ether_type =3D htons(etype); - - if (((ifp->if_flags & (IFF_MONITOR | IFF_UP)) !=3D IFF_UP) || - (ifp->if_drv_flags & IFF_DRV_RUNNING) !=3D IFF_DRV_RUNNING) = { - if_printf(ifp, "netdump_ether_output: Interface isn't = up\n"); - m_freem(m); - return (ENETDOWN); - } - return ((ifp->if_transmit)(ifp, m)); -} =20 /* * Unreliable transmission of an mbuf chain to the netdump server @@ -343,7 +262,7 @@ netdump_udp_output(struct mbuf *m) struct udpiphdr *ui; struct ip *ip; =20 - MPASS(nd_ifp !=3D NULL); + MPASS(nd_networking.ifp !=3D NULL); =20 M_PREPEND(m, sizeof(struct udpiphdr), M_NOWAIT); if (m =3D=3D NULL) { @@ -355,11 +274,11 @@ netdump_udp_output(struct mbuf *m) ui->ui_pr =3D IPPROTO_UDP; ui->ui_len =3D htons(m->m_pkthdr.len - sizeof(struct ip)); ui->ui_ulen =3D ui->ui_len; - ui->ui_src =3D nd_client; - ui->ui_dst =3D nd_server; + ui->ui_src =3D nd_networking.local; + ui->ui_dst =3D nd_networking.remote; /* Use this src port so that the server can connect() the socket = */ ui->ui_sport =3D htons(NETDUMP_ACKPORT); - ui->ui_dport =3D htons(nd_server_port); + ui->ui_dport =3D htons(nd_remote_port); ui->ui_sum =3D 0; if ((ui->ui_sum =3D in_cksum(m, m->m_pkthdr.len)) =3D=3D 0) ui->ui_sum =3D 0xffff; @@ -375,59 +294,15 @@ netdump_udp_output(struct mbuf *m) ip->ip_sum =3D 0; ip->ip_sum =3D in_cksum(m, sizeof(struct ip)); =20 - if (m->m_pkthdr.len > nd_ifp->if_mtu) { + if (m->m_pkthdr.len > nd_networking.ifp->if_mtu) { printf("netdump_udp_output: Packet is too big: %u > MTU = %lu\n", - m->m_pkthdr.len, (unsigned long)nd_ifp->if_mtu); + m->m_pkthdr.len, (unsigned = long)nd_networking.ifp->if_mtu); m_freem(m); return (ENOBUFS); } - return (netdump_ether_output(m, nd_ifp, nd_gw_mac, = ETHERTYPE_IP)); + return (network_debug_ether_output(m, nd_networking.ifp, = nd_networking.remote_mac, ETHERTYPE_IP)); } =20 -/* - * Builds and sends a single ARP request to locate the server - * - * Parameters: - * void - * - * Return value: - * 0 on success - * errno on error - */ -static int -netdump_send_arp() -{ - struct ether_addr bcast; - struct mbuf *m; - struct arphdr *ah; - int pktlen; - - MPASS(nd_ifp !=3D NULL); - - /* Fill-up a broadcast address. */ - memset(&bcast, 0xFF, ETHER_ADDR_LEN); - MGETHDR(m, M_NOWAIT, MT_DATA); - if (m =3D=3D NULL) { - printf("netdump_send_arp: Out of mbufs"); - return (ENOBUFS); - } - pktlen =3D arphdr_len2(ETHER_ADDR_LEN, sizeof(struct in_addr)); - m->m_len =3D pktlen; - m->m_pkthdr.len =3D pktlen; - MH_ALIGN(m, pktlen); - ah =3D mtod(m, struct arphdr *); - ah->ar_hrd =3D htons(ARPHRD_ETHER); - ah->ar_pro =3D htons(ETHERTYPE_IP); - ah->ar_hln =3D ETHER_ADDR_LEN; - ah->ar_pln =3D sizeof(struct in_addr); - ah->ar_op =3D htons(ARPOP_REQUEST); - memcpy(ar_sha(ah), IF_LLADDR(nd_ifp), ETHER_ADDR_LEN); - memcpy(ar_spa(ah), &nd_client.s_addr, sizeof(nd_client.s_addr)); - bzero(ar_tha(ah), ETHER_ADDR_LEN); - memcpy(ar_tpa(ah), &nd_gw.s_addr, sizeof(nd_gw.s_addr)); - - return (netdump_ether_output(m, nd_ifp, bcast, ETHERTYPE_ARP)); -} =20 /* * Sends ARP requests to locate the server and waits for a response @@ -444,20 +319,20 @@ netdump_arp_server() { int err, polls, retries; =20 - for (retries =3D 0; retries < nd_retries && have_server_mac =3D=3D= 0; + for (retries =3D 0; (retries < nd_networking.retries) && = (nd_networking.have_remote_mac =3D=3D 0); retries++) { - err =3D netdump_send_arp(); + err =3D network_debug_send_arp(&nd_networking); if (err !=3D 0) return (err); - for (polls =3D 0; polls < nd_polls && have_server_mac =3D=3D= 0; + for (polls =3D 0; (polls < nd_networking.poll_time) && = (nd_networking.have_remote_mac =3D=3D 0); polls++) { - netdump_network_poll(); + network_debug_poll(&nd_networking); DELAY(500); } - if (have_server_mac =3D=3D 0) + if (nd_networking.have_remote_mac =3D=3D 0) printf("(ARP retry)"); } - if (have_server_mac !=3D 0) + if (nd_networking.have_remote_mac !=3D 0) return (0); =20 printf("\nARP timed out.\n"); @@ -492,7 +367,7 @@ netdump_send(uint32_t type, off_t offset, unsigned = char *data, uint32_t datalen) rcvd_acks =3D 0; retries =3D 0; =20 - MPASS(nd_ifp !=3D NULL); + MPASS(nd_networking.ifp !=3D NULL); =20 retransmit: =20 @@ -505,7 +380,7 @@ retransmit: pktlen =3D min(pktlen, NETDUMP_DATASIZE); =20 /* Second bound: the interface MTU (assume no IP = options). */ - pktlen =3D min(pktlen, nd_ifp->if_mtu - sizeof(struct = udpiphdr) - + pktlen =3D min(pktlen, nd_networking.ifp->if_mtu - = sizeof(struct udpiphdr) - sizeof(struct netdump_msg_hdr)); =20 /* @@ -543,7 +418,7 @@ retransmit: printf("netdump_send: Out of mbufs!\n"); return (ENOBUFS); } - MEXTADD(m2, data+sent_so_far, pktlen, = netdump_mbuf_nop, + MEXTADD(m2, data+sent_so_far, pktlen, = network_debug_mbuf_nop, NULL, NULL, M_RDONLY, EXT_MOD_TYPE); m2->m_len =3D pktlen; m->m_next =3D m2; @@ -568,13 +443,13 @@ retransmit: */ polls =3D 0; while (rcvd_acks !=3D want_acks) { =09 - if (polls++ > nd_polls) { - if (retries++ > nd_retries) + if (polls++ > nd_networking.poll_time) { + if (retries++ > nd_networking.retries) return (ETIMEDOUT); printf(". "); goto retransmit; } - netdump_network_poll(); + network_debug_poll(&nd_networking); DELAY(500); } nd_seqno +=3D i; @@ -582,6 +457,97 @@ retransmit: } =20 /* + * Determine if we want this IP packet. + */ +static int +nd_packet_wanted(struct network_debug *ndp, struct mbuf *m) +{ + int retval =3D 0; + struct udpiphdr *udp; + struct ip *ip; + unsigned short hlen; + + ip =3D mtod(m, struct ip*); + + /* IP version. */ + if (ip->ip_v !=3D IPVERSION) { + NETDDEBUG("%s: Bad IP version %d\n", __FUNCTION__, = ip->ip_v); + goto done; + } + +#ifdef INVARIANTS + if (((ntohl(ip->ip_dst.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET || + (ntohl(ip->ip_src.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET) && + (m->m_pkthdr.rcvif->if_flags & IFF_LOOPBACK) =3D=3D 0) { + NETDDEBUG("%s: Bad IP header (RFC1122)\n", = __FUNCTION__); + goto done; + } +#endif + + /* Header length. */ + hlen =3D ip->ip_hl << 2; + if (hlen < sizeof(struct ip)) { + NETDDEBUG("%s: Bad IP header length (%hu)\n", = __FUNCTION__, hlen); + goto done; + } + /* Checksum. */ + if ((m->m_pkthdr.csum_flags & CSUM_IP_CHECKED) !=3D 0) { + if ((m->m_pkthdr.csum_flags & CSUM_IP_VALID) =3D=3D 0) { + NETDDEBUG("%s: Bad IP checksum\n", = __FUNCTION__); + goto done; + } + } + + if (ntohs(ip->ip_len) < hlen) { + NETDDEBUG("%s: IP packet smaller (%hu) than header = (%hu)\n", + __FUNCTION__, ntohs(ip->ip_len), hlen); + goto done; + } + if (m->m_pkthdr.len < ntohs(ip->ip_len)) { + NETDDEBUG("%s: IP packet bigger (%hu) than ethernet = packet (%hu)\n", + __FUNCTION__, ntohs(ip->ip_len), = m->m_pkthdr.len); + goto done; + } + /* Ignore packets with IP options. */ + if (hlen > sizeof(struct ip)) { + NETDDEBUG("%s: Drop packet with IP options\n", = __FUNCTION__); + goto done; + } + if (ip->ip_p !=3D IPPROTO_UDP) { + NETDDEBUG("%s: Drop non-UDP packet\n", __FUNCTION__); + goto done; + } + + /* Do not deal with fragments. */ + if ((ntohs(ip->ip_off) & (IP_MF | IP_OFFMASK)) !=3D 0) { + NETDDEBUG("%s: Drop fragmented packet\n", __FUNCTION__); + goto done; + } + /* Check that the source is the server's IP. */ + if (ip->ip_src.s_addr !=3D ndp->remote.s_addr) { + NETDDEBUG("%s: Drop packet not from server\n", = __FUNCTION__); + goto done; + } + + /* Check if the destination IP is ours. */ + if (ip->ip_dst.s_addr !=3D ndp->local.s_addr) { + NETDDEBUGV("%s: Drop packet not to our IP\n", = __FUNCTION__); + goto done; + } + udp =3D mtod(m, struct udpiphdr *); + + if (ntohs(udp->ui_u.uh_dport) !=3D NETDUMP_ACKPORT) { + NETDDEBUG("%s: not on the netdump port.\n", = __FUNCTION__); + goto done; + } +=09 + retval =3D 1; + +done: + return retval; +} + +/* * Handler for IP packets: checks their sanity and then processes any = netdump * ACK packets it finds. * @@ -589,6 +555,7 @@ retransmit: * udp_input(). * * Parameters: + * ndp A pointer to a network_debug struct * mb a pointer to an mbuf * containing the packet received * Updates *mb if m_pullup et al change the pointer * Assumes the calling function will take care of freeing = the mbuf @@ -597,27 +564,26 @@ retransmit: * void */ static void -nd_handle_ip(struct mbuf **mb) -{ - struct ip *ip; +nd_handle_ip(struct network_debug *ndp, struct mbuf **mb) +{=09 struct udpiphdr *udp; - struct netdump_ack *nd_ack; + struct ip *ip; struct mbuf *m; + struct netdump_ack *nd_ack; int rcv_ackno; unsigned short hlen; =20 /* IP processing. */ m =3D *mb; if (m->m_pkthdr.len < sizeof(struct ip)) { - NETDDEBUG("nd_handle_ip: dropping packet too small for = IP " - "header\n"); + NETDDEBUG("%s: dropping packet too small for IP = header\n", __FUNCTION__); return; } if (m->m_len < sizeof(struct ip)) { m =3D m_pullup(m, sizeof(struct ip)); *mb =3D m; if (m =3D=3D NULL) { - NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + NETDDEBUG("%s: m_pullup failed\n", = __FUNCTION__); return; } } @@ -625,21 +591,21 @@ nd_handle_ip(struct mbuf **mb) =20 /* IP version. */ if (ip->ip_v !=3D IPVERSION) { - NETDDEBUG("nd_handle_ip: Bad IP version %d\n", = ip->ip_v); + NETDDEBUG("%s: Bad IP version %d\n", __FUNCTION__, = ip->ip_v); return; } =20 /* Header length. */ hlen =3D ip->ip_hl << 2; if (hlen < sizeof(struct ip)) { - NETDDEBUG("nd_handle_ip: Bad IP header length (%hu)\n", = hlen); + NETDDEBUG("%s: Bad IP header length (%hu)\n", = __FUNCTION__, hlen); return; } if (hlen > m->m_len) { m =3D m_pullup(m, hlen); *mb =3D m; if (m =3D=3D NULL) { - NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + NETDDEBUG("%s: m_pullup failed\n", = __FUNCTION__); return; } ip =3D mtod(m, struct ip *); @@ -649,7 +615,7 @@ nd_handle_ip(struct mbuf **mb) if (((ntohl(ip->ip_dst.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET || (ntohl(ip->ip_src.s_addr) >> IN_CLASSA_NSHIFT) =3D=3D = IN_LOOPBACKNET) && (m->m_pkthdr.rcvif->if_flags & IFF_LOOPBACK) =3D=3D 0) { - NETDDEBUG("nd_handle_ip: Bad IP header (RFC1122)\n"); + NETDDEBUG("%s: Bad IP header (RFC1122)\n", = __FUNCTION__); return; } #endif @@ -657,24 +623,24 @@ nd_handle_ip(struct mbuf **mb) /* Checksum. */ if ((m->m_pkthdr.csum_flags & CSUM_IP_CHECKED) !=3D 0) { if ((m->m_pkthdr.csum_flags & CSUM_IP_VALID) =3D=3D 0) { - NETDDEBUG("nd_handle_ip: Bad IP checksum\n"); + NETDDEBUG("%s: Bad IP checksum\n", = __FUNCTION__); return; } } else - NETDDEBUG("nd_handle_ip: HW didn't check IP cksum\n"); + NETDDEBUG("%s: HW didn't check IP cksum\n", = __FUNCTION__); =20 /* Convert fields to host byte order. */ ip->ip_len =3D ntohs(ip->ip_len); if (ip->ip_len < hlen) { - NETDDEBUG("nd_handle_ip: IP packet smaller (%hu) than header = (%hu)\n", - ip->ip_len, hlen); + NETDDEBUG("%s: IP packet smaller (%hu) than header = (%hu)\n", + __FUNCTION__, ip->ip_len, hlen); return; } ip->ip_off =3D ntohs(ip->ip_off); - +=09 if (m->m_pkthdr.len < ip->ip_len) { -NETDDEBUG("nd_handle_ip: IP packet bigger (%hu) than ethernet packet = (%hu)\n", - ip->ip_len, m->m_pkthdr.len); + NETDDEBUG("%s: IP packet bigger (%hu) than ethernet = packet (%hu)\n", + __FUNCTION__, ip->ip_len, m->m_pkthdr.len); return; } if (m->m_pkthdr.len > ip->ip_len) { @@ -689,42 +655,48 @@ NETDDEBUG("nd_handle_ip: IP packet bigger (%hu) = than ethernet packet (%hu)\n", =20 /* Ignore packets with IP options. */ if (hlen > sizeof(struct ip)) { - NETDDEBUG("nd_handle_ip: Drop packet with IP = options\n"); - return; - } - - /* Check that the source is the server's IP. */ - if (ip->ip_src.s_addr !=3D nd_server.s_addr) { - NETDDEBUG("nd_handle_ip: Drop packet not from = server\n"); - return; - } - - /* Check if the destination IP is ours. */ - if (ip->ip_dst.s_addr !=3D nd_client.s_addr) { - NETDDEBUGV("nd_handle_ip: Drop packet not to our IP\n"); + NETDDEBUG("%s: Drop packet with IP options\n", = __FUNCTION__); return; } =20 if (ip->ip_p !=3D IPPROTO_UDP) { - NETDDEBUG("nd_handle_ip: Drop non-UDP packet\n"); + NETDDEBUG("%s: Drop non-UDP packet\n", __FUNCTION__); return; } =20 /* Do not deal with fragments. */ if ((ip->ip_off & (IP_MF | IP_OFFMASK)) !=3D 0) { - NETDDEBUG("nd_handle_ip: Drop fragmented packet\n"); + NETDDEBUG("%s: Drop fragmented packet\n", __FUNCTION__); return; } =20 /* UDP custom is to have packet length not include IP header. */ ip->ip_len -=3D hlen; =20 + /* UDP processing. */ =20 + /* + * Note that the rest of the code below is hard-wired for + * network dump, and cannot be used for other purposes. + */ + + /* Check that the source is the server's IP. */ + if (ip->ip_src.s_addr !=3D ndp->remote.s_addr) { + NETDDEBUG("%s: Drop packet not from server\n", = __FUNCTION__); + return; + } + + /* Check if the destination IP is ours. */ + if (ip->ip_dst.s_addr !=3D ndp->local.s_addr) { + NETDDEBUGV("%s: Drop packet not to our IP\n", = __FUNCTION__); + return; + } + /* Get IP and UDP headers together, along with the netdump = packet. */ if (m->m_pkthdr.len < sizeof(struct udpiphdr) + sizeof(struct netdump_ack)) { - NETDDEBUG("nd_handle_ip: Ignoring small packet\n"); + NETDDEBUG("%s: Ignoring small packet\n", __FUNCTION__); return; } if (m->m_len < sizeof(struct udpiphdr) + sizeof(struct = netdump_ack)) { @@ -732,19 +704,17 @@ NETDDEBUG("nd_handle_ip: IP packet bigger (%hu) = than ethernet packet (%hu)\n", sizeof(struct netdump_ack)); *mb =3D m; if (m =3D=3D NULL) { - NETDDEBUG("nd_handle_ip: m_pullup failed\n"); + NETDDEBUG("%s: m_pullup failed\n", = __FUNCTION__); return; } } udp =3D mtod(m, struct udpiphdr *); =20 if (ntohs(udp->ui_u.uh_dport) !=3D NETDUMP_ACKPORT) { - NETDDEBUG("not on the netdump port.\n"); + NETDDEBUG("%s: not on the netdump port.\n", = __FUNCTION__); return; } =20 - /* Netdump processing. */ - /* * Packet is meant for us. Extract the ack sequence number and = the * port number if necessary. @@ -752,233 +722,30 @@ NETDDEBUG("nd_handle_ip: IP packet bigger (%hu) = than ethernet packet (%hu)\n", nd_ack =3D (struct netdump_ack *)(mtod(m, caddr_t) + sizeof(struct udpiphdr)); rcv_ackno =3D ntohl(nd_ack->na_seqno); - if (nd_server_port =3D=3D NETDUMP_PORT) - nd_server_port =3D ntohs(udp->ui_u.uh_sport); + if (nd_remote_port =3D=3D NETDUMP_PORT) + nd_remote_port =3D ntohs(udp->ui_u.uh_sport); if (rcv_ackno >=3D nd_seqno + 64) - printf("nd_handle_ip: ACK %d too far in future!\n", = rcv_ackno); + printf("%s: ACK %d too far in future!\n", __FUNCTION__, = rcv_ackno); else if (rcv_ackno >=3D nd_seqno) { - /* We're interested in this ack. Record it. */ rcvd_acks |=3D 1 << (rcv_ackno-nd_seqno); } -} - -/* - * Handler for ARP packets: checks their sanity and then - * 1. If the ARP is a request for our IP, respond with our MAC address - * 2. If the ARP is a response from our server, record its MAC address - * - * It needs to replicate partially the behaviour of arpintr() and - * in_arpinput(). - * - * Parameters: - * mb a pointer to an mbuf * containing the packet received - * Updates *mb if m_pullup et al change the pointer - * Assumes the calling function will take care of freeing = the mbuf - * - * Return value: - * void - */ -static void -nd_handle_arp(struct mbuf **mb) -{ - char buf[INET_ADDRSTRLEN]; - struct in_addr isaddr, itaddr, myaddr; - struct ether_addr dst; - struct mbuf *m; - struct arphdr *ah; - struct ifnet *ifp; - uint8_t *enaddr; - int req_len, op; - - m =3D *mb; - ifp =3D m->m_pkthdr.rcvif; - if (m->m_len < sizeof(struct arphdr)) { - m =3D m_pullup(m, sizeof(struct arphdr)); - *mb =3D m; - if (m =3D=3D NULL) { - NETDDEBUG("nd_handle_arp: runt packet: m_pullup = failed\n"); - return; - } - } - ah =3D mtod(m, struct arphdr *); - - if (ntohs(ah->ar_hrd) !=3D ARPHRD_ETHER) { - NETDDEBUG("nd_handle_arp: unknown hardware address = 0x%2D)\n", - (unsigned char *)&ah->ar_hrd, ""); - return; - } - if (ntohs(ah->ar_pro) !=3D ETHERTYPE_IP) { - NETDDEBUG("nd_handle_arp: Drop ARP for unknown protocol = %d\n", - ntohs(ah->ar_pro)); - return; - } - req_len =3D arphdr_len2(ifp->if_addrlen, sizeof(struct = in_addr)); - if (m->m_len < req_len) { - m =3D m_pullup(m, req_len); - *mb =3D m; - if (m =3D=3D NULL) { - NETDDEBUG("nd_handle_arp: runt packet: m_pullup = failed\n"); - return; - } - } - ah =3D mtod(m, struct arphdr *); - - op =3D ntohs(ah->ar_op); - memcpy(&isaddr, ar_spa(ah), sizeof(isaddr)); - memcpy(&itaddr, ar_tpa(ah), sizeof(itaddr)); - enaddr =3D (uint8_t *)IF_LLADDR(ifp); - myaddr =3D nd_client; - - if (!bcmp(ar_sha(ah), enaddr, ifp->if_addrlen)) { - NETDDEBUG("nd_handle_arp: ignoring ARP from myself\n"); - return; - } - - if (isaddr.s_addr =3D=3D nd_client.s_addr) { - printf("nd_handle_arp: %*D is using my IP address = %s!\n", - ifp->if_addrlen, (u_char *)ar_sha(ah), ":", - inet_ntoa(isaddr)); - return; - } =20 - if (!bcmp(ar_sha(ah), ifp->if_broadcastaddr, ifp->if_addrlen)) { - NETDDEBUG("nd_handle_arp: ignoring ARP from broadcast = address\n"); - return; - } - - if (op =3D=3D ARPOP_REPLY) { - if (isaddr.s_addr !=3D nd_gw.s_addr) { - inet_ntoa_r(isaddr, buf); -NETDDEBUG("nd_handle_arp: ignoring ARP reply from %s (not netdump = server)\n", - buf); - return; - } - memcpy(nd_gw_mac.octet, ar_sha(ah), - min(ah->ar_hln, ETHER_ADDR_LEN)); - have_server_mac =3D 1; - NETDDEBUG("\nnd_handle_arp: Got server MAC address = %6D\n", - nd_gw_mac.octet, ":"); - return; - } - - if (op !=3D ARPOP_REQUEST) { - NETDDEBUG("nd_handle_arp: Ignoring ARP = non-request/reply\n"); - return; - } - - if (itaddr.s_addr !=3D nd_client.s_addr) { - NETDDEBUG("nd_handle_arp: ignoring ARP not to our = IP\n"); - return; - } - - memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); - memcpy(ar_sha(ah), enaddr, ah->ar_hln); - memcpy(ar_tpa(ah), ar_spa(ah), ah->ar_pln); - memcpy(ar_spa(ah), &itaddr, ah->ar_pln); - ah->ar_op =3D htons(ARPOP_REPLY); - ah->ar_pro =3D htons(ETHERTYPE_IP); - m->m_flags &=3D ~(M_BCAST|M_MCAST); - m->m_len =3D sizeof(*ah) + (2 * ah->ar_pln) + (2 * ah->ar_hln); - m->m_pkthdr.len =3D m->m_len; - - memcpy(dst.octet, ar_tha(ah), ETHER_ADDR_LEN); - netdump_ether_output(m, ifp, dst, ETHERTYPE_ARP); - *mb =3D NULL; } =20 /* - * Handler for incoming packets directly from the network adapter - * Identifies the packet type (IP or ARP) and passes it along to one of = the - * helper functions nd_handle_ip or nd_handle_arp. - * - * It needs to replicate partially the behaviour of ether_input() and - * ether_demux(). - * - * Parameters: - * ifp the interface the packet came from (should be nd_ifp) - * m an mbuf containing the packet received - * - * Return value: - * void + * netdump_pkt_in + * Simple wrapper for network_debug_pkt_in; this is called + * as ifp->if_input() by the ethernet driver. We simply let + * network_debug_pkt_in() procss it, and decide which helper + * function to call. */ static void netdump_pkt_in(struct ifnet *ifp, struct mbuf *m) { - struct ether_header *eh; - u_short etype; - - /* Ethernet processing. */ - if ((m->m_flags & M_PKTHDR) =3D=3D 0) { - NETDDEBUG_IF(ifp, - "netdump_pkt_in: Discard frame without packet = header\n"); - goto done; - } - if (m->m_len < ETHER_HDR_LEN) { - NETDDEBUG_IF(ifp, -"netdump_pkt_in: Discard frame without leading eth header (len %u = pktlen %u)\n", - m->m_len, m->m_pkthdr.len); - goto done; - } - if ((m->m_flags & M_HASFCS) !=3D 0) { - m_adj(m, -ETHER_CRC_LEN); - m->m_flags &=3D ~M_HASFCS; - } - eh =3D mtod(m, struct ether_header *); - m->m_pkthdr.PH_loc.ptr =3D eh; - etype =3D ntohs(eh->ether_type); - if ((m->m_flags & M_VLANTAG) !=3D 0 || etype =3D=3D = ETHERTYPE_VLAN) { - NETDDEBUG_IF(ifp, "netdump_pkt_in: Ignoring vlan = packets\n"); - goto done; - } - - /* XXX: Probably must also check if we're the recipient MAC = address. */ - - /* Done ethernet processing. Strip off the ethernet header. */ - m_adj(m, ETHER_HDR_LEN); - switch (etype) { - case ETHERTYPE_ARP: - nd_handle_arp(&m); - break; - case ETHERTYPE_IP: - nd_handle_ip(&m); - break; - default: - NETDDEBUG_IF(ifp, - "netdump_pkt_in: Dropping unknown ethertype = %hu\n", - etype); - break; - } -done: - if (m !=3D NULL) - m_freem(m); + network_debug_pkt_in(&nd_networking, m); } =20 -/* - * After trapping, instead of assuming that most of the network stack = is sane - * just poll the driver directly for packets. - * - * Parameters: - * void - * - * Returns: - * void - */ -static void -netdump_network_poll() -{ - - MPASS(nd_ifp !=3D NULL); - -#if defined(KDB) && !defined(KDB_UNATTENDED) - if (panicstr !=3D NULL) - nd_ifp->if_ndumpfuncs->ne_poll_unlocked(nd_ifp, - POLL_AND_CHECK_STATUS, 1000); - else -#endif - nd_ifp->if_ndumpfuncs->ne_poll_locked(nd_ifp, - POLL_AND_CHECK_STATUS, 1000); -} =20 /*- * Dumping specific primitives. @@ -1033,6 +800,25 @@ netdump_dumper(void *priv __unused, void *virtual, } =20 /* + * Preparation routine. This sets some things up, and also ensures + * there we have an interface and address. + */ +static int +nd_prepare(struct network_debug *ndp) +{ + int error =3D 0; + + if (ndp->remote.s_addr =3D=3D INADDR_ANY) { + printf("%s: Can't netdump; no server IP given\n", = __FUNCTION__); + return (EINVAL); + } + + error =3D network_debug_prepare(ndp); + return error; +} + + +/* * Dumper routine, specular to dumpsys(). * * Parameters: @@ -1045,13 +831,11 @@ int netdumpsys() { struct dumperinfo dumper; - void (*old_if_input)(struct ifnet *, struct mbuf *); - int error, found, must_lock, nd_gw_unset; + int error, found, must_lock; + void (*old_if_input)(struct ifnet *, struct mbuf *) =3D NULL; =20 - old_if_input =3D NULL; error =3D 0; found =3D 0; - nd_gw_unset =3D 0; must_lock =3D 1; #if defined(KDB) && !defined(KDB_UNATTENDED) if (panicstr !=3D NULL) @@ -1062,51 +846,21 @@ netdumpsys() if (nd_enable =3D=3D 0) return (EINVAL); =20 - /* Lookup the right if device to be used in the dump. */ - if (must_lock !=3D 0) - IFNET_RLOCK_NOSLEEP(); - TAILQ_FOREACH(nd_ifp, &V_ifnet, if_link) { - if (!strncmp(nd_ifp->if_xname, nd_ifp_str, - strlen(nd_ifp->if_xname)) && - netdump_supported_nic(nd_ifp)) { - found =3D 1; - break; - } - } - if (must_lock !=3D 0) - IFNET_RUNLOCK_NOSLEEP(); - if (found =3D=3D 0) { - printf("netdumpsys: Can't netdump: no valid NIC = given\n"); - return (EINVAL); - } - - MPASS(nd_ifp !=3D NULL); - - if (nd_server.s_addr =3D=3D INADDR_ANY) { - printf("netdumpsys: Can't netdump; no server IP = given\n"); - return (EINVAL); - } - if (nd_client.s_addr =3D=3D INADDR_ANY) { - printf("netdumpsys: Can't netdump; no client IP = given\n"); - return (EINVAL); + error =3D nd_prepare(&nd_networking); + if (error) { + goto trig_abort; } =20 /* * nd_server_port could have switched after the first ack the * first time it gets called. Adjust it accordingly. */ - nd_server_port =3D NETDUMP_PORT; - if ((nd_ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && must_lock = !=3D 0) - nd_ifp->if_ndumpfuncs->ne_disable_intr(nd_ifp); + nd_remote_port =3D NETDUMP_PORT; =20 /* Make the card use *our* receive callback. */ - old_if_input =3D nd_ifp->if_input; - nd_ifp->if_input =3D netdump_pkt_in; + old_if_input =3D nd_networking.ifp->if_input; + nd_networking.ifp->if_input =3D netdump_pkt_in; =20 - if (nd_gw.s_addr =3D=3D INADDR_ANY) { - nd_gw.s_addr =3D nd_server.s_addr; - nd_gw_unset =3D 1; - } printf("\n-----------------------------------\n"); printf("netdump in progress. searching for server.. "); if (netdump_arp_server()) { @@ -1119,7 +873,7 @@ netdumpsys() error =3D EINVAL; goto trig_abort; } - printf("dumping to %s (%6D)\n", inet_ntoa(nd_server), = nd_gw_mac.octet, + printf("dumping to %s (%6D)\n", inet_ntoa(nd_networking.remote), = nd_networking.remote_mac.octet, ":"); printf("-----------------------------------\n"); =20 @@ -1142,12 +896,9 @@ netdumpsys() } printf("\nnetdump finished.\n"); trig_abort: - if (nd_gw_unset !=3D 0) - nd_gw.s_addr =3D INADDR_ANY; + network_debug_finish(&nd_networking); if (old_if_input) - nd_ifp->if_input =3D old_if_input; - if ((nd_ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && must_lock = !=3D 0) - nd_ifp->if_ndumpfuncs->ne_enable_intr(nd_ifp); + nd_networking.ifp->if_input =3D old_if_input; return (error); } =20 @@ -1160,6 +911,10 @@ trig_abort: * (locates the first available NIC and uses the first IPv4 IP on that = card as * the client IP). Leaves the server IP unconfigured. * + * The parenthical comment above is completely wrong. Not only does it = not do anything + * w.r.t. IP address, but it is called during kernel initialization -- = so no NICs + * are configured at the time. We instead do this in nd_prepare(). + * * Parameters: * void *, unused * @@ -1169,22 +924,21 @@ trig_abort: static void netdump_config_defaults(void *dummy __unused) { - struct ifnet *ifp; - int found; - - nd_ifp =3D NULL; - nd_server.s_addr =3D INADDR_ANY; - nd_client.s_addr =3D INADDR_ANY; - nd_gw.s_addr =3D INADDR_ANY; - - if (nd_server_tun[0] !=3D '\0') - inet_aton(nd_server_tun, &nd_server); - if (nd_client_tun[0] !=3D '\0') - inet_aton(nd_client_tun, &nd_client); + nd_networking.ifp =3D NULL; + nd_networking.remote.s_addr =3D INADDR_ANY; + nd_networking.local.s_addr =3D INADDR_ANY; + nd_networking.gw.s_addr =3D INADDR_ANY; + + if (nd_remote_tun[0] !=3D '\0') + inet_aton(nd_remote_tun, &nd_networking.remote); + if (nd_local_tun[0] !=3D '\0') + inet_aton(nd_local_tun, &nd_networking.local); if (nd_gw_tun[0] !=3D '\0') - inet_aton(nd_gw_tun, &nd_gw); + inet_aton(nd_gw_tun, &nd_networking.gw); if (nd_nic_tun[0] !=3D '\0') { - found =3D 0; +#if 0 + int found =3D 0; + struct ifnet *ifp; IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { if (!strncmp(ifp->if_xname, nd_nic_tun, @@ -1194,8 +948,13 @@ netdump_config_defaults(void *dummy __unused) } } IFNET_RUNLOCK_NOSLEEP(); - if (found !=3D 0 && netdump_supported_nic(ifp)) - nd_ifp =3D ifp; + if (found !=3D 0 && network_debug_supported_nic(ifp)) + nd_networking.ifp =3D ifp; +#else + strlcpy(nd_networking.if_name, nd_nic_tun, = sizeof(nd_networking.if_name)); +#endif } } + SYSINIT(netdump, SI_SUB_KLD, SI_ORDER_ANY, netdump_config_defaults, = NULL); + diff --git a/sys/netinet/network_debug.c b/sys/netinet/network_debug.c new file mode 100644 index 0000000..933aec5 --- /dev/null +++ b/sys/netinet/network_debug.c @@ -0,0 +1,574 @@ +/*- + * + * Copyright (c) 2014 iXSystems, Inc. All rights reserved. + * + * Copyright (c) 2005-2011 Sandvine Incorporated. All rights reserved. + * Copyright (c) 2000 Darrell Anderson + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in = the + * documentation and/or other materials provided with the = distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF + * SUCH DAMAGE. + */ + +/* + * network_debug.c + * A FreeBSD subsystem supporting network I/O in a minimal kernel = environment -- + * specifically, to do a network coredump, or debugging over ethernet. + */ + +#include "opt_ddb.h" +#include "opt_kdb.h" +#include "opt_netdump.h" + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#ifdef DDB +#include +#endif + +//#define NETWORK_DEBUG_DEBUG 10 + +#ifdef NETWORK_DEBUG_DEBUG +# define NETDDEBUG(f, ...) printf((f), ## = __VA_ARGS__) +# define NETDDEBUG_IF(i, f, ...) if_printf((i), (f), ## = __VA_ARGS__) +# if NETWORK_DEBUG_DEBUG > 1 +# define NETDDEBUGV(f, ...) printf((f), ## = __VA_ARGS__) +# define NETDDEBUGV_IF(i, f, ...) if_printf((i), (f), ## = __VA_ARGS__) +# else +# define NETDDEBUGV(f, ...) +# define NETDDEBUGV_IF(i, f, ...) +# endif +#else +# define NETDDEBUG(f, ...) +# define NETDDEBUG_IF(i, f, ...) +# define NETDDEBUGV(f, ...) +# define NETDDEBUGV_IF(i, f, ...) +#endif + +/* + * Handles creation of the ethernet header, then places outgoing = packets into + * the tx buffer for the NIC + * + * Parameters: + * m The mbuf containing the packet to be sent (will be freed = by + * this function or the NIC driver) + * ifp The interface to send on + * dst The destination ethernet address (source address will be = looked + * up using ifp) + * etype The ETHERTYPE_* value for the protocol that is being = sent + * + * Returns: + * int see errno.h, 0 for success + */ +int +network_debug_ether_output(struct mbuf *m, struct ifnet *ifp, struct = ether_addr dst, + u_short etype) +{ + struct ether_header *eh; + + /* Fill in the ethernet header. */ + M_PREPEND(m, ETHER_HDR_LEN, M_NOWAIT); + if (m =3D=3D NULL) { + NETDDEBUG("%s: Out of mbufs\n", __FUNCTION__); + return (ENOBUFS); + } + eh =3D mtod(m, struct ether_header *); + memcpy(eh->ether_shost, IF_LLADDR(ifp), ETHER_ADDR_LEN); + memcpy(eh->ether_dhost, dst.octet, ETHER_ADDR_LEN); + eh->ether_type =3D htons(etype); + + if (((ifp->if_flags & (IFF_MONITOR | IFF_UP)) !=3D IFF_UP) || + (ifp->if_drv_flags & IFF_DRV_RUNNING) !=3D IFF_DRV_RUNNING) = { + if_printf(ifp, "%s: Interface isn't up\n", = __FUNCTION__); + m_freem(m); + return (ENETDOWN); + } + return ((ifp->if_transmit)(ifp, m)); +} + +/* + * Netdump wraps external mbufs around address ranges. unlike most = sane + * counterparts, netdump uses a stop-and-wait approach to flow control = and + * retransmission, so the ack obviates the need for mbuf reference + * counting. We still need to tell other mbuf handlers not to do = anything + * special with our mbufs, so specify this nop handler. + * + * Parameters: + * ptr data to free (ignored) + * opt_args callback pointer (ignored) + * + * Returns: + * void + */ +int +network_debug_mbuf_nop(struct mbuf *mp __unused, void *ptr __unused, = void *opt_args __unused) +{ + return EXT_FREE_OK; +} +/* + * Handler for ARP packets: checks their sanity and then + * 1. If the ARP is a request for our IP, respond with our MAC address + * 2. If the ARP is a response from our server, record its MAC address + * + * It needs to replicate partially the behaviour of arpintr() and + * in_arpinput(). + * + * Parameters: + * mb a pointer to an mbuf * containing the packet received + * Updates *mb if m_pullup et al change the pointer + * Assumes the calling function will take care of freeing = the mbuf + * + * Return value: + * void + */ +void +network_debug_handle_arp(struct network_debug *ndp, struct mbuf **mb) +{ + char buf[INET_ADDRSTRLEN]; + struct in_addr isaddr, itaddr, myaddr; + struct ether_addr dst; + struct mbuf *m; + struct arphdr *ah; + struct ifnet *ifp; + uint8_t *enaddr; + int req_len, op; + + NETDDEBUG("%s(%d)\n", __FUNCTION__, __LINE__); + + m =3D *mb; + ifp =3D m->m_pkthdr.rcvif; + if (m->m_len < sizeof(struct arphdr)) { + m =3D m_pullup(m, sizeof(struct arphdr)); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("%s: runt packet: m_pullup failed\n", = __FUNCTION__); + return; + } + } + ah =3D mtod(m, struct arphdr *); + + if (ntohs(ah->ar_hrd) !=3D ARPHRD_ETHER) { + NETDDEBUG("%s: unknown hardware address 0x%2D)\n", + __FUNCTION__, + (unsigned char *)&ah->ar_hrd, ""); + return; + } + if (ntohs(ah->ar_pro) !=3D ETHERTYPE_IP) { + NETDDEBUG("%s: Drop ARP for unknown protocol %d\n", + __FUNCTION__, + ntohs(ah->ar_pro)); + return; + } + req_len =3D arphdr_len2(ifp->if_addrlen, sizeof(struct = in_addr)); + if (m->m_len < req_len) { + m =3D m_pullup(m, req_len); + *mb =3D m; + if (m =3D=3D NULL) { + NETDDEBUG("%s: runt packet: m_pullup failed\n", = __FUNCTION__); + return; + } + } + ah =3D mtod(m, struct arphdr *); + + op =3D ntohs(ah->ar_op); + memcpy(&isaddr, ar_spa(ah), sizeof(isaddr)); + memcpy(&itaddr, ar_tpa(ah), sizeof(itaddr)); + enaddr =3D (uint8_t *)IF_LLADDR(ifp); + myaddr =3D ndp->local; + + if (!bcmp(ar_sha(ah), enaddr, ifp->if_addrlen)) { + NETDDEBUG("%s: ignoring ARP from myself\n", = __FUNCTION__); + return; + } + + if (ndp->local.s_addr !=3D INADDR_ANY && + isaddr.s_addr =3D=3D ndp->local.s_addr) { + NETDDEBUG("%s: %*D is using my IP address %s!\n", + __FUNCTION__, + ifp->if_addrlen, (u_char *)ar_sha(ah), ":", + inet_ntoa(isaddr)); + return; + } + + if (!bcmp(ar_sha(ah), ifp->if_broadcastaddr, ifp->if_addrlen)) { + NETDDEBUG("%s: ignoring ARP from broadcast address\n", = __FUNCTION__); + return; + } + + if (op =3D=3D ARPOP_REPLY) { + if (ndp->gw.s_addr =3D=3D INADDR_ANY && = ndp->remote.s_addr =3D=3D INADDR_ANY) { + NETDDEBUG("%s: Ignoring ARP because we do not = have an address yet", __FUNCTION__); + return; + } + if (isaddr.s_addr !=3D (ndp->gw.s_addr =3D=3D INADDR_ANY = ? ndp->remote.s_addr : ndp->gw.s_addr)) { + inet_ntoa_r(isaddr, buf); + NETDDEBUG("%sp: ignoring ARP reply from %s (not = desired remote)\n", + __FUNCTION__, buf); + return; + } + memcpy(ndp->remote_mac.octet, ar_sha(ah), + min(ah->ar_hln, ETHER_ADDR_LEN)); + ndp->have_remote_mac =3D 1; + NETDDEBUG("\n%s: Got remote MAC address %6D\n", = __FUNCTION__, + ndp->remote_mac.octet, ":"); + return; + } + + if (op !=3D ARPOP_REQUEST) { + NETDDEBUG("%s: Ignoring ARP non-request/reply\n", = __FUNCTION__); + return; + } + + if (itaddr.s_addr !=3D ndp->local.s_addr) { + NETDDEBUG("%s: ignoring ARP not to our IP\n", = __FUNCTION__); + return; + } + + memcpy(ar_tha(ah), ar_sha(ah), ah->ar_hln); + memcpy(ar_sha(ah), enaddr, ah->ar_hln); + memcpy(ar_tpa(ah), ar_spa(ah), ah->ar_pln); + memcpy(ar_spa(ah), &itaddr, ah->ar_pln); + ah->ar_op =3D htons(ARPOP_REPLY); + ah->ar_pro =3D htons(ETHERTYPE_IP); + m->m_flags &=3D ~(M_BCAST|M_MCAST); + m->m_len =3D sizeof(*ah) + (2 * ah->ar_pln) + (2 * ah->ar_hln); + m->m_pkthdr.len =3D m->m_len; + + memcpy(dst.octet, ar_tha(ah), ETHER_ADDR_LEN); + network_debug_ether_output(m, ifp, dst, ETHERTYPE_ARP); + *mb =3D NULL; +} +/* + * Builds and sends a single ARP request to locate the server + * + * Parameters: + * void + * + * Return value: + * 0 on success + * errno on error + */ +int +network_debug_send_arp(struct network_debug *ndp) +{ + struct ether_addr bcast; + struct mbuf *m; + struct arphdr *ah; + int pktlen; + + MPASS(ndp->ifp !=3D NULL); + + /* Fill-up a broadcast address. */ + memset(&bcast, 0xFF, ETHER_ADDR_LEN); + MGETHDR(m, M_NOWAIT, MT_DATA); + if (m =3D=3D NULL) { + NETDDEBUG("%s: Out of mbufs", __FUNCTION__); + return (ENOBUFS); + } + pktlen =3D arphdr_len2(ETHER_ADDR_LEN, sizeof(struct in_addr)); + m->m_len =3D pktlen; + m->m_pkthdr.len =3D pktlen; + MH_ALIGN(m, pktlen); + ah =3D mtod(m, struct arphdr *); + ah->ar_hrd =3D htons(ARPHRD_ETHER); + ah->ar_pro =3D htons(ETHERTYPE_IP); + ah->ar_hln =3D ETHER_ADDR_LEN; + ah->ar_pln =3D sizeof(struct in_addr); + ah->ar_op =3D htons(ARPOP_REQUEST); + memcpy(ar_sha(ah), IF_LLADDR(ndp->ifp), ETHER_ADDR_LEN); + memcpy(ar_spa(ah), &ndp->local.s_addr, = sizeof(ndp->local.s_addr)); + bzero(ar_tha(ah), ETHER_ADDR_LEN); + memcpy(ar_tpa(ah), (ndp->gw.s_addr =3D=3D INADDR_ANY) ? = &ndp->remote.s_addr : &ndp->gw.s_addr, sizeof(ndp->gw.s_addr)); + + return (network_debug_ether_output(m, ndp->ifp, bcast, = ETHERTYPE_ARP)); +} +/* + * After trapping, instead of assuming that most of the network stack = is sane + * just poll the driver directly for packets. + * + * Parameters: + * void + * + * Returns: + * void + */ +void +network_debug_poll(struct network_debug *ndp) +{ + + MPASS(ndp->ifp !=3D NULL); + +#if defined(KDB) && !defined(KDB_UNATTENDED) + if (panicstr !=3D NULL) + ndp->ifp->if_ndumpfuncs->ne_poll_unlocked(ndp->ifp, + POLL_AND_CHECK_STATUS, 1000); + else +#endif + ndp->ifp->if_ndumpfuncs->ne_poll_locked(ndp->ifp, + POLL_AND_CHECK_STATUS, 1000); +} + +/* + * Handler for incoming packets directly from the network adapter + * Identifies the packet type (IP or ARP) and passes it along to one of = the + * helper functions. + * + * Note: Each subsystem that wants to use this will need to set the + * interface if_input pointer to something that calls this. E.g. + * static void packet_in(struct ifnet *ifp, struct mbuf *m) { + * network_debug_pkt_in(&my_netdeb, m); + * } + * and + * ndp->ifp->if_input =3D packet_in; + * + * It needs to replicate partially the behaviour of ether_input() and + * ether_demux(). + * + * Parameters: + * ndp The network_debug structure for our purposes. + * m an mbuf containing the packet received + * + * Return value: + * void + */ + +void +network_debug_pkt_in(struct network_debug *ndp, struct mbuf *m) +{ + static const u_char no_mac[ETHER_ADDR_LEN] =3D { 0 }; + static const u_char broadcast_mac[ETHER_ADDR_LEN] =3D { 0xff, = 0xff, 0xff, 0xff, 0xff, 0xff }; + + struct ether_header *eh, temp_eh; + u_short etype; + int send_through =3D 1; + + NETDDEBUG("%s(%d): Got an mbuf\n", __FUNCTION__, __LINE__); + + /* Ethernet processing. */ + if ((m->m_flags & M_PKTHDR) =3D=3D 0) { + NETDDEBUG_IF(ndp->ifp, "%s: Discard frame without packet = header\n", __FUNCTION__); + goto done; + } + if (m->m_len < ETHER_HDR_LEN) { + NETDDEBUG_IF(ndp->ifp, + "%s: Discard frame without leading eth = header (len %u pktlen %u)\n", + __FUNCTION__, m->m_len, m->m_pkthdr.len); + goto done; + } + if ((m->m_flags & M_HASFCS) !=3D 0) { + m_adj(m, -ETHER_CRC_LEN); + m->m_flags &=3D ~M_HASFCS; + } + eh =3D mtod(m, struct ether_header *); + temp_eh =3D *eh; // Not a big structure, so should be = safe to have on stack and copy + m->m_pkthdr.PH_loc.ptr =3D eh; + etype =3D ntohs(eh->ether_type); + if ((m->m_flags & M_VLANTAG) !=3D 0 || etype =3D=3D = ETHERTYPE_VLAN) { + NETDDEBUG_IF(ndp->ifp, "netdump_pkt_in: Ignoring vlan = packets\n"); + goto done; + } + + /* + * XXX: Probably must also check if we're the recipient MAC = address. + * But we need to also check for broadcast, and whether or not = we have + * a MAC address set. + */ + + if (bcmp(&ndp->local_mac, no_mac, sizeof(no_mac)) !=3D 0 && + bcmp(&eh->ether_dhost, broadcast_mac, sizeof(broadcast_mac)) = !=3D 0 && + bcmp(&ndp->local_mac, &eh->ether_dhost, = sizeof(ndp->local_mac)) !=3D 0) { + NETDDEBUG("%s: Ethernet packet for destination %6D is = not for me (%6D)\n", + __FUNCTION__, + eh->ether_dhost, ":", + ndp->local_mac.octet, ":"); + goto done; + } + + /* Done ethernet processing. Strip off the ethernet header. */ + m_adj(m, ETHER_HDR_LEN); + switch (etype) { + case ETHERTYPE_ARP: + NETDDEBUG("%s(%d): ETHERTYPE_ARP\n", __FUNCTION__, = __LINE__); + network_debug_handle_arp(ndp, &m); + break; + case ETHERTYPE_IP: + NETDDEBUG("%s(%d: ETHERTYPE_IP\n", __FUNCTION__, = __LINE__); + if (ndp->wanted) { + send_through =3D (ndp->wanted)(ndp, m); + if (send_through) { + /* + * If this packet is wanted, then we = stash + * the source information into ndp + */ + NETDDEBUG("%s(%d): Setting = ndp->remote_mac to %6D\n", __FUNCTION__, __LINE__, eh->ether_shost, = ":"); + bcopy(&eh->ether_shost, = &ndp->remote_mac, sizeof(ndp->remote_mac)); + ndp->remote =3D (mtod(m, struct = udpiphdr*))->ui_src; + NETDDEBUG("%s(%d): Set ndp->remote to = %#x\n", __FUNCTION__, __LINE__, ndp->remote.s_addr); + } + } + if (send_through && ndp->ip_handler) + (*ndp->ip_handler)(ndp, &m); + NETDDEBUG("%s(%d): Done with ip_handler\n", = __FUNCTION__, __LINE__); + break; + default: + NETDDEBUG_IF(ndp->ifp, + "%s: Dropping unknown ethertype %hu\n", + __FUNCTION__, + etype); + break; + } +done: + if (m !=3D NULL) + m_freem(m); + NETDDEBUG("%s(%d): Done here\n", __FUNCTION__, __LINE__); +} + +int +network_debug_prepare(struct network_debug *ndp) +{ + int must_lock =3D 1; + int error =3D 0; + +#if defined(KDB) && !defined(KDB_UNATTENDED) + if (panicstr !=3D NULL) + must_lock =3D 0; +#endif + /* Lookup the right if device to be used */ + if (must_lock !=3D 0) + IFNET_RLOCK_NOSLEEP(); + + if (ndp->ifp =3D=3D NULL) { + struct ifnet *ifp =3D NULL; + /* + * Look for an interface. If one was specified via = sysctl + * or tunable, look for that name; otherwise, pick the = first + * one that claims to support netdump. + */ + TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ndp->if_name[0] =3D=3D 0 || + !strcmp(ifp->if_xname, ndp->if_name)) && + network_debug_supported_nic(ifp)) { + ndp->ifp =3D ifp; + break; + } + } + if (must_lock !=3D 0) + IFNET_RUNLOCK_NOSLEEP(); + if (ndp->ifp =3D=3D NULL) { + NETDDEBUG("%s: No valid NIC found\n", = __FUNCTION__); + return (EINVAL); + } + } + + MPASS(ndp->ifp !=3D NULL); +=09 + bcopy((void*)IF_LLADDR(ndp->ifp), (void*)&ndp->local_mac, = sizeof(ndp->local_mac)); + + if (ndp->local.s_addr =3D=3D INADDR_ANY) { + /* + * If we don't have an address yet, let's try to + * find it. + */ + struct ifreq req; + struct sockaddr_in *sin; + if (ndp->ifp->if_addr =3D=3D NULL || + ndp->ifp->if_addr->ifa_addr =3D=3D NULL) { + NETDDEBUG("%s: Can't netdump; no local IP given, = and can't figure it out\n", __FUNCTION__); + return (EINVAL); + } + bzero(&req, sizeof(req)); + if ((error =3D in_control(NULL, SIOCGIFADDR, = (caddr_t)&req, ndp->ifp, NULL)) !=3D 0) { + NETDDEBUG("%s: Can't ask interface for IP = address to do error %d\n", __FUNCTION__, error); + return (error); + } + sin =3D (struct sockaddr_in*)&req.ifr_addr; + if (sin->sin_family !=3D AF_INET) { + NETDDEBUG("%s: Address family for interface = %s%u is %d, not AF_INET (%d)\n", __FUNCTION__, ndp->ifp->if_dname, = ndp->ifp->if_dunit, sin->sin_family, AF_INET); + return (EINVAL); + } + if (sin->sin_addr.s_addr =3D=3D INADDR_ANY) { + NETDDEBUG("%s: Address for interface is = INADDR_ANY\n", __FUNCTION__); + return (EINVAL); + } + ndp->local.s_addr =3D sin->sin_addr.s_addr; + NETDDEBUG("%s(%d): ndp->local.s_addr =3D %#x\n", = __FUNCTION__, __LINE__, ndp->local.s_addr); + } + + if ((ndp->ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && = must_lock !=3D 0) + ndp->ifp->if_ndumpfuncs->ne_disable_intr(ndp->ifp); + + return 0; +} + +void +network_debug_finish(struct network_debug *ndp) +{ + int must_lock =3D 1; + +#if defined(KDB) && !defined(KDB_UNATTENDED) + if (panicstr !=3D NULL) + must_lock =3D 0; +#endif + + if (ndp->local.s_addr !=3D INADDR_ANY) { + if ((ndp->ifp->if_capenable & IFCAP_POLLING) =3D=3D 0 && = must_lock !=3D 0) + = ndp->ifp->if_ndumpfuncs->ne_enable_intr(ndp->ifp); + } + return; +} diff --git a/sys/netinet/network_debug.h b/sys/netinet/network_debug.h new file mode 100644 index 0000000..af7c7b7 --- /dev/null +++ b/sys/netinet/network_debug.h @@ -0,0 +1,89 @@ +#ifndef _NETINET_NETWORK_DEBUG_H +# define _NETINET_NETWORK_DEBUG_H + +# include + +struct network_debug { + int poll_time; // Time to poll the nic = (0.5ms for each poll) + int retries; // Number of times to = try retransmitting un-acked packets + int have_remote_mac; // Indicates whether we = got the gw / remote mac yet (via ARP) + char if_name[IFNAMSIZ]; // The interface name to = use + struct ifnet *ifp; // Interface to use + struct in_addr gw; // Address of gateway, = if any + struct ether_addr remote_mac; // MAC address to which = to send packets (may be remote, may be router) + struct in_addr remote; // IP address for remote = end + struct ether_addr local_mac; // Our MAC address + struct in_addr local; // Our IP address! + + /* + * If non-NULL, this function will be called on the receipt of + * a packet to determine if it is a desired packet. The mbuf* + * will be the start of the IP packet -- the ethernet header + * will have been stripped off by this point. The function may + * want to check the source address/port and/or the destination + * port. (If the destination address or MAC don't match the + * ones in this structure, it won't get that far.) The caller + * should not alter the mbuf in any way. + * + * If the function returns non-zero, then remote_mac and + * remote will be set appropriately based on the packet. + * + * This function is called from network_debug_pkt_in, and only + * for IP packets. + * + * Saying the packet is wanted doesn't guarantee it'll be = processed + * by the ip_handler. But if a wanted function is set, it = should + * make an effort to indicate whether it really wants the = packet. + */ + int (*wanted)(struct network_debug *, struct mbuf = *); + + /* + * If non-NULL, this function will be called to process any IP = packets. + * Currently, only UDP is supported. The mbuf** will refer to = the + * UDP packet. + * + * The caller is allowed to manipulate the mbuf. + */ + void (*ip_handler)(struct network_debug *, struct = mbuf **); +}; + +/* + * Checks for netdump support on a network interface + * + * Parameters: + * ifp The network interface that is being tested for support + * + * Returns: + * int 1 if the interface is supported, 0 if not + */ +static __inline int +network_debug_supported_nic(struct ifnet *ifp) +{ + + return (ifp->if_ndumpfuncs !=3D NULL); +} + +int network_debug_ether_output(struct mbuf *, struct ifnet *, struct = ether_addr dst, u_short etype); + +int network_debug_mbuf_nop(struct mbuf *mp, void *, void *); + +void network_debug_handle_arp(struct network_debug *ndp, struct mbuf = **); + +int network_debug_send_arp(struct network_debug *ndp); + +void network_debug_poll(struct network_debug *ndp); + +void network_debug_pkt_in(struct network_debug *, struct mbuf *); + +/* + * Call to initialize the network_debug structure -- it will look for = an interface, + * IP address, etc. + */ +int network_debug_prepare(struct network_debug *ndp); + +/* + * Call when done. It does less than the previous function. + */ +void network_debug_finish(struct network_debug *ndp); + +#endif /* _NETINET_NETWORK_DEBUG_H */ diff --git a/usr.sbin/netdumpsrv/netdumpsrv.c = b/usr.sbin/netdumpsrv/netdumpsrv.c index af6ac6d..283a003 100644 --- a/usr.sbin/netdumpsrv/netdumpsrv.c +++ b/usr.sbin/netdumpsrv/netdumpsrv.c @@ -454,7 +454,7 @@ handle_kdh(struct netdump_client *client, struct = netdump_msg *msg) return; =20 client->any_data_rcvd =3D 1; - h =3D (struct kerneldumpheader *)msg->nm_data; + h =3D (void*)msg->nm_data; if (msg->nm_hdr.mh_len < sizeof(struct kerneldumpheader)) { LOGERR("Bad KDH from %s [%s]: packet too small\n", client->hostname, client_ntoa(client)); --Apple-Mail=_B5ABEE4A-F6E8-4BE7-9CF0-3958B48D3DE3-- From owner-freebsd-net@FreeBSD.ORG Thu Apr 3 20:15:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C69C3267; Thu, 3 Apr 2014 20:15:34 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 38DE6A19; Thu, 3 Apr 2014 20:15:32 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s33KIOGD071235; Thu, 3 Apr 2014 13:18:30 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s33KIJrw071229; Thu, 3 Apr 2014 13:18:19 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Thu, 3 Apr 2014 13:18:19 -0700 (PDT) Message-ID: <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> In-Reply-To: <20140402020803.GB2938@michelle.cdnetworks.com> References: <2598eeb4c68e23df0789e5e3e8f46d76.authenticated@ultimatedns.net> <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> Date: Thu, 3 Apr 2014 13:18:19 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: multipart/mixed;boundary="----=_20140403131819_44728" X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 20:15:34 -0000 ------=_20140403131819_44728 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> >> >> [...] >> >> >> miibus0: on nfe0 >> >> >> rlphy0: PHY 0 on miibus0 >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> >> >> rlphy1: PHY 1 on miibus0 >> >> > [...]---big-snip--8<--- >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> >> >> As you can see, it looks much the same. I have no idea what >> >> >> I should do to better inform the driver/kernel how to better >> >> >> handle it. Or is it the driver, itself? >> >> >> >> >> >> Thank you again, for your thoughtful response. >> >> >> >> >> >> --Chris >> >> >> >> >> > >> >> > I think the way to fix a phy that responds at all addresses is to set a >> >> > hint in loader.conf masking out the ones that aren't real, like so: >> >> > >> >> > hint.miibus.0.phymask="1" >> >> > >> >> > You might be able to set ="0x00000001" to make it more clear it's a >> >> > bitmask, but I'm not sure of that. >> >> >> >> Thank you very much for the hint. I'll give it a shot. >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia >> >> chipset, and the nfe(4) driver. But they don't respond this way. >> >> >> > >> > If some nfe(4) variants badly behave in probing stage, this should >> > be handled by driver. We already have too many hints and tunables >> > and I don't think most users know that. In addition, adding >> > additional NIC may change miibus instance number. >> > >> > Could you show me the output of 'kenv | grep smbios'? >> Yes, of course. >> >> Here it is: >> >> smbios.bios.reldate="11/22/2010" >> smbios.bios.vendor="American Megatrends Inc." >> smbios.bios.version="V2.7" >> smbios.chassis.maker="MSI" >> smbios.chassis.serial="To Be Filled By O.E.M." >> smbios.chassis.tag="To Be Filled By O.E.M." >> smbios.chassis.version="2.0" >> smbios.memory.enabled="2097152" >> smbios.planar.maker="MSI" >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" >> smbios.planar.serial="To be filled by O.E.M." >> smbios.planar.version="2.0" >> smbios.socket.enabled="1" >> smbios.socket.populated="1" >> smbios.system.maker="MSI" >> smbios.system.product="MS-7309" >> smbios.system.serial="To Be Filled By O.E.M." >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" >> smbios.system.version="2.0" >> smbios.version="2.6" >> >> Hope this helps, and thank you for all your time, and trouble. >> > > Thanks for the info. Try attached patch and let me know how it > works. Make sure to remove the hint(hint.miibus.0.phymask="1") > set in loader.conf before testing it. Hello, and thanks for all the attention. Sorry for the delay. I chose to perform a dump(8) before attempting the KERn rebuild with the patch. But the kernel threw a read error message on one of the drives. So I had to sort out the problem on the drive before I could complete the dump. Then, of course I had to reslice, and format another drive to replace the ailing one, before I could perform a restore(8), and start the nfe patch; build && install kernel. Weird; the drive had only a few hours on it. Well, anyway. The patch applied cleanly. So I built, and installed a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" in loader.conf(5), and bounced the box. Bad news: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 miibus0: mii_mediachg: can't handle non-zero PHY instance 30 miibus0: mii_mediachg: can't handle non-zero PHY instance 29 miibus0: mii_mediachg: can't handle non-zero PHY instance 28 miibus0: mii_mediachg: can't handle non-zero PHY instance 27 miibus0: mii_mediachg: can't handle non-zero PHY instance 26 miibus0: mii_mediachg: can't handle non-zero PHY instance 25 miibus0: mii_mediachg: can't handle non-zero PHY instance 24 miibus0: mii_mediachg: can't handle non-zero PHY instance 23 miibus0: mii_mediachg: can't handle non-zero PHY instance 22 miibus0: mii_mediachg: can't handle non-zero PHY instance 21 miibus0: mii_mediachg: can't handle non-zero PHY instance 20 miibus0: mii_mediachg: can't handle non-zero PHY instance 19 miibus0: mii_mediachg: can't handle non-zero PHY instance 18 miibus0: mii_mediachg: can't handle non-zero PHY instance 17 miibus0: mii_mediachg: can't handle non-zero PHY instance 16 miibus0: mii_mediachg: can't handle non-zero PHY instance 15 miibus0: mii_mediachg: can't handle non-zero PHY instance 14 miibus0: mii_mediachg: can't handle non-zero PHY instance 13 miibus0: mii_mediachg: can't handle non-zero PHY instance 12 miibus0: mii_mediachg: can't handle non-zero PHY instance 11 miibus0: mii_mediachg: can't handle non-zero PHY instance 10 miibus0: mii_mediachg: can't handle non-zero PHY instance 9 miibus0: mii_mediachg: can't handle non-zero PHY instance 8 miibus0: mii_mediachg: can't handle non-zero PHY instance 7 miibus0: mii_mediachg: can't handle non-zero PHY instance 6 miibus0: mii_mediachg: can't handle non-zero PHY instance 5 miibus0: mii_mediachg: can't handle non-zero PHY instance 4 miibus0: mii_mediachg: can't handle non-zero PHY instance 3 miibus0: mii_mediachg: can't handle non-zero PHY instance 2 miibus0: mii_mediachg: can't handle non-zero PHY instance 1 Just as before. In case it should make any difference; I'm going to attach my copy of src/sys/dev/nfe/if_nfe.c in case there are any differences in mine, that do not coincide with your version/copy (I'm on releng_9 - 9.2-STABLE) FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: Thu Apr 3 12:42:03 PDT 2014 root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 Best wishes, and thanks again. --Chris > >> --Chris >> >> > >> > ------=_20140403131819_44728 Content-Type: application/octet-stream; name="if_nfe.c.tar.xz" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="if_nfe.c.tar.xz" /Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4Wf/QItdABcLyacZgBFGLc+VgFjtbL8VroppXOmgEW64 8EsUlFZhhG7JIVEUFNmzkVLzerBQQeHqEhkN2wgf7Qx7gWLOkMO9yjJgWQMunWK3F15h68RTNBUv awxFBGD58yCktJwFJQUzOv7vsGWGbsBORd7jnHBTyfg7vTcBsUUyuz1tqIOng/oxs9YLfIBKsVPy l6my4QkBwYARTYiI6mzt4YJL/eibdsT3e/sgT7yb07Isxh1Xkf6ROLhb7Mu6Y11B8AEoAWyDy6CN LQGexasYwrb0W/LfHPHJKJb33aNCLW4oKBwlvOZendqIcDuz63SLz6U+YR7AV+pAF3/wX8gZWGip SE67uYMT2rPCL7/8LbwDfLitpeHSvfcBYqFii0q0L1gI/vBnHCHPCFNCj5HBwI1YpZweP0m8iXwd dpIjxWAh2S4WirOYD2swRLZBgPZVP1xGBL3AaTJZ41dPerlB25jCYmefahcQMrRuhxziX3FWde51 MKWrwvJegDGX03jnSzTTDvScANb5ffwtdFrdSmV6/r1W7scDBkjg6GY4SMvBwoVJnxN6QwEaFkXV Ms9+JTHD73EH23zAbxd1ojh7Xvt6OWlzKfy/wr4ccXF1q4iB3eeDBRI50/JDCIhwA+UGaP0RQuLc AX0Y0j9gb2wsAF17NP1iMy9+Pus/+5jB4HLE/kYfZl6qGrXT7M4GQgdiLmhwHZByzgLutgBbSc9D K0167mIZG+GbztsVTw6gjDk+3E0FKbiubI6URx6sEAlp5WrarLxqXf3J4rq35VZMEbqXygeyb2tG BHZCBqB9PiFd7y7NopWGF7OjHTUQsdcjXOtm6vAdmDD5rWkOhOM207+ELRIT0yoTmsnTrn3qEni7 4+dSVbvPJdOWKljUxfUAldBCbT4nkCKRkFyOenYzTpUp+ZGDfQz1JOHXYyHT0Yf3HW4F4cGg9z1m HiyIXdDjMjkFANWJTPFMtpnmmoxUXW3P0iUEoZcjutHv8ID6Z7KhZWO7SecoqrdanDuO68+VYTu4 LECJcsDgdEZxf12Po5W2NHuAR25aYORzeLKPSuZ+XwMIDLl0K4/oKMIJxAIPN0QqWtThyrlDr24m NEcbpssRoZ6F8o1fKzVRJvPU64i9m/gelm4ABRZ++e20brZ9H/JXV8VM7a1gensd4cm6kSqPQhxV b2UkpsisELBcOAUc5IsUEQBphAM3Uyd+X3NwADFHJt+5W4UiXaEAyDD+GxEIhynRf1njJe1oS7TZ xRYdu/gNuPUno8voBrB2dW4JsxOjU/Q3YCZJe01F+R1XZVcpmQs0P/9Sm23YXA4qdYjY2K+HKpuY C01sZJL/VZm8UeBjjxiJrfXyEJqeBOpujaPbBKxDdcUAjeElek/VpQn+Ezd8B/Iv3KtPYgpY9+a0 EqmgNbmqHiRDUSL9F/XQEqehBUZcVlhQSK/FiDg0FWe04YIkpW2uOYxaPp0GC6SymfDiZC3PIuBa SpcsK2lFpTwoezZTqKuoGH2v8tCOpuWHrGA1PxIxDdcaJrDfVOsnm+puCt6OTZXVGXOy4bKbLBtC s9xBT97GLO+aw9PPY3/BNKr4xvaz4YRiGUayb1/hxGqtWvRLvwU07SvHZSwFzsN1en8R97RkQmp6 kT6J+FQdkAztRRfb2fWSFN4snTWsAOmrvrA4Z70C/XyYom67K5JElva73lm66g9FaTYnhqq8npmD r3GlwlXKWyP5xawuzh1VmZX5h6wEyqfpTHz6jcgLSG23oSnFq6tKbDupFqHAMAWyu5tOYn2du+f9 KOEDgglHHWmhWC9mpkI0CYqJ1ntRC49x2s+UtUZZNRPX4pJGNjAshq7/rxMm3mkZfEvBnDVhkTUm 5XHF2e9TAjI1xIWoq6zqueZi8PX5HJ7hjpa/46Mzovu/j2w0qFdHznA6gNva0cd70paWeBf9HOvz yBHsDD45k5oLUVSII8sJFbd7JPsa1Rcp42LQik93A+2OxQupHWQroaIJfI+AWmCGvtBVHlGP87hr Jcj/Fzn0gmDAFhWJsKmGUp/NVOOx1xNb/E/jQQEv1CzgpwYJveWQagNHNc0B46tXskWP2UmhR5SY OfX+TT82T9hxUpkHdwkgletuGOu/TKPOwlU/SFGVrdYCuT1X8/TCc8HJP/eSyZc/+I8v5gQxaRnm g0KG9OwLLc4pr77tzP5YVqYLgAPxijHeYGGorwduCxU46awHXqYZ4n1WaEAT46xCpL0OB0BpAyOR zYK/PuKCZhMdgZLKL2/WcVxKdHk9zkCmXwkr10JbFTsz2QshhQaHUXS3K0UwgeqCVEikDmRwEgFG 1V6+Bt/xMLroTI8KAUSDC6RAJ0QJKNr7NjSzRcVdSWJzdNQWcoo/JFLTlk24GdYrx+QKXzaZJQ+v 1ebwVY7Sn0wWUnOguYBQasG3MlvyE2M3xqeuvOZx3iPUf1/5JtQubDZKPiPePzrznGbSz4o9VZuO JiDYKGhGFds0y+e8zGS9tAe34J5P/kF1fVXYwoe4kuB4rICNsxsuCsJLAJpwgFGoz2zziXattOJN GqZjU5qm48tDZ3ASzIZzIfOV0MtNSMX2rxD0ca5++tMjT/cad93PHiHYKsFmA922bLLtFFGKW0Ai bdSMcJcucmYH/HH5/wvI/pb2r/btNew0YEgeayIf03cC/bMjdyK0lSqGrqtXCxm1TpmhGov6DnSz /hGAILXl7wKc4ckYPLS608X7s63z5t+L1f/7uSkVl3ZPfXVgmIxlckXuhcsFCjPM30pJzf8qkLM2 T2sJLrEoToWFcMWHCAFEzABlOOzKKwhaV32eI1iQGeQMr+n/La6S81jMz/SMkiVYhb5Aruu7+X0Q INpeZsAlILw8S+htAg1BJ1vaGRrDNXPb8mOsccRIrT7Oyv2yQcGNjs3rSTTq5j7uHp4B6YCFHTAA P+z8g0Xpi5rFWPNbPuxgW5NpKuf/VQaJ7gl7BNqrvofP8ouPxWK7Cy9cZPyy3H2gAYa1RfTL1mk7 moTiyRIA+ZWaDvedPUzpoiu1/XwDCxz1zIZaZAyEkqjiU3AED+QGhcC82OJAjgF2xmO9+WmCn2QY oVh42DArShr92oBrRnOocSu0+o2hK+KiLZuE/Bz1xl5fIYhY9dbfUJAjHbn8W2GAL1UJL/yZv25A Fa7blnEQKdGhYMAgcWUaeOCJ6dDHnGybnxArC6MYx91ZpWDwNfRf6lDpQn/1ZNJxq6MdwEDKCJQg Cujj3Tr9y+ccRNE44u5+FQOgdd+6VaKEqKusIRZHJEDf2txiDmwsJJElh8XaYqHnaiX6ZmJT1pRf n0A+hBlRZ2EyQ5on+Og88uB5fx7Oo9EJ4BBsf+m8ZpkWoAW8mf2RSMHPlUnuDIE4Y58msHsHqOO7 qsEvCrcBVGi/ZslrD14dfGdFgmTu/yIGCIshVQCcR0a/h6bLow9pvWu0ffvu1TdVCJO7ZTh8gMoV HcAcCsHsZ6v5fGFxrwAc/UnAYZysTXf7ciwzk977wdFz2gTm3P0PFoMzsl1UTGuC4W+GJ5zJNZ6u QraH3GfyTqhUD9qtlUSMQRFMGSFeyRzapGInlFkbpgryJXj7U9LMy0SwQ9QSnib5YIj2pgKaiXvG Qe5JaaL8b5ueILR6lnOA25xQxbZQTKKMXJNbGjeOdk5gwh9TI/xRSv8Vu9A/EwgwLIdC3VVjQDpO Q185oem4yl2rj0YpAvE+8x9+RuAPzIOs02LClGrNxCB5DsCs6h7mbFLAWwbgHsAy9bR2+DlEuH2L +iqCNrKL7LdmqI1EoD5OLtOj5zckqKBuEd/4FRth0XYmxpNVyY9/6Pf/pyx+Q5CWaDBbs/ja0OXB KxYzTeGTS87+i24HaY/YPVKwmezqc04ySBmPyVsvMtXEiJXWWywytBaXcN1LQ7VkReqilu9dcfTn hXk5XPX9AkSOmYJ8HN1cuFGcjeYU9XtX4XJhXX2QXMDxlpWOLcnnEsax8Zls7aJUI0ZAD7y1CU0q IZCdKcP9V8P2wf+VKFJpOSVxXGkeNjkVgSLCZ542Gg6zbj/bwKLGMxjBObqrMw/gMkd5V08VopoJ dtCBrsGsmzrZTM1j7T7i/F1G3NBDWtTIW/sXGkTLLjFZmKU+6Ivf5S8KM/ft4Ukg+svud7E/NStM 1LKYISPQekJ8hs9tq9+EWJExPN6Kfwathe/mdRL8wYyTUcAvD5b9Ht/4soZd4ZhV487iXarGFCAF kbUK9Z6Y+rJ29lJIxlgAHmpq52xNJXY/uJPuOaRlgihUVAd0c8yhCHVQ4eVevmcbFmPQtXAk2cWp BlhG7S6+Gu2T7rCfmd9iksy2VgVeEKghwn1MSxKEOaZs53jyjxChUOaWf37tyn2UOrb/YXI2qeaC 7CiJNSe4YFOC6ZiJ0eh2a/FkS/suCXB8FeQ5O6FSL1N7kagNcZQR2sjOVRAMBORQfkbPB1lZemSR 2aW9mzsBaxEj3oZZRMQnonQfugoAEq/cQAUxwKr0wNw/ygzQpm7sn4mmYbeT0RkzlwAOK8bYSIAC rHu9B18kQncO2/oleTW8bnTuuS41umUSNjwCLp/zrJwmeZGJ0i/c+WiTTFYFUSIRb0tjzP9ci4VE EtlQ0azmWPSnFE4dOpbDHrXfsKhWxeTE4FguNEs/pXeljlE0JUmEnRFc+vba00T+mjELAVckNTsA CwmogiJxOkSAbcqN/CgCiiU9wPmtMoNVW1Up23cRnJEtAS+3KEG1So07pyMvhe1mfLBk/w4oknRH F7k+LoYUUfr09HwNwfHvHMfua1Zs8B0xabFaMACrUf52TabiBnAGebb2BFUridkkpVEBvDUMieFp DLVj2lpVLwEbxJPd5oN9g7lhHaBgWp/kbyaPtOe5d9kFMC21emQ7AAUo1NYcjXdjtPWEwS2XbCjN ejGr8lmccF4y41PyzX4/6/vPiUb1Xr+e79/MaGMx7qkfm6xK3V48M4naqrzI1xsYVBS/bWKwy5h7 wodTLvkTn4d6cWk6B2pfb60TyZGV75O+Bw9mqYCNL/c4CGZIGJznFxRvt8Rb6cTFakx4hSH/yblO +1AR75XrKIELQrYVSAMXxWtslMgi6P90XMRgMyfdFl0lV2uCleU8PM35u8QrKmtd7ssAuDC6QaBk 5NQHYHW8ex8fC02KR/v5FEH1Q5lFrm/RzYqpaOdK1EQKouBYoSs709uEaiAvWw3toDFe5NQ7FVJx /CdStSvhsUpbgAWX0cSQYKn/anhZKQMRdl7Q96lbezZtRFtrOZ0jI4GDUapHLezSgeWA0vxeB2KQ H88kn/7OELX5I5VapRp/DoAExPNxQDkO2myaOG2fNPnoNnNd7VZdkg83iYHfAoMGv5Hh+vtLFmx+ wT+Qm/L7VfD4NUJmu9tcu8t4tJVXCDlBbrSHZoKPl5atmPauBEjRBiHXVXZ2Dj7QJD3GPqCWD+ih xF7eyeVZqsTpMtxETiGZhWEUM7l5N2S0SdGQ/3fD7RAFU9G9KZSylmP8r6G7epD1XoP4WSuu5/eV rxVscNA82Sq0cexWvgiqbEy3yJ/jVcgFZT6D1jjA/IOB+EvkG+bc6QFw47yx8fPrj9YVWw//5yAU QXPNHEgqw8XZjxdM0i4WS57rRM1n4LQVoqLU2nre9QlRQwv0IiJGdswcXHhEpIZDDiBvTQNJedyM W0CVBXxxwG75vOyQE5oCDZpBPKTXhKyPzL8JbGu3LY9VUk2YaExeeqx4XpYyqToQBusb+1rmcrHf mZm6ygBS6zdJ972pcGwkhfb+HPkVbOUhxA30/fmErLhrUMZpxTXtaL7W848a82NKDAmb8Y+qDNVI 1MWQtq7mThmWakTmHeQJ52INKxeo3QfwbB6hsf84FT6D6YaJEJVjUD6zXxlsJgTOYJcJ2EsA0Ia5 SJZMlBAZskmvhXvOD6Oqo0w4zMT4k41+MLQD7cYzfJMj9FWdXJG3sfWNjQje1NCi0PLsfGlIdwfT 496Gt6XtH9sYGkMjKNxNp+N5MMEDjmCyhfCCOHYuX7O3+1HlN5sAMD0+HxKxcaPyEdnBM7wGfoVx zNzg2OKuwtUKErbHPshqmN1dqBJzIR73nTdO/x6FaYWzDxSI7nyazf+bmFZFX1P7mSYGc4MgKiDJ gF1bjtRsuLOmJQN2DOqYVdMWBuaPbNY8vY4Cugury6yBei0Soy3eqXzf577rkx13L/OTTjA4OCmb 46Lwp6lPAZv3wsQhyIcVVNPTOHzPKptEMV3OQjlEIlYpai2lTd2P1lM1pdKV+VWOCDlyaUVZTOFK SeAIerPyA8TPwCFBhrTgxBAtJq6aLoq1gUMkgUO93I+KU6vivTGzaKwFlWQY51hyoGUbieYAucGB nyEMdmPORIIKTktKWnOrJxmiunsY+RgA9yxgPxQYm+1rHGkD0j3sZnNG9RHWK0QlZ8zqeuxSvFI0 EybQtkNzvf+DCiGdW/4VAQPV0xyJ4qZzHJ0j2OcGc12L9ru+YlpyaANNjH+XJyTviFy0Rd2SFU3Z g10JcxKOWVEvg5ANLGXlvWUS94wydetjiBEuPkUb41Gepxoe74v/SN9Qnmb11AqM15bYOdCq6v3K Xqhtm+okdGLrJUA4b0X55eBoT3Cu5+IxEzf1SnaeCPKiRuGyukp14+GnkmfNLTcwQK1Ikdzg2YUm 8Ak8a5USxKKHj8SlpBWFamO/Ffdk569C+TfgqubiAjxugaTJGMYWjfDA1030kGRxUtad0wXGQxYS A3NGqHDJuDz1j+HvWKieWQzR4iGBBZOOivcNBHIiXJVQASA9EmqUQiKB3S1TOl2lMNF91ayjrqb5 8/Kqor5XddZH8B7VekSwBojYVd9edgwFlDdu9LSoFXKZp6/T8mLXGGSXfF9EKYFMTwivVedyPay6 hRSQXyb94nWck4ixk0y4qSxf0TGJeDhhYWf9RgGuX0vtHnuHyy6Z+s9NtxpKvBk1MR8M27RAuT1E 6/Y/ib3dZXw2+vSFQc1RIOetwl42vy28jSgc3pAEYe3XCQA89hysLx+6v1qyaxjRqwiFRKAF4Jbj 3IQ/fFU90VOUSVcI7NWHxDcQvJBYXCP5mY0r59QTc1sPZ2TAYSN0JmEt+oWMGt3mfnlQ3fk0eRIr VgwGzUiIkJJ/SpXBKIkryZcUDLn6WnCNeCKlITq6wDrre2d+s0vebjj3OYQ+j/W+eP5T5YwwVl8L Cpga7JNNsLSWk9Ry23HG+RJg4G1Nr6miU+v7c39em03Pkxg4JNyURmPRWjvFY3kk9ULPu5DuLYWZ M/Rm2RvLXnN2lVMVvkxI2XyWM7jJ0J0JjgHZPEe3Wwqb6aoP9vQtEqQUm8n434XH6V8eudO4ETUN E77L9FJLhY/WGNh4SC8XZ+9qcDUmXW6DkE1ZA1DQkOi0oRE8oGAYXpWeCnTCK35fBrC9nYkCWVjT TyQ5WpeoC2owNSNyPoQwzHXhXfiUq5m7Rp3aKDoHRoOf4QNBdMZh19zUmW8nv/AuGLPoGZwgnN+d szy+b4nRqD0tZCfHwZNqHttMvkSetxiYisSnot4HuIndZvwE2o7V7OqbNAyDygJWgC3TBe2mm+wI X7c1wLQ9/U5r53OXWDOm2m48BJ7mA289ChdiAeumKBkbeOoGaQY0J48AFL/7KwNr5OiIu1m0seAG YU6KyS4lhzH+jgLvYimgK1J1VMO697bCGM5DjNeWAGvilE413k4+5n5MkYs9IhN1R6i9EeqGFVY1 4JAyr+84DSE49x6TM1WW02SVryiwUFcMjpwE+SkGBaIyKYF0TD0QG5SS7havq5D/GbqBAn504GcK dE7ICxrJT7uX7Djs6gDizwnd/sYlTx5pehED4ahQQ5ETkFn8Qn/djeANPZRU9XKxrbcXpA9PC2rj ncvy1wnH1HyHcXZcYhCJ2xAFJOre9s2LxZkQClc2QifJaxr4mDptYMVOkLwlP8eqc1F3LdQj5GSq xcdi+9Sk3b+JYM23u4WsQw9EPk9wL+Uty8W6/Iqr2E9BCfQL31rcc5P8MzOe58DCi6gWmbA8emmV S4y59Ye0CJrJvUa0oJ0aEYTwq174XDj3cEup1JQl1EzqJnpqHFcOFN4d92u8gT0HzeQ5sdo9J/+D vMgWDs47Qgot2Mag/SDCLJaCCiMHYSEBxFmZBu46PGTpLcpmA7pebAfVivL/ljfrJPqyaC5JpZeC MDnqtvuJAJnewF9YSR3Pz/X+wDn2HS9bMkvzWPdWl0mJg+pQRjkWmQ2sJgmCUzQTXHuSWPvakUmn mu/T252/DxkzCQgazVkazkBRPfmtVLaL8F5DEgHq3dayR2JeYikzMZ6BPqhrAoxDtaKhcQZPf0hf ofL+uECNVaMyhEjiIJKxqeKnA6EU9OlR1mmjoRtfP9qW0U38ipPitDhtTZUK0omeWcmyDiu6BY5q 47b9dwJ9ASBcCfurQCuEfWhoLs5UGR0GItA5wo90t6iTzX2BHLjVhV25rVdZQONUZ0XJYvyaynWx tnyGu1L3Vzx7lB+4BdhdXHzEmU+tI88+xPGp/kUD5EtxDWS6sz38s7VysGczBuTQTMMtLmc3HFB+ vO553XihP34e3qZ7UfETrivbD68wgHAvJk0qz2HSFKPs19A+E2mHjaxq+b5+vNj8bBrktvyEqsBy jc7nzROcr1x3uW89HX8rvll8kKTUVRrpxfLXHJOFTEMdBAVSEdFTkRU65ptBGg/Q6W0OBkhFhnuU woL4hs1zU/YXzoSMREQywRIwz381yD5nLNEGR/F4hOxJEm7DNlqmOEycyx01M2g32ImnTv2SZI2+ pTJ1o+CE4AV+aNRbJQ4scQpgqbqMJXd5d1Zh/djZAFvl7lNsplnsHkEb9Rp/75DHLXrm1hiO++y1 7EHil/drA0RVuFHy17ZebrAkwdt16ki382AdPbmpB/yaX71YlaeLPjLVJ+s5HxeY4dlDOON05gAO VCNFQcAzDmtWRCdtVn59WPfUDTerVrncvdCbFB37fyPQVYlLS6woBENP8RkbG+2rs7lwBgwzD+XG k+PyOZhksD1FDpD6RY8QsNWgndYhfxUktf4A+95X1BhN0wKittx3+DJTt8qBsbvo+VCMp8ETUKum b5/g0GiF9JQVfRcLzIeDdSZA9RoZoKvnTNMfMc4kW3QBr5zVSmyDehALMSHPCxWbAabkjQm1fRJ5 6WynVtZ7ImpN3nfq+IwpblrMQ7bHL641fLvxFnEb8tPRgtmI97sLDF8Zlrm/aTkeLoKpvscvuUTW ybTK8w0tyv/r8LgNw1tv0QuTRC6JHMVkXb5cdBtGFxqspJBEGVkQ/pZt30ibcjEhM2eSZWgeDpVd R1KLVvx5zxwFOGIgLA6saPxj6LBqiu37eb/muDUes57JOuiSqZEA2YhP9vlEsfFt2DVFuOW1vhYf Srpc+ugpiQji3FAe+Ttz7DtcPek7ETf5ZSrBpXeVUne6ghUOS8HKBpv3R9ZQ1KEVw4FTaNphhTzb YL13Y1R2haQ4XNpWOXN4jL2gSKcnvRdPrB6lXnr1q8FT2nj9ciWMZ+36etXVM33MbFRelk/5FCc2 p172cUVK+BXvE7F9PABPmvlTDJPMqR0saKl5rtO4pNOuZUO1p/TdravX8qccMaGllxeSm0EnzRId tl6vnh+8AyWeADHwdGl88oWm8mNAMGr5kxklnBX8GM3v1uL9DZm+ZsAYBeB+FCeDJRv24AFFknBt eGCX8orb1+9acMIIGxM02MvB+EmIhHfpCwABAlAnyiXdzs0bn3DB41aOoPx7KucBUJowDzUyxw/X LmhLg1hupPVBT5FseTkRYX3rliC7XShHk+/4C6S3Yokh56donOfdaN8l6ja08rfLqC1mE/DeTafK fdSLo05InT1xy2HWyGO72+ffu3u/MkZgLVnEmaRSeAG7Loa7zHS3L3W19k1JRgFiO3WPQMIq3j9F KDh8Bt+w/TDT79Pf6TwwyBqnGc5/KzYQnjgpVLGBPamjY/7w9vpHDZ2WnMNv/wQC7M2tDyPL0mnY AdJxdsSph/3tK3tMbSLUcAXJ0bAg1NaLqj+7Ce81OwD2WE2pRHMESTxG06VCKa7SrlR1fDEbpLVa YZdloGXy/4+vqtV+CBi/sAOol3+48Pfiz37QVL8k2KBH7UWN9Z2h9FnyrHUvrTm0rG0xQbF2xBij ojzI2OTnk6DPbxXy4nteoDPxCc6ufrPv79X0bsMPZCZ7d26xmYATAduJ6A9mlk82fPUV8FkWr+u8 EjmlX5gH25WDGhxrwaBOoyHY4xgetU+9cpJCJe5nF1JBt/QK8rls8WPBQ8tpw+KX9vMw4bCamlMV ETaIIsMh0HJH21uO7EuI7KjDtu/EuBHCk+Zk3t58Hs3HXqG8wTl3LHeoJbjkY3DycwSqYLdQtCKV Zv2EYY/idJVwPEhN/5ndrOhhI04TlKixCc348odeDEZp7HKslRVitFUPoGF4muXKVPKw7QaEOFdb 953Xy7/6rhgSdfDZzHmd6Y2E8MBK1Z/C8SOWJIHn+vUD1+IWrmK4GHPSZL0CTxaxR8tG8SrHrHAI S5eWVvTP6EV32mz0zOx6t/8mlDBf0Grlg1XXc9sWmUCMTpHQO6YjxHeqWu3CvkxYhILsgym97aJ+ d7F9N+kIb9YG3qyJ8KY6Rf93ybjcuV9HTvdDN9nEH7k1wMPXva6eALU1v8JVfBCv+lxhiLaubLZu CpLTI+7MArRJPgNv4eQvEbxUSlUObbFgIfR8RNlserzo0HoA4mNU6a2eBgkoozZHvjMr9linJj8x Q3zkNTh93H3mw09z08eTZWyljRS1VM8urt4JLQafveL+L8srRu3MGlrBZfuhY7b8sUY+wAq5PadX z1d3iFEhHJxPCNhOFJ8waQ4whwuCF8z/su2TAHELGofMp3qSUxW+c9qXKb85OadbxUfkGdeK6ZC1 0+mQL83IAn69UUBPfA533HDSVSKSNTaQl4lC0bSp+aak13jwyTEcUHdnAR7Gu6z2u6ANcfKpDdwC yirwmlgKKsV6/wgQjr6AyUN4KYfRzTpQZHP3dSOpecOCeYtW8CSyxuC2zs9gy8qVK0kfeVm1rCrZ A5kzcIphghTYSDo7m/Ei50cG2XaVz5pnnisiPM/2SLGjmX5LvTkLx8/wCKIhjQUmuotFl/WjmIXJ RzFaKuBf7ergJODPC09GXXkUVg9xdz10yvJfLEYVpXGgG/fImo1WMKiurfAGGMcSH37kLTJUD5hY YgWDdmTjqERf2WatyQDeFXx/5wwLxEdbfA76heZKa8SQ3vEZtxqYrpDXyrPsEHMD/q8jYK+1Z2mM UbTOXzmrfKr0/TKnLQU+631h2NSC7ndx+lxCgb2XyeUELJxGRZJ7VhCg9Um8z00IYd9PFijFmYyI UMOOATljB6GVGXUsQ8kg30qEhRY0GepVy0FU1C0WGXiuezlMFAX7vp5bajDyKe8Jv/mMW6SpvR7E 4Ue9VO0Tm3hx6WwC5c5oBv7JrDhT16aDVAFufm4bieyP5GisG5FeEb4Od6I/Ya7fs9/egx2bJ7zd Yveg6bDwjcfiRw1SUtKR8HL90kVktzigcDhYbVXui301/JcvYR4tNTF2rtdpQml2kMZ/e/e+1CSj DeW1GsaFwoIZeNH7FC3U+3LG8d8ZfUU/IuoAHUfj6Vthqlb7UKbTnpji5Jr56QAfIENmeC5liMaz xQNte6hMeRWBJ34HNx9RlLWmmtutXiocTK9Blc92p1/5wPnNkAJWlg+0mfSpXvpWICglrL/+y5ID wDqEAtGGCr/NZBjrTJraGwvxE9MFo2hu4ZvUiDmvItj99b1Yd/RscsDwbipbdtHI3TwrxT/2DqF7 0tUGNuedqCqbIUQfRHfiQp67S6mL0X8LB8ChRauyFIEXr3Oh9BFMRcUhbsYSzKuVJBYWmAVEGgU2 kccxaJ1PGhJzjkf2tpIK+sbyJPanYsXnQdGcMZfTtoZQf0rbnmAmeovNvIEDGB8fvglbevLXsDbd xVCu5zpHEH5WAogiBP3VKzx42y+dhbe3ndmgv9APswYCKGDjG55TU1faraRVFGJRtPBvR4HWZULb LX9DxOJwPhMs26t/eGGIB08L0RZSU8J4YT3xgkSaFE2LNTTyuLO4Eg3rLxy8Sjb0XgXDCXbdN6Gv odhnkJmivR73YiuR6TYMITEhZS/btSG/O6fNd/FgKe1B9lg3sjOT625UpaQQerQxb4tOKuH66ca3 QM/umNfimZRND0r0HM9Q9gvOOsCzDyMgmBj9XiGt1uBEanB+GhrTaarzEo1c1GqU+R+RwXZVyLRo AyL+1VpXAyC66xiuU4IxGiqcOkqX7N3KypPVqcJahZGsDCXg+YU2boXPkTqzyg8Ab4CAr5vwRJhP i/z16FJPKfKD5vaaXM3N8yMCR3sRrobmAZFaX+KJjHytKks4fYQtxiInJN/GVjqJMeEx/SIKMS1a RRNFLGojA6wrhSRkn73QlW4wGQSwRsgPCRwxhfpkIdJecmAHoA4D5AGzI0Hb4XSGg96k8aCyS6CJ 5Iv7jYmaOqIVhlUzNw/s3dH/76zyqcFk3YCmNEHEc0wOctl4FLs0akWzzO6lAAHxy+v2hGxtsdbI lVbHfr9QZX9gHhrwsT1lo0Mt6f3g/COuhH11lWfNmCj/01PbRx9d6quIYtlFx6l6GDa+2Y3xhvvH zpPImhLB2eB8zWd8Wh9U5G+RpdM9dkMncf/3SMRtbs48zvoXy8JFUJcnXTqV9yBrswNue6DWk7Sx YoVer2BtbEWVonfO6YpCcKKWfeiliTEuIcdIEtsmYNJbF2fU85GmCBPFPObVMGifQ9xz3A3Fjf/t 3J5fIDhQ3Z6232UPZ1EGdTyiENGnByf6p9Z9hnvJaLqG1yj9sKs4osdAtBPS8m78vuJ9GLkjeiIh SvW3g/yHRb4LnusSIUJ+w3qwigKYQa1z5bZKkqmBr3XncSBKjoyg9o+OKrYUqLx810mBkNgp4Kwb P3ZJQFqQ01GGfukN4Un0n5aJr+wpoeAjB3ncoonhI/tN/3jrSlhkiJl2ZpU1MZ5qj1z7BgBPoGgI nw7Za9un6epfGMqGcJ4qCZ1tzo1+igAdyXEs+Qh10R2fyyK9lrivYTzbrTYcJnFH6EXNfxl9o3Ao nLbszxhT/w4Nx1OEKVxbGBCRFXd/lQFmKtMgzV3kYVT2HHwqsxQZ/hGp9GIEClBZvUFuBj7ADltT 1HWmfRUm4vSaluNH9+xO+nKCeDvgPpDK8+cMpelBYqWB7NFZ2a3JsvqNPULz9GaKMiq/CS2umjUC uYAnJ9Cwrmp8W51xnVkjysVCj0FLARd8yUP/WGs9/GDviCQS+bn5mM2aH4gNQ6x06aX6lNYBFOLl 9hCJ0QgneN8xQe7W8aJ0xsxClZpdj0ZCV7Pvm4wl1/gKgkwBxMr9I1yDCcRzVB6gfenuncz/noHb HFKWsZA5K0SOj7gF4o0SFzFixVvMQsDfAurlnpsRihyjvxOxhwURerpFp/DXxKXAeMWFLW3fTL4N mioTdp+/ZPFGf+gWFRtlwEMZAes1NPOzbduc8tHFVTUTa8nCLx6Rll0/rQ9jcatMqel8YMMWQhYL su0hTeHlUznTStVwKle+4ACkXGs8IXzkssMM+0tDY7XKKXhIvkb2AQ/7HxJuEiyjsqALoFSiGbB2 fSpw2JBUmFHmti0rem5nAbWppz4um8xcd+0NwU3v+EiUpmMqls5WYy0BXb5GuYoF/rn1AUmT77A7 gjST2DcGdXnFegIVBDK9jE+WwJpYBhHFvLYAGzEk+olwKUF03jrSjZ3ckaP6UcfdzH58EPhS0X3h RX99UdTvfNKI/7UNeonDTBQAGc02WlFiNQP1vMMa3cB5OXSkCFESDB2K2JP6RS0JuTt9HTwGREdq dOWthetPCVvMjR/RXIxBPGaK/UQCUwHNPGKhVh3hb1jp+gywzA39EsGnGUUwqE6oufgtWcfjRL8H lrLN+5ck6nn2X6NjmXkmwiD4gnwmy25I9xkbVULIC5YAIbHjSsc2ScaaU1z4bXaLu8yNh/gPPDIM ZD3Xsd4QwJaCVPqzlBe1MIBzV/ANjWzfq2hBP3s3unQfVzJR3snmc09V9PAPAtKlSeki2C3ewuaW aSBiQShVh52zAf2jxGKyQ8VUpiLW3kygZchde+GJEt3EwcvIR47lp6jOM3dIABjKmrTME5Xr+Uhw WCg6gPwn0XhpbTedCALNGKDMQianXaMrWXkzPqelufl1ydmdvpzqW/IBsgUs21/U0LCjXYOslEFO LgoT/y2/x03TuP4ASt4QOUy4G3HsEnIA4wYc3WhtjMnP7/tLsJpVKeS8xQR8QWj+2Im1ci3vXcsD XMbPrh7ogQuL11BYXYlxzzhO9c1cbuhkZ4iCW2J7z1jQ1xNvuFro3lsNAe1AVGmBMJ6in1jdIFcF 9SFn5Oo2Bpgs40ZzB0AhDm7Fa8lLrOEDu3Y2V2eap9ccrJxVhvePthAkLZJZ818llWIasvMptrmN icbJHXo5lR5p1+wYOz+xt0UXuImm1Mk2KHXrA7DwDsK10M6Mi3A/Phq2GS3iEChgnW78qLDT1fyR VOvdFJ/Rk/sJSMMdzpXhZgS2vX4g8XoOrSdvnbjunnKPK8P/8skeMyg2+fjVAXh0BmdZbjiC9L2G 3YY7JJO/jGxMAdJVUtfHVgiD9amez00DCYGS2wes37Rnzqcp6IkxxUEtZ84lbr/Z4aC9OhAVBeBq 4t4tmUKCuHJr1Bp2NYNIr5SuAe/w5CxgIMZKNWQcQ6mJ6zfy0OCiAYrh46gyJb0gXMIHVUOUyw18 CqFB+ku78ZQZDOxuagkfpyUMIMXMaqw5Ht/2xRXHsXja+VYlVelx62njAV5aOr/1VdFcQwZdR/B6 GzNaYGA+TUY8LTT6XdQpA6mncbB9ipl4yKO8dRwWVT2rjKTJp0gxzAK5wj++zG1+ygTQsHbVmu4h lDsRAaQ1dzT7H4JgFi99/8+uQ6np2TdFBt0LvvsGSs3tvGWTvErbVHw6hwHbXJuQ1hAsNHUvQ1fa BC95y7m0g/uG1lIavAUhsrI5hl1/tLmun/QOQHq/+xkMxccWO7AKMTxxJFy2SzhpIGJ2IJH5sWHS jUZPp/DLTYleEdk24V45gGLyKczlyGE1tPzGYRD9k36IZMxj5woLZXAkmvk3CTo0hv9BGoLQ4HRy T5c/Ak8iyFtg5yK1K9aFPoa/rS/wnOBzYSgQMuOFROqbIRCSwmTwoYoi96jFv8tbHDf2sgmbBFU5 VplYHD3FSG15CBi6gncgl6CwFPHPp96NA3/q0PuuqrvCkS0r3jUOF2fHua1a1iLwgKzsxdqyj+oA zdLBkiU5DmikQRrUJCSMgYSfGfekWP7cqtHf+mFrPj6U1QxNAl5gUeIb1KNXxrf12OrsFOoNBihq 650U091GGX0X6V8pSryDPKix9vmID+Ezpr9VpLyeH8FD3SB7xYSMHkSkN/DgAVlo0lVltA1kEbwU VhVT2Qqpxbhpf89VGQE2Hm+UliltZfvy15ZKxHlvOiXlBiVNy2KP/jYvN+rWNwwD5zbQQNE6Y0Ix ULRPqtgOUE7h1A9KXIqsq1o/SjZFr7YKbiqEiJ2+LywDKARMI7/YicdYyTF556a1NgKlilj5vMsh KuHqySuMMOqJR3FVs9J5CmjSO0IazkmmilhYzYaMVxhlglJVNKF6ayC3uxgh5coZD3bF4vp8UpD6 cxZJ0HV4hXepXFp4JKdkkOFmVLI8IXonKz/Z/tcG2oqFz7I2amx7ZFip4aBp0lpl8bZIkB43taYu JcEak40C6In2+34OaE9XsfIN9S6MmsidSL4TvadTxROYLpNha9/cQ6MfloOivA8CPfWKsczm7G7z UJb9dIVanDJXyMy4t549s6U9QxQWolDf7/tLWR3JiqqnCRdbj4FNdDC8PfIFBnbnmHA5MnnfeyUH 5FRr9nmAvu0f56oJ6dkoSG5qSkdCO1riAH/Em63PGtTKYy825cxQ139Toudf6kwPBrYOAcwDMXtR b94/lwe7ZdVPDikVPnGBH6URcqbv8ExipYTYV5nWq1LGYr83IzD1oFCcNFHYN6ylVEoUSMDLVo7S 6PQqbTob8lEtoVTQ1oCvbsYIazL0KrZ/gMH0kK1R+451GuQ69WBfuyjSCDjtD46s3ZQSR/GPGMxp DMPDPWjFEZ2i4SQB21AjbvraYZ9L2SUB6rNRmLF4hwTeKbxr88L5YF5LKiUELx6WWkR0CmtkUQiT MSe29aAu/eKRiYdSxKJ1m4/7TrzWsomZvzdDPj11xoDagwjYgu4KOPNweH9+NpJZZOaTM7ljofKF 3Nd1sH0llTBB7wHI5aGRSfzVVit8SqAI0zP/SXN3Qqy40+rVbiL3iPwzFp0U7yaMCckUPoRuWU8H vah8ZEHNL6QG7oNJ4iIQlq4hF4Cw1bblBdfA1R/LwBeJtG7XAU4JRqFAX1lmht3RFUdLCrcEPCGg cRfCIGPtiGndH4otKavR1+gQFppQQEl8GQEtBkZZJSjZ3OP3Ot7FUNJB1nYDs97/Zx4aucFK3Tus 3Zp6JNKMa+h6FEyOXSdGGIWtcYbMF9kX4uw+73H8JDW1RLkoMJ2CWxxMdsW+MVjy3sv9dkeWY2jR GRbWywvZh7MHswZ8aZn87gD6huJFJqB4V16E+sdSVZPW+Bk4ich/He02cwoEeQbx16TpvariBUCh HiOWAsiFwg6Q/diKsjFcZngSMHfb3MkukobIS4JqKsZMGvudWRpPWIhFzMrbk06T042AaqAV+u4E YMkgRWHl9IEPUxaCTBxWW/1UfmlbFYWDQ3tsJ9omoc+1Az8l8CXTUa9TUz6LLbdiAp5y0ajA0ZnZ 2fTpO8XQnbzcITHu3q6eW1V1u4IoY8affmzHjfjWLxi25yxF7cc6Te0yDWbStcL4ndt54YdMFV2I pblvUi58qusgwiJ+tOkFQGiZ2/9XyRjUTwf678XS/vHCh2RpgSE0EmTdYZVjiTNC3aPPmtk8Igaw oBedmtNGiRoDD3nWtt9uyY7o2YAdOWVxCQNIMHugSoGzNRaTwpNm0D6jmv+1Set+TkBQqKaJ0EqU rNzDZEFrhPz0tMCRBu1JAtRdyN0t7vHRtQ10K4PcTj1b9FIdhYLoEwFJLCo1k1TCAekTew+mVZOn K7GsQT4LZ9MqE/3BJlCBm4QdFWkVSxHdu4qxvUB1xmz8xBqNyiK/ElE6HQvfxY9OFAK3QOG2HF5n pZ+HGffGfkrTiLYXr/aWvK9Il9+QxCNAZH84Elgoi30SX536v8dvYiOfh2ydn330+YlQIWq4KLTV BGLxh7A654xVlDCoUhPIpRy0C1bvM/hIPU19xq1rIdeCPkpRvDNWTkkDLSqcL83wO4nD6BxTCfTE Ei+iIk/MhVqDPYx0J3dco6cacKvXpY1x0cWgBpi1oz7xbfIv797OJja0VGzVr/3Mc3g1WEFTg42k EdlJ6zzeqSCshtnsA0A7dfQYRE1s0l7cj8I/JvL3LnaIaBPzfuJXL2rd2x4lKy9j0CFeUN714qoh /RRN83Xunx1wGVzwQZoX7Pj7WK247EYMn7lXP1tHR5WSajhpHrvvg0illCIKwpWtubZDKx5u89tS Jg98jnvR2JM3xL5G8DYxRdqBxzacguLYatKDOmQj6tuoQEfmE4pbHgQbg/TinLXIIRrJFzFW2L5W KTMxB8slgVmCJEDigKHA+a4t6xwF40/dJVO05q4ko1Ei+js+I3zrAB7d6a7+314Xj55RNkHqi5sD zjt7/GXvEtHixESpxYFxRGL26onfcoAYVxlqzRxPIaZVIsTHsmD60DCUZ5foZwKiayswtAHIHPix O3ZMmsjw1VDn0hKspxSn0ihhLlLLUyLwlREP5rBRQpfmLfIjxG5Aw01SRLIURJmLm36OeO9fSfUb 3Z7fZH/UC5lhFEIbQFKzSyh7XFp307xvsFk0fo5uVxxfUsevJo/pNFTXjtrg9F3ElpIROmuLRLNQ CLnGJ5qvQrjZWBvVEd3NLE1gg2ceTVs9r2mljJj/+9QgHuINUxaUiDMVzqaKB1sfTwgL34Jpusjg nwXqkOu1Rer0k/j38T9XK3uJW0M9Lom/96uHQiIBDgzSz9yIP1HhjuXgU4/jfWHcM96Oi0OS/SZW IQdmav+PjactDydtjnZoZzsJDrhe6vuWlaCoEDdw5pAg7Aa6Yfpw7aewkEGf3pkYSs2MAJQVAPEI fqAdCZjuRyBt3O4pLC8b0Wo5L0Zwt5DuegCX4KoZoDoaPjjstHSWDSNag+B2D6+4rFNXNfrEXwdn vBfdUygOZokwcpFLu6n5Nw11in8/JI0KGiCLvwNRkug2fNpfkjuPk8AJC6L3jvuRAb+JtdyRZ5IV oLa+UgOEmPmb9A+VleeNGeQJnDqXneeXAaxep85CLI4ReQtk4gTPEe5LBxUnqaMGe4IarF6zFvTA SxvJfvoLkaGcJu9U79GGU3IXqYFGjFGjisvC7cH5q4IMuQadbBbtiyUxXiik1h2nRN9SluHBBpup mP37rNAPGdFoSDlF+MWZV+2ncgKtRDGr4w652bRaFsupWdbOhaTrzsyY7wjmZTSK73s2n9I6tG6T GzAPvOdQLe0w3qe60OOksEhVegcQHRxStUMxT6e51P5fI0NgiCB323KYIcqTIEX/3UEZ4pu1JPoS hE9Q9QxB5XALwOVAbgTgcB0Rypa1X2IeE6faBBu9jtggQZMoqPa8eUBo0yEgktaUghaAwsNPntcJ du8xmOMEi6iCOp3N0AbJ9OYPTOv2m5EZFYj19cPjvpwqN7YHD7b0+rjkBlYbNDbAupMoScOG7NMs jiWu+dEHI34DNsMTEH4Rsiw+kA62wiq1DowuT3Q8wZkdTJ9WwcLT6gPeoFX3aS3o+B8e8ECh9QJS a1qnroFA1wgnWyV+iz+Tl/64zi8zBuF609zP+E2FAWuffTdxNt2A7ZXCWhR7yKbFLUB4MZ+KUArQ gTn8th/p5aUltqOsbryxAGsJrSQlBXFPTDNWwfM45+wWhbkvSA+u/7UiMcrp7uZJSwK+NMDt/bJ1 314AzjOUmLyMjla0jGcp1D43KJkGTUYNbbhi0BcE3pmpsqevCJO45t7iOOsxJPKlprgw5vsav4sn UhOzIgJKLA5kpQ8VljUdhY7aBWGQQe6jJ3jw37s6Mev2Xqx3trDX1pmjs9n5WY9/4eHwArUHYLAh 9r+xQa9lg0iXcuBb+AHbvL8qfOUOTjIj4KVE2bZ4ryyEITFsAHkuwbpHsd+HuxI4Fdf3F9zFt6/x GDZByU+kw+DwkLPvPPwxHw4CW5NmG4PZaTmmQ5u2THZrh7QgUkXZBnwjxlIjM0tzUts0V/PdsMt4 J5ofwaxQjGwQotqD1ZspTPOL9FVoXSKcuZ/bbjamIkPZpVZQGJUBi8yFT4CFYBeeXVds/pT8CnSR SibXxEqe839DlKwEP+yLs9h7tVtO3pXfFe/O9J2ghcUf6v76J9lLIX0UckO3iFvth6tGjv16CVRu lvolB9uMzyOq/3dMW5Ux/dnQUXN5FsZUiHPAHbqeW+VVdtr5WhXjJXYLSJ31fj0mBL21judEukoh gG648HNMmxjEA54wzhRxOvGN9rPDBIlkI4SFOoPCGlVAk8TMtPdJM7RLjwyahlcCCweB5GmHIXjR OSBESDgTifbrZ1/ghoKzanbSL/E+Z3G3zZJEr5yTDSURroWq1nddrX/Uoahu2VxamPn85hL+lJoc dMD2SG6W7Wl3jW/fxMAsBW1y+Wvj8bn7ZNQCb9pkz8Mwur7Qn5yNdTv3dBsEITjavpkkJ0ov0cOr wvt3W59XH3WMhp6jya7wZ7u0pAswGSlmYbDPQWWlZcBSG6MNtyWbkL86OLZERvKPX94d2ciQ99Ao hUhqDJ0tPDNnlxUefkzW8/n3kgumfCaj/G7FM3o4qbXWkdLuV44pewnxrI2VP4XFhozfaR/R8oPe 1+STtbFYU4dEc5qKpp4k3rpH9zPo5WkScMf6U5ok0sYosUKGZYxQQhqGwY5on3wUQVsOsEojYrfA 5xxP5pS6/yR4frK9DaTTCpJFXdReHW0sTzSzOrnr3lcOtRugbnE0s21y1GV9COjxZvDhi8ChePem EeJjUmo6gQJ0uL/bbxyIuwnAvQp2ZBvpGpH233mHxyf25dOQ8UxEV7JcWYwQshSKCxOrYimz1j6n gXjoBTgtNAw3mkJ5vG8yTlHlXmdyDFRfXtigKrLbW1PxTd5og+nLO1t/82LI7lL/WEcE6/C+0sWo W7OSezkm2ZgklpTRgKerHbNhkFdTN9jjRYrGjWNtLTNQ/1BiR7BxG6txnxfGDe7/xWQ/d53r780U qWZYofoH+m9AmEQ6FXT5sfyd7PALD6SYf5Kckt5nZ7n/s/OOriX1HMqFqARt/geZpY1ti44Nf2W7 gG90QkWwELQxpTlCjoshsYnXfxydM+YSz1kuSHZ0kgudGZ5289jH5SYLhx03Kc15joBUwkzYdzeT BVD/fbXOMeb9ir+Ba8b5FoWs6lrYUKiB7/giiTSHZWENSBXaRl8wavKmTFJzdFqm74BCYEDUoxmU IRwF4sS5oRTDL4MpJ4DIZv0mB7pq/5/f5WwywQNDi6zGuIxsDfsFEs3ctzmkBWzACVtre9UFGgBk 10Vtdc15/PtwF0rQTobxaISTvzws+S7WrcAFnkdUFYCnB8B3iUKh/EDw3wuPG9AEgLpliKRub1Cq HQP3sGNibKvsAhi02BlqsUQiwPy5xtWKoPxv3caiqLBf1MnGjv80v8Lv8lhujmSX6h8vM69d8c3w 5YNwyN7hyC/i65CTxzwNTWdkXlk88eewDo+NZe7FXRKStjVDBMaOuHqiDUhy7c4M5KeGlAR55UbM pR565UPWQLLdtoVYPicHs0yBt7mpb87BfloOxlijJaE5AraEtKj9+Y7cOP6EmzdY2fFw+wA4MXSk auHeT3bitjMoLvgjDFvxuVe+R7cYkE+GZfe3W1vRseSCj2aDZqOfY71vVkQjdcqjMIxb7cUGPf1w 9Ji+tehLFrnIsAVaqH9/zovg2F5PAylqGQ6bKxFb7hz76K2ArRWQ9rNoJQKexmb8UrraW6wKgxjZ 5tKA9NU0P3V2n7aLYJZ+EENhlVtQGj4LQPa/NbcQQe92dAc/qkl628iBynBUPabVbs2aUFDnBCEN 9wR9O4Gw3KV2YU0TslVPKrEP8X2YhDM5+P7cwTkfFPn2JlDeY/i8nFZnH6OoxPlChHPH3g/m+0fg P5aJIksfE6iePi3r2Dqy9nSLbOVVX/mEGSkb6nTdW9qQ3sWNmlJ6rKMSnoj4Bb49nD1AtV3v6ZEt SMk8/Pb8HY5jtHXJtKY5oh9YWFOgkhZXJBZHXPiC8CY22rfJJGqL+8R8rTuL157YWjSzh9hi68BW nMtTJjob94aLjcZFkA4adBWFTxRYleWYh6zxw209x9uxLAG1aBiHK/K1Mhaw75XTtM0biz4isa6o wMucFWv79IIRExWQKd/mcmyMgV+KeUmQbJ8LQCoKWnmd6w75yFKRpajlO1G3kTBf61Q1afQSdIM6 mk890p3UTCtnaPFdQYBgWK+y/Nabmbi0l73x3ajKHmTDRVTqt5jrQ9Ga3WQW+6CGKL03Q9MY3o3R 6Yk/VaBuTbibRNPrs9g4YCA0npnBTkeFKLF+z+E/rtRfhDovIr3ZnWbQekUtxDURBrrBd+ovufgA aMw/Mjqz+yYrp91OqKqdymOAAu0AAhyVsf6w15rl+p8948/kSbOyJiRfHyo+lgM9sAEz/SH9tAGF +1aFm1CWiN/77lPuMvbrS9fWV1QITzU7kzRMYx+eC+mV1CovOGjIxgGfQ9RPbUDkRIuKiNkQouLa 09tN7Zx2RxOVrcioAnr9VD/FBlmFNFnqCj3KyKJX5zJnePQ8VoB78huSLkI6p/mmLMvfjc+40mfK pp3J8AJcYDcp1LnGwwfwIGQTKTeCUL3SbO1p0XcSdy1Aa7sb4c3DybQSky1FAg5xN1CUZV4k6CjH 9dkghXQ4Ol8WeRsqz25YrAdry/ou7cQAAABCdYdMg1WMMQABp4EBgNAF3CER47HEZ/sCAAAAAARZ Wg== ------=_20140403131819_44728-- From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 02:34:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65DBC47C; Fri, 4 Apr 2014 02:34:01 +0000 (UTC) Received: from feynman.konjz.org (feynman.konjz.org [64.147.119.39]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1150FC79; Fri, 4 Apr 2014 02:34:00 +0000 (UTC) Received: from 127.0.0.1 (tor.t-3.net [64.113.32.29]) (authenticated bits=0) by feynman.konjz.org (8.14.7/8.14.4) with ESMTP id s342J8bq075512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 3 Apr 2014 22:19:23 -0400 (EDT) (envelope-from george@ceetonetechnology.com) Message-ID: <533E1699.5050901@ceetonetechnology.com> Date: Thu, 03 Apr 2014 22:19:05 -0400 From: George Rosamond MIME-Version: 1.0 To: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org Subject: net.inet.ip.random_id Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 02:34:01 -0000 Sorry for the cross-post, but assume this is relevant to both lists. It's regarding the default setting of net.inet.ip.random_id to 0 This disclosure has caused a bit of a stir on the Tor-relays list starting with this post: https://lists.torproject.org/pipermail/tor-relays/2014-March/004199.html FreeBSD is one of the OS's not implementing random IP IDs by default. I know OpenBSD has for a long while. I vaguely remember someone referring to some specific application compatibility with IP ID randomness in the distant past... Or should it just default to '1'? g From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 03:02:42 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 492258BD for ; Fri, 4 Apr 2014 03:02:42 +0000 (UTC) Received: from os-mail-2.tamu.edu (os-mail-2.tamu.edu [165.91.23.189]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.tamu.edu", Issuer "InCommon Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCD5BF59 for ; Fri, 4 Apr 2014 03:02:41 +0000 (UTC) X-TAMU-Auth-ID: daved X-TAMU-SenderIP: 128.194.199.23 X-HAT: SG None, P $RELAY, L incoming_auth X-SRBS: None X-EXTLoop1: 128.194.199.23 X-IronPort-AV: E=Sophos;i="4.97,792,1389765600"; d="p7s'?scan'208";a="479725956" Content-Type: multipart/signed; boundary="Apple-Mail=_D524DF0E-4849-426E-8D0A-CF394FCA4E75"; protocol="application/pkcs7-signature"; micalg=sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: questions about (system) dhclient From: Dave Duchscher In-Reply-To: <533ADB41.8000002@rcn.com> Date: Thu, 3 Apr 2014 22:01:31 -0500 Message-Id: References: <533ADB41.8000002@rcn.com> To: Robert Huff X-Mailer: Apple Mail (2.1874) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 03:02:42 -0000 --Apple-Mail=_D524DF0E-4849-426E-8D0A-CF394FCA4E75 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 1, 2014, at 10:29 AM, Robert Huff wrote: >=20 > Synopsis of my (apparent) problem: DISCOVER, OFFER, REQUEST, > and ACKNOWLEDGEMENT all happen correctly ... but the information > doesn't make it to ifconfig or the routing table. Have you tried commenting everything out of dhclient.conf? I don=92t see any wrong but its sometimes useful return a system to its default state. To see what is happening with the /sbin/dhclient-script which does the actual configuration, you will need to modify it to output some debugging info. Add the following two lines to the start of the script. exec 1>>/tmp/dhclient.log 2>&1 set -x Then restart dhclient. Hopefully that output will give you some insight of what is going on. =97 DaveD --Apple-Mail=_D524DF0E-4849-426E-8D0A-CF394FCA4E75 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIO2DCCBIow ggNyoAMCAQICECf06hH0eobEbp27bqkXBwcwDQYJKoZIhvcNAQEFBQAwbzELMAkGA1UEBhMCU0Ux FDASBgNVBAoTC0FkZFRydXN0IEFCMSYwJAYDVQQLEx1BZGRUcnVzdCBFeHRlcm5hbCBUVFAgTmV0 d29yazEiMCAGA1UEAxMZQWRkVHJ1c3QgRXh0ZXJuYWwgQ0EgUm9vdDAeFw0wNTA2MDcwODA5MTBa Fw0yMDA1MzAxMDQ4MzhaMIGuMQswCQYDVQQGEwJVUzELMAkGA1UECBMCVVQxFzAVBgNVBAcTDlNh bHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxITAfBgNVBAsTGGh0 dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UEAxMtVVROLVVTRVJGaXJzdC1DbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIEVtYWlsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsjmF pPJ9q0E7YkY3rs3BYHW8OWX5ShpHornMSMxqmNVNNRm5pELlzkniii8efNIxB8dOtINknS4p1aJk xIW9hVE1eaROaJB7HHqkkqgX8pgV8pPMyaQylbsMTzC9mKALi+VuG6JG+ni8om+rWV6lL8/K2m2q L+usobNqqrcuZzWLeeEeaYji5kbNoKXqvgvOdjp6Dpvq/NonWz1zHyLmSGHGTPNpsaguG7bUMSAs vIKKjqQOpdeJQ/wWWq8dcdcRWdq6hw2v+vPhwvCkxWeM1tZUOt4KpLoDd7NlyP0e03RiqhjKaJMe oYV+9Udly/hNVyh00jT/MLbu9mIwFIws6wIDAQABo4HhMIHeMB8GA1UdIwQYMBaAFK29mHo0tCb3 +sQmVO8DveAky1QaMB0GA1UdDgQWBBSJgmd9xJ0mcABLtFBIfN49rgRufTAOBgNVHQ8BAf8EBAMC AQYwDwYDVR0TAQH/BAUwAwEB/zB7BgNVHR8EdDByMDigNqA0hjJodHRwOi8vY3JsLmNvbW9kb2Nh LmNvbS9BZGRUcnVzdEV4dGVybmFsQ0FSb290LmNybDA2oDSgMoYwaHR0cDovL2NybC5jb21vZG8u bmV0L0FkZFRydXN0RXh0ZXJuYWxDQVJvb3QuY3JsMA0GCSqGSIb3DQEBBQUAA4IBAQAZ2IkRbyis pgCi54fBm5AD236hEv0e8+LwAamUVEJrmgnEoG3XkJIEA2Z5Q3H8+G+v23ZF4jcaPd3kWQR4rBz0 g0bzes9bhHIt5UbBuhgRKfPLSXmHPLptBZ2kbWhPrXIUNqi5sf2/z3/wpGqUNVCPz4FtVbHdWTBK 322gnGQfSXzvNrv042n0+DmPWq1LhTq3Du3Tzw1EovsEv+QvcI4l+1pUBrPQxLxtjftzMizpm4Qk LdZ/kXpoAlAfDj9N6cz1u2fo3BwuO/xOzf4CjuOoEwqlJkRl6RDyTVKnrtw+ymsyXEFs/vVdoOr/ 0fqbhlhtPZZH5f4ulQTCAMyOofK7MIIE6jCCA9KgAwIBAgIQJ5o/CcavxpGv2uo3G++mbzANBgkq hkiG9w0BAQUFADCBrjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYDVQQHEw5TYWx0IExh a2UgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYDVQQLExhodHRwOi8v d3d3LnVzZXJ0cnVzdC5jb20xNjA0BgNVBAMTLVVUTi1VU0VSRmlyc3QtQ2xpZW50IEF1dGhlbnRp Y2F0aW9uIGFuZCBFbWFpbDAeFw0xMTAyMTEwMDAwMDBaFw0yMDA1MzAxMDQ4MzhaMGQxCzAJBgNV BAYTAlVTMRIwEAYDVQQKEwlJbnRlcm5ldDIxETAPBgNVBAsTCEluQ29tbW9uMS4wLAYDVQQDEyVJ bkNvbW1vbiBTdGFuZGFyZCBBc3N1cmFuY2UgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEA0+AVAnv5doLp/pGW8/AeIWi1DWEwvgmXZt0dI6f7H4Z4OuaXasfBpH+JTbOe WbM4Cx85ImsCKBK/KZbRNfSVKmwGmBwZZFymfkVuIGuRuHQPPFPIgxlrEaPHLIUFHofUy6AIBQYu Zok9eKeDk1S32Bfq0+59ZtQ03onKGf03ytCD3lFo2Fr1dOvdggJa+iDOqs9AK9PInoXpgTOb1vFP kUXZjTMpHF0HmoXp/8kSskwMQirtMPTX3Jm1zwuA6nnepyNin7XplqLWshpF0NgTHZJ59ISBPbYV j+5vLBrG7F2sk48LR006OwnV3gYiOhGVkG9Oex9yLieRMz6IV8K/CQIDAQABo4IBSzCCAUcwHwYD VR0jBBgwFoAUiYJnfcSdJnAAS7RQSHzePa4Ebn0wHQYDVR0OBBYEFOjXvZaq3dAI76Eznl5ZmDwS t5uRMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/AgEAMBEGA1UdIAQKMAgwBgYEVR0g ADBYBgNVHR8EUTBPME2gS6BJhkdodHRwOi8vY3JsLnVzZXJ0cnVzdC5jb20vVVROLVVTRVJGaXJz dC1DbGllbnRBdXRoZW50aWNhdGlvbmFuZEVtYWlsLmNybDB0BggrBgEFBQcBAQRoMGYwPQYIKwYB BQUHMAKGMWh0dHA6Ly9jcnQudXNlcnRydXN0LmNvbS9VVE5BZGRUcnVzdENsaWVudF9DQS5jcnQw JQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnVzZXJ0cnVzdC5jb20wDQYJKoZIhvcNAQEFBQADggEB AA7ttuA4nzpoogHB8ITlJu78gdLXjMkbJEOgybGF5p/k4CrRRN7yoHCtiBgoSJYTEkdJ+co5o1fX zVOFTbf3Lx/Zq1N6aQdOyEK/4uxeO3v0PrKkSZaRFnv5T2peQnIx0dNDoTkFiT7LtdHpPhHGhHXO Nb/1Leiiq/WkAU5wHCZlqxbDvTLmKRAva2qySNZwBgb1KOhUfswQxWmuEekF6zs4wWJa0B4Y6VxN U/j1KuHdkjWE+iWPH8cjWRr4oiWlVmLi3jis1N8zV2IBOCpRKjtMuUewcenJIcFhjtJjrbpJKyIA rDY1EEgOXsHsHbHbYl6TA+rlUEjvUashivtgxcIwggVYMIIEQKADAgECAhEAkBk77adN1ykDnrpD b/1TqDANBgkqhkiG9w0BAQUFADBkMQswCQYDVQQGEwJVUzESMBAGA1UEChMJSW50ZXJuZXQyMREw DwYDVQQLEwhJbkNvbW1vbjEuMCwGA1UEAxMlSW5Db21tb24gU3RhbmRhcmQgQXNzdXJhbmNlIENs aWVudCBDQTAeFw0xMTA3MjIwMDAwMDBaFw0xNjA3MjEyMzU5NTlaMHgxETAPBgNVBAoTCFRBTVUt RU5DMR0wGwYDVQQJFBRUZXhhcyBBJk0gVW5pdmVyc2l0eTELMAkGA1UEBhMCVVMxGDAWBgNVBAMT D0RhdmlkIER1Y2hzY2hlcjEdMBsGCSqGSIb3DQEJARYOZGF2ZWRAdGFtdS5lZHUwggEiMA0GCSqG SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCE2CTSrO/PJr+XWeE/zAwd5QkIBbNg77hAImBmFsq9iiIi JY9zAbrXA0XHbUKh/tR1nTcJgg9B6vr/gbIAakO2seJVXwYOFrmSlZDF+7uI+8ugQNAstDzH98DI JJlH2qGMKFkukgCvtyga6wcaafxxXdktNoPlP4JHaUv3FfO9d7pAYORxH9wsvTX/kdbKXjMSSF9C K2zeGBWXobi9HalRHXjeVGyvtigk/TkF6Y6XwPzuqvMuAUR2F0gGTepUaMppfHVSFCdo0L3myu9b JJY0VvjsQLgbgtIeYTGw9FHpAvUD+NvBUm2IP09NVMvPyCGkfX3ROTiTgO82yfCmBSDRAgMBAAGj ggHvMIIB6zAfBgNVHSMEGDAWgBTo172Wqt3QCO+hM55eWZg8ErebkTAdBgNVHQ4EFgQUMrJ3IU0V oDB9a0RtN20wvYam0eUwDgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYI KwYBBQUHAwQGCCsGAQUFBwMCMGoGA1UdIARjMGEwXwYNKwYBBAGuIwEEAwMAATBOMEwGCCsGAQUF BwIBFkBodHRwczovL3d3dy5pbmNvbW1vbi5vcmcvY2VydC9yZXBvc2l0b3J5L2Nwc19zdGFuZGFy ZF9jbGllbnQucGRmME4GA1UdHwRHMEUwQ6BBoD+GPWh0dHA6Ly9jcmwuaW5jb21tb24ub3JnL0lu Q29tbW9uU3RhbmRhcmRBc3N1cmFuY2VDbGllbnRDQS5jcmwwgYAGCCsGAQUFBwEBBHQwcjBKBggr BgEFBQcwAoY+aHR0cDovL2NlcnQuaW5jb21tb24ub3JnL0luQ29tbW9uU3RhbmRhcmRBc3N1cmFu Y2VDbGllbnRDQS5jcnQwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmluY29tbW9uLm9yZzAtBgNV HREEJjAkgQ5kYXZlZEB0YW11LmVkdYESZGF2ZWRAbmV0LnRhbXUuZWR1MA0GCSqGSIb3DQEBBQUA A4IBAQAi5uRmXDzXPU/OmCpkk5Qpf0sLynUMVrOMi8UXyVJO2ZzSL5jtPW2brDKAZGPjvn8waZ2W fbxQ9+uRBcAAdYy02YAa06Y3FPIA2TXqI1gGJoZvNsQED9ltJ4MWXb/XKsY/GKNcPh5dZcAMWrvE xWN2/FomudJMOTY+FVwgMr2DMWckTWzsr70wW1Hf1OEBI0x4bd8vHfd4rthl2BJ8JL1mUqxTFW+v Y8X47m7IXZA0wnCUPUJOGoDz5izC5mGnwyDwOK5QqAzJyZba3P1h0a1OoCXb4Dsq+N4xrGUQi0Ld v3s+xybTPnW4gmeea4PM8XyJ4tyBKBMBIe9hP4Bp3Zs0MYIDGTCCAxUCAQEweTBkMQswCQYDVQQG EwJVUzESMBAGA1UEChMJSW50ZXJuZXQyMREwDwYDVQQLEwhJbkNvbW1vbjEuMCwGA1UEAxMlSW5D b21tb24gU3RhbmRhcmQgQXNzdXJhbmNlIENsaWVudCBDQQIRAJAZO+2nTdcpA566Q2/9U6gwCQYF Kw4DAhoFAKCCAXUwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQw NDA0MDMwMTMyWjAjBgkqhkiG9w0BCQQxFgQUZR3AmaXd2QHrL2LCbaJKuro2F30wgYgGCSsGAQQB gjcQBDF7MHkwZDELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUludGVybmV0MjERMA8GA1UECxMISW5D b21tb24xLjAsBgNVBAMTJUluQ29tbW9uIFN0YW5kYXJkIEFzc3VyYW5jZSBDbGllbnQgQ0ECEQCQ GTvtp03XKQOeukNv/VOoMIGKBgsqhkiG9w0BCRACCzF7oHkwZDELMAkGA1UEBhMCVVMxEjAQBgNV BAoTCUludGVybmV0MjERMA8GA1UECxMISW5Db21tb24xLjAsBgNVBAMTJUluQ29tbW9uIFN0YW5k YXJkIEFzc3VyYW5jZSBDbGllbnQgQ0ECEQCQGTvtp03XKQOeukNv/VOoMA0GCSqGSIb3DQEBAQUA BIIBAEC0rfboj0HaA248QocWWXEKwlJR+QE5ho6oT3inGdEUc5EkM+I3HTLUg4RXwfxnIAH/d0t3 YJVUKuu0udDwdJTZ2HUTNO3hgTiUFcyMgu5OO6pqadMgZVB6zDDVOHonxmWb8lQywzVpNS9b7qdI SIBlX6fki1oMBBshvCa1iv2le6DzEfuGRQQmRuiuGWnDrVEQ/0WqAAX9yvCb2ECgZGOQEMQ08Mad uRA9C652aH55CUXEJMPFQ8k78jxgdtx0u1hry44upnuXHRIKvyiIjvnCnTZw3fwAx+shLghWKcYa VRgjXnFLB62ALPXB/mOi7O6Ap2pwElyMYC71wphzE+IAAAAAAAA= --Apple-Mail=_D524DF0E-4849-426E-8D0A-CF394FCA4E75-- From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 04:54:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B5656F8 for ; Fri, 4 Apr 2014 04:54:55 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC5BDA3E for ; Fri, 4 Apr 2014 04:54:54 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id n12so2870458wgh.12 for ; Thu, 03 Apr 2014 21:54:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=syPvgC64pa0YOCQCjqc8i6kstQtYdaOaa4zRkZbuF/o=; b=SWvNdp+J/VAdwFXuYodRrW0mW+2yJpobTcqlOo8GdIR8dWdgXXgS5dbsEwLWZLmgOe CUjtT5g65Li29vB8Z9t0nVzcK5tMHCUCS9go2y60kZGfnMyrF8G0R1Wqphw0wKfaO11Y rJzFBINOeiodegnIpjQ6w6RzP1fRxYAUXGZoDCzjTskeg/Chx++soLtTIxitn8WFWg4z uFgcUTW3LeyjbZXUrOG2cRQcF9V6m5YcpYG9WjFxATJF+cUIVdnBjC+5y9EBPp7l8nNQ QvvJhiYG8tcSuOFczVPtwx+UZdD6oh0NE3+I6jEH/c0zd1+jlgqqklEdzrVSwQRXWjCJ A2XQ== MIME-Version: 1.0 X-Received: by 10.180.81.228 with SMTP id d4mr1064372wiy.49.1396587292987; Thu, 03 Apr 2014 21:54:52 -0700 (PDT) Received: by 10.216.202.1 with HTTP; Thu, 3 Apr 2014 21:54:52 -0700 (PDT) Date: Fri, 4 Apr 2014 07:54:52 +0300 Message-ID: Subject: netisr 0 : %100 and other netisr threads are waiting From: =?ISO-8859-1?Q?=D6zkan_KIRIK?= To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 04:54:55 -0000 Hi, I am trying to use suricata on FreeBSD 10 amd64. FreeBSD behaves as a VLAN router and NAT Box. Traffic is about 400Mbps. When i diverted traffic to suricata, swi: netisr 0 thread gets %100 cpu. other netisr threads are %0. And Even I remove the divert rule, netisr still eats %100 cpu. I think that something looping :) And after 1-2 minutes, one of igb0 and igb1 stops working. Only reboot solves problem. Hardware has 8 cores, 24GB Ram My loader.conf : hw.igb.txd="4096" hw.igb.rxd="4096" hw.igb.rx_process_limit=1024 hw.igb.num_queues=3 net.isr.maxthreads=3 net.isr.bindthreads=1 net.isr.defaultqlimit=4096 net.isr.maxqlimit=20480 net.link.ifqmaxlen=10240 How can I debug this situation? Any suggestions? Best regards From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 08:13:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7958AC1 for ; Fri, 4 Apr 2014 08:13:19 +0000 (UTC) Received: from mailstore06.sysedata.no (b.mail.tornado.no [195.159.29.130]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C493C50 for ; Fri, 4 Apr 2014 08:13:18 +0000 (UTC) Received: from [195.159.29.130] (helo=www.eposttjener.no) by mailstore06.sysedata.no with esmtpa (Exim 4.71) (envelope-from ) id 1WVyyw-0003da-FP for freebsd-net@freebsd.org; Fri, 04 Apr 2014 09:56:10 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 04 Apr 2014 09:56:10 +0200 From: Daniel Engberg To: freebsd-net@freebsd.org Subject: ALTQ patch for the octe =?UTF-8?Q?driver=3F?= Message-ID: <1ef23ad226c956b755938db70e9fa9e8@pyret.net> X-Sender: daniel.engberg.lists@pyret.net User-Agent: Roundcube Webmail/0.9.4 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 08:13:19 -0000 Hi, I've pinged -mips about this earlier but no response unfortunately, a few devs on irc have raised interest but ENOTIME. Anyhow, as I don't have the knowledge so is there someone that can whip up a patch enabling ALTQ? I'm willing to test and give feedback if such a patch is provided. As far as I can tell it's just a few lines http://people.freebsd.org/~mlaier/ALTQ_driver/ but I'm not sure how up to date these patches are. Best regards, Daniel From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:37:59 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE15D6D0; Fri, 4 Apr 2014 09:37:59 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A34073D7; Fri, 4 Apr 2014 09:37:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s349bxJT042405; Fri, 4 Apr 2014 09:37:59 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s349bxXI042404; Fri, 4 Apr 2014 09:37:59 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 09:37:59 GMT Message-Id: <201404040937.s349bxXI042404@freefall.freebsd.org> To: ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/65616: IPSEC can't detunnel GRE packets after real ESP encryption X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:37:59 -0000 Synopsis: IPSEC can't detunnel GRE packets after real ESP encryption Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Fri Apr 4 09:37:42 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=65616 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:41:30 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6729D77E; Fri, 4 Apr 2014 09:41:30 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39EF03EC; Fri, 4 Apr 2014 09:41:30 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s349fUCs045123; Fri, 4 Apr 2014 09:41:30 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s349fU69045122; Fri, 4 Apr 2014 09:41:30 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 09:41:30 GMT Message-Id: <201404040941.s349fU69045122@freefall.freebsd.org> To: ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/147894: [ipsec] IPv6-in-IPv4 does not work inside an ESP-only IPsec tunnel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:41:30 -0000 Synopsis: [ipsec] IPv6-in-IPv4 does not work inside an ESP-only IPsec tunnel Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Fri Apr 4 09:41:15 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=147894 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:45:48 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF6B292A; Fri, 4 Apr 2014 09:45:48 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A478766D; Fri, 4 Apr 2014 09:45:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s349jmmc045286; Fri, 4 Apr 2014 09:45:48 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s349jm5q045285; Fri, 4 Apr 2014 09:45:48 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 09:45:48 GMT Message-Id: <201404040945.s349jm5q045285@freefall.freebsd.org> To: ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/143208: [ipsec] [gif] IPSec over gif interface not working X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:45:48 -0000 Synopsis: [ipsec] [gif] IPSec over gif interface not working Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Fri Apr 4 09:45:33 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=143208 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:50:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D724AF93; Fri, 4 Apr 2014 09:50:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ACC046D7; Fri, 4 Apr 2014 09:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s349o1wb045500; Fri, 4 Apr 2014 09:50:01 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s349o1i1045499; Fri, 4 Apr 2014 09:50:01 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 09:50:01 GMT Message-Id: <201404040950.s349o1i1045499@freefall.freebsd.org> To: ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/143593: [ipsec] When using IPSec, tcpdump doesn't show outgoing packets on gif interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:50:01 -0000 Synopsis: [ipsec] When using IPSec, tcpdump doesn't show outgoing packets on gif interface Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Fri Apr 4 09:49:47 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=143593 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:51:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE5C51B6 for ; Fri, 4 Apr 2014 09:51:33 +0000 (UTC) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com [209.85.215.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24B877BA for ; Fri, 4 Apr 2014 09:51:32 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id pv20so2333656lab.9 for ; Fri, 04 Apr 2014 02:51:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=VOicdCnivV4ZwFLYV8uThhyaISgOqO4eBp6Q7wMMp3o=; b=OnyEQ8UfEOH8sf5QOs37d8XqvGMJGyl6bBlSqIqmSxEfO9bOktrUwXmMA1PamQXKn9 8YBdtc+yOxE+nrzNd5PRlSqY/tHqy58HtfJTpVUvgYWWmwlEzK2S5k1VSJV7fmd96ZS7 syZ2pQ1sWOCHSO+H0k1lc3dmXJpFU7u2myusgDwZ/PYaRWeXJZb30TsB5Az/UeKMHBXK bSAQjAKljsAawkp3+jB27uP9cjiTKlLNhbqpLV2fWIcWGk9I3WeGHqe9lwnLBf3B8rRv wU13+Vsgu07bT18o64mxPj/1BaUqzgoyHheFRnYk9Di10sinnltomXCF6XjogfeexPrN logg== X-Gm-Message-State: ALoCoQnU5qlVI0egLeiJjRMoB5NKN5lr784oIQjg6jcn9y+MYBPYC3BABqfBk+fTEeXq/qhKn27p X-Received: by 10.152.18.229 with SMTP id z5mr7962493lad.27.1396605084090; Fri, 04 Apr 2014 02:51:24 -0700 (PDT) Received: from grey.office.se.prisjakt.nu ([212.16.170.194]) by mx.google.com with ESMTPSA id j2sm7410456lag.12.2014.04.04.02.51.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 04 Apr 2014 02:51:23 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: LACP lagg and CARP - ENETDOWN (was: FreeBSD 10-STABLE and CARP states) From: mxb In-Reply-To: <1E20234E-4F81-4BA1-BA57-FADD80F782F4@alumni.chalmers.se> Date: Fri, 4 Apr 2014 11:51:28 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A818132-757F-4BAD-8137-CDB1F6F0681C@alumni.chalmers.se> <1E20234E-4F81-4BA1-BA57-FADD80F782F4@alumni.chalmers.se> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1874) Cc: freebsd-pf@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:51:33 -0000 According my own research around this problem ip_output() at line 839 of ip_carp.c returns ENETDOWN then lagg is = configured in LACP mode. On 2 apr 2014, at 15:13, mxb wrote: >=20 > OK, thanks everyone whom replayed. E.g. NONE. >=20 > The problem seems to be related to LACP trunking. > Disabling LACP and configuring trunk in =91loadbalance=92 mode puts = all in desired state (even after reboot). >=20 > lagg0: flags=3D8943 = metric 0 mtu 9000 > = options=3D8407bb > ether 00:25:90:e3:71:f2 > inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 > inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 > inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 > inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 > nd6 options=3D29 > media: Ethernet autoselect > status: active > carp: MASTER vhid 201 advbase 1 advskew 1 > carp: BACKUP vhid 202 advbase 5 advskew 100 > laggproto loadbalance lagghash l2,l3,l4 > laggport: ix1 flags=3D4 > laggport: ix0 flags=3D4 > vlan2: flags=3D8943 = metric 0 mtu 9000 > options=3D303 > ether 00:25:90:e3:71:f2 > inet 10.11.11.201 netmask 0xffffff00 broadcast 10.11.11.255 > inet6 fe80::225:90ff:fee3:71f2%vlan2 prefixlen 64 scopeid 0x6 > inet 10.11.12.203 netmask 0xffffff00 broadcast 10.11.12.255 vhid = 12 > nd6 options=3D29 > media: Ethernet autoselect > status: active > vlan: 2 parent interface: lagg0 > carp: BACKUP vhid 12 advbase 1 advskew 100 >=20 > //mxb >=20 > On 2 apr 2014, at 09:35, mxb wrote: >=20 >>=20 >> Moving this to freebsd-pf. >>=20 >> On 31 mar 2014, at 22:21, mxb wrote: >>=20 >>>=20 >>> Manually setting net.inet.carp.demotion brought BOTH VHIDs in = desired state. >>> pfsync bulk update seems to not put everything back as it should. >>>=20 >>> lagg0: flags=3D8943 = metric 0 mtu 9000 >>> = options=3D8407bb >>> ether 00:25:90:e3:71:f2 >>> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >>> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >>> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >>> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >>> nd6 options=3D29 >>> media: Ethernet autoselect >>> status: active >>> carp: MASTER vhid 201 advbase 1 advskew 1 >>> carp: BACKUP vhid 202 advbase 5 advskew 100 >>> laggproto lacp lagghash l2,l3,l4 >>> laggport: ix1 flags=3D1c >>> laggport: ix0 flags=3D1c >>>=20 >>>=20 >>> On 31 mar 2014, at 20:42, mxb wrote: >>>=20 >>>>=20 >>>> Hi list, >>>>=20 >>>> hopefully this is the right place to have my question regarding = CARP on 10-STABLE. >>>>=20 >>>> I have two nodes with following setup(node1): >>>>=20 >>>> lagg0: flags=3D8943 = metric 0 mtu 9000 >>>> = options=3D8407bb >>>> ether 00:25:90:e3:71:f2 >>>> inet 172.16.0.234 netmask 0xfffff800 broadcast 172.16.7.255 >>>> inet6 fe80::225:90ff:fee3:71f2%lagg0 prefixlen 64 scopeid 0x5 >>>> inet 172.16.0.231 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 201 >>>> inet 172.16.0.233 netmask 0xfffff800 broadcast 172.16.7.255 vhid = 202 >>>> nd6 options=3D29 >>>> media: Ethernet autoselect >>>> status: active >>>> carp: BACKUP vhid 201 advbase 1 advskew 1 >>>> carp: BACKUP vhid 202 advbase 5 advskew 100 >>>> laggproto lacp lagghash l2,l3,l4 >>>> laggport: ix1 flags=3D1c >>>> laggport: ix0 flags=3D1c >>>>=20 >>>> net.inet.carp.preempt=3D1 on both nodes. as well as PSYNC as this: >>>>=20 >>>> pfsync0: flags=3D41 metric 0 mtu 1500 >>>> pfsync: syncdev: vlan22 syncpeer: 10.22.22.2 maxupd: 128 defer: = off >>>>=20 >>>> The problem is (if it is not clear from the ifconfig-output for the = lagg0) the state of VHID 201. >>>> Node2 with advskew of 100 is currently MASTER, but it SHOULD NOT as = of setup. >>>>=20 >>>> Am I hitting a bug or doing something wrong? >>>>=20 >>>> I also have noted that after the pfsync bulk update the demotion = counter never setts to 0, but stays on 480, >>>> thus preventing node1 become a MASTER 201(?). Or is this a normal = behavior? >>>>=20 >>>> Regards, >>>> mxb >>>>=20 >>>>=20 >>>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 09:53:02 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 796333C4; Fri, 4 Apr 2014 09:53:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E39F7E2; Fri, 4 Apr 2014 09:53:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s349r2Us048120; Fri, 4 Apr 2014 09:53:02 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s349r2gF048119; Fri, 4 Apr 2014 09:53:02 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 09:53:02 GMT Message-Id: <201404040953.s349r2gF048119@freefall.freebsd.org> To: ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/174602: [gif] [ipsec] traceroute issue on gif tunnel with ipsec X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 09:53:02 -0000 Synopsis: [gif] [ipsec] traceroute issue on gif tunnel with ipsec Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Fri Apr 4 09:52:50 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=174602 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 10:20:53 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 74166E9B; Fri, 4 Apr 2014 10:20:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4923EA50; Fri, 4 Apr 2014 10:20:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s34AKrGk057759; Fri, 4 Apr 2014 10:20:53 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s34AKq1W057758; Fri, 4 Apr 2014 10:20:52 GMT (envelope-from ae) Date: Fri, 4 Apr 2014 10:20:52 GMT Message-Id: <201404041020.s34AKq1W057758@freefall.freebsd.org> To: dom@goodforbusiness.co.uk, ae@FreeBSD.org, freebsd-net@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/176484: [ipsec] [enc] [patch] panic: IPsec + enc(4); device name clash with CAM X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 10:20:53 -0000 Synopsis: [ipsec] [enc] [patch] panic: IPsec + enc(4); device name clash with CAM State-Changed-From-To: open->closed State-Changed-By: ae State-Changed-When: Fri Apr 4 10:20:27 UTC 2014 State-Changed-Why: Fixed with r247424. Thanks! http://www.freebsd.org/cgi/query-pr.cgi?pr=176484 From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 12:30:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF325D99 for ; Fri, 4 Apr 2014 12:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C1D63847 for ; Fri, 4 Apr 2014 12:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s34CU1b9094877 for ; Fri, 4 Apr 2014 12:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s34CU1sd094876; Fri, 4 Apr 2014 12:30:01 GMT (envelope-from gnats) Date: Fri, 4 Apr 2014 12:30:01 GMT Message-Id: <201404041230.s34CU1sd094876@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Eugene M. Zheganin" Subject: Re: kern/164475: [gre] gre misses RUNNING flag after a reboot X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: "Eugene M. Zheganin" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 12:30:02 -0000 The following reply was made to PR kern/164475; it has been noted by GNATS. From: "Eugene M. Zheganin" To: bug-followup@FreeBSD.org, eugene@zhegan.in Cc: Subject: Re: kern/164475: [gre] gre misses RUNNING flag after a reboot Date: Fri, 04 Apr 2014 18:27:29 +0600 Still reproducible on 10.x, including RELEASE and STABLE. From owner-freebsd-net@FreeBSD.ORG Fri Apr 4 20:07:56 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0DFEBA0 for ; Fri, 4 Apr 2014 20:07:56 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 58C0EBBC for ; Fri, 4 Apr 2014 20:07:56 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s34K7mck039768 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 4 Apr 2014 14:07:48 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s34K7mti039765 for ; Fri, 4 Apr 2014 14:07:48 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Fri, 4 Apr 2014 14:07:48 -0600 (MDT) From: Warren Block To: freebsd-net@FreeBSD.org Subject: Re: Gigabyte BIOS/UEFI and WOL In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Fri, 04 Apr 2014 14:07:48 -0600 (MDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 20:07:56 -0000 On Tue, 1 Apr 2014, Warren Block wrote: > On Tue, 1 Apr 2014, Warren Block wrote: > >> So far I've tried and failed to get a Gigabyte GA-Z68A-D3H-B3 to wake from >> the network. It had the most recent BIOS (F13), which did not have a WOL >> option. >> >> A UEFI BIOS is available, so I've installed that, and still failed to get >> it to wake up. There are a bewildering number of undocumented options, >> none of which mentions WOL. >> >> Adding an Intel PCI card made no difference. The card LEDs are on when the >> system is off, but it still doesn't wake up. >> >> Once manually started, the system works fine, and ifconfig shows WOL_MAGIC. >> >> Any suggestions on things to try? I'm open to going back to a normal >> BIOS... if it will let me. It runs fine either way. > > And of course I tried it one more time out of desperation and it worked. I'll > document the settings ...if they work again. It appears that the reason this did not work was a combination of changes. The card on the server pointing changed, so some of the problem was sending wake packets out on the wrong card. Changing settings in UEFI and then powering off the system does not leave the system ready to boot with WOL. It must be shut down from FreeBSD. For now, I've found on Gigabyte boards that very little is required to enable WOL for UEFI. In the BIOS Features screen, I have these settings: OS Type: Other OS Boot Mode Selection: Legacy ... (only or first) Storage Boot Option Control: Legacy First Other PCI Device ROM Priority: Legacy OpROM The last is only needed for an added Ethernet card rather than the built-in one. It does not matter if PXE boot is disabled. When an added Ethernet card is used, it does not matter if the built-in one is enabled or not. In fact, there does not seem to be a purposeful way to defeat WOL. The only way that looks to definitely work is to enable the ErP mode on the Power Saving screen. This shuts the system down into a very low-power mode, and (I think) disables standby power to the Ethernet so WOL packets will not be detected. Untested, though. From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 02:56:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C55F8677 for ; Sat, 5 Apr 2014 02:56:50 +0000 (UTC) Received: from smtp.webfaction.com (mail6.webfaction.com [74.55.86.74]) by mx1.freebsd.org (Postfix) with ESMTP id AAB02164 for ; Sat, 5 Apr 2014 02:56:49 +0000 (UTC) Received: from [10.71.101.130] (unknown [203.86.207.104]) by smtp.webfaction.com (Postfix) with ESMTP id 60038227E9BF for ; Sat, 5 Apr 2014 02:22:40 +0000 (UTC) Message-ID: <533F68EF.8060607@nevermind.co.nz> Date: Sat, 05 Apr 2014 15:22:39 +1300 From: Chris Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Multihomed system with jails routing issues Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 02:56:50 -0000 Hi All, I have a system with 1 network interface with 2 extra VLANs off it and I'm having some trouble getting the routing working correctly with it and jails. bge0 - management - 10.71.100.0/24 bge0.101 - LAN - 10.71.101.0/24 bge0.103 - DMZ - 10.71.101.0/24 Here's what I want to achieve... Host: I want the host system to only listen on one interface, bge0. I want NO ip addresses of the host on the vlan interfaces. The only service it will be exposing is its sshd. The management address for this system is 10.71.100.50. Jails: The system will also host a variety of jails, each with an IP either on the LAN or DMZ. I am using ezjail to manage the jails. Router: There is a router at the .254 address of every subnet that can route between each network. I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host does not have an address configured on bge0.101, I configured the jail address as /24 instead of the default /32. My issues: * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot communicate with the jail. * When the jail is up and 10.71.101.51/24 is active, SSHing from the LAN to the mgmt interface via the router fails, as the host tries to send return traffic via the bge0.101 interface, even though traffic arrived via the bge0 interface. So I did a whole lot of research for people having these apparently problems, and decided to try the multiple routing table/fib approach. So I recompiled my kernel, configured fib 1 with the LAN interface route (setfib route add 10.71.101.0/24 -iface bge0.101), set the jail fib and set the tunable net.addr_all_fibs = 0. I still can't get this working correctly. ezjail still seems to add the interface route to fib 0 by default (but it won't if i run ezjail with the setfib 1 command). Using FIB 1 and trying to ping hosts on the LAN gives an error like: sendto failed: invalid argument. Does anybody have any best practices for doing this, or anything else I can try? I'm happy to share/pastebin any configuration and I've tried most things I've found on the internet. I'm using FreeBSD 10.0 with a custom kernel for multiple routing tables. Thanks in advance! Chris. From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 07:02:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99A4BC6C for ; Sat, 5 Apr 2014 07:02:48 +0000 (UTC) Received: from mail-wg0-x22a.google.com (mail-wg0-x22a.google.com [IPv6:2a00:1450:400c:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 35143903 for ; Sat, 5 Apr 2014 07:02:48 +0000 (UTC) Received: by mail-wg0-f42.google.com with SMTP id y10so4380995wgg.13 for ; Sat, 05 Apr 2014 00:02:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=5ri169ykUNuR89LokUl+2aQJpEW9tcvSoaSzvtPpeRk=; b=Qu3YmTtRwk/jmaqYWE7RaKWxkzUqcIGcOlHRFEuv8Is/thzEJkLHZ0g1RyCZQPtHSM knER+W253NpKhE5BqlqMh1nh5w6eacUCZDGtw9qrkIjHwkn6Qu+tZlZiGW5Ywn4vsYTb KiMz0G0r/WINQSccZfY0ifEDqXhqsoQxOKcBmOfcqMAhMshsOXAGbiO8NBKEz2mSHh2n LcXqfxc9C/DLwm9+BJEATfuGJHcB9vsNO6+VXNYhGD59zXs9r2DdRlN2mVR7BXyix7AW zw6/j2XSryBg7ewp44fLlqFfS+pRag3sDkgJzETFA1UoCxN1o7/JJOMN5/Mu9gGlYwMy bpTg== MIME-Version: 1.0 X-Received: by 10.180.13.8 with SMTP id d8mr7929474wic.13.1396681366452; Sat, 05 Apr 2014 00:02:46 -0700 (PDT) Received: by 10.14.211.134 with HTTP; Sat, 5 Apr 2014 00:02:46 -0700 (PDT) In-Reply-To: References: Date: Sat, 5 Apr 2014 00:02:46 -0700 Message-ID: Subject: Re: netisr 0 : %100 and other netisr threads are waiting From: hiren panchasara To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 07:02:48 -0000 On Thu, Apr 3, 2014 at 9:54 PM, =C3=96zkan KIRIK wr= ote: > Hi, > > I am trying to use suricata on FreeBSD 10 amd64. > FreeBSD behaves as a VLAN router and NAT Box. > > Traffic is about 400Mbps. > When i diverted traffic to suricata, swi: netisr 0 thread gets %100 cpu. > other netisr threads are %0. And Even I remove the divert rule, netisr > still eats %100 cpu. I think that something looping :) To be clear, this happens only *after* you divert traffic to suricata, righ= t? > And after 1-2 minutes, one of igb0 and igb1 stops working. > Only reboot solves problem. > > Hardware has 8 cores, 24GB Ram > > My loader.conf : > > hw.igb.txd=3D"4096" > hw.igb.rxd=3D"4096" > hw.igb.rx_process_limit=3D1024 > hw.igb.num_queues=3D3 > net.isr.maxthreads=3D3 > net.isr.bindthreads=3D1 > net.isr.defaultqlimit=3D4096 > net.isr.maxqlimit=3D20480 > net.link.ifqmaxlen=3D10240 > > How can I debug this situation? > Any suggestions? I am not an expert here but please upload o/p for "sysctl net.isr" and "sysctl dev.igb" which would show error counters to get some idea about why igb0 or igb1 stops working. Whether we are running out of some resources or something else is going on. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 14:16:59 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F98CDCE for ; Sat, 5 Apr 2014 14:16:59 +0000 (UTC) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64CCA30D for ; Sat, 5 Apr 2014 14:16:58 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.0 cv=G8ee4qY5 c=1 sm=1 a=uNsD4W5u/UlQopoDAqU1YA==:17 a=fZBWQ0Qh6m4A:10 a=phkJtM7cgHoA:10 a=AaUjGI9IrlcA:10 a=IkcTkHD0fZMA:10 a=OA2lqS22AAAA:8 a=EYX-r1snQh2KckPJDw0A:9 a=QEXdDO2ut3YA:10 a=ZZAfTtC2Ym4A:10 a=uNsD4W5u/UlQopoDAqU1YA==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: cm9iZXJ0aHVmZkByY24uY29t Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=roberthuff@rcn.com; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=roberthuff@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=roberthuff; auth=pass (PLAIN) Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 209.6.39.223 is neither permitted nor denied by domain of rcn.com) Received: from [209.6.39.223] ([209.6.39.223:2545] helo=[10.0.0.3]) by smtp.rcn.com (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA id 76/7B-08709-4AB00435; Sat, 05 Apr 2014 09:56:52 -0400 Message-ID: <53400B8F.40402@rcn.com> Date: Sat, 05 Apr 2014 09:56:31 -0400 From: Robert Huff User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Dave Duchscher Subject: Re: questions about (system) dhclient Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Huff , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 14:16:59 -0000 Dave Duchscher writes: > Robert Huff wrote: > > Synopsis of my (apparent) problem: DISCOVER, OFFER, REQUEST, > and ACKNOWLEDGEMENT all happen correctly ... but the information > doesn't make it to ifconfig or the routing table. > Have you tried commenting everything out of dhclient.conf? I don't > see any wrong but its sometimes useful return a system to its > default state. Yes. Same result. > To see what is happening with the /sbin/dhclient-script which does > the actual configuration, you will need to modify it to output some > debugging info. Add the following two lines to the start of the > script. > > exec 1>>/tmp/dhclient.log 2>&1 > set -x When I do this, I get: execve (/sbin/dhclient-script, ...): Exec format error Hopefully, Robert Huff From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 14:33:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E360F208 for ; Sat, 5 Apr 2014 14:33:05 +0000 (UTC) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DAE6D876 for ; Sat, 5 Apr 2014 14:33:04 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id d1so2788573wiv.7 for ; Sat, 05 Apr 2014 07:33:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=avG8lxXLuVyOicjpoYIQ0ZKsiuqKo0Cptj/7roxBIFk=; b=zta4KFAl12oj9guMdwcj4cgX0vlCSnXstBY9NEP4zZRfduA0ZsuWSEhe0fP4Skkrak w3BQU8/nO+poAEieWKduqAHDoRhmxkRqOAANf7wVoDTYxS1wobZtk6xpQzGmq4ljCoUx fUOdl3P2oc2ZbYw4gaIG2wUqIw9cWmKgqfb1XrgvfAH4j4cltaaDdwA1ATPQsAYGt9zz yJRMRvjwKSPLi1hDtu+/jXIf6F7nOCQQ0hlqsBbrrzaScqcUvDcLnqHN7ZvOX24QgdgF qnHb1chJPT02/ASR03eb4vJYPuAYUPVqrCilV4MOIAbuoz0GG7/ofkThYT6SUS7sVp/I HUxg== MIME-Version: 1.0 X-Received: by 10.180.23.99 with SMTP id l3mr11776589wif.47.1396701946457; Sat, 05 Apr 2014 05:45:46 -0700 (PDT) Received: by 10.216.202.1 with HTTP; Sat, 5 Apr 2014 05:45:46 -0700 (PDT) In-Reply-To: References: Date: Sat, 5 Apr 2014 15:45:46 +0300 Message-ID: Subject: Re: netisr 0 : %100 and other netisr threads are waiting From: =?ISO-8859-1?Q?=D6zkan_KIRIK?= To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 14:33:06 -0000 hi, / I saw that netisr 0 ip has Queue Drops ( 94902 ). Thank you # sysctl net.isr net.isr.dispatch: direct net.isr.maxthreads: 3 net.isr.bindthreads: 1 net.isr.maxqlimit: 20480 net.isr.defaultqlimit: 4096 net.isr.maxprot: 16 net.isr.numthreads: 3 # sysctl net.route. net.route.netisr_maxqlen: 4096 # netstat -Q Configuration: Setting Current Limit Thread count 3 3 Default queue limit 4096 20480 Dispatch policy direct n/a Threads bound to CPUs enabled n/a Protocols: Name Proto QLimit Policy Dispatch Flags ip 1 4096 flow default --- igmp 2 4096 source default --- rtsock 3 4096 source default --- arp 7 4096 source default --- ether 9 4096 source direct --- ip6 10 4096 flow default --- Workstreams: WSID CPU Name Len WMark Disp'd HDisp'd QDrops Queued Handled 0 0 ip 0 4096 37379235 0 94902 22919506 60298741 0 0 igmp 0 0 0 0 0 0 0 0 0 rtsock 0 412 0 0 0 94653 94653 0 0 arp 0 0 12006983 0 0 0 12006983 0 0 ether 0 0 738671396 0 0 0 73867139= 6 0 0 ip6 0 3105 46250868 0 0 717393 46968261 1 1 ip 0 4096 36662705 0 7136 1722812 38385517 1 1 igmp 0 0 0 0 0 0 0 1 1 rtsock 0 0 0 0 0 0 0 1 1 arp 0 0 2335 0 0 0 2335 1 1 ether 0 0 398136593 0 0 0 39813659= 3 1 1 ip6 0 2076 58246220 0 0 28286 58274506 2 2 ip 0 74 40025929 0 0 4621109 44647038 2 2 igmp 0 0 0 0 0 0 0 2 2 rtsock 0 0 0 0 0 0 0 2 2 arp 0 0 3145 0 0 0 3145 2 2 ether 0 0 391770318 0 0 0 39177031= 8 2 2 ip6 0 3 39938460 0 0 694 39939154 # sysctl dev.igb. dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.0.%driver: igb dev.igb.0.%location: slot=3D0 function=3D0 dev.igb.0.%pnpinfo: vendor=3D0x8086 device=3D0x1521 subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 dev.igb.0.%parent: pci9 dev.igb.0.nvm: -1 dev.igb.0.enable_aim: 1 dev.igb.0.fc: 0 dev.igb.0.rx_processing_limit: 1024 dev.igb.0.dmac: 0 dev.igb.0.eee_disabled: 0 dev.igb.0.link_irq: 10 dev.igb.0.dropped: 0 dev.igb.0.tx_dma_fail: 0 dev.igb.0.rx_overruns: 0 dev.igb.0.watchdog_timeouts: 0 dev.igb.0.device_control: 1209795137 dev.igb.0.rx_control: 67141658 dev.igb.0.interrupt_mask: 4 dev.igb.0.extended_int_mask: 2147483663 dev.igb.0.tx_buf_alloc: 0 dev.igb.0.rx_buf_alloc: 0 dev.igb.0.fc_high_water: 33168 dev.igb.0.fc_low_water: 33152 dev.igb.0.queue0.no_desc_avail: 6 dev.igb.0.queue0.tx_packets: 516852382 dev.igb.0.queue0.rx_packets: 529687186 dev.igb.0.queue0.rx_bytes: 22241465754 dev.igb.0.queue0.lro_queued: 0 dev.igb.0.queue0.lro_flushed: 0 dev.igb.0.queue1.no_desc_avail: 6 dev.igb.0.queue1.tx_packets: 515982331 dev.igb.0.queue1.rx_packets: 381070968 dev.igb.0.queue1.rx_bytes: 14255780030 dev.igb.0.queue1.lro_queued: 0 dev.igb.0.queue1.lro_flushed: 0 dev.igb.0.queue2.no_desc_avail: 6 dev.igb.0.queue2.tx_packets: 513824025 dev.igb.0.queue2.rx_packets: 368240158 dev.igb.0.queue2.rx_bytes: 26077348141 dev.igb.0.queue2.lro_queued: 0 dev.igb.0.queue2.lro_flushed: 0 dev.igb.0.mac_stats.excess_coll: 0 dev.igb.0.mac_stats.single_coll: 0 dev.igb.0.mac_stats.multiple_coll: 0 dev.igb.0.mac_stats.late_coll: 0 dev.igb.0.mac_stats.collision_count: 0 dev.igb.0.mac_stats.symbol_errors: 0 dev.igb.0.mac_stats.sequence_errors: 0 dev.igb.0.mac_stats.defer_count: 0 dev.igb.0.mac_stats.missed_packets: 427447 dev.igb.0.mac_stats.recv_no_buff: 71 dev.igb.0.mac_stats.recv_undersize: 0 dev.igb.0.mac_stats.recv_fragmented: 0 dev.igb.0.mac_stats.recv_oversize: 325 dev.igb.0.mac_stats.recv_jabber: 0 dev.igb.0.mac_stats.recv_errs: 0 dev.igb.0.mac_stats.crc_errs: 0 dev.igb.0.mac_stats.alignment_errs: 0 dev.igb.0.mac_stats.coll_ext_errs: 0 dev.igb.0.mac_stats.xon_recvd: 0 dev.igb.0.mac_stats.xon_txd: 0 dev.igb.0.mac_stats.xoff_recvd: 0 dev.igb.0.mac_stats.xoff_txd: 0 dev.igb.0.mac_stats.total_pkts_recvd: 1279285268 dev.igb.0.mac_stats.good_pkts_recvd: 1278803003 dev.igb.0.mac_stats.bcast_pkts_recvd: 22508587 dev.igb.0.mac_stats.mcast_pkts_recvd: 30424048 dev.igb.0.mac_stats.rx_frames_64: 493315593 dev.igb.0.mac_stats.rx_frames_65_127: 491205147 dev.igb.0.mac_stats.rx_frames_128_255: 25874063 dev.igb.0.mac_stats.rx_frames_256_511: 24981098 dev.igb.0.mac_stats.rx_frames_512_1023: 38860430 dev.igb.0.mac_stats.rx_frames_1024_1522: 204566672 dev.igb.0.mac_stats.good_octets_recvd: 412107586597 dev.igb.0.mac_stats.good_octets_txd: 1777183006840 dev.igb.0.mac_stats.total_pkts_txd: 1546643040 dev.igb.0.mac_stats.good_pkts_txd: 1546643040 dev.igb.0.mac_stats.bcast_pkts_txd: 284306 dev.igb.0.mac_stats.mcast_pkts_txd: 212244 dev.igb.0.mac_stats.tx_frames_64: 154006784 dev.igb.0.mac_stats.tx_frames_65_127: 111698251 dev.igb.0.mac_stats.tx_frames_128_255: 38858200 dev.igb.0.mac_stats.tx_frames_256_511: 55855569 dev.igb.0.mac_stats.tx_frames_512_1023: 47293645 dev.igb.0.mac_stats.tx_frames_1024_1522: 1138930591 dev.igb.0.mac_stats.tso_txd: 0 dev.igb.0.mac_stats.tso_ctx_fail: 0 dev.igb.0.interrupts.asserts: 1720418170 dev.igb.0.interrupts.rx_pkt_timer: 1278783566 dev.igb.0.interrupts.rx_abs_timer: 0 dev.igb.0.interrupts.tx_pkt_timer: 0 dev.igb.0.interrupts.tx_abs_timer: 0 dev.igb.0.interrupts.tx_queue_empty: 1546622059 dev.igb.0.interrupts.tx_queue_min_thresh: 3275762942 dev.igb.0.interrupts.rx_desc_min_thresh: 0 dev.igb.0.interrupts.rx_overrun: 0 dev.igb.0.host.breaker_tx_pkt: 0 dev.igb.0.host.host_tx_pkt_discard: 0 dev.igb.0.host.rx_pkt: 19438 dev.igb.0.host.breaker_rx_pkts: 0 dev.igb.0.host.breaker_rx_pkt_drop: 0 dev.igb.0.host.tx_good_pkt: 20981 dev.igb.0.host.breaker_tx_pkt_drop: 0 dev.igb.0.host.rx_good_bytes: 412107590415 dev.igb.0.host.tx_good_bytes: 1777183006840 dev.igb.0.host.length_errors: 1929 dev.igb.0.host.serdes_violation_pkt: 0 dev.igb.0.host.header_redir_missed: 0 dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.1.%driver: igb dev.igb.1.%location: slot=3D0 function=3D1 dev.igb.1.%pnpinfo: vendor=3D0x8086 device=3D0x1521 subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 dev.igb.1.%parent: pci9 dev.igb.1.nvm: -1 dev.igb.1.enable_aim: 1 dev.igb.1.fc: 0 dev.igb.1.rx_processing_limit: 1024 dev.igb.1.dmac: 0 dev.igb.1.eee_disabled: 0 dev.igb.1.link_irq: 2 dev.igb.1.dropped: 0 dev.igb.1.tx_dma_fail: 0 dev.igb.1.rx_overruns: 0 dev.igb.1.watchdog_timeouts: 0 dev.igb.1.device_control: 1209795137 dev.igb.1.rx_control: 67141658 dev.igb.1.interrupt_mask: 4 dev.igb.1.extended_int_mask: 2147483663 dev.igb.1.tx_buf_alloc: 0 dev.igb.1.rx_buf_alloc: 0 dev.igb.1.fc_high_water: 33168 dev.igb.1.fc_low_water: 33152 dev.igb.1.queue0.no_desc_avail: 0 dev.igb.1.queue0.tx_packets: 523930609 dev.igb.1.queue0.rx_packets: 680643711 dev.igb.1.queue0.rx_bytes: 708218088144 dev.igb.1.queue0.lro_queued: 0 dev.igb.1.queue0.lro_flushed: 0 dev.igb.1.queue1.no_desc_avail: 0 dev.igb.1.queue1.tx_packets: 529405009 dev.igb.1.queue1.rx_packets: 669699145 dev.igb.1.queue1.rx_bytes: 694129068004 dev.igb.1.queue1.lro_queued: 0 dev.igb.1.queue1.lro_flushed: 0 dev.igb.1.queue2.no_desc_avail: 0 dev.igb.1.queue2.tx_packets: 500794589 dev.igb.1.queue2.rx_packets: 654545718 dev.igb.1.queue2.rx_bytes: 671933390519 dev.igb.1.queue2.lro_queued: 0 dev.igb.1.queue2.lro_flushed: 0 dev.igb.1.mac_stats.excess_coll: 0 dev.igb.1.mac_stats.single_coll: 0 dev.igb.1.mac_stats.multiple_coll: 0 dev.igb.1.mac_stats.late_coll: 0 dev.igb.1.mac_stats.collision_count: 0 dev.igb.1.mac_stats.symbol_errors: 0 dev.igb.1.mac_stats.sequence_errors: 0 dev.igb.1.mac_stats.defer_count: 0 dev.igb.1.mac_stats.missed_packets: 114803 dev.igb.1.mac_stats.recv_no_buff: 7 dev.igb.1.mac_stats.recv_undersize: 0 dev.igb.1.mac_stats.recv_fragmented: 0 dev.igb.1.mac_stats.recv_oversize: 0 dev.igb.1.mac_stats.recv_jabber: 0 dev.igb.1.mac_stats.recv_errs: 0 dev.igb.1.mac_stats.crc_errs: 0 dev.igb.1.mac_stats.alignment_errs: 0 dev.igb.1.mac_stats.coll_ext_errs: 0 dev.igb.1.mac_stats.xon_recvd: 0 dev.igb.1.mac_stats.xon_txd: 0 dev.igb.1.mac_stats.xoff_recvd: 0 dev.igb.1.mac_stats.xoff_txd: 0 dev.igb.1.mac_stats.total_pkts_recvd: 2005019592 dev.igb.1.mac_stats.good_pkts_recvd: 2004886386 dev.igb.1.mac_stats.bcast_pkts_recvd: 4241129 dev.igb.1.mac_stats.mcast_pkts_recvd: 6973905 dev.igb.1.mac_stats.rx_frames_64: 206735844 dev.igb.1.mac_stats.rx_frames_65_127: 329507211 dev.igb.1.mac_stats.rx_frames_128_255: 46979194 dev.igb.1.mac_stats.rx_frames_256_511: 47697263 dev.igb.1.mac_stats.rx_frames_512_1023: 53530798 dev.igb.1.mac_stats.rx_frames_1024_1522: 1320436076 dev.igb.1.mac_stats.good_octets_recvd: 2082949543090 dev.igb.1.mac_stats.good_octets_txd: 796399612004 dev.igb.1.mac_stats.total_pkts_txd: 1554128026 dev.igb.1.mac_stats.good_pkts_txd: 1554128026 dev.igb.1.mac_stats.bcast_pkts_txd: 75022 dev.igb.1.mac_stats.mcast_pkts_txd: 49 dev.igb.1.mac_stats.tx_frames_64: 433361117 dev.igb.1.mac_stats.tx_frames_65_127: 563082322 dev.igb.1.mac_stats.tx_frames_128_255: 26328309 dev.igb.1.mac_stats.tx_frames_256_511: 28329889 dev.igb.1.mac_stats.tx_frames_512_1023: 50096025 dev.igb.1.mac_stats.tx_frames_1024_1522: 452930364 dev.igb.1.mac_stats.tso_txd: 0 dev.igb.1.mac_stats.tso_ctx_fail: 0 dev.igb.1.interrupts.asserts: 1412655129 dev.igb.1.interrupts.rx_pkt_timer: 2004854126 dev.igb.1.interrupts.rx_abs_timer: 0 dev.igb.1.interrupts.tx_pkt_timer: 0 dev.igb.1.interrupts.tx_abs_timer: 0 dev.igb.1.interrupts.tx_queue_empty: 1554108764 dev.igb.1.interrupts.tx_queue_min_thresh: 118190592 dev.igb.1.interrupts.rx_desc_min_thresh: 0 dev.igb.1.interrupts.rx_overrun: 0 dev.igb.1.host.breaker_tx_pkt: 0 dev.igb.1.host.host_tx_pkt_discard: 0 dev.igb.1.host.rx_pkt: 32260 dev.igb.1.host.breaker_rx_pkts: 0 dev.igb.1.host.breaker_rx_pkt_drop: 0 dev.igb.1.host.tx_good_pkt: 19262 dev.igb.1.host.breaker_tx_pkt_drop: 0 dev.igb.1.host.rx_good_bytes: 2082949636304 dev.igb.1.host.tx_good_bytes: 796399612004 dev.igb.1.host.length_errors: 1926 dev.igb.1.host.serdes_violation_pkt: 0 dev.igb.1.host.header_redir_missed: 0 dev.igb.2.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.2.%driver: igb dev.igb.2.%location: slot=3D0 function=3D2 dev.igb.2.%pnpinfo: vendor=3D0x8086 device=3D0x1521 subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 dev.igb.2.%parent: pci9 dev.igb.2.nvm: -1 dev.igb.2.enable_aim: 1 dev.igb.2.fc: 0 dev.igb.2.rx_processing_limit: 1024 dev.igb.2.dmac: 0 dev.igb.2.eee_disabled: 0 dev.igb.2.link_irq: 2 dev.igb.2.dropped: 0 dev.igb.2.tx_dma_fail: 0 dev.igb.2.rx_overruns: 0 dev.igb.2.watchdog_timeouts: 0 dev.igb.2.device_control: 1209795137 dev.igb.2.rx_control: 67141658 dev.igb.2.interrupt_mask: 4 dev.igb.2.extended_int_mask: 2147483663 dev.igb.2.tx_buf_alloc: 0 dev.igb.2.rx_buf_alloc: 0 dev.igb.2.fc_high_water: 33168 dev.igb.2.fc_low_water: 33152 dev.igb.2.queue0.no_desc_avail: 0 dev.igb.2.queue0.tx_packets: 26500409 dev.igb.2.queue0.rx_packets: 35480078 dev.igb.2.queue0.rx_bytes: 31515581084 dev.igb.2.queue0.lro_queued: 0 dev.igb.2.queue0.lro_flushed: 0 dev.igb.2.queue1.no_desc_avail: 0 dev.igb.2.queue1.tx_packets: 26617642 dev.igb.2.queue1.rx_packets: 35427096 dev.igb.2.queue1.rx_bytes: 31684312453 dev.igb.2.queue1.lro_queued: 0 dev.igb.2.queue1.lro_flushed: 0 dev.igb.2.queue2.no_desc_avail: 0 dev.igb.2.queue2.tx_packets: 30206184 dev.igb.2.queue2.rx_packets: 41459725 dev.igb.2.queue2.rx_bytes: 42527725829 dev.igb.2.queue2.lro_queued: 0 dev.igb.2.queue2.lro_flushed: 0 dev.igb.2.mac_stats.excess_coll: 0 dev.igb.2.mac_stats.single_coll: 0 dev.igb.2.mac_stats.multiple_coll: 0 dev.igb.2.mac_stats.late_coll: 0 dev.igb.2.mac_stats.collision_count: 0 dev.igb.2.mac_stats.symbol_errors: 0 dev.igb.2.mac_stats.sequence_errors: 0 dev.igb.2.mac_stats.defer_count: 0 dev.igb.2.mac_stats.missed_packets: 0 dev.igb.2.mac_stats.recv_no_buff: 0 dev.igb.2.mac_stats.recv_undersize: 0 dev.igb.2.mac_stats.recv_fragmented: 0 dev.igb.2.mac_stats.recv_oversize: 0 dev.igb.2.mac_stats.recv_jabber: 0 dev.igb.2.mac_stats.recv_errs: 0 dev.igb.2.mac_stats.crc_errs: 0 dev.igb.2.mac_stats.alignment_errs: 0 dev.igb.2.mac_stats.coll_ext_errs: 0 dev.igb.2.mac_stats.xon_recvd: 0 dev.igb.2.mac_stats.xon_txd: 0 dev.igb.2.mac_stats.xoff_recvd: 0 dev.igb.2.mac_stats.xoff_txd: 0 dev.igb.2.mac_stats.total_pkts_recvd: 112367081 dev.igb.2.mac_stats.good_pkts_recvd: 112366789 dev.igb.2.mac_stats.bcast_pkts_recvd: 23942 dev.igb.2.mac_stats.mcast_pkts_recvd: 131827 dev.igb.2.mac_stats.rx_frames_64: 5888966 dev.igb.2.mac_stats.rx_frames_65_127: 25732916 dev.igb.2.mac_stats.rx_frames_128_255: 6139316 dev.igb.2.mac_stats.rx_frames_256_511: 5777835 dev.igb.2.mac_stats.rx_frames_512_1023: 2826165 dev.igb.2.mac_stats.rx_frames_1024_1522: 66001591 dev.igb.2.mac_stats.good_octets_recvd: 106177003543 dev.igb.2.mac_stats.good_octets_txd: 16031387076 dev.igb.2.mac_stats.total_pkts_txd: 83324159 dev.igb.2.mac_stats.good_pkts_txd: 83324159 dev.igb.2.mac_stats.bcast_pkts_txd: 3629 dev.igb.2.mac_stats.mcast_pkts_txd: 14 dev.igb.2.mac_stats.tx_frames_64: 25588411 dev.igb.2.mac_stats.tx_frames_65_127: 42179767 dev.igb.2.mac_stats.tx_frames_128_255: 5508485 dev.igb.2.mac_stats.tx_frames_256_511: 2598777 dev.igb.2.mac_stats.tx_frames_512_1023: 2774048 dev.igb.2.mac_stats.tx_frames_1024_1522: 4674671 dev.igb.2.mac_stats.tso_txd: 0 dev.igb.2.mac_stats.tso_ctx_fail: 0 dev.igb.2.interrupts.asserts: 158093007 dev.igb.2.interrupts.rx_pkt_timer: 112365086 dev.igb.2.interrupts.rx_abs_timer: 0 dev.igb.2.interrupts.tx_pkt_timer: 0 dev.igb.2.interrupts.tx_abs_timer: 0 dev.igb.2.interrupts.tx_queue_empty: 83323166 dev.igb.2.interrupts.tx_queue_min_thresh: 1986860 dev.igb.2.interrupts.rx_desc_min_thresh: 0 dev.igb.2.interrupts.rx_overrun: 0 dev.igb.2.host.breaker_tx_pkt: 0 dev.igb.2.host.host_tx_pkt_discard: 0 dev.igb.2.host.rx_pkt: 1703 dev.igb.2.host.breaker_rx_pkts: 0 dev.igb.2.host.breaker_rx_pkt_drop: 0 dev.igb.2.host.tx_good_pkt: 993 dev.igb.2.host.breaker_tx_pkt_drop: 0 dev.igb.2.host.rx_good_bytes: 106177008073 dev.igb.2.host.tx_good_bytes: 16031387076 dev.igb.2.host.length_errors: 0 dev.igb.2.host.serdes_violation_pkt: 0 dev.igb.2.host.header_redir_missed: 0 dev.igb.3.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.3.%driver: igb dev.igb.3.%location: slot=3D0 function=3D3 dev.igb.3.%pnpinfo: vendor=3D0x8086 device=3D0x1521 subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 dev.igb.3.%parent: pci9 dev.igb.3.nvm: -1 dev.igb.3.enable_aim: 1 dev.igb.3.fc: 0 dev.igb.3.rx_processing_limit: 1024 dev.igb.3.dmac: 0 dev.igb.3.eee_disabled: 0 dev.igb.3.link_irq: 0 dev.igb.3.dropped: 0 dev.igb.3.tx_dma_fail: 0 dev.igb.3.rx_overruns: 0 dev.igb.3.watchdog_timeouts: 0 dev.igb.3.device_control: 136053313 dev.igb.3.rx_control: 0 dev.igb.3.interrupt_mask: 0 dev.igb.3.extended_int_mask: 2147483648 dev.igb.3.tx_buf_alloc: 0 dev.igb.3.rx_buf_alloc: 0 dev.igb.3.fc_high_water: 33168 dev.igb.3.fc_low_water: 33152 dev.igb.3.queue0.no_desc_avail: 0 dev.igb.3.queue0.tx_packets: 0 dev.igb.3.queue0.rx_packets: 0 dev.igb.3.queue0.rx_bytes: 0 dev.igb.3.queue0.lro_queued: 0 dev.igb.3.queue0.lro_flushed: 0 dev.igb.3.queue1.no_desc_avail: 0 dev.igb.3.queue1.tx_packets: 0 dev.igb.3.queue1.rx_packets: 0 dev.igb.3.queue1.rx_bytes: 0 dev.igb.3.queue1.lro_queued: 0 dev.igb.3.queue1.lro_flushed: 0 dev.igb.3.queue2.no_desc_avail: 0 dev.igb.3.queue2.tx_packets: 0 dev.igb.3.queue2.rx_packets: 0 dev.igb.3.queue2.rx_bytes: 0 dev.igb.3.queue2.lro_queued: 0 dev.igb.3.queue2.lro_flushed: 0 dev.igb.3.mac_stats.excess_coll: 0 dev.igb.3.mac_stats.single_coll: 0 dev.igb.3.mac_stats.multiple_coll: 0 dev.igb.3.mac_stats.late_coll: 0 dev.igb.3.mac_stats.collision_count: 0 dev.igb.3.mac_stats.symbol_errors: 0 dev.igb.3.mac_stats.sequence_errors: 0 dev.igb.3.mac_stats.defer_count: 0 dev.igb.3.mac_stats.missed_packets: 0 dev.igb.3.mac_stats.recv_no_buff: 0 dev.igb.3.mac_stats.recv_undersize: 0 dev.igb.3.mac_stats.recv_fragmented: 0 dev.igb.3.mac_stats.recv_oversize: 0 dev.igb.3.mac_stats.recv_jabber: 0 dev.igb.3.mac_stats.recv_errs: 0 dev.igb.3.mac_stats.crc_errs: 0 dev.igb.3.mac_stats.alignment_errs: 0 dev.igb.3.mac_stats.coll_ext_errs: 0 dev.igb.3.mac_stats.xon_recvd: 0 dev.igb.3.mac_stats.xon_txd: 0 dev.igb.3.mac_stats.xoff_recvd: 0 dev.igb.3.mac_stats.xoff_txd: 0 dev.igb.3.mac_stats.total_pkts_recvd: 0 dev.igb.3.mac_stats.good_pkts_recvd: 0 dev.igb.3.mac_stats.bcast_pkts_recvd: 0 dev.igb.3.mac_stats.mcast_pkts_recvd: 0 dev.igb.3.mac_stats.rx_frames_64: 0 dev.igb.3.mac_stats.rx_frames_65_127: 0 dev.igb.3.mac_stats.rx_frames_128_255: 0 dev.igb.3.mac_stats.rx_frames_256_511: 0 dev.igb.3.mac_stats.rx_frames_512_1023: 0 dev.igb.3.mac_stats.rx_frames_1024_1522: 0 dev.igb.3.mac_stats.good_octets_recvd: 0 dev.igb.3.mac_stats.good_octets_txd: 0 dev.igb.3.mac_stats.total_pkts_txd: 0 dev.igb.3.mac_stats.good_pkts_txd: 0 dev.igb.3.mac_stats.bcast_pkts_txd: 0 dev.igb.3.mac_stats.mcast_pkts_txd: 0 dev.igb.3.mac_stats.tx_frames_64: 0 dev.igb.3.mac_stats.tx_frames_65_127: 0 dev.igb.3.mac_stats.tx_frames_128_255: 0 dev.igb.3.mac_stats.tx_frames_256_511: 0 dev.igb.3.mac_stats.tx_frames_512_1023: 0 dev.igb.3.mac_stats.tx_frames_1024_1522: 0 dev.igb.3.mac_stats.tso_txd: 0 dev.igb.3.mac_stats.tso_ctx_fail: 0 dev.igb.3.interrupts.asserts: 0 dev.igb.3.interrupts.rx_pkt_timer: 0 dev.igb.3.interrupts.rx_abs_timer: 0 dev.igb.3.interrupts.tx_pkt_timer: 0 dev.igb.3.interrupts.tx_abs_timer: 0 dev.igb.3.interrupts.tx_queue_empty: 0 dev.igb.3.interrupts.tx_queue_min_thresh: 0 dev.igb.3.interrupts.rx_desc_min_thresh: 0 dev.igb.3.interrupts.rx_overrun: 0 dev.igb.3.host.breaker_tx_pkt: 0 dev.igb.3.host.host_tx_pkt_discard: 0 dev.igb.3.host.rx_pkt: 0 dev.igb.3.host.breaker_rx_pkts: 0 dev.igb.3.host.breaker_rx_pkt_drop: 0 dev.igb.3.host.tx_good_pkt: 0 dev.igb.3.host.breaker_tx_pkt_drop: 0 dev.igb.3.host.rx_good_bytes: 0 dev.igb.3.host.tx_good_bytes: 0 dev.igb.3.host.length_errors: 0 dev.igb.3.host.serdes_violation_pkt: 0 dev.igb.3.host.header_redir_missed: 0 On Sat, Apr 5, 2014 at 10:02 AM, hiren panchasara < hiren.panchasara@gmail.com> wrote: > On Thu, Apr 3, 2014 at 9:54 PM, =D6zkan KIRIK wro= te: > > Hi, > > > > I am trying to use suricata on FreeBSD 10 amd64. > > FreeBSD behaves as a VLAN router and NAT Box. > > > > Traffic is about 400Mbps. > > When i diverted traffic to suricata, swi: netisr 0 thread gets %100 cpu= . > > other netisr threads are %0. And Even I remove the divert rule, netisr > > still eats %100 cpu. I think that something looping :) > > To be clear, this happens only *after* you divert traffic to suricata, > right? > > > And after 1-2 minutes, one of igb0 and igb1 stops working. > > Only reboot solves problem. > > > > Hardware has 8 cores, 24GB Ram > > > > My loader.conf : > > > > hw.igb.txd=3D"4096" > > hw.igb.rxd=3D"4096" > > hw.igb.rx_process_limit=3D1024 > > hw.igb.num_queues=3D3 > > net.isr.maxthreads=3D3 > > net.isr.bindthreads=3D1 > > net.isr.defaultqlimit=3D4096 > > net.isr.maxqlimit=3D20480 > > net.link.ifqmaxlen=3D10240 > > > > How can I debug this situation? > > Any suggestions? > > I am not an expert here but please upload o/p for "sysctl net.isr" and > "sysctl dev.igb" which would show error counters to get some idea > about why igb0 or igb1 stops working. Whether we are running out of > some resources or something else is going on. > > cheers, > Hiren > From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 15:27:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A6C2802 for ; Sat, 5 Apr 2014 15:27:47 +0000 (UTC) Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DE3ACEA for ; Sat, 5 Apr 2014 15:27:47 +0000 (UTC) Received: by mail-pb0-f53.google.com with SMTP id rp16so4717614pbb.26 for ; Sat, 05 Apr 2014 08:27:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Henmy3mgVV4jafpHqci/7S53cpm6ck3ln2EKx+OBsTw=; b=XcweRaCQAv75vZVc5NbkEJ/44JVqh+eJnZm1iiUE10HtaBgvoaxW4N759ZdMHRQ70t amDryjM/C4n4c0NggS56ck+lIL9iwScgcrTQs95Ss0mGbvwNLH5QX+hOX6qe8l9icelC p6VEdCvyLZ9Iix30kMUCek7NX8i505f2TV9XsoLcXt+CGFr4tgAyJMoLP+BylL9X3OJ/ gDKeHWnFrxVAkheXZM57/S69DdhmfGz4gL5TqLqrkbH4oa6YOYU/Khha+ws8xoEg8kLV F4o2/TsZKzdeT/GGLwEhW0p+uNTMdo1sXy4lc5J2T9qSRODGJEsHoUSZdckBntAxI3ny aLsg== MIME-Version: 1.0 X-Received: by 10.66.141.12 with SMTP id rk12mr62933pab.152.1396711666088; Sat, 05 Apr 2014 08:27:46 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.70.88.109 with HTTP; Sat, 5 Apr 2014 08:27:46 -0700 (PDT) In-Reply-To: References: Date: Sat, 5 Apr 2014 17:27:46 +0200 X-Google-Sender-Auth: nAZZMc6Z-twO9QA88ohfDuzc6pQ Message-ID: Subject: Re: netisr 0 : %100 and other netisr threads are waiting From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: =?ISO-8859-1?Q?=D6zkan_KIRIK?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 15:27:47 -0000 Hello, what are you using to divert packets, ipfw(4) or pf(4)? Can you show your configuration on that as well! On Fri, Apr 4, 2014 at 6:54 AM, =D6zkan KIRIK wrote= : > Hi, > > I am trying to use suricata on FreeBSD 10 amd64. > FreeBSD behaves as a VLAN router and NAT Box. > > Traffic is about 400Mbps. > When i diverted traffic to suricata, swi: netisr 0 thread gets %100 cpu. > other netisr threads are %0. And Even I remove the divert rule, netisr > still eats %100 cpu. I think that something looping :) > And after 1-2 minutes, one of igb0 and igb1 stops working. > Only reboot solves problem. > > Hardware has 8 cores, 24GB Ram > > My loader.conf : > > hw.igb.txd=3D"4096" > hw.igb.rxd=3D"4096" > hw.igb.rx_process_limit=3D1024 > hw.igb.num_queues=3D3 > net.isr.maxthreads=3D3 > net.isr.bindthreads=3D1 > net.isr.defaultqlimit=3D4096 > net.isr.maxqlimit=3D20480 > net.link.ifqmaxlen=3D10240 > > How can I debug this situation? > Any suggestions? > > Best regards > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 Ermal From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 16:21:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91791919 for ; Sat, 5 Apr 2014 16:21:14 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4AF60218 for ; Sat, 5 Apr 2014 16:21:13 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s35GL3wp053414 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 5 Apr 2014 09:21:05 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53402D68.4030500@freebsd.org> Date: Sun, 06 Apr 2014 00:20:56 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Chris Smith , freebsd-net@freebsd.org Subject: Re: Multihomed system with jails routing issues References: <533F68EF.8060607@nevermind.co.nz> In-Reply-To: <533F68EF.8060607@nevermind.co.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 16:21:14 -0000 On 4/5/14, 10:22 AM, Chris Smith wrote: > Hi All, > > I have a system with 1 network interface with 2 extra VLANs off it > and I'm having some trouble getting the routing working correctly > with it and jails. > > bge0 - management - 10.71.100.0/24 > bge0.101 - LAN - 10.71.101.0/24 > bge0.103 - DMZ - 10.71.101.0/24 > > Here's what I want to achieve... > > Host: > I want the host system to only listen on one interface, bge0. I want > NO ip addresses of the host on the vlan interfaces. The only service > it will be exposing is its sshd. The management address for this > system is 10.71.100.50. > Sounds to me that you want to use vimage jails. check the vnet command to jail . > Jails: > The system will also host a variety of jails, each with an IP either > on the LAN or DMZ. I am using ezjail to manage the jails. > > Router: > There is a router at the .254 address of every subnet that can route > between each network. > > I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host > does not have an address configured on bge0.101, I configured the > jail address as /24 instead of the default /32. > > My issues: > > * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot > communicate with the jail. > > * When the jail is up and 10.71.101.51/24 is active, SSHing from the > LAN to the mgmt interface via the router fails, as the host tries to > send return traffic via the bge0.101 interface, even though traffic > arrived via the bge0 interface. > > So I did a whole lot of research for people having these apparently > problems, and decided to try the multiple routing table/fib > approach. So I recompiled my kernel, configured fib 1 with the LAN > interface route (setfib route add 10.71.101.0/24 -iface bge0.101), > set the jail fib and set the tunable net.addr_all_fibs = 0. I still > can't get this working correctly. ezjail still seems to add the > interface route to fib 0 by default (but it won't if i run ezjail > with the setfib 1 command). > > Using FIB 1 and trying to ping hosts on the LAN gives an error like: > sendto failed: invalid argument. > > Does anybody have any best practices for doing this, or anything > else I can try? I'm happy to share/pastebin any configuration and > I've tried most things I've found on the internet. I'm using FreeBSD > 10.0 with a custom kernel for multiple routing tables. > > Thanks in advance! > Chris. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Apr 5 21:03:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 036CB455 for ; Sat, 5 Apr 2014 21:03:34 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 88CBED7F for ; Sat, 5 Apr 2014 21:03:32 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s35L2x4V063907 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sat, 5 Apr 2014 23:03:00 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s35L2kLm034273 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 5 Apr 2014 23:02:46 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s35L2kCF065073; Sat, 5 Apr 2014 23:02:46 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s35L2kqK065072; Sat, 5 Apr 2014 23:02:46 +0200 (CEST) (envelope-from ticso) Date: Sat, 5 Apr 2014 23:02:46 +0200 From: Bernd Walter To: freebsd-net@freebsd.org Subject: SCTP binds to IPs outside of jail Message-ID: <20140405210246.GB58138@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_FRT_STOCK2=0.01, T_RP_MATCHES_RCVD=-0.01 autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 21:03:34 -0000 So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. The modifications are to allow IPv6 multicast support within jail which only makes a difference for multicast addresses and some multicast loopback checksum bugs - both changes are open PR. I've created an AF_INET6 SCTP one to many socket to receive incoming messages. The process was started within a jail. Now netstat -anW lists all host IPv6 IPs, not just those of the jail. Also not sure why this AF_INET6 socket is shown as sctp46. This is the relevant C++ code part to open the socket: int setup_sctp_socket(uint16_t port) { int sc = socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); { // reuse address long val = 1; setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val)); // XXX error handling } { // no delay long val = 1; setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, sizeof(val)); // XXX error handling } { // eeor mode (last write needs MSG_EOR to declare end of message) // Linux has MSG_MORE negative send flag long val = 1; setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, sizeof(val)); // XXX error handling } #if 0 { struct sctp_initmsg init; bzero(&init, sizeof(init)); init.sinit_num_ostreams = HDB_STREAMS; init.sinit_max_instreams = HDB_STREAMS; // SOL_SCTP instead of IPPROTO_SCTP on Linux setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, (socklen_t)sizeof(struct sctp_initmsg)); // XXX error handling } #endif { struct sockaddr_in6 addr; bzero(&addr, sizeof(addr)); addr.sin6_len = sizeof(addr); addr.sin6_family = AF_INET6; addr.sin6_port = htons(port); bind(sc, (struct sockaddr *)&addr, sizeof(struct sockaddr_in)); // XXX error handling } { // enable heartbeats at 1000ms struct sctp_paddrparams paddr_params; bzero(&paddr_params, sizeof(paddr_params)); paddr_params.spp_address.ss_family = AF_INET6; paddr_params.spp_flags = SPP_HB_ENABLE; paddr_params.spp_hbinterval = 1000; // SOL_SCTP instead of IPPROTO_SCTP on Linux setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, &paddr_params, sizeof(paddr_params)); // XXX error handling } { struct sctp_event_subscribe events; bzero(&events, sizeof(events)); events.sctp_data_io_event = 1; // we need io_events to know where the message came from // subscribe to other events as well for testing events.sctp_association_event = 1; events.sctp_address_event = 1; events.sctp_send_failure_event = 1; events.sctp_peer_error_event = 1; events.sctp_shutdown_event = 1; events.sctp_partial_delivery_event = 1; events.sctp_adaptation_layer_event = 1; events.sctp_authentication_event = 1; events.sctp_sender_dry_event = 1; events.sctp_stream_reset_event = 1; setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, sizeof(events)); // XXX error handling } { // setup send and receive buffers (default on FreeBSD 9.x) long val; val = 1864135; setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, sizeof(val)); // XXX error handling val = 1864135; setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, sizeof(val)); // XXX error handling } listen (sc, 1); // listen is required to allow incoming associations, but no listen queue // XXX error handling return sc; } -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 02:32:52 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6D1DB72 for ; Sun, 6 Apr 2014 02:32:52 +0000 (UTC) Received: from mail.lariat.net (mail.lariat.net [66.62.230.51]) by mx1.freebsd.org (Postfix) with ESMTP id 48A16978 for ; Sun, 6 Apr 2014 02:32:52 +0000 (UTC) Received: from Toshi.lariat.net (IDENT:ppp1000.lariat.net@localhost [127.0.0.1]) by mail.lariat.net (8.9.3/8.9.3) with ESMTP id UAA10958 for ; Sat, 5 Apr 2014 20:26:44 -0600 (MDT) Message-Id: <201404060226.UAA10958@mail.lariat.net> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 05 Apr 2014 20:26:00 -0600 To: net@freebsd.org From: Brett Glass Subject: IPFW and VLANs Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 02:32:52 -0000 Everyone: I'm writing some new rulesets for IPFW on a machine that has only one built-in Ethernet interface. It connects to a few different Ethernets via a VLAN switch. (The physical interface leads to a "trunk;" that is to say, all packets passing om and out of the parent interface ought to be tagged with a VLAN number associated with one or more of the ports on the external switch. There shouldn't be any untagged packets on the interface.) One of the things I wanted to do in my rules was block IPv4 multicast packets on some -- or possibly all -- of the interfaces, depending on whether an interface needed to use routing protocols that did multicasting. I became curious: When there are VLANs (which are implemented as "child" interfaces in FreeBSD), is each Layer 2 packet that passes through the "parent" interface handed to IPFW twice -- once for the parent interface and once for the child for which the packet is tagged? (This would be inefficient, but at least I could minimize the inefficiency by putting in an early rule to pass all packets on the parent and then filter on the children... or just filter on the parent if I wanted to block all multicasting.) Figuring that it would be fastest to do an empirical test to see how the packets were handled, I set the sysctl variable net.link.ether.ipfw to 1 and set up some rules to check the behavior. The rules counted all of the raw (layer2) packets on the parent interface (re0) and also on one of the children (re0_1); one of them also looked for non-Layer 2 traffic on the parent (which I didn't expect to find). I then let the machine, which was set up as a router, process a bit of traffic. What I saw, when I looked at the results, was downright strange: 00001 4290 1268452 count ip from any to any layer2 via re0_1 00002 3878 1251586 count ip from any to any layer2 via re0 00003 0 0 count ip from any to any not layer2 via re0 According to these counts, IPFW wasn't getting Layer 3 packets from the parent interface (Rule 3). That made sense, because the parent did not even have an IP address assigned to it. However, IPFW seemed to be counting more packets passing through one of the "child" interfaces (Rule 1) than through the parent (Rule 2), even though other "child" interfaces were also quite active. I added a few more rules, with "recv" and "xmit" options, and checked the counts again after zeroing them and letting the router run for a bit: 00001 20591 8769298 count ip from any to any layer2 via re0_1 00002 18715 8725085 count ip from any to any layer2 via re0 00003 0 0 count ip from any to any not layer2 via re0 00004 18715 8725085 count ip from any to any layer2 recv re0 00005 18715 8725085 count ip from any to any layer2 xmit re0 00006 12746 1324342 count ip from any to any layer2 recv re0_1 00007 20592 8770798 count ip from any to any layer2 xmit re0_1 Maybe I am missing something (as I often do), but this seems just plain wrong. What gives? Help in interpreting these results would be much appreciated. --Brett Glass From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 04:27:46 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B5D2C10 for ; Sun, 6 Apr 2014 04:27:45 +0000 (UTC) Received: from mail.lariat.net (mail.lariat.net [66.62.230.51]) by mx1.freebsd.org (Postfix) with ESMTP id B00021AE for ; Sun, 6 Apr 2014 04:27:45 +0000 (UTC) Received: from Toshi.lariat.net (IDENT:ppp1000.lariat.net@localhost [127.0.0.1]) by mail.lariat.net (8.9.3/8.9.3) with ESMTP id WAA11607 for ; Sat, 5 Apr 2014 22:27:41 -0600 (MDT) Message-Id: <201404060427.WAA11607@mail.lariat.net> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 05 Apr 2014 22:27:17 -0600 To: net@freebsd.org From: Brett Glass Subject: More on odd IPFW behavior Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 04:27:46 -0000 A bit more investigation of IPFW's behavior on VLAN interfaces has revealed some even stranger stuff. Consider the tallies on the following firewall rules: # ipfw show | head 00001 65071 36685513 count ip from any to any layer2 via re0 00002 65303 36856334 count ip from any to any layer2 via re0_1 00003 6 3381 count ip from any to any layer2 via re0_2 00004 49338 35208527 count ip from any to any layer2 via re0_3 00005 0 0 count ip from any to any layer2 in recv re0 00006 65071 36685513 count ip from any to any layer2 out xmit re0 00007 0 0 count ip from any to any layer2 in recv re0_1 00008 65303 36856334 count ip from any to any layer2 out xmit re0_1 It looks as if, when one adds "in" and "out" to the rules, one never sees any Layer 2 packets coming "in" on either a vlan(4) interface or its parent. There might be a problem with general brokenness in IPFW's "in" and "out" qualifiers when dealing with Layer 2 packets, or something else might be wrong.... Not sure, but this behavior is definitely weird. And note that, again, re0_1 (a child interface) shows more packets than re0 (the parent). Weird. Do not have experience with pf, so do not know if it would do better, but IPFW certainly has something broken. Help in figuring out what to propose as a patch would be MUCH appreciated. --Brett Glass From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 09:04:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D32862C for ; Sun, 6 Apr 2014 09:04:15 +0000 (UTC) Received: from smtp.webfaction.com (mail6.webfaction.com [74.55.86.74]) by mx1.freebsd.org (Postfix) with ESMTP id 800F5914 for ; Sun, 6 Apr 2014 09:04:14 +0000 (UTC) Received: from [10.71.101.130] (unknown [203.86.207.104]) by smtp.webfaction.com (Postfix) with ESMTP id B50D02079252 for ; Sun, 6 Apr 2014 09:04:07 +0000 (UTC) Message-ID: <53411885.7030206@nevermind.co.nz> Date: Sun, 06 Apr 2014 21:04:05 +1200 From: Chris Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Multihomed system with jails routing issues References: <533F68EF.8060607@nevermind.co.nz> <53402D68.4030500@freebsd.org> In-Reply-To: <53402D68.4030500@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 09:04:15 -0000 On 06/04/14 04:20, Julian Elischer wrote: > On 4/5/14, 10:22 AM, Chris Smith wrote: >> Hi All, >> >> I have a system with 1 network interface with 2 extra VLANs off it >> and I'm having some trouble getting the routing working correctly >> with it and jails. >> >> bge0 - management - 10.71.100.0/24 >> bge0.101 - LAN - 10.71.101.0/24 >> bge0.103 - DMZ - 10.71.101.0/24 >> >> Here's what I want to achieve... >> >> Host: >> I want the host system to only listen on one interface, bge0. I want >> NO ip addresses of the host on the vlan interfaces. The only service >> it will be exposing is its sshd. The management address for this >> system is 10.71.100.50. >> > Sounds to me that you want to use vimage jails. > check the vnet command to jail . > Hey Julian, Thanks for that. I did come across it but all of the documentation I found indicated that it was experimental. After a day or so messing around with VIMAGE/vnet and their various gotchas and interactions with jails on FreeBSD 10, I have something working that I'm happy with. I've made a bunch of notes so I hope to write something up for it since most of the documentation around this is thin, old or outdated. Cheers, Chris. From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 10:57:44 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C198DD4D for ; Sun, 6 Apr 2014 10:57:44 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DA5114C for ; Sun, 6 Apr 2014 10:57:43 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s36AvKYc082262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Apr 2014 12:57:25 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: net@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s36AvEFr064633; Sun, 6 Apr 2014 17:57:15 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <5341330A.70603@grosbein.net> Date: Sun, 06 Apr 2014 17:57:14 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Brett Glass Subject: Re: IPFW and VLANs References: <201404060226.UAA10958@mail.lariat.net> In-Reply-To: <201404060226.UAA10958@mail.lariat.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 10:57:44 -0000 On 06.04.2014 09:26, Brett Glass wrote: > I added a few more rules, with "recv" and "xmit" options, and > checked the counts again after zeroing them and letting the router > run for a bit: > > 00001 20591 8769298 count ip from any to any layer2 via re0_1 > 00002 18715 8725085 count ip from any to any layer2 via re0 > 00003 0 0 count ip from any to any not layer2 via re0 > 00004 18715 8725085 count ip from any to any layer2 recv re0 > 00005 18715 8725085 count ip from any to any layer2 xmit re0 > 00006 12746 1324342 count ip from any to any layer2 recv re0_1 > 00007 20592 8770798 count ip from any to any layer2 xmit re0_1 > > Maybe I am missing something (as I often do), but this seems just plain wrong. > > What gives? Help in interpreting these results would be much appreciated. You should use "in recv" and "out xmit" instead of just recv/xmit as routed packet will match BOTH of "recv $in_if" and "xmit $out_if". From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 11:02:50 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9880ED1 for ; Sun, 6 Apr 2014 11:02:50 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4988D1DB for ; Sun, 6 Apr 2014 11:02:49 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s36B2ja9082297 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Apr 2014 13:02:46 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: net@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s36B2f8t064764; Sun, 6 Apr 2014 18:02:41 +0700 (NOVT) (envelope-from eugen@grosbein.net) Message-ID: <53413451.4080400@grosbein.net> Date: Sun, 06 Apr 2014 18:02:41 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Brett Glass Subject: Re: More on odd IPFW behavior References: <201404060427.WAA11607@mail.lariat.net> In-Reply-To: <201404060427.WAA11607@mail.lariat.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=3.2 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * 2.9 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-Spam-Level: *** Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 11:02:50 -0000 On 06.04.2014 11:27, Brett Glass wrote: > A bit more investigation of IPFW's behavior on VLAN interfaces has > revealed some even stranger stuff. Consider the tallies on the > following firewall rules: > > # ipfw show | head > 00001 65071 36685513 count ip from any to any layer2 via re0 > 00002 65303 36856334 count ip from any to any layer2 via re0_1 > 00003 6 3381 count ip from any to any layer2 via re0_2 > 00004 49338 35208527 count ip from any to any layer2 via re0_3 > 00005 0 0 count ip from any to any layer2 in recv re0 > 00006 65071 36685513 count ip from any to any layer2 out xmit re0 > 00007 0 0 count ip from any to any layer2 in recv re0_1 > 00008 65303 36856334 count ip from any to any layer2 out xmit re0_1 > > It looks as if, when one adds "in" and "out" to the rules, one > never sees any Layer 2 packets coming "in" on either a vlan(4) > interface or its parent. There might be a problem with general > brokenness in IPFW's "in" and "out" qualifiers when dealing with > Layer 2 packets, or something else might be wrong.... Not sure, but > this behavior is definitely weird. And note that, again, re0_1 (a > child interface) shows more packets than re0 (the parent). Weird. > Do not have experience with pf, so do not know if it would do > better, but IPFW certainly has something broken. Help in figuring > out what to propose as a patch would be MUCH appreciated. Try to replace "count" with "allow" in the rule 6 and "count" with "count log" in the rule 8 and look at /var/log/security to find out "extra" packets that hit rule 8. From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 11:42:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7C055E1 for ; Sun, 6 Apr 2014 11:42:46 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7356E6CB for ; Sun, 6 Apr 2014 11:42:46 +0000 (UTC) Received: from [192.168.1.103] (p508F3C2C.dip0.t-ipconnect.de [80.143.60.44]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 92A571C0E9788; Sun, 6 Apr 2014 13:42:42 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: Michael Tuexen In-Reply-To: <20140405210246.GB58138@cicely7.cicely.de> Date: Sun, 6 Apr 2014 13:42:36 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> References: <20140405210246.GB58138@cicely7.cicely.de> To: ticso@cicely.de X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 11:42:47 -0000 On 05 Apr 2014, at 23:02, Bernd Walter wrote: > So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. > The modifications are to allow IPv6 multicast support within jail > which only makes a difference for multicast addresses and some = multicast > loopback checksum bugs - both changes are open PR. >=20 > I've created an AF_INET6 SCTP one to many socket to receive incoming > messages. > The process was started within a jail. > Now netstat -anW lists all host IPv6 IPs, not just those of the jail. > Also not sure why this AF_INET6 socket is shown as sctp46. This should be handled as a v6 only socket depending on your setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. However, netstat has no information about this and can not distinguish between sctp6 and sctp46, so it reports sctp46 always. You can file a PR about this. The questions about the addresses and the jails: The SCTP code has no jail specific code. If you bind a socket to the wildcard address (which is what to do by not binding at all), the SCTP stack lists all addresses it know about. I'm not sure what would happen, if you send a packet to an address not owned by the jail. You might want to file a separate PR about the support of jails. Best regards Michael >=20 > This is the relevant C++ code part to open the socket: > int > setup_sctp_socket(uint16_t port) > { > int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); > { > // reuse address > long val =3D 1; > setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); > // XXX error handling > } > { > // no delay > long val =3D 1; > setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); > // XXX error handling > } > { > // eeor mode (last write needs MSG_EOR to declare end = of message) > // Linux has MSG_MORE negative send flag > long val =3D 1; > setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); > // XXX error handling > } > #if 0 > { > struct sctp_initmsg init; > bzero(&init, sizeof(init)); > init.sinit_num_ostreams =3D HDB_STREAMS; > init.sinit_max_instreams =3D HDB_STREAMS; > // SOL_SCTP instead of IPPROTO_SCTP on Linux > setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); > // XXX error handling > } > #endif > { > struct sockaddr_in6 addr; > bzero(&addr, sizeof(addr)); > addr.sin6_len =3D sizeof(addr); > addr.sin6_family =3D AF_INET6; > addr.sin6_port =3D htons(port); > bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); > // XXX error handling > } > { > // enable heartbeats at 1000ms > struct sctp_paddrparams paddr_params; > bzero(&paddr_params, sizeof(paddr_params)); > paddr_params.spp_address.ss_family =3D AF_INET6; > paddr_params.spp_flags =3D SPP_HB_ENABLE; > paddr_params.spp_hbinterval =3D 1000; > // SOL_SCTP instead of IPPROTO_SCTP on Linux > setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 > // XXX error handling > } > { > struct sctp_event_subscribe events; > bzero(&events, sizeof(events)); >=20 > events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >=20 > // subscribe to other events as well for testing > events.sctp_association_event =3D 1; > events.sctp_address_event =3D 1; > events.sctp_send_failure_event =3D 1; > events.sctp_peer_error_event =3D 1; > events.sctp_shutdown_event =3D 1; > events.sctp_partial_delivery_event =3D 1; > events.sctp_adaptation_layer_event =3D 1; > events.sctp_authentication_event =3D 1; > events.sctp_sender_dry_event =3D 1; > events.sctp_stream_reset_event =3D 1; >=20 > setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); > // XXX error handling > } > { > // setup send and receive buffers (default on FreeBSD = 9.x) > long val; > val =3D 1864135; > setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); > // XXX error handling > val =3D 1864135; > setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); > // XXX error handling > } > listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue > // XXX error handling >=20 > return sc; > } >=20 > --=20 > B.Walter http://www.bwct.de > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 13:36:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24A5250B for ; Sun, 6 Apr 2014 13:36:52 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EAC0FE86 for ; Sun, 6 Apr 2014 13:36:51 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s36Daj7S056514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 6 Apr 2014 06:36:48 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53415866.1030107@freebsd.org> Date: Sun, 06 Apr 2014 21:36:38 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Chris Smith , freebsd-net@freebsd.org Subject: Re: Multihomed system with jails routing issues References: <533F68EF.8060607@nevermind.co.nz> <53402D68.4030500@freebsd.org> <53411885.7030206@nevermind.co.nz> In-Reply-To: <53411885.7030206@nevermind.co.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 13:36:52 -0000 On 4/6/14, 5:04 PM, Chris Smith wrote: > On 06/04/14 04:20, Julian Elischer wrote: >> On 4/5/14, 10:22 AM, Chris Smith wrote: >>> Hi All, >>> >>> I have a system with 1 network interface with 2 extra VLANs off it >>> and I'm having some trouble getting the routing working correctly >>> with it and jails. >>> >>> bge0 - management - 10.71.100.0/24 >>> bge0.101 - LAN - 10.71.101.0/24 >>> bge0.103 - DMZ - 10.71.101.0/24 >>> >>> Here's what I want to achieve... >>> >>> Host: >>> I want the host system to only listen on one interface, bge0. I >>> want NO ip addresses of the host on the vlan interfaces. The only >>> service it will be exposing is its sshd. The management address >>> for this system is 10.71.100.50. >>> >> Sounds to me that you want to use vimage jails. >> check the vnet command to jail . >> > Hey Julian, > > Thanks for that. I did come across it but all of the documentation I > found indicated that it was experimental. > > After a day or so messing around with VIMAGE/vnet and their various > gotchas and interactions with jails on FreeBSD 10, I have something > working that I'm happy with. as long as you steer clear of pf and do only 'vanilla' stuff, you should be ok. let us know what you think and I'd like to see your notes published, if not officially then at least put here so that others can find it in the archives. > > I've made a bunch of notes so I hope to write something up for it > since most of the documentation around this is thin, old or outdated. > > Cheers, > Chris. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 15:06:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EB4F51E for ; Sun, 6 Apr 2014 15:06:14 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D2E936EE for ; Sun, 6 Apr 2014 15:06:13 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 6243225D3897; Sun, 6 Apr 2014 15:06:04 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 94B7AC22BA7; Sun, 6 Apr 2014 15:06:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 2WsZK6TJZQcV; Sun, 6 Apr 2014 15:06:01 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493] (unknown [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 7F39FC22B9B; Sun, 6 Apr 2014 15:05:59 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: "Bjoern A. Zeeb" In-Reply-To: <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> Date: Sun, 6 Apr 2014 15:05:50 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> To: Michael Tuexen X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 15:06:14 -0000 On 06 Apr 2014, at 11:42 , Michael Tuexen = wrote: > On 05 Apr 2014, at 23:02, Bernd Walter = wrote: >=20 >> So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. >> The modifications are to allow IPv6 multicast support within jail >> which only makes a difference for multicast addresses and some = multicast >> loopback checksum bugs - both changes are open PR. >>=20 >> I've created an AF_INET6 SCTP one to many socket to receive incoming >> messages. >> The process was started within a jail. >> Now netstat -anW lists all host IPv6 IPs, not just those of the jail. >> Also not sure why this AF_INET6 socket is shown as sctp46. > This should be handled as a v6 only socket depending on your > setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. > However, netstat has no information about this and can not distinguish > between sctp6 and sctp46, so it reports sctp46 always. You can file > a PR about this. >=20 > The questions about the addresses and the jails: The SCTP code has > no jail specific code. If you bind a socket to the wildcard address > (which is what to do by not binding at all), the SCTP stack lists > all addresses it know about. I'm not sure what would happen, if > you send a packet to an address not owned by the jail. > You might want to file a separate PR about the support of jails. Aehm, the SCTP code was filtering addresses at one point and made sure = only jail-visible addresses were seen or bound very much like normal PCB = handling. If this is not the case (anymore) SCTP shall not be allowed = inside jails again.=20 >=20 > Best regards > Michael >>=20 >> This is the relevant C++ code part to open the socket: >> int >> setup_sctp_socket(uint16_t port) >> { >> int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); >> { >> // reuse address >> long val =3D 1; >> setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); >> // XXX error handling >> } >> { >> // no delay >> long val =3D 1; >> setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); >> // XXX error handling >> } >> { >> // eeor mode (last write needs MSG_EOR to declare end = of message) >> // Linux has MSG_MORE negative send flag >> long val =3D 1; >> setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); >> // XXX error handling >> } >> #if 0 >> { >> struct sctp_initmsg init; >> bzero(&init, sizeof(init)); >> init.sinit_num_ostreams =3D HDB_STREAMS; >> init.sinit_max_instreams =3D HDB_STREAMS; >> // SOL_SCTP instead of IPPROTO_SCTP on Linux >> setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); >> // XXX error handling >> } >> #endif >> { >> struct sockaddr_in6 addr; >> bzero(&addr, sizeof(addr)); >> addr.sin6_len =3D sizeof(addr); >> addr.sin6_family =3D AF_INET6; >> addr.sin6_port =3D htons(port); >> bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); >> // XXX error handling >> } >> { >> // enable heartbeats at 1000ms >> struct sctp_paddrparams paddr_params; >> bzero(&paddr_params, sizeof(paddr_params)); >> paddr_params.spp_address.ss_family =3D AF_INET6; >> paddr_params.spp_flags =3D SPP_HB_ENABLE; >> paddr_params.spp_hbinterval =3D 1000; >> // SOL_SCTP instead of IPPROTO_SCTP on Linux >> setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 >> // XXX error handling >> } >> { >> struct sctp_event_subscribe events; >> bzero(&events, sizeof(events)); >>=20 >> events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >>=20 >> // subscribe to other events as well for testing >> events.sctp_association_event =3D 1; >> events.sctp_address_event =3D 1; >> events.sctp_send_failure_event =3D 1; >> events.sctp_peer_error_event =3D 1; >> events.sctp_shutdown_event =3D 1; >> events.sctp_partial_delivery_event =3D 1; >> events.sctp_adaptation_layer_event =3D 1; >> events.sctp_authentication_event =3D 1; >> events.sctp_sender_dry_event =3D 1; >> events.sctp_stream_reset_event =3D 1; >>=20 >> setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); >> // XXX error handling >> } >> { >> // setup send and receive buffers (default on FreeBSD = 9.x) >> long val; >> val =3D 1864135; >> setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); >> // XXX error handling >> val =3D 1864135; >> setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); >> // XXX error handling >> } >> listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue >> // XXX error handling >>=20 >> return sc; >> } >>=20 >> --=20 >> B.Walter http://www.bwct.de >> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" =97=20 Bjoern A. Zeeb ????????? ??? ??????? ??????: '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? ???? ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", ?.??? From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 16:36:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FD1C20A; Sun, 6 Apr 2014 16:36:40 +0000 (UTC) Received: from tampoco.espindola.nl (tampoco.espindola.nl [149.210.133.191]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82980E58; Sun, 6 Apr 2014 16:36:38 +0000 (UTC) Received: from [127.0.0.1] (corfu.internal.deze.org [192.168.1.2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: frank) by tampoco.espindola.nl (Postfix) with ESMTPSA id 00E4B371E93; Sun, 6 Apr 2014 18:26:57 +0200 (CEST) Message-ID: <53418053.5020105@deze.org> Date: Sun, 06 Apr 2014 18:26:59 +0200 From: Frank Volf User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: re0: watchdog timeout Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 16:36:40 -0000 Hello, I'm experiencing watchdog timeouts with my Realtek interface card. I'm using a fairly new system (Shuttle DS47), running FreeBSD 10-STABLE. For this shuttle a patch has been recently committed to SVN to make this card work at all (revision *262391* ). The timeout is only experienced under heavy network load (the system is running a bacula backup server that backups to NFS connected storage), and typically large full backups trigger this. Normal traffic works fine (this system is e.g. also my firewall to the Internet). What might not be standard is that I use sub-interfaces on this system. First of all, the only way that I can get the sub-interfaces to work at all is by using ifconfig_re0="-vlanhwtag" I'm not sure that is related. The question is how can we debug this to solve the problem? I have non clue, but I'm happy to assist if somebody can tell me what I should do. Some information that might be useful: root@drawbridge:/usr/local/etc/bacula # dmesg | grep re0 re0: port 0xd000-0xd0ff mem 0xf7a00000-0xf7a00fff,0xf0100000-0xf0103fff irq 17 at device 0.0 on pci2 re0: Using 1 MSI-X message re0: ASPM disabled re0: Chip rev. 0x4c000000 re0: MAC rev. 0x00000000 miibus0: on re0 re0: Ethernet address: 80:ee:73:77:e9:ab re0: watchdog timeout re0: link state changed to DOWN re0.98: link state changed to DOWN re0.10: link state changed to DOWN re0.11: link state changed to DOWN re0.12: link state changed to DOWN re0: link state changed to UP re0.98: link state changed to UP re0.10: link state changed to UP re0.11: link state changed to UP re0.12: link state changed to UP ... root@drawbridge:/usr/local/etc/bacula # uname -a FreeBSD drawbridge.internal.deze.org 10.0-STABLE FreeBSD 10.0-STABLE #0 r262433: Mon Feb 24 16:25:35 CET 2014 root@drawbridge-new.internal.deze.org:/usr/obj/usr/sources/src10-stable/sys/SHUTTLE i386 root@drawbridge:/usr/local/etc/bacula # pciconf -lv re0 re0@pci0:2:0:0: class=0x020000 card=0x40181297 chip=0x816810ec rev=0x0c hdr=0x00 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' class = network subclass = ethernet Kind regards, Frank From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 16:42:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84A765B3 for ; Sun, 6 Apr 2014 16:42:42 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10F55F25 for ; Sun, 6 Apr 2014 16:42:42 +0000 (UTC) Received: from [192.168.1.103] (p508F3C2C.dip0.t-ipconnect.de [80.143.60.44]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id C23901C104664; Sun, 6 Apr 2014 18:42:37 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: Michael Tuexen In-Reply-To: <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> Date: Sun, 6 Apr 2014 18:42:34 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 16:42:42 -0000 On 06 Apr 2014, at 17:05, Bjoern A. Zeeb = wrote: >=20 > On 06 Apr 2014, at 11:42 , Michael Tuexen = wrote: >=20 >> On 05 Apr 2014, at 23:02, Bernd Walter = wrote: >>=20 >>> So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. >>> The modifications are to allow IPv6 multicast support within jail >>> which only makes a difference for multicast addresses and some = multicast >>> loopback checksum bugs - both changes are open PR. >>>=20 >>> I've created an AF_INET6 SCTP one to many socket to receive incoming >>> messages. >>> The process was started within a jail. >>> Now netstat -anW lists all host IPv6 IPs, not just those of the = jail. >>> Also not sure why this AF_INET6 socket is shown as sctp46. >> This should be handled as a v6 only socket depending on your >> setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. >> However, netstat has no information about this and can not = distinguish >> between sctp6 and sctp46, so it reports sctp46 always. You can file >> a PR about this. >>=20 >> The questions about the addresses and the jails: The SCTP code has >> no jail specific code. If you bind a socket to the wildcard address >> (which is what to do by not binding at all), the SCTP stack lists >> all addresses it know about. I'm not sure what would happen, if >> you send a packet to an address not owned by the jail. >> You might want to file a separate PR about the support of jails. >=20 > Aehm, the SCTP code was filtering addresses at one point and made sure = only jail-visible addresses were seen or bound very much like normal PCB = handling. If this is not the case (anymore) SCTP shall not be allowed = inside jails again.=20 Can you point me to the "normal PCB handling"? Maybe I'm just = overlooking something... Best regards Michael >=20 >=20 >=20 >=20 >>=20 >> Best regards >> Michael >>>=20 >>> This is the relevant C++ code part to open the socket: >>> int >>> setup_sctp_socket(uint16_t port) >>> { >>> int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); >>> { >>> // reuse address >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> { >>> // no delay >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> { >>> // eeor mode (last write needs MSG_EOR to declare end = of message) >>> // Linux has MSG_MORE negative send flag >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> #if 0 >>> { >>> struct sctp_initmsg init; >>> bzero(&init, sizeof(init)); >>> init.sinit_num_ostreams =3D HDB_STREAMS; >>> init.sinit_max_instreams =3D HDB_STREAMS; >>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>> setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); >>> // XXX error handling >>> } >>> #endif >>> { >>> struct sockaddr_in6 addr; >>> bzero(&addr, sizeof(addr)); >>> addr.sin6_len =3D sizeof(addr); >>> addr.sin6_family =3D AF_INET6; >>> addr.sin6_port =3D htons(port); >>> bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); >>> // XXX error handling >>> } >>> { >>> // enable heartbeats at 1000ms >>> struct sctp_paddrparams paddr_params; >>> bzero(&paddr_params, sizeof(paddr_params)); >>> paddr_params.spp_address.ss_family =3D AF_INET6; >>> paddr_params.spp_flags =3D SPP_HB_ENABLE; >>> paddr_params.spp_hbinterval =3D 1000; >>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>> setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 >>> // XXX error handling >>> } >>> { >>> struct sctp_event_subscribe events; >>> bzero(&events, sizeof(events)); >>>=20 >>> events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >>>=20 >>> // subscribe to other events as well for testing >>> events.sctp_association_event =3D 1; >>> events.sctp_address_event =3D 1; >>> events.sctp_send_failure_event =3D 1; >>> events.sctp_peer_error_event =3D 1; >>> events.sctp_shutdown_event =3D 1; >>> events.sctp_partial_delivery_event =3D 1; >>> events.sctp_adaptation_layer_event =3D 1; >>> events.sctp_authentication_event =3D 1; >>> events.sctp_sender_dry_event =3D 1; >>> events.sctp_stream_reset_event =3D 1; >>>=20 >>> setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); >>> // XXX error handling >>> } >>> { >>> // setup send and receive buffers (default on FreeBSD = 9.x) >>> long val; >>> val =3D 1864135; >>> setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); >>> // XXX error handling >>> val =3D 1864135; >>> setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue >>> // XXX error handling >>>=20 >>> return sc; >>> } >>>=20 >>> --=20 >>> B.Walter http://www.bwct.de >>> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner = uvm. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >=20 > =97=20 > Bjoern A. Zeeb ????????? ??? ??????? = ??????: > '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? > ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 17:04:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFD95C6A for ; Sun, 6 Apr 2014 17:04:53 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79793151 for ; Sun, 6 Apr 2014 17:04:53 +0000 (UTC) Received: from [192.168.1.103] (p508F3C2C.dip0.t-ipconnect.de [80.143.60.44]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 74DA11C0B4626; Sun, 6 Apr 2014 19:04:50 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: Michael Tuexen In-Reply-To: <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> Date: Sun, 6 Apr 2014 19:04:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <5785F386-DC41-4D0A-BBBE-6DA935095451@lurchi.franken.de> References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 17:04:54 -0000 On 06 Apr 2014, at 17:05, Bjoern A. Zeeb = wrote: >=20 > On 06 Apr 2014, at 11:42 , Michael Tuexen = wrote: >=20 >> On 05 Apr 2014, at 23:02, Bernd Walter = wrote: >>=20 >>> So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. >>> The modifications are to allow IPv6 multicast support within jail >>> which only makes a difference for multicast addresses and some = multicast >>> loopback checksum bugs - both changes are open PR. >>>=20 >>> I've created an AF_INET6 SCTP one to many socket to receive incoming >>> messages. >>> The process was started within a jail. >>> Now netstat -anW lists all host IPv6 IPs, not just those of the = jail. >>> Also not sure why this AF_INET6 socket is shown as sctp46. >> This should be handled as a v6 only socket depending on your >> setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. >> However, netstat has no information about this and can not = distinguish >> between sctp6 and sctp46, so it reports sctp46 always. You can file >> a PR about this. >>=20 >> The questions about the addresses and the jails: The SCTP code has >> no jail specific code. If you bind a socket to the wildcard address >> (which is what to do by not binding at all), the SCTP stack lists >> all addresses it know about. I'm not sure what would happen, if >> you send a packet to an address not owned by the jail. >> You might want to file a separate PR about the support of jails. >=20 > Aehm, the SCTP code was filtering addresses at one point and made sure = only jail-visible addresses were seen or bound very much like normal PCB = handling. If this is not the case (anymore) SCTP shall not be allowed = inside jails again.=20 Are you referring to prison_local_ip4() and prison_local_ip6() calls? These are used while explicit binding. However, I don't think we do the corresponding filtering when sending INIT-/INIT-ACKs or export the list of address via the sysctl interface used by netstat. I guess this needs to be added, right? Best regards Michael >=20 >=20 >=20 >=20 >>=20 >> Best regards >> Michael >>>=20 >>> This is the relevant C++ code part to open the socket: >>> int >>> setup_sctp_socket(uint16_t port) >>> { >>> int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); >>> { >>> // reuse address >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> { >>> // no delay >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> { >>> // eeor mode (last write needs MSG_EOR to declare end = of message) >>> // Linux has MSG_MORE negative send flag >>> long val =3D 1; >>> setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> #if 0 >>> { >>> struct sctp_initmsg init; >>> bzero(&init, sizeof(init)); >>> init.sinit_num_ostreams =3D HDB_STREAMS; >>> init.sinit_max_instreams =3D HDB_STREAMS; >>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>> setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); >>> // XXX error handling >>> } >>> #endif >>> { >>> struct sockaddr_in6 addr; >>> bzero(&addr, sizeof(addr)); >>> addr.sin6_len =3D sizeof(addr); >>> addr.sin6_family =3D AF_INET6; >>> addr.sin6_port =3D htons(port); >>> bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); >>> // XXX error handling >>> } >>> { >>> // enable heartbeats at 1000ms >>> struct sctp_paddrparams paddr_params; >>> bzero(&paddr_params, sizeof(paddr_params)); >>> paddr_params.spp_address.ss_family =3D AF_INET6; >>> paddr_params.spp_flags =3D SPP_HB_ENABLE; >>> paddr_params.spp_hbinterval =3D 1000; >>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>> setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 >>> // XXX error handling >>> } >>> { >>> struct sctp_event_subscribe events; >>> bzero(&events, sizeof(events)); >>>=20 >>> events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >>>=20 >>> // subscribe to other events as well for testing >>> events.sctp_association_event =3D 1; >>> events.sctp_address_event =3D 1; >>> events.sctp_send_failure_event =3D 1; >>> events.sctp_peer_error_event =3D 1; >>> events.sctp_shutdown_event =3D 1; >>> events.sctp_partial_delivery_event =3D 1; >>> events.sctp_adaptation_layer_event =3D 1; >>> events.sctp_authentication_event =3D 1; >>> events.sctp_sender_dry_event =3D 1; >>> events.sctp_stream_reset_event =3D 1; >>>=20 >>> setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); >>> // XXX error handling >>> } >>> { >>> // setup send and receive buffers (default on FreeBSD = 9.x) >>> long val; >>> val =3D 1864135; >>> setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); >>> // XXX error handling >>> val =3D 1864135; >>> setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); >>> // XXX error handling >>> } >>> listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue >>> // XXX error handling >>>=20 >>> return sc; >>> } >>>=20 >>> --=20 >>> B.Walter http://www.bwct.de >>> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner = uvm. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >=20 > =97=20 > Bjoern A. Zeeb ????????? ??? ??????? = ??????: > '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? > ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 18:44:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 712C84F5 for ; Sun, 6 Apr 2014 18:44:37 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 18AC7E59 for ; Sun, 6 Apr 2014 18:44:36 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 850C225D3AB9; Sun, 6 Apr 2014 18:44:32 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 91C13C22BA7; Sun, 6 Apr 2014 18:44:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id ggleoK82JJz3; Sun, 6 Apr 2014 18:44:28 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493] (unknown [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 8A900C22B9B; Sun, 6 Apr 2014 18:44:26 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: "Bjoern A. Zeeb" In-Reply-To: Date: Sun, 6 Apr 2014 18:44:05 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <798F4E1E-693B-4B94-847D-2A2106A47C0A@lists.zabbadoz.net> References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> To: Michael Tuexen X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 18:44:37 -0000 On 06 Apr 2014, at 16:42 , Michael Tuexen = wrote: > On 06 Apr 2014, at 17:05, Bjoern A. Zeeb = wrote: >=20 >>=20 >> On 06 Apr 2014, at 11:42 , Michael Tuexen = wrote: >>=20 >>> On 05 Apr 2014, at 23:02, Bernd Walter = wrote: >>>=20 >>>> So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. >>>> The modifications are to allow IPv6 multicast support within jail >>>> which only makes a difference for multicast addresses and some = multicast >>>> loopback checksum bugs - both changes are open PR. >>>>=20 >>>> I've created an AF_INET6 SCTP one to many socket to receive = incoming >>>> messages. >>>> The process was started within a jail. >>>> Now netstat -anW lists all host IPv6 IPs, not just those of the = jail. >>>> Also not sure why this AF_INET6 socket is shown as sctp46. >>> This should be handled as a v6 only socket depending on your >>> setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. >>> However, netstat has no information about this and can not = distinguish >>> between sctp6 and sctp46, so it reports sctp46 always. You can file >>> a PR about this. >>>=20 >>> The questions about the addresses and the jails: The SCTP code has >>> no jail specific code. If you bind a socket to the wildcard address >>> (which is what to do by not binding at all), the SCTP stack lists >>> all addresses it know about. I'm not sure what would happen, if >>> you send a packet to an address not owned by the jail. >>> You might want to file a separate PR about the support of jails. >>=20 >> Aehm, the SCTP code was filtering addresses at one point and made = sure only jail-visible addresses were seen or bound very much like = normal PCB handling. If this is not the case (anymore) SCTP shall not = be allowed inside jails again.=20 > Can you point me to the "normal PCB handling"? Maybe I'm just = overlooking something=85 I guess what helps you more is looking for prison_* calls in the SCTP = stack (and equally in in*_pcb*, tcp_*, udp_*). >>> Best regards >>> Michael >>>>=20 >>>> This is the relevant C++ code part to open the socket: >>>> int >>>> setup_sctp_socket(uint16_t port) >>>> { >>>> int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); >>>> { >>>> // reuse address >>>> long val =3D 1; >>>> setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); >>>> // XXX error handling >>>> } >>>> { >>>> // no delay >>>> long val =3D 1; >>>> setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); >>>> // XXX error handling >>>> } >>>> { >>>> // eeor mode (last write needs MSG_EOR to declare end = of message) >>>> // Linux has MSG_MORE negative send flag >>>> long val =3D 1; >>>> setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); >>>> // XXX error handling >>>> } >>>> #if 0 >>>> { >>>> struct sctp_initmsg init; >>>> bzero(&init, sizeof(init)); >>>> init.sinit_num_ostreams =3D HDB_STREAMS; >>>> init.sinit_max_instreams =3D HDB_STREAMS; >>>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>>> setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); >>>> // XXX error handling >>>> } >>>> #endif >>>> { >>>> struct sockaddr_in6 addr; >>>> bzero(&addr, sizeof(addr)); >>>> addr.sin6_len =3D sizeof(addr); >>>> addr.sin6_family =3D AF_INET6; >>>> addr.sin6_port =3D htons(port); >>>> bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); >>>> // XXX error handling >>>> } >>>> { >>>> // enable heartbeats at 1000ms >>>> struct sctp_paddrparams paddr_params; >>>> bzero(&paddr_params, sizeof(paddr_params)); >>>> paddr_params.spp_address.ss_family =3D AF_INET6; >>>> paddr_params.spp_flags =3D SPP_HB_ENABLE; >>>> paddr_params.spp_hbinterval =3D 1000; >>>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>>> setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 >>>> // XXX error handling >>>> } >>>> { >>>> struct sctp_event_subscribe events; >>>> bzero(&events, sizeof(events)); >>>>=20 >>>> events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >>>>=20 >>>> // subscribe to other events as well for testing >>>> events.sctp_association_event =3D 1; >>>> events.sctp_address_event =3D 1; >>>> events.sctp_send_failure_event =3D 1; >>>> events.sctp_peer_error_event =3D 1; >>>> events.sctp_shutdown_event =3D 1; >>>> events.sctp_partial_delivery_event =3D 1; >>>> events.sctp_adaptation_layer_event =3D 1; >>>> events.sctp_authentication_event =3D 1; >>>> events.sctp_sender_dry_event =3D 1; >>>> events.sctp_stream_reset_event =3D 1; >>>>=20 >>>> setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); >>>> // XXX error handling >>>> } >>>> { >>>> // setup send and receive buffers (default on FreeBSD = 9.x) >>>> long val; >>>> val =3D 1864135; >>>> setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); >>>> // XXX error handling >>>> val =3D 1864135; >>>> setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); >>>> // XXX error handling >>>> } >>>> listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue >>>> // XXX error handling >>>>=20 >>>> return sc; >>>> } >>>>=20 >>>> --=20 >>>> B.Walter http://www.bwct.de >>>> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner = uvm. >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>=20 >>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 >> =97=20 >> Bjoern A. Zeeb ????????? ??? ??????? = ??????: >> '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? >> ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >>=20 >>=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" =97=20 Bjoern A. Zeeb ????????? ??? ??????? ??????: '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? ???? ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", ?.??? From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 18:44:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3B8557C for ; Sun, 6 Apr 2014 18:44:44 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 74A34E5A for ; Sun, 6 Apr 2014 18:44:44 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id EFFDB25D3AC0; Sun, 6 Apr 2014 18:44:41 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 780ECC22BA8; Sun, 6 Apr 2014 18:44:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id Ewq7fq_hJKb7; Sun, 6 Apr 2014 18:44:40 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493] (unknown [IPv6:fde9:577b:c1a9:4410:395f:c902:48ef:f493]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 65F47C22B9B; Sun, 6 Apr 2014 18:44:38 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: "Bjoern A. Zeeb" In-Reply-To: <5785F386-DC41-4D0A-BBBE-6DA935095451@lurchi.franken.de> Date: Sun, 6 Apr 2014 18:44:50 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> <5785F386-DC41-4D0A-BBBE-6DA935095451@lurchi.franken.de> To: Michael Tuexen X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 18:44:44 -0000 On 06 Apr 2014, at 17:04 , Michael Tuexen = wrote: >> Aehm, the SCTP code was filtering addresses at one point and made = sure only jail-visible addresses were seen or bound very much like = normal PCB handling. If this is not the case (anymore) SCTP shall not = be allowed inside jails again.=20 > Are you referring to prison_local_ip4() and prison_local_ip6() calls? > These are used while explicit binding. However, I don't think we > do the corresponding filtering when sending INIT-/INIT-ACKs or > export the list of address via the sysctl interface used by netstat. > I guess this needs to be added, right? Yes. =97=20 Bjoern A. Zeeb ????????? ??? ??????? ??????: '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? ???? ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", ?.??? From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 19:44:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B09A1B1 for ; Sun, 6 Apr 2014 19:44:58 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37BC9321 for ; Sun, 6 Apr 2014 19:44:58 +0000 (UTC) Received: from [192.168.1.103] (p508F3041.dip0.t-ipconnect.de [80.143.48.65]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 5E6371C10466A; Sun, 6 Apr 2014 21:44:54 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: Michael Tuexen In-Reply-To: Date: Sun, 6 Apr 2014 21:44:52 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> <5785F386-DC41-4D0A-BBBE-6DA935095451@lurchi.franken.de> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 19:44:58 -0000 On 06 Apr 2014, at 20:44, Bjoern A. Zeeb = wrote: >=20 > On 06 Apr 2014, at 17:04 , Michael Tuexen = wrote: >=20 >>> Aehm, the SCTP code was filtering addresses at one point and made = sure only jail-visible addresses were seen or bound very much like = normal PCB handling. If this is not the case (anymore) SCTP shall not = be allowed inside jails again.=20 >> Are you referring to prison_local_ip4() and prison_local_ip6() calls? >> These are used while explicit binding. However, I don't think we >> do the corresponding filtering when sending INIT-/INIT-ACKs or >> export the list of address via the sysctl interface used by netstat. >> I guess this needs to be added, right? >=20 > Yes. OK. Give me a couple of days and I'll try to fix the SCTP stack (need to set up a test environment for it). Best regards Michael >=20 > =97=20 > Bjoern A. Zeeb ????????? ??? ??????? = ??????: > '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? > ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 19:46:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5086F258 for ; Sun, 6 Apr 2014 19:46:39 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF51333B for ; Sun, 6 Apr 2014 19:46:38 +0000 (UTC) Received: from [192.168.1.103] (p508F3041.dip0.t-ipconnect.de [80.143.48.65]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id C56EA1C0E96FA; Sun, 6 Apr 2014 21:46:35 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: SCTP binds to IPs outside of jail From: Michael Tuexen In-Reply-To: <798F4E1E-693B-4B94-847D-2A2106A47C0A@lists.zabbadoz.net> Date: Sun, 6 Apr 2014 21:46:33 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140405210246.GB58138@cicely7.cicely.de> <7D1ABA78-D48D-48B7-9CE7-152BD59DB1B0@lurchi.franken.de> <77B6DEC1-D7E8-446E-A057-A692379D9EFB@lists.zabbadoz.net> <798F4E1E-693B-4B94-847D-2A2106A47C0A@lists.zabbadoz.net> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 19:46:39 -0000 On 06 Apr 2014, at 20:44, Bjoern A. Zeeb = wrote: >=20 > On 06 Apr 2014, at 16:42 , Michael Tuexen = wrote: >=20 >> On 06 Apr 2014, at 17:05, Bjoern A. Zeeb = wrote: >>=20 >>>=20 >>> On 06 Apr 2014, at 11:42 , Michael Tuexen = wrote: >>>=20 >>>> On 05 Apr 2014, at 23:02, Bernd Walter = wrote: >>>>=20 >>>>> So far I've tested this on FreeBSD-9.2 BETA2 r254053M only. >>>>> The modifications are to allow IPv6 multicast support within jail >>>>> which only makes a difference for multicast addresses and some = multicast >>>>> loopback checksum bugs - both changes are open PR. >>>>>=20 >>>>> I've created an AF_INET6 SCTP one to many socket to receive = incoming >>>>> messages. >>>>> The process was started within a jail. >>>>> Now netstat -anW lists all host IPv6 IPs, not just those of the = jail. >>>>> Also not sure why this AF_INET6 socket is shown as sctp46. >>>> This should be handled as a v6 only socket depending on your >>>> setting of net.inet6.ip6.v6only sysctl variable by the SCTP stack. >>>> However, netstat has no information about this and can not = distinguish >>>> between sctp6 and sctp46, so it reports sctp46 always. You can file >>>> a PR about this. >>>>=20 >>>> The questions about the addresses and the jails: The SCTP code has >>>> no jail specific code. If you bind a socket to the wildcard address >>>> (which is what to do by not binding at all), the SCTP stack lists >>>> all addresses it know about. I'm not sure what would happen, if >>>> you send a packet to an address not owned by the jail. >>>> You might want to file a separate PR about the support of jails. >>>=20 >>> Aehm, the SCTP code was filtering addresses at one point and made = sure only jail-visible addresses were seen or bound very much like = normal PCB handling. If this is not the case (anymore) SCTP shall not = be allowed inside jails again.=20 >> Can you point me to the "normal PCB handling"? Maybe I'm just = overlooking something=85 >=20 > I guess what helps you more is looking for prison_* calls in the SCTP = stack (and equally in in*_pcb*, tcp_*, udp_*). Thanks for the hint. Best regards Michael >=20 >=20 >=20 >>>> Best regards >>>> Michael >>>>>=20 >>>>> This is the relevant C++ code part to open the socket: >>>>> int >>>>> setup_sctp_socket(uint16_t port) >>>>> { >>>>> int sc =3D socket(AF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP); >>>>> { >>>>> // reuse address >>>>> long val =3D 1; >>>>> setsockopt(sc, SOL_SOCKET, SO_REUSEADDR, &val, = sizeof(val)); >>>>> // XXX error handling >>>>> } >>>>> { >>>>> // no delay >>>>> long val =3D 1; >>>>> setsockopt(sc, SOL_SOCKET, SCTP_NODELAY, &val, = sizeof(val)); >>>>> // XXX error handling >>>>> } >>>>> { >>>>> // eeor mode (last write needs MSG_EOR to declare end = of message) >>>>> // Linux has MSG_MORE negative send flag >>>>> long val =3D 1; >>>>> setsockopt(sc, SOL_SOCKET, SCTP_EXPLICIT_EOR, &val, = sizeof(val)); >>>>> // XXX error handling >>>>> } >>>>> #if 0 >>>>> { >>>>> struct sctp_initmsg init; >>>>> bzero(&init, sizeof(init)); >>>>> init.sinit_num_ostreams =3D HDB_STREAMS; >>>>> init.sinit_max_instreams =3D HDB_STREAMS; >>>>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>>>> setsockopt(sc, IPPROTO_SCTP, SCTP_INITMSG, &init, = (socklen_t)sizeof(struct sctp_initmsg)); >>>>> // XXX error handling >>>>> } >>>>> #endif >>>>> { >>>>> struct sockaddr_in6 addr; >>>>> bzero(&addr, sizeof(addr)); >>>>> addr.sin6_len =3D sizeof(addr); >>>>> addr.sin6_family =3D AF_INET6; >>>>> addr.sin6_port =3D htons(port); >>>>> bind(sc, (struct sockaddr *)&addr, sizeof(struct = sockaddr_in)); >>>>> // XXX error handling >>>>> } >>>>> { >>>>> // enable heartbeats at 1000ms >>>>> struct sctp_paddrparams paddr_params; >>>>> bzero(&paddr_params, sizeof(paddr_params)); >>>>> paddr_params.spp_address.ss_family =3D AF_INET6; >>>>> paddr_params.spp_flags =3D SPP_HB_ENABLE; >>>>> paddr_params.spp_hbinterval =3D 1000; >>>>> // SOL_SCTP instead of IPPROTO_SCTP on Linux >>>>> setsockopt(sc, IPPROTO_SCTP, SCTP_PEER_ADDR_PARAMS, = &paddr_params, sizeof(paddr_params));=20 >>>>> // XXX error handling >>>>> } >>>>> { >>>>> struct sctp_event_subscribe events; >>>>> bzero(&events, sizeof(events)); >>>>>=20 >>>>> events.sctp_data_io_event =3D 1; // we need io_events = to know where the message came from >>>>>=20 >>>>> // subscribe to other events as well for testing >>>>> events.sctp_association_event =3D 1; >>>>> events.sctp_address_event =3D 1; >>>>> events.sctp_send_failure_event =3D 1; >>>>> events.sctp_peer_error_event =3D 1; >>>>> events.sctp_shutdown_event =3D 1; >>>>> events.sctp_partial_delivery_event =3D 1; >>>>> events.sctp_adaptation_layer_event =3D 1; >>>>> events.sctp_authentication_event =3D 1; >>>>> events.sctp_sender_dry_event =3D 1; >>>>> events.sctp_stream_reset_event =3D 1; >>>>>=20 >>>>> setsockopt(sc, IPPROTO_SCTP, SCTP_EVENTS, &events, = sizeof(events)); >>>>> // XXX error handling >>>>> } >>>>> { >>>>> // setup send and receive buffers (default on FreeBSD = 9.x) >>>>> long val; >>>>> val =3D 1864135; >>>>> setsockopt(sc, SOL_SOCKET, SO_RCVBUF, &val, = sizeof(val)); >>>>> // XXX error handling >>>>> val =3D 1864135; >>>>> setsockopt(sc, SOL_SOCKET, SO_SNDBUF, &val, = sizeof(val)); >>>>> // XXX error handling >>>>> } >>>>> listen (sc, 1); // listen is required to allow incoming = associations, but no listen queue >>>>> // XXX error handling >>>>>=20 >>>>> return sc; >>>>> } >>>>>=20 >>>>> --=20 >>>>> B.Walter http://www.bwct.de >>>>> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner = uvm. >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>=20 >>>>=20 >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>> =97=20 >>> Bjoern A. Zeeb ????????? ??? ??????? = ??????: >>> '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? >>> ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >>>=20 >>>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >=20 > =97=20 > Bjoern A. Zeeb ????????? ??? ??????? = ??????: > '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? = ???? > ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", = ?.??? >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sun Apr 6 23:37:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57955689; Sun, 6 Apr 2014 23:37:16 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id D71A0B2D; Sun, 6 Apr 2014 23:37:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAADkQVODaFve/2dsb2JhbABYg0FXgwq5WIZmUYEudIIlAQEBAwEBAQEgKyALBRYYAgINGQIpAQkmBggHBAEcBIdQCA2pPqF/F4EpjF0IARACAQYVNAeCb4FJBJYEhAuRC4NMITF8QQ X-IronPort-AV: E=Sophos;i="4.97,805,1389762000"; d="scan'208";a="112222027" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 06 Apr 2014 19:37:08 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 864AAB4044; Sun, 6 Apr 2014 19:37:08 -0400 (EDT) Date: Sun, 6 Apr 2014 19:37:08 -0400 (EDT) From: Rick Macklem To: Frank Volf Message-ID: <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> In-Reply-To: <53418053.5020105@deze.org> Subject: Re: re0: watchdog timeout MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 23:37:16 -0000 Frank Volf wrote: > > Hello, > > I'm experiencing watchdog timeouts with my Realtek interface card. > > I'm using a fairly new system (Shuttle DS47), running FreeBSD > 10-STABLE. > For this shuttle a patch has been recently committed to SVN to make > this > card work at all (revision *262391* > ). > > The timeout is only experienced under heavy network load (the system > is > running a bacula backup server that backups to NFS connected > storage), > and typically large full backups trigger this. Normal traffic works > fine > (this system is e.g. also my firewall to the Internet). > Since you mention NFS, you could try disabling TSO on the interface and see if that helps. (I'm beginning to feel like a parrot saying this, but...) If you care about why it might help, read this email thread: http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root If it happens to help, please email again, since there are probably better ways to fix the problem than disabling TSO. Good luck with it, rick > What might not be standard is that I use sub-interfaces on this > system. > First of all, the only way that I can get the sub-interfaces to work > at > all is by using > > ifconfig_re0="-vlanhwtag" > > I'm not sure that is related. > > The question is how can we debug this to solve the problem? > I have non clue, but I'm happy to assist if somebody can tell me what > I > should do. > > Some information that might be useful: > > root@drawbridge:/usr/local/etc/bacula # dmesg | grep re0 > re0: port > 0xd000-0xd0ff mem 0xf7a00000-0xf7a00fff,0xf0100000-0xf0103fff irq 17 > at > device 0.0 on pci2 > re0: Using 1 MSI-X message > re0: ASPM disabled > re0: Chip rev. 0x4c000000 > re0: MAC rev. 0x00000000 > miibus0: on re0 > re0: Ethernet address: 80:ee:73:77:e9:ab > re0: watchdog timeout > re0: link state changed to DOWN > re0.98: link state changed to DOWN > re0.10: link state changed to DOWN > re0.11: link state changed to DOWN > re0.12: link state changed to DOWN > re0: link state changed to UP > re0.98: link state changed to UP > re0.10: link state changed to UP > re0.11: link state changed to UP > re0.12: link state changed to UP > ... > > root@drawbridge:/usr/local/etc/bacula # uname -a > FreeBSD drawbridge.internal.deze.org 10.0-STABLE FreeBSD 10.0-STABLE > #0 > r262433: Mon Feb 24 16:25:35 CET 2014 > root@drawbridge-new.internal.deze.org:/usr/obj/usr/sources/src10-stable/sys/SHUTTLE > i386 > > root@drawbridge:/usr/local/etc/bacula # pciconf -lv re0 > re0@pci0:2:0:0: class=0x020000 card=0x40181297 chip=0x816810ec > rev=0x0c > hdr=0x00 > vendor = 'Realtek Semiconductor Co., Ltd.' > device = 'RTL8111/8168B PCI Express Gigabit Ethernet > controller' > class = network > subclass = ethernet > > > Kind regards, > > Frank > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 01:22:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12E53C5; Mon, 7 Apr 2014 01:22:46 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF265329; Mon, 7 Apr 2014 01:22:45 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id p10so5791020pdj.40 for ; Sun, 06 Apr 2014 18:22:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=tyqwmYNNTTf9AvPZELGE7Jd5K7lyXE4aCP62uCzgfh8=; b=q2jKroFvpdjb6dtinGGaKAngX7rDyukAVk32q1K5z+fnN2TUaH34F185GGDHybLqOy ZuYEYgEi8uyWCTtgiC0sMgKsLV6nkHgziwVSF3SRmvx/5OCou6jj1jHu+OaxzaDxKocn hS7FuZzF3ocU8SDJTGpPwlJ6jVPg/bFhYmamvOmxKB9OwGM5TqP2EFg3LpUru7BJR8JP zngM7IuTxyrjzET4cnLhwTnNUcq2nq/T06vuo/8BREMIWbv3qif//b6vxcY/Av8Ijrp5 6d7bT8rDeuLVXUru2VFuQpu5h2sDY18zKiAq6baLhfj7yLhQOJwsuu1ooFJMI8/aOnn1 XsIA== X-Received: by 10.66.190.4 with SMTP id gm4mr634627pac.116.1396833765468; Sun, 06 Apr 2014 18:22:45 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id ss2sm77719005pab.8.2014.04.06.18.22.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 06 Apr 2014 18:22:44 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 07 Apr 2014 10:22:41 +0900 From: Yonghyeon PYUN Date: Mon, 7 Apr 2014 10:22:41 +0900 To: Rick Macklem Subject: Re: re0: watchdog timeout Message-ID: <20140407012241.GA3543@michelle.cdnetworks.com> References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 01:22:46 -0000 On Sun, Apr 06, 2014 at 07:37:08PM -0400, Rick Macklem wrote: > Frank Volf wrote: > > > > Hello, > > > > I'm experiencing watchdog timeouts with my Realtek interface card. > > > > I'm using a fairly new system (Shuttle DS47), running FreeBSD > > 10-STABLE. > > For this shuttle a patch has been recently committed to SVN to make > > this > > card work at all (revision *262391* > > ). > > > > The timeout is only experienced under heavy network load (the system > > is > > running a bacula backup server that backups to NFS connected > > storage), > > and typically large full backups trigger this. Normal traffic works > > fine > > (this system is e.g. also my firewall to the Internet). > > > Since you mention NFS, you could try disabling TSO on the interface > and see if that helps. (I'm beginning to feel like a parrot saying this, > but...) If you care about why it might help, read this email thread: > http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > > If it happens to help, please email again, since there are probably > better ways to fix the problem than disabling TSO. > re(4) controllers support TSO but it was disabled long time ago(r217832). It's still allowed to enable TSO but users have to explicitly enable it with ifconfig. If Frank didn't explicitly enable TSO on the box, TSO may have nothing to do with watchdog timeout, I guess. > Good luck with it, rick > > > What might not be standard is that I use sub-interfaces on this > > system. > > First of all, the only way that I can get the sub-interfaces to work > > at > > all is by using > > > > ifconfig_re0="-vlanhwtag" > > > > I'm not sure that is related. > > > > The question is how can we debug this to solve the problem? > > I have non clue, but I'm happy to assist if somebody can tell me what > > I > > should do. > > > > Some information that might be useful: > > > > root@drawbridge:/usr/local/etc/bacula # dmesg | grep re0 > > re0: port > > 0xd000-0xd0ff mem 0xf7a00000-0xf7a00fff,0xf0100000-0xf0103fff irq 17 > > at > > device 0.0 on pci2 > > re0: Using 1 MSI-X message > > re0: ASPM disabled > > re0: Chip rev. 0x4c000000 > > re0: MAC rev. 0x00000000 > > miibus0: on re0 > > re0: Ethernet address: 80:ee:73:77:e9:ab > > re0: watchdog timeout > > re0: link state changed to DOWN > > re0.98: link state changed to DOWN > > re0.10: link state changed to DOWN > > re0.11: link state changed to DOWN > > re0.12: link state changed to DOWN > > re0: link state changed to UP > > re0.98: link state changed to UP > > re0.10: link state changed to UP > > re0.11: link state changed to UP > > re0.12: link state changed to UP > > ... > > > > root@drawbridge:/usr/local/etc/bacula # uname -a > > FreeBSD drawbridge.internal.deze.org 10.0-STABLE FreeBSD 10.0-STABLE > > #0 > > r262433: Mon Feb 24 16:25:35 CET 2014 > > root@drawbridge-new.internal.deze.org:/usr/obj/usr/sources/src10-stable/sys/SHUTTLE > > i386 > > > > root@drawbridge:/usr/local/etc/bacula # pciconf -lv re0 > > re0@pci0:2:0:0: class=0x020000 card=0x40181297 chip=0x816810ec > > rev=0x0c > > hdr=0x00 > > vendor = 'Realtek Semiconductor Co., Ltd.' > > device = 'RTL8111/8168B PCI Express Gigabit Ethernet > > controller' > > class = network > > subclass = ethernet > > > > > > Kind regards, > > > > Frank > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 01:51:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CE0F5FF; Mon, 7 Apr 2014 01:51:09 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E8AF783; Mon, 7 Apr 2014 01:51:09 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so5971782pab.22 for ; Sun, 06 Apr 2014 18:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=4JIjes/2O+r2UX4kmz4XHB+I3FDeXC9FomZdL60sXyI=; b=XQIJkI64oV3Q1SpgCBG3NE48M/fadWTPcbLKW+D4HIpm/dtpwsx3qXQ0rvel2MOJdX zUqukHKmUOCH1NLcl2DFWlvQinEbSWzHAsyXIdZaYFqzCRbvdpiuJIYNWgThOKJBHcuq amjZJWd/8GDhK0Ih8/4yG9jc/ksBwGHqsl0tVllbu8FZJgqLTzIaTbrymonQRoUDAoPh AjbMBlVeZx8+nN4TI0DkNMY8If8fSoQVdMLkyU1Izm1tf7C1RZGNDj7+A4u4BfVnbkDi Wc5ExZNDbfQDxVjqyfyfrZcPf3V5MhEhS8rGISQ8xp6OG6v1J93YNc5nZmoiaKyNB9o1 w7WQ== X-Received: by 10.68.252.165 with SMTP id zt5mr28219060pbc.17.1396835468624; Sun, 06 Apr 2014 18:51:08 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id pl10sm32950998pbb.56.2014.04.06.18.51.05 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 06 Apr 2014 18:51:07 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 07 Apr 2014 10:51:04 +0900 From: Yonghyeon PYUN Date: Mon, 7 Apr 2014 10:51:04 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140407015104.GC3543@michelle.cdnetworks.com> References: <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 01:51:09 -0000 On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: > > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: > >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: > >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > >> >> >> [...] > >> >> >> miibus0: on nfe0 > >> >> >> rlphy0: PHY 0 on miibus0 > >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > >> >> >> rlphy1: PHY 1 on miibus0 > >> >> > [...]---big-snip--8<--- > >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> >> >> > >> >> >> As you can see, it looks much the same. I have no idea what > >> >> >> I should do to better inform the driver/kernel how to better > >> >> >> handle it. Or is it the driver, itself? > >> >> >> > >> >> >> Thank you again, for your thoughtful response. > >> >> >> > >> >> >> --Chris > >> >> >> > >> >> > > >> >> > I think the way to fix a phy that responds at all addresses is to set a > >> >> > hint in loader.conf masking out the ones that aren't real, like so: > >> >> > > >> >> > hint.miibus.0.phymask="1" > >> >> > > >> >> > You might be able to set ="0x00000001" to make it more clear it's a > >> >> > bitmask, but I'm not sure of that. > >> >> > >> >> Thank you very much for the hint. I'll give it a shot. > >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia > >> >> chipset, and the nfe(4) driver. But they don't respond this way. > >> >> > >> > > >> > If some nfe(4) variants badly behave in probing stage, this should > >> > be handled by driver. We already have too many hints and tunables > >> > and I don't think most users know that. In addition, adding > >> > additional NIC may change miibus instance number. > >> > > >> > Could you show me the output of 'kenv | grep smbios'? > >> Yes, of course. > >> > >> Here it is: > >> > >> smbios.bios.reldate="11/22/2010" > >> smbios.bios.vendor="American Megatrends Inc." > >> smbios.bios.version="V2.7" > >> smbios.chassis.maker="MSI" > >> smbios.chassis.serial="To Be Filled By O.E.M." > >> smbios.chassis.tag="To Be Filled By O.E.M." > >> smbios.chassis.version="2.0" > >> smbios.memory.enabled="2097152" > >> smbios.planar.maker="MSI" > >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" > >> smbios.planar.serial="To be filled by O.E.M." > >> smbios.planar.version="2.0" > >> smbios.socket.enabled="1" > >> smbios.socket.populated="1" > >> smbios.system.maker="MSI" > >> smbios.system.product="MS-7309" > >> smbios.system.serial="To Be Filled By O.E.M." > >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" > >> smbios.system.version="2.0" > >> smbios.version="2.6" > >> > >> Hope this helps, and thank you for all your time, and trouble. > >> > > > > Thanks for the info. Try attached patch and let me know how it > > works. Make sure to remove the hint(hint.miibus.0.phymask="1") > > set in loader.conf before testing it. > > Hello, and thanks for all the attention. > Sorry for the delay. I chose to perform a dump(8) before attempting > the KERn rebuild with the patch. But the kernel threw a read error > message on one of the drives. So I had to sort out the problem on > the drive before I could complete the dump. Then, of course I had > to reslice, and format another drive to replace the ailing one, > before I could perform a restore(8), and start the nfe patch; build > && install kernel. Weird; the drive had only a few hours on it. > Well, anyway. The patch applied cleanly. So I built, and installed > a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" > in loader.conf(5), and bounced the box. Bad news: > miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > miibus0: mii_mediachg: can't handle non-zero PHY instance 30 > miibus0: mii_mediachg: can't handle non-zero PHY instance 29 > miibus0: mii_mediachg: can't handle non-zero PHY instance 28 > miibus0: mii_mediachg: can't handle non-zero PHY instance 27 > miibus0: mii_mediachg: can't handle non-zero PHY instance 26 > miibus0: mii_mediachg: can't handle non-zero PHY instance 25 > miibus0: mii_mediachg: can't handle non-zero PHY instance 24 > miibus0: mii_mediachg: can't handle non-zero PHY instance 23 > miibus0: mii_mediachg: can't handle non-zero PHY instance 22 > miibus0: mii_mediachg: can't handle non-zero PHY instance 21 > miibus0: mii_mediachg: can't handle non-zero PHY instance 20 > miibus0: mii_mediachg: can't handle non-zero PHY instance 19 > miibus0: mii_mediachg: can't handle non-zero PHY instance 18 > miibus0: mii_mediachg: can't handle non-zero PHY instance 17 > miibus0: mii_mediachg: can't handle non-zero PHY instance 16 > miibus0: mii_mediachg: can't handle non-zero PHY instance 15 > miibus0: mii_mediachg: can't handle non-zero PHY instance 14 > miibus0: mii_mediachg: can't handle non-zero PHY instance 13 > miibus0: mii_mediachg: can't handle non-zero PHY instance 12 > miibus0: mii_mediachg: can't handle non-zero PHY instance 11 > miibus0: mii_mediachg: can't handle non-zero PHY instance 10 > miibus0: mii_mediachg: can't handle non-zero PHY instance 9 > miibus0: mii_mediachg: can't handle non-zero PHY instance 8 > miibus0: mii_mediachg: can't handle non-zero PHY instance 7 > miibus0: mii_mediachg: can't handle non-zero PHY instance 6 > miibus0: mii_mediachg: can't handle non-zero PHY instance 5 > miibus0: mii_mediachg: can't handle non-zero PHY instance 4 > miibus0: mii_mediachg: can't handle non-zero PHY instance 3 > miibus0: mii_mediachg: can't handle non-zero PHY instance 2 > miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > > Just as before. In case it should make any difference; I'm > going to attach my copy of src/sys/dev/nfe/if_nfe.c in case > there are any differences in mine, that do not coincide with > your version/copy (I'm on releng_9 - 9.2-STABLE) > > FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: > Thu Apr 3 12:42:03 PDT 2014 > root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 > > Best wishes, and thanks again. > Oops, could you show me the output of "pciconf -lcbv"? From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 02:06:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84C1AE86 for ; Mon, 7 Apr 2014 02:06:21 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41FA8918 for ; Mon, 7 Apr 2014 02:06:21 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id cm18so2463930qab.32 for ; Sun, 06 Apr 2014 19:06:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=EDshUn3iFmhcu/3or86fMK1MW5jGz95Fd5JiWuc0qaI=; b=C5x0P5aGX1DOKRb1d7AbJ0J0Xjp9FtkrhcGYVcoO3BJOn+MQBNOkUjEWcebfZFDVkn 0ftF2uboywNdUQnWCf3shdC8AgH1uu5nQ/+mn9yoCHVL1nzThsOAYNwkNJ7IgKCHqkhY LVycWLdlDlRfz7fIRSxUoI0TzanT/TwuLC0Ajh5Cx7MoGfmJc+dx9H6RUx910lV9MTCw eF3gCXtnGOnuXohyciF3CTTMRNguJbJmKZsXAINynw0vm0XWlpqekVWnjnac2GCap1j9 gHdU1pE/M6sY+hUqCyOGh4E63tq9CQHQW6/kXizb0NLWn49QXELB8LZtgGnj9tgwkiXY 7q+g== X-Received: by 10.229.12.197 with SMTP id y5mr2394qcy.22.1396836379495; Sun, 06 Apr 2014 19:06:19 -0700 (PDT) Received: from [10.100.82.100] (cpe-075-184-002-012.nc.res.rr.com. [75.184.2.12]) by mx.google.com with ESMTPSA id x8sm31533418qat.36.2014.04.06.19.06.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 06 Apr 2014 19:06:18 -0700 (PDT) Message-ID: <53420819.8070806@gmail.com> Date: Sun, 06 Apr 2014 22:06:17 -0400 From: Jason Unovitch User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, chris@nevermind.co.nz Subject: Re: freebsd-net Digest, Vol 574, Issue 9 References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 02:06:21 -0000 > Date: Sat, 05 Apr 2014 15:22:39 +1300 > From: Chris Smith > To: freebsd-net@freebsd.org > Subject: Multihomed system with jails routing issues > Message-ID: <533F68EF.8060607@nevermind.co.nz> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi All, > > I have a system with 1 network interface with 2 extra VLANs off it and > I'm having some trouble getting the routing working correctly with it > and jails. > > bge0 - management - 10.71.100.0/24 > bge0.101 - LAN - 10.71.101.0/24 > bge0.103 - DMZ - 10.71.101.0/24 Do you mean .102.0/24 for the DMZ subnet? An overlap is bound to cause issues. Just so I understand, is the router side configured as a trunk port with a native vlan set for that management address? > Here's what I want to achieve... > > Host: > I want the host system to only listen on one interface, bge0. I want NO > ip addresses of the host on the vlan interfaces. The only service it > will be exposing is its sshd. The management address for this system is > 10.71.100.50. Just make sure every daemon running on the host is bound to an address. ListenAddress in /etc/ssh/sshd_config. Use '-b' flags for a bunch of parameters in rc.conf. Unfortunately, keep in mind that not everything supports being bound to a specific address. > Jails: > The system will also host a variety of jails, each with an IP either on > the LAN or DMZ. I am using ezjail to manage the jails. > > Router: > There is a router at the .254 address of every subnet that can route > between each network. > > I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host does > not have an address configured on bge0.101, I configured the jail > address as /24 instead of the default /32. > > My issues: > > * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot > communicate with the jail. I can't comment on the difference between /24 and /32 here as I've only set my jails to a /32. I use DHCP on my em0 with a bunch of /32 aliases on a DMZ fib and static address on my em1 with a bunch of /32 aliases on a LAN fib. All my host services have the bind address set to the static IP on my LAN interface. > * When the jail is up and 10.71.101.51/24 is active, SSHing from the LAN > to the mgmt interface via the router fails, as the host tries to send > return traffic via the bge0.101 interface, even though traffic arrived > via the bge0 interface. It doesn't matter which interface the traffic arrived on, it's going to use the directly connected on when it has the whole routing table. Is the router using a source address of 10.71.101.254? > So I did a whole lot of research for people having these apparently > problems, and decided to try the multiple routing table/fib approach. So > I recompiled my kernel, configured fib 1 with the LAN interface route > (setfib route add 10.71.101.0/24 -iface bge0.101), set the jail fib and > set the tunable net.addr_all_fibs = 0. I still can't get this working > correctly. ezjail still seems to add the interface route to fib 0 by > default (but it won't if i run ezjail with the setfib 1 command). Here is a short version of what I do. I set the sysctl as you pointed out so that directly connected subnets don't automatically get added to every fib. echo 'net.add_addr_allfibs=0' >> /boot/loader.conf I set the alias and a static route for both the network and gateway in my rc.conf. ifconfig_em0="DHCP" ifconfig_em0_alias0="inet 192.168.102.11/32 fib 2" static_routes="fibnetwork fibdefault" route_fibnetwork="-net 192.168.102.0/24 -interface em0 -fib 2" route_fibdefault="default 192.168.102.1 -fib 2" Finally I set ezjail to use the fib. With all this I can run tcpdump on my router and see that connections from my jail behave as expected. ezjail-admin config -f 2 jail.example.com > Using FIB 1 and trying to ping hosts on the LAN gives an error like: > sendto failed: invalid argument. > > Does anybody have any best practices for doing this, or anything else I > can try? I'm happy to share/pastebin any configuration and I've tried > most things I've found on the internet. I'm using FreeBSD 10.0 with a > custom kernel for multiple routing tables. FYI, you don't need a custom kernel for this. It's set at boot time. Example for 4 fibs: echo 'net.fibs=4' >> /boot/loader.conf > Thanks in advance! > Chris. > I had to toy around with this quite a bit before I got the behaviour I was looking for. I hope this helps. Jason From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 02:58:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A28B0692 for ; Mon, 7 Apr 2014 02:58:35 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6724EC71 for ; Mon, 7 Apr 2014 02:58:35 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so5755662qgd.1 for ; Sun, 06 Apr 2014 19:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=RdGDS1JNE99X29en5PKbGZveFzV0WvdgOMkfBCmm4c8=; b=dljJ364/Fo1aIYlB6C6aAclkpRHcSUX+LKXaFR14BivxjBMRFTKxBTMH+wP5wDmtii +/sgH4S8LHJSTqgZ8auRxTjSaMONNaNLu1sYNvv3j6SSN4GRsKKMWiIQ1j9RDdYQWhqO +K5uodarr2O8bCOy7yDDQE5C7Qg/daEDNgQwj8zIUz4CgBgJ8nGrXz17Ph9L0T6+eL3T yGHmLm+Npd63ATtoODnPAmbdRZdbiMRiOkR7MuQCM2PiOXqT0/YtAYvV+yruseFbsk2m 5Y/MdNpCdtnuNWfBJb8FKqiS8lP3zKdihvGjN6/qM4UjJOTCsicA+gxyFZg4enPmEJDl srVA== MIME-Version: 1.0 X-Received: by 10.140.95.142 with SMTP id i14mr28991149qge.6.1396839514585; Sun, 06 Apr 2014 19:58:34 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Sun, 6 Apr 2014 19:58:34 -0700 (PDT) Date: Sun, 6 Apr 2014 19:58:34 -0700 X-Google-Sender-Auth: 7COk6gU6AGvPYgNWYxraa8fm-tQ Message-ID: Subject: panic in -HEAD multicast code From: Adrian Chadd To: FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 02:58:35 -0000 hiya, I'm seeing a panic in the multicast code path. I reproduce this by trying to browse network sharing on VLC. The panic: http://people.freebsd.org/~adrian/ath/core.txt.0 Any ideas? -a From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 05:10:00 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06149355; Mon, 7 Apr 2014 05:10:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCD4D969; Mon, 7 Apr 2014 05:09:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3759xZe021121; Mon, 7 Apr 2014 05:09:59 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3759xOr021120; Mon, 7 Apr 2014 05:09:59 GMT (envelope-from linimon) Date: Mon, 7 Apr 2014 05:09:59 GMT Message-Id: <201404070509.s3759xOr021120@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/188245: [vlan] Critical vlan problem with OpenBGP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 05:10:00 -0000 Old Synopsis: Critical vlan problem with OpenBGP New Synopsis: [vlan] Critical vlan problem with OpenBGP Responsible-Changed-From-To: freebsd-amd64->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Mon Apr 7 05:08:04 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=188245 From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 05:46:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A6C9C50; Mon, 7 Apr 2014 05:46:27 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C99D0BB6; Mon, 7 Apr 2014 05:46:26 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s375nWJS021614; Sun, 6 Apr 2014 22:49:38 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s375nQB4021610; Sun, 6 Apr 2014 22:49:26 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Sun, 6 Apr 2014 22:49:27 -0700 (PDT) Message-ID: <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> In-Reply-To: <20140407015104.GC3543@michelle.cdnetworks.com> References: <20140331050002.GC1359@michelle.cdnetworks.com> <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> <20140407015104.GC3543@michelle.cdnetworks.com> Date: Sun, 6 Apr 2014 22:49:27 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 05:46:27 -0000 > On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: >> > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: >> >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: >> >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> >> >> >> [...] >> >> >> >> miibus0: on nfe0 >> >> >> >> rlphy0: PHY 0 on miibus0 >> >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> >> >> >> rlphy1: PHY 1 on miibus0 >> >> >> > [...]---big-snip--8<--- >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> >> >> >> >> As you can see, it looks much the same. I have no idea what >> >> >> >> I should do to better inform the driver/kernel how to better >> >> >> >> handle it. Or is it the driver, itself? >> >> >> >> >> >> >> >> Thank you again, for your thoughtful response. >> >> >> >> >> >> >> >> --Chris >> >> >> >> >> >> >> > >> >> >> > I think the way to fix a phy that responds at all addresses is to set a >> >> >> > hint in loader.conf masking out the ones that aren't real, like so: >> >> >> > >> >> >> > hint.miibus.0.phymask="1" >> >> >> > >> >> >> > You might be able to set ="0x00000001" to make it more clear it's a >> >> >> > bitmask, but I'm not sure of that. >> >> >> >> >> >> Thank you very much for the hint. I'll give it a shot. >> >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia >> >> >> chipset, and the nfe(4) driver. But they don't respond this way. >> >> >> >> >> > >> >> > If some nfe(4) variants badly behave in probing stage, this should >> >> > be handled by driver. We already have too many hints and tunables >> >> > and I don't think most users know that. In addition, adding >> >> > additional NIC may change miibus instance number. >> >> > >> >> > Could you show me the output of 'kenv | grep smbios'? >> >> Yes, of course. >> >> >> >> Here it is: >> >> >> >> smbios.bios.reldate="11/22/2010" >> >> smbios.bios.vendor="American Megatrends Inc." >> >> smbios.bios.version="V2.7" >> >> smbios.chassis.maker="MSI" >> >> smbios.chassis.serial="To Be Filled By O.E.M." >> >> smbios.chassis.tag="To Be Filled By O.E.M." >> >> smbios.chassis.version="2.0" >> >> smbios.memory.enabled="2097152" >> >> smbios.planar.maker="MSI" >> >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" >> >> smbios.planar.serial="To be filled by O.E.M." >> >> smbios.planar.version="2.0" >> >> smbios.socket.enabled="1" >> >> smbios.socket.populated="1" >> >> smbios.system.maker="MSI" >> >> smbios.system.product="MS-7309" >> >> smbios.system.serial="To Be Filled By O.E.M." >> >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" >> >> smbios.system.version="2.0" >> >> smbios.version="2.6" >> >> >> >> Hope this helps, and thank you for all your time, and trouble. >> >> >> > >> > Thanks for the info. Try attached patch and let me know how it >> > works. Make sure to remove the hint(hint.miibus.0.phymask="1") >> > set in loader.conf before testing it. >> >> Hello, and thanks for all the attention. >> Sorry for the delay. I chose to perform a dump(8) before attempting >> the KERn rebuild with the patch. But the kernel threw a read error >> message on one of the drives. So I had to sort out the problem on >> the drive before I could complete the dump. Then, of course I had >> to reslice, and format another drive to replace the ailing one, >> before I could perform a restore(8), and start the nfe patch; build >> && install kernel. Weird; the drive had only a few hours on it. >> Well, anyway. The patch applied cleanly. So I built, and installed >> a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" >> in loader.conf(5), and bounced the box. Bad news: >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 30 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 29 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 28 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 27 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 26 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 25 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 24 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 23 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 22 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 21 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 20 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 19 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 18 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 17 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 16 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 15 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 14 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 13 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 12 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 11 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 10 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 9 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 8 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 7 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 6 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 5 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 4 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 3 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 2 >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> Just as before. In case it should make any difference; I'm >> going to attach my copy of src/sys/dev/nfe/if_nfe.c in case >> there are any differences in mine, that do not coincide with >> your version/copy (I'm on releng_9 - 9.2-STABLE) >> >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: >> Thu Apr 3 12:42:03 PDT 2014 >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 >> >> Best wishes, and thanks again. >> > > Oops, could you show me the output of "pciconf -lcbv"? Yes. Of course. none0@pci0:0:0:0: class=0x050000 card=0x73091462 chip=0x03ea10de rev=0xa1 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 Memory Controller' class = memory subclass = RAM cap 08[44] = HT slave cap 08[dc] = HT MSI address window enabled at 0xfee00000 isab0@pci0:0:1:0: class=0x060100 card=0xcb8410de chip=0x03e010de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 LPC Bridge' class = bridge subclass = PCI-ISA bar [10] = type I/O Port, range 32, base 0x4f00, size 256, enabled none1@pci0:0:1:1: class=0x0c0500 card=0x73091462 chip=0x03eb10de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 SMBus' class = serial bus subclass = SMBus bar [10] = type I/O Port, range 32, base 0x4900, size 64, enabled bar [20] = type I/O Port, range 32, base 0x4d00, size 64, enabled bar [24] = type I/O Port, range 32, base 0x4e00, size 64, enabled cap 01[44] = powerspec 2 supports D0 D3 current D0 none2@pci0:0:1:2: class=0x050000 card=0x73091462 chip=0x03f510de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 Memory Controller' class = memory subclass = RAM ohci0@pci0:0:2:0: class=0x0c0310 card=0x73091462 chip=0x03f110de rev=0xa3 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 USB Controller' class = serial bus subclass = USB bar [10] = type Memory, range 32, base 0xdff7f000, size 4096, enabled cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 ehci0@pci0:0:2:1: class=0x0c0320 card=0x73091462 chip=0x03f210de rev=0xa3 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 USB Controller' class = serial bus subclass = USB bar [10] = type Memory, range 32, base 0xdff7ec00, size 256, enabled cap 0a[44] = EHCI Debug Port at offset 0x98 in map 0x14 cap 01[80] = powerspec 2 supports D0 D1 D2 D3 current D0 pcib1@pci0:0:4:0: class=0x060401 card=0xcb8410de chip=0x03f310de rev=0xa1 hdr=0x01 vendor = 'nVidia Corporation' device = 'MCP61 PCI bridge' class = bridge subclass = PCI-PCI cap 0d[b8] = PCI Bridge card=0xcb8410de cap 08[8c] = HT MSI address window disabled at 0xfee00000 hdac0@pci0:0:5:0: class=0x040300 card=0x10ec0888 chip=0x03f010de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 High Definition Audio' class = multimedia subclass = HDA bar [10] = type Memory, range 32, base 0xdff78000, size 16384, enabled cap 01[44] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks enabled with 1 message cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 atapci0@pci0:0:6:0: class=0x01018a card=0x73091462 chip=0x03ec10de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 IDE' class = mass storage subclass = ATA bar [20] = type I/O Port, range 32, base 0xffa0, size 16, enabled cap 01[44] = powerspec 2 supports D0 D3 current D0 nfe0@pci0:0:7:0: class=0x068000 card=0x73091462 chip=0x03ef10de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 Ethernet' class = bridge bar [10] = type Memory, range 32, base 0xdff7d000, size 4096, enabled bar [14] = type I/O Port, range 32, base 0xe480, size 8, enabled cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 cap 05[50] = MSI supports 8 messages, 64 bit, vector masks enabled with 8 messages cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 atapci1@pci0:0:8:0: class=0x010185 card=0x73091462 chip=0x03f610de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 SATA Controller' class = mass storage subclass = ATA bar [10] = type I/O Port, range 32, base 0xe400, size 8, enabled bar [14] = type I/O Port, range 32, base 0xe080, size 4, enabled bar [18] = type I/O Port, range 32, base 0xe000, size 8, enabled bar [1c] = type I/O Port, range 32, base 0xdc00, size 4, enabled bar [20] = type I/O Port, range 32, base 0xd880, size 16, enabled bar [24] = type Memory, range 32, base 0xdff7c000, size 4096, enabled cap 01[44] = powerspec 2 supports D0 D3 current D0 cap 05[b0] = MSI supports 4 messages, 64 bit cap 08[cc] = HT MSI fixed address window disabled at 0xfee00000 atapci2@pci0:0:8:1: class=0x010185 card=0x73091462 chip=0x03f610de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP61 SATA Controller' class = mass storage subclass = ATA bar [10] = type I/O Port, range 32, base 0xd800, size 8, enabled bar [14] = type I/O Port, range 32, base 0xd480, size 4, enabled bar [18] = type I/O Port, range 32, base 0xd400, size 8, enabled bar [1c] = type I/O Port, range 32, base 0xd080, size 4, enabled bar [20] = type I/O Port, range 32, base 0xd000, size 16, enabled bar [24] = type Memory, range 32, base 0xdff77000, size 4096, enabled cap 01[44] = powerspec 2 supports D0 D3 current D0 cap 05[b0] = MSI supports 4 messages, 64 bit cap 08[cc] = HT MSI fixed address window disabled at 0xfee00000 pcib2@pci0:0:9:0: class=0x060400 card=0x73091462 chip=0x03e810de rev=0xa2 hdr=0x01 vendor = 'nVidia Corporation' device = 'MCP61 PCI Express bridge' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x73091462 cap 01[48] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 2 messages, 64 bit cap 08[60] = HT MSI address window disabled at 0xfee00000 cap 10[80] = PCI-Express 1 root port slot max data 128(256) link x16(x16) speed 2.5(2.5) ASPM disabled(L0s/L1) ecap 0002[100] = VC 1 max VC0 pcib3@pci0:0:11:0: class=0x060400 card=0x73091462 chip=0x03e910de rev=0xa2 hdr=0x01 vendor = 'nVidia Corporation' device = 'MCP61 PCI Express bridge' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x73091462 cap 01[48] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 2 messages, 64 bit cap 08[60] = HT MSI address window disabled at 0xfee00000 cap 10[80] = PCI-Express 1 root port slot max data 128(256) link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) ecap 0002[100] = VC 1 max VC0 pcib4@pci0:0:12:0: class=0x060400 card=0x73091462 chip=0x03e910de rev=0xa2 hdr=0x01 vendor = 'nVidia Corporation' device = 'MCP61 PCI Express bridge' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x73091462 cap 01[48] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 2 messages, 64 bit cap 08[60] = HT MSI address window disabled at 0xfee00000 cap 10[80] = PCI-Express 1 root port slot max data 128(256) link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) ecap 0002[100] = VC 1 max VC0 vgapci0@pci0:0:13:0: class=0x030000 card=0x73091462 chip=0x03d010de rev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'C61 [GeForce 6150SE nForce 430]' class = display subclass = VGA bar [10] = type Memory, range 32, base 0xde000000, size 16777216, enabled bar [14] = type Prefetchable Memory, range 64, base 0xc0000000, size 268435456, enabled bar [1c] = type Memory, range 64, base 0xdd000000, size 16777216, enabled cap 01[48] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit hostb0@pci0:0:24:0: class=0x060000 card=0x00000000 chip=0x12001022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'Family 10h Processor HyperTransport Configuration' class = bridge subclass = HOST-PCI cap 08[80] = HT host hostb1@pci0:0:24:1: class=0x060000 card=0x00000000 chip=0x12011022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'Family 10h Processor Address Map' class = bridge subclass = HOST-PCI hostb2@pci0:0:24:2: class=0x060000 card=0x00000000 chip=0x12021022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'Family 10h Processor DRAM Controller' class = bridge subclass = HOST-PCI hostb3@pci0:0:24:3: class=0x060000 card=0x00000000 chip=0x12031022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'Family 10h Processor Miscellaneous Control' class = bridge subclass = HOST-PCI cap 0f[f0] = unknown hostb4@pci0:0:24:4: class=0x060000 card=0x00000000 chip=0x12041022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'Family 10h Processor Link Control' class = bridge subclass = HOST-PCI Thanks for all your time, and efforts. --Chris > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 06:04:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 688266F; Mon, 7 Apr 2014 06:04:00 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26784D2E; Mon, 7 Apr 2014 06:04:00 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id kq14so6248040pab.37 for ; Sun, 06 Apr 2014 23:03:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=J1goa8/m05zJ39Zbz5bEi6rTI1ZJrDk4D6fVBhnzzNU=; b=ofyBXS5CRPCBZwXZbbeSadL3YxAJnOAmMK3gR/hYhdaLXAStBFfXhHmNuid8g1I8eT qF+aQvd6cH2Nlw2XFCRMdBpraoQ+IpRABn8/NzpNAPKvtAc6NZQ4D0KK1yX2jkebPYWr wSR4mm9VYIxjpur/gGtikqIgeh95aLoqKDpUWlKNmJbj8SFNMLzvn0h7JO2kPkiYvinZ 71THIcahSyi0LIfWTZ6DvcHlc8sZU7Nt471+hNGZPWvVZq+IwvSdlIHWryZ/MzkW2RPt w+jKkbcNgtrPWnbVMAlvY1rgthHFYpMFbR3D+bZ3WaqBXfUdbELq7mXbo/1nQjedg1jg b0Pg== X-Received: by 10.66.153.80 with SMTP id ve16mr1060756pab.143.1396850639571; Sun, 06 Apr 2014 23:03:59 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id xk3sm34158880pbb.65.2014.04.06.23.03.55 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 06 Apr 2014 23:03:58 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 07 Apr 2014 15:03:51 +0900 From: Yonghyeon PYUN Date: Mon, 7 Apr 2014 15:03:51 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140407060351.GA1357@michelle.cdnetworks.com> References: <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> <20140407015104.GC3543@michelle.cdnetworks.com> <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="EeQfGwPcQSOJBaQU" Content-Disposition: inline In-Reply-To: <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 06:04:00 -0000 --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Apr 06, 2014 at 10:49:27PM -0700, Chris H wrote: > > On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: > >> > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: > >> >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: > >> >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > >> >> >> >> [...] > >> >> >> >> miibus0: on nfe0 > >> >> >> >> rlphy0: PHY 0 on miibus0 > >> >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > >> >> >> >> rlphy1: PHY 1 on miibus0 > >> >> >> > [...]---big-snip--8<--- > >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> >> >> >> > >> >> >> >> As you can see, it looks much the same. I have no idea what > >> >> >> >> I should do to better inform the driver/kernel how to better > >> >> >> >> handle it. Or is it the driver, itself? > >> >> >> >> > >> >> >> >> Thank you again, for your thoughtful response. > >> >> >> >> > >> >> >> >> --Chris > >> >> >> >> > >> >> >> > > >> >> >> > I think the way to fix a phy that responds at all addresses is to set a > >> >> >> > hint in loader.conf masking out the ones that aren't real, like so: > >> >> >> > > >> >> >> > hint.miibus.0.phymask="1" > >> >> >> > > >> >> >> > You might be able to set ="0x00000001" to make it more clear it's a > >> >> >> > bitmask, but I'm not sure of that. > >> >> >> > >> >> >> Thank you very much for the hint. I'll give it a shot. > >> >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia > >> >> >> chipset, and the nfe(4) driver. But they don't respond this way. > >> >> >> > >> >> > > >> >> > If some nfe(4) variants badly behave in probing stage, this should > >> >> > be handled by driver. We already have too many hints and tunables > >> >> > and I don't think most users know that. In addition, adding > >> >> > additional NIC may change miibus instance number. > >> >> > > >> >> > Could you show me the output of 'kenv | grep smbios'? > >> >> Yes, of course. > >> >> > >> >> Here it is: > >> >> > >> >> smbios.bios.reldate="11/22/2010" > >> >> smbios.bios.vendor="American Megatrends Inc." > >> >> smbios.bios.version="V2.7" > >> >> smbios.chassis.maker="MSI" > >> >> smbios.chassis.serial="To Be Filled By O.E.M." > >> >> smbios.chassis.tag="To Be Filled By O.E.M." > >> >> smbios.chassis.version="2.0" > >> >> smbios.memory.enabled="2097152" > >> >> smbios.planar.maker="MSI" > >> >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" > >> >> smbios.planar.serial="To be filled by O.E.M." > >> >> smbios.planar.version="2.0" > >> >> smbios.socket.enabled="1" > >> >> smbios.socket.populated="1" > >> >> smbios.system.maker="MSI" > >> >> smbios.system.product="MS-7309" > >> >> smbios.system.serial="To Be Filled By O.E.M." > >> >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" > >> >> smbios.system.version="2.0" > >> >> smbios.version="2.6" > >> >> > >> >> Hope this helps, and thank you for all your time, and trouble. > >> >> > >> > > >> > Thanks for the info. Try attached patch and let me know how it > >> > works. Make sure to remove the hint(hint.miibus.0.phymask="1") > >> > set in loader.conf before testing it. > >> > >> Hello, and thanks for all the attention. > >> Sorry for the delay. I chose to perform a dump(8) before attempting > >> the KERn rebuild with the patch. But the kernel threw a read error > >> message on one of the drives. So I had to sort out the problem on > >> the drive before I could complete the dump. Then, of course I had > >> to reslice, and format another drive to replace the ailing one, > >> before I could perform a restore(8), and start the nfe patch; build > >> && install kernel. Weird; the drive had only a few hours on it. > >> Well, anyway. The patch applied cleanly. So I built, and installed > >> a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" > >> in loader.conf(5), and bounced the box. Bad news: > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 30 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 29 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 28 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 27 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 26 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 25 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 24 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 23 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 22 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 21 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 20 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 19 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 18 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 17 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 16 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 15 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 14 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 13 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 12 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 11 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 10 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 9 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 8 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 7 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 6 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 5 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 4 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 3 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 2 > >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> > >> Just as before. In case it should make any difference; I'm > >> going to attach my copy of src/sys/dev/nfe/if_nfe.c in case > >> there are any differences in mine, that do not coincide with > >> your version/copy (I'm on releng_9 - 9.2-STABLE) > >> > >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: > >> Thu Apr 3 12:42:03 PDT 2014 > >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 > >> > >> Best wishes, and thanks again. > >> > > > > Oops, could you show me the output of "pciconf -lcbv"? > > Yes. Of course. > [...] > nfe0@pci0:0:7:0: class=0x068000 card=0x73091462 chip=0x03ef10de rev=0xa2 hdr=0x00 > vendor = 'nVidia Corporation' > device = 'MCP61 Ethernet' > class = bridge > bar [10] = type Memory, range 32, base 0xdff7d000, size 4096, enabled > bar [14] = type I/O Port, range 32, base 0xe480, size 8, enabled > cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 > cap 05[50] = MSI supports 8 messages, 64 bit, vector masks enabled with 8 messages > cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 Thanks a lot for the info. It seems I missed there are 4 variants for MCP61. Try attached patch again. --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="nfe.msi.diff2" Index: sys/dev/nfe/if_nfe.c =================================================================== --- sys/dev/nfe/if_nfe.c (revision 264061) +++ sys/dev/nfe/if_nfe.c (working copy) @@ -79,6 +79,7 @@ static int nfe_suspend(device_t); static int nfe_resume(device_t); static int nfe_shutdown(device_t); static int nfe_can_use_msix(struct nfe_softc *); +static int nfe_detect_msik9(struct nfe_softc *); static void nfe_power(struct nfe_softc *); static int nfe_miibus_readreg(device_t, int, int); static int nfe_miibus_writereg(device_t, int, int, int); @@ -334,13 +335,38 @@ nfe_alloc_msix(struct nfe_softc *sc, int count) } } + static int +nfe_detect_msik9(struct nfe_softc *sc) +{ + static char *maker = "MSI"; + static char *product = "K9N6PGM2-V2 (MS-7309)"; + char *m, *p; + int found; + + found = 0; + m = getenv("smbios.planar.maker"); + p = getenv("smbios.planar.product"); + if (m != NULL && p != NULL) { + if (strcmp(m, maker) == 0 && strcmp(p, product) == 0) + found = 1; + } + if (m != NULL) + freeenv(m); + if (p != NULL) + freeenv(p); + + return (found); +} + + +static int nfe_attach(device_t dev) { struct nfe_softc *sc; struct ifnet *ifp; bus_addr_t dma_addr_max; - int error = 0, i, msic, reg, rid; + int error = 0, i, msic, phyloc, reg, rid; sc = device_get_softc(dev); sc->nfe_dev = dev; @@ -608,8 +634,16 @@ nfe_attach(device_t dev) #endif /* Do MII setup */ + phyloc = MII_PHY_ANY; + if (sc->nfe_devid == PCI_PRODUCT_NVIDIA_MCP61_LAN1 || + sc->nfe_devid == PCI_PRODUCT_NVIDIA_MCP61_LAN2 || + sc->nfe_devid == PCI_PRODUCT_NVIDIA_MCP61_LAN3 || + sc->nfe_devid == PCI_PRODUCT_NVIDIA_MCP61_LAN4) { + if (nfe_detect_msik9(sc) != 0) + phyloc = 0; + } error = mii_attach(dev, &sc->nfe_miibus, ifp, nfe_ifmedia_upd, - nfe_ifmedia_sts, BMSR_DEFCAPMASK, MII_PHY_ANY, MII_OFFSET_ANY, + nfe_ifmedia_sts, BMSR_DEFCAPMASK, phyloc, MII_OFFSET_ANY, MIIF_DOPAUSE); if (error != 0) { device_printf(dev, "attaching PHYs failed\n"); --EeQfGwPcQSOJBaQU-- From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 06:09:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DDB5341; Mon, 7 Apr 2014 06:09:34 +0000 (UTC) Received: from tampoco.espindola.nl (tampoco.espindola.nl [149.210.133.191]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DE61D82; Mon, 7 Apr 2014 06:09:33 +0000 (UTC) Received: from [127.0.0.1] (corfu.internal.deze.org [192.168.1.2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: frank) by tampoco.espindola.nl (Postfix) with ESMTPSA id C22A2371BC6; Mon, 7 Apr 2014 08:09:01 +0200 (CEST) Message-ID: <53424100.7030103@deze.org> Date: Mon, 07 Apr 2014 08:09:04 +0200 From: Frank Volf User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pyunyh@gmail.com, Rick Macklem Subject: Re: re0: watchdog timeout References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> In-Reply-To: <20140407012241.GA3543@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 06:09:34 -0000 Yonghyeon PYUN schreef op 7-4-2014 3:22: > On Sun, Apr 06, 2014 at 07:37:08PM -0400, Rick Macklem wrote: >> Frank Volf wrote: >>> Hello, >>> >>> I'm experiencing watchdog timeouts with my Realtek interface card. >>> >>> I'm using a fairly new system (Shuttle DS47), running FreeBSD >>> 10-STABLE. >>> For this shuttle a patch has been recently committed to SVN to make >>> this >>> card work at all (revision *262391* >>> ). >>> >>> The timeout is only experienced under heavy network load (the system >>> is >>> running a bacula backup server that backups to NFS connected >>> storage), >>> and typically large full backups trigger this. Normal traffic works >>> fine >>> (this system is e.g. also my firewall to the Internet). >>> >> Since you mention NFS, you could try disabling TSO on the interface >> and see if that helps. (I'm beginning to feel like a parrot saying this, >> but...) If you care about why it might help, read this email thread: >> http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root >> >> If it happens to help, please email again, since there are probably >> better ways to fix the problem than disabling TSO. >> > re(4) controllers support TSO but it was disabled long time > ago(r217832). > It's still allowed to enable TSO but users have to explicitly > enable it with ifconfig. If Frank didn't explicitly enable TSO on > the box, TSO may have nothing to do with watchdog timeout, I guess. I haven't explicitly enabled TSO, the only option that has been explicitly set is -vlanhwtag, here is the interface config: re0: flags=8843 metric 0 mtu 1500 options=8208b ether 80:ee:73:77:e9:ab nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Regards, Frank From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 08:32:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4986318; Mon, 7 Apr 2014 08:32:44 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABBD3B23; Mon, 7 Apr 2014 08:32:44 +0000 (UTC) Received: by mail-pd0-f176.google.com with SMTP id r10so6241004pdi.35 for ; Mon, 07 Apr 2014 01:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Sb0FZg9bxMmAXVsjBw0ow7cRBIsOWgeEQKlPWQQblDw=; b=ZB521wF//gQPRhx193el44RmpUJHjFypFQT0dtQqPBnSvPbLV/M2n0pFO36AbWNvph AqXnGcR2QBjJsygkRll7BZigPNvTnGZKIJI+os4rAZXPiqoMCTevH57Ux8H4nhHipuK7 npZxjTXZrxlMCm1qhiVfQkeEKEXlBR8hwGjqPRvaGH4Q4mkIP6L/E2nF+cGKZ/ezqY7R jAZELPRxrsHtqoy7Yygksn3tUdUCqp37F6qk58TjQrUDzBBfcQLZhWgm1nlaRYPMmXIG E2rJXNkYdY9YbeyLOlR5dTM1ONPUkr9JZu43v9nZxf6bv9quvWKJQT09B3W6jEADi6aQ Bu7w== X-Received: by 10.68.138.227 with SMTP id qt3mr29962514pbb.6.1396859564244; Mon, 07 Apr 2014 01:32:44 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id tk5sm35111366pbc.63.2014.04.07.01.32.40 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 07 Apr 2014 01:32:43 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 07 Apr 2014 17:32:37 +0900 From: Yonghyeon PYUN Date: Mon, 7 Apr 2014 17:32:37 +0900 To: Frank Volf Subject: Re: re0: watchdog timeout Message-ID: <20140407083236.GB1357@michelle.cdnetworks.com> References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> <53424100.7030103@deze.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53424100.7030103@deze.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, marius@freebsd.org, Rick Macklem , yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 08:32:45 -0000 On Mon, Apr 07, 2014 at 08:09:04AM +0200, Frank Volf wrote: > Yonghyeon PYUN schreef op 7-4-2014 3:22: > >On Sun, Apr 06, 2014 at 07:37:08PM -0400, Rick Macklem wrote: > >>Frank Volf wrote: > >>>Hello, > >>> > >>>I'm experiencing watchdog timeouts with my Realtek interface card. > >>> > >>>I'm using a fairly new system (Shuttle DS47), running FreeBSD > >>>10-STABLE. > >>>For this shuttle a patch has been recently committed to SVN to make > >>>this > >>>card work at all (revision *262391* > >>>). > >>> > >>>The timeout is only experienced under heavy network load (the system > >>>is > >>>running a bacula backup server that backups to NFS connected > >>>storage), > >>>and typically large full backups trigger this. Normal traffic works > >>>fine > >>>(this system is e.g. also my firewall to the Internet). > >>> > >>Since you mention NFS, you could try disabling TSO on the interface > >>and see if that helps. (I'm beginning to feel like a parrot saying this, > >>but...) If you care about why it might help, read this email thread: > >> http://docs.FreeBSD.org/cgi/mid.cgi?1850411724.1687820.1395621539316.JavaMail.root > >> > >>If it happens to help, please email again, since there are probably > >>better ways to fix the problem than disabling TSO. > >> > >re(4) controllers support TSO but it was disabled long time > >ago(r217832). > >It's still allowed to enable TSO but users have to explicitly > >enable it with ifconfig. If Frank didn't explicitly enable TSO on > >the box, TSO may have nothing to do with watchdog timeout, I guess. > > I haven't explicitly enabled TSO, the only option that has been > explicitly set is -vlanhwtag, here is the interface config: > > re0: flags=8843 metric 0 mtu 1500 > options=8208b > ether 80:ee:73:77:e9:ab > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > It would be even better to know your network configuration. I'm not sure why you have to disable VLAN hardware tagging. But given that you've disabled it, could you also try disabling VLAN hardware checksum offloading? > > Regards, > > Frank > From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 09:14:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E77464C9 for ; Mon, 7 Apr 2014 09:14:31 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BEBD9FB4 for ; Mon, 7 Apr 2014 09:14:31 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so6507992pad.28 for ; Mon, 07 Apr 2014 02:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=3E/E4P3LsQilRbCi1qafTrC8X+srfrLh9hvgnowwYRo=; b=KkQx3efiDD6toF08IbFTsbuIL5Ad6P74V2qGFfRONZIsotZbOyAnfs0bnhkmBvDfcY 2AVGfLRni5c7ZdYPZOX/HeJhrzlodOCakpKi76bxVFvtZl2TGqIe1UAUTz4+sMou+Bxq He56cBFmo6XylHhsVeeQ7bgaRQbazeeOQ2GxpZTShlQlO1nozKHoeKjvEcKI/U4AoDnh ElV023tpemKDpC3gRPd3srrqLnq3fxrBrNnFoZg7HriXEs2RuCO8TWY33vei+AhL86n4 hhqxTgYB9RmnXyb0htPUTXQBr978pVuJRwFnOILbIwJ/q7+lLo4kxz+XltKXqEl9ilra GRVw== X-Received: by 10.68.196.226 with SMTP id ip2mr6610436pbc.106.1396862071382; Mon, 07 Apr 2014 02:14:31 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.121]) by mx.google.com with ESMTPSA id ek2sm24046377pbd.30.2014.04.07.02.14.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Apr 2014 02:14:30 -0700 (PDT) Message-ID: <53426C6F.5060609@gmail.com> Date: Mon, 07 Apr 2014 17:14:23 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: FreeBSD Net Subject: may some issue related to lo0 or socket Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 09:14:32 -0000 Hi,lists, I ran a forward squid instance and a haproxy instanse on one FreeBSD 9.1 box, it's works fine. When I upgraded it to 10-stable, I found I can't buffer videos normally unless I set "tcp_recv_bufsize" to a higher value, eg. 32768. The squid instance forward http request to the haproxy and accept the response from the haproxy via lo0. I think maybe there is some issue, I have tried upgraded the os to r264098 and disable lo0's csum, but it's not helped. How can I investigate the issue deeply ? Regards Simon From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 09:40:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29E67B82 for ; Mon, 7 Apr 2014 09:40:11 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D901B20A for ; Mon, 7 Apr 2014 09:40:10 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WX5nY-0002dr-9N for freebsd-net@freebsd.org; Mon, 07 Apr 2014 11:25:00 +0200 Received: from 217.30.88.7 ([217.30.88.7]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 07 Apr 2014 11:25:00 +0200 Received: from jcharbon by 217.30.88.7 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 07 Apr 2014 11:25:00 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: panic in -HEAD multicast code Date: Mon, 07 Apr 2014 11:24:54 +0200 Lines: 38 Message-ID: <53426EE6.5020409@verisign.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 217.30.88.7 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 09:40:11 -0000 Hi Adrian, On 07/04/14 04:58, Adrian Chadd wrote: > I'm seeing a panic in the multicast code path. I reproduce this by > trying to browse network sharing on VLC. > > The panic: > > http://people.freebsd.org/~adrian/ath/core.txt.0 > > Any ideas? We believe this issue is due to a race condition in multicast code. We found it tracking down another unrelated bug: http://www.freebsd.org/cgi/query-pr.cgi?pr=185043 See provided details on this race condition in PR description section starting with: "With even this patch in place, we have further found a subsequent race condition that...". In short, it looks like a race between a thread setting/allocating a new multicast address on an interface and calling: inp_setmoptions() -> inp_join_group() -> inp_join_group_locked() -> in_getmulti() and a thread detaching this interface, purging all associated multicast addresses and calling: if_detach() -> if_detach_internal() -> if_purgemaddrs() -> if_delmulti_locked() -> if_freemulti() However, we did not find a way to fix it without unwanted side effects and we asked for comments/ideas. -- Julien From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 11:06:48 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C731B4F for ; Mon, 7 Apr 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 78B81BFF for ; Mon, 7 Apr 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s37B6mIC071139 for ; Mon, 7 Apr 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s37B6lK9071137 for freebsd-net@FreeBSD.org; Mon, 7 Apr 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 7 Apr 2014 11:06:47 GMT Message-Id: <201404071106.s37B6lK9071137@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/188245 net [vlan] Critical vlan problem with OpenBGP o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 461 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 15:31:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25981B99 for ; Mon, 7 Apr 2014 15:31:03 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD21DDC7 for ; Mon, 7 Apr 2014 15:31:02 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id n12so6905187wgh.0 for ; Mon, 07 Apr 2014 08:31:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PVwI6kv45XHw1RMU6cl6/u4RwwY4wvwchgoPlb2f56s=; b=BL74D9W2oQLSS0sIIP1cMCa+aV73LoavR2LoQp5vHzgG+slnccQnbsXbUcnb2s6LZL 51KytUsissPpccDKEbod8LR0VMTlHR8EIf1gHXFZb7XBwigdbKvT/APwHTn40DlTPgSb tDTGLofE6RNe+FZwqvqlPoda1Ju53OT3UYkpcsJhhzb2P5KWdeM9uvvgpAalfAvXkfz7 Dh/HgUMizCzbLrG0taQBEL9EC7BSJaIObBI/XvFjss5xS34MBwgmFnyo97FsV33mHFWP yRDJ9rFMFrxI9FW8Sr2CZvhJWewUn7a6CzC/9c+fMzDETKrmmwN3aJvoobP04GBQ1dTa o6HA== MIME-Version: 1.0 X-Received: by 10.180.187.16 with SMTP id fo16mr26341355wic.26.1396884660997; Mon, 07 Apr 2014 08:31:00 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Mon, 7 Apr 2014 08:31:00 -0700 (PDT) In-Reply-To: <533F68EF.8060607@nevermind.co.nz> References: <533F68EF.8060607@nevermind.co.nz> Date: Mon, 7 Apr 2014 09:31:00 -0600 X-Google-Sender-Auth: Fp__wGkccCq4qFPu-7-0-eijrh8 Message-ID: Subject: Re: Multihomed system with jails routing issues From: Alan Somers To: Chris Smith Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 15:31:03 -0000 On Fri, Apr 4, 2014 at 8:22 PM, Chris Smith wrote: > Hi All, > > I have a system with 1 network interface with 2 extra VLANs off it and I'm > having some trouble getting the routing working correctly with it and jails. > > bge0 - management - 10.71.100.0/24 > bge0.101 - LAN - 10.71.101.0/24 > bge0.103 - DMZ - 10.71.101.0/24 > > Here's what I want to achieve... > > Host: > I want the host system to only listen on one interface, bge0. I want NO ip > addresses of the host on the vlan interfaces. The only service it will be > exposing is its sshd. The management address for this system is > 10.71.100.50. > > Jails: > The system will also host a variety of jails, each with an IP either on the > LAN or DMZ. I am using ezjail to manage the jails. > > Router: > There is a router at the .254 address of every subnet that can route between > each network. > > I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host does not > have an address configured on bge0.101, I configured the jail address as /24 > instead of the default /32. > > My issues: > > * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot > communicate with the jail. > > * When the jail is up and 10.71.101.51/24 is active, SSHing from the LAN to > the mgmt interface via the router fails, as the host tries to send return > traffic via the bge0.101 interface, even though traffic arrived via the bge0 > interface. > > So I did a whole lot of research for people having these apparently > problems, and decided to try the multiple routing table/fib approach. So I > recompiled my kernel, configured fib 1 with the LAN interface route (setfib > route add 10.71.101.0/24 -iface bge0.101), set the jail fib and set the > tunable net.addr_all_fibs = 0. I still can't get this working correctly. > ezjail still seems to add the interface route to fib 0 by default (but it > won't if i run ezjail with the setfib 1 command). Routing is very broken when you use multiple FIBs and set net.addr_all_fibs=0. See bugs http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/167947 http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/187549 http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/187550 http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/187551 http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/187552 http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/187553 http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/187699 I'm actively working on fixing the above, but getting consensus on the solution is slow. If you can't solve your problem without using multiple FIBs, I will suggest a set of patches that may help you. I will need 1) Your ifconfig_bge0 etc lines from /etc/rc.conf 2) The output for "setfib X netstat -rn -f inet" for each fib in use. 3) The rc.conf lines for any manually create routes that you're using, for example gateways. Also you may want to look at this possibly relevant bug. I'm not involved with this one: http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/181794 -Alan > > Using FIB 1 and trying to ping hosts on the LAN gives an error like: sendto > failed: invalid argument. > > Does anybody have any best practices for doing this, or anything else I can > try? I'm happy to share/pastebin any configuration and I've tried most > things I've found on the internet. I'm using FreeBSD 10.0 with a custom > kernel for multiple routing tables. > > Thanks in advance! > Chris. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 16:37:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F0EB71; Mon, 7 Apr 2014 16:37:46 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 89361694; Mon, 7 Apr 2014 16:37:45 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s37Gewm1008702; Mon, 7 Apr 2014 09:41:04 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s37GeqtQ008698; Mon, 7 Apr 2014 09:40:52 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from unavailable02.ultimatedns.net ([209.180.214.228]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Mon, 7 Apr 2014 09:40:53 -0700 (PDT) Message-ID: <366dff3399e9d5960e5a89902cb81dc3.authenticated@ultimatedns.net> In-Reply-To: <20140407060351.GA1357@michelle.cdnetworks.com> References: <20140401065842.GA1364@michelle.cdnetworks.com> <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> <20140407015104.GC3543@michelle.cdnetworks.com> <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> <20140407060351.GA1357@michelle.cdnetworks.com> Date: Mon, 7 Apr 2014 09:40:53 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 16:37:46 -0000 > On Sun, Apr 06, 2014 at 10:49:27PM -0700, Chris H wrote: >> > On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: >> >> > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: >> >> >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: >> >> >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> >> >> >> >> [...] >> >> >> >> >> miibus0: on nfe0 >> >> >> >> >> rlphy0: PHY 0 on miibus0 >> >> >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> >> >> >> >> rlphy1: PHY 1 on miibus0 >> >> >> >> > [...]---big-snip--8<--- >> >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> >> >> >> >> >> >> As you can see, it looks much the same. I have no idea what >> >> >> >> >> I should do to better inform the driver/kernel how to better >> >> >> >> >> handle it. Or is it the driver, itself? >> >> >> >> >> >> >> >> >> >> Thank you again, for your thoughtful response. >> >> >> >> >> >> >> >> >> >> --Chris >> >> >> >> >> >> >> >> >> > >> >> >> >> > I think the way to fix a phy that responds at all addresses is to set a >> >> >> >> > hint in loader.conf masking out the ones that aren't real, like so: >> >> >> >> > >> >> >> >> > hint.miibus.0.phymask="1" >> >> >> >> > >> >> >> >> > You might be able to set ="0x00000001" to make it more clear it's a >> >> >> >> > bitmask, but I'm not sure of that. >> >> >> >> >> >> >> >> Thank you very much for the hint. I'll give it a shot. >> >> >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia >> >> >> >> chipset, and the nfe(4) driver. But they don't respond this way. >> >> >> >> >> >> >> > >> >> >> > If some nfe(4) variants badly behave in probing stage, this should >> >> >> > be handled by driver. We already have too many hints and tunables >> >> >> > and I don't think most users know that. In addition, adding >> >> >> > additional NIC may change miibus instance number. >> >> >> > >> >> >> > Could you show me the output of 'kenv | grep smbios'? >> >> >> Yes, of course. >> >> >> >> >> >> Here it is: >> >> >> >> >> >> smbios.bios.reldate="11/22/2010" >> >> >> smbios.bios.vendor="American Megatrends Inc." >> >> >> smbios.bios.version="V2.7" >> >> >> smbios.chassis.maker="MSI" >> >> >> smbios.chassis.serial="To Be Filled By O.E.M." >> >> >> smbios.chassis.tag="To Be Filled By O.E.M." >> >> >> smbios.chassis.version="2.0" >> >> >> smbios.memory.enabled="2097152" >> >> >> smbios.planar.maker="MSI" >> >> >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" >> >> >> smbios.planar.serial="To be filled by O.E.M." >> >> >> smbios.planar.version="2.0" >> >> >> smbios.socket.enabled="1" >> >> >> smbios.socket.populated="1" >> >> >> smbios.system.maker="MSI" >> >> >> smbios.system.product="MS-7309" >> >> >> smbios.system.serial="To Be Filled By O.E.M." >> >> >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" >> >> >> smbios.system.version="2.0" >> >> >> smbios.version="2.6" >> >> >> >> >> >> Hope this helps, and thank you for all your time, and trouble. >> >> >> >> >> > >> >> > Thanks for the info. Try attached patch and let me know how it >> >> > works. Make sure to remove the hint(hint.miibus.0.phymask="1") >> >> > set in loader.conf before testing it. >> >> >> >> Hello, and thanks for all the attention. >> >> Sorry for the delay. I chose to perform a dump(8) before attempting >> >> the KERn rebuild with the patch. But the kernel threw a read error >> >> message on one of the drives. So I had to sort out the problem on >> >> the drive before I could complete the dump. Then, of course I had >> >> to reslice, and format another drive to replace the ailing one, >> >> before I could perform a restore(8), and start the nfe patch; build >> >> && install kernel. Weird; the drive had only a few hours on it. >> >> Well, anyway. The patch applied cleanly. So I built, and installed >> >> a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" >> >> in loader.conf(5), and bounced the box. Bad news: >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 30 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 29 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 28 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 27 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 26 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 25 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 24 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 23 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 22 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 21 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 20 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 19 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 18 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 17 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 16 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 15 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 14 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 13 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 12 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 11 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 10 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 9 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 8 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 7 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 6 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 5 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 4 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 3 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 2 >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> Just as before. In case it should make any difference; I'm >> >> going to attach my copy of src/sys/dev/nfe/if_nfe.c in case >> >> there are any differences in mine, that do not coincide with >> >> your version/copy (I'm on releng_9 - 9.2-STABLE) >> >> >> >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: >> >> Thu Apr 3 12:42:03 PDT 2014 >> >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 >> >> >> >> Best wishes, and thanks again. >> >> >> > >> > Oops, could you show me the output of "pciconf -lcbv"? >> >> Yes. Of course. >> > > [...] > >> nfe0@pci0:0:7:0: class=0x068000 card=0x73091462 chip=0x03ef10de rev=0xa2 hdr=0x00 >> vendor = 'nVidia Corporation' >> device = 'MCP61 Ethernet' >> class = bridge >> bar [10] = type Memory, range 32, base 0xdff7d000, size 4096, enabled >> bar [14] = type I/O Port, range 32, base 0xe480, size 8, enabled >> cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 >> cap 05[50] = MSI supports 8 messages, 64 bit, vector masks enabled with 8 messages >> cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 > > Thanks a lot for the info. It seems I missed there are 4 variants > for MCP61. Try attached patch again. Greetings, and thank you for your continued efforts. I just applied the patch, and built/installed a kernel with it. The results are in: /boot/loader.conf: loader_logo="beastiebw" #hint.miibus.0.phymask="0x00000001" relevant output from dmesg(8): nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff irq 20 at device 7.0 on pci0 nfe0: attempting to allocate 8 MSI vectors (8 supported) msi: routing MSI IRQ 257 to local APIC 0 vector 56 msi: routing MSI IRQ 258 to local APIC 0 vector 57 msi: routing MSI IRQ 259 to local APIC 0 vector 58 msi: routing MSI IRQ 260 to local APIC 0 vector 59 msi: routing MSI IRQ 261 to local APIC 0 vector 60 msi: routing MSI IRQ 262 to local APIC 0 vector 61 msi: routing MSI IRQ 263 to local APIC 0 vector 62 msi: routing MSI IRQ 264 to local APIC 0 vector 63 nfe0: using IRQs 257-264 for MSI nfe0: Using 8 MSI messages miibus0: on nfe0 rlphy0: PHY 0 on miibus0 rlphy0: OUI 0x000004, model 0x0020, rev. 1 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow nfe0: bpf attached nfe0: Ethernet address: 40:61:86:cd:44:97 ... vlan: initialized, using hash tables with chaining ... lo0: bpf attached ... Hmmm. I think we have a winner. No? The only thing that I noticed that appeared to be different. Is that it takes some 5 seconds to get from: nfe0: link state changed to DOWN nfe0: link state changed to UP to: Starting Network: ... output. I don't know that your patch had anything to do with that. Or it's just another anomaly with the driver/NIC. But it was a long enough period/change, that I thought it worth mentioning. Thank you again, for all your time, and efforts. --Chris > From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 18:45:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4330F6F2; Mon, 7 Apr 2014 18:45:07 +0000 (UTC) Received: from tampoco.espindola.nl (tampoco.espindola.nl [149.210.133.191]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 73D45362; Mon, 7 Apr 2014 18:45:06 +0000 (UTC) Received: from [127.0.0.1] (corfu.internal.deze.org [192.168.1.2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: frank) by tampoco.espindola.nl (Postfix) with ESMTPSA id 05E1A371E7A; Mon, 7 Apr 2014 20:44:58 +0200 (CEST) Message-ID: <5342F22C.5080606@deze.org> Date: Mon, 07 Apr 2014 20:45:00 +0200 From: Frank Volf User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: re0: watchdog timeout References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> <53424100.7030103@deze.org> <20140407083236.GB1357@michelle.cdnetworks.com> In-Reply-To: <20140407083236.GB1357@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 18:45:07 -0000 Yonghyeon PYUN schreef op 7-4-2014 10:32: > It would be even better to know your network configuration. I'm not > sure why you have to disable VLAN hardware tagging. But given that > you've disabled it, could you also try disabling VLAN hardware > checksum offloading? Hi, The reason that I disable VLAN hardware tagging is that the system does not work with it enabled. To show this, see the following transcript (on a freshly booted system): Script started on Mon Apr 7 20:30:43 2014 root@drawbridge:~ # ifconfig re0 re0: flags=8802 metric 0 mtu 1500 options=8209b ether 80:ee:73:77:e9:ab nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active root@drawbridge:~ # ifconfig re0.10 re0.10: flags=8843 metric 0 mtu 1500 options=3 ether 80:ee:73:77:e9:ab inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::82ee:73ff:fe77:e9ab%re0.10 prefixlen 64 scopeid 0x5 inet6 2001:470:7af9:10::1 prefixlen 64 nd6 options=21 media: Ethernet autoselect (100baseTX ) status: active vlan: 10 parent interface: re0 root@drawbridge:~ # ping 192.168.1.2 PING 192.168.1.2 (192.168.1.2): 56 data bytes ping: sendto: Host is down ping: sendto: Host is down ping: sendto: Host is down ping: sendto: Host is down ^C --- 192.168.1.2 ping statistics --- 9 packets transmitted, 0 packets received, 100.0% packet loss root@drawbridge:~ # ifconfig re0 -vlanhwtag root@drawbridge:~ # ifconfig re0 down root@drawbridge:~ # ifconfig re0 up root@drawbridge:~ # ifconfig re0 re0: flags=8843 metric 0 mtu 1500 options=8208b ether 80:ee:73:77:e9:ab nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active root@drawbridge:~ # ifconfig re0.10 re0.10: flags=8843 metric 0 mtu 1500 ether 80:ee:73:77:e9:ab inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::82ee:73ff:fe77:e9ab%re0.10 prefixlen 64 scopeid 0x5 inet6 2001:470:7af9:10::1 prefixlen 64 nd6 options=21 media: Ethernet autoselect (1000baseT ) status: active vlan: 10 parent interface: re0 root@drawbridge:~ # ping 192.168.1.2 PING 192.168.1.2 (192.168.1.2): 56 data bytes 64 bytes from 192.168.1.2: icmp_seq=0 ttl=64 time=0.250 ms 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.232 ms 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.283 ms 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=0.238 ms 64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=0.138 ms 64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=0.217 ms ^C --- 192.168.1.2 ping statistics --- 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.138/0.226/0.283/0.044 ms root@drawbridge:~ # exit Script done on Mon Apr 7 20:32:27 2014 From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 18:50:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC3B9832; Mon, 7 Apr 2014 18:50:23 +0000 (UTC) Received: from tampoco.espindola.nl (tampoco.espindola.nl [149.210.133.191]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AC0043BA; Mon, 7 Apr 2014 18:50:23 +0000 (UTC) Received: from [127.0.0.1] (corfu.internal.deze.org [192.168.1.2]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: frank) by tampoco.espindola.nl (Postfix) with ESMTPSA id B4383371BEF; Mon, 7 Apr 2014 20:50:21 +0200 (CEST) Message-ID: <5342F370.4090202@deze.org> Date: Mon, 07 Apr 2014 20:50:24 +0200 From: Frank Volf User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: re0: watchdog timeout References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> <53424100.7030103@deze.org> <20140407083236.GB1357@michelle.cdnetworks.com> In-Reply-To: <20140407083236.GB1357@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 18:50:24 -0000 Yonghyeon PYUN schreef op 7-4-2014 10:32: > It would be even better to know your network configuration. I'm not > sure why you have to disable VLAN hardware tagging. But given that > you've disabled it, could you also try disabling VLAN hardware > checksum offloading? HI Again, Sorry, forgot to answer the second part of your question, the disabling of VLAN hardware checksum. I can't get that to work, it won't enable even though the command has been accepted. Regards, Frank Script started on Mon Apr 7 20:46:33 2014 root@drawbridge:~ # ifconfig re0 re0: flags=8843 metric 0 mtu 1500 options=8208b ether 80:ee:73:77:e9:ab nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active root@drawbridge:~ # ifconfig re0 -vlanhwcsum root@drawbridge:~ # ifconfig re0 down root@drawbridge:~ # ifconfig re0 up root@drawbridge:~ # ifconfig re0 re0: flags=8843 metric 0 mtu 1500 options=8208b ether 80:ee:73:77:e9:ab nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active root@drawbridge:~ # exit exit Script done on Mon Apr 7 20:47:18 2014 From owner-freebsd-net@FreeBSD.ORG Mon Apr 7 22:23:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C07EB06 for ; Mon, 7 Apr 2014 22:23:06 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3FEB4C0D for ; Mon, 7 Apr 2014 22:23:06 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id uz6so100694obc.15 for ; Mon, 07 Apr 2014 15:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=9z8yf8sZP6tkqxydJU/oGsyKeemKnWoD//Hy2de2SmY=; b=tbhtnE6fSfexp7cTCu402CLfdJOSuCc/OyQlBIl4trTOKBvZF7d9/RsWIpnwa8jNxO +sDOruDs6wEuS3dI7yWHZsPUXjQX7kHEpk7UYcu5N+JaX/ouBb8GJb/Azn2v4zmBc2Qs c1rgPMZ1lmYPpwA+XKjS67yaZ8XI6XnDguJkfHm9My0N5IbDtbdl6EBB2jYKMAXR5k9C iUTKR3Egm6BN1iycyc3n7Ve9Z21FtyDAjSO/qLBanXiSTUMKUg9p9zv3xaIwmn4j1JRO Tya03XeriYDBhwj8VSWTln6rttBEv+5tPAQWsiI7hLdYDRq2sC6kRsVx+MXBsQSbr0Pl GLFQ== MIME-Version: 1.0 X-Received: by 10.182.65.199 with SMTP id z7mr25172obs.7.1396909385553; Mon, 07 Apr 2014 15:23:05 -0700 (PDT) Received: by 10.76.12.34 with HTTP; Mon, 7 Apr 2014 15:23:05 -0700 (PDT) In-Reply-To: References: <531771C8.1040207@yandex.ru> <20140306145231.Q75313@sola.nimnet.asn.au> <20140307024724.T75313@sola.nimnet.asn.au> Date: Tue, 8 Apr 2014 00:23:05 +0200 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: Ian Smith , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 22:23:06 -0000 On Wed, Mar 26, 2014 at 5:42 PM, Andreas Nilsson wrote: > ... snip ... > > >>> I'm wondering what's happening on the outbound path, most of your rules >>> handle inbound (to kernel) and it seems that rule 65535 deals with most >>> outbound, except those specifically acting on both paths. >>> >> So do I :) >> >>> >>> Maybe try adding to the above: >>> ipfw add 63510 count log ip from table(1) to any out recv table(8) >>> and >>> ipfw add 64510 count log ip from table(1) to any out recv table(8) >>> >>> which should catch them on the outbound path before the default rule; >>> outbound packets have both receive and transmit interfaces available. >>> >>> They never get that far :( Those rules log nothing. So I see the packet >> coming in, triggering the skipto, triggering the count log ... in recv but >> not the count log ... out. >> >> >>> I guess you're sure that these packets are actually going out to some >>> external address, not a localhost or alias address (which of course are >>> received and not sent out anywhere)? >>> >> Yes, when the go through they go to external address, which is in the >> routing table. >> >>> >>> I guess the above new log lines should show the interface (if any) >>> these packets are leaving on, after routing (maybe a routing issue?) >>> >>> Good luck hunting. If in doubt, throw ever more logging at it! :) And >>> please let the list know if you solve it - or not! >>> >>> cheers, Ian >>> >> >> To make it even more interesting, it tested this on another machine, >> where I'm unable to reproduce this "problem". I'm thinking that a reboot >> might help, but this is kind of fascinating so I would like to actually >> find the cause. I will investigate further. >> >> Best regards >> Andreas >> > > And now it happened on another machine, but different type of traffic. > Traffic triggering the divert rule work fine, some traffic not triggering > the divert rule does not. After everything works as expected. > > Before reboot I flushed the firewall so that only default rule was in play > ( allow all from any to any ), and then *no* traffic for that destination > went out of the box. > > There are something like ~1000 routes in the routing table. Routes are > added and removed according to some triggers, so during uptime of the > machine there is a bit of messing with the routing table. At this stage I'm > at a loss on how to continue investigating this, so I'll just implement the > forwarding via fwd rules in ipfw and altogether ignore the routing table. > > Best regards > Andreas > Ok, found the culprit: -# $FreeBSD: releng/9.1/etc/rc.d/netif 212579 2010-09-13 19:55:40Z hrs $ +# $FreeBSD: releng/9.2/etc/rc.d/netif 253238 2013-07-12 01:34:24Z hrs $ + if [ -f /etc/rc.d/routing -a -n "$cmdifn" ] ; then + for _if in $cmdifn; do + /etc/rc.d/routing start any $_if + done + fi etc/rc.d/routing enforces gateway_enable setting in rc.conf which does not play well with setting net.inet.ip.forwarding=1 in sysctl.conf directly. Very annoying. I can find nothing i UPDATING on the subject. Even more annoying. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 04:38:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1628CB2B; Tue, 8 Apr 2014 04:38:55 +0000 (UTC) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA91F1058; Tue, 8 Apr 2014 04:38:54 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id jt11so466809pbb.0 for ; Mon, 07 Apr 2014 21:38:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=aQTTd3PpwSsqEsWbxZbtvouaOif0lNU/515pS0zToVE=; b=I6n8sHo5iro6vhXFu71aWy1Tb4zJMIb3YNgrQAradMl+Mjuscj+MmOv6EA1aO0hiBf NZofRRVapu+OogUG15+waVb3lTEzD/TMADKmaT0ijL4GtXJ3j/aOsm4nxcfr+EECsIXM czVAf5OuZCC+6Dew4HJhF6DldSGvVRAmKUT8jw+Mec3gHlmbVe8aX6di4l5QiYz6fOz4 lBQasKt7RZ9ra0kP7Lc9L318R75rQmwlkBmPYbid4mcz3SmePYaO7oF802oNc2OHVLif HoPKQcd6lR+ZdoCuBqG5XtwsvL3P4Tr7tZavsdZzoPb+GEUV0yzD51ciYE/GXZ9ylW2S rDgw== X-Received: by 10.66.233.72 with SMTP id tu8mr1785423pac.112.1396931933780; Mon, 07 Apr 2014 21:38:53 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id fg12sm4071855pac.28.2014.04.07.21.38.50 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 07 Apr 2014 21:38:52 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 08 Apr 2014 13:38:48 +0900 From: Yonghyeon PYUN Date: Tue, 8 Apr 2014 13:38:48 +0900 To: Chris H Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 Message-ID: <20140408043848.GA1372@michelle.cdnetworks.com> References: <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> <20140407015104.GC3543@michelle.cdnetworks.com> <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> <20140407060351.GA1357@michelle.cdnetworks.com> <366dff3399e9d5960e5a89902cb81dc3.authenticated@ultimatedns.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <366dff3399e9d5960e5a89902cb81dc3.authenticated@ultimatedns.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 04:38:55 -0000 On Mon, Apr 07, 2014 at 09:40:53AM -0700, Chris H wrote: > > On Sun, Apr 06, 2014 at 10:49:27PM -0700, Chris H wrote: > >> > On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: > >> >> > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: > >> >> >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: > >> >> >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: > >> >> >> >> >> [...] > >> >> >> >> >> miibus0: on nfe0 > >> >> >> >> >> rlphy0: PHY 0 on miibus0 > >> >> >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > >> >> >> >> >> rlphy1: PHY 1 on miibus0 > >> >> >> >> > [...]---big-snip--8<--- > >> >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> >> >> >> >> > >> >> >> >> >> As you can see, it looks much the same. I have no idea what > >> >> >> >> >> I should do to better inform the driver/kernel how to better > >> >> >> >> >> handle it. Or is it the driver, itself? > >> >> >> >> >> > >> >> >> >> >> Thank you again, for your thoughtful response. > >> >> >> >> >> > >> >> >> >> >> --Chris > >> >> >> >> >> > >> >> >> >> > > >> >> >> >> > I think the way to fix a phy that responds at all addresses is to set a > >> >> >> >> > hint in loader.conf masking out the ones that aren't real, like so: > >> >> >> >> > > >> >> >> >> > hint.miibus.0.phymask="1" > >> >> >> >> > > >> >> >> >> > You might be able to set ="0x00000001" to make it more clear it's a > >> >> >> >> > bitmask, but I'm not sure of that. > >> >> >> >> > >> >> >> >> Thank you very much for the hint. I'll give it a shot. > >> >> >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia > >> >> >> >> chipset, and the nfe(4) driver. But they don't respond this way. > >> >> >> >> > >> >> >> > > >> >> >> > If some nfe(4) variants badly behave in probing stage, this should > >> >> >> > be handled by driver. We already have too many hints and tunables > >> >> >> > and I don't think most users know that. In addition, adding > >> >> >> > additional NIC may change miibus instance number. > >> >> >> > > >> >> >> > Could you show me the output of 'kenv | grep smbios'? > >> >> >> Yes, of course. > >> >> >> > >> >> >> Here it is: > >> >> >> > >> >> >> smbios.bios.reldate="11/22/2010" > >> >> >> smbios.bios.vendor="American Megatrends Inc." > >> >> >> smbios.bios.version="V2.7" > >> >> >> smbios.chassis.maker="MSI" > >> >> >> smbios.chassis.serial="To Be Filled By O.E.M." > >> >> >> smbios.chassis.tag="To Be Filled By O.E.M." > >> >> >> smbios.chassis.version="2.0" > >> >> >> smbios.memory.enabled="2097152" > >> >> >> smbios.planar.maker="MSI" > >> >> >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" > >> >> >> smbios.planar.serial="To be filled by O.E.M." > >> >> >> smbios.planar.version="2.0" > >> >> >> smbios.socket.enabled="1" > >> >> >> smbios.socket.populated="1" > >> >> >> smbios.system.maker="MSI" > >> >> >> smbios.system.product="MS-7309" > >> >> >> smbios.system.serial="To Be Filled By O.E.M." > >> >> >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" > >> >> >> smbios.system.version="2.0" > >> >> >> smbios.version="2.6" > >> >> >> > >> >> >> Hope this helps, and thank you for all your time, and trouble. > >> >> >> > >> >> > > >> >> > Thanks for the info. Try attached patch and let me know how it > >> >> > works. Make sure to remove the hint(hint.miibus.0.phymask="1") > >> >> > set in loader.conf before testing it. > >> >> > >> >> Hello, and thanks for all the attention. > >> >> Sorry for the delay. I chose to perform a dump(8) before attempting > >> >> the KERn rebuild with the patch. But the kernel threw a read error > >> >> message on one of the drives. So I had to sort out the problem on > >> >> the drive before I could complete the dump. Then, of course I had > >> >> to reslice, and format another drive to replace the ailing one, > >> >> before I could perform a restore(8), and start the nfe patch; build > >> >> && install kernel. Weird; the drive had only a few hours on it. > >> >> Well, anyway. The patch applied cleanly. So I built, and installed > >> >> a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" > >> >> in loader.conf(5), and bounced the box. Bad news: > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 30 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 29 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 28 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 27 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 26 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 25 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 24 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 23 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 22 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 21 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 20 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 19 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 18 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 17 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 16 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 15 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 14 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 13 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 12 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 11 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 10 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 9 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 8 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 7 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 6 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 5 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 4 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 3 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 2 > >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 > >> >> > >> >> Just as before. In case it should make any difference; I'm > >> >> going to attach my copy of src/sys/dev/nfe/if_nfe.c in case > >> >> there are any differences in mine, that do not coincide with > >> >> your version/copy (I'm on releng_9 - 9.2-STABLE) > >> >> > >> >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: > >> >> Thu Apr 3 12:42:03 PDT 2014 > >> >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 > >> >> > >> >> Best wishes, and thanks again. > >> >> > >> > > >> > Oops, could you show me the output of "pciconf -lcbv"? > >> > >> Yes. Of course. > >> > > > > [...] > > > >> nfe0@pci0:0:7:0: class=0x068000 card=0x73091462 chip=0x03ef10de rev=0xa2 hdr=0x00 > >> vendor = 'nVidia Corporation' > >> device = 'MCP61 Ethernet' > >> class = bridge > >> bar [10] = type Memory, range 32, base 0xdff7d000, size 4096, enabled > >> bar [14] = type I/O Port, range 32, base 0xe480, size 8, enabled > >> cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 > >> cap 05[50] = MSI supports 8 messages, 64 bit, vector masks enabled with 8 messages > >> cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 > > > > Thanks a lot for the info. It seems I missed there are 4 variants > > for MCP61. Try attached patch again. > > Greetings, and thank you for your continued efforts. > I just applied the patch, and built/installed a kernel with it. > The results are in: > > /boot/loader.conf: > loader_logo="beastiebw" > #hint.miibus.0.phymask="0x00000001" > > relevant output from dmesg(8): > nfe0: port 0xe480-0xe487 mem 0xdff7d000-0xdff7dfff > irq 20 at device 7.0 on pci0 > nfe0: attempting to allocate 8 MSI vectors (8 supported) > msi: routing MSI IRQ 257 to local APIC 0 vector 56 > msi: routing MSI IRQ 258 to local APIC 0 vector 57 > msi: routing MSI IRQ 259 to local APIC 0 vector 58 > msi: routing MSI IRQ 260 to local APIC 0 vector 59 > msi: routing MSI IRQ 261 to local APIC 0 vector 60 > msi: routing MSI IRQ 262 to local APIC 0 vector 61 > msi: routing MSI IRQ 263 to local APIC 0 vector 62 > msi: routing MSI IRQ 264 to local APIC 0 vector 63 > nfe0: using IRQs 257-264 for MSI > nfe0: Using 8 MSI messages > miibus0: on nfe0 > rlphy0: PHY 0 on miibus0 > rlphy0: OUI 0x000004, model 0x0020, rev. 1 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow > nfe0: bpf attached > nfe0: Ethernet address: 40:61:86:cd:44:97 > ... > vlan: initialized, using hash tables with chaining > ... > lo0: bpf attached > ... > > Hmmm. I think we have a winner. No? Thanks a lot for testing. I'll commit it within a couple of days. > The only thing that I noticed that appeared to be different. > Is that it takes some 5 seconds to get from: > > nfe0: link state changed to DOWN > nfe0: link state changed to UP > > to: > > Starting Network: ... > > output. I don't know that your patch had anything to do with I don't think the patch may result in the change you've seen. Auto-negotiation may take several seconds and that might be reason you see the difference. Link up message does not necessarily mean it resolved speed/duplex of link against link partner. nfe(4) explicitly checks whether it established a valid link(i.e. link should be up and speed/duplex also should be resolved). > that. Or it's just another anomaly with the driver/NIC. But it > was a long enough period/change, that I thought it worth > mentioning. > > Thank you again, for all your time, and efforts. > > --Chris > > > > > From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 04:59:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 821E4E17; Tue, 8 Apr 2014 04:59:45 +0000 (UTC) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4965011D8; Tue, 8 Apr 2014 04:59:45 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id kx10so489481pab.33 for ; Mon, 07 Apr 2014 21:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=UGgsozGKmYSW3phCGdT0XnUizSfhdePwJ4ofST1ovFY=; b=bNq+bvwk+EwHJmZ4WYtvtVCVVlSToHPVeMDKkNwoYt526wyLQNCIa3s4x+9jYalU59 oUEYebt3ceTlUwDHSnqnTh7MeQMuZrA87RI1AIVmYSq/cVtD6aWoYHuXavbi6nAylVkp PTCA1M3oZ/4Sb1rsb+eS9qaiyjhcPoEKCOOdjw9+YLogqzAXrwujUoO/pgeiRVDvgKMr 1J6FEZLX2hxzRuEq9VfFs9vnwfKKySoiG/blNXPDwtXZWlvhBDT9yVBz5WylMSmPM0s1 jLpKNCW/oiZQFfcfYBUEyKbOy6BLNysdtsVV/XcN3pxNiE4ut0lZMRW2RTj9zUaYhfwB /fwA== X-Received: by 10.66.177.168 with SMTP id cr8mr1898649pac.128.1396933184928; Mon, 07 Apr 2014 21:59:44 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id dn1sm1640714pbb.54.2014.04.07.21.59.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 07 Apr 2014 21:59:44 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 08 Apr 2014 13:59:39 +0900 From: Yonghyeon PYUN Date: Tue, 8 Apr 2014 13:59:39 +0900 To: Frank Volf Subject: Re: re0: watchdog timeout Message-ID: <20140408045939.GB1372@michelle.cdnetworks.com> References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> <53424100.7030103@deze.org> <20140407083236.GB1357@michelle.cdnetworks.com> <5342F22C.5080606@deze.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5342F22C.5080606@deze.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, marius@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 04:59:45 -0000 On Mon, Apr 07, 2014 at 08:45:00PM +0200, Frank Volf wrote: > Yonghyeon PYUN schreef op 7-4-2014 10:32: > >It would be even better to know your network configuration. I'm not > >sure why you have to disable VLAN hardware tagging. But given that > >you've disabled it, could you also try disabling VLAN hardware > >checksum offloading? > > Hi, > > The reason that I disable VLAN hardware tagging is that the system does > not work with it enabled. > To show this, see the following transcript (on a freshly booted system): [...] Okat, I'll check VLAN hardware tagging with RTL8168G but watchdog timeout is different issue. I have no idea why this happens at this moment but I'll let you know if I find a clue. Anyway, thanks for reporting. From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 08:22:18 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1966136B for ; Tue, 8 Apr 2014 08:22:18 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 69F2E13C4 for ; Tue, 8 Apr 2014 08:22:17 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA21061 for ; Tue, 08 Apr 2014 11:22:09 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WXRIH-000Lhn-Jo for freebsd-net@FreeBSD.org; Tue, 08 Apr 2014 11:22:09 +0300 Message-ID: <5343B178.8070604@FreeBSD.org> Date: Tue, 08 Apr 2014 11:21:12 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net Subject: re(4) causes memory corruption? X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 08:22:18 -0000 I have this network card (it's actually integrated into a motherboard): re0: port 0xde00-0xdeff mem 0xfdaff000-0xfdafffff,0xfdae0000-0xfdaeffff irq 18 at device 0.0 on pci2 re0: Using 1 MSI-X message re0: Chip rev. 0x3c000000 re0: MAC rev. 0x00400000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow When there is little traffic through the interface I do not observe any problems with it. But within 15 seconds of applying some moderate traffic I would always observe a heavy screen corruption often followed by a total freeze or a hardware self-reset. An example of the moderate traffic is 6 MBytes/s which results in about 10K interrupts per seconds. I am not sure what causes the problem. Could it be some driver using memoery that it should not or hardware writing where it should not or if this something completely in the hardware. I will appreciate any hints on possible ways to analyze this issue. Thanks! -- Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 08:25:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4532441 for ; Tue, 8 Apr 2014 08:25:44 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 864771435 for ; Tue, 8 Apr 2014 08:25:44 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id s7so34321qap.7 for ; Tue, 08 Apr 2014 01:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=CDI0/cjb42Si6H6KUFCa37LegIqBS4cV/ZxWzjxACMY=; b=kfgZZuWWNPY8c8TMycLEmr39QmEzaeaGx8qYsfU08v4Z65b3aPXLsIR3uzP2ZsZYx8 4mJ2k+b7K1l+ZHUlx/72ynF0fzMSX+nHhO+r4x57gFyLeABlnUga6CYm94urNptQRNnH xtkzrchyavB5CsJEgaf4/eUfi8Us65iB5156dgRUD+k+iD3WZc7TwEK9ZYAXV1KxJkQR ypVW8xQODx3V9dYPD7NnHbYWQIoerqzpqTMJvYY/E5CxMyIL+C4Uh6FTPzFpgqZIBKQJ lVo8Nl7Uha/d2ILvBz7zjDRYgrMgW3/SQ6iPi3iMGgC1wumJchkD0I8eO/AdwTW9hk/z zQKQ== MIME-Version: 1.0 X-Received: by 10.140.109.100 with SMTP id k91mr2547355qgf.12.1396945543704; Tue, 08 Apr 2014 01:25:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Tue, 8 Apr 2014 01:25:43 -0700 (PDT) In-Reply-To: <53426EE6.5020409@verisign.com> References: <53426EE6.5020409@verisign.com> Date: Tue, 8 Apr 2014 01:25:43 -0700 X-Google-Sender-Auth: HFSafEZ2lv3TAEYb2hh34vm1fUw Message-ID: Subject: Re: panic in -HEAD multicast code From: Adrian Chadd To: Julien Charbon Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 08:25:44 -0000 Hm, how's this happening here? I'm not detaching the interface. -a From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 10:46:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25E25461; Tue, 8 Apr 2014 10:46:15 +0000 (UTC) Received: from exprod6og114.obsmtp.com (exprod6og114.obsmtp.com [64.18.1.33]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A22B0114F; Tue, 8 Apr 2014 10:46:13 +0000 (UTC) Received: from peregrine.verisign.com ([216.168.239.74]) (using TLSv1) by exprod6ob114.postini.com ([64.18.5.12]) with SMTP ID DSNKU0PTbtv42f0TFJ0nXAFbUevxcfHzT0UO@postini.com; Tue, 08 Apr 2014 03:46:14 PDT Received: from DUL1WNSMTP01.vcorp.ad.vrsn.com (dul1wnexcn03.vcorp.ad.vrsn.com [10.170.12.113]) by peregrine.verisign.com (8.13.6/8.13.4) with ESMTP id s38Ak2vi030221; Tue, 8 Apr 2014 06:46:06 -0400 Received: from FRI2JCHARBON-M1.local ([10.100.64.29]) by DUL1WNSMTP01.vcorp.ad.vrsn.com with Microsoft SMTPSVC(7.5.7601.17514); Tue, 8 Apr 2014 06:46:02 -0400 Message-ID: <5343D368.6020308@verisign.com> Date: Tue, 08 Apr 2014 12:46:00 +0200 From: Julien Charbon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 Newsgroups: gmane.os.freebsd.devel.net To: Adrian Chadd Subject: Re: panic in -HEAD multicast code References: <53426EE6.5020409@verisign.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 08 Apr 2014 10:46:02.0783 (UTC) FILETIME=[BC6FCAF0:01CF5317] Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 10:46:15 -0000 Hi Adrian, On 08/04/14 10:25, Adrian Chadd wrote: > Hm, how's this happening here? I'm not detaching the interface. Hm, if your are positive that nothing is detaching the interface on your behalf (like a /etc/rc.d/netif restart somewhere), then it looks like you got an unrelated case that just drives the same stacktrace than the race condition we found. -- Julien From owner-freebsd-net@FreeBSD.ORG Tue Apr 8 18:40:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5FD49B; Tue, 8 Apr 2014 18:40:13 +0000 (UTC) Received: from udns.ultimateDNS.NET (ultimatedns.net [209.180.214.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37B39136C; Tue, 8 Apr 2014 18:40:12 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s38IhQEo005129; Tue, 8 Apr 2014 11:43:32 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s38IhJem005126; Tue, 8 Apr 2014 11:43:19 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net ([209.180.214.225]) (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Tue, 8 Apr 2014 11:43:21 -0700 (PDT) Message-ID: In-Reply-To: <20140408043848.GA1372@michelle.cdnetworks.com> References: <1396384167.81853.210.camel@revolution.hippie.lan> <41917a9e67d0f4519df4b55f3aa6ebe3.authenticated@ultimatedns.net> <20140402003912.GA2938@michelle.cdnetworks.com> <3f97f5646629043fed5e34a77c9c2f3d.authenticated@ultimatedns.net> <20140402020803.GB2938@michelle.cdnetworks.com> <70cd5de109845e30fcb6516af7a6f9de.authenticated@ultimatedns.net> <20140407015104.GC3543@michelle.cdnetworks.com> <16934e2751614ab9235ac00c6b6bb2d7.authenticated@ultimatedns.net> <20140407060351.GA1357@michelle.cdnetworks.com> <366dff3399e9d5960e5a89902cb81dc3.authenticated@ultimatedns.net> <20140408043848.GA1372@michelle.cdnetworks.com> Date: Tue, 8 Apr 2014 11:43:21 -0700 (PDT) Subject: Re: miibus0: mii_mediachg: can't handle non-zero PHY instance 31 From: "Chris H" To: pyunyh@gmail.com User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net , freebsd-stable , Ian Lepore X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 18:40:13 -0000 > On Mon, Apr 07, 2014 at 09:40:53AM -0700, Chris H wrote: >> > On Sun, Apr 06, 2014 at 10:49:27PM -0700, Chris H wrote: >> >> > On Thu, Apr 03, 2014 at 01:18:19PM -0700, Chris H wrote: >> >> >> > On Tue, Apr 01, 2014 at 05:53:51PM -0700, Chris H wrote: >> >> >> >> > On Tue, Apr 01, 2014 at 01:40:58PM -0700, Chris H wrote: >> >> >> >> >> > On Tue, 2014-04-01 at 13:19 -0700, Chris H wrote: >> >> >> >> >> >> [...] >> >> >> >> >> >> miibus0: on nfe0 >> >> >> >> >> >> rlphy0: PHY 0 on miibus0 >> >> >> >> >> >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> >> >> >> >> >> rlphy1: PHY 1 on miibus0 >> >> >> >> >> > [...]---big-snip--8<--- >> >> >> >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> >> >> >> >> >> >> >> >> As you can see, it looks much the same. I have no idea what >> >> >> >> >> >> I should do to better inform the driver/kernel how to better >> >> >> >> >> >> handle it. Or is it the driver, itself? >> >> >> >> >> >> >> >> >> >> >> >> Thank you again, for your thoughtful response. >> >> >> >> >> >> >> >> >> >> >> >> --Chris >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> > I think the way to fix a phy that responds at all addresses is to set a >> >> >> >> >> > hint in loader.conf masking out the ones that aren't real, like so: >> >> >> >> >> > >> >> >> >> >> > hint.miibus.0.phymask="1" >> >> >> >> >> > >> >> >> >> >> > You might be able to set ="0x00000001" to make it more clear it's a >> >> >> >> >> > bitmask, but I'm not sure of that. >> >> >> >> >> >> >> >> >> >> Thank you very much for the hint. I'll give it a shot. >> >> >> >> >> Any idea why this is happening? I have 4 other MB's using the Nvidia >> >> >> >> >> chipset, and the nfe(4) driver. But they don't respond this way. >> >> >> >> >> >> >> >> >> > >> >> >> >> > If some nfe(4) variants badly behave in probing stage, this should >> >> >> >> > be handled by driver. We already have too many hints and tunables >> >> >> >> > and I don't think most users know that. In addition, adding >> >> >> >> > additional NIC may change miibus instance number. >> >> >> >> > >> >> >> >> > Could you show me the output of 'kenv | grep smbios'? >> >> >> >> Yes, of course. >> >> >> >> >> >> >> >> Here it is: >> >> >> >> >> >> >> >> smbios.bios.reldate="11/22/2010" >> >> >> >> smbios.bios.vendor="American Megatrends Inc." >> >> >> >> smbios.bios.version="V2.7" >> >> >> >> smbios.chassis.maker="MSI" >> >> >> >> smbios.chassis.serial="To Be Filled By O.E.M." >> >> >> >> smbios.chassis.tag="To Be Filled By O.E.M." >> >> >> >> smbios.chassis.version="2.0" >> >> >> >> smbios.memory.enabled="2097152" >> >> >> >> smbios.planar.maker="MSI" >> >> >> >> smbios.planar.product="K9N6PGM2-V2 (MS-7309)" >> >> >> >> smbios.planar.serial="To be filled by O.E.M." >> >> >> >> smbios.planar.version="2.0" >> >> >> >> smbios.socket.enabled="1" >> >> >> >> smbios.socket.populated="1" >> >> >> >> smbios.system.maker="MSI" >> >> >> >> smbios.system.product="MS-7309" >> >> >> >> smbios.system.serial="To Be Filled By O.E.M." >> >> >> >> smbios.system.uuid="00000000-0000-0000-0000-406186cd4497" >> >> >> >> smbios.system.version="2.0" >> >> >> >> smbios.version="2.6" >> >> >> >> >> >> >> >> Hope this helps, and thank you for all your time, and trouble. >> >> >> >> >> >> >> > >> >> >> > Thanks for the info. Try attached patch and let me know how it >> >> >> > works. Make sure to remove the hint(hint.miibus.0.phymask="1") >> >> >> > set in loader.conf before testing it. >> >> >> >> >> >> Hello, and thanks for all the attention. >> >> >> Sorry for the delay. I chose to perform a dump(8) before attempting >> >> >> the KERn rebuild with the patch. But the kernel threw a read error >> >> >> message on one of the drives. So I had to sort out the problem on >> >> >> the drive before I could complete the dump. Then, of course I had >> >> >> to reslice, and format another drive to replace the ailing one, >> >> >> before I could perform a restore(8), and start the nfe patch; build >> >> >> && install kernel. Weird; the drive had only a few hours on it. >> >> >> Well, anyway. The patch applied cleanly. So I built, and installed >> >> >> a new kernel with it. X's out the hint.miibus.0.phymask="0x00000001" >> >> >> in loader.conf(5), and bounced the box. Bad news: >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 31 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 30 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 29 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 28 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 27 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 26 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 25 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 24 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 23 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 22 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 21 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 20 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 19 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 18 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 17 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 16 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 15 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 14 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 13 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 12 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 11 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 10 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 9 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 8 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 7 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 6 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 5 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 4 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 3 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 2 >> >> >> miibus0: mii_mediachg: can't handle non-zero PHY instance 1 >> >> >> >> >> >> Just as before. In case it should make any difference; I'm >> >> >> going to attach my copy of src/sys/dev/nfe/if_nfe.c in case >> >> >> there are any differences in mine, that do not coincide with >> >> >> your version/copy (I'm on releng_9 - 9.2-STABLE) >> >> >> >> >> >> FreeBSD demon0 9.2-STABLE FreeBSD 9.2-STABLE #0 r263756M: >> >> >> Thu Apr 3 12:42:03 PDT 2014 >> >> >> root@demon0:/usr/obj/usr/src/sys/DEMON0 amd64 >> >> >> >> >> >> Best wishes, and thanks again. >> >> >> >> >> > >> >> > Oops, could you show me the output of "pciconf -lcbv"? >> >> >> >> Yes. Of course. >> >> >> > >> > [...] >> > >> >> nfe0@pci0:0:7:0: class=0x068000 card=0x73091462 chip=0x03ef10de rev=0xa2 hdr=0x00 >> >> vendor = 'nVidia Corporation' >> >> device = 'MCP61 Ethernet' >> >> class = bridge >> >> bar [10] = type Memory, range 32, base 0xdff7d000, size 4096, enabled >> >> bar [14] = type I/O Port, range 32, base 0xe480, size 8, enabled >> >> cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 >> >> cap 05[50] = MSI supports 8 messages, 64 bit, vector masks enabled with 8 messages >> >> cap 08[6c] = HT MSI fixed address window enabled at 0xfee00000 >> > >> > Thanks a lot for the info. It seems I missed there are 4 variants >> > for MCP61. Try attached patch again. >> >> Greetings, and thank you for your continued efforts. >> I just applied the patch, and built/installed a kernel with it. >> The results are in: >> >> /boot/loader.conf: >> loader_logo="beastiebw" >> #hint.miibus.0.phymask="0x00000001" >> >> relevant output from dmesg(8): >> nfe0: port 0xe480-0xe487 mem >> 0xdff7d000-0xdff7dfff >> irq 20 at device 7.0 on pci0 >> nfe0: attempting to allocate 8 MSI vectors (8 supported) >> msi: routing MSI IRQ 257 to local APIC 0 vector 56 >> msi: routing MSI IRQ 258 to local APIC 0 vector 57 >> msi: routing MSI IRQ 259 to local APIC 0 vector 58 >> msi: routing MSI IRQ 260 to local APIC 0 vector 59 >> msi: routing MSI IRQ 261 to local APIC 0 vector 60 >> msi: routing MSI IRQ 262 to local APIC 0 vector 61 >> msi: routing MSI IRQ 263 to local APIC 0 vector 62 >> msi: routing MSI IRQ 264 to local APIC 0 vector 63 >> nfe0: using IRQs 257-264 for MSI >> nfe0: Using 8 MSI messages >> miibus0: on nfe0 >> rlphy0: PHY 0 on miibus0 >> rlphy0: OUI 0x000004, model 0x0020, rev. 1 >> rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow >> nfe0: bpf attached >> nfe0: Ethernet address: 40:61:86:cd:44:97 >> ... >> vlan: initialized, using hash tables with chaining >> ... >> lo0: bpf attached >> ... >> >> Hmmm. I think we have a winner. No? > > Thanks a lot for testing. Thank YOU for working on this! > I'll commit it within a couple of days. Good news. > >> The only thing that I noticed that appeared to be different. >> Is that it takes some 5 seconds to get from: >> >> nfe0: link state changed to DOWN >> nfe0: link state changed to UP >> >> to: >> >> Starting Network: ... >> >> output. I don't know that your patch had anything to do with > > I don't think the patch may result in the change you've seen. > Auto-negotiation may take several seconds and that might be reason > you see the difference. Link up message does not necessarily mean > it resolved speed/duplex of link against link partner. nfe(4) > explicitly checks whether it established a valid link(i.e. link > should be up and speed/duplex also should be resolved). OK. I have 3 others using nfe(4), and they don't take as long. So thought it might be worth mentioning -- no finger pointing, just saying. :) I'll keep an eye on it. I'm guessing it's probably just the difference in version of chip, or something similar. Thanks again. --Chris > >> that. Or it's just another anomaly with the driver/NIC. But it >> was a long enough period/change, that I thought it worth >> mentioning. >> >> Thank you again, for all your time, and efforts. >> >> --Chris >> >> >> > >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 03:38:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6E455C5 for ; Wed, 9 Apr 2014 03:38:53 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7E2110D2 for ; Wed, 9 Apr 2014 03:38:53 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id r5so2108487qcx.4 for ; Tue, 08 Apr 2014 20:38:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dFwXnpG7aE+/rcf3ktmk0YXN95cDKbDUl1yCDCUS0iU=; b=r5EIOzjDbC5v0NsxX/n0ma2YsWjsIpgjYOkTYbHA7OYQ8gFc4w4TyInx8HmGgyeTKf rqwRfSZvY0XVIRaZKn6MrmcRGLdrFKla+hTT4y13k3HM369gscF2onAnChyN3THrBRsL 8YX/qL1/OZgDIaWE5jqo32gfTP+S9wQzybhuEAsz6rMoRKaDKdmY0NSvZiqo3vz5o4N9 ntc0354q5Pnge9W0rlkGO9BNmUx7Bw8FoyteouKveFw3nnRVFfAvWZKdlmmskWrQHmsS 27bBB73ViJXLe4rmug1LeyD+cMKIUzgyMRiYdoZZPJa7c848hDY74mbRCXqKHD9+zwoO WSUg== MIME-Version: 1.0 X-Received: by 10.140.93.22 with SMTP id c22mr8885740qge.53.1397014732806; Tue, 08 Apr 2014 20:38:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Tue, 8 Apr 2014 20:38:52 -0700 (PDT) In-Reply-To: <5343D368.6020308@verisign.com> References: <53426EE6.5020409@verisign.com> <5343D368.6020308@verisign.com> Date: Tue, 8 Apr 2014 20:38:52 -0700 X-Google-Sender-Auth: CFVUjt0JJHtLGvh2glXESoRcs-M Message-ID: Subject: Re: panic in -HEAD multicast code From: Adrian Chadd To: Julien Charbon Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 03:38:54 -0000 On 8 April 2014 03:46, Julien Charbon wrote: > > Hi Adrian, > > > On 08/04/14 10:25, Adrian Chadd wrote: >> >> Hm, how's this happening here? I'm not detaching the interface. > > > Hm, if your are positive that nothing is detaching the interface on your > behalf (like a /etc/rc.d/netif restart somewhere), then it looks like you > got an unrelated case that just drives the same stacktrace than the race > condition we found. Hm, interesting. I wonder who i have to beat over the head about this. -a From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 03:46:40 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F51AA39 for ; Wed, 9 Apr 2014 03:46:40 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DC1F119C for ; Wed, 9 Apr 2014 03:46:40 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id q108so1738562qgd.37 for ; Tue, 08 Apr 2014 20:46:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=xcs4rKh/dJdQPNDX83MySWUaNlGWHGJnUJqOni1fxZk=; b=fpZGduVGvoUHLyYDSAzsratc2Uw5GurF2aBHemzMXl0tamNRbD2s9j1hIIfXVyPuoS jPhtVzd5hwDyjcpqx+0gsr+dl3TtUhyxTWHaMGZUZVbLE4XqqXfPIkt6f+ecVMRpWY2v 9xSyBtroJYeCd35my2tFDhuAGhN2cCUYFwIArgqJLqD+mi+FKxpzWQjVvJTIxOlpAk3A QP9RKc4rxsqQqKA8Bk8AycEyLh8rL2zK+/yEHdnjScA0vHZi3c6NIOGQ2QGE2JIBTtIZ J3U2IueZGcQAd1FdvZqvWrVuL7Ncnq26XmH941hPX9ETbNP2/8DL/9BDW7kqX+MwaDiI 7olg== MIME-Version: 1.0 X-Received: by 10.224.115.68 with SMTP id h4mr9292761qaq.35.1397015199440; Tue, 08 Apr 2014 20:46:39 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Tue, 8 Apr 2014 20:46:39 -0700 (PDT) In-Reply-To: <533BF902.7010506@sfc.wide.ad.jp> References: <533BF902.7010506@sfc.wide.ad.jp> Date: Tue, 8 Apr 2014 20:46:39 -0700 X-Google-Sender-Auth: fj4WGskNIIIZr6SABAdl4sDrEiY Message-ID: Subject: Re: ECN marking implenetation for dummynet From: Adrian Chadd To: Midori Kato Content-Type: text/plain; charset=ISO-8859-1 Cc: "Eggert, Lars" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 03:46:40 -0000 Hi! Cool! can you file a FreeBSD PR with this? -a On 2 April 2014 04:48, Midori Kato wrote: > Hi FreeBSD developers, > > I'm Midori Kato. I was working with Lars Eggert about DCTCP. > I would like to share our patch for an ECN marking mechanism on > dummynet, which I used for DCTCP testing. > > My implementation allows to set ECN with RED as an AQM scheme. The > following command is an example: > $ ipfw pipe 9999 config red 1/10/10/0.0 ecn > > Our implementation includes both DCTCP and RFC 3168 ECN marking methodology. > > If you are interested in our ECN implemention, I'm very happy to receive > your review! (I have already submitted my patch to Luigi and hope he > will merge ours in near future.) > > Regards, > -- Midori > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 04:14:55 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 108A325A; Wed, 9 Apr 2014 04:14:55 +0000 (UTC) Received: from mail-ee0-x22a.google.com (mail-ee0-x22a.google.com [IPv6:2a00:1450:4013:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79EF913D8; Wed, 9 Apr 2014 04:14:54 +0000 (UTC) Received: by mail-ee0-f42.google.com with SMTP id d17so1366980eek.1 for ; Tue, 08 Apr 2014 21:14:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=VAog8C3GSgIxWBRxaXxh8VxZiYImp4yuTV02Pss9mAs=; b=vVgEl0YSVmG8nFF7+Bm2cF/bSe9pNjgLRtlboEMwFWX0c+ZNO7bgXJww9UvgiAwwpZ BZItc9bFElkQNuqrJeHwADNtzueXVz194ygqhKL7tbV4PwioHoy+60Hg0jfpza9wmb5a ppYmatLpRUmJfCi4Se7CA6923hR9GSxHMUNapLkAjgs/T24hNe0kr+nUjJfkziMVD1bV Yb2snsuPaNZiKF0CZJg4Kv/J265fO6H0sZ+hgrG4589B2qWCVxpWIn5oRfeh8GSFskdC BAhxlSgCg3goDVToR7MK3DXfck94ZKCgLLszF/REEqxSVBi8VU6k100n51N07mCv1n/w RqYA== MIME-Version: 1.0 X-Received: by 10.15.53.69 with SMTP id q45mr9335386eew.22.1397016892729; Tue, 08 Apr 2014 21:14:52 -0700 (PDT) Received: by 10.14.213.194 with HTTP; Tue, 8 Apr 2014 21:14:52 -0700 (PDT) In-Reply-To: References: <533BF902.7010506@sfc.wide.ad.jp> Date: Tue, 8 Apr 2014 21:14:52 -0700 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: hiren panchasara To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: Midori Kato , "Eggert, Lars" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 04:14:55 -0000 On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd wrote: > Hi! Cool! can you file a FreeBSD PR with this? I'm testing this patch right now. I will make sure it doesn't get lost. :-) cheers, Hiren > > > -a > > > On 2 April 2014 04:48, Midori Kato wrote: >> Hi FreeBSD developers, >> >> I'm Midori Kato. I was working with Lars Eggert about DCTCP. >> I would like to share our patch for an ECN marking mechanism on >> dummynet, which I used for DCTCP testing. >> >> My implementation allows to set ECN with RED as an AQM scheme. The >> following command is an example: >> $ ipfw pipe 9999 config red 1/10/10/0.0 ecn >> >> Our implementation includes both DCTCP and RFC 3168 ECN marking methodology. >> >> If you are interested in our ECN implemention, I'm very happy to receive >> your review! (I have already submitted my patch to Luigi and hope he >> will merge ours in near future.) >> >> Regards, >> -- Midori >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 05:58:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F3C90C2A; Wed, 9 Apr 2014 05:58:10 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C329A1C04; Wed, 9 Apr 2014 05:58:10 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id lj1so2046179pab.6 for ; Tue, 08 Apr 2014 22:58:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ADcNyEJYt+otHdvWwAgEc1YRQkTCoe9b5j6fYtU0YrE=; b=WnCGFWWb/V9PuSd7gEn6EhvkHTsLQzvsTGOLmAmChekyZmm8q/Gl4rLtaufKALheOb 9FRuzTJDYOhhdfLtFNEuVH3+WQ657Tr6WORF++DZGOdYcf+ALZhrnvhBLuw3kR8BJTSH JLPdoG3FE/FvL9S3nsqnpEyb5Qtisbu9m961wkTisyTDn3olHyKvB6xFWAAUHrqoXuYA 0yQdi+gyeEuxNOwXakRLJPYOAxNgCS1Rz/7DOQpWcqJ2H6tDoyxhZr1K6vCiFZLAwTM7 yKjIlfpJ5Makk5QIWsBZTDTss0Vl/kmMKRdsMoYEh507nwtmbmCWXUBCc1A5qcy0qFwi 0ZQw== X-Received: by 10.68.202.194 with SMTP id kk2mr9641257pbc.156.1397023090335; Tue, 08 Apr 2014 22:58:10 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id kl1sm8726699pbd.73.2014.04.08.22.58.07 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 08 Apr 2014 22:58:09 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 09 Apr 2014 14:58:05 +0900 From: Yonghyeon PYUN Date: Wed, 9 Apr 2014 14:58:05 +0900 To: Andriy Gapon Subject: Re: re(4) causes memory corruption? Message-ID: <20140409055805.GA1398@michelle.cdnetworks.com> References: <5343B178.8070604@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5343B178.8070604@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 05:58:11 -0000 On Tue, Apr 08, 2014 at 11:21:12AM +0300, Andriy Gapon wrote: > > I have this network card (it's actually integrated into a motherboard): > > re0: port > 0xde00-0xdeff mem 0xfdaff000-0xfdafffff,0xfdae0000-0xfdaeffff irq 18 at device > 0.0 on pci2 > re0: Using 1 MSI-X message > re0: Chip rev. 0x3c000000 > re0: MAC rev. 0x00400000 > miibus0: on re0 > rgephy0: PHY 1 on miibus0 > rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, > 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, > 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow > > When there is little traffic through the interface I do not observe any problems > with it. > But within 15 seconds of applying some moderate traffic I would always observe a > heavy screen corruption often followed by a total freeze or a hardware self-reset. > An example of the moderate traffic is 6 MBytes/s which results in about 10K > interrupts per seconds. > PCIe re(4) controllers do not seem to have intelligent interrupt moderation feature. At least it's not documented at all. To overcome the H/W limitation, re(4) uses one-shot timer interrupt to mitigate interrupt processing overhead. However the maximum time allowed to set for one-shot timer is less than or equal to 65us so you may still see lots of interrupts under heavy load. > I am not sure what causes the problem. Could it be some driver using memoery > that it should not or hardware writing where it should not or if this something > completely in the hardware. > I will appreciate any hints on possible ways to analyze this issue. It seems your controller is old RTL8168C and I'm not aware of any memory corruption issues with the RTL8168C. There were a couple of re(4) instability reports but they were using relatively recent re(4) controllers and none of them showed memory corruption. > > Thanks! > -- > Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 19:16:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5F4615D for ; Wed, 9 Apr 2014 19:16:43 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 941B3134E for ; Wed, 9 Apr 2014 19:16:43 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id y20so2921334ier.13 for ; Wed, 09 Apr 2014 12:16:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=AasCX+VtaaxIZnebvRJR+HeS9PXIrGk7nwnFT/1Lgak=; b=PxyqdtCUmA0lVDigCtCGyEyCPKbOEwf/XBua3uhv6jDOhb1QKtz1zaw+j5y+1cnpJG +Ri8e7F48psuB45fmj9DdrwmzkgcAwq/nenfr3OC1kfcMUix3o3yVZGEjrUvrtiXqx0k bMEq+cINv02Ss0PnRoOsiIfKRiYLcNJbYjmjx7Sg56aXWfSJtC8TfahH+IxJv0Zoi2cj P+wuN2XGL3P/WPheDPwTMqZK0wxafFUsOFpXy3z8VrzR2qBL12GFDlWihrOIVydSiosU qWnizl63mI/zEAWGifJgp9GmV0NtAGoT/x5YO+kERZf3J41cLplfnmBrtT8isPCLBH6O +iWw== X-Received: by 10.50.36.66 with SMTP id o2mr11906269igj.24.1397071002890; Wed, 09 Apr 2014 12:16:42 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id vu3sm7201462igc.6.2014.04.09.12.16.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Apr 2014 12:16:42 -0700 (PDT) Message-ID: <53459C96.5040304@gmail.com> Date: Wed, 09 Apr 2014 15:16:38 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@FreeBSD.org Subject: Preventing ng_callout() timeouts to trigger packet queuing Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 19:16:43 -0000 Hi List, I'm calling out to the general wisdom ... I have seen an issue in netgraph where, if called, a callout routine registered by ng_callout() will trigger packet queuing inside the worklist of netgraph since ng_callout() makes my node suddenly a WRITER node (therefore non reentrant) for the duration of the call. So as soon as the callout function returns, all following packets will get directly passed to the node again and when the ngintr thread gets executed then only then will I get the queued packets. This introduces out of order packets in the flow. I am using the current patch below to solve the issue and I am wondering if there is anything wrong with it (and maybe contribute back :): @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p hook, int ticks, if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) return (ENOMEM); - item->el_flags |= NGQF_WRITER; + item->el_flags = NGQF_READER; NG_NODE_REF(node); /* and one for the item */ NGI_SET_NODE(item, node); if (hook) { Best regards, Karim. From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 20:14:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A972B9F0 for ; Wed, 9 Apr 2014 20:14:22 +0000 (UTC) Received: from mx1.rpsol.net (mx1.rpsol.net [74.206.97.74]) by mx1.freebsd.org (Postfix) with ESMTP id 8E0C019B6 for ; Wed, 9 Apr 2014 20:14:22 +0000 (UTC) Received: from [172.16.1.100] (wsip-72-215-202-18.ph.ph.cox.net [72.215.202.18]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.rpsol.net (Postfix) with ESMTPSA id 53A87FFE309 for ; Wed, 9 Apr 2014 13:14:06 -0700 (MST) Message-ID: <5345AA7C.3050700@soliddataservices.com> Date: Wed, 09 Apr 2014 13:15:56 -0700 From: Matt Lager User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Racoon/IPSEC Tunnel in 9.2 vs 10.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-RPS-MailScanner-Information: Please contact the ISP for more information X-RPS-MailScanner-ID: 53A87FFE309.AADF8 X-RPS-MailScanner: Found to be clean X-RPS-MailScanner-From: matt@soliddataservices.com X-Spam-Status: No X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 20:14:22 -0000 I have used IPSEC tunnels w/ racoon to establish point to point VPN connections for a long time, with great success. I recently decided to upgrade one of my endpoints to 10.0-RELEASE from 9.2-RELEASE-p3. I didn't do an upgrade but did a fresh installation of 10.0-RELEASE, but applied the identical VPN configuration that was working in 9.2-RELEASE-p3. The tunnels came up fine, and setkey -D shows that keys had been generated, connectivity appeared to be working at first glance. I then started to work as normal through my VPN with things like RDP, SQL Server, and other protocols, where I found that connectivity started then came to a dead halt (not ICMP, which always works fine). I did another fresh install of 9.2-RELEASE-p3, applied the config, and everything worked as expected. I've read a lot about MTU's and fragmented traffic, but I'm trying to figure out where I should be looking to fix things up. Something obviously changed. I do use PF, and I know PF underwent some big changes, so maybe it's a PF problem, but I thought I'd post here first. I'm using the same PF config on the 10.0 system as I did on the 9.2, of course making sure interfaces were all named properly and whatnot. Any advice would be appreciated. Thanks! Matt -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 21:14:37 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A32562C for ; Wed, 9 Apr 2014 21:14:37 +0000 (UTC) Received: from frv191.fwdcdn.com (frv191.fwdcdn.com [212.42.77.191]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3532B114F for ; Wed, 9 Apr 2014 21:14:36 +0000 (UTC) Received: from [10.10.1.30] (helo=frv196.fwdcdn.com) by frv191.fwdcdn.com with esmtp ID 1WXzpE-000JMu-1M for net@freebsd.org; Thu, 10 Apr 2014 00:14:28 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:Cc:To:Subject:From:Date; bh=jupy+tg9D/Jim0E+jNBnR/vdasU7Y2pUFqWwqwTMwIA=; b=tVtoGNSYa2camIT1t5S9ojhVodMOmxpl+NILOYNUcu3ShlBhRlmirZEXH9MaEO2xilwnrS2Xhw5KlWuXtOyd0EIItIeBMnAsxBK3SSkGotgxD4FPYxX51BRyTqQ0sTglu4UwYedApa68N+y2SZE89xmWdRdPGZbbAUhQGURvoUQ=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv196.fwdcdn.com with smtp ID 1WXzow-000Grv-Ci for net@freebsd.org; Thu, 10 Apr 2014 00:14:10 +0300 Date: Thu, 10 Apr 2014 00:14:10 +0300 From: Vladislav Prodan Subject: Some gruesome moments with performance of FreeBSD at over 20K interfaces To: stable@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Thu, 10 Apr 2014 00:14:10 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: hackers@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 21:14:37 -0000 Dear Colleagues! I had a task, using FreeBSD 10.0-STABLE: 1) Receive 20-30 Q-in-Q VLAN (IEEE 802.1ad ), inside of which 2k-4k vlan (IEEE 802.1Q). Total ~60K vlan 2) To every vlan interface assign ipv4 and ipv6 addresses, define routes to ipv4 and ipv6 addresses on another side of vlan (ip unnumbered), and also prescribe ipv6 network /64 by size through ipv6 address on another side of vlan. 3) Perform routing from the world to all of these ipv4/ipv6 addresses Ðļ ipv6 networks inside ~60K vlan To accomplish the 1st task I have no alternatives to using Netgraph. I noticed incorrect behavior of ngctl(8) after addition of 560th vlan (bin/187835) Than speed of addition 4k, 8k, 12k vlans was damnably slow: 10 minutes for first 4k vlans 18 minutes for first 5k vlans 28 minutes for first 6k vlans 52 minutes for first 8k vlans Than I added more 4К vlans 20 minutes - 9500 vlans 33 minutes - 10500 vlans 58 minutes - 12К vlans In total speed of addition of 4k, 8k, 12k vlans was subsequently 10m/52m/110m It’s hard to imagine, how many time is needed to add ~60K vlan :( Process was accelerated a little by shooting off devd, bsnmpd, ntpd services, but it found another problems and limitations. For example, a) Service ntpd refuse to start at 12K interfaces: ntpd[2195]: Too many sockets in use, FD_SETSIZE 16384 exceeded I remind, that in files /usr/src/sys/sys/select.h and /usr/include/sys/select.h FD_SETSIZE value is only 1024U b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU at 80-100% last pid: 64011; load averages: 1.00, 0.97, 0.90 up 0+05:25:39 21:26:36 58 processes: 3 running, 54 sleeping, 1 waiting CPU: 68.2% user, 0.0% nice, 30.6% system, 1.2% interrupt, 0.0% idle Mem: 125M Active, 66M Inact, 435M Wired, 200K Cache, 525M Free ARC: 66M Total, 28M MFU, 36M MRU, 16K Anon, 614K Header, 2035K Other Swap: 1024M Total, 1024M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 63863 root 1 96 0 136M 119M RUN 35:31 79.98% bsnmpd ... c) Size of fields during output of command netstat(1) - netstat -inW is unsufficient (bin/188153) d) If indicate in command netstat of interface it’s impossible to understand, which ipv4/ipv6 neworks are indicated here. # netstat -I ngeth123.223 -nW Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ngeth12 1500 08:00:27:cd:9b:8e 0 0 0 1 5 0 ngeth12 - 172.18.206.13 172.18.206.139 0 - - 0 - - ngeth12 - fe80::a00:27f fe80::a00:27ff:fe 0 - - 1 - - ngeth12 - 2001:570:28:1 2001:570:28:140:: 0 - - 0 - - e) Very low output of command arp: # ngctl list | grep ngeth | wc -l 12003 # ifconfig -a | egrep -e 'inet ' | wc -l 12007 # time /usr/sbin/arp -na > /dev/null 150.661u 551.002s 11:53.71 98.3% 20+172k 1+0io 0pf+0w More info at http://freebsd.1045724.n5.nabble.com/arp-8-performance-use-if-nameindex-instead-of-if-indextoname-td5898205.html After using of patch, speed became acceptable: # time /usr/sbin/arp -na > /dev/null 0.114u 0.090s 0:00.14 142.8% 20+170k 0+0io 0pf+0w I suspect, that output of standard network stack will be too low to accomplish a 3rd task, routing of ~60K vlan I have no idea, how to use netmap(4) in this situation :( Please, help me in fulfillment of assigned task. P.S. Colleague-Linuxoid is adjusting the same task and bragging: At Debian, in test (kernel 3.13), 80K vlans arose in 20 minutes. It takes 3 GB RAM. And deleting of these vlans also took 20 minutes. -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 21:26:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA010FE9 for ; Wed, 9 Apr 2014 21:26:05 +0000 (UTC) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97E4512B8 for ; Wed, 9 Apr 2014 21:26:05 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id rl12so3092599iec.32 for ; Wed, 09 Apr 2014 14:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=2qHdAKTg+HrLeLP33KMMWVN5D/XFVnU6NVEuQAo1x/c=; b=Q+Gj69JD5i3yMoTfdBmkZrPaV+mQzedh7n8DB+0zHOh26dg71LmqnAuln3ocnFBvM0 Nu7/pGd9GizBqsSgQpXSVmI918GuqyVG3WrDUmmCm2nlveDaRe0llfRUgJ/y/VnMb3Eo ZX10mW+pBarH2red0sLNCFUrYD2vTZbiEclOZ/iXcxIc3zEeNsTgMrzBkdmSDisIUB8d dGYgcMYJ3vToyWxuVZfLRRjr9sxBfF504yLBJwGAiDSkbBtXX6kTCrGUEMHjvBn6V2ha IqTRDuQ6Ae8Kv/iSWoqCI935Bst3V6cnQOr52VOoedk4PsNq3TU630Wi+5sCZXx6xeQX h3uQ== X-Received: by 10.42.13.210 with SMTP id e18mr10033352ica.10.1397078764974; Wed, 09 Apr 2014 14:26:04 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id i16sm14196927igf.11.2014.04.09.14.26.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Apr 2014 14:26:04 -0700 (PDT) Message-ID: <5345BAE7.4010501@gmail.com> Date: Wed, 09 Apr 2014 17:25:59 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> In-Reply-To: <53459C96.5040304@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 21:26:05 -0000 Hi, Below is a revised patch for this issue. It accounts for nodes or hooks that explicitly need to be queuing: @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p hook, int ticks, if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) return (ENOMEM); - item->el_flags |= NGQF_WRITER; + if ((node->nd_flags & NGF_FORCE_WRITER) || + (hook && (hook->hk_flags & HK_FORCE_WRITER))) + item->el_flags |= NGQF_WRITER; + else + item->el_flags |= NGQF_READER; + NG_NODE_REF(node); /* and one for the item */ NGI_SET_NODE(item, node); if (hook) { Regards, Karim. On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: > Hi List, > > I'm calling out to the general wisdom ... I have seen an issue in > netgraph where, if called, a callout routine registered by > ng_callout() will trigger packet queuing inside the worklist of > netgraph since ng_callout() makes my node suddenly a WRITER node > (therefore non reentrant) for the duration of the call. > > So as soon as the callout function returns, all following packets will > get directly passed to the node again and when the ngintr thread gets > executed then only then will I get the queued packets. This introduces > out of order packets in the flow. I am using the current patch below > to solve the issue and I am wondering if there is anything wrong with > it (and maybe contribute back :): > > > @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, > hook_p hook, int ticks, > if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) > return (ENOMEM); > > - item->el_flags |= NGQF_WRITER; > + item->el_flags = NGQF_READER; > NG_NODE_REF(node); /* and one for the item */ > NGI_SET_NODE(item, node); > if (hook) { > > > Best regards, > > Karim. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 21:29:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D96D6338 for ; Wed, 9 Apr 2014 21:29:28 +0000 (UTC) Received: from frv196.fwdcdn.com (frv196.fwdcdn.com [212.42.77.196]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 965FC12E7 for ; Wed, 9 Apr 2014 21:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:Cc:To:Subject:From:Date; bh=JZf2Z/93GDeyUrJl9tcNCQ/IJP0oZ42R+f8jqFX1Zus=; b=eyWoFnJmeZgBQHUWfdw8Ie/o4bXWs2q+B/TdBLaBmAvlVS58XA1wxQo/3Ax+U6NoqOOJ7LZI4NGHZ8AzhRnI2c7d76JaNkrl8vnvvjYypFcxN/oj49Y+mO8MSOoCw1g4CNETKHCASjyVhmuKYxRBMa35ERsZyGuEEfBBSLs+NMw=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv196.fwdcdn.com with smtp ID 1WY03i-0007CT-Jq for freebsd-net@freebsd.org; Thu, 10 Apr 2014 00:29:26 +0300 Date: Thu, 10 Apr 2014 00:29:26 +0300 From: Vladislav Prodan Subject: Some gruesome moments with performance of FreeBSD at over 20K interfaces To: freebsd-stable@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1397078965.914029340.wn98okr9@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Thu, 10 Apr 2014 00:29:26 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 21:29:29 -0000 Dear Colleagues! I had a task, using FreeBSD 10.0-STABLE: 1) Receive 20-30 Q-in-Q VLAN (IEEE 802.1ad ), inside of which 2k-4k vlan (IEEE 802.1Q). Total ~60K vlan 2) To every vlan interface assign ipv4 and ipv6 addresses, define routes to ipv4 and ipv6 addresses on another side of vlan (ip unnumbered), and also prescribe ipv6 network /64 by size through ipv6 address on another side of vlan. 3) Perform routing from the world to all of these ipv4/ipv6 addresses Ðļ ipv6 networks inside ~60K vlan To accomplish the 1st task I have no alternatives to using Netgraph. I noticed incorrect behavior of ngctl(8) after addition of 560th vlan (bin/187835) Than speed of addition 4k, 8k, 12k vlans was damnably slow: 10 minutes for first 4k vlans 18 minutes for first 5k vlans 28 minutes for first 6k vlans 52 minutes for first 8k vlans Than I added more 4К vlans 20 minutes - 9500 vlans 33 minutes - 10500 vlans 58 minutes - 12К vlans In total speed of addition of 4k, 8k, 12k vlans was subsequently 10m/52m/110m It’s hard to imagine, how many time is needed to add ~60K vlan :( Process was accelerated a little by shooting off devd, bsnmpd, ntpd services, but it found another problems and limitations. For example, a) Service ntpd refuse to start at 12K interfaces: ntpd[2195]: Too many sockets in use, FD_SETSIZE 16384 exceeded I remind, that in files /usr/src/sys/sys/select.h and /usr/include/sys/select.h FD_SETSIZE value is only 1024U b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU at 80-100% last pid: 64011; load averages: 1.00, 0.97, 0.90 up 0+05:25:39 21:26:36 58 processes: 3 running, 54 sleeping, 1 waiting CPU: 68.2% user, 0.0% nice, 30.6% system, 1.2% interrupt, 0.0% idle Mem: 125M Active, 66M Inact, 435M Wired, 200K Cache, 525M Free ARC: 66M Total, 28M MFU, 36M MRU, 16K Anon, 614K Header, 2035K Other Swap: 1024M Total, 1024M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 63863 root 1 96 0 136M 119M RUN 35:31 79.98% bsnmpd ... c) Size of fields during output of command netstat(1) - netstat -inW is unsufficient (bin/188153) d) If indicate in command netstat of interface it’s impossible to understand, which ipv4/ipv6 neworks are indicated here. # netstat -I ngeth123.223 -nW Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ngeth12 1500 08:00:27:cd:9b:8e 0 0 0 1 5 0 ngeth12 - 172.18.206.13 172.18.206.139 0 - - 0 - - ngeth12 - fe80::a00:27f fe80::a00:27ff:fe 0 - - 1 - - ngeth12 - 2001:570:28:1 2001:570:28:140:: 0 - - 0 - - e) Very low output of command arp: # ngctl list | grep ngeth | wc -l 12003 # ifconfig -a | egrep -e 'inet ' | wc -l 12007 # time /usr/sbin/arp -na > /dev/null 150.661u 551.002s 11:53.71 98.3% 20+172k 1+0io 0pf+0w More info at http://freebsd.1045724.n5.nabble.com/arp-8-performance-use-if-nameindex-instead-of-if-indextoname-td5898205.html After using of patch, speed became acceptable: # time /usr/sbin/arp -na > /dev/null 0.114u 0.090s 0:00.14 142.8% 20+170k 0+0io 0pf+0w I suspect, that output of standard network stack will be too low to accomplish a 3rd task, routing of ~60K vlan I have no idea, how to use netmap(4) in this situation :( Please, help me in fulfillment of assigned task. P.S. Colleague-Linuxoid is adjusting the same task and bragging: At Debian, in test (kernel 3.13), 80K vlans arose in 20 minutes. It takes 3 GB RAM. And deleting of these vlans also took 20 minutes. -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 01:09:19 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE25D952; Thu, 10 Apr 2014 01:09:19 +0000 (UTC) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B88B1A71; Thu, 10 Apr 2014 01:09:19 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id m20so3641293qcx.38 for ; Wed, 09 Apr 2014 18:09:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=264EbCMXLHTmbq+aW9cndHLnV5e9ZHW389ZjxR05FuU=; b=dErqPCaQBQCSJFnT5mv2UZptgdxBTXFbtG+SuiyB3DlouolyEokw33WtU92DIghv9C rtw+hfe6MFxcO+53eb/zBiM8Ui+hdNpkPqXD+lW6Ps1T63XUWMUpUJuYNXKeR6vdcZWc V0hF3nU3Gj9CDZsQMjTgwwy1ZMvTg4kK6fRMXf3Ox/PLVgzuFxnNYB3f7mQtWUEeg4vT NsLxNjNktvPu+xI3SoPMaYX9aGSEERTOTOHg6lfQSWSMM84N/yIM27GjQ90seJ64DBe4 lq7ZWSukPtird9e+pywSV09EijZ+MYbJoZzE4LIwxDA2rO4MAlJH4xzJi0iHy45lw5Ib DvMQ== MIME-Version: 1.0 X-Received: by 10.140.42.38 with SMTP id b35mr15773695qga.87.1397092158477; Wed, 09 Apr 2014 18:09:18 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Wed, 9 Apr 2014 18:09:18 -0700 (PDT) In-Reply-To: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> Date: Wed, 9 Apr 2014 18:09:18 -0700 X-Google-Sender-Auth: hJIhuCiqDd6ywEO6kT0dNkZDL8s Message-ID: Subject: Re: Some gruesome moments with performance of FreeBSD at over 20K interfaces From: Adrian Chadd To: Vladislav Prodan Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "stable@freebsd.org" , "freebsd-hackers@freebsd.org" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 01:09:19 -0000 Hi, There's likely many more places where these aren't O(1) operations. The patch in question should be in -HEAD now. a-a On 9 April 2014 14:14, Vladislav Prodan wrote: > Dear Colleagues! > > I had a task, using FreeBSD 10.0-STABLE: > 1) Receive 20-30 Q-in-Q VLAN (IEEE 802.1ad ), inside of which 2k-4k vlan = (IEEE 802.1Q). Total ~60K vlan > 2) To every vlan interface assign ipv4 and ipv6 addresses, define routes = to ipv4 and ipv6 addresses on another side of vlan (ip unnumbered), and als= o prescribe ipv6 network /64 by size through ipv6 address on another side o= f vlan. > 3) Perform routing from the world to all of these ipv4/ipv6 addresses =D0= =B8 ipv6 networks inside ~60K vlan > > > > To accomplish the 1st task I have no alternatives to using Netgraph. > I noticed incorrect behavior of ngctl(8) after addition of 560th vlan (bi= n/187835) > Than speed of addition 4k, 8k, 12k vlans was damnably slow: > 10 minutes for first 4k vlans > 18 minutes for first 5k vlans > 28 minutes for first 6k vlans > 52 minutes for first 8k vlans > Than I added more 4=D0=BA vlans > 20 minutes - 9500 vlans > 33 minutes - 10500 vlans > 58 minutes - 12=D0=BA vlans > > In total speed of addition of 4k, 8k, 12k vlans was subsequently 10m/52m/= 110m > It=E2=80=99s hard to imagine, how many time is needed to add ~60K vlan :( > Process was accelerated a little by shooting off devd, bsnmpd, ntpd servi= ces, but it found another problems and limitations. > > For example, > a) Service ntpd refuse to start at 12K interfaces: > ntpd[2195]: Too many sockets in use, FD_SETSIZE 16384 exceeded > I remind, that in files /usr/src/sys/sys/select.h and /usr/include/sys/se= lect.h FD_SETSIZE value is only 1024U > > b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU a= t 80-100% > > last pid: 64011; load averages: 1.00, 0.97, 0.90 = up 0+05:25:39 21:26:36 > 58 processes: 3 running, 54 sleeping, 1 waiting > CPU: 68.2% user, 0.0% nice, 30.6% system, 1.2% interrupt, 0.0% idle > Mem: 125M Active, 66M Inact, 435M Wired, 200K Cache, 525M Free > ARC: 66M Total, 28M MFU, 36M MRU, 16K Anon, 614K Header, 2035K Other > Swap: 1024M Total, 1024M Free > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAN= D > 63863 root 1 96 0 136M 119M RUN 35:31 79.98% bsnmpd > ... > > c) Size of fields during output of command netstat(1) - netstat -inW is u= nsufficient (bin/188153) > > d) If indicate in command netstat of interface it=E2=80=99s impossible to= understand, which ipv4/ipv6 neworks are indicated here. > > # netstat -I ngeth123.223 -nW > Name Mtu Network Address Ipkts Ierrs Idrop Opk= ts Oerrs Coll > ngeth12 1500 08:00:27:cd:9b:8e 0 0 0 = 1 5 0 > ngeth12 - 172.18.206.13 172.18.206.139 0 - - = 0 - - > ngeth12 - fe80::a00:27f fe80::a00:27ff:fe 0 - - = 1 - - > ngeth12 - 2001:570:28:1 2001:570:28:140:: 0 - - = 0 - - > > e) Very low output of command arp: > # ngctl list | grep ngeth | wc -l > 12003 > # ifconfig -a | egrep -e 'inet ' | wc -l > 12007 > # time /usr/sbin/arp -na > /dev/null > 150.661u 551.002s 11:53.71 98.3% 20+172k 1+0io 0pf+0w > > > More info at http://freebsd.1045724.n5.nabble.com/arp-8-performance-use-i= f-nameindex-instead-of-if-indextoname-td5898205.html > > After using of patch, speed became acceptable: > > # time /usr/sbin/arp -na > /dev/null > 0.114u 0.090s 0:00.14 142.8% 20+170k 0+0io 0pf+0w > > I suspect, that output of standard network stack will be too low to accom= plish a 3rd task, routing of ~60K vlan > I have no idea, how to use netmap(4) in this situation :( > Please, help me in fulfillment of assigned task. > > P.S. > Colleague-Linuxoid is adjusting the same task and bragging: > At Debian, in test (kernel 3.13), 80K vlans arose in 20 minutes. It takes= 3 GB RAM. And deleting of these vlans also took 20 minutes. > > -- > Vladislav V. Prodan > System & Network Administrator > http://support.od.ua > +380 67 4584408, +380 99 4060508 > VVP88-RIPE > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 01:09:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9929EBA3 for ; Thu, 10 Apr 2014 01:09:54 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AC461A86 for ; Thu, 10 Apr 2014 01:09:54 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id j15so3273644qaq.30 for ; Wed, 09 Apr 2014 18:09:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=LLTdDil7vlMg8Xrxax9Qw2xk4MK+0Hb7rztVfqITtMc=; b=dIXXYanHW1Bw/2jwqoUJjEYqohqn+9DJdA67Necp1K4EyWFfQY5fYiXL30OR68ny1r s/YpngT8//nsjeH9URgFXfDC3mtc4TKAvLJAlR7y5tGGsM9toHeOf7ls7fZ7qw2XXeEM 50izx0CqtcCshE23QvXBsuYYZFhAAisidU47gIJ6kK5XE0mnoMTy+po1aD89Lm3om3hU 6GjJIXMDp+hrthaGda2ZdEkMJ5Qgv88B1I3Y7a51mpLoLPXHiE4BmzHXB0LNsVJalvfy Vnuq7tIfjPn6j+qlI1zycCACKtHHPmlGZ2QsvYln5Mo10/hB414+pVVHAtiUo0vJ7sLg Xyrw== MIME-Version: 1.0 X-Received: by 10.140.42.38 with SMTP id b35mr15775940qga.87.1397092193592; Wed, 09 Apr 2014 18:09:53 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Wed, 9 Apr 2014 18:09:53 -0700 (PDT) In-Reply-To: <5345BAE7.4010501@gmail.com> References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> Date: Wed, 9 Apr 2014 18:09:53 -0700 X-Google-Sender-Auth: d1zjfM23LM1kXrjhHVmb08MfYJ0 Message-ID: Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing From: Adrian Chadd To: Karim Fodil-Lemelin Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 01:09:54 -0000 Hi! Would you mind filing a PR for this? www.freebsd.org/send-pr.html That way it won't (hopefully!) get lost. Thanks! -a On 9 April 2014 14:25, Karim Fodil-Lemelin wrote: > Hi, > > Below is a revised patch for this issue. It accounts for nodes or hooks that > explicitly need to be queuing: > > @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p > hook, int ticks, > > if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) > return (ENOMEM); > > - item->el_flags |= NGQF_WRITER; > + if ((node->nd_flags & NGF_FORCE_WRITER) || > + (hook && (hook->hk_flags & HK_FORCE_WRITER))) > > + item->el_flags |= NGQF_WRITER; > + else > > + item->el_flags |= NGQF_READER; > + > NG_NODE_REF(node); /* and one for the item */ > NGI_SET_NODE(item, node); > if (hook) { > > Regards, > > Karim. > > > On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >> >> Hi List, >> >> I'm calling out to the general wisdom ... I have seen an issue in netgraph >> where, if called, a callout routine registered by ng_callout() will trigger >> packet queuing inside the worklist of netgraph since ng_callout() makes my >> node suddenly a WRITER node (therefore non reentrant) for the duration of >> the call. >> >> So as soon as the callout function returns, all following packets will get >> directly passed to the node again and when the ngintr thread gets executed >> then only then will I get the queued packets. This introduces out of order >> packets in the flow. I am using the current patch below to solve the issue >> and I am wondering if there is anything wrong with it (and maybe contribute >> back :): >> >> >> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >> hook, int ticks, >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + item->el_flags = NGQF_READER; >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> >> Best regards, >> >> Karim. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 07:17:19 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D49C3E7D; Thu, 10 Apr 2014 07:17:19 +0000 (UTC) Received: from mail-pb0-x232.google.com (mail-pb0-x232.google.com [IPv6:2607:f8b0:400e:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DA381C41; Thu, 10 Apr 2014 07:17:19 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id md12so3607373pbc.9 for ; Thu, 10 Apr 2014 00:17:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=KJMYIPwoZHluFWhKdWoXM+tp2I1coko7DRWkQ0cq4rw=; b=aCRzcV5e6S1Br4kfJ+8BZj2tGyXaKMx6M0q6iK2HWY7ULvVZpZL7PKBkor79hDfkxN QzQ4R7E7ZzqVar38AA65FBCJ++UhnZL4aqCtVf9rwWaD60Aq+tAsI3dx795PpirGS/db kbEpiS6Av/YjJFWNx3AbA/1Qb41Z104tMhykS0XrxxCAHpH2MLsK61hh4iNtaQIT8CW6 RStsug7om22oYKySQa8B7+vu0w0vNNvFnO0uNreSpqQwHa0uKhVISS91UljHh7MLQTnA tQJaQLLJ6oEbHvIFnLG0eSLVVdQCn/9kDXOAmDpP00n/73ZT8w61fsB641pWBJUOqC07 tCtw== MIME-Version: 1.0 X-Received: by 10.68.202.8 with SMTP id ke8mr17891380pbc.86.1397114239144; Thu, 10 Apr 2014 00:17:19 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.70.88.109 with HTTP; Thu, 10 Apr 2014 00:17:19 -0700 (PDT) In-Reply-To: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> Date: Thu, 10 Apr 2014 09:17:19 +0200 X-Google-Sender-Auth: nDCaiWbz6h330UUkNh-TgVjcCV8 Message-ID: Subject: Re: Some gruesome moments with performance of FreeBSD at over 20K interfaces From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Vladislav Prodan Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: stable@freebsd.org, hackers@freebsd.org, "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 07:17:19 -0000 >From experience with large number of interfaces and configuring them. Its not that the kernel cannot handle it the problem is that you call generic utilities to do this job. I.E. to setup an ip on the interface ifconfig has first to get the whole list of interfaces to determine if that interface exists and extra checkings. This is what slows down the whole thing. In pfSense by using custom utilities the time for configuring 8K interfaces went from around 30 minutes to mere seconds or about a minute. It has been long time not testing such scenarios and if you can generate a config(xml format) with all the information for pfSense i can give a look to see what is the bottleneck there. On Wed, Apr 9, 2014 at 11:14 PM, Vladislav Prodan wrote= : > Dear Colleagues! > > I had a task, using FreeBSD 10.0-STABLE: > 1) Receive 20-30 Q-in-Q VLAN (IEEE 802.1ad ), inside of which 2k-4k vlan > (IEEE 802.1Q). Total ~60K vlan > 2) To every vlan interface assign ipv4 and ipv6 addresses, define routes > to ipv4 and ipv6 addresses on another side of vlan (ip unnumbered), and > also prescribe ipv6 network /64 by size through ipv6 address on another > side of vlan. > 3) Perform routing from the world to all of these ipv4/ipv6 addresses =C9 > ipv6 networks inside ~60K vlan > > > > To accomplish the 1st task I have no alternatives to using Netgraph. > I noticed incorrect behavior of ngctl(8) after addition of 560th vlan > (bin/187835) > Than speed of addition 4k, 8k, 12k vlans was damnably slow: > 10 minutes for first 4k vlans > 18 minutes for first 5k vlans > 28 minutes for first 6k vlans > 52 minutes for first 8k vlans > Than I added more 4=CB vlans > 20 minutes - 9500 vlans > 33 minutes - 10500 vlans > 58 minutes - 12=CB vlans > > In total speed of addition of 4k, 8k, 12k vlans was subsequently > 10m/52m/110m > It's hard to imagine, how many time is needed to add ~60K vlan :( > Process was accelerated a little by shooting off devd, bsnmpd, ntpd > services, but it found another problems and limitations. > > For example, > a) Service ntpd refuse to start at 12K interfaces: > ntpd[2195]: Too many sockets in use, FD_SETSIZE 16384 exceeded > I remind, that in files /usr/src/sys/sys/select.h and > /usr/include/sys/select.h FD_SETSIZE value is only 1024U > > b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU a= t > 80-100% > > last pid: 64011; load averages: 1.00, 0.97, 0.90 > up 0+05:25:39 21:26:36 > 58 processes: 3 running, 54 sleeping, 1 waiting > CPU: 68.2% user, 0.0% nice, 30.6% system, 1.2% interrupt, 0.0% idle > Mem: 125M Active, 66M Inact, 435M Wired, 200K Cache, 525M Free > ARC: 66M Total, 28M MFU, 36M MRU, 16K Anon, 614K Header, 2035K Other > Swap: 1024M Total, 1024M Free > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAN= D > 63863 root 1 96 0 136M 119M RUN 35:31 79.98% bsnmpd > ... > > c) Size of fields during output of command netstat(1) - netstat -inW is > unsufficient (bin/188153) > > d) If indicate in command netstat of interface it's impossible to > understand, which ipv4/ipv6 neworks are indicated here. > > # netstat -I ngeth123.223 -nW > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll > ngeth12 1500 08:00:27:cd:9b:8e 0 0 0 > 1 5 0 > ngeth12 - 172.18.206.13 172.18.206.139 0 - - > 0 - - > ngeth12 - fe80::a00:27f fe80::a00:27ff:fe 0 - - > 1 - - > ngeth12 - 2001:570:28:1 2001:570:28:140:: 0 - - > 0 - - > > e) Very low output of command arp: > # ngctl list | grep ngeth | wc -l > 12003 > # ifconfig -a | egrep -e 'inet ' | wc -l > 12007 > # time /usr/sbin/arp -na > /dev/null > 150.661u 551.002s 11:53.71 98.3% 20+172k 1+0io 0pf+0w > > > More info at > http://freebsd.1045724.n5.nabble.com/arp-8-performance-use-if-nameindex-i= nstead-of-if-indextoname-td5898205.html > > After using of patch, speed became acceptable: > > # time /usr/sbin/arp -na > /dev/null > 0.114u 0.090s 0:00.14 142.8% 20+170k 0+0io 0pf+0w > > I suspect, that output of standard network stack will be too low to > accomplish a 3rd task, routing of ~60K vlan > I have no idea, how to use netmap(4) in this situation :( > Please, help me in fulfillment of assigned task. > > P.S. > Colleague-Linuxoid is adjusting the same task and bragging: > At Debian, in test (kernel 3.13), 80K vlans arose in 20 minutes. It takes > 3 GB RAM. And deleting of these vlans also took 20 minutes. > > -- > Vladislav V. Prodan > System & Network Administrator > http://support.od.ua > +380 67 4584408, +380 99 4060508 > VVP88-RIPE > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Ermal From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 10:08:25 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDCBB946; Thu, 10 Apr 2014 10:08:25 +0000 (UTC) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.32]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mailhost.dlr.de", Issuer "DLR CA - G02" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E8621CEC; Thu, 10 Apr 2014 10:08:25 +0000 (UTC) Received: from DLREXHUB01.intra.dlr.de (172.21.152.130) by dlrexedge01.dlr.de (172.21.163.100) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 12:08:01 +0200 Received: from KNOP-BEAGLE.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 12:08:00 +0200 Date: Thu, 10 Apr 2014 12:08:00 +0200 From: Harti Brandt X-X-Sender: brandt_h@KNOP-BEAGLE.kn.op.dlr.de To: Vladislav Prodan Subject: Re: Some gruesome moments with performance of FreeBSD at over 20K interfaces In-Reply-To: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> Message-ID: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key: harti@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Cc: stable@freebsd.org, hackers@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Harti Brandt List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 10:08:25 -0000 On Wed, 9 Apr 2014, Vladislav Prodan wrote: VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU VP>at 80-100% I could imagine that this is because of the statistics polling. bsnmp implements 64-bit interface statistics but we have only 32-bit statistics in the kernel. So it polls the kernel statistics for each interface on a rate that ensures that 32-bit don't overflow. If the interfaces are GBit or, worse, 10GBit interfaces the polling rate is rather high (in the order of seconds). You should either make sure that the interfaces report sensible bitrates (I doubt that 20k interfaces could all be GBit interfaces) or force a slower polling interval by setting begemotIfForcePoll.0 to some large value. harti From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 11:22:45 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0D3DC1E; Thu, 10 Apr 2014 11:22:45 +0000 (UTC) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.33]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mailhost.dlr.de", Issuer "DLR CA - G02" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 728EB1660; Thu, 10 Apr 2014 11:22:45 +0000 (UTC) Received: from DLREXHUB01.intra.dlr.de (172.21.152.130) by dlrexedge02.dlr.de (172.21.163.101) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 13:22:15 +0200 Received: from KNOP-BEAGLE.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 13:22:14 +0200 Date: Thu, 10 Apr 2014 13:22:52 +0200 From: Hartmut Brandt X-X-Sender: brandt_h@KNOP-BEAGLE.kn.op.dlr.de To: Vladislav Prodan Subject: Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces In-Reply-To: <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> Message-ID: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Cc: stable@freebsd.org, hackers@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 11:22:46 -0000 On Thu, 10 Apr 2014, Vladislav Prodan wrote: VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote: VP>> VP>> VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU VP>> VP>at 80-100% VP>> VP>> I could imagine that this is because of the statistics polling. bsnmp VP>> implements 64-bit interface statistics but we have only 32-bit statistics VP>> in the kernel. So it polls the kernel statistics for each interface on a VP>> rate that ensures that 32-bit don't overflow. If the interfaces are GBit VP>> or, worse, 10GBit interfaces the polling rate is rather high (in the order VP>> of seconds). VP>> VP>> You should either make sure that the interfaces report sensible bitrates VP>> (I doubt that 20k interfaces could all be GBit interfaces) or force a slower VP>> polling interval by setting begemotIfForcePoll.0 to some large value. VP>> VP>> harti VP>> VP> VP>Thanks for the tip. VP> VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M. VP>BegemotIfForcePoll parameter in this case a little help, you will be forced to stand another value for Gigabit Interface begemotIfForcePoll ... Yeah. There is only one parameter. You are running -STABLE, right? On current the statistics are 64-bit (I wonder whether the operation on these values is automatically atomic on all our platforms, though). harti From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 11:26:52 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CFC3ED7 for ; Thu, 10 Apr 2014 11:26:52 +0000 (UTC) Received: from frv189.fwdcdn.com (frv189.fwdcdn.com [212.42.77.189]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0471816A3 for ; Thu, 10 Apr 2014 11:26:51 +0000 (UTC) Received: from [10.10.1.29] (helo=frv197.fwdcdn.com) by frv189.fwdcdn.com with esmtp ID 1WYCo6-00037s-AZ for net@freebsd.org; Thu, 10 Apr 2014 14:06:10 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=znTrwNIHwNvgGMhZwpXKmoJJdn0w2U/UwtfH5CVLoRQ=; b=tJtWeEDpDPOxde0PqtRoMbMCwi1yOJzBGz1d+MHvGVFHij5OIPOkL/pkI71HepXZUo/t2dH5IXMPQ7msH68tHRHm3xXld35271hj+xC9n4bilZqTA6xRv1EaOZA/nxDEmtmiA9TEiIzNwmSkltwLZxfHfiyIDboDyQ1K7BQOEh4=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1WYCns-000DRG-Tc for net@freebsd.org; Thu, 10 Apr 2014 14:05:56 +0300 Date: Thu, 10 Apr 2014 14:05:56 +0300 From: Vladislav Prodan Subject: Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces To: Harti Brandt X-Mailer: mail.ukr.net 5.0 Message-Id: <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> In-Reply-To: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Thu, 10 Apr 2014 14:05:56 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: stable@freebsd.org, hackers@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 11:26:52 -0000 > On Wed, 9 Apr 2014, Vladislav Prodan wrote: > > VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU > VP>at 80-100% > > I could imagine that this is because of the statistics polling. bsnmp > implements 64-bit interface statistics but we have only 32-bit statistics > in the kernel. So it polls the kernel statistics for each interface on a > rate that ensures that 32-bit don't overflow. If the interfaces are GBit > or, worse, 10GBit interfaces the polling rate is rather high (in the order > of seconds). > > You should either make sure that the interfaces report sensible bitrates > (I doubt that 20k interfaces could all be GBit interfaces) or force a slower > polling interval by setting begemotIfForcePoll.0 to some large value. > > harti > Thanks for the tip. At least 10 interfaces to be 1Gb, and the rest no more than 50M. BegemotIfForcePoll parameter in this case a little help, you will be forced to stand another value for Gigabit Interface begemotIfForcePoll ... -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 12:38:36 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05C93DE; Thu, 10 Apr 2014 12:38:36 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C18A41CBC; Thu, 10 Apr 2014 12:38:35 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id v10so3807851pde.25 for ; Thu, 10 Apr 2014 05:38:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6gpWfHaYsRexDGvuNlPrJmc1sslnJOlW1f0XZEe3y8Q=; b=lMU17bz7NYq3kiWUOmC7WhvFcGdstp7QwMezr1g/x0nc3r2DLDcoii9/C99MNJok4G uIf6UO8zWRZMMhJjBlImem5AMTkr747APFowdlGa4RNmGqt2zvQD7of4W0GRyWqeIyCh xocsFBa6cAJ1oP+hSAHGAotRnwVJJkgMEZEcX2J5dd830TkoMCfgZXvIefodkhRYyDC8 5aQPhz8kD0Onsgc0ioCEezHrXhTJN5q+cLgLrYKqpfPVWqYkOecq68Ii0Sw75q2v34Ti o2MaJI9Qprf013fQCAj0WbmaCogLTmO+xfgPlzR926A1gqSrXRXqTf/zzEIqHzfzsJWA H2vg== MIME-Version: 1.0 X-Received: by 10.68.236.229 with SMTP id ux5mr19320244pbc.98.1397133515355; Thu, 10 Apr 2014 05:38:35 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.70.88.109 with HTTP; Thu, 10 Apr 2014 05:38:35 -0700 (PDT) In-Reply-To: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> Date: Thu, 10 Apr 2014 14:38:35 +0200 X-Google-Sender-Auth: yI4THH4Pe0R7_PwLx408BAWe5S4 Message-ID: Subject: Re: Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Hartmut Brandt Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: hackers@freebsd.org, stable@freebsd.org, Vladislav Prodan , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 12:38:36 -0000 Another note related to Q-in-Q. You would probably be better of creating standard vlans for the first vlan layer and use ng_vlan for the second++ part of the Q-in-Q on top of the first ones. This also give better usability and will speedup a bit your times. On Thu, Apr 10, 2014 at 1:22 PM, Hartmut Brandt wrote: > On Thu, 10 Apr 2014, Vladislav Prodan wrote: > > VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote: > VP>> > VP>> VP>b) Service bsnmpd started at 12K interfaces, but immediately > loaded CPU > VP>> VP>at 80-100% > VP>> > VP>> I could imagine that this is because of the statistics polling. bsnmp > VP>> implements 64-bit interface statistics but we have only 32-bit > statistics > VP>> in the kernel. So it polls the kernel statistics for each interface > on a > VP>> rate that ensures that 32-bit don't overflow. If the interfaces are > GBit > VP>> or, worse, 10GBit interfaces the polling rate is rather high (in the > order > VP>> of seconds). > VP>> > VP>> You should either make sure that the interfaces report sensible > bitrates > VP>> (I doubt that 20k interfaces could all be GBit interfaces) or force a > slower > VP>> polling interval by setting begemotIfForcePoll.0 to some large value. > VP>> > VP>> harti > VP>> > VP> > VP>Thanks for the tip. > VP> > VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M. > VP>BegemotIfForcePoll parameter in this case a little help, you will be > forced to stand another value for Gigabit Interface begemotIfForcePoll ... > > Yeah. There is only one parameter. > > You are running -STABLE, right? On current the statistics are 64-bit > (I wonder whether the operation on these values is automatically atomic on > all our platforms, though). > > harti > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Ermal From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 12:39:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4AC13343; Thu, 10 Apr 2014 12:39:56 +0000 (UTC) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D40B51CD8; Thu, 10 Apr 2014 12:39:55 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.7/8.14.7/ALCHEMY.FRANKEN.DE) with ESMTP id s3ACdr4p003972 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 10 Apr 2014 14:39:53 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.7/8.14.7/Submit) id s3ACdqi2003129; Thu, 10 Apr 2014 14:39:52 +0200 (CEST) (envelope-from marius) Date: Thu, 10 Apr 2014 14:39:52 +0200 From: Marius Strobl To: Yonghyeon PYUN Subject: Re: re0: watchdog timeout Message-ID: <20140410123952.GM86304@alchemy.franken.de> References: <53418053.5020105@deze.org> <280407068.6916295.1396827428537.JavaMail.root@uoguelph.ca> <20140407012241.GA3543@michelle.cdnetworks.com> <53424100.7030103@deze.org> <20140407083236.GB1357@michelle.cdnetworks.com> <5342F22C.5080606@deze.org> <20140408045939.GB1372@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140408045939.GB1372@michelle.cdnetworks.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 12:39:56 -0000 On Tue, Apr 08, 2014 at 01:59:39PM +0900, Yonghyeon PYUN wrote: > On Mon, Apr 07, 2014 at 08:45:00PM +0200, Frank Volf wrote: > > Yonghyeon PYUN schreef op 7-4-2014 10:32: > > >It would be even better to know your network configuration. I'm not > > >sure why you have to disable VLAN hardware tagging. But given that > > >you've disabled it, could you also try disabling VLAN hardware > > >checksum offloading? > > > > Hi, > > > > The reason that I disable VLAN hardware tagging is that the system does > > not work with it enabled. > > To show this, see the following transcript (on a freshly booted system): > > [...] > > Okat, I'll check VLAN hardware tagging with RTL8168G but watchdog > timeout is different issue. I have no idea why this happens at > this moment but I'll let you know if I find a clue. > Sorry for the late reply. I didn't hit any timeouts when testing the changes making re(4) work with DS47, including not when benchmarking performance. Unfortunately, that machine now is at a remote location not allowing further tests. Marius From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 12:42:13 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D464D4D6 for ; Thu, 10 Apr 2014 12:42:13 +0000 (UTC) Received: from frv198.fwdcdn.com (frv198.fwdcdn.com [212.42.77.198]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AA8F1D82 for ; Thu, 10 Apr 2014 12:42:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=MgtXFb7I6q1mbw2jOo94Diaf3Debe5Aa8QkbofjH1sU=; b=U6lMUU3fiV+Dy9RB88UviRw3wnEYWFfZYBSC+Di8DDHkstAYdG9jEZPwsHqqQNZw32mrAg5WAFVbadCZuDbnE8IT3TmmL67VeGrRazei8NVXRAOmBNtEPGWIX7ukJZZGj2EdaZOtx1S/qgEnACfy7Qa97KA0W7W+wKlkf4VCx+0=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1WYEIw-000Hx0-BB for net@freebsd.org; Thu, 10 Apr 2014 15:42:06 +0300 Date: Thu, 10 Apr 2014 15:42:05 +0300 From: Vladislav Prodan Subject: Re[2]: Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces To: Ermal =?iso-8859-1?b?THXnaQ==?= X-Mailer: mail.ukr.net 5.0 Message-Id: <1397133674.156493243.42smhi32@frv35.fwdcdn.com> In-Reply-To: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Thu, 10 Apr 2014 15:42:05 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: stable@freebsd.org, hackers@freebsd.org, "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 12:42:13 -0000 > > Another note related to Q-in-Q. > > > You would probably be better of creating standard vlans for the first vlan layer and use ng_vlan for the second++ part of the Q-in-Q on top of the first ones. > This also give better usability and will speedup a bit your times. > > > So I implemented the q-in-q. -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 15:18:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6FE7BF8 for ; Thu, 10 Apr 2014 15:18:26 +0000 (UTC) Received: from mail-ie0-x22c.google.com (mail-ie0-x22c.google.com [IPv6:2607:f8b0:4001:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 916CA1E4F for ; Thu, 10 Apr 2014 15:18:26 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id as1so4098262iec.17 for ; Thu, 10 Apr 2014 08:18:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=OWS1+6Eu/xpEtv00R0v8PyUWCrVvzGf1mpGGOJ4dILw=; b=QZ3jCcBus/b5qaiyJbM49RB8uIBY2RF/tHWxClKy9ZQdIiuffZZiSdp26OTmGrKCIH 3qPDIOImhW0v/M9h4Wg9PgkxlfCVMQ/s9T5WtrSo4Ri1dWvwWyML1sCdin8VwV/Cl03N +x4EUKX5NE036pVRpQccDIzuI9PrZNPlNqIpgAIhXsGEo9Ju6LoOFsAqWnXAzbtMNDEV APC7FL+a7HwMNq4qGAwHsLuMX4f61/TXJL8bTfBS/GiPanChC/aDAyHdK8mC+546XtQu f03vDW4/q9RFZwlXGRCCYyEkuONr3R1gbeDpYYdbzbUnTDEOl0Ctpcg5EEvaD5y3h6NL zYyw== X-Received: by 10.50.49.44 with SMTP id r12mr11225810ign.41.1397143105899; Thu, 10 Apr 2014 08:18:25 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id ix1sm20273631igc.15.2014.04.10.08.18.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 08:18:25 -0700 (PDT) Message-ID: <5346B63D.4080707@gmail.com> Date: Thu, 10 Apr 2014 11:18:21 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:18:26 -0000 Hi, By the way this change has opened the gates to greater performance for us when using ng_callout() inside nodes. In some cases we see twice as much pps since packets are direct dispatched instead of being queued in software interrupts threads (swi*). Thanks, Karim PS: I did file a PR : kern/188421 On 09/04/2014 9:09 PM, Adrian Chadd wrote: > Hi! > > Would you mind filing a PR for this? > > www.freebsd.org/send-pr.html > > That way it won't (hopefully!) get lost. > > Thanks! > > > -a > > > On 9 April 2014 14:25, Karim Fodil-Lemelin wrote: >> Hi, >> >> Below is a revised patch for this issue. It accounts for nodes or hooks that >> explicitly need to be queuing: >> >> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p >> hook, int ticks, >> >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + if ((node->nd_flags & NGF_FORCE_WRITER) || >> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >> >> + item->el_flags |= NGQF_WRITER; >> + else >> >> + item->el_flags |= NGQF_READER; >> + >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> Regards, >> >> Karim. >> >> >> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>> Hi List, >>> >>> I'm calling out to the general wisdom ... I have seen an issue in netgraph >>> where, if called, a callout routine registered by ng_callout() will trigger >>> packet queuing inside the worklist of netgraph since ng_callout() makes my >>> node suddenly a WRITER node (therefore non reentrant) for the duration of >>> the call. >>> >>> So as soon as the callout function returns, all following packets will get >>> directly passed to the node again and when the ngintr thread gets executed >>> then only then will I get the queued packets. This introduces out of order >>> packets in the flow. I am using the current patch below to solve the issue >>> and I am wondering if there is anything wrong with it (and maybe contribute >>> back :): >>> >>> >>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >>> hook, int ticks, >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + item->el_flags = NGQF_READER; >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> >>> Best regards, >>> >>> Karim. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 16:41:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2548F7C9 for ; Thu, 10 Apr 2014 16:41:43 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9D981849 for ; Thu, 10 Apr 2014 16:41:42 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so4121625qgf.7 for ; Thu, 10 Apr 2014 09:41:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gm51Cath61NqHXbQHfXhQ4G9BVRjDBMPUcGZ83ARmtc=; b=NrY4M4WReciNxsfCK/R3Pv/orpW09hSIIZtjjyubqibvxn7sFSUOKIwkBGGx2u6MqW wArsLRslNnP523EuG58NASwUZrXjT5l7hCcC5SxUU3K7v0Qp+/uumZbBzZn2yv7CIatf 5KzxUQrEDGT3bA7qnm2d245pJ2Qc1YzUiCMDxhyfTIzgg/2fTIXLH76dVwVB7xPgeUvP QtwegZc/u6AnvCacj+oD/6yTw+m1kEe4eevgyBzL/5weRQzfItgNlod32jhPaNuiUj4s aEHvjL1GVYcVhfu1n8Nvo2tIKywQneoiI03yfMlUia15b3ieRoIPx/zvrDQvAsdguHSV TYKw== MIME-Version: 1.0 X-Received: by 10.224.13.142 with SMTP id c14mr22439771qaa.76.1397148101966; Thu, 10 Apr 2014 09:41:41 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 09:41:41 -0700 (PDT) In-Reply-To: <5346B63D.4080707@gmail.com> References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> <5346B63D.4080707@gmail.com> Date: Thu, 10 Apr 2014 09:41:41 -0700 X-Google-Sender-Auth: G1ZuEJbYlPP__kwB-JFZNW79VH0 Message-ID: Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing From: Adrian Chadd To: Karim Fodil-Lemelin Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 16:41:43 -0000 Sweet! I'll ask around and see if anyone netmap clued can review. :) -a On 10 April 2014 08:18, Karim Fodil-Lemelin wrote: > Hi, > > By the way this change has opened the gates to greater performance for us > when using ng_callout() inside nodes. In some cases we see twice as much pps > since packets are direct dispatched instead of being queued in software > interrupts threads (swi*). > > Thanks, > > Karim > > PS: I did file a PR : kern/188421 > > > On 09/04/2014 9:09 PM, Adrian Chadd wrote: >> >> Hi! >> >> Would you mind filing a PR for this? >> >> www.freebsd.org/send-pr.html >> >> That way it won't (hopefully!) get lost. >> >> Thanks! >> >> >> -a >> >> >> On 9 April 2014 14:25, Karim Fodil-Lemelin >> wrote: >>> >>> Hi, >>> >>> Below is a revised patch for this issue. It accounts for nodes or hooks >>> that >>> explicitly need to be queuing: >>> >>> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p >>> hook, int ticks, >>> >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + if ((node->nd_flags & NGF_FORCE_WRITER) || >>> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >>> >>> + item->el_flags |= NGQF_WRITER; >>> + else >>> >>> + item->el_flags |= NGQF_READER; >>> + >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> Regards, >>> >>> Karim. >>> >>> >>> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>>> >>>> Hi List, >>>> >>>> I'm calling out to the general wisdom ... I have seen an issue in >>>> netgraph >>>> where, if called, a callout routine registered by ng_callout() will >>>> trigger >>>> packet queuing inside the worklist of netgraph since ng_callout() makes >>>> my >>>> node suddenly a WRITER node (therefore non reentrant) for the duration >>>> of >>>> the call. >>>> >>>> So as soon as the callout function returns, all following packets will >>>> get >>>> directly passed to the node again and when the ngintr thread gets >>>> executed >>>> then only then will I get the queued packets. This introduces out of >>>> order >>>> packets in the flow. I am using the current patch below to solve the >>>> issue >>>> and I am wondering if there is anything wrong with it (and maybe >>>> contribute >>>> back :): >>>> >>>> >>>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >>>> hook, int ticks, >>>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>>> return (ENOMEM); >>>> >>>> - item->el_flags |= NGQF_WRITER; >>>> + item->el_flags = NGQF_READER; >>>> NG_NODE_REF(node); /* and one for the item */ >>>> NGI_SET_NODE(item, node); >>>> if (hook) { >>>> >>>> >>>> Best regards, >>>> >>>> Karim. >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 02:17:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4042EF62 for ; Fri, 11 Apr 2014 02:17:56 +0000 (UTC) Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C09F212B5 for ; Fri, 11 Apr 2014 02:17:55 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id e49so3506179eek.25 for ; Thu, 10 Apr 2014 19:17:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=7TrUrv4NSh9tE9Bhydw9PuJoG9f28Et47m8hpUcwFtM=; b=HLZW5ACAeNQK4F1R9z46dEGsd6SiD9yRAltWfIUjQxsMtClKftkm4B+Ui6xnCrwaiN AfLG1fb3gKA4BGSmk2OZXHQQ8GbRm3WduEi0DWL4cQ7kfWb7MEeA+8FVWFSTHOIzTciV upma8ZLV3cUFo+mNs3kMcnP4098T1NpMttG5A6gTPYeY+L2v4Iawu+L2yjPIx0SVFr+z 0cN1TknqjZuhJ8uMrEM1BpsNyaMvldzs0Xl92MOoEn1bjMShsKOEr6CalsZbfHla0UJ+ wnOcyrwXSE0wY+Yvmg+M7nWqFcgzuDGutA8NtSilQULWuVNqnIcmkfqJL5R2Qn1g1fKc 1Y7Q== MIME-Version: 1.0 X-Received: by 10.14.176.193 with SMTP id b41mr7598611eem.55.1397182672262; Thu, 10 Apr 2014 19:17:52 -0700 (PDT) Received: by 10.14.213.194 with HTTP; Thu, 10 Apr 2014 19:17:52 -0700 (PDT) Date: Thu, 10 Apr 2014 19:17:52 -0700 Message-ID: Subject: netisr observations From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 02:17:56 -0000 (Note: This may seem more like a rant than an actual problem report.) I am on a stable-10ish box with igb0. Workload is mainly inbound nfs traffic. About 2K connections at any point in time. device igb # Intel PRO/1000 PCIE Server Gigabit Family hw.igb.rxd: 4096 hw.igb.txd: 4096 hw.igb.enable_aim: 1 hw.igb.enable_msix: 1 hw.igb.max_interrupt_rate: 32768 hw.igb.buf_ring_size: 4096 hw.igb.header_split: 0 hw.igb.num_queues: 0 hw.igb.rx_process_limit: 100 dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.0.%driver: igb dev.igb.0.%location: slot=0 function=0 dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x103c subdevice=0x323f class=0x020000 -bash-4.2$ netstat -I igb0 -i 1 input igb0 output packets errs idrops bytes packets errs bytes colls 18332 0 0 19096474 22946 0 18211000 0 19074 0 0 11408912 28280 0 29741195 0 15753 0 0 15499238 21234 0 16779695 0 12914 0 0 9583719 17945 0 14599603 0 13677 0 0 10818359 19050 0 15069889 0 -bash-4.2$ sysctl net.isr net.isr.dispatch: direct net.isr.maxthreads: 8 net.isr.bindthreads: 0 net.isr.maxqlimit: 10240 net.isr.defaultqlimit: 256 net.isr.maxprot: 16 net.isr.numthreads: 8 -bash-4.2$ sysctl -a | grep igb.0 | grep rx_bytes dev.igb.0.queue0.rx_bytes: 65473003127 dev.igb.0.queue1.rx_bytes: 73982776038 dev.igb.0.queue2.rx_bytes: 57669494795 dev.igb.0.queue3.rx_bytes: 57830053867 dev.igb.0.queue4.rx_bytes: 75087429774 dev.igb.0.queue5.rx_bytes: 69252615374 dev.igb.0.queue6.rx_bytes: 70565370833 dev.igb.0.queue7.rx_bytes: 90210083223 I am seeing something interesting in "top": PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 12 root 68 -72 - 0K 1088K WAIT 0 279:36 65.77% intr I see "intr" in on of top 3 slots almost all the time. turning on -H (thread view) shows me: 12 root -72 - 0K 1088K WAIT 2 69:04 20.36% intr{swi1: netisr 3} (Does this mean netisr has swi (software interrupt) on cpu3?) also, I see this process jumping to all different CPUs (so its not sticking to 1 cpu) -bash-4.2$ vmstat -i interrupt total rate irq4: uart0 1538 0 cpu0:timer 23865486 1108 irq256: igb0:que 0 46111948 2140 irq257: igb0:que 1 49820986 2313 irq258: igb0:que 2 41914519 1945 irq259: igb0:que 3 40926921 1900 irq260: igb0:que 4 49549124 2300 irq261: igb0:que 5 47066777 2185 irq262: igb0:que 6 50945395 2365 irq263: igb0:que 7 47147662 2188 irq264: igb0:link 2 0 irq274: ahci0:ch0 196869 9 cpu1:timer 23866170 1108 cpu10:timer 23805794 1105 cpu4:timer 23870757 1108 cpu11:timer 23806733 1105 cpu13:timer 23806644 1105 cpu2:timer 23858811 1107 cpu3:timer 23862250 1107 cpu15:timer 23805634 1105 cpu7:timer 23863865 1107 cpu9:timer 23810503 1105 cpu5:timer 23864136 1107 cpu12:timer 23808397 1105 cpu8:timer 23806059 1105 cpu6:timer 23874612 1108 cpu14:timer 23807698 1105 Total 755065290 35055 So, i seems all queues are being used uniformly. -bash-4.2$ netstat -Q Configuration: Setting Current Limit Thread count 8 8 Default queue limit 256 10240 Dispatch policy direct n/a Threads bound to CPUs disabled n/a Protocols: Name Proto QLimit Policy Dispatch Flags ip 1 1024 flow default --- igmp 2 256 source default --- rtsock 3 256 source default --- arp 7 256 source default --- ether 9 256 source direct --- ip6 10 256 flow default --- But the *interesting* part from it: -bash-4.2$ netstat -Q | grep "ip " (looking at just ip in workstreams) Workstreams: WSID CPU Name Len WMark Disp'd HDisp'd QDrops Queued Handled 0 0 ip 0 0 73815267 0 0 0 73815267 1 1 ip 0 0 68975084 0 0 0 68975084 2 2 ip 0 0 48943960 0 0 0 48943960 3 3 ip 0 67 59306618 0 0 203888563 263168729 4 4 ip 0 0 77025108 0 0 0 77025108 5 5 ip 0 0 58537310 0 0 0 58537310 6 6 ip 0 0 81896427 0 0 0 81896427 7 7 ip 0 0 69535857 0 0 0 69535857 So, looks like only cpu3 is doing all the queuing. But it doesn't look like it's getting hammered or anything: last pid: 75181; load averages: 27.81, 27.08, 26.93 up 0+06:12:37 19:04:33 508 processes: 23 running, 476 sleeping, 1 waiting, 8 lock CPU 0: 71.8% user, 0.0% nice, 13.7% system, 14.5% interrupt, 0.0% idle CPU 1: 80.9% user, 0.0% nice, 14.5% system, 4.6% interrupt, 0.0% idle CPU 2: 77.1% user, 0.0% nice, 17.6% system, 5.3% interrupt, 0.0% idle CPU 3: 88.5% user, 0.0% nice, 9.2% system, 2.3% interrupt, 0.0% idle CPU 4: 80.2% user, 0.0% nice, 14.5% system, 5.3% interrupt, 0.0% idle CPU 5: 79.4% user, 0.0% nice, 16.8% system, 3.1% interrupt, 0.8% idle CPU 6: 83.2% user, 0.0% nice, 11.5% system, 4.6% interrupt, 0.8% idle CPU 7: 68.7% user, 0.0% nice, 18.3% system, 13.0% interrupt, 0.0% idle CPU 8: 88.5% user, 0.0% nice, 11.5% system, 0.0% interrupt, 0.0% idle CPU 9: 87.8% user, 0.0% nice, 10.7% system, 0.0% interrupt, 1.5% idle CPU 10: 87.0% user, 0.0% nice, 10.7% system, 2.3% interrupt, 0.0% idle CPU 11: 80.9% user, 0.0% nice, 16.8% system, 2.3% interrupt, 0.0% idle CPU 12: 86.3% user, 0.0% nice, 11.5% system, 2.3% interrupt, 0.0% idle CPU 13: 84.7% user, 0.0% nice, 14.5% system, 0.8% interrupt, 0.0% idle CPU 14: 87.0% user, 0.0% nice, 12.2% system, 0.8% interrupt, 0.0% idle CPU 15: 87.8% user, 0.0% nice, 9.9% system, 2.3% interrupt, 0.0% idle Mem: 17G Active, 47G Inact, 3712M Wired, 674M Cache, 1655M Buf, 1300M Free Swap: 8192M Total, 638M Used, 7554M Free, 7% Inuse, 4K In My conclusion after lookinag it for a bunch of times that all CPUs are equally doing work (if we believe top -P stats) Finally, the question: why is cpu3 doing all the queuing. and what does that actually mean? Can I improve performance OR reduce cpu load any other way? Should I change anything in my netisr settings? cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 06:48:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 449DE358 for ; Fri, 11 Apr 2014 06:48:49 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 074501BE8 for ; Fri, 11 Apr 2014 06:48:48 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so5436312qcx.6 for ; Thu, 10 Apr 2014 23:48:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9XXPheSpRJy9jcokx/zVBNVGxdYx1cL9zrGvwzA4wKU=; b=i0rDg5MBuBsjPp80NmhmL27EF9kDsMk4nP1zLcg57MkYpa0l3cvv+S5ABeHDR3WyRq UYDBeT5E6hH8uaWDw3loc0KgpGMzNDvF0AvCHnOxjNIJX7vdoaIOl4QhNiRxSBbwwsdE M56tUWe3UobSu/EBErLzpGrFmvmKRofBiN+xrmn4pXKJI2bY4HoW0y/OKC8X2uDkcIxU 517nr3pRNhmWfk713hCV8Abhwsp2U2dBmuSnlENxGKqMiZbr+grrQ0MdPCZV8i5Cju70 N8irSW05TmUtUZXtp5WhLKTtkpa5Hep0RG9h+4XYbc4oKh0R+7idvaiYuvE9Xjq2T7CX 5+fA== MIME-Version: 1.0 X-Received: by 10.140.16.37 with SMTP id 34mr24721792qga.37.1397198927462; Thu, 10 Apr 2014 23:48:47 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 23:48:47 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Apr 2014 23:48:47 -0700 X-Google-Sender-Auth: p2tkU8-X2c3VU0gOtgzPVexNKk8 Message-ID: Subject: Re: netisr observations From: Adrian Chadd To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 06:48:49 -0000 [snip] So, hm, the thing that comes to mind is the flowid. What's the various flowid's for flows? Are they all mapping to CPU 3 somehow? -a From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 07:42:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38B9ECB7 for ; Fri, 11 Apr 2014 07:42:33 +0000 (UTC) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2001:470:1f09:14c0::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Norma UNIX CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A1B0B128F for ; Fri, 11 Apr 2014 07:42:32 +0000 (UTC) Received: from bsdrookie.norma.com. (bsdrookie.norma.com [192.168.7.224]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id s3B7gTKD070403 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 11 Apr 2014 13:42:29 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <53479CE5.9070700@norma.perm.ru> Date: Fri, 11 Apr 2014 13:42:29 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: tunnels, mtu and payload length Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [192.168.3.10]); Fri, 11 Apr 2014 13:42:29 +0600 (YEKT) X-Spam-Status: No hits=-101.0 bayes=0.5 testhits ALL_TRUSTED=-1, USER_IN_WHITELIST=-100 autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 07:42:33 -0000 Hi. Can someone explain me where are the 4 missing bytes when capturing traffic on a gif interface with a tcpdump ? I expect to see the length of the first fragment (offset = 0) to be equal to an mtu (1280 bytes), but clearly it's 1276 bytes. Same thing happens to a gre tunnel. # ifconfig gif0 gif0: flags=8051 metric 0 mtu 1280 tunnel inet 192.168.3.24 --> 192.168.3.17 inet 172.16.5.40 --> 172.16.5.41 netmask 0xffffffff inet6 fe80::21a:64ff:fe21:8e80%gif0 prefixlen 64 scopeid 0x1c nd6 options=21 # ping -s 4096 172.16.5.41 PING 172.16.5.41 (172.16.5.41): 4096 data bytes 4104 bytes from 172.16.5.41: icmp_seq=0 ttl=64 time=0.837 ms 4104 bytes from 172.16.5.41: icmp_seq=1 ttl=64 time=0.870 ms 4104 bytes from 172.16.5.41: icmp_seq=2 ttl=64 time=0.779 ms 4104 bytes from 172.16.5.41: icmp_seq=3 ttl=64 time=0.823 ms 4104 bytes from 172.16.5.41: icmp_seq=4 ttl=64 time=0.794 ms tcpdump: 12:58:33.430450 IP (tos 0x0, ttl 64, id 40760, offset 0, flags [+], proto ICMP (1), length 1276) 172.16.5.40 > 172.16.5.41: ICMP echo request, id 62980, seq 17, length 1256 12:58:33.430467 IP (tos 0x0, ttl 64, id 40760, offset 1256, flags [+], proto ICMP (1), length 1276) 172.16.5.40 > 172.16.5.41: ip-proto-1 12:58:33.430481 IP (tos 0x0, ttl 64, id 40760, offset 2512, flags [+], proto ICMP (1), length 1276) 172.16.5.40 > 172.16.5.41: ip-proto-1 12:58:33.430494 IP (tos 0x0, ttl 64, id 40760, offset 3768, flags [none], proto ICMP (1), length 356) 172.16.5.40 > 172.16.5.41: ip-proto-1 Thanks. Eugene. From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 08:58:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9BE4E12 for ; Fri, 11 Apr 2014 08:58:48 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 682B01A6A for ; Fri, 11 Apr 2014 08:58:47 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1WYXJ1-000M2M-D0 for freebsd-net@freebsd.org; Fri, 11 Apr 2014 12:59:27 +0400 Message-ID: <5347AEAA.9090801@smartspb.net> Date: Fri, 11 Apr 2014 12:58:18 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: dummynet/ipfw high load? X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 140410-1, 10.04.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 08:58:48 -0000 Good day, gurus! We have a servers on the FreeBSD. They do NAT, shaping and traffic accounting for our home (mainly) customers. NAT realized with pf nat, shaping with ipfw dummynet and traffic accounting with ng_netflow via ipfw ng_tee. The problem is performance on (relatively) high traffic. On Xeon E3-1270, whereas use Intel 10Gbit/sec 82599-based NIC(ix) or Intel I350 (82579) in lagg transit traffic in 800 Mbit/sec and 100 kpps [to customers] cause CPU load almost at 100% by interrupts from NIC or, in case of net.isr.dispatch=deferred and net.inet.ip.fastforwarding=0. Deleting ipfw pipe decrease load at ~30% per cpu. Deleting ipfw ng_tee (to ng_netflow) decrease load at 15% per cpu. Turning off ipfw (sysctl net.inet.ip.fw.enable=0) decrease load more, so what server can pass (nat'ed!) traffic on 1600 Mbit/sec and 200 kpps with only load ~40% per cpu. So my questions are: 1. Are there any way to decrease system load caused by dummynet/ipfw? 2. Why dummynet/ipfw increase *interrupts* load, not kernel or something like that? 3. Are there any way to profiling that kind of load? Existing DTrace and pmcstat examples almost useless or I just doesn't know how to do it properly. Huge size of debugging info (including dtrace and pmcstat samples), sysctl settings and so on, I opened appropriate topic at russian network operator's forum: http://forum.nag.ru/forum/index.php?showtopic=93674 In english it's available via google translate: http://translate.google.com/translate?hl=en&sl=auto&tl=en&u=http%3A%2F%2Fforum.nag.ru%2Fforum%2Findex.php%3Fshowtopic%3D93674 Feel free to ask me any question and do actions on the server! I would be VERY appreciate for any help and can take any measuring and debugging on the one server. Moreover, I'm ready to give root access to any of the appropriate person (as I already did it to Gleb Smirnoff when we were investigate pf state problem). -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 11:15:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 689E83C8; Fri, 11 Apr 2014 11:15:14 +0000 (UTC) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx2.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AD59193F; Fri, 11 Apr 2014 11:15:13 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.97,841,1389772800"; d="asc'?scan'208";a="82152512" Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx2-out.netapp.com with ESMTP; 11 Apr 2014 04:15:13 -0700 Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.246]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Fri, 11 Apr 2014 04:15:13 -0700 From: "Eggert, Lars" To: "freebsd-net@freebsd.org" Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 Thread-Topic: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 Thread-Index: AQHPIYzc3HaTSw//+U6b0HMYTUDKSZsNINqA Date: Fri, 11 Apr 2014 11:15:12 +0000 Message-ID: References: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> In-Reply-To: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.105.18] Content-Type: multipart/signed; boundary="Apple-Mail=_D1FFEC48-26FD-4FAE-9F8E-C6BF320B08A9"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: Adrian Chadd , hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 11:15:14 -0000 --Apple-Mail=_D1FFEC48-26FD-4FAE-9F8E-C6BF320B08A9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, since folks are playing with Midori's DCTCP patch, I wanted to make sure = that you were also aware of the patches that Aris did for PRR and = NewCWV... Lars On 2014-2-4, at 10:38, Eggert, Lars wrote: > Hi, >=20 > below are two patches that implement RFC6937 ("Proportional Rate = Reduction for TCP") and draft-ietf-tcpm-newcwv-00 ("Updating TCP to = support Rate-Limited Traffic"). They were done by Aris = Angelogiannopoulos for his MS thesis, which is at = https://eggert.org/students/angelogiannopoulos-thesis.pdf. >=20 > The patches should apply to -CURRENT as of Sep 17, 2013. (Sorry for = the delay in sending them, we'd been trying to get some feedback from = committers first, without luck.) >=20 > Please note that newcwv is still a work in progress in the IETF, and = the patch has some limitations with regards to the "pipeACK Sampling = Period" mentioned in the Internet-Draft. Aris says this in his thesis = about what exactly he implemented: >=20 > "The second implementation choice, is in regards with the measurement = of pipeACK. This variable is the most important introduced by the method = and is used to compute the phase that the sender currently lies in. In = order to compute pipeACK the approach suggested by the Internet Draft = (ID) is followed [ncwv]. During initialization, pipeACK is set to the = maximum possible value. A helper variable prevHighACK is introduced that = is initialized to the initial sequence number (iss). prevHighACK holds = the value of the highest acknowledged byte so far. pipeACK is measured = once per RTT meaning that when an ACK covering prevHighACK is received, = pipeACK becomes the difference between the current ACK and prevHighACK. = This is called a pipeACK sample. A newer version of the draft suggests = that multiple pipeACK samples can be used during the pipeACK sampling = period." >=20 > Lars >=20 > --Apple-Mail=_D1FFEC48-26FD-4FAE-9F8E-C6BF320B08A9 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU0fOv9ZcnpRveo1xAQKPnAP+M4t/ovAfeWD9rBGqyB3pfUzGASg7B5OW vAL3K6aaWc08OxduE5tw4KlsrdAAIa8p65fEBVWCBwxwthuzzQCVfIPf7948LLf5 2lWsVWSvuoAKmFS6+LS2iha+lDFbYgkV9GqrDsBahk53SLv2eCI3mx+048JAlw80 MOMAOtj2YNU= =UgVh -----END PGP SIGNATURE----- --Apple-Mail=_D1FFEC48-26FD-4FAE-9F8E-C6BF320B08A9-- From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 16:05:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF45F1CE for ; Fri, 11 Apr 2014 16:05:55 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6A8C1C05 for ; Fri, 11 Apr 2014 16:05:55 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id x10so5461640pdj.20 for ; Fri, 11 Apr 2014 09:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cApkMMC+DqWO/oQJpHnmqv1MpN8R9ZBRWDbXRoj1WOU=; b=Egxdox7jC7az+2mxP/IDQwNKRH2lt6RlPSShS0ProPQw9pLKCRmWsa6MVxaaKriROS GJdL6mbk6Wllx+vyMIy4DwzFk9miF/Wuc0exNolFjcohVkTWpTDXf1Sfeq7xFESAT+b5 QPOz+rt7GEBvVld66k8dekP4EBFsyAxHAKdNHpTN8tj1DsuONCfmorItqgv4vFamPIfY kLdTucwdF983KrfY7/F0RleOl13sHfRR2t3g/pXGhkE7hHuyAyoi6Y5RB6sxtCum+H4t YIxVJM7eSprJ4yE8267AZxqPXRLNG/Hauy3KHFlNABNFezhoU/remeXe4Jts9dWAbyWG VKDw== MIME-Version: 1.0 X-Received: by 10.66.140.104 with SMTP id rf8mr27962930pab.107.1397232355307; Fri, 11 Apr 2014 09:05:55 -0700 (PDT) Received: by 10.70.127.136 with HTTP; Fri, 11 Apr 2014 09:05:55 -0700 (PDT) In-Reply-To: <5347AEAA.9090801@smartspb.net> References: <5347AEAA.9090801@smartspb.net> Date: Fri, 11 Apr 2014 19:05:55 +0300 Message-ID: Subject: Re: dummynet/ipfw high load? From: Sami Halabi To: Dennis Yusupoff Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 16:05:55 -0000 Hi, I had similar problem on the past and it turned to be the ammount of rules in ipfe. Using reduced subset with tables actually reduced the load. Sami =E2=80=8F=D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=A9=D7= =99=D7=A9=D7=99, 11 =D7=91=D7=90=D7=A4=D7=A8=D7=99=D7=9C 2014, Dennis Yusup= off =D7=9B=D7=AA=D7=91: > Good day, gurus! > > We have a servers on the FreeBSD. They do NAT, shaping and traffic > accounting for our home (mainly) customers. > NAT realized with pf nat, shaping with ipfw dummynet and traffic > accounting with ng_netflow via ipfw ng_tee. > The problem is performance on (relatively) high traffic. > On Xeon E3-1270, whereas use Intel 10Gbit/sec 82599-based NIC(ix) or > Intel I350 (82579) in lagg transit traffic in 800 Mbit/sec and 100 kpps > [to customers] cause CPU load almost at 100% by interrupts from NIC or, > in case of net.isr.dispatch=3Ddeferred and net.inet.ip.fastforwarding=3D0= . > Deleting ipfw pipe decrease load at ~30% per cpu. > Deleting ipfw ng_tee (to ng_netflow) decrease load at 15% per cpu. > Turning off ipfw (sysctl net.inet.ip.fw.enable=3D0) decrease load more, s= o > what server can pass (nat'ed!) traffic on 1600 Mbit/sec and 200 kpps > with only load ~40% per cpu. > > So my questions are: > 1. Are there any way to decrease system load caused by dummynet/ipfw? > 2. Why dummynet/ipfw increase *interrupts* load, not kernel or > something like that? > 3. Are there any way to profiling that kind of load? Existing DTrace > and pmcstat examples almost useless or I just doesn't know how to do it > properly. > > Huge size of debugging info (including dtrace and pmcstat samples), > sysctl settings and so on, I opened appropriate topic at russian network > operator's forum: http://forum.nag.ru/forum/index.php?showtopic=3D93674 > In english it's available via google translate: > > http://translate.google.com/translate?hl=3Den&sl=3Dauto&tl=3Den&u=3Dhttp%= 3A%2F%2Fforum.nag.ru%2Fforum%2Findex.php%3Fshowtopic%3D93674 > > Feel free to ask me any question and do actions on the server! > > I would be VERY appreciate for any help and can take any measuring and > debugging on the one server. Moreover, I'm ready to give root access to > any of the appropriate person (as I already did it to Gleb Smirnoff when > we were investigate pf state problem). > > > -- > Best regards, > Dennis Yusupoff, > network engineer of > Smart-Telecom ISP > Russia, Saint-Petersburg > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > --=20 Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 16:16:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DA6BA16; Fri, 11 Apr 2014 16:16:07 +0000 (UTC) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3F731D78; Fri, 11 Apr 2014 16:16:06 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id b15so4335002eek.34 for ; Fri, 11 Apr 2014 09:16:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=d0RS7miKnPBDax+rIAdszU+Z1ChzfV95L4JMV4PR5e4=; b=X9FiXJDX/hm6EPE+vDbq54MAd6tyDqGU+qXyO7varL9gNNZYd8D9vmJj0ypD7fwPT2 5bnS49+8suXW9Le3m1ydq/yG48YIjyQaUThWUgZ5+IiORIiqxL8Mj10h6XZFGSNXmw6J 0Ot01XvaDIgJHPGa12rOhRR2XKmFVFJoHQsNbUbjcQ3Pf3gGMco/s2AvjO+SchlNuAyz YeygG8ub8GH8JSdbaMrQwjqbl6TFqd+V/vOgvW5EwzQD9PUVEWatnA94EIskcakQrs/e DB3omHC3fa4WdYXD8ZtHlNaHIY/jAElhP2po1iViEVZslQ+aJ+igydF2eFhVjADbp8Nk Xm1g== MIME-Version: 1.0 X-Received: by 10.15.53.135 with SMTP id r7mr3196084eew.102.1397232964972; Fri, 11 Apr 2014 09:16:04 -0700 (PDT) Received: by 10.14.213.194 with HTTP; Fri, 11 Apr 2014 09:16:04 -0700 (PDT) In-Reply-To: References: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> Date: Fri, 11 Apr 2014 09:16:04 -0700 Message-ID: Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 From: hiren panchasara To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 16:16:07 -0000 On Fri, Apr 11, 2014 at 4:15 AM, Eggert, Lars wrote: > Hi, > > since folks are playing with Midori's DCTCP patch, I wanted to make sure that you were also aware of the patches that Aris did for PRR and NewCWV... >> >> Lars, There are no actual patches attached here. (Or the mailing-list dropped them.) cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 18:30:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB05E8BE; Fri, 11 Apr 2014 18:30:47 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 229B41CCC; Fri, 11 Apr 2014 18:30:46 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id z11so3744392lbi.22 for ; Fri, 11 Apr 2014 11:30:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=N8tW2BGPT1qKRq8wvR80x/auqRYVlgEB9X1AsS8cNuA=; b=rBJxbQXuR7mH3fcuL2NtTjSaff9fy7VFpTZWsw6FhKXw/3M7w5yCSXsApl+dXHUbaE YvTwxs122AbAtnEhphDrF2YUokMz4i/SQxKNhdb9aNRz07OLfn6nhKJtUx2bgc0gWwNI OiQfjSoXjKW0P6ss749mLf6MPf9hP+83XFC5qgheb4SKXpwM40Pf5wSN+emz7OPerYCE wnRXqcb9GyzL0e+HqMm6y2M+PeY9yp038lbG4/cztBEolCOSfHh3FVACBvYm49ONwzTp 1MXo/qK4zDXLesbLK8T18Q8bV/advHCB3BYwlKNFfHnKwYBRdLB4tYML0QTTCKytDmdB +BuQ== MIME-Version: 1.0 X-Received: by 10.152.44.234 with SMTP id h10mr106068lam.68.1397241044572; Fri, 11 Apr 2014 11:30:44 -0700 (PDT) Sender: pkelsey@gmail.com Received: by 10.112.141.196 with HTTP; Fri, 11 Apr 2014 11:30:44 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Apr 2014 14:30:44 -0400 X-Google-Sender-Auth: jcs4ZdF4AHAVEZhiKCkt9rQrdG0 Message-ID: Subject: Re: netisr observations From: Patrick Kelsey To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:30:47 -0000 On Fri, Apr 11, 2014 at 2:48 AM, Adrian Chadd wrote: > [snip] > > So, hm, the thing that comes to mind is the flowid. What's the various > flowid's for flows? Are they all mapping to CPU 3 somehow The output of netstat -Q shows IP dispatch is set to default, which is direct (NETISR_DISPATCH_DIRECT). That means each IP packet will be processed on the same CPU that the Ethernet processing for that packet was performed on, so CPU selection for IP packets will not be based on flowid. The output of netstat -Q shows Ethernet dispatch is set to direct (NETISR_DISPATCH_DIRECT if you wind up reading the code), so the Ethernet processing for each packet will take place on the same CPU that the driver receives that packet on. For the igb driver with queues autoconfigured and msix enabled, as the sysctl output shows you have, the driver will create a number of queues subject to device limitations, msix message limitations, and the number of CPUs in the system, establish a separate interrupt handler for each one, and bind each of those interrupt handlers to a separate CPU. It also creates a separate single-threaded taskqueue for each queue. Each queue interrupt handler sends work to its associated taskqueue when the interrupt fires. Those taskqueues are where the Ethernet packets are received and processed by the driver. The question is where those taskqueue threads will be run. I don't see anything in the driver that makes an attempt to bind those taskqueue threads to specific CPUs, so really the location of all of the packet processing is up to the scheduler (i.e., arbitrary). The summary is: 1. the hardware schedules each received packet to one of its queues and raises the interrupt for that queue 2. that queue interrupt is serviced on the same CPU all the time, which is different from the CPUs for all other queues on that interface 3. the interrupt handler notifies the corresponding task queue, which runs its task in a thread on whatever CPU the scheduler chooses 4. that task dispatches the packet for Ethernet processing via netisr, which processes it on whatever the current CPU is 5. Ethernet processing dispatches that packet for IP processing via netisr, which processes it on whatever the current CPU is You might want to try changing the default netisr dispatch policy to 'deferred' (sysctl net.isr.dispatch=deferred). If you do that, the Ethernet processing will still happen on an arbitrary CPU chosen by the scheduler, but the IP processing should then get mapped to a CPU based on the flowid assigned by the driver. Since igb assigns flowids based on received queue number, all IP (and above) processing for that packet should then be performed on the same CPU the queue interrupt was bound to. Unfortunately, I don't have a system with igb interfaces to try that on. -Patrick From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 18:59:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A59FCB6 for ; Fri, 11 Apr 2014 18:59:35 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D84FD112E for ; Fri, 11 Apr 2014 18:59:34 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3BIxUKU004077 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 11 Apr 2014 11:59:33 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53483B8B.7030409@freebsd.org> Date: Sat, 12 Apr 2014 02:59:23 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> In-Reply-To: <5345BAE7.4010501@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:59:35 -0000 disclaimer: I'm not looking at the code now.. I want to go to bed: :-) When I wrote that code, the idea was that even a direct node execution should become a queuing operation if there was already something else on the queue. so in that model packets were not supposed to get re-ordered. does that not still work? Either that, or you need to explain the problem to me a bit better.. On 4/10/14, 5:25 AM, Karim Fodil-Lemelin wrote: > Hi, > > Below is a revised patch for this issue. It accounts for nodes or > hooks that explicitly need to be queuing: > > @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, > hook_p hook, int ticks, > if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) > return (ENOMEM); > > - item->el_flags |= NGQF_WRITER; > + if ((node->nd_flags & NGF_FORCE_WRITER) || > + (hook && (hook->hk_flags & HK_FORCE_WRITER))) > + item->el_flags |= NGQF_WRITER; > + else > + item->el_flags |= NGQF_READER; > + > NG_NODE_REF(node); /* and one for the item */ > NGI_SET_NODE(item, node); > if (hook) { > > Regards, > > Karim. > > On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >> Hi List, >> >> I'm calling out to the general wisdom ... I have seen an issue in >> netgraph where, if called, a callout routine registered by >> ng_callout() will trigger packet queuing inside the worklist of >> netgraph since ng_callout() makes my node suddenly a WRITER node >> (therefore non reentrant) for the duration of the call. >> >> So as soon as the callout function returns, all following packets >> will get directly passed to the node again and when the ngintr >> thread gets executed then only then will I get the queued packets. >> This introduces out of order packets in the flow. I am using the >> current patch below to solve the issue and I am wondering if there >> is anything wrong with it (and maybe contribute back :): >> >> >> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, >> hook_p hook, int ticks, >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + item->el_flags = NGQF_READER; >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> >> Best regards, >> >> Karim. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 19:51:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 397A0C82; Fri, 11 Apr 2014 19:51:10 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEA9815F4; Fri, 11 Apr 2014 19:51:09 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id m20so6445575qcx.35 for ; Fri, 11 Apr 2014 12:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Y4X9hykRQGajzVTU1qAE3wxhe5y8kEd5uChh7a/1cvY=; b=pSMMc5RcqttygHj+jPxmKsDEO15v+5OArTc0bMocCHZPRJEIOaIw3BsGDtphXFkxEr c1ERM2clrzIrQJHkf1mj2kwrh/IKkN/YMTpBuQKCx4fFaDacF43JLhLjh9GWnB1rxihC 1jMY1A3proe9v/DjXVfo2XAUUXxo0Vr4qF82rfU7PSpVwtdrIvIeoNaaU+R1niTiqGMi ttDfYj+Ne+7IyG/p1ZzwZ2UVU8UbIfxFPZph8yRWtheSZqeM8qlc4jHlD+1t/7A1q+fI lDssvao2UUQOA0vPfpeVm2XR9JzwSJqHZE95ELpLBQO8+12sMIcOy9mO5nXvhKzh7BPo mosg== MIME-Version: 1.0 X-Received: by 10.229.53.136 with SMTP id m8mr32381680qcg.4.1397245868679; Fri, 11 Apr 2014 12:51:08 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Fri, 11 Apr 2014 12:51:08 -0700 (PDT) In-Reply-To: <53483B8B.7030409@freebsd.org> References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> <53483B8B.7030409@freebsd.org> Date: Fri, 11 Apr 2014 12:51:08 -0700 X-Google-Sender-Auth: YP-mfjcIlbZEbQhEadWSwER5ZgM Message-ID: Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing From: Adrian Chadd To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 19:51:10 -0000 Well, ethernet drivers nowdays seem to be doing: * always queue * then pop the head item off the queue and transmit that. -a On 11 April 2014 11:59, Julian Elischer wrote: > disclaimer: I'm not looking at the code now.. I want to go to bed: :-) > > When I wrote that code, the idea was that even a direct node execution > should become a queuing operation if there was already something else on the > queue. so in that model packets were not supposed to get re-ordered. does > that not still work? > > Either that, or you need to explain the problem to me a bit better.. > > > > On 4/10/14, 5:25 AM, Karim Fodil-Lemelin wrote: >> >> Hi, >> >> Below is a revised patch for this issue. It accounts for nodes or hooks >> that explicitly need to be queuing: >> >> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p >> hook, int ticks, >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + if ((node->nd_flags & NGF_FORCE_WRITER) || >> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >> + item->el_flags |= NGQF_WRITER; >> + else >> + item->el_flags |= NGQF_READER; >> + >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> Regards, >> >> Karim. >> >> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>> >>> Hi List, >>> >>> I'm calling out to the general wisdom ... I have seen an issue in >>> netgraph where, if called, a callout routine registered by ng_callout() will >>> trigger packet queuing inside the worklist of netgraph since ng_callout() >>> makes my node suddenly a WRITER node (therefore non reentrant) for the >>> duration of the call. >>> >>> So as soon as the callout function returns, all following packets will >>> get directly passed to the node again and when the ngintr thread gets >>> executed then only then will I get the queued packets. This introduces out >>> of order packets in the flow. I am using the current patch below to solve >>> the issue and I am wondering if there is anything wrong with it (and maybe >>> contribute back :): >>> >>> >>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >>> hook, int ticks, >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + item->el_flags = NGQF_READER; >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> >>> Best regards, >>> >>> Karim. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Apr 11 22:00:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5663B80A; Fri, 11 Apr 2014 22:00:27 +0000 (UTC) Received: from mail-ee0-x229.google.com (mail-ee0-x229.google.com [IPv6:2a00:1450:4013:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCE46134C; Fri, 11 Apr 2014 22:00:26 +0000 (UTC) Received: by mail-ee0-f41.google.com with SMTP id t10so4591665eei.14 for ; Fri, 11 Apr 2014 15:00:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vRMwkzvXQkxSwFuwupg1Bzw7l1AC0h46Jhn/DBR0Mcg=; b=mz1QAmGmRi+S+pbM5hFjBZ+OQSnar5mAiR0RpfVCHfQFuyjNP04Syiq7azmgbNGpCk BwQb3JVOZchsBJEg3U0rmThG6jV4vydUKzctRV854L4Nhm6J2Ds3vNQSmINaQGQ+XXeZ LeWCwcQ9AsNc78cjgXeVWLugbOSAYJpWE4T2dgReGs75/hPBLgiltZtDJQeTsapkFICf ubZILmNXaxZJ+not5pc5hj8uDvvqbXwK/wL+sQPXEIfldjI/+uLKu+vCjgSoG9+lvMLS ZwRX9+INI28aRACt5EKLV8AM8X84jJq/l57nRf9x+C+SbVicWNVtYCzj4p0sJjLr1L/B 8fDA== MIME-Version: 1.0 X-Received: by 10.14.194.70 with SMTP id l46mr4840717een.95.1397253625069; Fri, 11 Apr 2014 15:00:25 -0700 (PDT) Received: by 10.14.213.194 with HTTP; Fri, 11 Apr 2014 15:00:24 -0700 (PDT) In-Reply-To: References: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> Date: Fri, 11 Apr 2014 22:00:24 +0000 Message-ID: Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 From: hiren panchasara To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 22:00:27 -0000 On Fri, Apr 11, 2014 at 4:16 PM, hiren panchasara wrote: > On Fri, Apr 11, 2014 at 4:15 AM, Eggert, Lars wrote: >> Hi, >> >> since folks are playing with Midori's DCTCP patch, I wanted to make sure that you were also aware of the patches that Aris did for PRR and NewCWV... > >>> >>> > > Lars, > > There are no actual patches attached here. (Or the mailing-list dropped them.) Ah, my bad. I think you are referring to the patches in original email. I can see them. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 00:23:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6839C9AA; Sat, 12 Apr 2014 00:23:14 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1839C10B8; Sat, 12 Apr 2014 00:23:14 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id hw13so5915331qab.17 for ; Fri, 11 Apr 2014 17:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aolNYDhR0bYYU4OdhozqDTB5II36pR7yiTLntVUrSck=; b=Z/bsZl2+JnyeI3w6fJIqWYgATnj7ArxgtSDO4m6Cbe9u2GXn572y5iQamlabo+JTHK Mg4cGxQ/cAGarL3lZbi4cyCP6Imxncc/v93VFUO+UDMiHBFCB/Z99G49QJ8iENuCuRP0 BqF4ebFn8xUIulX2Y3075pb0+sOsLcKJTBX72k3Z9sRCnbDjQAPLf1p3qRgpweKoKteA lGEAxU1ZJZD7sCc+T6fC+Qr7n9higZptsLQlxHpZsiMx93rmddWLTii/h2X9Mhg5q2lq IhK/dIevpA5tO3t8Vb/NrkDDNHigTiAJfAYNevczsPivKU2RJR/ecc8JTFtoSbCcSYxT a63g== MIME-Version: 1.0 X-Received: by 10.140.41.200 with SMTP id z66mr6364421qgz.102.1397262193302; Fri, 11 Apr 2014 17:23:13 -0700 (PDT) Received: by 10.140.101.151 with HTTP; Fri, 11 Apr 2014 17:23:13 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Apr 2014 17:23:13 -0700 Message-ID: Subject: Re: netisr observations From: hiren panchasara To: Patrick Kelsey Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 00:23:14 -0000 On Fri, Apr 11, 2014 at 11:30 AM, Patrick Kelsey wrote: > > The output of netstat -Q shows IP dispatch is set to default, which is > direct (NETISR_DISPATCH_DIRECT). That means each IP packet will be > processed on the same CPU that the Ethernet processing for that packet was > performed on, so CPU selection for IP packets will not be based on flowid. > The output of netstat -Q shows Ethernet dispatch is set to direct > (NETISR_DISPATCH_DIRECT if you wind up reading the code), so the Ethernet > processing for each packet will take place on the same CPU that the driver > receives that packet on. > > For the igb driver with queues autoconfigured and msix enabled, as the > sysctl output shows you have, the driver will create a number of queues > subject to device limitations, msix message limitations, and the number of > CPUs in the system, establish a separate interrupt handler for each one, and > bind each of those interrupt handlers to a separate CPU. It also creates a > separate single-threaded taskqueue for each queue. Each queue interrupt > handler sends work to its associated taskqueue when the interrupt fires. > Those taskqueues are where the Ethernet packets are received and processed > by the driver. The question is where those taskqueue threads will be run. > I don't see anything in the driver that makes an attempt to bind those > taskqueue threads to specific CPUs, so really the location of all of the > packet processing is up to the scheduler (i.e., arbitrary). > > The summary is: > > 1. the hardware schedules each received packet to one of its queues and > raises the interrupt for that queue > 2. that queue interrupt is serviced on the same CPU all the time, which is > different from the CPUs for all other queues on that interface > 3. the interrupt handler notifies the corresponding task queue, which runs > its task in a thread on whatever CPU the scheduler chooses > 4. that task dispatches the packet for Ethernet processing via netisr, which > processes it on whatever the current CPU is > 5. Ethernet processing dispatches that packet for IP processing via netisr, > which processes it on whatever the current CPU is I really appreciate you taking time and explaining this. Thank you. I am specially confused with ip "Queued" column from netstat -Q showing 203888563 only for cpu3. Does this mean that cpu3 queues everything and then distributes among other cpus? Where does this queuing on cpu3 happens out of 5 stages you mentioned above? This value gets populated in snwp->snw_queued field for each cpu inside sysctl_netisr_work(). > > You might want to try changing the default netisr dispatch policy to > 'deferred' (sysctl net.isr.dispatch=deferred). If you do that, the Ethernet > processing will still happen on an arbitrary CPU chosen by the scheduler, > but the IP processing should then get mapped to a CPU based on the flowid > assigned by the driver. Since igb assigns flowids based on received queue > number, all IP (and above) processing for that packet should then be > performed on the same CPU the queue interrupt was bound to. I will give this a try and see how things behave. I was also thinking about net.isr.bindthreads. netisr_start_swi() does intr_event_bind() if we have it bindthreads set to 1. What would that gain me, if anything? Would it stop moving intr{swi1: netisr 3} on to different cpus (as I am seeing in 'top' o/p) and bind it to a single cpu? I've came across a thread discussing some side-effects of this though: http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037597.html Thanks a ton, again. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 02:42:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0949792; Sat, 12 Apr 2014 02:42:29 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 462E11B92; Sat, 12 Apr 2014 02:42:28 +0000 (UTC) Received: by mail-la0-f46.google.com with SMTP id hr17so4119151lab.33 for ; Fri, 11 Apr 2014 19:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9nCmvqdJAz/XC3o2M7hgF+j1x2+7bnxzVIqkYm2F+sw=; b=rvNX5vVTmYS7KypskkQD7D2QlF3lL6AF1p79ZOOZRESy55olzfH8EhP2G15hnbZexq 5XJHlrjg+m7vuYQnfM7tqmykA+UMGcZcD6A4ACRJJ14SK+CFkLKUCPegV8YZxeY2KeZS SnSTevSonRuw/tRpEN/tR/GNCQiBpFPugn/tkDqoPW43lWCXoVNEsq5SsbjigTh+NknU XFq8h0sVHG/Pfh2+R9okO7KE9NUHWuweuawA8fb2RqheFqPcfVdrTdK7qAlR+QvY48hB CIeM32qN+INUQD+E6XjYI84QU+6gVqBciJHxER4sSkgZQNSnEXUA32hKq1j7axQsRbsa p8mw== MIME-Version: 1.0 X-Received: by 10.112.205.35 with SMTP id ld3mr18011437lbc.1.1397270546104; Fri, 11 Apr 2014 19:42:26 -0700 (PDT) Sender: pkelsey@gmail.com Received: by 10.112.141.196 with HTTP; Fri, 11 Apr 2014 19:42:26 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Apr 2014 22:42:26 -0400 X-Google-Sender-Auth: Ak4T-5LrPi8YahSa5YqSDXaYJdM Message-ID: Subject: Re: netisr observations From: Patrick Kelsey To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 02:42:29 -0000 On Fri, Apr 11, 2014 at 8:23 PM, hiren panchasara < hiren.panchasara@gmail.com> wrote: > On Fri, Apr 11, 2014 at 11:30 AM, Patrick Kelsey wrote: > > > > The output of netstat -Q shows IP dispatch is set to default, which is > > direct (NETISR_DISPATCH_DIRECT). That means each IP packet will be > > processed on the same CPU that the Ethernet processing for that packet > was > > performed on, so CPU selection for IP packets will not be based on > flowid. > > The output of netstat -Q shows Ethernet dispatch is set to direct > > (NETISR_DISPATCH_DIRECT if you wind up reading the code), so the Ethernet > > processing for each packet will take place on the same CPU that the > driver > > receives that packet on. > > > > For the igb driver with queues autoconfigured and msix enabled, as the > > sysctl output shows you have, the driver will create a number of queues > > subject to device limitations, msix message limitations, and the number > of > > CPUs in the system, establish a separate interrupt handler for each one, > and > > bind each of those interrupt handlers to a separate CPU. It also > creates a > > separate single-threaded taskqueue for each queue. Each queue interrupt > > handler sends work to its associated taskqueue when the interrupt fires. > > Those taskqueues are where the Ethernet packets are received and > processed > > by the driver. The question is where those taskqueue threads will be > run. > > I don't see anything in the driver that makes an attempt to bind those > > taskqueue threads to specific CPUs, so really the location of all of the > > packet processing is up to the scheduler (i.e., arbitrary). > > > > The summary is: > > > > 1. the hardware schedules each received packet to one of its queues and > > raises the interrupt for that queue > > 2. that queue interrupt is serviced on the same CPU all the time, which > is > > different from the CPUs for all other queues on that interface > > 3. the interrupt handler notifies the corresponding task queue, which > runs > > its task in a thread on whatever CPU the scheduler chooses > > 4. that task dispatches the packet for Ethernet processing via netisr, > which > > processes it on whatever the current CPU is > > 5. Ethernet processing dispatches that packet for IP processing via > netisr, > > which processes it on whatever the current CPU is > > I really appreciate you taking time and explaining this. Thank you. > Sure thing. I've had my head in the netisr code frequently lately, and it's nice to be able to share :) > > I am specially confused with ip "Queued" column from netstat -Q > showing 203888563 only for cpu3. Does this mean that cpu3 queues > everything and then distributes among other cpus? Where does this > queuing on cpu3 happens out of 5 stages you mentioned above? > > This value gets populated in snwp->snw_queued field for each cpu > inside sysctl_netisr_work(). > The way your system is configured, all inbound packets are being direct-dispatched. Those packets will bump the dispatched and handled counters, but not the queued counter. The queued counter only gets bumped when something is queued to a netisr thread. You can figure out where that is happening, despite everything apparently being configured for direct dispatch, by looking at where netisr_queue() and netisr_queue_src() are being called from. netisr_queue() is called during ipv6 forwarding and output and ipv4 output when the destination is a local address, gre processing, route socket processing, if_simloop() (which is called to loop back multicast packets, for example)... netisr_queue_src() is called during ipsec and divert processing. One thing to consider also when thinking about what the netisr per-cpu counters represent is that netisr really maintains per-cpu workstream context, not per-netisr-thread. Direct-dispatched packets contribute to the statistics of the workstream context of whichever CPU they are being direct-dispatched on. Packets handled by a netisr thread contribute to the statistics of the workstream context of the CPU it was created for, whether or not it was bound to, or is currently running on, that CPU. So when you look at the statistics in netstat -Q output for CPU 3, dispatched is the number of packets direct-dispatched on CPU 3, queued is the number of packets queued to the netisr thread associated with CPU 3 (but that may be running all over the place if net.isr.bindthreads is 0), and handled is the number of packets processed directly on CPU 3 or in the netisr thread associated with CPU3. > > > > > You might want to try changing the default netisr dispatch policy to > > 'deferred' (sysctl net.isr.dispatch=deferred). If you do that, the > Ethernet > > processing will still happen on an arbitrary CPU chosen by the scheduler, > > but the IP processing should then get mapped to a CPU based on the flowid > > assigned by the driver. Since igb assigns flowids based on received > queue > > number, all IP (and above) processing for that packet should then be > > performed on the same CPU the queue interrupt was bound to. > > I will give this a try and see how things behave. > > I was also thinking about net.isr.bindthreads. netisr_start_swi() does > intr_event_bind() if we have it bindthreads set to 1. What would that > gain me, if anything? > > That's a good point. If you move to deferred dispatch and bind the threads, then you keep the interrupt processing and IP-and-above protocol processing for packets from a given igb queue on the same CPU always. If you don't bind the netisr threads, then all IP-and-above protocol processing for packets from a given igb queue will always happen in the same netisr thread and you will get whatever locality benefits the scheduler manages to give you. I think the choice depends on what else you have going on in the system and what your priorities are. Binding the netisr threads will get you the best locality benefits for input packet processing, but might create hot-spot problems if you have other system activities you want bound to certain CPUs from an overlapping CPU set. Not binding the netisr threads probably gives up some locality benefits in packet processing, but the scheduler can move the network processing work away from other workloads you might have bound to some CPUs (and might care more about getting the locality benefit). > Would it stop moving intr{swi1: netisr 3} on to different cpus (as I > am seeing in 'top' o/p) and bind it to a single cpu? > > Yes it would. > I've came across a thread discussing some side-effects of this though: > http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037597.html > > Looks like the suggested fix was incorporated into the kernel about a month after that thread (so, 2 years ago) in r230984 ( http://svnweb.freebsd.org/base?view=revision&revision=230984). That's in 10-stable as well as -current. -Patrick From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 10:16:18 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21338209 for ; Sat, 12 Apr 2014 10:16:18 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9813C15C9 for ; Sat, 12 Apr 2014 10:16:17 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 5C52C7F540; Sat, 12 Apr 2014 14:16:13 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 5C52C7F540 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1397297773; bh=bPNwFzcmhaq8nqjaP9kJdfEyKJ/TLF1+HwjTw5Doc7s=; l=3196; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type; b=qyc2gDXQYcJB2D2yYM/deX/d/toUBJiCArnYngIq2LArat8Um8WmInwsXNPpDa+O/ 7seN+utsTKG87tfdNnebpHcjaOL2KjoYOGwi/q37TlmDEREZWeO/Q1W+LtgKSXi5pa X9Xy7sFNLyeXUH0uEJq5kGO/0Tbyl290ih5ayChw= Message-ID: <5349126E.2060209@oktetlabs.ru> Date: Sat, 12 Apr 2014 14:16:14 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 2/3] sfxge: TXQ index (not label) comes from FW in flush done event Content-Type: multipart/mixed; boundary="------------010101060407030209010801" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 10:16:18 -0000 This is a multi-part message in MIME format. --------------010101060407030209010801 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Change the second argument name of the efx_txq_flush_done_ev_t prototype to highlight that TXQ index (not label) comes from FW in flush done event. --------------010101060407030209010801 Content-Type: text/x-patch; name="2-sfxge-txq_index.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="2-sfxge-txq_index.patch" sfxge: TXQ index (not label) comes from FW in flush done event Change the second argument name of the efx_txq_flush_done_ev_t prototype to highlight that TXQ index (not label) comes from FW in flush done event. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r 74ea9e0f7842 -r 42f27b037ebb sys/dev/sfxge/common/efx.h --- a/sys/dev/sfxge/common/efx.h Thu Apr 10 14:23:34 2014 +0400 +++ b/sys/dev/sfxge/common/efx.h Thu Apr 10 14:23:36 2014 +0400 @@ -1389,7 +1389,7 @@ typedef __checkReturn boolean_t (*efx_txq_flush_done_ev_t)( __in_opt void *arg, - __in uint32_t label); + __in uint32_t txq_index); typedef __checkReturn boolean_t (*efx_software_ev_t)( diff -r 74ea9e0f7842 -r 42f27b037ebb sys/dev/sfxge/common/efx_ev.c --- a/sys/dev/sfxge/common/efx_ev.c Thu Apr 10 14:23:34 2014 +0400 +++ b/sys/dev/sfxge/common/efx_ev.c Thu Apr 10 14:23:36 2014 +0400 @@ -406,16 +406,16 @@ switch (EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_SUBCODE)) { case FSE_AZ_TX_DESCQ_FLS_DONE_EV: { - uint32_t label; + uint32_t txq_index; EFX_EV_QSTAT_INCR(eep, EV_DRIVER_TX_DESCQ_FLS_DONE); - label = EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_SUBDATA); + txq_index = EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_SUBDATA); - EFSYS_PROBE1(tx_descq_fls_done, uint32_t, label); + EFSYS_PROBE1(tx_descq_fls_done, uint32_t, txq_index); EFSYS_ASSERT(eecp->eec_txq_flush_done != NULL); - should_abort = eecp->eec_txq_flush_done(arg, label); + should_abort = eecp->eec_txq_flush_done(arg, txq_index); break; } diff -r 74ea9e0f7842 -r 42f27b037ebb sys/dev/sfxge/sfxge_ev.c --- a/sys/dev/sfxge/sfxge_ev.c Thu Apr 10 14:23:34 2014 +0400 +++ b/sys/dev/sfxge/sfxge_ev.c Thu Apr 10 14:23:36 2014 +0400 @@ -260,16 +260,17 @@ } static boolean_t -sfxge_ev_txq_flush_done(void *arg, uint32_t label) +sfxge_ev_txq_flush_done(void *arg, uint32_t txq_index) { struct sfxge_evq *evq; struct sfxge_softc *sc; struct sfxge_txq *txq; + unsigned int label; uint16_t magic; evq = (struct sfxge_evq *)arg; sc = evq->sc; - txq = sc->txq[label]; + txq = sc->txq[txq_index]; KASSERT(txq != NULL, ("txq == NULL")); KASSERT(txq->init_state == SFXGE_TXQ_INITIALIZED, @@ -278,6 +279,7 @@ /* Resend a software event on the correct queue */ evq = sc->evq[txq->evq_index]; + label = txq_index; KASSERT((label & SFXGE_MAGIC_DMAQ_LABEL_MASK) == label, ("(label & SFXGE_MAGIC_DMAQ_LABEL_MASK) != label")); magic = SFXGE_MAGIC_TX_QFLUSH_DONE | label; --------------010101060407030209010801-- From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 10:17:27 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EADD52AC for ; Sat, 12 Apr 2014 10:17:26 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70DDE15DC for ; Sat, 12 Apr 2014 10:17:26 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 8337B7F4CE; Sat, 12 Apr 2014 14:17:23 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 8337B7F4CE DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1397297843; bh=Rr6BZuiS34ocgxKhDFexEP8HOar2XjzvwJWUW/ZL0xY=; l=3662; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type; b=rk9xAtOcRInLiwwhXj9OmE0eQjyIHmAjl6sMsfkAStNwE3w6IQscgQ7B0FjXLNEyL szgO/bZf+eCafyt1a0OEx1x0tErMMlyk1eke4JaCtw0nA3j2rdKQigZok7vsnET+Wh raC9x7yXZHs9uCzPkhOtj1XUGIJeACYmVGtAVZp4= Message-ID: <534912B4.106@oktetlabs.ru> Date: Sat, 12 Apr 2014 14:17:24 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 3/3] sfxge: use TXQ type as label to support more than 32 TXQs Content-Type: multipart/mixed; boundary="------------090407050909080702060908" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 10:17:27 -0000 This is a multi-part message in MIME format. --------------090407050909080702060908 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit There are 3 TXQs in event queue 0 and 1 TXQ (with TCP/UDP checksum offload) in all other event queues. --------------090407050909080702060908 Content-Type: text/x-patch; name="3-sfxge-txq_type_as_label.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="3-sfxge-txq_type_as_label.patch" sfxge: use TXQ type as label to support more than 32 TXQs There are 3 TXQs in event queue 0 and 1 TXQ (with TCP/UDP checksum offload) in all other event queues. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r 42f27b037ebb -r ab60166f0df9 sys/dev/sfxge/sfxge_ev.c --- a/sys/dev/sfxge/sfxge_ev.c Thu Apr 10 14:23:36 2014 +0400 +++ b/sys/dev/sfxge/sfxge_ev.c Fri Apr 11 15:40:47 2014 +0400 @@ -218,18 +218,27 @@ return (B_FALSE); } +static struct sfxge_txq * +sfxge_get_txq_by_label(struct sfxge_evq *evq, enum sfxge_txq_type label) +{ + unsigned int index; + + KASSERT((evq->index == 0 && label < SFXGE_TXQ_NTYPES) || + (label == SFXGE_TXQ_IP_TCP_UDP_CKSUM), ("unexpected txq label")); + index = (evq->index == 0) ? label : (evq->index - 1 + SFXGE_TXQ_NTYPES); + return evq->sc->txq[index]; +} + static boolean_t sfxge_ev_tx(void *arg, uint32_t label, uint32_t id) { struct sfxge_evq *evq; - struct sfxge_softc *sc; struct sfxge_txq *txq; unsigned int stop; unsigned int delta; evq = (struct sfxge_evq *)arg; - sc = evq->sc; - txq = sc->txq[label]; + txq = sfxge_get_txq_by_label(evq, label); KASSERT(txq != NULL, ("txq == NULL")); KASSERT(evq->index == txq->evq_index, @@ -279,7 +288,7 @@ /* Resend a software event on the correct queue */ evq = sc->evq[txq->evq_index]; - label = txq_index; + label = txq->type; KASSERT((label & SFXGE_MAGIC_DMAQ_LABEL_MASK) == label, ("(label & SFXGE_MAGIC_DMAQ_LABEL_MASK) != label")); magic = SFXGE_MAGIC_TX_QFLUSH_DONE | label; @@ -336,7 +345,7 @@ break; } case SFXGE_MAGIC_TX_QFLUSH_DONE: { - struct sfxge_txq *txq = sc->txq[label]; + struct sfxge_txq *txq = sfxge_get_txq_by_label(evq, label); KASSERT(txq != NULL, ("txq == NULL")); KASSERT(evq->index == txq->evq_index, diff -r 42f27b037ebb -r ab60166f0df9 sys/dev/sfxge/sfxge_tx.c --- a/sys/dev/sfxge/sfxge_tx.c Thu Apr 10 14:23:36 2014 +0400 +++ b/sys/dev/sfxge/sfxge_tx.c Fri Apr 11 15:40:47 2014 +0400 @@ -27,6 +27,21 @@ * SUCH DAMAGE. */ +/* Theory of operation: + * + * Tx queues allocation and mapping + * + * One Tx queue with enabled checksum offload is allocated per Rx channel + * (event queue). Also 2 Tx queues (one without checksum offload and one + * with IP checksum offload only) are allocated and bound to event queue 0. + * sfxge_txq_type is used as Tx queue label. + * + * So, event queue plus label mapping to Tx queue index is: + * if event queue index is 0, TxQ-index = TxQ-label * [0..SFXGE_TXQ_NTYPES) + * else TxQ-index = SFXGE_TXQ_NTYPES + EvQ-index - 1 + * See sfxge_get_txq_by_label() sfxge_ev.c + */ + #include __FBSDID("$FreeBSD$"); @@ -1179,7 +1194,7 @@ } /* Create the common code transmit queue. */ - if ((rc = efx_tx_qcreate(sc->enp, index, index, esmp, + if ((rc = efx_tx_qcreate(sc->enp, index, txq->type, esmp, SFXGE_NDESCS, txq->buf_base_id, flags, evq->common, &txq->common)) != 0) goto fail; --------------090407050909080702060908-- From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 10:19:39 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CAE1352 for ; Sat, 12 Apr 2014 10:19:39 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [188.134.15.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB98915EA for ; Sat, 12 Apr 2014 10:19:38 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 987B87F4E9; Sat, 12 Apr 2014 14:12:10 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 987B87F4E9 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1397297530; bh=SBhNDfENeMCi4+pZ01dQGibWbBj6qXSU41UJKgp4k+U=; l=4720; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type; b=Vtpm03xLbfXnpCU7S/6sHZOhvUmokmVv4QyFceMrCz14TbBjX/omwonm+xustvfIn Fdv0Cr8sruPZG/ac/kzyLVIoYDZcx6GtKwbiw4vwSR/KGQTswIBzTbD4DGbbaIuBOK waWFt5AN6m575A9l4+Bu5mcMOsD4VFWSfFEWb6T8= Message-ID: <5349117B.4050505@oktetlabs.ru> Date: Sat, 12 Apr 2014 14:12:11 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: net@FreeBSD.org Subject: [PATCH 1/3] sfxge: RXQ index (not label) comes from FW in flush done/failed events Content-Type: multipart/mixed; boundary="------------080206030102090207050106" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 10:19:39 -0000 This is a multi-part message in MIME format. --------------080206030102090207050106 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit sfxge: RXQ index (not label) comes from FW in flush done/failed events Change the second argument name of the efx_rxq_flush_done_ev_t and efx_rxq_flush_failed_ev_t prototypes to highlight that RXQ index (not label) comes from FW in flush done and failed events. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. --------------080206030102090207050106 Content-Type: text/x-patch; name="1-sfxge-rxq_index.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="1-sfxge-rxq_index.patch" sfxge: RXQ index (not label) comes from FW in flush done/failed events Change the second argument name of the efx_rxq_flush_done_ev_t and efx_rxq_flush_failed_ev_t prototypes to highlight that RXQ index (not label) comes from FW in flush done and failed events. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r 303a89ceaefe -r 74ea9e0f7842 sys/dev/sfxge/common/efx.h --- a/sys/dev/sfxge/common/efx.h Wed Mar 19 14:41:05 2014 +0000 +++ b/sys/dev/sfxge/common/efx.h Thu Apr 10 14:23:34 2014 +0400 @@ -1379,12 +1379,12 @@ typedef __checkReturn boolean_t (*efx_rxq_flush_done_ev_t)( __in_opt void *arg, - __in uint32_t label); + __in uint32_t rxq_index); typedef __checkReturn boolean_t (*efx_rxq_flush_failed_ev_t)( __in_opt void *arg, - __in uint32_t label); + __in uint32_t rxq_index); typedef __checkReturn boolean_t (*efx_txq_flush_done_ev_t)( diff -r 303a89ceaefe -r 74ea9e0f7842 sys/dev/sfxge/common/efx_ev.c --- a/sys/dev/sfxge/common/efx_ev.c Wed Mar 19 14:41:05 2014 +0000 +++ b/sys/dev/sfxge/common/efx_ev.c Thu Apr 10 14:23:34 2014 +0400 @@ -420,10 +420,10 @@ break; } case FSE_AZ_RX_DESCQ_FLS_DONE_EV: { - uint32_t label; + uint32_t rxq_index; uint32_t failed; - label = EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_RX_DESCQ_ID); + rxq_index = EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_RX_DESCQ_ID); failed = EFX_QWORD_FIELD(*eqp, FSF_AZ_DRIVER_EV_RX_FLUSH_FAIL); EFSYS_ASSERT(eecp->eec_rxq_flush_done != NULL); @@ -432,15 +432,15 @@ if (failed) { EFX_EV_QSTAT_INCR(eep, EV_DRIVER_RX_DESCQ_FLS_FAILED); - EFSYS_PROBE1(rx_descq_fls_failed, uint32_t, label); + EFSYS_PROBE1(rx_descq_fls_failed, uint32_t, rxq_index); - should_abort = eecp->eec_rxq_flush_failed(arg, label); + should_abort = eecp->eec_rxq_flush_failed(arg, rxq_index); } else { EFX_EV_QSTAT_INCR(eep, EV_DRIVER_RX_DESCQ_FLS_DONE); - EFSYS_PROBE1(rx_descq_fls_done, uint32_t, label); + EFSYS_PROBE1(rx_descq_fls_done, uint32_t, rxq_index); - should_abort = eecp->eec_rxq_flush_done(arg, label); + should_abort = eecp->eec_rxq_flush_done(arg, rxq_index); } break; diff -r 303a89ceaefe -r 74ea9e0f7842 sys/dev/sfxge/sfxge_ev.c --- a/sys/dev/sfxge/sfxge_ev.c Wed Mar 19 14:41:05 2014 +0000 +++ b/sys/dev/sfxge/sfxge_ev.c Thu Apr 10 14:23:34 2014 +0400 @@ -155,17 +155,18 @@ } static boolean_t -sfxge_ev_rxq_flush_done(void *arg, uint32_t label) +sfxge_ev_rxq_flush_done(void *arg, uint32_t rxq_index) { struct sfxge_evq *evq; struct sfxge_softc *sc; struct sfxge_rxq *rxq; unsigned int index; + unsigned int label; uint16_t magic; evq = (struct sfxge_evq *)arg; sc = evq->sc; - rxq = sc->rxq[label]; + rxq = sc->rxq[rxq_index]; KASSERT(rxq != NULL, ("rxq == NULL")); @@ -173,6 +174,7 @@ index = rxq->index; evq = sc->evq[index]; + label = rxq_index; KASSERT((label & SFXGE_MAGIC_DMAQ_LABEL_MASK) == label, ("(label & SFXGE_MAGIC_DMAQ_LABEL_MASK) != level")); magic = SFXGE_MAGIC_RX_QFLUSH_DONE | label; @@ -185,17 +187,18 @@ } static boolean_t -sfxge_ev_rxq_flush_failed(void *arg, uint32_t label) +sfxge_ev_rxq_flush_failed(void *arg, uint32_t rxq_index) { struct sfxge_evq *evq; struct sfxge_softc *sc; struct sfxge_rxq *rxq; unsigned int index; + unsigned int label; uint16_t magic; evq = (struct sfxge_evq *)arg; sc = evq->sc; - rxq = sc->rxq[label]; + rxq = sc->rxq[rxq_index]; KASSERT(rxq != NULL, ("rxq == NULL")); @@ -203,6 +206,7 @@ index = rxq->index; evq = sc->evq[index]; + label = rxq_index; KASSERT((label & SFXGE_MAGIC_DMAQ_LABEL_MASK) == label, ("(label & SFXGE_MAGIC_DMAQ_LABEL_MASK) != label")); magic = SFXGE_MAGIC_RX_QFLUSH_FAILED | label; --------------080206030102090207050106-- From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 10:58:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58D27B97 for ; Sat, 12 Apr 2014 10:58:04 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1384218EA for ; Sat, 12 Apr 2014 10:58:03 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WYvdD-00015W-7k for freebsd-net@freebsd.org; Sat, 12 Apr 2014 12:57:55 +0200 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 12 Apr 2014 12:57:55 +0200 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 12 Apr 2014 12:57:55 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: Re: Multihomed system with jails routing issues Date: Sat, 12 Apr 2014 03:57:42 -0700 Lines: 26 Message-ID: References: <533F68EF.8060607@nevermind.co.nz> <53402D68.4030500@freebsd.org> <53411885.7030206@nevermind.co.nz> <53415866.1030107@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Thunderbird/28.0 In-Reply-To: <53415866.1030107@freebsd.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 10:58:04 -0000 On 4/6/2014 6:36 AM, Julian Elischer wrote: > On 4/6/14, 5:04 PM, Chris Smith wrote: >> On 06/04/14 04:20, Julian Elischer wrote: >> Hey Julian, >> >> Thanks for that. I did come across it but all of the documentation I >> found indicated that it was experimental. >> >> After a day or so messing around with VIMAGE/vnet and their various >> gotchas and interactions with jails on FreeBSD 10, I have something >> working that I'm happy with. > > as long as you steer clear of pf and do only 'vanilla' stuff, you should > be ok. > let us know what you think and I'd like to see your notes published, if > not officially then at least put here so that others can find it in the > archives. There have been long standing memory leaks in stopping VNET jails. For instance kern/164763. Is there anyone looking into this? Is there any will to enable VNET by default in -CURRENT? Regards, Kevin From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 16:05:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC11732D for ; Sat, 12 Apr 2014 16:05:43 +0000 (UTC) Received: from mail-yk0-x234.google.com (mail-yk0-x234.google.com [IPv6:2607:f8b0:4002:c07::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 805C015B1 for ; Sat, 12 Apr 2014 16:05:43 +0000 (UTC) Received: by mail-yk0-f180.google.com with SMTP id 19so5958432ykq.25 for ; Sat, 12 Apr 2014 09:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5YjAZYd7+Ogjz0hUeMz7JqCGYnhLW6NilBGqumeCXpM=; b=tMCZ30Bq0ac+KfHjRMaz99DxbF3uHITGhdyYQwVroKcQQ5u2pWI92E0uSXPCZS/fh3 WseZPJfLnQ4kIF0dGM77KchzEPZcTkmeeokpaxZbTJJ79ka9fLVHe390ulX5Zrvv7HoO 6ZDJbxrEyP56/23569M7zTapW9CVkG5eNhBFGhgwW/KFfbJ8KIO5y+3+ankefwkIAEbK 41eBpdaPGYkx61+aqZ5/SpojXOaRcxNl4R1KEiFzfY0ajPMKqWXWBzcGD9MrvuLCHBvE eJCjkHm1O/xR1SSAdMD0fiOkVtPWqAwTeCADoAWLwxaJBI8ZZXTFDh0h08Ah6ZEOdqcr oyCg== MIME-Version: 1.0 X-Received: by 10.236.113.69 with SMTP id z45mr42468316yhg.0.1397318742702; Sat, 12 Apr 2014 09:05:42 -0700 (PDT) Received: by 10.170.217.67 with HTTP; Sat, 12 Apr 2014 09:05:42 -0700 (PDT) Date: Sat, 12 Apr 2014 19:05:42 +0300 Message-ID: Subject: [patch] ifconfig -L shows inet6 addresses pltime and vltime as zero since 10.0-RELEASE From: Guy Yur To: freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary=20cf3010eb114c0f2a04f6da9faa X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 16:05:43 -0000 --20cf3010eb114c0f2a04f6da9faa Content-Type: text/plain; charset=UTF-8 Hi, "ifconfig -L" shows inet6 addresses pltime and vltime as zero since 10.0-RELEASE. Seems that ifconfig was missed in r253970 which changed usage from time_second to time_uptime. I filed bin/188520 with a patch to use clock_gettime(CLOCK_MONOTONIC_FAST) in ifconfig per r253970 comment. http://www.freebsd.org/cgi/query-pr.cgi?pr=188520 ifconfig -L ... inet6 XXXX:XXXX:XXXX:XXXX:YYYY:YYYY:YYYY:YYYY prefixlen 64 autoconf temporary pltime 0 vltime 0 The prefix is learned via router advertisment. tcpdump shows non zero values in the icmp6 packet. A test program that calls SIOCGIFALIFETIME_IN6 shows the value is < wall clock which is the test done by ifconfig. time = 1397314436 ia6t_preferred = 608907 ia6t_expire = 2596107 Thanks, Guy --20cf3010eb114c0f2a04f6da9faa Content-Type: application/octet-stream; name="ifconfig_af_inet6_monotonic.patch" Content-Disposition: attachment; filename="ifconfig_af_inet6_monotonic.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_htx3ba0p0 SW5kZXg6IHNiaW4vaWZjb25maWcvYWZfaW5ldDYuYwo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzYmluL2lmY29u ZmlnL2FmX2luZXQ2LmMJKHJldmlzaW9uIDI2NDM2NikKKysrIHNiaW4vaWZjb25maWcvYWZfaW5l dDYuYwkod29ya2luZyBjb3B5KQpAQCAtNDIsNiArNDIsNyBAQCBzdGF0aWMgY29uc3QgY2hhciBy Y3NpZFtdID0KICNpbmNsdWRlIDxzdGRsaWIuaD4KICNpbmNsdWRlIDxzdHJpbmcuaD4KICNpbmNs dWRlIDx1bmlzdGQuaD4KKyNpbmNsdWRlIDx0aW1lLmg+CiAjaW5jbHVkZSA8aWZhZGRycy5oPgog CiAjaW5jbHVkZSA8YXJwYS9pbmV0Lmg+CkBAIC05OCwxMCArOTksMTEgQEAgc3RhdGljIHZvaWQK IHNldGlwNmxpZmV0aW1lKGNvbnN0IGNoYXIgKmNtZCwgY29uc3QgY2hhciAqdmFsLCBpbnQgcywg CiAgICAgY29uc3Qgc3RydWN0IGFmc3d0Y2ggKmFmcCkKIHsKLQl0aW1lX3QgbmV3dmFsLCB0Owor CXN0cnVjdCB0aW1lc3BlYyBub3c7CisJdGltZV90IG5ld3ZhbDsKIAljaGFyICplcDsKIAotCXQg PSB0aW1lKE5VTEwpOworCWNsb2NrX2dldHRpbWUoQ0xPQ0tfTU9OT1RPTklDX0ZBU1QsICZub3cp OwogCW5ld3ZhbCA9ICh0aW1lX3Qpc3RydG91bCh2YWwsICZlcCwgMCk7CiAJaWYgKHZhbCA9PSBl cCkKIAkJZXJyeCgxLCAiaW52YWxpZCAlcyIsIGNtZCk7CkBAIC0xMDgsMTAgKzExMCwxMCBAQCBz ZXRpcDZsaWZldGltZShjb25zdCBjaGFyICpjbWQsIGNvbnN0IGNoYXIgKnZhbCwgaQogCWlmIChh ZnAtPmFmX2FmICE9IEFGX0lORVQ2KQogCQllcnJ4KDEsICIlcyBub3QgYWxsb3dlZCBmb3IgdGhl IEFGIiwgY21kKTsKIAlpZiAoc3RyY21wKGNtZCwgInZsdGltZSIpID09IDApIHsKLQkJaW42X2Fk ZHJlcS5pZnJhX2xpZmV0aW1lLmlhNnRfZXhwaXJlID0gdCArIG5ld3ZhbDsKKwkJaW42X2FkZHJl cS5pZnJhX2xpZmV0aW1lLmlhNnRfZXhwaXJlID0gbm93LnR2X3NlYyArIG5ld3ZhbDsKIAkJaW42 X2FkZHJlcS5pZnJhX2xpZmV0aW1lLmlhNnRfdmx0aW1lID0gbmV3dmFsOwogCX0gZWxzZSBpZiAo c3RyY21wKGNtZCwgInBsdGltZSIpID09IDApIHsKLQkJaW42X2FkZHJlcS5pZnJhX2xpZmV0aW1l LmlhNnRfcHJlZmVycmVkID0gdCArIG5ld3ZhbDsKKwkJaW42X2FkZHJlcS5pZnJhX2xpZmV0aW1l LmlhNnRfcHJlZmVycmVkID0gbm93LnR2X3NlYyArIG5ld3ZhbDsKIAkJaW42X2FkZHJlcS5pZnJh X2xpZmV0aW1lLmlhNnRfcGx0aW1lID0gbmV3dmFsOwogCX0KIH0KQEAgLTE3Miw5ICsxNzQsMTEg QEAgaW42X3N0YXR1cyhpbnQgcyBfX3VudXNlZCwgY29uc3Qgc3RydWN0IGlmYWRkcnMgKmkKIAlp bnQgczY7CiAJdV9pbnQzMl90IGZsYWdzNjsKIAlzdHJ1Y3QgaW42X2FkZHJsaWZldGltZSBsaWZl dGltZTsKLQl0aW1lX3QgdCA9IHRpbWUoTlVMTCk7CisJc3RydWN0IHRpbWVzcGVjIG5vdzsKIAlp bnQgZXJyb3I7CiAKKwljbG9ja19nZXR0aW1lKENMT0NLX01PTk9UT05JQ19GQVNULCAmbm93KTsK KwogCW1lbXNldCgmbnVsbF9zaW4sIDAsIHNpemVvZihudWxsX3NpbikpOwogCiAJc2luID0gKHN0 cnVjdCBzb2NrYWRkcl9pbjYgKilpZmEtPmlmYV9hZGRyOwpAQCAtMjU4LDE1ICsyNjIsMTUgQEAg aW42X3N0YXR1cyhpbnQgcyBfX3VudXNlZCwgY29uc3Qgc3RydWN0IGlmYWRkcnMgKmkKIAlpZiAo aXA2bGlmZXRpbWUgJiYgKGxpZmV0aW1lLmlhNnRfcHJlZmVycmVkIHx8IGxpZmV0aW1lLmlhNnRf ZXhwaXJlKSkgewogCQlwcmludGYoInBsdGltZSAiKTsKIAkJaWYgKGxpZmV0aW1lLmlhNnRfcHJl ZmVycmVkKSB7Ci0JCQlwcmludGYoIiVzICIsIGxpZmV0aW1lLmlhNnRfcHJlZmVycmVkIDwgdAot CQkJCT8gIjAiIDogc2VjMnN0cihsaWZldGltZS5pYTZ0X3ByZWZlcnJlZCAtIHQpKTsKKwkJCXBy aW50ZigiJXMgIiwgbGlmZXRpbWUuaWE2dF9wcmVmZXJyZWQgPCBub3cudHZfc2VjCisJCQkJPyAi MCIgOiBzZWMyc3RyKGxpZmV0aW1lLmlhNnRfcHJlZmVycmVkIC0gbm93LnR2X3NlYykpOwogCQl9 IGVsc2UKIAkJCXByaW50ZigiaW5mdHkgIik7CiAKIAkJcHJpbnRmKCJ2bHRpbWUgIik7CiAJCWlm IChsaWZldGltZS5pYTZ0X2V4cGlyZSkgewotCQkJcHJpbnRmKCIlcyAiLCBsaWZldGltZS5pYTZ0 X2V4cGlyZSA8IHQKLQkJCQk/ICIwIiA6IHNlYzJzdHIobGlmZXRpbWUuaWE2dF9leHBpcmUgLSB0 KSk7CisJCQlwcmludGYoIiVzICIsIGxpZmV0aW1lLmlhNnRfZXhwaXJlIDwgbm93LnR2X3NlYwor CQkJCT8gIjAiIDogc2VjMnN0cihsaWZldGltZS5pYTZ0X2V4cGlyZSAtIG5vdy50dl9zZWMpKTsK IAkJfSBlbHNlCiAJCQlwcmludGYoImluZnR5ICIpOwogCX0K --20cf3010eb114c0f2a04f6da9faa-- From owner-freebsd-net@FreeBSD.ORG Sat Apr 12 23:39:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E75D63EA for ; Sat, 12 Apr 2014 23:39:11 +0000 (UTC) Received: from mail-oa0-x231.google.com (mail-oa0-x231.google.com [IPv6:2607:f8b0:4003:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1EF71AE9 for ; Sat, 12 Apr 2014 23:39:11 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id o6so7754638oag.36 for ; Sat, 12 Apr 2014 16:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=mtbovXn2K123zUJlfwuDUNaWCUumZc5WjFOJ0/oYohU=; b=EOGUGv2ZWoLORY4UdzLGYak8O0vRlpZEMMfbsA8VJU10yP9X6rZoygvNZyifH39dXt /5tZjPWLXOr/5qTLLNwFX0efNGpY5vHo+Z+pC1J+CxoPQB7DofFZM0ugY4wVWsUx2aOO E8lchzNUOBRK9Yu0pBNWyZjAdmYi3/FiHWfUwi8lPlSiAnPEufDZyEnfVwu9gB3mtTjU WveGEZvT4/YqNA/VDhF/4GdPtQOMkKtXjCdH5YTxF7HbdevSPBeZ9AiLnwsKbkbEBzvw pl+fRpXVIAHG45LfFT0U+CCJhApSTa1V+xexBlfRMfqP5Frhr92JAvlDhO0FbOv83vpo 5qTw== X-Received: by 10.182.125.161 with SMTP id mr1mr119944obb.47.1397345951047; Sat, 12 Apr 2014 16:39:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.232.72 with HTTP; Sat, 12 Apr 2014 16:38:50 -0700 (PDT) In-Reply-To: <5347AEAA.9090801@smartspb.net> References: <5347AEAA.9090801@smartspb.net> From: Raimundo Santos Date: Sat, 12 Apr 2014 20:38:50 -0300 Message-ID: Subject: Re: dummynet/ipfw high load? To: Dennis Yusupoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 23:39:12 -0000 On 11 April 2014 05:58, Dennis Yusupoff wrote: > NAT realized with pf nat, shaping with ipfw dummynet and traffic > accounting with ng_netflow via ipfw ng_tee. > Good time, Dennis. May I ask how much clients do you nat, shape and account? Why you do that with both engines (pf + ipfw)? Why not ipfw only with in-kernel NAT? Best regards, Raimundo From owner-freebsd-net@FreeBSD.ORG Mon Apr 14 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27F7AE8 for ; Mon, 14 Apr 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14B791664 for ; Mon, 14 Apr 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3EB6nI9025950 for ; Mon, 14 Apr 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3EB6mbk025948 for freebsd-net@FreeBSD.org; Mon, 14 Apr 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 14 Apr 2014 11:06:48 GMT Message-Id: <201404141106.s3EB6mbk025948@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/188245 net [vlan] Critical vlan problem with OpenBGP o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net Server hangs if -arp is present on an IP address o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] freebsd 8.4, 9.1 and 9.2 panic Double-Fault with o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 461 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 14 21:35:44 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B29B45C for ; Mon, 14 Apr 2014 21:35:44 +0000 (UTC) Received: from ihost.wananchi.com (ihost.wananchi.com [62.8.88.47]) by mx1.freebsd.org (Postfix) with ESMTP id EE7D71FD2 for ; Mon, 14 Apr 2014 21:35:43 +0000 (UTC) Received: from chris-PC (unknown [197.237.193.240]) by ihost.wananchi.com (Postfix) with ESMTP id 8299BA4CE2 for ; Tue, 15 Apr 2014 00:23:21 +0300 (EAT) Message-ID: <02c9758a-41743-802a5997128472@chris-pc> From: "Email list" To: net@freebsd.org Subject: EMAIL LIST KENYA Date: Mon, 14 Apr 2014 14:23:38 -0700 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Priority: 3 X-Mailer: Tuggex X-MimeOLE: Kinetic.4.6 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 21:35:44 -0000 Do you need to market your products or services in the most cost effective and efficient way? We have over 450,000 Valid Kenyan Email addresses and a Mass E-mailing software that can send up to 1000 emails per minute. With a one off cost of Ksh 35,000 you can acquire all this and increase your client base. For more information on how to get this please call 0724008640 From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 04:15:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21DFB7AD for ; Tue, 15 Apr 2014 04:15:02 +0000 (UTC) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2001:470:1f09:14c0::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Norma UNIX CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 689CC1451 for ; Tue, 15 Apr 2014 04:15:00 +0000 (UTC) Received: from bsdrookie.norma.com. (bsdrookie.norma.com [192.168.7.224]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id s3F4Esj3091468 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 15 Apr 2014 10:14:55 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <534CB23E.8060304@norma.perm.ru> Date: Tue, 15 Apr 2014 10:14:54 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Racoon/IPSEC Tunnel in 9.2 vs 10.0 References: <5345AA7C.3050700@soliddataservices.com> In-Reply-To: <5345AA7C.3050700@soliddataservices.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [192.168.3.10]); Tue, 15 Apr 2014 10:14:55 +0600 (YEKT) X-Spam-Status: No hits=-101.0 bayes=0.5 testhits ALL_TRUSTED=-1, USER_IN_WHITELIST=-100 autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 04:15:02 -0000 Hi. On 10.04.2014 02:15, Matt Lager wrote: > I have used IPSEC tunnels w/ racoon to establish point to point VPN > connections for a long time, with great success. I recently decided to > upgrade one of my endpoints to 10.0-RELEASE from 9.2-RELEASE-p3. I > didn't do an upgrade but did a fresh installation of 10.0-RELEASE, but > applied the identical VPN configuration that was working in > 9.2-RELEASE-p3. The tunnels came up fine, and setkey -D shows that > keys had been generated, connectivity appeared to be working at first > glance. I then started to work as normal through my VPN with things > like RDP, SQL Server, and other protocols, where I found that > connectivity started then came to a dead halt (not ICMP, which always > works fine). I did another fresh install of 9.2-RELEASE-p3, applied > the config, and everything worked as expected. > > I've read a lot about MTU's and fragmented traffic, but I'm trying to > figure out where I should be looking to fix things up. Something > obviously changed. I do use PF, and I know PF underwent some big > changes, so maybe it's a PF problem, but I thought I'd post here > first. I'm using the same PF config on the 10.0 system as I did on the > 9.2, of course making sure interfaces were all named properly and > whatnot. > > Any advice would be appreciated. Thanks! > I'm using FreeBSD on a variety of VPN/ipsec links. Nothing really changed in 10.x. In fact, I've skipped the 9.x branch entirely, because it has been worst release over many years. You should really investigate the problem, since it looks like it has nothing to do with the versining. As a wild guess I can assume you have FLOWTABLE in your kernel; if I'm right you should get rid of it. Eugene. From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 04:25:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7CA8D8A3 for ; Tue, 15 Apr 2014 04:25:59 +0000 (UTC) Received: from mx1.rpsol.net (mx1.rpsol.net [74.206.97.74]) by mx1.freebsd.org (Postfix) with ESMTP id 596DB151E for ; Tue, 15 Apr 2014 04:25:58 +0000 (UTC) Received: from [172.16.1.100] (wsip-72-215-202-18.ph.ph.cox.net [72.215.202.18]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.rpsol.net (Postfix) with ESMTPSA id A121CFFEA07 for ; Mon, 14 Apr 2014 21:25:40 -0700 (MST) Message-ID: <534CB52C.60203@soliddataservices.com> Date: Mon, 14 Apr 2014 21:27:24 -0700 From: Matt Lager User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Racoon/IPSEC Tunnel in 9.2 vs 10.0 References: <5345AA7C.3050700@soliddataservices.com> <534CB23E.8060304@norma.perm.ru> In-Reply-To: <534CB23E.8060304@norma.perm.ru> X-RPS-MailScanner-Information: Please contact the ISP for more information X-RPS-MailScanner-ID: A121CFFEA07.A0604 X-RPS-MailScanner: Found to be clean X-RPS-MailScanner-From: matt@soliddataservices.com X-Spam-Status: No Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 04:25:59 -0000 Do you utilize PF as your firewalling platform, because I'm slightly suspicious that could be the cause. In fact, it's a bit silly I haven't disabled it entirely to see if that resolves my issues. I'll probably run tests doing that tomorrow and will report back. PF did undergo a lot of changes as I understand it between 9 and 10. On 4/14/2014 9:14 PM, Eugene M. Zheganin wrote: > Hi. > > On 10.04.2014 02:15, Matt Lager wrote: >> I have used IPSEC tunnels w/ racoon to establish point to point VPN >> connections for a long time, with great success. I recently decided to >> upgrade one of my endpoints to 10.0-RELEASE from 9.2-RELEASE-p3. I >> didn't do an upgrade but did a fresh installation of 10.0-RELEASE, but >> applied the identical VPN configuration that was working in >> 9.2-RELEASE-p3. The tunnels came up fine, and setkey -D shows that >> keys had been generated, connectivity appeared to be working at first >> glance. I then started to work as normal through my VPN with things >> like RDP, SQL Server, and other protocols, where I found that >> connectivity started then came to a dead halt (not ICMP, which always >> works fine). I did another fresh install of 9.2-RELEASE-p3, applied >> the config, and everything worked as expected. >> >> I've read a lot about MTU's and fragmented traffic, but I'm trying to >> figure out where I should be looking to fix things up. Something >> obviously changed. I do use PF, and I know PF underwent some big >> changes, so maybe it's a PF problem, but I thought I'd post here >> first. I'm using the same PF config on the 10.0 system as I did on the >> 9.2, of course making sure interfaces were all named properly and >> whatnot. >> >> Any advice would be appreciated. Thanks! >> > I'm using FreeBSD on a variety of VPN/ipsec links. Nothing really > changed in 10.x. In fact, I've skipped the 9.x branch entirely, because > it has been worst release over many years. You should really investigate > the problem, since it looks like it has nothing to do with the > versining. As a wild guess I can assume you have FLOWTABLE in your > kernel; if I'm right you should get rid of it. > > Eugene. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Solid Data Services Matt Lager / President *Office:* 480-351-5122 *Mobile:* 501-269-8606 www.SolidDataServices.com This e-mail message may contain confidential or legally privileged information and is intended only for the use of the intended recipient(s). Any unauthorized disclosure, dissemination, distribution, copying or the taking of any action in reliance on the information herein is prohibited. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, or contain viruses. Anyone who communicates with us by e-mail is deemed to have accepted these risks. Solid Data Services is not responsible for errors or omissions in this message and denies any responsibility for any damage arising from the use of e-mail. Any opinion and other statement contained in this message and any attachment are solely those of the author and do not necessarily represent those of the company. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 12:13:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5DC1FF for ; Tue, 15 Apr 2014 12:13:05 +0000 (UTC) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2001:470:1f09:14c0::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Norma UNIX CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F21B1291 for ; Tue, 15 Apr 2014 12:13:03 +0000 (UTC) Received: from bsdrookie.norma.com. (bsdrookie.norma.com [192.168.7.224]) by elf.hq.norma.perm.ru (8.14.5/8.14.5) with ESMTP id s3FCD0Bh017100 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 15 Apr 2014 18:13:00 +0600 (YEKT) (envelope-from emz@norma.perm.ru) Message-ID: <534D224C.8020000@norma.perm.ru> Date: Tue, 15 Apr 2014 18:13:00 +0600 From: "Eugene M. Zheganin" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Racoon/IPSEC Tunnel in 9.2 vs 10.0 References: <5345AA7C.3050700@soliddataservices.com> <534CB23E.8060304@norma.perm.ru> <534CB52C.60203@soliddataservices.com> In-Reply-To: <534CB52C.60203@soliddataservices.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (elf.hq.norma.perm.ru [192.168.3.10]); Tue, 15 Apr 2014 18:13:00 +0600 (YEKT) X-Spam-Status: No hits=-101.0 bayes=0.5 testhits ALL_TRUSTED=-1, USER_IN_WHITELIST=-100 autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on elf.hq.norma.perm.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 12:13:05 -0000 Hi. On 15.04.2014 10:27, Matt Lager wrote: > Do you utilize PF as your firewalling platform, because I'm slightly > suspicious that could be the cause. I do. Eugene. From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 15:17:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1EA3BAA9 for ; Tue, 15 Apr 2014 15:17:26 +0000 (UTC) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D4241176E for ; Tue, 15 Apr 2014 15:17:25 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.8/8.14.8) with ESMTP id s3FFHIB9088933; Tue, 15 Apr 2014 11:17:19 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <534D4D72.8040807@sentex.net> Date: Tue, 15 Apr 2014 11:17:06 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Przemyslaw Frasunek , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> <531C517F.5020506@frasunek.com> <531C5877.8050401@sentex.net> <532335B6.7070105@frasunek.com> In-Reply-To: <532335B6.7070105@frasunek.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 15:17:26 -0000 On 3/14/2014 1:00 PM, Przemyslaw Frasunek wrote: >>> FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 >>> enabled. >> What sort of uptime have you seen with ipv6 enabled ? > > Now it's 19 days and still no crash occurred. Hi, Has all been stable still with ipv6 enabled ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 19:10:03 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 285DE4DE for ; Tue, 15 Apr 2014 19:10:03 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14F8D1146 for ; Tue, 15 Apr 2014 19:10:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3FJA20d077675 for ; Tue, 15 Apr 2014 19:10:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3FJA2fk077674; Tue, 15 Apr 2014 19:10:02 GMT (envelope-from gnats) Date: Tue, 15 Apr 2014 19:10:02 GMT Message-Id: <201404151910.s3FJA2fk077674@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: =?koi8-r?B?7cnIwcnMIOzB18vJzg==?= Subject: Re: kern/183391: [oce] 10gigabit networking problems with Emulex OCE 11102 CNA X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: =?koi8-r?B?7cnIwcnMIOzB18vJzg==?= List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 19:10:03 -0000 The following reply was made to PR kern/183391; it has been noted by GNATS. From: =?koi8-r?B?7cnIwcnMIOzB18vJzg==?= To: Cc: Subject: Re: kern/183391: [oce] 10gigabit networking problems with Emulex OCE 11102 CNA Date: Tue, 15 Apr 2014 23:06:32 +0400 This is a multipart message in MIME format. ------=_NextPart_000_0313_01CF58FF.584FC000 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit FIX Use latest driver version http://www.emulex.com/downloads/emulex/drivers/freebsd/freebsd-91-amd64/driv ers/ I put it to 10.0 Release src tree and rebuild system. Now it work without hangs and panics. On 9.2 I have compile error. I think then new oce version can go to CURRENT. ------=_NextPart_000_0313_01CF58FF.584FC000 Content-Type: text/html; charset="koi8-r" Content-Transfer-Encoding: quoted-printable

FIX

 

Use latest driver version

http://www.emulex.com/downloads/emulex/drivers/freebsd/f= reebsd-91-amd64/drivers/

 

I put it to 10.0 Release src tree = and rebuild system. Now it work without hangs and = panics.

On = 9.2 I have compile error.

 

I think then new oce version can go = to CURRENT.

 

------=_NextPart_000_0313_01CF58FF.584FC000-- From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 22:46:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C00C89A; Tue, 15 Apr 2014 22:46:07 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C6F9B16C1; Tue, 15 Apr 2014 22:46:06 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id C8EEB25D3897; Tue, 15 Apr 2014 22:46:03 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 04011C22BD0; Tue, 15 Apr 2014 22:46:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 8LdjCu25KtGK; Tue, 15 Apr 2014 22:46:01 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:7c90:1de7:5946:cc76] (unknown [IPv6:fde9:577b:c1a9:4410:7c90:1de7:5946:cc76]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 08CDBC22B98; Tue, 15 Apr 2014 22:46:00 +0000 (UTC) From: "Bjoern A. Zeeb" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: route(8) fix Date: Tue, 15 Apr 2014 22:45:27 +0000 Message-Id: To: Hiroki Sato , Gleb Smirnoff Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:46:07 -0000 Hi, can someone please review the below patch and see if it=92s correct? I did a simple 1:1 replacement without thinking about the code. ! When switching variables to flags in r243185 a few cases were missed. ! After r263152 this leaves unused variables if route(8) is compiled ! without INET support. https://people.freebsd.org/~bz/20140415-01-sbin-route.diff =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Tue Apr 15 23:55:21 2014 Return-Path: Delivered-To: FreeBSD-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBC615E6 for ; Tue, 15 Apr 2014 23:55:21 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9DC1CE5 for ; Tue, 15 Apr 2014 23:55:16 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s3FNsZSb002212 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 16 Apr 2014 01:54:51 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s3FNsJ3m013200 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 16 Apr 2014 01:54:20 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s3FNsJEJ032326; Wed, 16 Apr 2014 01:54:19 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s3FNsJsf032325; Wed, 16 Apr 2014 01:54:19 +0200 (CEST) (envelope-from ticso) Date: Wed, 16 Apr 2014 01:54:19 +0200 From: Bernd Walter To: FreeBSD-net@freebsd.org Subject: SCTP stream socket read(2) can block a process Message-ID: <20140415235419.GG30429@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 23:55:21 -0000 When using SCTP in TCP like stream style. socket(2) connect(2) write(2) request read(2) answer When the other side disconnects the association (e.g. process restarted) after the write and before read call the read never returns. With TCP stream sockets the read would return with an error instead. SCTP are not TCP, but the other side can't associate back in on such a connect(2) socket, so returning would be expected. I've switched to nonblocking with timeout in the meantime - I wanted to have a response timeout anyway. In my test case the other side was a one2many socket on the same host. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:02:04 2014 Return-Path: Delivered-To: FreeBSD-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBAF5378 for ; Wed, 16 Apr 2014 01:02:04 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 345471439 for ; Wed, 16 Apr 2014 01:02:03 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s3G11fOG003078 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 16 Apr 2014 03:01:41 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s3G11ZdO015110 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 16 Apr 2014 03:01:35 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s3G11ZxU033318; Wed, 16 Apr 2014 03:01:35 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s3G11ZHu033317; Wed, 16 Apr 2014 03:01:35 +0200 (CEST) (envelope-from ticso) Date: Wed, 16 Apr 2014 03:01:35 +0200 From: Bernd Walter To: FreeBSD-net@freebsd.org Subject: Re: SCTP stream socket read(2) can block a process Message-ID: <20140416010135.GC32538@cicely7.cicely.de> References: <20140415235419.GG30429@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140415235419.GG30429@cicely7.cicely.de> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:02:04 -0000 On Wed, Apr 16, 2014 at 01:54:19AM +0200, Bernd Walter wrote: > When using SCTP in TCP like stream style. > socket(2) > connect(2) > write(2) request > read(2) answer > > When the other side disconnects the association (e.g. process restarted) > after the write and before read call the read never returns. I wasn't correct - it happened to me after calling read(2), but before the other side send anything back, so the read(2) was already in kernel when the other side closed it's end. > With TCP stream sockets the read would return with an error instead. > SCTP are not TCP, but the other side can't associate back in on such > a connect(2) socket, so returning would be expected. > I've switched to nonblocking with timeout in the meantime - I wanted > to have a response timeout anyway. > In my test case the other side was a one2many socket on the same host. > > -- > B.Walter http://www.bwct.de > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:16:20 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AFBD9BA; Wed, 16 Apr 2014 01:16:20 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6ECB3158D; Wed, 16 Apr 2014 01:16:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G1GKfL005106; Wed, 16 Apr 2014 01:16:20 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G1GKDq005105; Wed, 16 Apr 2014 01:16:20 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 01:16:20 GMT Message-Id: <201404160116.s3G1GKDq005105@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187451: [vlan] [carp] Some vlans in bridge + carp result in hung server X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:16:20 -0000 Old Synopsis: Some vlans in bride + carp result hung server New Synopsis: [vlan] [carp] Some vlans in bridge + carp result in hung server Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 01:15:19 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=187451 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:17:49 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51071A9E; Wed, 16 Apr 2014 01:17:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2538015A0; Wed, 16 Apr 2014 01:17:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G1Hn6o005215; Wed, 16 Apr 2014 01:17:49 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G1HmgG005214; Wed, 16 Apr 2014 01:17:48 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 01:17:48 GMT Message-Id: <201404160117.s3G1HmgG005214@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187718: [igb] UDP bad performance and out-of-order packet with iperf3 and igb(4) drivers X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:17:49 -0000 Old Synopsis: UDP bad performance and out-of-order packet with iperf3 and igb(4) drivers New Synopsis: [igb] UDP bad performance and out-of-order packet with iperf3 and igb(4) drivers Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 01:17:25 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=187718 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:27:45 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C89B7DD9; Wed, 16 Apr 2014 01:27:45 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9D7AE166A; Wed, 16 Apr 2014 01:27:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G1RjTC008775; Wed, 16 Apr 2014 01:27:45 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G1Rj8A008774; Wed, 16 Apr 2014 01:27:45 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 01:27:45 GMT Message-Id: <201404160127.s3G1Rj8A008774@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/188032: [lo] IPv6 on lo never leaves 'tentative' state if configured with prefixlen 128 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:27:45 -0000 Old Synopsis: IPv6 on lo never leaves 'tentative' state if configured with prefixlen 128 New Synopsis: [lo] IPv6 on lo never leaves 'tentative' state if configured with prefixlen 128 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 01:26:32 UTC 2014 Responsible-Changed-Why: assign. http://www.freebsd.org/cgi/query-pr.cgi?pr=188032 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:37:54 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E49EC380; Wed, 16 Apr 2014 01:37:54 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8F531764; Wed, 16 Apr 2014 01:37:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G1bskf012120; Wed, 16 Apr 2014 01:37:54 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G1bse9012119; Wed, 16 Apr 2014 01:37:54 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 01:37:54 GMT Message-Id: <201404160137.s3G1bse9012119@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/188421: [libnetgraph] [patch] ng_callout() timeouts trigger packets queuing and out of order packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:37:55 -0000 Old Synopsis: ng_callout() timeouts trigger packets queuing and out of order packets New Synopsis: [libnetgraph] [patch] ng_callout() timeouts trigger packets queuing and out of order packets Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 01:36:26 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=188421 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 01:42:55 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBE9359C; Wed, 16 Apr 2014 01:42:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B02891827; Wed, 16 Apr 2014 01:42:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G1gtVV015610; Wed, 16 Apr 2014 01:42:55 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G1gtQh015609; Wed, 16 Apr 2014 01:42:55 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 01:42:55 GMT Message-Id: <201404160142.s3G1gtQh015609@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187149: [patch] [netmap] fix netmap's pkt-gen behavior with IP addresses or port range X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 01:42:55 -0000 Old Synopsis: Patch for fixng netmap's pkt-gen behavior with IP addresses or port range New Synopsis: [patch] [netmap] fix netmap's pkt-gen behavior with IP addresses or port range Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 01:42:19 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=187149 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 02:04:52 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B423ABBC; Wed, 16 Apr 2014 02:04:52 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 88CCD19DA; Wed, 16 Apr 2014 02:04:52 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G24qPb023132; Wed, 16 Apr 2014 02:04:52 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G24qJC023131; Wed, 16 Apr 2014 02:04:52 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 02:04:52 GMT Message-Id: <201404160204.s3G24qJC023131@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/168913: tcpdump(8) causing interface to drop X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 02:04:52 -0000 Old Synopsis: TCPDUMP causing interface to drop New Synopsis: tcpdump(8) causing interface to drop Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 02:04:37 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=168913 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 02:08:07 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26AD5CDB; Wed, 16 Apr 2014 02:08:07 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EF89E1A0D; Wed, 16 Apr 2014 02:08:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3G286x6023644; Wed, 16 Apr 2014 02:08:06 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3G286Ww023643; Wed, 16 Apr 2014 02:08:06 GMT (envelope-from linimon) Date: Wed, 16 Apr 2014 02:08:06 GMT Message-Id: <201404160208.s3G286Ww023643@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/186449: ifconfig(8) fails to autoload if_tap.ko when creating vmnet interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 02:08:07 -0000 Synopsis: ifconfig(8) fails to autoload if_tap.ko when creating vmnet interface Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed Apr 16 02:05:37 UTC 2014 Responsible-Changed-Why: this is kind of a generic problem with interface naming. http://www.freebsd.org/cgi/query-pr.cgi?pr=186449 From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 06:52:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F20E756; Wed, 16 Apr 2014 06:52:01 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ABB3E159D; Wed, 16 Apr 2014 06:51:59 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s3G6pOR2036766 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 16 Apr 2014 10:51:24 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s3G6pOLk036765; Wed, 16 Apr 2014 10:51:24 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 16 Apr 2014 10:51:24 +0400 From: Gleb Smirnoff To: "Bjoern A. Zeeb" Subject: Re: route(8) fix Message-ID: <20140416065124.GZ902@glebius.int.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: FreeBSD Net , Hiroki Sato X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 06:52:01 -0000 On Tue, Apr 15, 2014 at 10:45:27PM +0000, Bjoern A. Zeeb wrote: B> can someone please review the below patch and see if it’s correct? B> I did a simple 1:1 replacement without thinking about the code. B> B> ! When switching variables to flags in r243185 a few cases were missed. B> ! After r263152 this leaves unused variables if route(8) is compiled B> ! without INET support. B> B> https://people.freebsd.org/~bz/20140415-01-sbin-route.diff Looks entirely correct and probably should fix some bugs. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 19:01:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 507FDD0B for ; Wed, 16 Apr 2014 19:01:02 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 272A6150F for ; Wed, 16 Apr 2014 19:01:02 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id lf10so11280743pab.13 for ; Wed, 16 Apr 2014 12:01:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:message-id:date:to:mime-version; bh=oAL9prr4CzRKnndG/xQ5ODjLPca9PtCHUdSrHwfa1jI=; b=h3spVGDp1+HepKSwgm61H3QE8mMdvAIga+qIUAueQ1+VAGg8zOf4mp270+fUnOgJpy 579Vk5EvOHoQr2gN0HnZn9bmZeVOK3EDhpR51N/BSRovvdrlZ9GuO178Tl2UKY2YTBv6 xSD1xYPfRhvq0WOu7tcrUk+gKkhj5BYVlG2s2ifYB97h90dFKoJEwt0V4sELaT+P5hhJ VzXP84OPDaDwnN8cC1ock1luCXxRkuQQcz/8nJuG0lQc9Ue0ZVBE1LInO29USDkWw7i3 4O76O9M7sAz3L9/oIHvxCLf+XWNLAcNU8lSz5qo7R384d0TX17veHaOgYoLV8/D5bMHI yMKQ== X-Received: by 10.68.204.162 with SMTP id kz2mr10400655pbc.13.1397674861766; Wed, 16 Apr 2014 12:01:01 -0700 (PDT) Received: from briankrusicw.logan.tv ([64.17.255.138]) by mx.google.com with ESMTPSA id op3sm48578578pbc.40.2014.04.16.12.01.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Apr 2014 12:01:01 -0700 (PDT) From: aurfalien Subject: Solarflare LACP bug? Message-Id: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> Date: Wed, 16 Apr 2014 12:00:58 -0700 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 19:01:02 -0000 Hi, I=92ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. While the card works fine as individual ports, upon configuring LACP the = machine suddenly reboots. Here are my commands; ifconfig sfxge0 up ifconfig sfxge1 up ifconfig lagg0 create * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 = 10.0.10.99/16 * This is were the system reboots. I believe this to be a bug, should i post this on = freebsd-bugs@freebsd.org The only thing in /var/crash is minfree. - aurf "Janitorial Services" From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 21:59:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 221FD3C7 for ; Wed, 16 Apr 2014 21:59:31 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E27F7163B for ; Wed, 16 Apr 2014 21:59:30 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id hl10so1598668igb.12 for ; Wed, 16 Apr 2014 14:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=X/K6L4IFDsUEREQLWcNGlAgcV8/T7hfxruu12m5zdi4=; b=aE/+cpY/Wd0bT2NjxkC4LWquy+8ZEW757HtN0ZJPfqP7FIT5kRYWJVQriTXhn/RMt0 gv+ldWesVbHcbILlHNbVGFmkIGg9hJ01co9ckMvlwxXGyJ802KFuuMV8VCeMZ98eNTDC kF9sWlPpk47EjrqyPGHrGibbmGVqRJzR7Lvg8+hCsC+6QlGlNPH7TxjipzRnugp/mO44 fcLVoMGynXPeyDfXBOe77pDHxcd2XoMm6Kz/pXjrewyT16j9BhpGJ5cl/KkO7sB4Q1NO IjPZF3j4JBTUPs+Hnm9sAWdXqeTSEPSzDZEstgA+mVz4Zjr+tYvdHv9EyiRNtfK2C4YT gtXA== X-Received: by 10.50.111.135 with SMTP id ii7mr7430804igb.35.1397685570369; Wed, 16 Apr 2014 14:59:30 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id r4sm1170602igh.1.2014.04.16.14.59.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Apr 2014 14:59:29 -0700 (PDT) Message-ID: <534EFD3E.3010006@gmail.com> Date: Wed, 16 Apr 2014 17:59:26 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> <53483B8B.7030409@freebsd.org> In-Reply-To: <53483B8B.7030409@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 21:59:31 -0000 Hi Julian, The code behaves as you described. The intent is clearly there and I can confirm the behavior is correct for incoming packets. At this point it looks like turning off net.isr.direct makes a difference too so I'm trying to understand the relationship between the system's queuing behavior and netgraph's. The only common ground I can find at this point is they both use the "swi1: net" thread. There may be a higher level design concern with forcing WRITER on callout items. It seems that, in some cases, it is desirable to prevent queuing packets if a timer event (ng_callout) can execute asynchronously, but I digress. Regards, Karim. On 11/04/2014 2:59 PM, Julian Elischer wrote: > disclaimer: I'm not looking at the code now.. I want to go to bed: :-) > > When I wrote that code, the idea was that even a direct node execution > should become a queuing operation if there was already something else > on the queue. so in that model packets were not supposed to get > re-ordered. does that not still work? > > Either that, or you need to explain the problem to me a bit better.. > > > On 4/10/14, 5:25 AM, Karim Fodil-Lemelin wrote: >> Hi, >> >> Below is a revised patch for this issue. It accounts for nodes or >> hooks that explicitly need to be queuing: >> >> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, >> hook_p hook, int ticks, >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + if ((node->nd_flags & NGF_FORCE_WRITER) || >> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >> + item->el_flags |= NGQF_WRITER; >> + else >> + item->el_flags |= NGQF_READER; >> + >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> Regards, >> >> Karim. >> >> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>> Hi List, >>> >>> I'm calling out to the general wisdom ... I have seen an issue in >>> netgraph where, if called, a callout routine registered by >>> ng_callout() will trigger packet queuing inside the worklist of >>> netgraph since ng_callout() makes my node suddenly a WRITER node >>> (therefore non reentrant) for the duration of the call. >>> >>> So as soon as the callout function returns, all following packets >>> will get directly passed to the node again and when the ngintr >>> thread gets executed then only then will I get the queued packets. >>> This introduces out of order packets in the flow. I am using the >>> current patch below to solve the issue and I am wondering if there >>> is anything wrong with it (and maybe contribute back :): >>> >>> >>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, >>> hook_p hook, int ticks, >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + item->el_flags = NGQF_READER; >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> >>> Best regards, >>> >>> Karim. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 04:26:09 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8088E261; Thu, 17 Apr 2014 04:26:09 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53251106B; Thu, 17 Apr 2014 04:26:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3H4Q9jT060870; Thu, 17 Apr 2014 04:26:09 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3H4Q9i2060869; Thu, 17 Apr 2014 04:26:09 GMT (envelope-from linimon) Date: Thu, 17 Apr 2014 04:26:09 GMT Message-Id: <201404170426.s3H4Q9i2060869@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/173521: [ip6] remove useless assign in ip6_hopopts_input X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 04:26:09 -0000 Old Synopsis: remove useless assign in ip6_hopopts_input New Synopsis: [ip6] remove useless assign in ip6_hopopts_input Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Apr 17 04:25:45 UTC 2014 Responsible-Changed-Why: reassign. http://www.freebsd.org/cgi/query-pr.cgi?pr=173521 From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 06:40:58 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21281F47; Thu, 17 Apr 2014 06:40:58 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC8BE1C9C; Thu, 17 Apr 2014 06:40:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3H6evhN011355; Thu, 17 Apr 2014 06:40:57 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3H6esAd011354; Thu, 17 Apr 2014 06:40:54 GMT (envelope-from ae) Date: Thu, 17 Apr 2014 06:40:54 GMT Message-Id: <201404170640.s3H6esAd011354@freefall.freebsd.org> To: yizhouzhou@ict.ac.cn, ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/173521: [ip6] remove useless assign in ip6_hopopts_input X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 06:40:58 -0000 Synopsis: [ip6] remove useless assign in ip6_hopopts_input State-Changed-From-To: open->patched State-Changed-By: ae State-Changed-When: Thu Apr 17 06:40:24 UTC 2014 State-Changed-Why: Patched in head/. Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Thu Apr 17 06:40:24 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=173521 From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 20:59:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D46A336F for ; Thu, 17 Apr 2014 20:59:37 +0000 (UTC) Received: from mp1-smtp-2.eutelia.it (mp1-smtp-2.eutelia.it [62.94.10.162]) by mx1.freebsd.org (Postfix) with ESMTP id 55F4E12A9 for ; Thu, 17 Apr 2014 20:59:37 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-2.eutelia.it (Eutelia) with ESMTP id 85496E19F0 for ; Thu, 17 Apr 2014 22:39:11 +0200 (CEST) Received: from soth.ventu (adsl-ull-118-176.41-151.net24.it [151.41.176.118]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3HKcn4w059797 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 17 Apr 2014 22:39:08 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-118-176.41-151.net24.it [151.41.176.118] claimed to be soth.ventu Received: from guardian.ventu (bane.ventu [10.1.2.15]) (authenticated bits=0) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3HKcRiI088430 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 17 Apr 2014 22:38:31 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.ventu: Host bane.ventu [10.1.2.15] claimed to be guardian.ventu Message-ID: <53503BC3.6040806@netfence.it> Date: Thu, 17 Apr 2014 22:38:27 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Network troubles after 8.3 -> 8.4 upgrade Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 X-Scanned-By: MIMEDefang 2.74 on 10.1.2.13 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Thu, 17 Apr 2014 22:39:08 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 20:59:38 -0000 Hello. Three days ago I upgraded an amd64 8.3 box to the latest 8.4. Since then the outside network is misbehaving: large mails are not sended (although small ones do), svn operations will work for a while, then come to a sudden stop, etc... Perhaps the most evident test is "wget"ting a big file: it will download some chunk, halt; restart after a while and download another chunk; lose the connection once again, then restart and so on. I remember a couple of similar experiences in the past, from which I got out by disabling TSO; however those box had fxp cards, while this has an em. In any case disabling TSO did not help. This is the relevant part of rc.conf: > cloned_interfaces="lagg0 vlan1 vlan2 vlan3 carp0 carp1 carp3 carp4 carp6 carp7 carp9 carp10" > ifconfig_igb0="up" > ifconfig_igb1="up" > ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 192.168.101.4 netmask 255.255.255.0" > ifconfig_lagg0_alias0="inet 192.168.101.101 netmask 0xffffffff" > ifconfig_carp0="vhid 1 advskew 100 pass xxxxxxx 192.168.101.10" > ifconfig_carp1="vhid 2 pass xxxxxxxx 192.168.101.10" > ifconfig_em0="up" > ifconfig_vlan1="inet 81.174.30.11 netmask 255.255.255.248 vlan 4 vlandev em0" > ifconfig_vlan2="inet 83.211.188.186 netmask 255.255.255.248 vlan 2 vlandev em0" > ifconfig_vlan3="inet 192.168.2.202 netmask 255.255.255.0 vlan 3 vlandev em0" > ifconfig_carp3="vhid 4 advskew 100 pass xxxx 81.174.30.12" > ifconfig_carp4="vhid 5 pass xxxxxxx 81.174.30.12" > ifconfig_carp6="vhid 7 advskew 100 pass xxxxxx 83.211.188.187" > ifconfig_carp7="vhid 8 pass xxxxxxxxxxx 83.211.188.187" > ifconfig_carp9="vhid 10 advskew 100 pass xxxxxxxx 192.168.2.203" > ifconfig_carp10="vhid 11 pass xxxxxxxx 192.168.2.203" > ifconfig_lo0_alias0="inet 127.0.0.2 netmask 0xffffffff" > ifconfig_lo0_alias1="inet 127.0.0.3 netmask 0xffffffff" > ifconfig_lo0_alias2="inet 127.0.0.4 netmask 0xffffffff" As you can see the setup is quite complicated, but worked like a charm until the upgrade; actually the internal net (igb+lagg+carp) still does, so this is what points me toward em0, where I cannot seem to get any kind of stability. The card is > em0@pci0:6:0:0: class=0x020000 card=0x10828086 chip=0x107d8086 rev=0x06 hdr=0x00 > vendor = 'Intel Corporation' > device = 'PRO/1000 PT' > class = network > subclass = ethernet I tried disabling TSO, RXCSUM, TXCSUM, VLANHWTAG, VLANHWCSUM, VLANHWTSO... I tried putting the card into 10baseT/UTP mode... I tried sysctl net.inet.tcp.tso=0... None helped. Maybe I'm barking up the wrong tree, but nothing is in the logs to help... Nor did Google or wading through bug reports. Now I could restore the dumps I made before upgrading to 8.4 (but I'd really like to avoid this), try to upgrade even further to 9.2 (although this will be a lot of work and I'm not looking forward to it as a shot in the dark), drop in another NIC... What I'd really like, however, is some insight. Is this a known problem of some sort? Is this card or this driver known to be broken? Is there any way I could get some debugging info? Any hint is appreciated (and I need it badly :( !!!). bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 21:45:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F12BDD9 for ; Thu, 17 Apr 2014 21:45:36 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70F5F16EC for ; Thu, 17 Apr 2014 21:45:36 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s3HLjMnH009055 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 17 Apr 2014 17:45:23 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Network troubles after 8.3 -> 8.4 upgrade From: John Nielsen In-Reply-To: <53503BC3.6040806@netfence.it> Date: Thu, 17 Apr 2014 15:45:23 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <53503BC3.6040806@netfence.it> To: Andrea Venturoli X-Mailer: Apple Mail (2.1874) X-DCC-x.dcc-servers-Metrics: ns1.jnielsen.net 104; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 21:45:36 -0000 On Apr 17, 2014, at 2:38 PM, Andrea Venturoli wrote: > Three days ago I upgraded an amd64 8.3 box to the latest 8.4. > Since then the outside network is misbehaving: large mails are not = sended (although small ones do), svn operations will work for a while, = then come to a sudden stop, etc... > Perhaps the most evident test is "wget"ting a big file: it will = download some chunk, halt; restart after a while and download another = chunk; lose the connection once again, then restart and so on. >=20 > I remember a couple of similar experiences in the past, from which I = got out by disabling TSO; however those box had fxp cards, while this = has an em. > In any case disabling TSO did not help. My first thought was TSO as well, since I've seen the symptoms you = describe a few times on systems running 10.0. Do you use IPFW or any = kind of NAT on this system? When an application encounters a network = problem, does it report or log anything at all? Anything in the kernel = log/dmesg? A bit of a shot in the dark, but could you try applying r264517 (fixes a = problem with VLAN and TSO interaction)? = http://svnweb.freebsd.org/base/head/sys/net/if_vlan.c?r1=3D257241&r2=3D264= 517 Otherwise my only other thought would be the driver. Can you try = reverting only the em(4) driver back to 8.3? If that helps it would give = you both a workaround and a clue for where to look for a solution. Build = modules and a kernel without em(4) from unmodified 8.4 src, load em(4) = as a module, confirm that the problem persists. Replace the contents of = src/sys/dev/e1000, src/sys/modules/em and src/sys/conf/files with those = from an 8.3 src tree (or otherwise revert revision 247430), rebuild em = module, unload/reload or reboot, see if problem goes away. (Could be = somewhat complicated by the fact that you also have igb interfaces which = also use code from the e1000 directory, but rather than speculate I'll = leave solving that as an exercise for someone else.) JN > This is the relevant part of rc.conf: >> cloned_interfaces=3D"lagg0 vlan1 vlan2 vlan3 carp0 carp1 carp3 carp4 = carp6 carp7 carp9 carp10" >> ifconfig_igb0=3D"up" >> ifconfig_igb1=3D"up" >> ifconfig_lagg0=3D"laggproto lacp laggport igb0 laggport igb1 = 192.168.101.4 netmask 255.255.255.0" >> ifconfig_lagg0_alias0=3D"inet 192.168.101.101 netmask 0xffffffff" >> ifconfig_carp0=3D"vhid 1 advskew 100 pass xxxxxxx 192.168.101.10" >> ifconfig_carp1=3D"vhid 2 pass xxxxxxxx 192.168.101.10" >> ifconfig_em0=3D"up" >> ifconfig_vlan1=3D"inet 81.174.30.11 netmask 255.255.255.248 vlan 4 = vlandev em0" >> ifconfig_vlan2=3D"inet 83.211.188.186 netmask 255.255.255.248 vlan 2 = vlandev em0" >> ifconfig_vlan3=3D"inet 192.168.2.202 netmask 255.255.255.0 vlan 3 = vlandev em0" >> ifconfig_carp3=3D"vhid 4 advskew 100 pass xxxx 81.174.30.12" >> ifconfig_carp4=3D"vhid 5 pass xxxxxxx 81.174.30.12" >> ifconfig_carp6=3D"vhid 7 advskew 100 pass xxxxxx 83.211.188.187" >> ifconfig_carp7=3D"vhid 8 pass xxxxxxxxxxx 83.211.188.187" >> ifconfig_carp9=3D"vhid 10 advskew 100 pass xxxxxxxx 192.168.2.203" >> ifconfig_carp10=3D"vhid 11 pass xxxxxxxx 192.168.2.203" >> ifconfig_lo0_alias0=3D"inet 127.0.0.2 netmask 0xffffffff" >> ifconfig_lo0_alias1=3D"inet 127.0.0.3 netmask 0xffffffff" >> ifconfig_lo0_alias2=3D"inet 127.0.0.4 netmask 0xffffffff" >=20 > As you can see the setup is quite complicated, but worked like a charm = until the upgrade; actually the internal net (igb+lagg+carp) still does, = so this is what points me toward em0, where I cannot seem to get any = kind of stability. >=20 > The card is >> em0@pci0:6:0:0: class=3D0x020000 card=3D0x10828086 chip=3D0x107d8086 = rev=3D0x06 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'PRO/1000 PT' >> class =3D network >> subclass =3D ethernet >=20 > I tried disabling TSO, RXCSUM, TXCSUM, VLANHWTAG, VLANHWCSUM, = VLANHWTSO... > I tried putting the card into 10baseT/UTP mode... > I tried sysctl net.inet.tcp.tso=3D0... >=20 > None helped. >=20 > Maybe I'm barking up the wrong tree, but nothing is in the logs to = help... >=20 > Nor did Google or wading through bug reports. >=20 >=20 >=20 > Now I could restore the dumps I made before upgrading to 8.4 (but I'd = really like to avoid this), try to upgrade even further to 9.2 (although = this will be a lot of work and I'm not looking forward to it as a shot = in the dark), drop in another NIC... > What I'd really like, however, is some insight. >=20 > Is this a known problem of some sort? Is this card or this driver = known to be broken? > Is there any way I could get some debugging info? >=20 > Any hint is appreciated (and I need it badly :( !!!). >=20 > bye & Thanks > av. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 23:48:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A85D22B8 for ; Thu, 17 Apr 2014 23:48:06 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 31B621258 for ; Thu, 17 Apr 2014 23:48:05 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAPVnUFODaFve/2dsb2JhbABZg0FXgxO5HYZmUoE4dIIlAQEBAwEBAQEgBCcgCwUWDgoCAg0ZAikBCSYGCAcEARoCBIdTCA2oZ6MvF4EpjGgBAQ0ONAeCb4FJBIh0jSKED5EWg00hMX0HFyI X-IronPort-AV: E=Sophos;i="4.97,882,1389762000"; d="scan'208";a="115926903" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 17 Apr 2014 19:47:58 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 57E99B3F43; Thu, 17 Apr 2014 19:47:58 -0400 (EDT) Date: Thu, 17 Apr 2014 19:47:58 -0400 (EDT) From: Rick Macklem To: John Nielsen Message-ID: <2018376789.12960091.1397778478349.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Network troubles after 8.3 -> 8.4 upgrade MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 23:48:06 -0000 John Nielsen wrote: > On Apr 17, 2014, at 2:38 PM, Andrea Venturoli wrote: > > > Three days ago I upgraded an amd64 8.3 box to the latest 8.4. > > Since then the outside network is misbehaving: large mails are not > > sended (although small ones do), svn operations will work for a > > while, then come to a sudden stop, etc... > > Perhaps the most evident test is "wget"ting a big file: it will > > download some chunk, halt; restart after a while and download > > another chunk; lose the connection once again, then restart and so > > on. > > > > I remember a couple of similar experiences in the past, from which > > I got out by disabling TSO; however those box had fxp cards, while > > this has an em. > > In any case disabling TSO did not help. > > My first thought was TSO as well, since I've seen the symptoms you > describe a few times on systems running 10.0. Do you use IPFW or any > kind of NAT on this system? When an application encounters a network > problem, does it report or log anything at all? Anything in the > kernel log/dmesg? > > A bit of a shot in the dark, but could you try applying r264517 > (fixes a problem with VLAN and TSO interaction)? > http://svnweb.freebsd.org/base/head/sys/net/if_vlan.c?r1=257241&r2=264517 > Since the only net driver that sets if_hw_tsomax is Xen's netfront, the patch only affects systems that use that at this time. (The bug, which was also in if_lagg.c, was found during testing of an experimental patch for a net driver.) So, I'm pretty sure that patch won't help, rick > Otherwise my only other thought would be the driver. Can you try > reverting only the em(4) driver back to 8.3? If that helps it would > give you both a workaround and a clue for where to look for a > solution. Build modules and a kernel without em(4) from unmodified > 8.4 src, load em(4) as a module, confirm that the problem persists. > Replace the contents of src/sys/dev/e1000, src/sys/modules/em and > src/sys/conf/files with those from an 8.3 src tree (or otherwise > revert revision 247430), rebuild em module, unload/reload or reboot, > see if problem goes away. (Could be somewhat complicated by the fact > that you also have igb interfaces which also use code from the e1000 > directory, but rather than speculate I'll leave solving that as an > exercise for someone else.) > > JN > > > This is the relevant part of rc.conf: > >> cloned_interfaces="lagg0 vlan1 vlan2 vlan3 carp0 carp1 carp3 carp4 > >> carp6 carp7 carp9 carp10" > >> ifconfig_igb0="up" > >> ifconfig_igb1="up" > >> ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 > >> 192.168.101.4 netmask 255.255.255.0" > >> ifconfig_lagg0_alias0="inet 192.168.101.101 netmask 0xffffffff" > >> ifconfig_carp0="vhid 1 advskew 100 pass xxxxxxx 192.168.101.10" > >> ifconfig_carp1="vhid 2 pass xxxxxxxx 192.168.101.10" > >> ifconfig_em0="up" > >> ifconfig_vlan1="inet 81.174.30.11 netmask 255.255.255.248 vlan 4 > >> vlandev em0" > >> ifconfig_vlan2="inet 83.211.188.186 netmask 255.255.255.248 vlan 2 > >> vlandev em0" > >> ifconfig_vlan3="inet 192.168.2.202 netmask 255.255.255.0 vlan 3 > >> vlandev em0" > >> ifconfig_carp3="vhid 4 advskew 100 pass xxxx 81.174.30.12" > >> ifconfig_carp4="vhid 5 pass xxxxxxx 81.174.30.12" > >> ifconfig_carp6="vhid 7 advskew 100 pass xxxxxx 83.211.188.187" > >> ifconfig_carp7="vhid 8 pass xxxxxxxxxxx 83.211.188.187" > >> ifconfig_carp9="vhid 10 advskew 100 pass xxxxxxxx 192.168.2.203" > >> ifconfig_carp10="vhid 11 pass xxxxxxxx 192.168.2.203" > >> ifconfig_lo0_alias0="inet 127.0.0.2 netmask 0xffffffff" > >> ifconfig_lo0_alias1="inet 127.0.0.3 netmask 0xffffffff" > >> ifconfig_lo0_alias2="inet 127.0.0.4 netmask 0xffffffff" > > > > As you can see the setup is quite complicated, but worked like a > > charm until the upgrade; actually the internal net (igb+lagg+carp) > > still does, so this is what points me toward em0, where I cannot > > seem to get any kind of stability. > > > > The card is > >> em0@pci0:6:0:0: class=0x020000 card=0x10828086 chip=0x107d8086 > >> rev=0x06 hdr=0x00 > >> vendor = 'Intel Corporation' > >> device = 'PRO/1000 PT' > >> class = network > >> subclass = ethernet > > > > I tried disabling TSO, RXCSUM, TXCSUM, VLANHWTAG, VLANHWCSUM, > > VLANHWTSO... > > I tried putting the card into 10baseT/UTP mode... > > I tried sysctl net.inet.tcp.tso=0... > > > > None helped. > > > > Maybe I'm barking up the wrong tree, but nothing is in the logs to > > help... > > > > Nor did Google or wading through bug reports. > > > > > > > > Now I could restore the dumps I made before upgrading to 8.4 (but > > I'd really like to avoid this), try to upgrade even further to 9.2 > > (although this will be a lot of work and I'm not looking forward > > to it as a shot in the dark), drop in another NIC... > > What I'd really like, however, is some insight. > > > > Is this a known problem of some sort? Is this card or this driver > > known to be broken? > > Is there any way I could get some debugging info? > > > > Any hint is appreciated (and I need it badly :( !!!). > > > > bye & Thanks > > av. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Apr 18 15:12:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 516E5B63; Fri, 18 Apr 2014 15:12:15 +0000 (UTC) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 183B51A80; Fri, 18 Apr 2014 15:12:15 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:3815:a891:65b8:fb7e]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id D05944AC34; Fri, 18 Apr 2014 19:12:08 +0400 (MSK) Date: Fri, 18 Apr 2014 19:12:03 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1172266612.20140418191203@serebryakov.spb.ru> To: freebsd-net@freebsd.org Subject: amd(8) doesn't work in 10 and CURRENT (does auromound(8) support NFS?) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Edward Tomasz Napierala X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 15:12:15 -0000 Hello, Freebsd-net. I'm trying to use amd(8) with default config & map for "classic" task: access NFS shares without explicit "mount" commands (which need "sudo"). And it seems, that amd(8) is completely broken. It dumps core after 1-2 minutes of working with amd-mounted share (and NFS client complains on timeouts after that). Is it known problem? Does somebody use amd(8) these days? Does new "automound(8)", which "compatible with its counterparts in OS X, Solaris, and Linux" support NFS? -- // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Fri Apr 18 21:59:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DD8B689; Fri, 18 Apr 2014 21:59:13 +0000 (UTC) Received: from mail-ee0-x22b.google.com (mail-ee0-x22b.google.com [IPv6:2a00:1450:4013:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A16F514A0; Fri, 18 Apr 2014 21:59:12 +0000 (UTC) Received: by mail-ee0-f43.google.com with SMTP id e53so1961621eek.2 for ; Fri, 18 Apr 2014 14:59:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3XqXPU19TkyRUPguuiTY68CF9SJ+tVxWJAIMM5Ob5lY=; b=HJRSHK5pKZ5jFNNi1DtKnkKPFTM5sTnpMNePd3s2ujhP4dLo1l0yoX6N5ezekafZIH FLRuJVuyUlRbfIPd5kf2wwk5xUYyJDUZ9rU6SLdhTANM4yLSWJlTtFI4Kk8DHZHomXyu oBjPck2s0TasKuOICwHY6raBoNlbvMRcZB/fL9Rp/gzVSkYlGDeddkuhfAG5K/SK6NWZ kSvvLMUAIK5qMZ4GIBdnDFD96ZKNcrhwZMwTbO++KQzO0Sv0mgFZwKfgvJV9yMLbBbA1 x4Y4CeDpydigUnuS+72EaUN/ZcY9Qmyi7hHlN0eFmAwBk4nuNgIBiLa9RYcMCGlTWLjV 44eg== X-Received: by 10.15.50.136 with SMTP id l8mr6381863eew.73.1397858350959; Fri, 18 Apr 2014 14:59:10 -0700 (PDT) Received: from strashydlo.home (ajg1.neoplus.adsl.tpnet.pl. [83.25.240.1]) by mx.google.com with ESMTPSA id x45sm80115868eef.15.2014.04.18.14.59.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 18 Apr 2014 14:59:10 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: amd(8) doesn't work in 10 and CURRENT (does auromound(8) support NFS?) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= X-Priority: 3 (Normal) In-Reply-To: <1172266612.20140418191203@serebryakov.spb.ru> Date: Fri, 18 Apr 2014 23:59:08 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1172266612.20140418191203@serebryakov.spb.ru> To: lev@FreeBSD.org X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 21:59:13 -0000 Wiadomo=B6=E6 napisana przez Lev Serebryakov w dniu 18 kwi 2014, o godz. = 17:12: > Hello, Freebsd-net. >=20 > I'm trying to use amd(8) with default config & map for "classic" task: > access NFS shares without explicit "mount" commands (which need = "sudo"). >=20 > And it seems, that amd(8) is completely broken. It dumps core after = 1-2 > minutes of working with amd-mounted share (and NFS client complains on > timeouts after that). >=20 > Is it known problem? Does somebody use amd(8) these days? >=20 > Does new "automound(8)", which "compatible with its counterparts in = OS X, > Solaris, and Linux" support NFS? Well, it would be kind of pointless if it didn't, right? ;-) From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 06:40:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58294915 for ; Sat, 19 Apr 2014 06:40:14 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [84.52.89.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08F6E12F9 for ; Sat, 19 Apr 2014 06:40:13 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 56DA77F4D9 for ; Sat, 19 Apr 2014 10:33:14 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 56DA77F4D9 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1397889194; bh=OIwREe0wpFBw48n6LxuV6wlwYdbT1p3sn/sqEqKyXA0=; l=1148; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=HxoFD2Bf26WIcjH9zfltQ7Un+cBodQng9SqaOizqnZH1+72N/feijgd1rrqKAo83L UX/ghsjvWXYwWGRps+Al+LUlrT7YyogpZ72ID3ji/TysUq3HggdwIDMr8EGxkjEPMf ogcOoQElRl3ZpQNxB2VvkgpZKkSSnbhDI3qrOz+U= Message-ID: <535218AA.6050704@oktetlabs.ru> Date: Sat, 19 Apr 2014 10:33:14 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Solarflare LACP bug? References: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> In-Reply-To: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 06:40:14 -0000 Hi, I can repeat the bug. I'll investigate and return with the patch to fix it. Thanks for the report and sorry for delay with reply, Andrew. On 04/16/2014 11:00 PM, aurfalien wrote: > Hi, > > I’ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. > > While the card works fine as individual ports, upon configuring LACP the machine suddenly reboots. > > Here are my commands; > > ifconfig sfxge0 up > ifconfig sfxge1 up > ifconfig lagg0 create > * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 10.0.10.99/16 > > * This is were the system reboots. > > I believe this to be a bug, should i post this on freebsd-bugs@freebsd.org > > The only thing in /var/crash is minfree. > > - aurf > > "Janitorial Services" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Andrew Rybchenko OKTET Labs, St.-Petersburg, Russia Web: www.oktetlabs.ru Office: +7 812 7832191 Fax: +7 812 7846591 Mobile: +7 921 7479683 From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 06:40:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E90C9A7; Sat, 19 Apr 2014 06:40:54 +0000 (UTC) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0E0EF1370; Sat, 19 Apr 2014 06:40:53 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:3815:a891:65b8:fb7e]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 1D3084AC34; Sat, 19 Apr 2014 10:40:41 +0400 (MSK) Date: Sat, 19 Apr 2014 10:40:35 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1021530859.20140419104035@serebryakov.spb.ru> To: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= Subject: Re: amd(8) doesn't work in 10 and CURRENT (does auromound(8) support NFS?) In-Reply-To: References: <1172266612.20140418191203@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 06:40:54 -0000 Hello, Edward. You wrote 19 =E0=EF=F0=E5=EB=FF 2014 =E3., 1:59:08: >> Does new "automound(8)", which "compatible with its counterparts in OS = X, >> Solaris, and Linux" support NFS? ETN> Well, it would be kind of pointless if it didn't, right? ;-) Why not? As far as I understand, removable storage (USB sticks, all assortment of cards and cardreaders, optical discs, external USB HDD enclosures and things like this) is much more frequent than NFS in these days :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 11:17:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 470C1E3B for ; Sat, 19 Apr 2014 11:17:26 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [84.52.89.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B270F1876 for ; Sat, 19 Apr 2014 11:17:25 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 810DB7F659; Sat, 19 Apr 2014 15:17:22 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 810DB7F659 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1397906242; bh=Y2P4WrIcKZW5zSls6oOX4/CkU/zYeCApU3PYxNI4zaY=; l=2931; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type; b=g+YYF+51M5YTwl5nNzDhauYOhAz8qyfUnEzwqRH/6R8sAjoOn56c6cHGzfsctcLkz eEsY7acDBJRpiOqxaraSWkCLkTDaXiYwWkisAvXhjHm0cV05X/OQ+CmOv8c7MsjlKs nBh3qox0giMzDB1c7wGat8H4koyS2FACG3W4Isj4= Message-ID: <53525B43.9050602@oktetlabs.ru> Date: Sat, 19 Apr 2014 15:17:23 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: aurfalien , freebsd-net@freebsd.org Subject: Re: Solarflare LACP bug? References: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> In-Reply-To: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> Content-Type: multipart/mixed; boundary="------------080705090505090908040007" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 11:17:26 -0000 This is a multi-part message in MIME format. --------------080705090505090908040007 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Hi, On 04/16/2014 11:00 PM, aurfalien wrote: > Hi, > > I’ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. > > While the card works fine as individual ports, upon configuring LACP the machine suddenly reboots. > > Here are my commands; > > ifconfig sfxge0 up > ifconfig sfxge1 up > ifconfig lagg0 create > * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 10.0.10.99/16 > > * This is were the system reboots. please, find patch attached. It solves the problem for me. I'll discuss it with Solarflare and then submit patch to be pushed to subversion. Regards, Andrew. > I believe this to be a bug, should i post this on freebsd-bugs@freebsd.org > > The only thing in /var/crash is minfree. > > - aurf > > "Janitorial Services" --------------080705090505090908040007 Content-Type: text/x-patch; name="sfxge-lag-fix.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="sfxge-lag-fix.patch" sfxge: check that port is started when MAC filter is set MAC filter set may be called without softc_lock held in the case of SIOCADDMULTI and SIOCDELMULTI ioctls. ioctl handler checks IFF_DRV_RUNNING flag which implies port started, but it is not guaranteed to remain. softc_lock shared lock can't be held in the case of these ioctls processing, since it results in failure where kernel complains that non-sleepable lock is held in sleeping thread. Both problems are repeatable on LAG with LACP proto bring up. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff -r 8dc01b10eb64 sys/dev/sfxge/sfxge_port.c --- a/sys/dev/sfxge/sfxge_port.c Tue Apr 15 10:32:43 2014 +0100 +++ b/sys/dev/sfxge/sfxge_port.c Sat Apr 19 14:49:46 2014 +0400 @@ -357,10 +357,21 @@ struct sfxge_port *port = &sc->port; int rc; - KASSERT(port->init_state == SFXGE_PORT_STARTED, ("port not started")); - mtx_lock(&port->lock); - rc = sfxge_mac_filter_set_locked(sc); + /* + * The function may be called without softc_lock held in the + * case of SIOCADDMULTI and SIOCDELMULTI ioctls. ioctl handler + * checks IFF_DRV_RUNNING flag which implies port started, but + * it is not guaranteed to remain. softc_lock shared lock can't + * be held in the case of these ioctls processing, since it + * results in failure where kernel complains that non-sleepable + * lock is held in sleeping thread. Both problems are repeatable + * on LAG with LACP proto bring up. + */ + if (port->init_state == SFXGE_PORT_STARTED) + rc = sfxge_mac_filter_set_locked(sc); + else + rc = 0; mtx_unlock(&port->lock); return rc; } --------------080705090505090908040007-- From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 13:58:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FA8DC5 for ; Sat, 19 Apr 2014 13:58:26 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 434DE1543 for ; Sat, 19 Apr 2014 13:58:26 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id ld10so2299637pab.40 for ; Sat, 19 Apr 2014 06:58:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=hcuFfEYGJLD6oiBL4rACTblslJ4GeCMfi72mvCmnV4w=; b=c/vMqueeJzuGD8n4lgMQZ0udfm+FnxOU/oE1huqbnb/AO1qcazmoJOeOyZYROYc/ou Y5X5DIRlv336NShUR4/LapFwyxIiaGKBfFRY9z7cuc4C66WThvDrL1AGOQ2rY93XzCMk RCoOLesjdAxvORzaXJELmKfu7YQj6CDV8hPdS+/EMte86CDJjSIHlOKUeflNUdb3yT+A rejtWCcRR1ZxUoz+2hOOs2rCXwTMitYGUGKyXLybYNg8S3alhP6nwnChJEFxzZhEnr+d tdHRhnmby+OCbkWpWtNIN9djasbUsOM18nqjex6UYSovRVzzy0yesTbGl4gsGlCgT2HF AhMQ== X-Received: by 10.68.132.130 with SMTP id ou2mr27660783pbb.107.1397915905362; Sat, 19 Apr 2014 06:58:25 -0700 (PDT) Received: from [10.1.21.71] ([108.175.219.7]) by mx.google.com with ESMTPSA id ff4sm157241369pad.24.2014.04.19.06.58.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Apr 2014 06:58:24 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Solarflare LACP bug? From: aurfalien In-Reply-To: <53525B43.9050602@oktetlabs.ru> Date: Sat, 19 Apr 2014 06:58:22 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <0424FE83-E147-4F76-ACC8-D838140499F4@gmail.com> References: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> <53525B43.9050602@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 13:58:26 -0000 On Apr 19, 2014, at 4:17 AM, Andrew Rybchenko = wrote: > Hi, >=20 > On 04/16/2014 11:00 PM, aurfalien wrote: >> Hi, >>=20 >> I=92ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. >>=20 >> While the card works fine as individual ports, upon configuring LACP = the machine suddenly reboots. >>=20 >> Here are my commands; >>=20 >> ifconfig sfxge0 up >> ifconfig sfxge1 up >> ifconfig lagg0 create >> * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 = 10.0.10.99/16 >>=20 >> * This is were the system reboots. > please, find patch attached. It solves the problem for me. >=20 > I'll discuss it with Solarflare and then submit patch to be pushed to = subversion. >=20 > Regards, > Andrew. Wow, that was very fast actually, many many thanks Andrew! - aurf "Janitorial Services=94= From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 14:53:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3CB869D; Sat, 19 Apr 2014 14:53:12 +0000 (UTC) Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23301195C; Sat, 19 Apr 2014 14:53:11 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id e49so2377000eek.25 for ; Sat, 19 Apr 2014 07:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=JFWxROLaicIIr23vEYl8Oln2AB/Kf9dnHrrFrIQWyY0=; b=z2JouV0cldYJD/zOV+w74e8JIq3V4MH1LKyYueOsJYsQltxufncfKcUqHrblFaRjBB YRpLTzXovdTCxObXoQ/LV8YuTEpjfbWWH54enh+vLSqLUZUeFua4i4M2SCADHwZQfp8+ TH9wymjatK3cg3ZsZNPOM8zyWogAmVwOZY5+DWI6s/H0OQ375Y5g1Mu8b7o+m5uRLYff l71oiUqEPC0ezBvczC0By9K+Du3YDbrLA/Uk3lSjROSrPm8qPlEvpjhVacimLf/relcd eCvi2/VGSwOU1k0HnG0i+Z48PAYmESZVQmuBMoeisyUI44LcyGJlQkgbvGqsDMHdAfGM 1h8g== X-Received: by 10.14.47.133 with SMTP id t5mr4181284eeb.67.1397919190341; Sat, 19 Apr 2014 07:53:10 -0700 (PDT) Received: from strashydlo.home (adge23.neoplus.adsl.tpnet.pl. [79.184.134.23]) by mx.google.com with ESMTPSA id m44sm86271100eep.14.2014.04.19.07.53.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Apr 2014 07:53:08 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: amd(8) doesn't work in 10 and CURRENT (does auromound(8) support NFS?) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=utf-8 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= X-Priority: 3 (Normal) In-Reply-To: <1021530859.20140419104035@serebryakov.spb.ru> Date: Sat, 19 Apr 2014 16:53:07 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <60A615A8-BAF5-414D-8CED-CD52FAE117DA@FreeBSD.org> References: <1172266612.20140418191203@serebryakov.spb.ru> <1021530859.20140419104035@serebryakov.spb.ru> To: lev@FreeBSD.org X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 14:53:12 -0000 Wiadomo=C5=9B=C4=87 napisana przez Lev Serebryakov w dniu 19 kwi 2014, o = godz. 08:40: > Hello, Edward. > You wrote 19 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2014 =D0=B3., = 1:59:08: >=20 >>> Does new "automound(8)", which "compatible with its counterparts in = OS X, >>> Solaris, and Linux" support NFS? > ETN> Well, it would be kind of pointless if it didn't, right? ;-) > Why not? As far as I understand, removable storage (USB sticks, all > assortment of cards and cardreaders, optical discs, external USB HDD > enclosures and things like this) is much more frequent than NFS in = these > days :) Not in corporate environments, which are the main target for the automounter. Of course those will be supported as well, but in most cases mounting removable drives is done by some other desktop mechanism. From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 17:19:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF84597A for ; Sat, 19 Apr 2014 17:19:32 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 3F2AE17FC for ; Sat, 19 Apr 2014 17:19:32 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id B33416B916E for ; Sat, 19 Apr 2014 19:19:30 +0200 (CEST) Received: from soth.ventu (adsl-ull-118-176.41-151.net24.it [151.41.176.118]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3JHJEEp090147 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Sat, 19 Apr 2014 19:19:22 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-118-176.41-151.net24.it [151.41.176.118] claimed to be soth.ventu Received: from guardian.ventu (bane.ventu [10.1.2.15]) (authenticated bits=0) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3JHJ1Qw052589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 19 Apr 2014 19:19:03 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.ventu: Host bane.ventu [10.1.2.15] claimed to be guardian.ventu Message-ID: <5352B005.8090405@netfence.it> Date: Sat, 19 Apr 2014 19:19:01 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: John Nielsen Subject: Re: Network troubles after 8.3 -> 8.4 upgrade References: <53503BC3.6040806@netfence.it> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 X-Scanned-By: MIMEDefang 2.74 on 10.1.2.13 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Sat, 19 Apr 2014 19:19:23 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 17:19:32 -0000 On 04/17/14 23:45, John Nielsen wrote: Thanks for answering John. > My first thought was TSO as well, since I've seen the symptoms you describe a few times on systems running 10.0. > Do you use IPFW or any kind of NAT on this system? Yes, I use ipfw to firewall, to divert packets to natd and also to forward specific traffic to three different gateways. I think I added a "10 allow ip from any to any" rule for a brief period of time to exclude ipfw's interference, but I'm not sure and eventually I'll try again next week. > When an application encounters a network problem, > does it report or log anything at all? Simply a timeout. > Anything in the kernel log/dmesg? I wished so; unfortunately, as I wrote, unless there's some sysctl to tweak I get no suck info. > Otherwise my only other thought would be the driver. > Can you try reverting only the em(4) driver back to 8.3? Worth a try I guess! > Build modules and a kernel without em(4) from unmodified 8.4 src, load em(4) as a module, > confirm that the problem persists. Ok, easy. > Replace the contents of src/sys/dev/e1000, src/sys/modules/em and > src/sys/conf/files with those from an 8.3 src tree (or otherwise > revert revision 247430), rebuild em module, unload/reload or reboot, > see if problem goes away. (Could be somewhat complicated by the fact > that you also have igb interfaces which also use code from the e1000 > directory, but rather than speculate I'll leave solving that as an > exercise for someone else.) Hmmm, sounds a bit complicated... would simply dropping if_em.ko in from a 8.3 box work? bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Sat Apr 19 23:05:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 476949E1 for ; Sat, 19 Apr 2014 23:05:06 +0000 (UTC) Received: from mail-yh0-x230.google.com (mail-yh0-x230.google.com [IPv6:2607:f8b0:4002:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E2361B3D for ; Sat, 19 Apr 2014 23:05:05 +0000 (UTC) Received: by mail-yh0-f48.google.com with SMTP id z6so2480057yhz.7 for ; Sat, 19 Apr 2014 16:05:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=NILHKWOnGpcHHlmAxj7LCJmg+FRkAbpe/QzFIUtgyiQ=; b=wFy3nF1dd3SYcr9L+e5HjkXww5uL3ZZW6GrNZ9cwFWxDw7mfyVh9NJ9u96aJcfn0Pk Yqi3CDvtHf6RbeZRB7iB5pAU31AKWwdsEnajighOMOz2zINvwqRBKgbiBiBRd12voiw2 7Iq1W23zLq3m65xQDr49On/iGv1zbhnffAVDo/FEMSRUnfBduE4IRQ86RqZR/GDZIV23 WKltChLddPrhPK9FOqCogmVY8G9rvUMttNBnFw6QyENyypfdUc/w5yY7Bm1/nePmCD1I awb+uKzQRyu3q0rBtDHCsJD7GQLmp0l0tBJmWeyvV3TrA0116Gl9gMbLTI3x0qpZ1RaP UXjQ== MIME-Version: 1.0 X-Received: by 10.236.0.200 with SMTP id 48mr4924881yhb.72.1397948705191; Sat, 19 Apr 2014 16:05:05 -0700 (PDT) Received: by 10.170.185.208 with HTTP; Sat, 19 Apr 2014 16:05:05 -0700 (PDT) Date: Sun, 20 Apr 2014 01:05:05 +0200 Message-ID: Subject: Random disconnects on 10.0 with mpd5/pptp + PF From: martin i To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2014 23:05:06 -0000 Hi all, I've a FreeBSD 10.0 amd64 r264316 with custom built kernel (GENERIC + PF + cryptodev/crypto/aesni). I'm using mpd5 to provide pptp access to my samba shares hosted on the very same machine. PF is used for filtering/NAT (samba is bind on custom loopback interface only). All clients are experiencing random disconnects, even during the active session. There's no indication in mpd log why the session is closed, I can only see: Apr 19 00:09:32 foxi mpd: [cloud_b-1] IFACE: Up event Apr 19 00:16:30 foxi mpd: [cloud_link-1] PPTP call terminated Devd does log a lot of events: Apr 18 00:06:01 foxi devd: Executing '/etc/pccard_ether ng0 start' Apr 18 00:14:23 foxi devd: Executing '/etc/pccard_ether ng0 start' Apr 18 00:22:46 foxi devd: Executing '/etc/pccard_ether ng0 start' Apr 18 00:33:47 foxi devd: Executing '/etc/pccard_ether ng0 start' Apr 18 00:42:09 foxi devd: Executing '/etc/pccard_ether ng0 start' On apr 17, it was 107 of these messages. Disconnects are random, but usually the session can't be kept alive for more than 10 minutes. Session is terminated even during an active session (i.e. when client is copying data, etc.). On FreeBSD 9.2 with the same configuration I've no problems. I can't figure our why is this happening. I was hoping maybe somebody did already experience the same issues? Browsing the archive the closest match to my problem is the following: http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037864.html but the patch is (so it seems) already merged to the revision I'm using. Pls what can I do better to check where the issue is coming from ? Martin From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 00:11:43 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9CCCFD39; Sun, 20 Apr 2014 00:11:43 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 71D8C110E; Sun, 20 Apr 2014 00:11:43 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K0BhnZ055676; Sun, 20 Apr 2014 00:11:43 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K0Bhvv055675; Sun, 20 Apr 2014 00:11:43 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 00:11:43 GMT Message-Id: <201404200011.s3K0Bhvv055675@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/167793: [arp] mbuf leak if arp address is multicast X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 00:11:43 -0000 Old Synopsis: mbuf leak if arp address is multicast New Synopsis: [arp] mbuf leak if arp address is multicast Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 00:11:03 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=167793 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 00:37:22 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E3F920C; Sun, 20 Apr 2014 00:37:22 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6BF612B0; Sun, 20 Apr 2014 00:37:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K0bL7I063267; Sun, 20 Apr 2014 00:37:21 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K0bLSp063266; Sun, 20 Apr 2014 00:37:21 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 00:37:21 GMT Message-Id: <201404200037.s3K0bLSp063266@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/187980: [igmp] IGMP query not responded to between 2 FreeBSD hosts X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 00:37:22 -0000 Old Synopsis: Don't respod an IGMP query New Synopsis: [igmp] IGMP query not responded to between 2 FreeBSD hosts Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 00:36:15 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=187980 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 02:14:58 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E06E12EE; Sun, 20 Apr 2014 02:14:58 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B48DC19D1; Sun, 20 Apr 2014 02:14:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K2Ewaj097758; Sun, 20 Apr 2014 02:14:58 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K2EwsD097757; Sun, 20 Apr 2014 02:14:58 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 02:14:58 GMT Message-Id: <201404200214.s3K2EwsD097757@freefall.freebsd.org> To: jjasen@gmail.com, linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/183970: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 02:14:59 -0000 Old Synopsis: mellenox drivers and vlan usage causes kernel panic and reboot New Synopsis: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot State-Changed-From-To: open->open State-Changed-By: linimon State-Changed-When: Sun Apr 20 01:48:45 UTC 2014 State-Changed-Why: reclassify and assign. Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 01:48:45 UTC 2014 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=183970 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 02:53:17 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEFEADF4; Sun, 20 Apr 2014 02:53:17 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 840661CD3; Sun, 20 Apr 2014 02:53:17 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K2rHTl012295; Sun, 20 Apr 2014 02:53:17 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K2rHC6012294; Sun, 20 Apr 2014 02:53:17 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 02:53:17 GMT Message-Id: <201404200253.s3K2rHC6012294@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/168440: [ixgbe] [patch] ixgbe flow control tunable regression X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 02:53:17 -0000 Old Synopsis: ixgbe flow control tunable regression New Synopsis: [ixgbe] [patch] ixgbe flow control tunable regression Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 02:53:02 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=168440 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 02:54:03 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2967AED1; Sun, 20 Apr 2014 02:54:03 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F18051CE3; Sun, 20 Apr 2014 02:54:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K2s2qI012346; Sun, 20 Apr 2014 02:54:02 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K2s24X012345; Sun, 20 Apr 2014 02:54:02 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 02:54:02 GMT Message-Id: <201404200254.s3K2s24X012345@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/168414: [ixgbe] [patch] ixgbe commit r234137 introduces code/comment mismatch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 02:54:03 -0000 Old Synopsis: ixgbe commit r232874 introduces code/comment mismatch New Synopsis: [ixgbe] [patch] ixgbe commit r234137 introduces code/comment mismatch Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 02:53:27 UTC 2014 Responsible-Changed-Why: reclassify. http://www.freebsd.org/cgi/query-pr.cgi?pr=168414 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 03:21:06 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99C6B5D0; Sun, 20 Apr 2014 03:21:06 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C35410DB; Sun, 20 Apr 2014 03:21:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K3L6nh020702; Sun, 20 Apr 2014 03:21:06 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K3L6xX020701; Sun, 20 Apr 2014 03:21:06 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 03:21:06 GMT Message-Id: <201404200321.s3K3L6xX020701@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/185996: [ip6] For IPv6, ipsec_address returns pointer to corrupted data X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 03:21:06 -0000 Old Synopsis: For IPv6, ipsec_address returns pointer to corrupted data New Synopsis: [ip6] For IPv6, ipsec_address returns pointer to corrupted data Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 03:20:24 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=185996 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 03:22:26 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86C6A6E5; Sun, 20 Apr 2014 03:22:26 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B9EF1165; Sun, 20 Apr 2014 03:22:26 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K3MQqb023365; Sun, 20 Apr 2014 03:22:26 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K3MQC7023364; Sun, 20 Apr 2014 03:22:26 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 03:22:26 GMT Message-Id: <201404200322.s3K3MQC7023364@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/185967: [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 03:22:26 -0000 Old Synopsis: Link Aggregation LAGG: LACP not working in 10.0 New Synopsis: [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 03:22:00 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=185967 From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 16:05:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA3BB168 for ; Sun, 20 Apr 2014 16:05:22 +0000 (UTC) Received: from lagoon.freebsd.lublin.pl (lagoon.freebsd.lublin.pl [IPv6:2a02:2928:a::3]) by mx1.freebsd.org (Postfix) with ESMTP id 293841B36 for ; Sun, 20 Apr 2014 16:05:22 +0000 (UTC) Received: from [10.66.254.3] (mgmt.nette.pl [193.138.118.12]) by lagoon.freebsd.lublin.pl (Postfix) with ESMTPSA id 3EE79479566; Sun, 20 Apr 2014 18:04:45 +0200 (CEST) Message-ID: <5353F041.4080307@frasunek.com> Date: Sun, 20 Apr 2014 18:05:21 +0200 From: Przemyslaw Frasunek Organization: frasunek.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Mike Tancsa , freebsd-net@freebsd.org Subject: Re: mpd5/Netgraph issues after upgrading to 7.4 References: <20110513162311.GK95084@glebius.int.ru> <4DD298AD.2060905@frasunek.com> <20110517184613.GN74366@glebius.int.ru> <4FDB1D71.6050908@freebsd.lublin.pl> <20120615203142.GW28613@glebius.int.ru> <4FDBAFD7.9020606@freebsd.lublin.pl> <4FDF2F81.6030307@sentex.net> <4FDF3097.6080701@freebsd.lublin.pl> <4FE0EE62.5070905@freebsd.lublin.pl> <4FF7F2C6.5070401@freebsd.lublin.pl> <20120709081225.GJ21957@glebius.int.ru> <4FFBB7A8.90201@rdtc.ru> <4FFBCA96.3000605@freebsd.lublin.pl> <50032E10.3060607@sentex.net> <65702E3D-9373-4CC7-9CB4-81704EF44ED8@lists.zabbadoz.net> <50039923.2060802@rdtc.ru> <530085F3.9020205@frasunek.com> <53017B12.9050304@sentex.net> <531C517F.5020506@frasunek.com> <531C5877.8050401@sentex.net> <532335B6.7070105@frasunek.com> <534D4D72.8040807@sentex.net> In-Reply-To: <534D4D72.8040807@sentex.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 16:05:22 -0000 >>>> FYI -- after upgrade to 9-STABLE no further crashes occurred, even with IPv6 >>>> enabled. >>> What sort of uptime have you seen with ipv6 enabled ? >> Now it's 19 days and still no crash occurred. > Has all been stable still with ipv6 enabled ? Unfortunately, it crashed after ~30 days of uptime, but in different way than before: (kgdb) bt #0 doadump (textdump=) at pcpu.h:234 #1 0xffffffff80593466 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff80593967 in panic (fmt=0x1
) at /usr/src/sys/kern/kern_shutdown.c:637 #3 0xffffffff8082ba40 in trap_fatal (frame=0xc, eva=) at /usr/src/sys/amd64/amd64/trap.c:879 #4 0xffffffff8082bda1 in trap_pfault (frame=0xffffff8098c38b30, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 #5 0xffffffff8082c354 in trap (frame=0xffffff8098c38b30) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff80815683 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff8064642a in strlcpy (dst=0xffffff8098c38e36 "", src=0x0, siz=) at /usr/src/sys/libkern/strlcpy.c:39 #8 0xffffffff805831fe in fill_kinfo_thread (td=0xfffffe006a907920, kp=0xffffff8098c38c80, preferthread=0) at /usr/src/sys/kern/kern_proc.c:961 #9 0xffffffff8058410b in fill_kinfo_proc (p=0xfffffe006a9004a8, kp=0xffffff8098c38c80) at /usr/src/sys/kern/kern_proc.c:1033 #10 0xffffffff8058439d in kern_proc_out (p=0xfffffe006a9004a8, sb=0xffffff8098c39420, flags=1) at /usr/src/sys/kern/kern_proc.c:1208 #11 0xffffffff8058663d in sysctl_out_proc (p=0xfffffe006a9004a8, req=, flags=1, doingzomb=0) at /usr/src/sys/kern/kern_proc.c:1247 #12 0xffffffff80589019 in sysctl_kern_proc (oidp=, arg1=0xffffff8098c39a7c, arg2=, req=0xffffff8098c399b0) at /usr/src/sys/kern/kern_proc.c:1430 #13 0xffffffff8059e4a7 in sysctl_root (oidp=, arg1=0xffffff8098c39a7c, arg2=0, req=0xffffff8098c399b0) at /usr/src/sys/kern/kern_sysctl.c:1493 #14 0xffffffff8059e785 in userland_sysctl (td=, name=0xffffff8098c39a70, namelen=3, old=, oldlenp=0x0, inkernel=, new=0x0, newlen=0, retval=0xffffff8098c39ad8, flags=0) at /usr/src/sys/kern/kern_sysctl.c:1603 #15 0xffffffff8059ec9a in sys___sysctl (td=0xfffffe000b768490, uap=0xffffff8098c39bb0) at /usr/src/sys/kern/kern_sysctl.c:1529 #16 0xffffffff8082b1ea in amd64_syscall (td=0xfffffe000b768490, traced=0) at subr_syscall.c:135 #17 0xffffffff80815967 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:391 #18 0x0000000800d5413c in ?? () (kgdb) frame 8 #8 0xffffffff805831fe in fill_kinfo_thread (td=0xfffffe006a907920, kp=0xffffff8098c38c80, preferthread=0) at /usr/src/sys/kern/kern_proc.c:961 961 strlcpy(kp->ki_lockname, td->td_lockname, (kgdb) list 956 else 957 bzero(kp->ki_wmesg, sizeof(kp->ki_wmesg)); 958 strlcpy(kp->ki_tdname, td->td_name, sizeof(kp->ki_tdname)); 959 if (TD_ON_LOCK(td)) { 960 kp->ki_kiflag |= KI_LOCKBLOCK; 961 strlcpy(kp->ki_lockname, td->td_lockname, 962 sizeof(kp->ki_lockname)); 963 } else { 964 kp->ki_kiflag &= ~KI_LOCKBLOCK; 965 bzero(kp->ki_lockname, sizeof(kp->ki_lockname)); (kgdb) print td->td_lockname $3 = 0x0 (kgdb) print *td $4 = {td_lock = 0xfffffe003a3e6d80, td_proc = 0xfffffe006a9004a8, td_plist = {tqe_next = 0x0, tqe_prev = 0xfffffe006a9004b8}, td_runq = {tqe_next = 0x0, tqe_prev = 0xffffffff80c648a8}, td_slpq = { tqe_next = 0x0, tqe_prev = 0xfffffe0001b06d00}, td_lockq = {tqe_next = 0x0, tqe_prev = 0xfffffe0001780960}, td_hash = {le_next = 0x0, le_prev = 0xffffff80006918b8}, td_cpuset = 0xfffffe00014d2dc8, td_sel = 0x0, td_sleepqueue = 0xfffffe0001b06d00, td_turnstile = 0x0, td_umtxq = 0xfffffe0001b25d80, td_tid = 100119, td_sigqueue = {sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xfffffe006a9079d0}, sq_proc = 0xfffffe006a9004a8, sq_flags = 1}, td_lend_user_pri = 255 'ïŋ―', td_flags = 5, td_inhibitors = 8, td_pflags = 0, td_dupfd = 0, td_sqqueue = 0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 1 '\001', td_oncpu = 255 'ïŋ―', td_owepreempt = 0 '\0', td_tsqueue = 1 '\001', td_locks = 1, td_rw_rlocks = 0, td_lk_slocks = 0, td_stopsched = 0, td_blocked = 0xfffffe003a3e6d80, td_lockname = 0x0, td_contested = {lh_first = 0xfffffe0054d89300}, td_sleeplocks = 0x0, td_intr_nesting_level = 0, td_pinned = 0, td_ucred = 0xfffffe00014d4a00, td_estcpu = 0, td_slptick = 0, td_blktick = 584456809, td_swvoltick = 584456809, td_cow = 6, td_ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 4264, ru_ixrss = 36800, ru_idrss = 4673600, ru_isrss = 294400, ru_minflt = 688, ru_majflt = 0, ru_nswap = 0, ru_inblock = 0, ru_oublock = 0, ru_msgsnd = 633119, ru_msgrcv = 189464, ru_nsignals = 0, ru_nvcsw = 367402, ru_nivcsw = 50322}, td_rux = {rux_runtime = 49823813588, rux_uticks = 601, rux_sticks = 1699, rux_iticks = 0, rux_uu = 3570333, rux_su = 8456053, rux_tu = 12026387}, td_incruntime = 0, td_runtime = 49823813588, td_pticks = 0, td_sticks = 0, td_iticks = 0, td_uticks = 0, td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_sigmask = {__bits = {0, 0, 0, 0}}, td_generation = 417724, td_sigstk = {ss_sp = 0x0, ss_size = 0, ss_flags = 4}, td_xsig = 0, td_profil_addr = 0, td_profil_ticks = 0, td_name = "drad", '\0' , td_fpop = 0x0, td_dbgflags = 0, td_dbgksi = {ksi_link = {tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0, si_errno = 0, si_code = 0, si_pid = 0, si_uid = 0, si_status = 0, si_addr = 0x0, si_value = {sival_int = 0, sival_ptr = 0x0, sigval_int = 0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0}, _timer = {_timerid = 0, _overrun = 0}, _mesgq = {_mqd = 0}, _poll = {_band = 0}, __spare__ = {__spare1__ = 0, __spare2__ = {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags = 0, ksi_sigq = 0x0}, td_ng_outbound = 0, td_osd = {osd_nslots = 0, osd_slots = 0x0, osd_next = { le_next = 0x0, le_prev = 0x0}}, td_map_def_user = 0x0, td_dbg_forked = 0, td_rqindex = 30 '\036', td_base_pri = 120 'x', td_priority = 28 '\034', td_pri_class = 3 '\003', td_user_pri = 120 'x', td_base_user_pri = 120 'x', td_pcb = 0xffffff8098bb7d00, td_state = TDS_INHIBITED, td_retval = {0, 0}, td_slpcallout = {c_links = {sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xffffff800072c690}}, c_time = 584456809, c_arg = 0xfffffe006a907920, c_func = 0xffffffff805d8c00 , c_lock = 0x0, c_flags = 18, c_cpu = 2}, td_frame = 0xffffff8098bb7c40, td_kstack_obj = 0xfffffe0001b2b658, td_kstack = 18446743526516146176, td_kstack_pages = 4, td_critnest = 1, td_md = {md_spinlock_count = 1, md_saved_flags = 582, md_spurflt_addr = 0}, td_sched = 0xfffffe006a907d80, td_ar = 0x0, td_lprof = {{ lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0xfffffe0054c5ab00, td_errno = 0, td_vnet = 0x0, td_vnet_lpush = 0x0, td_intr_frame = 0x0, td_rfppwait_p = 0x0, td_ma = 0x0, td_ma_cnt = 0, td_rlqe = 0x0, td_vp_reserv = 0} From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 17:26:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59944EBA; Sun, 20 Apr 2014 17:26:55 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA676123F; Sun, 20 Apr 2014 17:26:54 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id m20so3260894qcx.7 for ; Sun, 20 Apr 2014 10:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=ONMoUIYbhdZ2aPsLEhXbAxPxtz6//GeiYUY4PUKoT/E=; b=V+1OAQY9mjEW48IalZN4N7B6kv5U0p0fwHwpDRflVNSqDtwadAcfxIFHDc3l4ot4j6 EOCnp46BZm8PfkzdNDxH2Xri+3/vevfDQFrZAjes9+nCYnvhfXIEiI3m1t3ff54B0sfu apJREalBDTpNd/7mn9Y43uoioTnyZjaFHFdKEdozOyyfyA+8TX1HuLHmZ+mu/+N0s2Sf Je+keyKnPGGurjdYm4Bbp+BVi2doKziSY/MbifuTuh5zZ8BvmlXw70Ky5jYU7LvFIKwJ faExLW02xRM6WS/2s0B1jBPNzvv72feDyEJcrQXyS0phphM5y8SkIDKCM7hEDhMBye1A J/nQ== X-Received: by 10.140.37.135 with SMTP id r7mr23730961qgr.61.1398014814072; Sun, 20 Apr 2014 10:26:54 -0700 (PDT) Received: from [10.0.0.215] (pool-96-244-212-235.bltmmd.fios.verizon.net. [96.244.212.235]) by mx.google.com with ESMTPSA id w2sm68597546qar.21.2014.04.20.10.26.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 20 Apr 2014 10:26:50 -0700 (PDT) Message-ID: <53540357.3000109@gmail.com> Date: Sun, 20 Apr 2014 13:26:47 -0400 From: John Jasen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org Subject: Re: kern/183970: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot References: <201404200214.s3K2EwsD097757@freefall.freebsd.org> In-Reply-To: <201404200214.s3K2EwsD097757@freefall.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 17:26:55 -0000 I've not checked 9.2, but the 181931 patch has been applied to 10-0-release, and it also fixes my problem. On 04/19/2014 10:14 PM, linimon@FreeBSD.org wrote: > Old Synopsis: mellenox drivers and vlan usage causes kernel panic and reboot > New Synopsis: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot > > State-Changed-From-To: open->open > State-Changed-By: linimon > State-Changed-When: Sun Apr 20 01:48:45 UTC 2014 > State-Changed-Why: > reclassify and assign. > > > Responsible-Changed-From-To: freebsd-bugs->freebsd-net > Responsible-Changed-By: linimon > Responsible-Changed-When: Sun Apr 20 01:48:45 UTC 2014 > Responsible-Changed-Why: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=183970 > From owner-freebsd-net@FreeBSD.ORG Sun Apr 20 23:30:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D96E8F9 for ; Sun, 20 Apr 2014 23:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A6C31015 for ; Sun, 20 Apr 2014 23:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3KNU1h0031154 for ; Sun, 20 Apr 2014 23:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3KNU1xU031153; Sun, 20 Apr 2014 23:30:01 GMT (envelope-from gnats) Date: Sun, 20 Apr 2014 23:30:01 GMT Message-Id: <201404202330.s3KNU1xU031153@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Mark Linimon Subject: Re: kern/183970: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Mark Linimon List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 23:30:01 -0000 The following reply was made to PR kern/183970; it has been noted by GNATS. From: Mark Linimon To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/183970: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot Date: Sun, 20 Apr 2014 18:23:44 -0500 ----- Forwarded message from John Jasen ----- Date: Sun, 20 Apr 2014 13:26:47 -0400 From: John Jasen To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org Subject: Re: kern/183970: [ofed] [vlan] [panic] mellanox drivers and vlan usage causes kernel panic and reboot User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 I've not checked 9.2, but the 181931 patch has been applied to 10-0-release, and it also fixes my problem. ----- End forwarded message ----- From owner-freebsd-net@FreeBSD.ORG Mon Apr 21 02:23:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47299A2F; Mon, 21 Apr 2014 02:23:53 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBC541DDB; Mon, 21 Apr 2014 02:23:52 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so3582535qgd.29 for ; Sun, 20 Apr 2014 19:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Uc9f6V6MZj+ls5t47xcd1AxwQnvLYDKz9mbGNotiSJ8=; b=fMLhcLqSZU2JqlFP0bWsPP7oKA7Qm6FeLF28kcgz6kMN8czJuy82nOODdG13ma+PyG 7hlTgNcXwRgnQMxFMUPjaGI5F7QS7jglC54uvYaLqowrnupUQIchmOv4DBe8KwztSAWG lwkGQyYmpnV5ciscFV/TlV0aiKquENPh7+ayU6UlkvgY8AnxHaHP89IzJjcTXkY1DN5S m6H5zdqalDVbpxHpe68e2gps3Bp1omphLylj/TCA3FaLC68MB58HGszFC0fRj3pXwJMk d4BzwFQMINzt4J62xlFo3IVqmEHcY1HeSriYHUWYwPOq9LS0C+tgNSVSe6X0lmpAImoR W7wA== MIME-Version: 1.0 X-Received: by 10.140.43.135 with SMTP id e7mr498160qga.95.1398047032073; Sun, 20 Apr 2014 19:23:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 20 Apr 2014 19:23:52 -0700 (PDT) In-Reply-To: References: Date: Sun, 20 Apr 2014 19:23:52 -0700 X-Google-Sender-Auth: SHCpCI0RKUeLkcKBTy79MtJAMro Message-ID: Subject: Re: Getting VNET/VIMAGE across the finish line From: Adrian Chadd To: Kevin Bowling Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 02:23:53 -0000 On 20 April 2014 18:56, Kevin Bowling wrote: > Hi, > > I'd like to see VNET/VIMAGE get added to GENERIC in -CURRENT and help stamp > out any remaining issues. > > I'm aware of two broad problems at the moment: > * http://www.freebsd.org/cgi/query-pr.cgi?pr=164763&cat= > * pf related things which seem to be getting addressed > > Is anyone tracking other issues? No, but it doesn't mean there aren't any. Try hotswapping devices on your PC - build everything as modules, then just decide to unload them with active interfaces on them. See if that makes your machine unhappy. Same with USB things. -a From owner-freebsd-net@FreeBSD.ORG Mon Apr 21 11:06:50 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5ACB9F for ; Mon, 21 Apr 2014 11:06:50 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 911E71964 for ; Mon, 21 Apr 2014 11:06:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3LB6oPE085784 for ; Mon, 21 Apr 2014 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3LB6o2C085782 for freebsd-net@FreeBSD.org; Mon, 21 Apr 2014 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 21 Apr 2014 11:06:50 GMT Message-Id: <201404211106.s3LB6o2C085782@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 11:06:50 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167793 net [arp] mbuf leak if arp address is multicast o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 475 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 21 12:10:41 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FEB1808; Mon, 21 Apr 2014 12:10:41 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 340681121; Mon, 21 Apr 2014 12:10:41 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3LCAfPa010664; Mon, 21 Apr 2014 12:10:41 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3LCAeiU010663; Mon, 21 Apr 2014 12:10:40 GMT (envelope-from ae) Date: Mon, 21 Apr 2014 12:10:40 GMT Message-Id: <201404211210.s3LCAeiU010663@freefall.freebsd.org> To: samspeed@mail.ru, ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/167793: [arp] mbuf leak if arp address is multicast X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 12:10:41 -0000 Synopsis: [arp] mbuf leak if arp address is multicast State-Changed-From-To: open->feedback State-Changed-By: ae State-Changed-When: Mon Apr 21 12:08:23 UTC 2014 State-Changed-Why: Do you able reproduce that? It seems the problem was fixed in r249742. Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Mon Apr 21 12:08:23 UTC 2014 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=167793 From owner-freebsd-net@FreeBSD.ORG Tue Apr 22 16:37:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AB1EAEF; Tue, 22 Apr 2014 16:37:30 +0000 (UTC) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AAF41CE2; Tue, 22 Apr 2014 16:37:29 +0000 (UTC) Received: from mh0.gentlemail.de (mh0.gentlemail.de [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id s3MGbLER071748; Tue, 22 Apr 2014 18:37:26 +0200 (CEST) (envelope-from h.schmalzbauer@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 069C53507; Tue, 22 Apr 2014 18:37:20 +0200 (CEST) Message-ID: <53569ABA.60007@omnilan.de> Date: Tue, 22 Apr 2014 18:37:14 +0200 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Deleting IPv4 iface-routes from extra FIBs X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig12E99B1A277853E263555BCD" X-Greylist: ACL 119 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Tue, 22 Apr 2014 18:37:26 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 16:37:30 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig12E99B1A277853E263555BCD Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, here, http://svnweb.freebsd.org/base?view=3Drevision&revision=3D248895 interface route protection was added (so the following problem arose with 9.2). Unfortunately, in my case, I must be able to delete these routes; not in the default FIB, but in jail's fibs, because: =C2=B7 Host is multihomed with multiple nics in different subnets. =C2=B7 Jail's IP (no vnet) is from a different subnet than host's default-router subnet =E2=80=93 jail has no ip in the range of host's default-router!!! =C2=B7 FIB used by jail contains valid default-router. Problem: If iface-routes exist in jail's FIB, answer-packets take the iface-shortcut, not trespassing the router (default gateway); hence 3way-handshake never finishes and firewall terminates (half-opened) TCP sessions. Workarround: =C2=B7 Abuse packet filter doing some kind of route-to=E2=80=A6 =C2=B7 Revert r248895, to be able to delete v4-iface-routes (inet6-routes= can be deleted without any hack) Desired solution: =C2=B7 Allow deletion of v4-iface-routes if FIB!=3D0. Unfortunately my C skills don't allow me to implement this myself :-( I can't even follow the code, I guess that was originally considered, but possibly doesn't work bacause of a simple bug?!? I took the lazy way and simply reverted r248895 instead of trying to understand rtrequest1_fib(). I wish I had the time to learn=E2=80=A6 Thanks for any help, -Harry --------------enig12E99B1A277853E263555BCD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlNWmsAACgkQLDqVQ9VXb8gAKACgowI4hoEKxrcWp0DrnUv+dXQS Nx4AoLJV8GyX4g0xPA5MIv1v1qOTaCOJ =CDJ2 -----END PGP SIGNATURE----- --------------enig12E99B1A277853E263555BCD-- From owner-freebsd-net@FreeBSD.ORG Tue Apr 22 19:07:50 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BCF2852; Tue, 22 Apr 2014 19:07:50 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 605381E3E; Tue, 22 Apr 2014 19:07:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3MJ7oD4067587; Tue, 22 Apr 2014 19:07:50 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3MJ7oP1067586; Tue, 22 Apr 2014 19:07:50 GMT (envelope-from linimon) Date: Tue, 22 Apr 2014 19:07:50 GMT Message-Id: <201404221907.s3MJ7oP1067586@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 19:07:50 -0000 Old Synopsis: dc ethernet driver seems to prevent the detection of other Nic chipsets New Synopsis: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Tue Apr 22 19:07:30 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=188897 From owner-freebsd-net@FreeBSD.ORG Tue Apr 22 20:38:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E343DA06; Tue, 22 Apr 2014 20:38:15 +0000 (UTC) Received: from mail-vc0-x233.google.com (mail-vc0-x233.google.com [IPv6:2607:f8b0:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F163192B; Tue, 22 Apr 2014 20:38:15 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id ij19so13295vcb.10 for ; Tue, 22 Apr 2014 13:38:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=24if34ULF+eRazH1AbbP1pbCCKgQ6qRGJYWOVqgaKno=; b=yvvcXFvt0iAeQiWN0HmgrN9N+zeyHwPqYzn6RpWUlLsAiT6LFBevCV/itnQvZz5w0X mTcxNv8Rb76/iWN20v8bzqOuzFJ6kGKbFoqm0qwO5OEvI7FTdLpirnQU0yRdJlWoY5Ge bfpMuSx3Lk4gfk1Cj86yzA3WOk9sLx4kyjVIKW7p968aqTSRT+28CKzO7Lcm3b6G/+To yKFTsBx1PR9x5oQtCKL1ZUMxee3RzEs4AQODScAsdzm9TCFniSh4MdcrecIIMWk0R4X/ ZpxSNuEvR6Cae+TTfrZ1m1ag7GRcJr0RW93l/EbHdmafWAOC0/HO7Zfw36ZGMzpr8SdC iwIA== MIME-Version: 1.0 X-Received: by 10.58.1.97 with SMTP id 1mr10432392vel.23.1398199094683; Tue, 22 Apr 2014 13:38:14 -0700 (PDT) Sender: ndenev@gmail.com Received: by 10.220.78.84 with HTTP; Tue, 22 Apr 2014 13:38:14 -0700 (PDT) In-Reply-To: <53569ABA.60007@omnilan.de> References: <53569ABA.60007@omnilan.de> Date: Tue, 22 Apr 2014 21:38:14 +0100 X-Google-Sender-Auth: 6S3gJSAXtC-aZDlzz7X0OjXA-O8 Message-ID: Subject: Re: Deleting IPv4 iface-routes from extra FIBs From: Nikolay Denev To: Harald Schmalzbauer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 20:38:16 -0000 On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer wrote: > Hello, > > here, http://svnweb.freebsd.org/base?view=3Drevision&revision=3D248895 > interface route protection was added (so the following problem arose > with 9.2). > > Unfortunately, in my case, I must be able to delete these routes; not in > the default FIB, but in jail's fibs, because: > =C2=B7 Host is multihomed with multiple nics in different subnets. > =C2=B7 Jail's IP (no vnet) is from a different subnet than host's > default-router subnet =E2=80=93 jail has no ip in the range of host's > default-router!!! > =C2=B7 FIB used by jail contains valid default-router. > > Problem: > If iface-routes exist in jail's FIB, answer-packets take the > iface-shortcut, not trespassing the router (default gateway); hence > 3way-handshake never finishes and firewall terminates (half-opened) TCP > sessions. > > Workarround: > =C2=B7 Abuse packet filter doing some kind of route-to=E2=80=A6 > =C2=B7 Revert r248895, to be able to delete v4-iface-routes (inet6-routes= can > be deleted without any hack) > > Desired solution: > =C2=B7 Allow deletion of v4-iface-routes if FIB!=3D0. > > Unfortunately my C skills don't allow me to implement this myself :-( > I can't even follow the code, I guess that was originally considered, > but possibly doesn't work bacause of a simple bug?!? I took the lazy way > and simply reverted r248895 instead of trying to understand > rtrequest1_fib(). I wish I had the time to learn=E2=80=A6 > > Thanks for any help, > > -Harry > Hi, As it was suggested before as immediate workaround you can set net.add_addr_allfibs=3D0 so that the interface routes are added only in the default FIB. --Nikolay From owner-freebsd-net@FreeBSD.ORG Tue Apr 22 22:03:31 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EDEEE6; Tue, 22 Apr 2014 22:03:31 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FD18133E; Tue, 22 Apr 2014 22:03:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3MM3Ufl026738; Tue, 22 Apr 2014 22:03:30 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3MM3UAG026737; Tue, 22 Apr 2014 22:03:30 GMT (envelope-from linimon) Date: Tue, 22 Apr 2014 22:03:30 GMT Message-Id: <201404222203.s3MM3UAG026737@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/188899: [cas] cas ethernet driver seems to have issues with some multiport card and mother board combinations X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 22:03:31 -0000 Old Synopsis: cas ethernet driver seems to have issues with some multiport card and mother board combinations New Synopsis: [cas] cas ethernet driver seems to have issues with some multiport card and mother board combinations Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Tue Apr 22 22:02:37 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=188899 From owner-freebsd-net@FreeBSD.ORG Wed Apr 23 07:55:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AA06155; Wed, 23 Apr 2014 07:55:59 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 51EC21D90; Wed, 23 Apr 2014 07:55:59 +0000 (UTC) Received: from Julian-MBP3.local (gw2.metromesh.com.au [110.5.117.243]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3N7taBW046816 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 23 Apr 2014 00:55:39 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <535771F3.4070007@freebsd.org> Date: Wed, 23 Apr 2014 15:55:31 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nikolay Denev , Harald Schmalzbauer Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: "freebsd-net@freebsd.org" , FreeBSD X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 07:55:59 -0000 On 4/23/14, 4:38 AM, Nikolay Denev wrote: > On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer > wrote: >> Hello, >> >> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >> interface route protection was added (so the following problem arose >> with 9.2). >> >> Unfortunately, in my case, I must be able to delete these routes; not in >> the default FIB, but in jail's fibs, because: >> · Host is multihomed with multiple nics in different subnets. >> · Jail's IP (no vnet) is from a different subnet than host's >> default-router subnet – jail has no ip in the range of host's >> default-router!!! >> · FIB used by jail contains valid default-router. >> >> Problem: >> If iface-routes exist in jail's FIB, answer-packets take the >> iface-shortcut, not trespassing the router (default gateway); hence >> 3way-handshake never finishes and firewall terminates (half-opened) TCP >> sessions. >> >> Workarround: >> · Abuse packet filter doing some kind of route-toâ€Ķ >> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes can >> be deleted without any hack) >> >> Desired solution: >> · Allow deletion of v4-iface-routes if FIB!=0. >> >> Unfortunately my C skills don't allow me to implement this myself :-( >> I can't even follow the code, I guess that was originally considered, >> but possibly doesn't work bacause of a simple bug?!? I took the lazy way >> and simply reverted r248895 instead of trying to understand >> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >> >> Thanks for any help, >> >> -Harry >> > Hi, > > As it was suggested before as immediate workaround you can set > net.add_addr_allfibs=0 so that the interface routes are added only in > the default FIB. yes, we made two behaviours. Add interface routes to all active FIBS or only add them to the first fib and let the user populate other fibs as needed. It appears you want the second behaviour, so I suggest you use that option and set up all your routes manually. > > --Nikolay > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Wed Apr 23 22:18:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D890282 for ; Wed, 23 Apr 2014 22:18:00 +0000 (UTC) Received: from smtp.webfaction.com (mail6.webfaction.com [74.55.86.74]) by mx1.freebsd.org (Postfix) with ESMTP id EA2AA1E63 for ; Wed, 23 Apr 2014 22:17:59 +0000 (UTC) Received: from [192.168.192.72] (unknown [120.136.4.251]) by smtp.webfaction.com (Postfix) with ESMTP id 781B020C07BC for ; Wed, 23 Apr 2014 21:56:04 +0000 (UTC) Message-ID: <535836F1.5070508@nevermind.co.nz> Date: Thu, 24 Apr 2014 09:56:01 +1200 From: Chris Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> In-Reply-To: <535771F3.4070007@freebsd.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 22:18:00 -0000 On 23/04/14 19:55, Julian Elischer wrote: > On 4/23/14, 4:38 AM, Nikolay Denev wrote: >> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >> wrote: >>> Hello, >>> >>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >>> interface route protection was added (so the following problem arose >>> with 9.2). >>> >>> Unfortunately, in my case, I must be able to delete these routes; >>> not in >>> the default FIB, but in jail's fibs, because: >>> · Host is multihomed with multiple nics in different subnets. >>> · Jail's IP (no vnet) is from a different subnet than host's >>> default-router subnet – jail has no ip in the range of host's >>> default-router!!! >>> · FIB used by jail contains valid default-router. >>> >>> Problem: >>> If iface-routes exist in jail's FIB, answer-packets take the >>> iface-shortcut, not trespassing the router (default gateway); hence >>> 3way-handshake never finishes and firewall terminates (half-opened) TCP >>> sessions. >>> >>> Workarround: >>> · Abuse packet filter doing some kind of route-toâ€Ķ >>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes >>> can >>> be deleted without any hack) >>> >>> Desired solution: >>> · Allow deletion of v4-iface-routes if FIB!=0. >>> >>> Unfortunately my C skills don't allow me to implement this myself :-( >>> I can't even follow the code, I guess that was originally considered, >>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>> way >>> and simply reverted r248895 instead of trying to understand >>> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >>> >>> Thanks for any help, >>> >>> -Harry >>> >> Hi, >> >> As it was suggested before as immediate workaround you can set >> net.add_addr_allfibs=0 so that the interface routes are added only in >> the default FIB. > > yes, we made two behaviours. > Add interface routes to all active FIBS or only add them to the first > fib and let the user populate other fibs as needed. > It appears you want the second behaviour, so I suggest you use that > option and set up all your routes manually. > Ah, this explains a thing or two. So when allfibs=0 and an interface is bought up, it's added to the first FIB automatically (and cannot be removed). Is there a way to change which fib the interface route is bought up on? I tried to 'setfib x ifconfig ....' which didn't work. Failing that, is there a way to change the systems global FIB without having to run every service with setfib? Basically, the behavour I want is for interface routes to be bought up on NO fibs, and manually add them to the fibs I need it on. >> >> --Nikolay >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Apr 23 23:16:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7811FEA1 for ; Wed, 23 Apr 2014 23:16:56 +0000 (UTC) Received: from mail-yh0-x22f.google.com (mail-yh0-x22f.google.com [IPv6:2607:f8b0:4002:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E8651512 for ; Wed, 23 Apr 2014 23:16:56 +0000 (UTC) Received: by mail-yh0-f47.google.com with SMTP id 29so1527200yhl.20 for ; Wed, 23 Apr 2014 16:16:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6084h9Q80BrxIWBtYPxTiz5amAvXuG04lBXx0czIEFs=; b=Pusww9WktJ95t+j//HhuCCnTkHkYTdGJp9zWbXyT6tw1aFiSXHCn3+P8SBuGwlJ8uZ GTIIGlhEoWBrgBzyZRjWa0et4IvDIbF9uo7AQGTaPVGqh42h/VuMJwvycW2Lmz40jXjL 9n0eOHv59brKGbySaI0IKIum95cV4IbkP++Kxvqpx50reIap5Ft9LR/DMdMoNZke3bvK p9FOSPkb1poFZKrqNLOurmkynJ6jrg5+j/EYK9gAploLV9f59ys6pXqqz60oZVb7YI77 YgY3u8uiOWG4KJk8Ss+qSn/AS1I4q57a5y6lRE9vodTDpY7dCFZz2gooYjJGSF1CRrDQ iEbQ== MIME-Version: 1.0 X-Received: by 10.236.31.40 with SMTP id l28mr73904485yha.17.1398295015421; Wed, 23 Apr 2014 16:16:55 -0700 (PDT) Received: by 10.170.185.208 with HTTP; Wed, 23 Apr 2014 16:16:55 -0700 (PDT) Date: Thu, 24 Apr 2014 01:16:55 +0200 Message-ID: Subject: Random disconnects on 10.0 with mpd5/pptp + PF From: martin i To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 23:16:56 -0000 I tried to analyse the PPTP traffic on both sides of the connection. Test consisted of client connecting to a server, logging to a samba share and staying idle. Prior to each test tcpdump/wireshark was launched on respective node. Every time session ended after certain time (time differed each time but was always less than 10 minutes). Connection started as expected, every second there was a live traffic (PPP/GRE) between these nodes. After a while though this traffic ended (no sign why, it just stopped). In ~45secs after this client requested echo-reply 5 times in 1 minute interval (5 mins total). Each time it got a successful reply. After that server sent PSH and RST packet which ended the session. In summary there's a live session, period of silence, 5 times echo and then PSH and RST which results in dropped connection. It doesn't matter if the client is active or stays idle. Question is: what can give me the hint why PPP/GRE traffic stopped? What to focus on? Other connections (not PPTP related) don't have this problem. If the client has an idle SSH connection to this server it stays up without a problem. The same PPTP configuration works ok on FreeBSD 9.2. Martin From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 06:24:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB59C26C for ; Thu, 24 Apr 2014 06:24:42 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7CE231BC4 for ; Thu, 24 Apr 2014 06:24:42 +0000 (UTC) Received: from secured.by.ipfw.ru ([95.143.220.47] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wd9Bz-0003IR-Ec; Thu, 24 Apr 2014 06:15:15 +0400 Message-ID: <5358AE0A.6000707@FreeBSD.org> Date: Thu, 24 Apr 2014 10:24:10 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Chris Smith , freebsd-net@freebsd.org Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> In-Reply-To: <535836F1.5070508@nevermind.co.nz> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 06:24:42 -0000 On 24.04.2014 01:56, Chris Smith wrote: > On 23/04/14 19:55, Julian Elischer wrote: >> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>> wrote: >>>> Hello, >>>> >>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >>>> interface route protection was added (so the following problem arose >>>> with 9.2). >>>> >>>> Unfortunately, in my case, I must be able to delete these routes; >>>> not in >>>> the default FIB, but in jail's fibs, because: >>>> · Host is multihomed with multiple nics in different subnets. >>>> · Jail's IP (no vnet) is from a different subnet than host's >>>> default-router subnet – jail has no ip in the range of host's >>>> default-router!!! >>>> · FIB used by jail contains valid default-router. >>>> >>>> Problem: >>>> If iface-routes exist in jail's FIB, answer-packets take the >>>> iface-shortcut, not trespassing the router (default gateway); hence >>>> 3way-handshake never finishes and firewall terminates (half-opened) TCP >>>> sessions. >>>> >>>> Workarround: >>>> · Abuse packet filter doing some kind of route-toâ€Ķ >>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes >>>> can >>>> be deleted without any hack) >>>> >>>> Desired solution: >>>> · Allow deletion of v4-iface-routes if FIB!=0. >>>> >>>> Unfortunately my C skills don't allow me to implement this myself :-( >>>> I can't even follow the code, I guess that was originally considered, >>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>> way >>>> and simply reverted r248895 instead of trying to understand >>>> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >>>> >>>> Thanks for any help, >>>> >>>> -Harry >>>> >>> Hi, >>> >>> As it was suggested before as immediate workaround you can set >>> net.add_addr_allfibs=0 so that the interface routes are added only in >>> the default FIB. >> >> yes, we made two behaviours. >> Add interface routes to all active FIBS or only add them to the first >> fib and let the user populate other fibs as needed. >> It appears you want the second behaviour, so I suggest you use that >> option and set up all your routes manually. >> > Ah, this explains a thing or two. There is an ongoing work to 1) make fibs/allfibs=0 to work better 2) Move forward to make allfibs=0 as default value. > > So when allfibs=0 and an interface is bought up, it's added to the first > FIB automatically (and cannot be removed). > > Is there a way to change which fib the interface route is bought up on? > I tried to 'setfib x ifconfig ....' which didn't work. This will be fixed in near future. > > Failing that, is there a way to change the systems global FIB without > having to run every service with setfib? Basically, the behavour I want > is for interface routes to be bought up on NO fibs, and manually add > them to the fibs I need it on. If ifconfig_ifaceX="fib X inet 1.2.3.4/30" works as expected (changes interface fib to chosen one and announce interface route and host route in this particular fib) - does this sound OK to you? > >>> >>> --Nikolay >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 06:57:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1353A79D for ; Thu, 24 Apr 2014 06:57:43 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D0F171E75 for ; Thu, 24 Apr 2014 06:57:42 +0000 (UTC) Received: from Julian-MBP3.local (gw2.metromesh.com.au [110.5.117.243]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3O6ZcXe051036 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 23 Apr 2014 23:35:44 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5358B0B4.4070805@freebsd.org> Date: Thu, 24 Apr 2014 14:35:32 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Chris Smith , freebsd-net@freebsd.org Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> In-Reply-To: <535836F1.5070508@nevermind.co.nz> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 06:57:43 -0000 On 4/24/14, 5:56 AM, Chris Smith wrote: > On 23/04/14 19:55, Julian Elischer wrote: >> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>> wrote: >>>> Hello, >>>> >>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >>>> interface route protection was added (so the following problem arose >>>> with 9.2). >>>> >>>> Unfortunately, in my case, I must be able to delete these routes; >>>> not in >>>> the default FIB, but in jail's fibs, because: >>>> · Host is multihomed with multiple nics in different subnets. >>>> · Jail's IP (no vnet) is from a different subnet than host's >>>> default-router subnet – jail has no ip in the range of host's >>>> default-router!!! >>>> · FIB used by jail contains valid default-router. >>>> >>>> Problem: >>>> If iface-routes exist in jail's FIB, answer-packets take the >>>> iface-shortcut, not trespassing the router (default gateway); hence >>>> 3way-handshake never finishes and firewall terminates (half-opened) TCP >>>> sessions. >>>> >>>> Workarround: >>>> · Abuse packet filter doing some kind of route-toâ€Ķ >>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes >>>> can >>>> be deleted without any hack) >>>> >>>> Desired solution: >>>> · Allow deletion of v4-iface-routes if FIB!=0. >>>> >>>> Unfortunately my C skills don't allow me to implement this myself :-( >>>> I can't even follow the code, I guess that was originally considered, >>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>> way >>>> and simply reverted r248895 instead of trying to understand >>>> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >>>> >>>> Thanks for any help, >>>> >>>> -Harry >>>> >>> Hi, >>> >>> As it was suggested before as immediate workaround you can set >>> net.add_addr_allfibs=0 so that the interface routes are added only in >>> the default FIB. >> yes, we made two behaviours. >> Add interface routes to all active FIBS or only add them to the first >> fib and let the user populate other fibs as needed. >> It appears you want the second behaviour, so I suggest you use that >> option and set up all your routes manually. >> > Ah, this explains a thing or two. > > So when allfibs=0 and an interface is bought up, it's added to the first > FIB automatically (and cannot be removed). > > Is there a way to change which fib the interface route is bought up on? > I tried to 'setfib x ifconfig ....' which didn't work. > > Failing that, is there a way to change the systems global FIB without > having to run every service with setfib? Basically, the behavour I want > is for interface routes to be bought up on NO fibs, and manually add > them to the fibs I need it on. > > I wrote the multi-fib code to "scratch my own itch" but I tried to imagine what other people might want to use it for. However one can never predict that with complete success. Since then we have added interface fibs, but I think that still needs some work. Don't feel shy about making suggestions and putting forwards patches. Even an addition to the man pages to explain the current behaviour more fully would be a good addition. From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 17:20:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7022AD94 for ; Thu, 24 Apr 2014 17:20:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B1AC15E9 for ; Thu, 24 Apr 2014 17:20:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3OHK1QO064928 for ; Thu, 24 Apr 2014 17:20:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3OHK1I9064927; Thu, 24 Apr 2014 17:20:01 GMT (envelope-from gnats) Date: Thu, 24 Apr 2014 17:20:01 GMT Message-Id: <201404241720.s3OHK1I9064927@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Mr. Clif" Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: "Mr. Clif" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 17:20:01 -0000 The following reply was made to PR kern/188897; it has been noted by GNATS. From: "Mr. Clif" To: bug-followup@FreeBSD.org, clif@eugeneweb.com Cc: Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Date: Thu, 24 Apr 2014 10:11:33 -0700 Well I guess we were over optimistic on PR 179033. Yes I can get some of the Nics to work some of the time but mostly they don't work. They DO list the correct mac addresses and Status lines though, so that is an improvement. If I have just one quad port card installed then dc3 works most often, then dc1. dc0 and dc2 I'm not sure I ever saw working. Even if an interface worked for a bit it often stops after a while. Id'e be happy to run some tests if anyone has some patches for me to try. Clif From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 18:44:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8ADC011B; Thu, 24 Apr 2014 18:44:02 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0D942000; Thu, 24 Apr 2014 18:44:01 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id t60so844016wes.8 for ; Thu, 24 Apr 2014 11:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=oTgT44UOJpHqUQDYc3VfR8ro/z/N94wO0u86+/3wnyM=; b=NEu0CUS/4jx6y4jdrw81Xc+XriV/2Ope16/jAvO+NjiZr+W0oKHZW6pTkd+D6rt2We AfYtDIaejoVcNGoG41ND/qI48O3qAvWe2mksETzYTnVABGEHrO6oSUB8t9YjCpRhEwgr lfEKNL6ZhQkP4GRm1jTdnjFfUcMqcaQbcylRpl4fhpCzv9ZwsunMdX33/zo/ZMkduwPx QONaTLc/7vbS2QtsL/vFXVHr7nKGtybklft02QofMKPIGjD/TW1ofkpRQqr3I62GAHZr 3ZoBAOMcvllnWASbB8y1sipbRj/U6+nHG/DIwKKWdZ4xpNPwouAHtSp/yh+FMIztkvMr YDdw== MIME-Version: 1.0 X-Received: by 10.180.212.107 with SMTP id nj11mr287992wic.40.1398365040229; Thu, 24 Apr 2014 11:44:00 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Thu, 24 Apr 2014 11:44:00 -0700 (PDT) In-Reply-To: <5358AE0A.6000707@FreeBSD.org> References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> <5358AE0A.6000707@FreeBSD.org> Date: Thu, 24 Apr 2014 12:44:00 -0600 X-Google-Sender-Auth: iYQNn6LBg2b7-DDu0kwX3INasLw Message-ID: Subject: Re: Deleting IPv4 iface-routes from extra FIBs From: Alan Somers To: "Alexander V. Chernikov" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , Chris Smith X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 18:44:02 -0000 On Thu, Apr 24, 2014 at 12:24 AM, Alexander V. Chernikov wrote: > On 24.04.2014 01:56, Chris Smith wrote: >> On 23/04/14 19:55, Julian Elischer wrote: >>> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>>> wrote: >>>>> Hello, >>>>> >>>>> here, http://svnweb.freebsd.org/base?view=3Drevision&revision=3D24889= 5 >>>>> interface route protection was added (so the following problem arose >>>>> with 9.2). >>>>> >>>>> Unfortunately, in my case, I must be able to delete these routes; >>>>> not in >>>>> the default FIB, but in jail's fibs, because: >>>>> =C2=B7 Host is multihomed with multiple nics in different subnets. >>>>> =C2=B7 Jail's IP (no vnet) is from a different subnet than host's >>>>> default-router subnet =E2=80=93 jail has no ip in the range of host's >>>>> default-router!!! >>>>> =C2=B7 FIB used by jail contains valid default-router. >>>>> >>>>> Problem: >>>>> If iface-routes exist in jail's FIB, answer-packets take the >>>>> iface-shortcut, not trespassing the router (default gateway); hence >>>>> 3way-handshake never finishes and firewall terminates (half-opened) T= CP >>>>> sessions. >>>>> >>>>> Workarround: >>>>> =C2=B7 Abuse packet filter doing some kind of route-to=E2=80=A6 >>>>> =C2=B7 Revert r248895, to be able to delete v4-iface-routes (inet6-ro= utes >>>>> can >>>>> be deleted without any hack) >>>>> >>>>> Desired solution: >>>>> =C2=B7 Allow deletion of v4-iface-routes if FIB!=3D0. >>>>> >>>>> Unfortunately my C skills don't allow me to implement this myself :-( >>>>> I can't even follow the code, I guess that was originally considered, >>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>>> way >>>>> and simply reverted r248895 instead of trying to understand >>>>> rtrequest1_fib(). I wish I had the time to learn=E2=80=A6 >>>>> >>>>> Thanks for any help, >>>>> >>>>> -Harry >>>>> >>>> Hi, >>>> >>>> As it was suggested before as immediate workaround you can set >>>> net.add_addr_allfibs=3D0 so that the interface routes are added only i= n >>>> the default FIB. >>> >>> yes, we made two behaviours. >>> Add interface routes to all active FIBS or only add them to the first >>> fib and let the user populate other fibs as needed. >>> It appears you want the second behaviour, so I suggest you use that >>> option and set up all your routes manually. >>> >> Ah, this explains a thing or two. > > There is an ongoing work to > 1) make fibs/allfibs=3D0 to work better > 2) Move forward to make allfibs=3D0 as default value. >> >> So when allfibs=3D0 and an interface is bought up, it's added to the fir= st >> FIB automatically (and cannot be removed). >> >> Is there a way to change which fib the interface route is bought up on? >> I tried to 'setfib x ifconfig ....' which didn't work. > This will be fixed in near future. Fixed in CURRENT by change 264887. >> >> Failing that, is there a way to change the systems global FIB without >> having to run every service with setfib? Basically, the behavour I want >> is for interface routes to be bought up on NO fibs, and manually add >> them to the fibs I need it on. > If ifconfig_ifaceX=3D"fib X inet 1.2.3.4/30" works as expected (changes > interface fib to chosen one and announce interface route and host route > in this particular fib) - does this sound OK to you? >> >>>> >>>> --Nikolay >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 21:17:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7506B6FE for ; Thu, 24 Apr 2014 21:17:46 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 2774A10EC for ; Thu, 24 Apr 2014 21:17:46 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id 793146B9343 for ; Thu, 24 Apr 2014 23:17:44 +0200 (CEST) Received: from soth.ventu (adsl-ull-163-141.41-151.net24.it [151.41.141.163]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3OLHcxe074935 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 24 Apr 2014 23:17:41 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-163-141.41-151.net24.it [151.41.141.163] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3OLHXbX022577 for ; Thu, 24 Apr 2014 23:17:33 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53597F6D.8090201@netfence.it> Date: Thu, 24 Apr 2014 23:17:33 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Network troubles after 8.3 -> 8.4 upgrade References: <53503BC3.6040806@netfence.it> <5352B005.8090405@netfence.it> In-Reply-To: <5352B005.8090405@netfence.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Thu, 24 Apr 2014 23:17:42 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 21:17:46 -0000 On 04/19/14 19:19, Andrea Venturoli wrote: > Hmmm, sounds a bit complicated... would simply dropping if_em.ko in from > a 8.3 box work? Ok, I'll answer myself. I'm now running 8.3's if_em.ko (binary from another system), on an 8.4 kernel. The behaviour is the same as before. However, further researches show em is not the culprit. Rather, ipfw is: in fact dynamic rules are created, but will always timeout after 20 s (no matter if idle or not). Someway the outgoing packet triggers the dynamic rule, but it doesn't seem to get past the SYN phase. The ruleset here is quite a mess, so I need to investigate it better. What suprise me is that it worked differently with 8.3! Well, at least now I know where to look... Thanks to anyone who replied. bye av. From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 23:00:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99941FC for ; Thu, 24 Apr 2014 23:00:24 +0000 (UTC) Received: from smtp.webfaction.com (mail6.webfaction.com [74.55.86.74]) by mx1.freebsd.org (Postfix) with ESMTP id 708151A9E for ; Thu, 24 Apr 2014 23:00:24 +0000 (UTC) Received: from [172.20.10.6] (153.71.224.49.dyn.cust.vf.net.nz [49.224.71.153]) by smtp.webfaction.com (Postfix) with ESMTP id F29A020CD113 for ; Thu, 24 Apr 2014 23:00:17 +0000 (UTC) Message-ID: <5359977C.8000308@nevermind.co.nz> Date: Fri, 25 Apr 2014 11:00:12 +1200 From: Chris Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> <5358AE0A.6000707@FreeBSD.org> In-Reply-To: <5358AE0A.6000707@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 23:00:24 -0000 On 24/04/14 18:24, Alexander V. Chernikov wrote: > On 24.04.2014 01:56, Chris Smith wrote: >> On 23/04/14 19:55, Julian Elischer wrote: >>> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>>> wrote: >>>>> Hello, >>>>> >>>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >>>>> interface route protection was added (so the following problem arose >>>>> with 9.2). >>>>> >>>>> Unfortunately, in my case, I must be able to delete these routes; >>>>> not in >>>>> the default FIB, but in jail's fibs, because: >>>>> · Host is multihomed with multiple nics in different subnets. >>>>> · Jail's IP (no vnet) is from a different subnet than host's >>>>> default-router subnet – jail has no ip in the range of host's >>>>> default-router!!! >>>>> · FIB used by jail contains valid default-router. >>>>> >>>>> Problem: >>>>> If iface-routes exist in jail's FIB, answer-packets take the >>>>> iface-shortcut, not trespassing the router (default gateway); hence >>>>> 3way-handshake never finishes and firewall terminates (half-opened) TCP >>>>> sessions. >>>>> >>>>> Workarround: >>>>> · Abuse packet filter doing some kind of route-toâ€Ķ >>>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes >>>>> can >>>>> be deleted without any hack) >>>>> >>>>> Desired solution: >>>>> · Allow deletion of v4-iface-routes if FIB!=0. >>>>> >>>>> Unfortunately my C skills don't allow me to implement this myself :-( >>>>> I can't even follow the code, I guess that was originally considered, >>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>>> way >>>>> and simply reverted r248895 instead of trying to understand >>>>> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >>>>> >>>>> Thanks for any help, >>>>> >>>>> -Harry >>>>> >>>> Hi, >>>> >>>> As it was suggested before as immediate workaround you can set >>>> net.add_addr_allfibs=0 so that the interface routes are added only in >>>> the default FIB. >>> yes, we made two behaviours. >>> Add interface routes to all active FIBS or only add them to the first >>> fib and let the user populate other fibs as needed. >>> It appears you want the second behaviour, so I suggest you use that >>> option and set up all your routes manually. >>> >> Ah, this explains a thing or two. > There is an ongoing work to > 1) make fibs/allfibs=0 to work better > 2) Move forward to make allfibs=0 as default value. >> So when allfibs=0 and an interface is bought up, it's added to the first >> FIB automatically (and cannot be removed). >> >> Is there a way to change which fib the interface route is bought up on? >> I tried to 'setfib x ifconfig ....' which didn't work. > This will be fixed in near future. >> Failing that, is there a way to change the systems global FIB without >> having to run every service with setfib? Basically, the behavour I want >> is for interface routes to be bought up on NO fibs, and manually add >> them to the fibs I need it on. > If ifconfig_ifaceX="fib X inet 1.2.3.4/30" works as expected (changes > interface fib to chosen one and announce interface route and host route > in this particular fib) - does this sound OK to you? Yes this sounds good. If I'm not mistaken the interface FIB only makes sense when the system is routing? Because the issue I have is that SYN ACKs from services are being routed via the wrong interfaces and interface FIBs do not appear to affect that. Allowing interface routes on different FIBs will fix that I think. Or being able to remove interface routes from a FIB. In the mean time, I will probably use FIBs (as opposed to vnet) for my jails, but find a way to run the hosts SSHd with a specific FIB. Any easy way to do that? Or to specify a system "default FIB" other than 0? >>>> --Nikolay >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 23:15:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B04E8904 for ; Thu, 24 Apr 2014 23:15:16 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 405AB1C3D for ; Thu, 24 Apr 2014 23:15:16 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id a1so2962907wgh.32 for ; Thu, 24 Apr 2014 16:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=4qAzIhPm2ol26VQ8/P4XsRDTqcEOYBuUnXup1pwo/Xc=; b=A821Uxf/Fsx3lKzmu/L9Lc7w9IDkjZ13ieXF99vuVez0FLOixUB4WHEh53MS5zH5bu tSxD23Ouw3mkq0NLj91RfNu+Ee0eHyoWbV7V0ydjUM2WgMvcKxP09qA/il7ixKCribD2 0CaG6IPfHiYLaI7A1WQcfKaNIe70BYVO24ADMovZGAaKoLvOjhOdoW5YUnL2LY+TR9hg rEwvUihGGAuCXOCfEOkkKMdHoMBzmiIT7yBm/cRUtZUiJtpi9T26eMhOvepMjH03F19n Y/k8aVJA/BBLmWdrwzZnEZFXp9sfNomNcSRMZ2lxhdNguQNcVSvUesOJOZo6pLNaqwlW CR4A== MIME-Version: 1.0 X-Received: by 10.194.20.65 with SMTP id l1mr3787789wje.39.1398381314484; Thu, 24 Apr 2014 16:15:14 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Thu, 24 Apr 2014 16:15:14 -0700 (PDT) In-Reply-To: <5359977C.8000308@nevermind.co.nz> References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> <5358AE0A.6000707@FreeBSD.org> <5359977C.8000308@nevermind.co.nz> Date: Thu, 24 Apr 2014 17:15:14 -0600 X-Google-Sender-Auth: KTifrh7ImatSrPcFKu2UpVkztlM Message-ID: Subject: Re: Deleting IPv4 iface-routes from extra FIBs From: Alan Somers To: Chris Smith Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 23:15:16 -0000 On Thu, Apr 24, 2014 at 5:00 PM, Chris Smith wrote: > On 24/04/14 18:24, Alexander V. Chernikov wrote: >> >> On 24.04.2014 01:56, Chris Smith wrote: >>> >>> On 23/04/14 19:55, Julian Elischer wrote: >>>> >>>> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>>>> >>>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>>>> wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> here, http://svnweb.freebsd.org/base?view=3Drevision&revision=3D2488= 95 >>>>>> interface route protection was added (so the following problem arose >>>>>> with 9.2). >>>>>> >>>>>> Unfortunately, in my case, I must be able to delete these routes; >>>>>> not in >>>>>> the default FIB, but in jail's fibs, because: >>>>>> =C2=B7 Host is multihomed with multiple nics in different subnets. >>>>>> =C2=B7 Jail's IP (no vnet) is from a different subnet than host's >>>>>> default-router subnet =E2=80=93 jail has no ip in the range of host'= s >>>>>> default-router!!! >>>>>> =C2=B7 FIB used by jail contains valid default-router. >>>>>> >>>>>> Problem: >>>>>> If iface-routes exist in jail's FIB, answer-packets take the >>>>>> iface-shortcut, not trespassing the router (default gateway); hence >>>>>> 3way-handshake never finishes and firewall terminates (half-opened) >>>>>> TCP >>>>>> sessions. >>>>>> >>>>>> Workarround: >>>>>> =C2=B7 Abuse packet filter doing some kind of route-to=E2=80=A6 >>>>>> =C2=B7 Revert r248895, to be able to delete v4-iface-routes (inet6-r= outes >>>>>> can >>>>>> be deleted without any hack) >>>>>> >>>>>> Desired solution: >>>>>> =C2=B7 Allow deletion of v4-iface-routes if FIB!=3D0. >>>>>> >>>>>> Unfortunately my C skills don't allow me to implement this myself :-= ( >>>>>> I can't even follow the code, I guess that was originally considered= , >>>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>>>> way >>>>>> and simply reverted r248895 instead of trying to understand >>>>>> rtrequest1_fib(). I wish I had the time to learn=E2=80=A6 >>>>>> >>>>>> Thanks for any help, >>>>>> >>>>>> -Harry >>>>>> >>>>> Hi, >>>>> >>>>> As it was suggested before as immediate workaround you can set >>>>> net.add_addr_allfibs=3D0 so that the interface routes are added only = in >>>>> the default FIB. >>>> >>>> yes, we made two behaviours. >>>> Add interface routes to all active FIBS or only add them to the first >>>> fib and let the user populate other fibs as needed. >>>> It appears you want the second behaviour, so I suggest you use that >>>> option and set up all your routes manually. >>>> >>> Ah, this explains a thing or two. >> >> There is an ongoing work to >> 1) make fibs/allfibs=3D0 to work better >> 2) Move forward to make allfibs=3D0 as default value. >>> >>> So when allfibs=3D0 and an interface is bought up, it's added to the fi= rst >>> FIB automatically (and cannot be removed). >>> >>> Is there a way to change which fib the interface route is bought up on? >>> I tried to 'setfib x ifconfig ....' which didn't work. >> >> This will be fixed in near future. >>> >>> Failing that, is there a way to change the systems global FIB without >>> having to run every service with setfib? Basically, the behavour I want >>> is for interface routes to be bought up on NO fibs, and manually add >>> them to the fibs I need it on. >> >> If ifconfig_ifaceX=3D"fib X inet 1.2.3.4/30" works as expected (changes >> interface fib to chosen one and announce interface route and host route >> in this particular fib) - does this sound OK to you? > > Yes this sounds good. > > If I'm not mistaken the interface FIB only makes sense when the system is > routing? Because the issue I have is that SYN ACKs from services are bein= g > routed via the wrong interfaces and interface FIBs do not appear to affec= t > that. The interface FIB is used when forwarding packets and when creating the initial subnet and host routes when you assign an interface address. It's not used for outbound traffic (except in that it determines where the host and subnet routes get created). There are several other FIB bugs that I'm actively working on. kern/187553 might be related to your problem; it would be great if you could make a test case. > > Allowing interface routes on different FIBs will fix that I think. Or bei= ng > able to remove interface routes from a FIB. > > In the mean time, I will probably use FIBs (as opposed to vnet) for my > jails, but find a way to run the hosts SSHd with a specific FIB. Any easy > way to do that? Or to specify a system "default FIB" other than 0? In FreeBSD 10 you can put "sshd_fib=3D1" in /etc/rc.conf to change that process's fib. That will affect the routing of sshd's outbound packets. If you also want to limit which interfaces sshd listens on, you can do that with pf or by setting the ListenAddress in sshd_config. -Alan > >>>>> --Nikolay >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>>>> >>>>> >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 23:23:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 742DD1000 for ; Thu, 24 Apr 2014 23:23:18 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36E5F1D1B for ; Thu, 24 Apr 2014 23:23:18 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id q108so3296643qgd.7 for ; Thu, 24 Apr 2014 16:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=RetQsJzApNVlzgNgez6XmvAnwNXjhC4le24gIzg9XW4=; b=ERdO9V3ybIqmlJXHpjTlA9bXoSWR8tWaz5GSBE5/g0wX3sTPTOk7jl5ZR31Us6Ez+O lMadc6vF9e3aO2APi5UYO2UKCXDCpekGmkXhAyTYmJEnosj1VIvP7JM4ra/pzz+pcBbE BkLa5JNaTSIKshSOmXFu12z8KAccQp8LWVa7qYPZkGdkCzAsuOwBisJYo0LEvxf/85sG ZmumxMm/m9t9Mk8dJSlNKiexuMGwemlDfGRmMi/3l4e1igBvg8cNoIOr9P0dVvAsIwMe wfn/2TOVTR0AkVK/Nhd0hGyJf3hglHUB/9kCCuzc3W2gXSzxkYk6fYGSDE50OTwUZTA8 ErNw== MIME-Version: 1.0 X-Received: by 10.224.13.142 with SMTP id c14mr7194981qaa.76.1398381797395; Thu, 24 Apr 2014 16:23:17 -0700 (PDT) Received: by 10.96.41.70 with HTTP; Thu, 24 Apr 2014 16:23:17 -0700 (PDT) Received: by 10.96.41.70 with HTTP; Thu, 24 Apr 2014 16:23:17 -0700 (PDT) Date: Thu, 24 Apr 2014 18:23:17 -0500 Message-ID: Subject: vnet - using a jail as a default firewall gateway to internet From: Rob J To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 23:23:18 -0000 Hi, I have been playing with vnet jails, and have a configuration working that I thought would not be (based on the docs out there), but it is. I have a box with 3 NICS - hme0, em0 and em1. Basically, with the assumption that the internet facing gateway is potentially a weak point, I set out to configure a jail on the above box to be the gateway, rather than the physical host itself. I recompiled the kernel, with the VIMAGE option, and setup a jail that uses em0 (192.168.x.y) as the lan side and hme0 (public IP a.b.c.d) is the ISP side. On the jail itself, its default route to the internet is public IP a.b.c.e (same network of interface hme0 above). Then I set the rest of my lan to point to 192.168.x.y (interface em0 above) as the default gateway. I have access to the internet with that configuration, routing through the jail (or at least I think so) - everything seems to work. The two errors I get upon starting the jail, are: "sysctl: net.inet.ip.sourceroute not permitted" and "sysctl: net.inet.ip.accept_sourceroute not permitted. Any body knows what may be broken with my configuration? All the docs I read about having a jail route traffic seemed to imply it is undoable. Did I create a glaring whole in my network by having this design as my firewall and router? I also noticed that the physical host is doing all the logging for dmesg and security, when I thought the jail would, but it is beginning to make sense that the kernel is only running on the physical host, and therefore does the logging of all kernel related activities. Any comments or suggestions welcome. Thanks, Robert From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 23:50:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F112B68E for ; Thu, 24 Apr 2014 23:50:14 +0000 (UTC) Received: from smtp.webfaction.com (mail6.webfaction.com [74.55.86.74]) by mx1.freebsd.org (Postfix) with ESMTP id C5DFF1049 for ; Thu, 24 Apr 2014 23:50:14 +0000 (UTC) Received: from [172.20.10.6] (153.71.224.49.dyn.cust.vf.net.nz [49.224.71.153]) by smtp.webfaction.com (Postfix) with ESMTP id 24BCB20BF766 for ; Thu, 24 Apr 2014 23:50:12 +0000 (UTC) Message-ID: <5359A32D.5060701@nevermind.co.nz> Date: Fri, 25 Apr 2014 11:50:05 +1200 From: Chris Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> <5358AE0A.6000707@FreeBSD.org> <5359977C.8000308@nevermind.co.nz> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 23:50:15 -0000 On 25/04/14 11:15, Alan Somers wrote: > On Thu, Apr 24, 2014 at 5:00 PM, Chris Smith wrote: >> On 24/04/14 18:24, Alexander V. Chernikov wrote: >>> On 24.04.2014 01:56, Chris Smith wrote: >>>> On 23/04/14 19:55, Julian Elischer wrote: >>>>> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>>>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>>>>> wrote: >>>>>>> Hello, >>>>>>> >>>>>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895 >>>>>>> interface route protection was added (so the following problem arose >>>>>>> with 9.2). >>>>>>> >>>>>>> Unfortunately, in my case, I must be able to delete these routes; >>>>>>> not in >>>>>>> the default FIB, but in jail's fibs, because: >>>>>>> · Host is multihomed with multiple nics in different subnets. >>>>>>> · Jail's IP (no vnet) is from a different subnet than host's >>>>>>> default-router subnet – jail has no ip in the range of host's >>>>>>> default-router!!! >>>>>>> · FIB used by jail contains valid default-router. >>>>>>> >>>>>>> Problem: >>>>>>> If iface-routes exist in jail's FIB, answer-packets take the >>>>>>> iface-shortcut, not trespassing the router (default gateway); hence >>>>>>> 3way-handshake never finishes and firewall terminates (half-opened) >>>>>>> TCP >>>>>>> sessions. >>>>>>> >>>>>>> Workarround: >>>>>>> · Abuse packet filter doing some kind of route-toâ€Ķ >>>>>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes >>>>>>> can >>>>>>> be deleted without any hack) >>>>>>> >>>>>>> Desired solution: >>>>>>> · Allow deletion of v4-iface-routes if FIB!=0. >>>>>>> >>>>>>> Unfortunately my C skills don't allow me to implement this myself :-( >>>>>>> I can't even follow the code, I guess that was originally considered, >>>>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy >>>>>>> way >>>>>>> and simply reverted r248895 instead of trying to understand >>>>>>> rtrequest1_fib(). I wish I had the time to learnâ€Ķ >>>>>>> >>>>>>> Thanks for any help, >>>>>>> >>>>>>> -Harry >>>>>>> >>>>>> Hi, >>>>>> >>>>>> As it was suggested before as immediate workaround you can set >>>>>> net.add_addr_allfibs=0 so that the interface routes are added only in >>>>>> the default FIB. >>>>> yes, we made two behaviours. >>>>> Add interface routes to all active FIBS or only add them to the first >>>>> fib and let the user populate other fibs as needed. >>>>> It appears you want the second behaviour, so I suggest you use that >>>>> option and set up all your routes manually. >>>>> >>>> Ah, this explains a thing or two. >>> There is an ongoing work to >>> 1) make fibs/allfibs=0 to work better >>> 2) Move forward to make allfibs=0 as default value. >>>> So when allfibs=0 and an interface is bought up, it's added to the first >>>> FIB automatically (and cannot be removed). >>>> >>>> Is there a way to change which fib the interface route is bought up on? >>>> I tried to 'setfib x ifconfig ....' which didn't work. >>> This will be fixed in near future. >>>> Failing that, is there a way to change the systems global FIB without >>>> having to run every service with setfib? Basically, the behavour I want >>>> is for interface routes to be bought up on NO fibs, and manually add >>>> them to the fibs I need it on. >>> If ifconfig_ifaceX="fib X inet 1.2.3.4/30" works as expected (changes >>> interface fib to chosen one and announce interface route and host route >>> in this particular fib) - does this sound OK to you? >> Yes this sounds good. >> >> If I'm not mistaken the interface FIB only makes sense when the system is >> routing? Because the issue I have is that SYN ACKs from services are being >> routed via the wrong interfaces and interface FIBs do not appear to affect >> that. > The interface FIB is used when forwarding packets and when creating > the initial subnet and host routes when you assign an interface > address. It's not used for outbound traffic (except in that it > determines where the host and subnet routes get created). There are > several other FIB bugs that I'm actively working on. kern/187553 > might be related to your problem; it would be great if you could make > a test case. The connections I've been testing with are TCP (SSH and Netcat) However, this: ifconfig bge0 fib 1 10.0.0.1/24 Adds the interface route to FIB 0 and nothing to FIB 1. FreeBSD 10 RELEASE amd64 >> Allowing interface routes on different FIBs will fix that I think. Or being >> able to remove interface routes from a FIB. >> >> In the mean time, I will probably use FIBs (as opposed to vnet) for my >> jails, but find a way to run the hosts SSHd with a specific FIB. Any easy >> way to do that? Or to specify a system "default FIB" other than 0? > In FreeBSD 10 you can put "sshd_fib=1" in /etc/rc.conf to change that > process's fib. That will affect the routing of sshd's outbound > packets. If you also want to limit which interfaces sshd listens on, > you can do that with pf or by setting the ListenAddress in > sshd_config. > > -Alan > >>>>>> --Nikolay >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 23:58:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40AA28FC for ; Thu, 24 Apr 2014 23:58:45 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C395910EF for ; Thu, 24 Apr 2014 23:58:44 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id b13so1752147wgh.35 for ; Thu, 24 Apr 2014 16:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=3+kl74txzVPSKzgVyllEfgZeQF4N7V6GJYwL75449g8=; b=OE4WPKB7CHFCc2lh4cWZKZrqIT2UCRp7n0ZNfoHcMezHQJTLd2jzSD3qWtDalLMxHI n4LoAUYQHA2lSYBjSrMotKtLUUhabyxAbThcPQlZFT89bx9gT8kkpeAcsjbiGvRjdnGn ZMBahgS0dRrbCHtW7aer8Ul5grJXfYzAdHauTlXUIm5d1hjK9djqVvyLNGcmoMKKK8nY 1SqYengEZtEGMiC3Y+h62nuEAEF60QEVBR4QeiivqfSyJ/RixC30RT3S4wCoQYp0JdA6 qtvizbmKXtf14Vzagg2+DX8rDS71D4Q22ZxEdECt8Uaj1226Q4fdS5N+zxXhzXnk5Lnp ddHg== MIME-Version: 1.0 X-Received: by 10.194.92.177 with SMTP id cn17mr4023037wjb.18.1398383923123; Thu, 24 Apr 2014 16:58:43 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Thu, 24 Apr 2014 16:58:43 -0700 (PDT) In-Reply-To: <5359A32D.5060701@nevermind.co.nz> References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <535836F1.5070508@nevermind.co.nz> <5358AE0A.6000707@FreeBSD.org> <5359977C.8000308@nevermind.co.nz> <5359A32D.5060701@nevermind.co.nz> Date: Thu, 24 Apr 2014 17:58:43 -0600 X-Google-Sender-Auth: -rBMKiKrnAf4IfcYUyCAE6G6mQQ Message-ID: Subject: Re: Deleting IPv4 iface-routes from extra FIBs From: Alan Somers To: Chris Smith Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 23:58:45 -0000 On Thu, Apr 24, 2014 at 5:50 PM, Chris Smith wrote: > On 25/04/14 11:15, Alan Somers wrote: >> >> On Thu, Apr 24, 2014 at 5:00 PM, Chris Smith >> wrote: >>> >>> On 24/04/14 18:24, Alexander V. Chernikov wrote: >>>> >>>> On 24.04.2014 01:56, Chris Smith wrote: >>>>> >>>>> On 23/04/14 19:55, Julian Elischer wrote: >>>>>> >>>>>> On 4/23/14, 4:38 AM, Nikolay Denev wrote: >>>>>>> >>>>>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer >>>>>>> wrote: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> here, http://svnweb.freebsd.org/base?view=3Drevision&revision=3D24= 8895 >>>>>>>> interface route protection was added (so the following problem aro= se >>>>>>>> with 9.2). >>>>>>>> >>>>>>>> Unfortunately, in my case, I must be able to delete these routes; >>>>>>>> not in >>>>>>>> the default FIB, but in jail's fibs, because: >>>>>>>> =C2=B7 Host is multihomed with multiple nics in different subnets. >>>>>>>> =C2=B7 Jail's IP (no vnet) is from a different subnet than host's >>>>>>>> default-router subnet =E2=80=93 jail has no ip in the range of hos= t's >>>>>>>> default-router!!! >>>>>>>> =C2=B7 FIB used by jail contains valid default-router. >>>>>>>> >>>>>>>> Problem: >>>>>>>> If iface-routes exist in jail's FIB, answer-packets take the >>>>>>>> iface-shortcut, not trespassing the router (default gateway); henc= e >>>>>>>> 3way-handshake never finishes and firewall terminates (half-opened= ) >>>>>>>> TCP >>>>>>>> sessions. >>>>>>>> >>>>>>>> Workarround: >>>>>>>> =C2=B7 Abuse packet filter doing some kind of route-to=E2=80=A6 >>>>>>>> =C2=B7 Revert r248895, to be able to delete v4-iface-routes (inet6= -routes >>>>>>>> can >>>>>>>> be deleted without any hack) >>>>>>>> >>>>>>>> Desired solution: >>>>>>>> =C2=B7 Allow deletion of v4-iface-routes if FIB!=3D0. >>>>>>>> >>>>>>>> Unfortunately my C skills don't allow me to implement this myself >>>>>>>> :-( >>>>>>>> I can't even follow the code, I guess that was originally >>>>>>>> considered, >>>>>>>> but possibly doesn't work bacause of a simple bug?!? I took the la= zy >>>>>>>> way >>>>>>>> and simply reverted r248895 instead of trying to understand >>>>>>>> rtrequest1_fib(). I wish I had the time to learn=E2=80=A6 >>>>>>>> >>>>>>>> Thanks for any help, >>>>>>>> >>>>>>>> -Harry >>>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> As it was suggested before as immediate workaround you can set >>>>>>> net.add_addr_allfibs=3D0 so that the interface routes are added onl= y in >>>>>>> the default FIB. >>>>>> >>>>>> yes, we made two behaviours. >>>>>> Add interface routes to all active FIBS or only add them to the firs= t >>>>>> fib and let the user populate other fibs as needed. >>>>>> It appears you want the second behaviour, so I suggest you use that >>>>>> option and set up all your routes manually. >>>>>> >>>>> Ah, this explains a thing or two. >>>> >>>> There is an ongoing work to >>>> 1) make fibs/allfibs=3D0 to work better >>>> 2) Move forward to make allfibs=3D0 as default value. >>>>> >>>>> So when allfibs=3D0 and an interface is bought up, it's added to the >>>>> first >>>>> FIB automatically (and cannot be removed). >>>>> >>>>> Is there a way to change which fib the interface route is bought up o= n? >>>>> I tried to 'setfib x ifconfig ....' which didn't work. >>>> >>>> This will be fixed in near future. >>>>> >>>>> Failing that, is there a way to change the systems global FIB without >>>>> having to run every service with setfib? Basically, the behavour I wa= nt >>>>> is for interface routes to be bought up on NO fibs, and manually add >>>>> them to the fibs I need it on. >>>> >>>> If ifconfig_ifaceX=3D"fib X inet 1.2.3.4/30" works as expected (change= s >>>> interface fib to chosen one and announce interface route and host rout= e >>>> in this particular fib) - does this sound OK to you? >>> >>> Yes this sounds good. >>> >>> If I'm not mistaken the interface FIB only makes sense when the system = is >>> routing? Because the issue I have is that SYN ACKs from services are >>> being >>> routed via the wrong interfaces and interface FIBs do not appear to >>> affect >>> that. >> >> The interface FIB is used when forwarding packets and when creating >> the initial subnet and host routes when you assign an interface >> address. It's not used for outbound traffic (except in that it >> determines where the host and subnet routes get created). There are >> several other FIB bugs that I'm actively working on. kern/187553 >> might be related to your problem; it would be great if you could make >> a test case. > > The connections I've been testing with are TCP (SSH and Netcat) > > However, this: > > ifconfig bge0 fib 1 10.0.0.1/24 > > Adds the interface route to FIB 0 and nothing to FIB 1. FreeBSD 10 RELEAS= E > amd64 That is exactly the bug I fixed earlier today with r264887. I'll MFC it to stable/10 in a few weeks. > > >>> Allowing interface routes on different FIBs will fix that I think. Or >>> being >>> able to remove interface routes from a FIB. >>> >>> In the mean time, I will probably use FIBs (as opposed to vnet) for my >>> jails, but find a way to run the hosts SSHd with a specific FIB. Any ea= sy >>> way to do that? Or to specify a system "default FIB" other than 0? >> >> In FreeBSD 10 you can put "sshd_fib=3D1" in /etc/rc.conf to change that >> process's fib. That will affect the routing of sshd's outbound >> packets. If you also want to limit which interfaces sshd listens on, >> you can do that with pf or by setting the ListenAddress in >> sshd_config. >> >> -Alan >> >>>>>>> --Nikolay >>>>>>> _______________________________________________ >>>>>>> freebsd-net@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>> To unsubscribe, send any mail to >>>>>>> "freebsd-net-unsubscribe@freebsd.org" >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.or= g" >>>>> >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>>>> >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 05:40:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D8A3477 for ; Fri, 25 Apr 2014 05:40:01 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3651F1076 for ; Fri, 25 Apr 2014 05:40:00 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3P5dpGw054914 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 24 Apr 2014 22:39:53 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5359F522.5080905@freebsd.org> Date: Fri, 25 Apr 2014 13:39:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rob J , freebsd-net@freebsd.org Subject: Re: vnet - using a jail as a default firewall gateway to internet References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 05:40:01 -0000 On 4/25/14, 7:23 AM, Rob J wrote: > Hi, > > I have been playing with vnet jails, and have a configuration working that > I thought would not be (based on the docs out there), but it is. I have a > box with 3 NICS - hme0, em0 and em1. Basically, with the assumption that > the internet facing gateway is potentially a weak point, I set out to > configure a jail on the above box to be the gateway, rather than the > physical host itself. I recompiled the kernel, with the VIMAGE option, and > setup a jail that uses em0 (192.168.x.y) as the lan side and hme0 (public > IP a.b.c.d) is the ISP side. Conceptually, the normal base system is just a single instance of a vnet jail, so any situation that you can do with a separate machine as router should be doable with a vnet jail in that role. the error messages you see are because some sysctls can not be done from within a jail. there may be a setting to allow them to happen in a jail... I have not checked. you may attach your regular 'base' system to teh jail using a physical ethernet, or it may have a shortcut with it's own epair or netgraph link to the router instance. this is exactly the sort of situation we wanted to write vnets for.. > On the jail itself, its default route to the internet is public IP a.b.c.e > (same network of interface hme0 above). Then I set the rest of my lan to > point to 192.168.x.y (interface em0 above) as the default gateway. I have > access to the internet with that configuration, routing through the jail > (or at least I think so) - everything seems to work. The two errors I get > upon starting the jail, are: "sysctl: net.inet.ip.sourceroute not > permitted" and "sysctl: net.inet.ip.accept_sourceroute not permitted. Any > body knows what may be broken with my configuration? All the docs I read > about having a jail route traffic seemed to imply it is undoable. > > Did I create a glaring whole in my network by having this design as my > firewall and router? I also noticed that the physical host is doing all > the logging for dmesg and security, when I thought the jail would, but it > is beginning to make sense that the kernel is only running on the physical > host, and therefore does the logging of all kernel related activities. > > Any comments or suggestions welcome. > > Thanks, > > Robert > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 07:48:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D97F82D for ; Fri, 25 Apr 2014 07:48:54 +0000 (UTC) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D90531A4F for ; Fri, 25 Apr 2014 07:48:53 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 32662 invoked from network); 25 Apr 2014 09:48:37 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398412118; bh=zncxVZFf/DZogM7UoZbFmdKSAry9SjOFSJF6Hvexmak=; h=From:To:Subject; b=d2/+DFoanaPJiPz50FuV0ovPlqJf1dEgl0f0gEiH2A3PobbraiHQZ6CiolenMAXq5 bxqzASCVyr7+j1gR2FacTXtPZS3U+11qa8z53JBTu7AVriHX82XifCCeLGLNbPrUnH Tvt9q6ARYkorPIwI+ugOtUqYezHjTX7CYZyRGlZo= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 09:48:37 +0200 Message-ID: <535A1354.2040309@wp.pl> Date: Fri, 25 Apr 2014 09:48:36 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: NFS over LAGG / lacp poor performance Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [EfPk] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 07:48:54 -0000 Hi list, I have two FreeBSD boxes (both based on SuperMicro X9DRD-7LN4F-JBOD motherboard, with 32GB RAM, 1 CPU :Intel(R) Xeon(R) CPU E5-2640 v2) storage1% uname -a FreeBSD storage1 9.1-RELEASE-p10 FreeBSD 9.1-RELEASE-p10 #0: Sun Jan 12 20:11:23 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 storage2% uname -a FreeBSD storage2 10.0-RELEASE-p1 FreeBSD 10.0-RELEASE-p1 #0: Tue Apr 8 06:45:06 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 They work as NFS storages for VMware ESXi. On both there are installed 4 igb 1Gbit NICs, with LACP and 2 VLANs (vlan900 is for NFS, vlan14 is for management/general purpose): on storage1: igb0: port 0xd060-0xd07f mem 0xfbb60000-0xfbb7ffff,0xfbb8c000-0xfbb8ffff irq 42 at device 0.0 on pci5 igb1: port 0xd040-0xd05f mem 0xfbb40000-0xfbb5ffff,0xfbb88000-0xfbb8bfff irq 45 at device 0.1 on pci5 igb2: port 0xd020-0xd03f mem 0xfbb20000-0xfbb3ffff,0xfbb84000-0xfbb87fff irq 44 at device 0.2 on pci5 igb3: port 0xd000-0xd01f mem 0xfbb00000-0xfbb1ffff,0xfbb80000-0xfbb83fff irq 46 at device 0.3 on pci5 on storage2: igb0: port 0xd060-0xd07f mem 0xfbb60000-0xfbb7ffff,0xfbb8c000-0xfbb8ffff irq 42 at device 0.0 on pci5 igb1: port 0xd040-0xd05f mem 0xfbb40000-0xfbb5ffff,0xfbb88000-0xfbb8bfff irq 45 at device 0.1 on pci5 igb2: port 0xd020-0xd03f mem 0xfbb20000-0xfbb3ffff,0xfbb84000-0xfbb87fff irq 44 at device 0.2 on pci5 igb3: port 0xd000-0xd01f mem 0xfbb00000-0xfbb1ffff,0xfbb80000-0xfbb83fff irq 46 at device 0.3 on pci5 storage1% ifconfig -a igb0: flags=8843 metric 0 mtu 9000 options=401bb ether 00:25:90:c1:1d:18 inet6 fe80::225:90ff:fec1:1d18%igb0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=8843 metric 0 mtu 9000 options=401bb ether 00:25:90:c1:1d:18 inet6 fe80::225:90ff:fec1:1d19%igb1 prefixlen 64 scopeid 0x2 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb2: flags=8843 metric 0 mtu 9000 options=401bb ether 00:25:90:c1:1d:18 inet6 fe80::225:90ff:fec1:1d1a%igb2 prefixlen 64 scopeid 0x3 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb3: flags=8843 metric 0 mtu 9000 options=401bb ether 00:25:90:c1:1d:18 inet6 fe80::225:90ff:fec1:1d1b%igb3 prefixlen 64 scopeid 0x4 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 lagg0: flags=8843 metric 0 mtu 9000 options=401bb ether 00:25:90:c1:1d:18 inet6 fe80::225:90ff:fec1:1d18%lagg0 prefixlen 64 scopeid 0x8 nd6 options=21 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: igb3 flags=1c laggport: igb2 flags=1c laggport: igb1 flags=1c laggport: igb0 flags=1c vlan14: flags=8843 metric 0 mtu 9000 options=103 ether 00:25:90:c1:1d:18 inet 192.168.1.65 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::225:90ff:fec1:1d18%vlan14 prefixlen 64 scopeid 0x9 nd6 options=29 media: Ethernet autoselect status: active vlan: 14 parent interface: lagg0 vlan900: flags=8843 metric 0 mtu 9000 options=103 ether 00:25:90:c1:1d:18 inet 172.25.25.65 netmask 0xffffff00 broadcast 172.25.25.255 inet6 fe80::225:90ff:fec1:1d18%vlan900 prefixlen 64 scopeid 0xa nd6 options=29 media: Ethernet autoselect status: active vlan: 900 parent interface: lagg0 storage2% ifconfig -a igb0: flags=8843 metric 0 mtu 9000 options=403bb ether 00:25:90:ca:3b:e0 inet6 fe80::225:90ff:feca:3be0%igb0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=8843 metric 0 mtu 9000 options=403bb ether 00:25:90:ca:3b:e0 inet6 fe80::225:90ff:feca:3be1%igb1 prefixlen 64 scopeid 0x2 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb2: flags=8843 metric 0 mtu 9000 options=403bb ether 00:25:90:ca:3b:e0 inet6 fe80::225:90ff:feca:3be2%igb2 prefixlen 64 scopeid 0x3 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active igb3: flags=8843 metric 0 mtu 9000 options=403bb ether 00:25:90:ca:3b:e0 inet6 fe80::225:90ff:feca:3be3%igb3 prefixlen 64 scopeid 0x4 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 lagg0: flags=8843 metric 0 mtu 9000 options=403bb ether 00:25:90:ca:3b:e0 inet6 fe80::225:90ff:feca:3be0%lagg0 prefixlen 64 scopeid 0x6 nd6 options=21 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: igb3 flags=1c laggport: igb2 flags=1c laggport: igb1 flags=1c laggport: igb0 flags=1c vlan14: flags=8843 metric 0 mtu 9000 options=303 ether 00:25:90:ca:3b:e0 inet 192.168.1.72 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::225:90ff:feca:3be0%vlan14 prefixlen 64 scopeid 0x7 nd6 options=29 media: Ethernet autoselect status: active vlan: 14 parent interface: lagg0 vlan900: flags=8843 metric 0 mtu 9000 options=303 ether 00:25:90:ca:3b:e0 inet 172.25.25.72 netmask 0xffffff00 broadcast 172.25.25.255 inet6 fe80::225:90ff:feca:3be0%vlan900 prefixlen 64 scopeid 0x8 nd6 options=29 media: Ethernet autoselect status: active vlan: 900 parent interface: lagg0 Both boxes are connected to the same switch (HP 1910-48G) I need to transfer around 10 TB of data from storage1 to storage2 I obeserve that during copying, only one NIC (instead of all 4) is used. Boxes are not stressed during copying What's more, apart from having 1 NIC saturated (transfer around 120 MB/s), I observe transfer rate on level of 70-80 MB/s I haven't changed any kernel configuration, nor sysctl's - am I missing sth ? Regards, Marek -- Marek Salwerowicz From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 09:46:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C170508 for ; Fri, 25 Apr 2014 09:46:53 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 913B01A1C for ; Fri, 25 Apr 2014 09:46:52 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 1986D200CD2; Fri, 25 Apr 2014 11:37:13 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 0CD79406AF1; Fri, 25 Apr 2014 11:37:13 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id C1E4C405889; Fri, 25 Apr 2014 11:37:12 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042511371137-88095 ; Fri, 25 Apr 2014 11:37:11 +0200 Date: Fri, 25 Apr 2014 11:37:11 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Marek Salwerowicz Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> In-Reply-To: <535A1354.2040309@wp.pl> References: <535A1354.2040309@wp.pl> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 11:37:11, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 11:37:11, Serialize complete at 04/25/2014 11:37:11 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.92722 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_2000_2999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 09:46:53 -0000 On Fri, 25 Apr 2014 09:48:36 +0200 Marek Salwerowicz wrote about NFS over LAGG / lacp poor performance: Hello Marek, MS> FreeBSD storage1 9.1-RELEASE-p10 FreeBSD 9.1-RELEASE-p10 #0: Sun Jan 12 MS> 20:11:23 UTC 2014 MS> root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Sorry for hijacking this (as I can probably not help you with your issue at hand), but I have a roughly similar setup here (9.1-REL nfs server on X9 mainboard with igb interfaces) and was wondering lately about some performance issues I have on a dedicated nfs link. While looking around on the system, I noticed that interrupts on the igb queues spread a bit strangely: --- root@storage:/root # vmstat -i interrupt total rate irq1: atkbd0 2743 0 irq18: ehci0 uhci5 4445137 2 irq21: uhci1 29 0 cpu0:timer 352597482 228 irq256: igb0:que 0 99396134 64 irq257: igb0:que 1 61496018 39 irq258: igb0:que 2 101687742 66 irq259: igb0:que 3 100824264 65 irq260: igb0:link 2 0 irq261: igb1:que 0 1666960 1 irq262: igb1:que 1 2325576555 1510 irq263: igb1:que 2 1563283 1 irq264: igb1:que 3 1897428 1 irq265: igb1:link 2 0 irq266: mps0 327734440 212 irq267: mps1 193113287 125 irq268: mps2 174367181 113 irq269: ahci0 59169140 38 cpu1:timer 416297615 270 cpu3:timer 327005767 212 cpu2:timer 325504623 211 Total 4874345832 3165 --- igb0 is only running standard/admin stuff, igb1 is running the mentioned dedicated nfs link. As you can see, igb1 gets interrupts on one queue only. I wonder if this is what it should be and if if has any impact on performace. I found several threads on the mailing lists about igb interfaces, queues and interrupts, but so far I could not really make out if this might be an issue for me. Could you have a look on your system and let me know how your interrupts are spread? cu Gerrit From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:37:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E830FB5D for ; Fri, 25 Apr 2014 11:37:06 +0000 (UTC) Received: from mail.blacquiere.nl (provider.blacquiere.nl [144.76.110.43]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92D391685 for ; Fri, 25 Apr 2014 11:37:05 +0000 (UTC) Received: from calendar.blacquiere.nl ([192.168.201.5] helo=shell.blacquiere.nl) by mail.blacquiere.nl with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1Wde4J-000CZs-Bn; Fri, 25 Apr 2014 13:13:43 +0200 Date: Fri, 25 Apr 2014 13:13:16 +0200 From: Robert Blacquiere To: freebsd-net@freebsd.org Message-ID: <20140425111316.GJ9177@calendar.blacquiere.nl> References: <535A1354.2040309@wp.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <535A1354.2040309@wp.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam_score: -4.4 X-Spam_score_int: -43 X-Spam_bar: ---- X-Spam_report: Spam detection software, running on the system "mail", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi Marek, On Fri, Apr 25, 2014 at 09:48:36AM +0200, Marek Salwerowicz wrote: > Hi list, > > Both boxes are connected to the same switch (HP 1910-48G) > > I need to transfer around 10 TB of data from storage1 to storage2 > I obeserve that during copying, only one NIC (instead of all 4) is used. > > Boxes are not stressed during copying > > What's more, apart from having 1 NIC saturated (transfer around 120 > MB/s), I observe transfer rate on level of 70-80 MB/s [...] Content analysis details: (-4.4 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-SA-Exim-Connect-IP: 192.168.201.5 X-SA-Exim-Mail-From: robert@blacquiere.nl X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail X-Spam-Level: X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,BAYES_00, RP_MATCHES_RCVD autolearn=ham version=3.3.2 Subject: Re: NFS over LAGG / lacp poor performance X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.blacquiere.nl) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:37:07 -0000 Hi Marek, On Fri, Apr 25, 2014 at 09:48:36AM +0200, Marek Salwerowicz wrote: > Hi list, > > Both boxes are connected to the same switch (HP 1910-48G) > > I need to transfer around 10 TB of data from storage1 to storage2 > I obeserve that during copying, only one NIC (instead of all 4) is used. > > Boxes are not stressed during copying > > What's more, apart from having 1 NIC saturated (transfer around 120 > MB/s), I observe transfer rate on level of 70-80 MB/s Default lacp 802.3ad works with mac based hashes to loadbalance traffic. So single host (mac) will be transfered by one ethernet adaptor as you have seen. > > I haven't changed any kernel configuration, nor sysctl's - am I missing > sth ? There are tuneables, i think, to change the hash from mac source : dest hash to session tcp source+port: dest:port but these might need also configuring on switch/network. Regards Robert Blacquiere From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:40:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BC3BCD0 for ; Fri, 25 Apr 2014 11:40:56 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3D2816C3 for ; Fri, 25 Apr 2014 11:40:55 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 26385 invoked from network); 25 Apr 2014 13:34:06 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398425646; bh=3LO5PEXlZ5gP08lhQ9m9Lqz2kg0mA3TC9/2GgghCOhM=; h=From:To:CC:Subject; b=md3f79iurp6JYHMxpq11QaFLp/aRd3FWEFkBs15PG4O6ZgvDD9OKxwg4SoFd7K8BD VwvXdYPtga/qUo09D3+7PaIhsvpSBXKRGQFsnjg5TQD+++dunEjLQt61C/uGdkGSQZ Mc6SJCblBz4RBtxKzDu+TNGgQ8hbs7dB8JfZIpro= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 13:34:06 +0200 Message-ID: <535A482E.1030106@wp.pl> Date: Fri, 25 Apr 2014 13:34:06 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Gerrit_K=FChn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> In-Reply-To: <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [cUOE] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:40:56 -0000 Hello Gerrit, W dniu 2014-04-25 11:37, Gerrit Kühn pisze: > On Fri, 25 Apr 2014 09:48:36 +0200 Marek Salwerowicz > wrote about NFS over LAGG / lacp poor performance: > > > > Could you have a look on your system and let me know how your interrupts > are spread? For me on storage1 (9.1-RELEASE) it looks like: > storage1% vmstat > -i > > interrupt total rate > irq1: atkbd0 77 0 > irq16: ehci0 11355459 1 > irq23: ehci1 26468060 3 > cpu0:timer 565127221 79 > irq264: mps0 1905530055 267 > irq265: igb0:que 0 2307223482 323 > irq266: igb0:link 4 0 > irq267: igb1:que 0 271641638 38 > irq268: igb1:link 6 0 > irq269: igb2:que 0 91665104 12 > irq270: igb2:link 6 0 > irq271: igb3:que 0 628139928 88 > irq272: igb3:link 5 0 > irq273: ahci0 1579878 0 > cpu1:timer 283555294 39 > cpu4:timer 285185117 40 > cpu10:timer 305482789 42 > cpu3:timer 239294364 33 > cpu7:timer 406658435 57 > cpu2:timer 353421522 49 > cpu8:timer 437284694 61 > cpu11:timer 261929365 36 > cpu5:timer 282975629 39 > cpu6:timer 248355888 34 > cpu9:timer 256679915 36 > Total 9169553935 1287 > marek@storage1:/home/marek% But in my case all igb links are aggregated using LACP (lagg0), then there are 2 vlans over lagg0 (vlan14 and vlan900) and the vlan900 is one dedicated for NFS And I don't have more than one queue per interface Cheers, Marek -- Marek Salwerowicz From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:46:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC5A5DE6 for ; Fri, 25 Apr 2014 11:46:18 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 801B41767 for ; Fri, 25 Apr 2014 11:46:18 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 1E0AB20E7088A; Fri, 25 Apr 2014 11:46:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=3.0 required=8.0 tests=AWL,BAYES_05,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id C64F920E70885; Fri, 25 Apr 2014 11:46:06 +0000 (UTC) Message-ID: <6312D8095E3B480DAEDDAB9C39A4B581@multiplay.co.uk> From: "Steven Hartland" To: "Robert Blacquiere" , References: <535A1354.2040309@wp.pl> <20140425111316.GJ9177@calendar.blacquiere.nl> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 12:46:10 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:46:18 -0000 ----- Original Message ----- From: "Robert Blacquiere" > Hi Marek, > > On Fri, Apr 25, 2014 at 09:48:36AM +0200, Marek Salwerowicz wrote: >> Hi list, >> > >> Both boxes are connected to the same switch (HP 1910-48G) >> >> I need to transfer around 10 TB of data from storage1 to storage2 >> I obeserve that during copying, only one NIC (instead of all 4) is used. >> >> Boxes are not stressed during copying >> >> What's more, apart from having 1 NIC saturated (transfer around 120 >> MB/s), I observe transfer rate on level of 70-80 MB/s > > Default lacp 802.3ad works with mac based hashes to loadbalance traffic. > So single host (mac) will be transfered by one ethernet adaptor as you > have seen. In FreeBSD 10 onwards the default is l2,l3 & l4 hash this is not true for the switch side, so may need to tune the switch. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:48:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5FFCEB6 for ; Fri, 25 Apr 2014 11:48:58 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 70AA7177B for ; Fri, 25 Apr 2014 11:48:57 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqUEAIVKWlODaFve/2dsb2JhbABZg1VXgmW6AoZnUYEldIIlAQEBAwEBAQEgKyALBRYYAgINGQIpAQkmBggCBQQBHASIGAgNpw2jTReBKYxfAQEbNAeCb4FKBJYrhBKRJINNITGBBDk X-IronPort-AV: E=Sophos;i="4.97,926,1389762000"; d="scan'208";a="117547210" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Apr 2014 07:48:50 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 470CBB40A2; Fri, 25 Apr 2014 07:48:50 -0400 (EDT) Date: Fri, 25 Apr 2014 07:48:50 -0400 (EDT) From: Rick Macklem To: Marek Salwerowicz Message-ID: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> In-Reply-To: <535A1354.2040309@wp.pl> Subject: Re: NFS over LAGG / lacp poor performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:48:59 -0000 Marek Salwerowicz wrote: > Hi list, > > I have two FreeBSD boxes (both based on SuperMicro X9DRD-7LN4F-JBOD > motherboard, with 32GB RAM, 1 CPU :Intel(R) Xeon(R) CPU E5-2640 v2) > > storage1% uname -a > FreeBSD storage1 9.1-RELEASE-p10 FreeBSD 9.1-RELEASE-p10 #0: Sun Jan > 12 > 20:11:23 UTC 2014 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC > amd64 > > > storage2% uname > -a > FreeBSD storage2 10.0-RELEASE-p1 FreeBSD 10.0-RELEASE-p1 #0: Tue Apr > 8 > 06:45:06 UTC 2014 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC > amd64 > > > > They work as NFS storages for VMware ESXi. > > On both there are installed 4 igb 1Gbit NICs, with LACP and 2 VLANs > (vlan900 is for NFS, vlan14 is for management/general purpose): > > on storage1: > igb0: port > 0xd060-0xd07f mem 0xfbb60000-0xfbb7ffff,0xfbb8c000-0xfbb8ffff irq 42 > at > device 0.0 on pci5 > igb1: port > 0xd040-0xd05f mem 0xfbb40000-0xfbb5ffff,0xfbb88000-0xfbb8bfff irq 45 > at > device 0.1 on pci5 > igb2: port > 0xd020-0xd03f mem 0xfbb20000-0xfbb3ffff,0xfbb84000-0xfbb87fff irq 44 > at > device 0.2 on pci5 > igb3: port > 0xd000-0xd01f mem 0xfbb00000-0xfbb1ffff,0xfbb80000-0xfbb83fff irq 46 > at > device 0.3 on pci5 > > on storage2: > igb0: port > 0xd060-0xd07f mem 0xfbb60000-0xfbb7ffff,0xfbb8c000-0xfbb8ffff irq 42 > at > device 0.0 on pci5 > igb1: port > 0xd040-0xd05f mem 0xfbb40000-0xfbb5ffff,0xfbb88000-0xfbb8bfff irq 45 > at > device 0.1 on pci5 > igb2: port > 0xd020-0xd03f mem 0xfbb20000-0xfbb3ffff,0xfbb84000-0xfbb87fff irq 44 > at > device 0.2 on pci5 > igb3: port > 0xd000-0xd01f mem 0xfbb00000-0xfbb1ffff,0xfbb80000-0xfbb83fff irq 46 > at > device 0.3 on pci5 > > > storage1% ifconfig -a > igb0: flags=8843 metric 0 mtu > 9000 > > options=401bb > ether 00:25:90:c1:1d:18 > inet6 fe80::225:90ff:fec1:1d18%igb0 prefixlen 64 scopeid 0x1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb1: flags=8843 metric 0 mtu > 9000 > > options=401bb > ether 00:25:90:c1:1d:18 > inet6 fe80::225:90ff:fec1:1d19%igb1 prefixlen 64 scopeid 0x2 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb2: flags=8843 metric 0 mtu > 9000 > > options=401bb > ether 00:25:90:c1:1d:18 > inet6 fe80::225:90ff:fec1:1d1a%igb2 prefixlen 64 scopeid 0x3 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb3: flags=8843 metric 0 mtu > 9000 > > options=401bb > ether 00:25:90:c1:1d:18 > inet6 fe80::225:90ff:fec1:1d1b%igb3 prefixlen 64 scopeid 0x4 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > lagg0: flags=8843 metric 0 > mtu 9000 > > options=401bb > ether 00:25:90:c1:1d:18 > inet6 fe80::225:90ff:fec1:1d18%lagg0 prefixlen 64 scopeid 0x8 > nd6 options=21 > media: Ethernet autoselect > status: active > laggproto lacp lagghash l2,l3,l4 > laggport: igb3 flags=1c > laggport: igb2 flags=1c > laggport: igb1 flags=1c > laggport: igb0 flags=1c > vlan14: flags=8843 metric 0 > mtu 9000 > options=103 > ether 00:25:90:c1:1d:18 > inet 192.168.1.65 netmask 0xffffff00 broadcast 192.168.1.255 > inet6 fe80::225:90ff:fec1:1d18%vlan14 prefixlen 64 scopeid > 0x9 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 14 parent interface: lagg0 > vlan900: flags=8843 metric 0 > mtu > 9000 > options=103 > ether 00:25:90:c1:1d:18 > inet 172.25.25.65 netmask 0xffffff00 broadcast 172.25.25.255 > inet6 fe80::225:90ff:fec1:1d18%vlan900 prefixlen 64 scopeid > 0xa > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 900 parent interface: lagg0 > > > storage2% ifconfig > -a > igb0: flags=8843 metric 0 mtu > 9000 > > options=403bb > ether 00:25:90:ca:3b:e0 > inet6 fe80::225:90ff:feca:3be0%igb0 prefixlen 64 scopeid 0x1 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb1: flags=8843 metric 0 mtu > 9000 > > options=403bb > ether 00:25:90:ca:3b:e0 > inet6 fe80::225:90ff:feca:3be1%igb1 prefixlen 64 scopeid 0x2 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb2: flags=8843 metric 0 mtu > 9000 > > options=403bb > ether 00:25:90:ca:3b:e0 > inet6 fe80::225:90ff:feca:3be2%igb2 prefixlen 64 scopeid 0x3 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > igb3: flags=8843 metric 0 mtu > 9000 > > options=403bb > ether 00:25:90:ca:3b:e0 > inet6 fe80::225:90ff:feca:3be3%igb3 prefixlen 64 scopeid 0x4 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > lagg0: flags=8843 metric 0 > mtu 9000 > > options=403bb > ether 00:25:90:ca:3b:e0 > inet6 fe80::225:90ff:feca:3be0%lagg0 prefixlen 64 scopeid 0x6 > nd6 options=21 > media: Ethernet autoselect > status: active > laggproto lacp lagghash l2,l3,l4 > laggport: igb3 flags=1c > laggport: igb2 flags=1c > laggport: igb1 flags=1c > laggport: igb0 flags=1c > vlan14: flags=8843 metric 0 > mtu 9000 > options=303 > ether 00:25:90:ca:3b:e0 > inet 192.168.1.72 netmask 0xffffff00 broadcast 192.168.1.255 > inet6 fe80::225:90ff:feca:3be0%vlan14 prefixlen 64 scopeid > 0x7 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 14 parent interface: lagg0 > vlan900: flags=8843 metric 0 > mtu > 9000 > options=303 > ether 00:25:90:ca:3b:e0 > inet 172.25.25.72 netmask 0xffffff00 broadcast 172.25.25.255 > inet6 fe80::225:90ff:feca:3be0%vlan900 prefixlen 64 scopeid > 0x8 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 900 parent interface: lagg0 > > > Both boxes are connected to the same switch (HP 1910-48G) > > I need to transfer around 10 TB of data from storage1 to storage2 > I obeserve that during copying, only one NIC (instead of all 4) is > used. > Well, you don't mention what command(s) you are using to transfer the data, but I would guess you have one serial data transfer for each command. (Put another way, if you are only running one command to transfer the data, there will only be one RPC happening at a time and that will only use one network interface.) I don't know anything about lagg, so I can't comment related to it, but if there is only one NFS RPC at a time, you'll only be transferring one message at a time on the wire.) Adding the mount option "readahead=8" to the machine receiving the data might help, if the data transfer command is being done there. (ie. The machine the data is being copied to has the other one NFS mounted and it is where you are running the data transfer command(s).) If you can run commands concurrently to transfer parts of the data, that might help, too. Good luck with it, rick > Boxes are not stressed during copying > > What's more, apart from having 1 NIC saturated (transfer around 120 > MB/s), I observe transfer rate on level of 70-80 MB/s > > I haven't changed any kernel configuration, nor sysctl's - am I > missing > sth ? > > Regards, > Marek > > -- > Marek Salwerowicz > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:58:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DB5730F for ; Fri, 25 Apr 2014 11:58:00 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00DD41870 for ; Fri, 25 Apr 2014 11:57:59 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 19820 invoked from network); 25 Apr 2014 13:57:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398427073; bh=KGhtPINQ/GZjGenWso+t5UUmPhypV59vqGjvQLZihZ8=; h=From:To:CC:Subject; b=JB53817nk9g2gHvU635zLw7srt63np+zTl/IoYg9WUkE/b7R1Du8HRGAr7CcxiPX6 jt2WzRpmuPJPxTqC3f5t0qEG4OBV8PKn4OypZ8uaOLTOoFRCJ+AVpnD1s7lOP2xhnz 3RjJTuDYPReOJUuYXbEQySp3Wr+0nLeGsj/nXUGw= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 13:57:53 +0200 Message-ID: <535A4DC2.605@wp.pl> Date: Fri, 25 Apr 2014 13:57:54 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rick Macklem Subject: Re: NFS over LAGG / lacp poor performance References: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> In-Reply-To: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [AdOE] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:58:00 -0000 W dniu 2014-04-25 13:48, Rick Macklem pisze: > Well, you don't mention what command(s) you are using to transfer the > data, but I would guess you have one serial data transfer for each command. > (Put another way, if you are only running one command to transfer the data, > there will only be one RPC happening at a time and that will only use one > network interface.) I don't know anything about lagg, so I can't comment > related to it, but if there is only one NFS RPC at a time, you'll only > be transferring one message at a time on the wire.) I need to transfer 15 files, each is about 1TB sized. >From 9.1-RELEASE[storage1] to 10-RELEASE[storage2] I have tried to run concurrent 'cp' and transfer 4 files at the same time: (executed on storage1) # cp -a file1 /net/storage2/ & # cp -a file2 /net/storage2/ & # cp -a file3 /net/storage2/ & # cp -a file4 /net/storage2/ & But in fact I did not observe bigger throughput Both servers have filesystem exported using NFS, so I can execute copy on source, or destination. Would you recommend running this on source-side, or rather destination-side ? > > Adding the mount option "readahead=8" to the machine receiving the data > might help, if the data transfer command is being done there. (ie. The machine > the data is being copied to has the other one NFS mounted and it is where > you are running the data transfer command(s).) Regarding what I wrote above - how should I mount the NFS volumes? Cheers, Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 12:01:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06B38855 for ; Fri, 25 Apr 2014 12:01:38 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 8956E195B for ; Fri, 25 Apr 2014 12:01:37 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 5AFF82009E7; Fri, 25 Apr 2014 14:01:34 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 4E780405889; Fri, 25 Apr 2014 14:01:34 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id 1F787406AF1; Fri, 25 Apr 2014 14:01:34 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042514012379-88367 ; Fri, 25 Apr 2014 14:01:23 +0200 Date: Fri, 25 Apr 2014 14:01:23 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Marek Salwerowicz Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> In-Reply-To: <535A482E.1030106@wp.pl> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 14:01:23, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 14:01:33, Serialize complete at 04/25/2014 14:01:33 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.114821 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1800_1899 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __RUS_OBFU_PHONE 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:01:38 -0000 On Fri, 25 Apr 2014 13:34:06 +0200 Marek Salwerowicz wrote about Re: NFS over LAGG / lacp poor performance: GK> irq256: igb0:que 0 99396134 64 GK> irq257: igb0:que 1 61496018 39 GK> irq258: igb0:que 2 101687742 66 GK> irq259: igb0:que 3 100824264 65 GK> irq260: igb0:link 2 0 GK> irq261: igb1:que 0 1666960 1 GK> irq262: igb1:que 1 2325576555 1510 GK> irq263: igb1:que 2 1563283 1 GK> irq264: igb1:que 3 1897428 1 GK> irq265: igb1:link 2 0 MS> For me on storage1 (9.1-RELEASE) it looks like: MS> irq265: igb0:que 0 2307223482 323 MS> irq266: igb0:link 4 0 MS> irq267: igb1:que 0 271641638 38 MS> irq268: igb1:link 6 0 MS> irq269: igb2:que 0 91665104 12 MS> irq270: igb2:link 6 0 MS> irq271: igb3:que 0 628139928 88 MS> irq272: igb3:link 5 0 MS> But in my case all igb links are aggregated using LACP (lagg0), then MS> there are 2 vlans over lagg0 (vlan14 and vlan900) and the vlan900 is MS> one dedicated for NFS MS> And I don't have more than one queue per interface Thanks for your input. As far as I understood so far, there should be one igb queue created per cpu core in the system by default (and this is what I see on my system). But my irq rate looks quite high to me (and it is only on one of these queues). Maybe I'll try to reduce this to one queue and see what happens. Does anybody else in here happen to know something about this? cu Gerrit From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 12:17:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E51F259 for ; Fri, 25 Apr 2014 12:17:47 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6A4F1AC4 for ; Fri, 25 Apr 2014 12:17:46 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 16843 invoked from network); 25 Apr 2014 14:17:44 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398428264; bh=LhZ0ijM8dZhGowCeyTLXjGT5GScunpkSwFFsv1XuB8s=; h=From:To:CC:Subject; b=WP4w2cJYFiQkhB/A1BTbdqXSMb7NKWTVrA/j1kAVQrgOAOux/H8eRmf/fupcn8CjO uk76+LXMShJ8GSK2Umy5OmWNVyh50mDjgaCbYCqNBwugeGx4rPXZs6HO4HZWRSviLP q5aYFwIvedCT9NOhxWr46Knm2AHZjOq4Td3uTtg4= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 14:17:44 +0200 Message-ID: <535A5268.100@wp.pl> Date: Fri, 25 Apr 2014 14:17:44 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Gerrit_K=FChn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> In-Reply-To: <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [YcO3] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:17:47 -0000 W dniu 2014-04-25 14:01, Gerrit Kühn pisze: > Thanks for your input. As far as I understood so far, there should be one > igb queue created per cpu core in the system by default (and this is what > I see on my system). But my irq rate looks quite high to me (and it is > only on one of these queues). My CPU has 8 cores: http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz So why do I have only 1 queue ? Cheers, Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 12:52:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 097D3CC for ; Fri, 25 Apr 2014 12:52:17 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8306D1FF9 for ; Fri, 25 Apr 2014 12:52:16 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 2575 invoked from network); 25 Apr 2014 14:45:33 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398429933; bh=yyam/GXU3bXOHT7PiV9z4Z2BfK6VKq1itNc1iqMNLM0=; h=From:To:CC:Subject; b=phKWKe+23Js1HyUpIBvfjmFduGEPvhCBACtUzFNahd7+FiBOyXb8Kk7+YcEfVtFCY r7AFj9DWw+vR9tG69sOAA6oL565FGS1svgyFgXLc80Me+lK2Zz7gIqPfflZvhDDTG0 7fj/ek6DiYdxhTB9M4mIzYKAVUTZmMyi0nkG4cLU= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 14:45:33 +0200 Message-ID: <535A58ED.3060608@wp.pl> Date: Fri, 25 Apr 2014 14:45:33 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Rick Macklem Subject: Re: NFS over LAGG / lacp poor performance References: <562289962.736994.1398426530226.JavaMail.root@uoguelph.ca> <535A4DC2.605@wp.pl> In-Reply-To: <535A4DC2.605@wp.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [8QOU] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:52:17 -0000 W dniu 2014-04-25 13:57, Marek Salwerowicz pisze: > (executed on storage1) > # cp -a file1 /net/storage2/ & > # cp -a file2 /net/storage2/ & > # cp -a file3 /net/storage2/ & > # cp -a file4 /net/storage2/ & Just to add: both source and destination are based on ZFS raidz2 pool with mirrored log SSD drive -- Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 12:55:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84B6339A for ; Fri, 25 Apr 2014 12:55:57 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 456591070 for ; Fri, 25 Apr 2014 12:55:57 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id A492820E7088B; Fri, 25 Apr 2014 12:55:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id B6E5120E7088A; Fri, 25 Apr 2014 12:55:51 +0000 (UTC) Message-ID: <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> From: "Steven Hartland" To: "Marek Salwerowicz" , =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 13:55:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:55:57 -0000 ----- Original Message ----- From: "Marek Salwerowicz" >W dniu 2014-04-25 14:01, Gerrit Kühn pisze: >> Thanks for your input. As far as I understood so far, there should be one >> igb queue created per cpu core in the system by default (and this is what >> I see on my system). But my irq rate looks quite high to me (and it is >> only on one of these queues). > > > My CPU has 8 cores: > > http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz > > So why do I have only 1 queue ? What does "sysctl hw.igb.num_queues" report? num_queues does default to 1 for Legacy or MSI so you might be hitting that. Do you see "Using MSIX interrupts with" in your dmesg? Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:06:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B56F0853 for ; Fri, 25 Apr 2014 13:06:37 +0000 (UTC) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 371131189 for ; Fri, 25 Apr 2014 13:06:36 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 19382 invoked from network); 25 Apr 2014 15:06:32 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398431192; bh=FuslYgNyY9CygrIReoNvM79EN5PWotnIhWtPeWl+aCQ=; h=From:To:CC:Subject; b=AVXG/xK6H5opF3tbnF5NnRxq8eXsnRgPGujkKsCjkzAVTRBQoaQAOWXm4s97Xl7tc 7dpFWmpfA/VPhU+MVrJ+vS2ISzH+cNF6usSQEcVcZmtdRcmO0YZNhc72lM2Zf3Kevt a4VDZzkJtB+6P4a/p0e53VqBMjWe+gHeOVlVnud0= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 15:06:32 +0200 Message-ID: <535A5DD9.9060206@wp.pl> Date: Fri, 25 Apr 2014 15:06:33 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steven Hartland , =?ISO-8859-1?Q?Gerrit_K=FC?= =?ISO-8859-1?Q?hn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> In-Reply-To: <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [YUMk] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:06:37 -0000 W dniu 2014-04-25 14:55, Steven Hartland pisze: > ----- Original Message ----- From: "Marek Salwerowicz" > > >> W dniu 2014-04-25 14:01, Gerrit Kühn pisze: >>> Thanks for your input. As far as I understood so far, there should >>> be one >>> igb queue created per cpu core in the system by default (and this is >>> what >>> I see on my system). But my irq rate looks quite high to me (and it is >>> only on one of these queues). >> >> >> My CPU has 8 cores: >> >> http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz >> >> >> So why do I have only 1 queue ? > > What does "sysctl hw.igb.num_queues" report? storage1% sysctl hw.igb.num_queues hw.igb.num_queues: 1 > > num_queues does default to 1 for Legacy or MSI so you might be hitting > that. > > Do you see "Using MSIX interrupts with" in your dmesg? storage% dmesg | grep MSIX igb0: Using MSIX interrupts with 2 vectors igb1: Using MSIX interrupts with 2 vectors igb2: Using MSIX interrupts with 2 vectors igb3: Using MSIX interrupts with 2 vectors igb0: Using MSIX interrupts with 2 vectors igb1: Using MSIX interrupts with 2 vectors igb2: Using MSIX interrupts with 2 vectors igb3: Using MSIX interrupts with 2 vectors -- Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:08:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06530B35 for ; Fri, 25 Apr 2014 13:08:52 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C06F611B0 for ; Fri, 25 Apr 2014 13:08:51 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAKRQWlODaFve/2dsb2JhbABZhCyCZb4rgw+BJXSCJQEBBAEjVgUWGAICDRkCWQYKiEIIpx2jTheBKYx8NAeCb4FKBKthg00hgW4 X-IronPort-AV: E=Sophos;i="4.97,927,1389762000"; d="scan'208";a="117959068" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 25 Apr 2014 09:08:41 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id AA832B3F4A; Fri, 25 Apr 2014 09:08:41 -0400 (EDT) Date: Fri, 25 Apr 2014 09:08:41 -0400 (EDT) From: Rick Macklem To: Marek Salwerowicz Message-ID: <31234037.791330.1398431321686.JavaMail.root@uoguelph.ca> In-Reply-To: <535A4DC2.605@wp.pl> Subject: Re: NFS over LAGG / lacp poor performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:08:52 -0000 Marek Salwerowicz wrote: > W dniu 2014-04-25 13:48, Rick Macklem pisze: > > Well, you don't mention what command(s) you are using to transfer > > the > > data, but I would guess you have one serial data transfer for each > > command. > > (Put another way, if you are only running one command to transfer > > the data, > > there will only be one RPC happening at a time and that will only > > use one > > network interface.) I don't know anything about lagg, so I can't > > comment > > related to it, but if there is only one NFS RPC at a time, you'll > > only > > be transferring one message at a time on the wire.) > > I need to transfer 15 files, each is about 1TB sized. > > From 9.1-RELEASE[storage1] to 10-RELEASE[storage2] > > I have tried to run concurrent 'cp' and transfer 4 files at the same > time: > > (executed on storage1) > # cp -a file1 /net/storage2/ & > # cp -a file2 /net/storage2/ & > # cp -a file3 /net/storage2/ & > # cp -a file4 /net/storage2/ & > Although I doubt it will make much difference, you might want to try "dd" with a fairly large blocksize (at least 64K). I don't know what blocksize "cp" uses and whether or not it does mmap'd file access. (mmap will only do I/O in page size blocks, so I think it will be slower.) > > But in fact I did not observe bigger throughput > > Both servers have filesystem exported using NFS, so I can execute > copy > on source, or destination. > Would you recommend running this on source-side, or rather > destination-side ? > Usually reads run faster than writes for NFS, so I'd try doing the mounts and running the commands on the destination side. That is also when "readahead=8" might help some. I'd add that option to the NFS mount (you can try any value you'd like, up to 16, but if 8 doesn't run faster than the default of 1, it probably isn't worth trying other values). > > > > Adding the mount option "readahead=8" to the machine receiving the > > data > > might help, if the data transfer command is being done there. (ie. > > The machine > > the data is being copied to has the other one NFS mounted and it is > > where > > you are running the data transfer command(s).) > > > Regarding what I wrote above - how should I mount the NFS volumes? > As above, I'd use nfsv3,readahead=8 options on the destination as a starting point. rick > Cheers, > Marek > From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:27:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D508227 for ; Fri, 25 Apr 2014 13:27:58 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id F23B0138D for ; Fri, 25 Apr 2014 13:27:57 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 382C620E7088B; Fri, 25 Apr 2014 13:27:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 31C0D20E70885; Fri, 25 Apr 2014 13:27:52 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Marek Salwerowicz" , =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 14:27:56 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:27:58 -0000 ----- Original Message ----- From: "Marek Salwerowicz" To: "Steven Hartland" ; "Gerrit Kühn" Cc: Sent: Friday, April 25, 2014 2:06 PM Subject: Re: NFS over LAGG / lacp poor performance >W dniu 2014-04-25 14:55, Steven Hartland pisze: >> ----- Original Message ----- From: "Marek Salwerowicz" >> >> >>> W dniu 2014-04-25 14:01, Gerrit Kühn pisze: >>>> Thanks for your input. As far as I understood so far, there should >>>> be one >>>> igb queue created per cpu core in the system by default (and this is >>>> what >>>> I see on my system). But my irq rate looks quite high to me (and it is >>>> only on one of these queues). >>> >>> >>> My CPU has 8 cores: >>> >>> http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz >>> >>> >>> So why do I have only 1 queue ? >> >> What does "sysctl hw.igb.num_queues" report? > > storage1% sysctl hw.igb.num_queues > hw.igb.num_queues: 1 >> >> num_queues does default to 1 for Legacy or MSI so you might be hitting >> that. >> >> Do you see "Using MSIX interrupts with" in your dmesg? > storage% dmesg | grep MSIX > igb0: Using MSIX interrupts with 2 vectors > igb1: Using MSIX interrupts with 2 vectors > igb2: Using MSIX interrupts with 2 vectors > igb3: Using MSIX interrupts with 2 vectors > igb0: Using MSIX interrupts with 2 vectors > igb1: Using MSIX interrupts with 2 vectors > igb2: Using MSIX interrupts with 2 vectors > igb3: Using MSIX interrupts with 2 vectors In that case I believe you've hard coded the number of queues, check /boot/loader.conf for references to this. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:51:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23DC2A8A for ; Fri, 25 Apr 2014 13:51:06 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DA8401602 for ; Fri, 25 Apr 2014 13:51:05 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAMphWlODaFve/2dsb2JhbABZg1VXgmW6AoZnUYEldIIlAQEBAwEBAQEgKyALBQcPDgMEAQEBAgINGQIpAQkeCAYIAgUEARwEiAwDCQgNpzCcTiGGWBeBKYxOEAIBBhUBMwcGgmmBSgSWK4QSkSSDTSExfEE X-IronPort-AV: E=Sophos;i="4.97,927,1389762000"; d="scan'208";a="117571142" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Apr 2014 09:51:04 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2C9D7B3EA2; Fri, 25 Apr 2014 09:51:04 -0400 (EDT) Date: Fri, 25 Apr 2014 09:51:04 -0400 (EDT) From: Rick Macklem To: Steven Hartland Message-ID: <1778189736.824550.1398433864172.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: NFS over LAGG / lacp poor performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, Marek Salwerowicz , Gerrit =?utf-8?B?S8O8aG4=?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:51:06 -0000 Steven Hartland wrote: >=20 > ----- Original Message ----- > From: "Marek Salwerowicz" > To: "Steven Hartland" ; "Gerrit K=C3=BChn" > > Cc: > Sent: Friday, April 25, 2014 2:06 PM > Subject: Re: NFS over LAGG / lacp poor performance >=20 >=20 > >W dniu 2014-04-25 14:55, Steven Hartland pisze: > >> ----- Original Message ----- From: "Marek Salwerowicz" > >> > >> > >> > >>> W dniu 2014-04-25 14:01, Gerrit K=C3=BChn pisze: > >>>> Thanks for your input. As far as I understood so far, there > >>>> should > >>>> be one > >>>> igb queue created per cpu core in the system by default (and > >>>> this is > >>>> what > >>>> I see on my system). But my irq rate looks quite high to me (and > >>>> it is > >>>> only on one of these queues). > >>> > >>> > >>> My CPU has 8 cores: > >>> > >>> http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-2= 0M-Cache-2_00-GHz > >>> > >>> > >>> So why do I have only 1 queue ? > >> > >> What does "sysctl hw.igb.num_queues" report? > > > > storage1% sysctl hw.igb.num_queues > > hw.igb.num_queues: 1 > >> > >> num_queues does default to 1 for Legacy or MSI so you might be > >> hitting > >> that. > >> > >> Do you see "Using MSIX interrupts with" in your dmesg? > > storage% dmesg | grep MSIX > > igb0: Using MSIX interrupts with 2 vectors > > igb1: Using MSIX interrupts with 2 vectors > > igb2: Using MSIX interrupts with 2 vectors > > igb3: Using MSIX interrupts with 2 vectors > > igb0: Using MSIX interrupts with 2 vectors > > igb1: Using MSIX interrupts with 2 vectors > > igb2: Using MSIX interrupts with 2 vectors > > igb3: Using MSIX interrupts with 2 vectors >=20 > In that case I believe you've hard coded the number of queues, check > /boot/loader.conf > for references to this. >=20 Not really replying to Steve's email, but... NFS uses a single TCP connection for a mount. I still know nothing about lagg, but if lagg/lacp requires multiple TCP connections to spread the load..I'd just switch to using something like ftp, given you are only moving a few large files. If you must use NFS, then to get multiple TCP connections, you'll need to do multiple mounts and then do the file transfers concurrently over the different mounts. rick > Regards > Steve >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 13:57:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF0D5CDD for ; Fri, 25 Apr 2014 13:57:37 +0000 (UTC) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4176B16E7 for ; Fri, 25 Apr 2014 13:57:36 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 7642 invoked from network); 25 Apr 2014 15:57:34 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398434254; bh=Vmr2PC2ASXIaaCp5URDizlskGQmRFyGk8ZyrG+xvslw=; h=From:To:CC:Subject; b=f6LWhfXm2gaieFGzNjVvptLn/2Uj38sDS2QvkBcBPOuEyKBUz//oK3F7QFjIv9P14 +y1WeuNRKcOD01RXZhk9Klu7JVBC3tdQla4TkHosk3JyQ95zORnQRS/wPkeRpabsnX SuAzlCvgOnJ9TbjTmXGv50n3lfJdX0ULT4+0LNhs= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 15:57:34 +0200 Message-ID: <535A69CE.9010800@wp.pl> Date: Fri, 25 Apr 2014 15:57:34 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steven Hartland , =?ISO-8859-1?Q?Gerrit_K=FC?= =?ISO-8859-1?Q?hn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [kdNk] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:57:37 -0000 W dniu 2014-04-25 15:27, Steven Hartland pisze: > In that case I believe you've hard coded the number of queues, check > /boot/loader.conf > for references to this. Yes, that's true: % cat /boot/loader.conf debug.acpi.max_tasks="128" if_lagg_load="YES" kern.ipc.nmbclusters="131072" hw.igb.num_queues="1" I am wondering why I have turned this on.. My box has raidz2 ZFS box with SSD mirrored log.. -- Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:02:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADE6316C for ; Fri, 25 Apr 2014 14:02:29 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 6DF1217E2 for ; Fri, 25 Apr 2014 14:02:29 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id A937120E7088B; Fri, 25 Apr 2014 14:02:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 0371820E70886; Fri, 25 Apr 2014 14:02:23 +0000 (UTC) Message-ID: <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> From: "Steven Hartland" To: "Marek Salwerowicz" , =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 15:02:15 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:02:29 -0000 ----- Original Message ----- From: "Marek Salwerowicz" To: "Steven Hartland" ; "Gerrit Kühn" Cc: Sent: Friday, April 25, 2014 2:57 PM Subject: Re: NFS over LAGG / lacp poor performance >W dniu 2014-04-25 15:27, Steven Hartland pisze: >> In that case I believe you've hard coded the number of queues, check >> /boot/loader.conf >> for references to this. > > Yes, that's true: > > % cat /boot/loader.conf > debug.acpi.max_tasks="128" > if_lagg_load="YES" > kern.ipc.nmbclusters="131072" > hw.igb.num_queues="1" > > > I am wondering why I have turned this on.. > My box has raidz2 ZFS box with SSD mirrored log.. We find that large numbers of queues causes high interrupt issues however at a guess you did this to enable the machine to boot with all nics due to lack of auto mbuf tuning in 9.x. I'd go with ~2 queues per nic. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:10:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7BBA358 for ; Fri, 25 Apr 2014 14:10:03 +0000 (UTC) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.wp.pl", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 674CD1875 for ; Fri, 25 Apr 2014 14:10:03 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 31614 invoked from network); 25 Apr 2014 16:10:01 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wp.pl; s=1024a; t=1398435001; bh=vf8tAIn67rxV5poGn6IMGKCHqcIgixwrex1SdHL9TcE=; h=From:To:CC:Subject; b=dsY+x5OxG3j5fELQirI0SyEurFIsxwKiUaMx9Z+FDUIfpJWJoXXZClFmk0YJk5uhc XkI1prhyIR4LorjZbrkaifdthzSHCil4fHoRnCGQXpYkKDdFYjZKW8XinKwXw5QZqv o52gcMO5wy2OcfGYrL+gbi8mo2Np2qc/FuuiWF9g= Received: from pb-d-128-141-237-169.cern.ch (HELO [128.141.237.169]) (marek_sal@[128.141.237.169]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with AES128-SHA encrypted SMTP for ; 25 Apr 2014 16:10:01 +0200 Message-ID: <535A6CB9.10107@wp.pl> Date: Fri, 25 Apr 2014 16:10:01 +0200 From: Marek Salwerowicz User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steven Hartland , =?ISO-8859-1?Q?Gerrit_K=FC?= =?ISO-8859-1?Q?hn?= Subject: Re: NFS over LAGG / lacp poor performance References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> In-Reply-To: <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [ATPk] Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:10:04 -0000 W dniu 2014-04-25 16:02, Steven Hartland pisze: > We find that large numbers of queues causes high interrupt issues > however at a guess you did this to enable the machine to boot with > all nics due to lack of auto mbuf tuning in 9.x. > > I'd go with ~2 queues per nic. Yes, probably you're right as the modification date points to the time period when I turned on the LAGG As I understand, on 10.x it's fixed and having more queues results in better performance? As far as I will have faster transfer between storage1 and storage2, I'd plan to upgrade 9.x to 10.x Cheers, Marek From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:13:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BA2F484 for ; Fri, 25 Apr 2014 14:13:26 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 0D6F71917 for ; Fri, 25 Apr 2014 14:13:25 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id E4F8C20110A; Fri, 25 Apr 2014 16:13:23 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id C4434405889; Fri, 25 Apr 2014 16:13:23 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id EBFAA406AF1; Fri, 25 Apr 2014 16:13:22 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042516131167-88690 ; Fri, 25 Apr 2014 16:13:11 +0200 Date: Fri, 25 Apr 2014 16:13:12 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "Steven Hartland" Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> In-Reply-To: <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 16:13:11, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 16:13:21, Serialize complete at 04/25/2014 16:13:21 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.140019 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_2000_2999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:13:26 -0000 On Fri, 25 Apr 2014 15:02:15 +0100 "Steven Hartland" wrote about Re: NFS over LAGG / lacp poor performance: SH> We find that large numbers of queues causes high interrupt issues Like the thing I am seeing with igb1 on my system? --- root@storage:/root # vmstat -i interrupt total rate irq1: atkbd0 2743 0 irq18: ehci0 uhci5 4445560 2 irq21: uhci1 29 0 cpu0:timer 355724275 227 irq256: igb0:que 0 99437514 63 irq257: igb0:que 1 61534816 39 irq258: igb0:que 2 101725601 65 irq259: igb0:que 3 100864440 64 irq260: igb0:link 2 0 irq261: igb1:que 0 1689527 1 irq262: igb1:que 1 2357590958 1510 irq263: igb1:que 2 1584474 1 irq264: igb1:que 3 1923144 1 irq265: igb1:link 2 0 irq266: mps0 332232450 212 irq267: mps1 194207894 124 irq268: mps2 176700834 113 irq269: ahci0 59175548 37 cpu1:timer 419838321 268 cpu3:timer 329696415 211 cpu2:timer 328219053 210 Total 4926593600 3156 --- irq262 sticks out like a sore thumb... SH> however at a guess you did this to enable the machine to boot with SH> all nics due to lack of auto mbuf tuning in 9.x. SH> I'd go with ~2 queues per nic. I was wondering what to try next for my system: either manually set the queues back to 2 or 1 per NIC, or try upgrading to either 9.2 or 10 as it looked like there have been improvements in the igb driver. Do you have any recommendations on that? cu Gerrit From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:25:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4FD7A68 for ; Fri, 25 Apr 2014 14:25:05 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 5B9FC1A74 for ; Fri, 25 Apr 2014 14:25:05 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id B639720E7088B; Fri, 25 Apr 2014 14:25:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id D7C8720E70885; Fri, 25 Apr 2014 14:25:00 +0000 (UTC) Message-ID: <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> From: "Steven Hartland" To: =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl><20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de><535A482E.1030106@wp.pl><20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de><535A5268.100@wp.pl><8247FE6336414E1F97ADA561D0680097@multiplay.co.uk><535A5DD9.9060206@wp.pl><535A69CE.9010800@wp.pl><4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 15:25:04 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:25:05 -0000 ----- Original Message ----- From: "Gerrit Kühn" To: "Steven Hartland" Cc: Sent: Friday, April 25, 2014 3:13 PM Subject: Re: NFS over LAGG / lacp poor performance > On Fri, 25 Apr 2014 15:02:15 +0100 "Steven Hartland" > wrote about Re: NFS over LAGG / lacp poor > performance: > > SH> We find that large numbers of queues causes high interrupt issues > > Like the thing I am seeing with igb1 on my system? > > --- > root@storage:/root # vmstat -i > interrupt total rate > irq1: atkbd0 2743 0 > irq18: ehci0 uhci5 4445560 2 > irq21: uhci1 29 0 > cpu0:timer 355724275 227 > irq256: igb0:que 0 99437514 63 > irq257: igb0:que 1 61534816 39 > irq258: igb0:que 2 101725601 65 > irq259: igb0:que 3 100864440 64 > irq260: igb0:link 2 0 > irq261: igb1:que 0 1689527 1 > irq262: igb1:que 1 2357590958 1510 > irq263: igb1:que 2 1584474 1 > irq264: igb1:que 3 1923144 1 > irq265: igb1:link 2 0 > irq266: mps0 332232450 212 > irq267: mps1 194207894 124 > irq268: mps2 176700834 113 > irq269: ahci0 59175548 37 > cpu1:timer 419838321 268 > cpu3:timer 329696415 211 > cpu2:timer 328219053 210 > Total 4926593600 3156 > --- > > > irq262 sticks out like a sore thumb... > > SH> however at a guess you did this to enable the machine to boot with > SH> all nics due to lack of auto mbuf tuning in 9.x. > SH> I'd go with ~2 queues per nic. > > I was wondering what to try next for my system: either manually set the > queues back to 2 or 1 per NIC, or try upgrading to either 9.2 or 10 as it > looked like there have been improvements in the igb driver. Do you have > any recommendations on that? We saw the issue with 10-RELEASE last weekend on a machine with 6 x igb's where by the system was burning CPU in the interrupt handlers, setting num_queues to 2 fixed the issue we where seeing. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:34:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D320C11 for ; Fri, 25 Apr 2014 14:34:45 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id DF3421B4F for ; Fri, 25 Apr 2014 14:34:44 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id EE2E320110D; Fri, 25 Apr 2014 16:34:43 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id E009740588A; Fri, 25 Apr 2014 16:34:43 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id C3FC6405889; Fri, 25 Apr 2014 16:34:43 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042516343306-88722 ; Fri, 25 Apr 2014 16:34:33 +0200 Date: Fri, 25 Apr 2014 16:34:33 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "Steven Hartland" Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> In-Reply-To: <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 16:34:33, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 16:34:43, Serialize complete at 04/25/2014 16:34:43 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.142420 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_500_599 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:34:45 -0000 On Fri, 25 Apr 2014 15:25:04 +0100 "Steven Hartland" wrote about Re: NFS over LAGG / lacp poor performance: SH> We saw the issue with 10-RELEASE last weekend on a machine with 6 x SH> igb's where by the system was burning CPU in the interrupt handlers, SH> setting num_queues to 2 fixed the issue we where seeing. Sounds like reducing the queues is the first thing to try then, thanks! How did you verify that the irqs were the culprit? Apart from the high irq rate in vmstat I do not see any other load values skyrocketing. cu Gerrit From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:55:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA16F381 for ; Fri, 25 Apr 2014 14:55:31 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 67A9E1D25 for ; Fri, 25 Apr 2014 14:55:31 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id AB34120E7088B; Fri, 25 Apr 2014 14:55:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id D438A20E70885; Fri, 25 Apr 2014 14:55:25 +0000 (UTC) Message-ID: <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> From: "Steven Hartland" To: =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl><20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de><535A482E.1030106@wp.pl><20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de><535A5268.100@wp.pl><8247FE6336414E1F97ADA561D0680097@multiplay.co.uk><535A5DD9.9060206@wp.pl><535A69CE.9010800@wp.pl><4E7A280CACC84910A3DE068085976E30@multiplay.co.uk><20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de><1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 15:55:29 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:55:31 -0000 ----- Original Message ----- From: "Gerrit Kühn" > On Fri, 25 Apr 2014 15:25:04 +0100 "Steven Hartland" > wrote about Re: NFS over LAGG / lacp poor > performance: > > SH> We saw the issue with 10-RELEASE last weekend on a machine with 6 x > SH> igb's where by the system was burning CPU in the interrupt handlers, > SH> setting num_queues to 2 fixed the issue we where seeing. > > Sounds like reducing the queues is the first thing to try then, thanks! > How did you verify that the irqs were the culprit? Apart from the high irq > rate in vmstat I do not see any other load values skyrocketing. It was more CPU usage in top. Run top then enable system processes (shift+S) and threads (shift+H) then disable idle process display (z) if your seeing significant CPU usage to the igb interrupt handlers try reducing the number. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 15:52:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F946AB4 for ; Fri, 25 Apr 2014 15:52:43 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 0D87314C6 for ; Fri, 25 Apr 2014 15:52:43 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id D44BB201110; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id C7C7B405889; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id A3A50406AF1; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042517523117-88837 ; Fri, 25 Apr 2014 17:52:31 +0200 Date: Fri, 25 Apr 2014 17:52:31 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "Steven Hartland" Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425175231.9714afc3.gerrit.kuehn@aei.mpg.de> In-Reply-To: <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 17:52:31, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 17:52:40, Serialize complete at 04/25/2014 17:52:40 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.153920 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1800_1899 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __STOCK_PHRASE_7 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 15:52:43 -0000 On Fri, 25 Apr 2014 15:55:29 +0100 "Steven Hartland" wrote about Re: NFS over LAGG / lacp poor performance: SH> Run top then enable system processes (shift+S) and threads (shift+H) SH> then disable idle process display (z) if your seeing significant CPU SH> usage to the igb interrupt handlers try reducing the number. Hm, I'm a bit unsure what I see here: --- last pid: 71026; load averages: 0.36, 0.37, 0.27 up 18+03:14:21 17:48:02 295 processes: 5 running, 259 sleeping, 31 waiting CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 3596K Active, 59M Inact, 11G Wired, 900K Cache, 1236M Buf, 1019M Free Swap: 16G Total, 14M Used, 16G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 0 root -16 0 0K 2672K - 2 272:01 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 0 271:55 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 3 271:54 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 1 271:52 0.00% kernel{zio_write_issue_} 1455 root 20 0 9912K 472K rpcsvc 2 173:53 0.00% nfsd{nfsd: master} 1455 root 20 0 9912K 472K rpcsvc 0 166:37 0.00% nfsd{nfsd: service} 1455 root 20 0 9912K 472K rpcsvc 0 164:25 0.00% nfsd{nfsd: service} 1455 root 20 0 9912K 472K rpcsvc 3 163:06 0.00% nfsd{nfsd: service} 12 root -92 - 0K 496K WAIT 1 155:36 0.00% intr{irq262: igb1:que} [...] --- The irq-igb1 clibs to top every now and then (always less than 1% cpu usage though), but there is never much cpu usage shown for any task. However, the overall load is significant. What might the system be doing? cu Gerrit From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 16:01:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DCCAE93 for ; Fri, 25 Apr 2014 16:01:01 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id BEDD315D0 for ; Fri, 25 Apr 2014 16:01:00 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 5A05020E7088B; Fri, 25 Apr 2014 16:00:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 67A0620E70885; Fri, 25 Apr 2014 16:00:55 +0000 (UTC) Message-ID: From: "Steven Hartland" To: =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> <20140425175231.9714afc3.gerrit.kuehn@aei.mpg.de> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 17:00:59 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 16:01:01 -0000 ----- Original Message ----- From: "Gerrit Kühn" To: "Steven Hartland" Cc: Sent: Friday, April 25, 2014 4:52 PM Subject: Re: NFS over LAGG / lacp poor performance > On Fri, 25 Apr 2014 15:55:29 +0100 "Steven Hartland" > wrote about Re: NFS over LAGG / lacp poor > performance: > > SH> Run top then enable system processes (shift+S) and threads (shift+H) > SH> then disable idle process display (z) if your seeing significant CPU > SH> usage to the igb interrupt handlers try reducing the number. > > Hm, I'm a bit unsure what I see here: > > --- > last pid: 71026; load averages: 0.36, 0.37, 0.27 > up 18+03:14:21 17:48:02 > 295 processes: 5 running, 259 sleeping, 31 waiting > CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > Mem: 3596K Active, 59M Inact, 11G Wired, 900K Cache, 1236M Buf, 1019M Free > Swap: 16G Total, 14M Used, 16G Free > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 0 root -16 0 0K 2672K - 2 272:01 0.00% kernel{zio_write_issue_} > 0 root -16 0 0K 2672K - 0 271:55 0.00% kernel{zio_write_issue_} > 0 root -16 0 0K 2672K - 3 271:54 0.00% kernel{zio_write_issue_} > 0 root -16 0 0K 2672K - 1 271:52 0.00% kernel{zio_write_issue_} > 1455 root 20 0 9912K 472K rpcsvc 2 173:53 0.00% nfsd{nfsd: master} > 1455 root 20 0 9912K 472K rpcsvc 0 166:37 0.00% nfsd{nfsd: service} > 1455 root 20 0 9912K 472K rpcsvc 0 164:25 0.00% nfsd{nfsd: service} > 1455 root 20 0 9912K 472K rpcsvc 3 163:06 0.00% nfsd{nfsd: service} > 12 root -92 - 0K 496K WAIT 1 155:36 0.00% intr{irq262: igb1:que} > [...] > --- > > > The irq-igb1 clibs to top every now and then (always less than 1% cpu > usage though), but there is never much cpu usage shown for any task. > However, the overall load is significant. What might the system be doing? Thats not your issue then. Regards Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 22:42:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5A4312F for ; Fri, 25 Apr 2014 22:42:59 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72CE6123C for ; Fri, 25 Apr 2014 22:42:59 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id x13so4148852wgg.33 for ; Fri, 25 Apr 2014 15:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=IkdiDouvXzqw7FXT1pHQqVFrsbNkH/LbnuN4koU0gT8=; b=LgngB9JmhW8vWvOKXcCzgAoKn1zxEaFhT8ZNvxG2EegOrSU19tlgCdsms57co02WRd 6FeOjIGYPABrIc45z6oNRe9/DHrMNf4LAUPgzgtt2OrP1IT5SqWURhJY8z/p512coai7 RB+R/2jKXKUG19MAbSqyQKcCG4t1tpfDj1ErF9rYh/hHllBM8/DwAjRMXBXLOGqJFacU QjkCa2ulC6+3BGdtHfjnhFBssI2r5xWwsHxATr+EDZWdLoHXGZdn3SrN5tjfmiVTpi44 HVcK6JOGDpc0vLBEYsohT8TOapUztV+0Wf9J8J1nWvOZ2SrytDtoL5z9CMuV+BqOJRSE lPmQ== MIME-Version: 1.0 X-Received: by 10.180.218.35 with SMTP id pd3mr5555544wic.26.1398465776868; Fri, 25 Apr 2014 15:42:56 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Fri, 25 Apr 2014 15:42:56 -0700 (PDT) Date: Fri, 25 Apr 2014 16:42:56 -0600 X-Google-Sender-Auth: Cm4eDnkRg_IVqmgECZW7iRtiK7g Message-ID: Subject: Please review: fix page fault panic in lacp_req From: Alan Somers To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 22:42:59 -0000 If you do an "ifconfig -am" in one thread while doing an "ifconfig lagg0 destroy" in another thread, at least two panics may result. One is in lacp_req(), caused by NULL == lsc. I opened kern/189003 to describe it. I'm still working on the other panic or panics. Full explanation and patch are at http://www.freebsd.org/cgi/query-pr.cgi?pr=189003&cat= . Can somebody please review the patch? -Alan From owner-freebsd-net@FreeBSD.ORG Sat Apr 26 04:31:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C465E5E for ; Sat, 26 Apr 2014 04:31:57 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DFE4D113C for ; Sat, 26 Apr 2014 04:31:56 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id ey11so2009563pad.31 for ; Fri, 25 Apr 2014 21:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DZsC8A0c3wLVBAIQqV3KkbeeK/JdwC2lan/VyGx3oBw=; b=f/pgbr0ZtB/VfFveUUtIkXilif7qHCdU+pHeyC6+l+TFfSMNd1TzCxQ/Qa+RcgIL4t dhObymZjipDcVCj6TdZfTFpnR0iYLjWFSoznR1IuwYswPwnjiYbw6iJJOVbAs0VjXu+7 IZofv96eulPfNjCgWIvIwhS0Fh/tCJhhzydpWzixkcmxDtwze78PN5vdXUc0hYB/qWlO UHKvlnux7aN8MYznsD5V0N1pYKls45Bh2MwU3MuYpnwODkAi0gIjGyhRDJAjaM2nL5ED raRvfOh8S2UzklGyjIHOk3sj+wShmL4VD5kUGsr6WqBCUbaz4dAd2nQmOzD2gxJBk0VC Cy3A== MIME-Version: 1.0 X-Received: by 10.68.163.197 with SMTP id yk5mr12093980pbb.57.1398486714617; Fri, 25 Apr 2014 21:31:54 -0700 (PDT) Received: by 10.70.55.169 with HTTP; Fri, 25 Apr 2014 21:31:54 -0700 (PDT) In-Reply-To: <20140425175231.9714afc3.gerrit.kuehn@aei.mpg.de> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> <20140425175231.9714afc3.gerrit.kuehn@aei.mpg.de> Date: Fri, 25 Apr 2014 23:31:54 -0500 Message-ID: Subject: Re: NFS over LAGG / lacp poor performance From: Adam Vande More To: =?UTF-8?B?R2Vycml0IEvDvGhu?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net , Steven Hartland X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2014 04:31:57 -0000 On Fri, Apr 25, 2014 at 10:52 AM, Gerrit K=C3=BChn wrote: > > > The irq-igb1 clibs to top every now and then (always less than 1% cpu > usage though), but there is never much cpu usage shown for any task. > However, the overall load is significant. What might the system be doing? > > http://lists.freebsd.org/pipermail/freebsd-current/2011-September/027138.ht= ml --=20 Adam From owner-freebsd-net@FreeBSD.ORG Sun Apr 27 15:00:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F5FB2DB for ; Sun, 27 Apr 2014 15:00:16 +0000 (UTC) Received: from mail-yk0-x232.google.com (mail-yk0-x232.google.com [IPv6:2607:f8b0:4002:c07::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36AB91BED for ; Sun, 27 Apr 2014 15:00:16 +0000 (UTC) Received: by mail-yk0-f178.google.com with SMTP id 200so3523045ykr.9 for ; Sun, 27 Apr 2014 08:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=REsQwZuiSfOV6/iKqwlOOuh//J7zimK7z1BI+1J/C6Y=; b=WGkTCAu/vbS154bBZmHLAqLqgj9B6HDB+2bnCQTaMc20tUFY3iI/JLhON4YAv3o4gP DEWyzpRqTPvD7ZwqWfhj7Q4auS7eRCiDlDaZIhrAcs6wvnrqplsIMu/hZDpZPS1f7is9 HugDbn/5AGiT0B4D806z6ECR/DpxfJKEK+f8pjlnKQfieq1xAXVvyTSSwxYydcwo4TWO VdqBW7mINKP+4ECkarmEXN8u4vwg2n2YT8b2deh0x4owiqUr9jn+bhcJKGPFtu8rU53r uAJ282xIdLY1iJxJVCOuYYj2vpWgpdRVcFnX9D7HiXcz8TZ+l3P+xYOU701fb9e19Jay pfZw== MIME-Version: 1.0 X-Received: by 10.236.31.40 with SMTP id l28mr30144824yha.17.1398610815415; Sun, 27 Apr 2014 08:00:15 -0700 (PDT) Received: by 10.170.185.208 with HTTP; Sun, 27 Apr 2014 08:00:15 -0700 (PDT) Date: Sun, 27 Apr 2014 17:00:15 +0200 Message-ID: Subject: Random disconnects on 10.0 with mpd5/pptp + PF From: martin i To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2014 15:00:16 -0000 I turned on verbose logging in mpd.conf and did another set of tests. Client just logged in and stayed idle. Traffic of the active PPTP session looked like: Test #1: Apr 27 02:09:35 fbsd10 mpd: pptp0: recv EchoRequest Apr 27 02:09:35 fbsd10 mpd: id=0x6000000 Apr 27 02:09:35 fbsd10 mpd: pptp0: send EchoReply msg Apr 27 02:09:35 fbsd10 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 02:09:35 fbsd10 mpd: id=0x6000000 result=1 err=0 ignore=0 Apr 27 02:10:35 fbsd10 mpd: pptp0: send EchoRequest msg Apr 27 02:10:35 fbsd10 mpd: len=16 msgType=1 magic=0x1a2b3c4d type=5 Apr 27 02:10:35 fbsd10 mpd: id=1 Apr 27 02:10:35 fbsd10 mpd: pptp0: recv EchoRequest Apr 27 02:10:35 fbsd10 mpd: id=0x7000000 Apr 27 02:10:35 fbsd10 mpd: pptp0: send EchoReply msg Apr 27 02:10:35 fbsd10 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 02:10:35 fbsd10 mpd: id=0x7000000 result=1 err=0 ignore=0 Apr 27 02:10:35 fbsd10 mpd: pptp0: recv EchoReply Apr 27 02:10:35 fbsd10 mpd: id=1 result=1 err=0 ignore=0 Apr 27 02:11:35 fbsd10 mpd: pptp0: send EchoRequest msg Apr 27 02:11:35 fbsd10 mpd: len=16 msgType=1 magic=0x1a2b3c4d type=5 Apr 27 02:11:35 fbsd10 mpd: id=2 Apr 27 02:11:35 fbsd10 mpd: pptp0: read: Connection reset by peer Apr 27 02:11:35 fbsd10 mpd: pptp0: ctrl state ESTABLISHED --> DYING Apr 27 02:11:35 fbsd10 mpd: pptp0: killing connection with 49595 Apr 27 02:11:35 fbsd10 mpd: pptp0-0: killing channel Apr 27 02:11:35 fbsd10 mpd: [cloud_link-1] PPTP call terminated Apr 27 02:11:35 fbsd10 mpd: [cloud_link-1] device: DOWN event Test #2: Apr 27 02:28:16 fbsd10 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 02:28:16 fbsd10 mpd: id=0x5000000 result=1 err=0 ignore=0 Apr 27 02:29:16 fbsd10 mpd: pptp0: recv EchoRequest Apr 27 02:29:16 fbsd10 mpd: id=0x6000000 Apr 27 02:29:16 fbsd10 mpd: pptp0: send EchoReply msg Apr 27 02:29:16 fbsd10 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 02:29:16 fbsd10 mpd: id=0x6000000 result=1 err=0 ignore=0 Apr 27 02:30:16 fbsd10 mpd: pptp0: send EchoRequest msg Apr 27 02:30:16 fbsd10 mpd: len=16 msgType=1 magic=0x1a2b3c4d type=5 Apr 27 02:30:16 fbsd10 mpd: id=1 Apr 27 02:30:16 fbsd10 mpd: pptp0: read: Connection reset by peer Apr 27 02:30:16 fbsd10 mpd: pptp0: ctrl state ESTABLISHED --> DYING Apr 27 02:30:16 fbsd10 mpd: pptp0: killing connection with 49732 In test #2 connection went down even sooner. It seems that the "type 5" message has something to do with it. I checked the RFC 1661 for PPP specification but I found no information on what that type might be. I looked at mpd5 sources (v5.7); it seems it's defined in pptp_ctrl.h as PPTP_EchoRequest (type 6 is PPTP_EchoReply). I rebooted back to 9.2 to check the behavior there. I saw only type 6 messages: Apr 27 16:40:33 fbsd9 mpd: pptp0: recv EchoRequest Apr 27 16:40:33 fbsd9 mpd: id=0x23000000 Apr 27 16:40:33 fbsd9 mpd: pptp0: send EchoReply msg Apr 27 16:40:33 fbsd9 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 16:40:33 fbsd9 mpd: id=0x23000000 result=1 err=0 ignore=0 Apr 27 16:41:33 fbsd9 mpd: pptp0: recv EchoRequest Apr 27 16:41:33 fbsd9 mpd: id=0x24000000 Apr 27 16:41:33 fbsd9 mpd: pptp0: send EchoReply msg Apr 27 16:41:33 fbsd9 mpd: len=20 msgType=1 magic=0x1a2b3c4d type=6 Apr 27 16:41:33 fbsd9 mpd: id=0x24000000 result=1 err=0 ignore=0 with no type 5 messages at all. Session stays up without problem. But logic to keep the link up is in the mpd5 which is the same version on 9.2 and 10.0. What can be the reason for sending those type 5 messages ? (or is my assumption even correct that it has something to do with my issue?) Martin From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 04:58:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8F60AE7 for ; Mon, 28 Apr 2014 04:58:14 +0000 (UTC) Received: from mail-qg0-x236.google.com (mail-qg0-x236.google.com [IPv6:2607:f8b0:400d:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E0A517AC for ; Mon, 28 Apr 2014 04:58:14 +0000 (UTC) Received: by mail-qg0-f54.google.com with SMTP id q107so5782190qgd.27 for ; Sun, 27 Apr 2014 21:58:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=X5krdE7+ktDgx8fFDOo46OrKRDbkKfsKLwMvXNO1JGo=; b=esn0eM2F7gRoRdDxy7tWrfMl26HXejdW3IocOy6s9zbUz144JRVWtwlX7sqNV/iki3 26tZvvdDQfeNzw9VZ2or8jfXjcKj7Y6HE0df0lk4BqfdFoFVVSJ2E+8WdNUoSDYNmkBj Ztf7VQi6gAEZ7xJotuak5AcWsSI76d5O987ZqvFZPtjQCcBC12nA6YAiNJNRg9gK3de4 yO1tt1HH0P/WGadQJ30J3fDG1AbD5cZmpmo41zRuk4Rbv5e+iiTvMZ7lp0ak7X+t8KLv hfVJzwU8uEPxjijSr0Uwb6uXDtJxCZ9rmayo6SWkBeDVvSWL2NoCOj6/rsI4kTAuOgy5 Ki/A== MIME-Version: 1.0 X-Received: by 10.140.19.212 with SMTP id 78mr28891822qgh.84.1398661093628; Sun, 27 Apr 2014 21:58:13 -0700 (PDT) Received: by 10.229.232.6 with HTTP; Sun, 27 Apr 2014 21:58:13 -0700 (PDT) Date: Mon, 28 Apr 2014 09:28:13 +0430 Message-ID: Subject: running netmap-ipfw with real NICs From: Mahnaz Talebi To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 04:58:14 -0000 Hi all, I am trying to run netmap-based ipfw with real NICs, but encounter error in opening netmap device. (I can run it with vale switch), what is problem??! From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 08:59:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A3B15D3 for ; Mon, 28 Apr 2014 08:59:04 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id E3AEB1CFF for ; Mon, 28 Apr 2014 08:59:03 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id 1192C6B8CBF for ; Mon, 28 Apr 2014 10:58:54 +0200 (CEST) Received: from soth.ventu (adsl-ull-90-150.41-151.net24.it [151.41.150.90]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3S8wmtX009564 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 28 Apr 2014 10:58:50 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-90-150.41-151.net24.it [151.41.150.90] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3S8wgMj066689 for ; Mon, 28 Apr 2014 10:58:42 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <535E1842.20905@netfence.it> Date: Mon, 28 Apr 2014 10:58:42 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Server with multiple public IP Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Mon, 28 Apr 2014 10:58:50 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 08:59:04 -0000 Hello. This has probably come up several times, however... I've got a server which has two (or more) interfaces with public IPs. Let's say, as an example (with fictional IPs): ifconfig_vlan1="inet 1.0.0.2 netmask 255.255.255.248..." ifconfig_vlan2="inet 2.0.0.2 netmask 255.255.255.248..." Of course, I can only have a default route, let's say 1.0.0.1. This is fine for outgoing traffic and for incoming connections on vlan1. However, when someone from the outside connects to 2.0.0.2, reply packets still go out through 1.0.0.1 (on vlan1), but they should go through vlan2 to 2.0.0.1 The only way I found so far to achieve this, is through ipfw: ipfw add 30 fwd 2.0.0.1 tcp from 2.0.0.2 to not 2.0.0.0/29 out This more or less works, but it will break ipfw firewalling (since after that rule matches, "the search terminates"). Besides, I don't feel this is a very clean solution. So I wonder: do other ways exist to achieve this? Any best practice? I thought natd might help, but found no reference to this functionality in its docs... Does any other program exists which I can "divert" packets to, which would modify and reinject them as natd does? Another thing I though of would be combining two firewalls (ipfw + pf/ipf), letting one do the filtering and leaving the above problem to the other. I'm not sure how hard this would be, however, so if a simple solution exists... Any hint appreciated. bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 09:16:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C86AFC00 for ; Mon, 28 Apr 2014 09:16:31 +0000 (UTC) Received: from mail.shmtech.biz (unknown [IPv6:2001:41c8:10:8c::4:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.shmtech.biz", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63F7A1047 for ; Mon, 28 Apr 2014 09:16:30 +0000 (UTC) Received: from fleabag.domlan.talk2dom.com ([46.233.116.122]) (authenticated bits=0) by mail.shmtech.biz (8.14.8/8.14.5) with ESMTP id s3S9GRsK031002 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 28 Apr 2014 10:16:28 +0100 (BST) (envelope-from dom@talk2dom.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=shmtech1; t=1398676588; bh=oMbibEkR2SKikU//AX5hwYHqM9W+9sce57kpNAU20pU=; h=Date:From:To:Subject:References:In-Reply-To; b=RIyuHYpOJjBHfzlWtD5J5lIhuOhbSXlobwSXTbaq6uHWytFu7niaLwf43Teu9IKBa nAF4HdQJ/vpW71lUG732OVzi0B4HXoA5sZtl9aO9FhoonASh6+UjulbVQKLtKYhOBJ HH44THtDs9z7ctnc6LQwk/a4hpFmjm5pgdr373Tk= X-Authentication-Warning: sendmail: Host [46.233.116.122] claimed to be fleabag.domlan.talk2dom.com Message-ID: <535E1C66.6090004@talk2dom.com> Date: Mon, 28 Apr 2014 10:16:22 +0100 From: Dominic Froud User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> In-Reply-To: <535E1842.20905@netfence.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:16:31 -0000 On 28/04/2014 09:58, Andrea Venturoli wrote: > I've got a server which has two (or more) interfaces with public IPs. > > Let's say, as an example (with fictional IPs): > ifconfig_vlan1="inet 1.0.0.2 netmask 255.255.255.248..." > ifconfig_vlan2="inet 2.0.0.2 netmask 255.255.255.248..." > > Of course, I can only have a default route, let's say 1.0.0.1. > This is fine for outgoing traffic and for incoming connections on vlan1. > However, when someone from the outside connects to 2.0.0.2, reply > packets still go out through 1.0.0.1 (on vlan1), but they should go > through vlan2 to 2.0.0.1 You want source-based routing. I have this situation and I used pf(4) to do it with a rule like: pass out quick route-to ( vlan2 ) from 2.0.0.0/29 to any no state As a variation you can give an optional next-hop address if you have a static router for that vlan, e.g. if your router is 2.0.0.1: pass out quick route-to ( vlan2 2.0.0.1 ) from 2.0.0.0/29 to any no state Also, you can run pf and ipfw at the same time! Hope this helps, Dominic From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 09:18:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83D78D41 for ; Mon, 28 Apr 2014 09:18:51 +0000 (UTC) Received: from mail-oa0-x230.google.com (mail-oa0-x230.google.com [IPv6:2607:f8b0:4003:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DFA31067 for ; Mon, 28 Apr 2014 09:18:51 +0000 (UTC) Received: by mail-oa0-f48.google.com with SMTP id m1so6803969oag.7 for ; Mon, 28 Apr 2014 02:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P/xrmX35r1lessBS12iTP0+uOFDeadzvgnAb9yc7kWQ=; b=zxFJIXxu18pYf+63LWXBYQZtYAWGIdSnv/GQej8h054F2PHPpxC+cbnluE+QIOQnx4 6sGvPBU8NdVEoX3T19hD3x7TDpGBegBlsCehsyE2mdzVH99vUzdV7vjIg7sD4QDFwbYY 9pZLHBnvHb20qwdQfov6CaTTB8asNZCR+hbmXQXX6cxKGU6buCvepylURhDTTRaqPWZh lxtFyqefHys64M7ZWdW9Q1SruXoSsz1Q2kO0GRuzfxIy8Zzhwz0cGUCRRjRZx0xTqIN3 li/IXsG3vGrWQHE8B6NemER/U9lCK147v5EqWO6zYreYGA7K2cEX6KjX4kQLcqVHqyDP S4zw== MIME-Version: 1.0 X-Received: by 10.60.144.200 with SMTP id so8mr21134667oeb.31.1398676730425; Mon, 28 Apr 2014 02:18:50 -0700 (PDT) Received: by 10.76.173.229 with HTTP; Mon, 28 Apr 2014 02:18:50 -0700 (PDT) In-Reply-To: <535E1C66.6090004@talk2dom.com> References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> Date: Mon, 28 Apr 2014 11:18:50 +0200 Message-ID: Subject: Re: Server with multiple public IP From: Andreas Nilsson To: Dominic Froud Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:18:51 -0000 On Mon, Apr 28, 2014 at 11:16 AM, Dominic Froud wrote: > On 28/04/2014 09:58, Andrea Venturoli wrote: > >> I've got a server which has two (or more) interfaces with public IPs. >> >> Let's say, as an example (with fictional IPs): >> ifconfig_vlan1="inet 1.0.0.2 netmask 255.255.255.248..." >> ifconfig_vlan2="inet 2.0.0.2 netmask 255.255.255.248..." >> >> Of course, I can only have a default route, let's say 1.0.0.1. >> This is fine for outgoing traffic and for incoming connections on vlan1. >> However, when someone from the outside connects to 2.0.0.2, reply packets >> still go out through 1.0.0.1 (on vlan1), but they should go through vlan2 >> to 2.0.0.1 >> > > You want source-based routing. > > I have this situation and I used pf(4) to do it with a rule like: > > pass out quick route-to ( vlan2 ) from 2.0.0.0/29 to any no state > > As a variation you can give an optional next-hop address if you have a > static router for that vlan, e.g. if your router is 2.0.0.1: > > pass out quick route-to ( vlan2 2.0.0.1 ) from 2.0.0.0/29 to any no state > > Also, you can run pf and ipfw at the same time! > > Hope this helps, > > Dominic > > You could put all the services which are on 2.0.0.2 in a separate fib and there have another default-route. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 09:45:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72824343 for ; Mon, 28 Apr 2014 09:45:25 +0000 (UTC) Received: from mp1-smtp-5.eutelia.it (mp1-smtp-5.eutelia.it [62.94.10.165]) by mx1.freebsd.org (Postfix) with ESMTP id 245BA12F2 for ; Mon, 28 Apr 2014 09:45:24 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-5.eutelia.it (Eutelia) with ESMTP id E59F91731E3 for ; Mon, 28 Apr 2014 11:45:22 +0200 (CEST) Received: from soth.ventu (adsl-ull-90-150.41-151.net24.it [151.41.150.90]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3S9j31Q014316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 28 Apr 2014 11:45:04 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-90-150.41-151.net24.it [151.41.150.90] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3S9iw8A068572 for ; Mon, 28 Apr 2014 11:44:58 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <535E231A.1050800@netfence.it> Date: Mon, 28 Apr 2014 11:44:58 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Mon, 28 Apr 2014 11:45:04 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:45:25 -0000 On 04/28/14 11:18, Andreas Nilsson wrote: > You could put all the services which are on 2.0.0.2 in a separate fib and > there have another default-route. Thanks, but unfortunately I can't, since some services must be able to answer on both addresses. Maybe I could use socket in one fib to proxy to the other, but that would probably make a mess in the logs when I have to identify who connects to what and from where. bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 10:01:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B37A65B1 for ; Mon, 28 Apr 2014 10:01:18 +0000 (UTC) Received: from mp1-smtp-5.eutelia.it (mp1-smtp-5.eutelia.it [62.94.10.165]) by mx1.freebsd.org (Postfix) with ESMTP id 65E1314BA for ; Mon, 28 Apr 2014 10:01:18 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-5.eutelia.it (Eutelia) with ESMTP id 2C20517347D for ; Mon, 28 Apr 2014 11:43:09 +0200 (CEST) Received: from soth.ventu (adsl-ull-90-150.41-151.net24.it [151.41.150.90]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s3S9h4lD014101 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 28 Apr 2014 11:43:05 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-90-150.41-151.net24.it [151.41.150.90] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.8/8.14.7) with ESMTP id s3S9gxNs068510; Mon, 28 Apr 2014 11:42:59 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <535E22A3.2090404@netfence.it> Date: Mon, 28 Apr 2014 11:42:59 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Dominic Froud Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> In-Reply-To: <535E1C66.6090004@talk2dom.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Mon, 28 Apr 2014 11:43:05 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.74 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 10:01:18 -0000 On 04/28/14 11:16, Dominic Froud wrote: > You want source-based routing. Thanks, that term will help me in my searches. > I have this situation and I used pf(4) to do it with a rule like: > > pass out quick route-to ( vlan2 ) from 2.0.0.0/29 to any no state > > As a variation you can give an optional next-hop address if you have a > static router for that vlan, e.g. if your router is 2.0.0.1: > > pass out quick route-to ( vlan2 2.0.0.1 ) from 2.0.0.0/29 to any no state > > Also, you can run pf and ipfw at the same time! Thanks a lot, I think I'll try this. bye av. From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 10:11:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB442753 for ; Mon, 28 Apr 2014 10:11:18 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BD61815BF for ; Mon, 28 Apr 2014 10:11:18 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3SABD9c068141 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 28 Apr 2014 03:11:16 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <535E293C.5050705@freebsd.org> Date: Mon, 28 Apr 2014 18:11:08 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Andrea Venturoli , freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> <535E231A.1050800@netfence.it> In-Reply-To: <535E231A.1050800@netfence.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 10:11:19 -0000 On 4/28/14, 5:44 PM, Andrea Venturoli wrote: > On 04/28/14 11:18, Andreas Nilsson wrote: > >> You could put all the services which are on 2.0.0.2 in a separate >> fib and >> there have another default-route. > > Thanks, but unfortunately I can't, since some services must be able > to answer on both addresses. the answer is to use the ipfw setfib rule for incoming packets on the second interface. setfib 1 ip from any to any in recv em0 In new freebsd kernels you can do this with ifconfig em0 fib 1 (I think that's the syntax) without involving ipfw. then the session will inherit that fib. Outgoing packets from that session will use fib 1 while other outgoing packets will use fib0. > > Maybe I could use socket in one fib to proxy to the other, but that > would probably make a mess in the logs when I have to identify who > connects to what and from where. > > bye & Thanks > av. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 10:15:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 425B38E7 for ; Mon, 28 Apr 2014 10:15:22 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EE6B615E1 for ; Mon, 28 Apr 2014 10:15:21 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3SAFGgD068178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 28 Apr 2014 03:15:20 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <535E2A2F.3030505@freebsd.org> Date: Mon, 28 Apr 2014 18:15:11 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Andrea Venturoli , freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> <535E231A.1050800@netfence.it> <535E293C.5050705@freebsd.org> In-Reply-To: <535E293C.5050705@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 10:15:22 -0000 replying to myself.. On 4/28/14, 6:11 PM, Julian Elischer wrote: > On 4/28/14, 5:44 PM, Andrea Venturoli wrote: >> On 04/28/14 11:18, Andreas Nilsson wrote: >> >>> You could put all the services which are on 2.0.0.2 in a separate >>> fib and >>> there have another default-route. >> >> Thanks, but unfortunately I can't, since some services must be able >> to answer on both addresses. > > the answer is to use the ipfw setfib rule for incoming packets on > the second interface. > setfib 1 ip from any to any in recv em0 > In new freebsd kernels you can do this with ifconfig em0 fib 1 (I > think that's the syntax) without involving ipfw. > > then the session will inherit that fib. Outgoing packets from that > session will use fib 1 while other outgoing packets will use fib0. from the ifconfig man page. (FreeBSD 11 but I think it's in 10 too.) fib fib_number Specify interface FIB. A FIB fib_number is assigned to all frames or packets received on that interface. The FIB is not inherited, e.g., vlans or other sub-interfaces will use the default FIB (0) irrespective of the parent interface's FIB. The kernel needs to be tuned to support more than the default FIB using the ROUTETABLES kernel configuration option, or the net.fibs tunable. this can be simulated using ipfw setfib should you not have it in the release you are running. > >> >> Maybe I could use socket in one fib to proxy to the other, but that >> would probably make a mess in the logs when I have to identify who >> connects to what and from where. >> >> bye & Thanks >> av. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 11:06:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E833A2A1 for ; Mon, 28 Apr 2014 11:06:31 +0000 (UTC) Received: from mail.shmtech.biz (unknown [IPv6:2001:41c8:10:8c::4:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.shmtech.biz", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 842891A75 for ; Mon, 28 Apr 2014 11:06:31 +0000 (UTC) Received: from fleabag.domlan.talk2dom.com ([46.233.116.122]) (authenticated bits=0) by mail.shmtech.biz (8.14.8/8.14.5) with ESMTP id s3SB6QcQ047662 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 28 Apr 2014 12:06:28 +0100 (BST) (envelope-from dom@talk2dom.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=shmtech1; t=1398683188; bh=o/35bchoNKl3vIx1jrN4Eo52E8lfvr8OZeWRFoLe8BI=; h=Date:From:To:Subject:References:In-Reply-To; b=uO1/UBkLshupDJLaGqqlFClfMohfTBFxRxbNldZ0xfQPto2Xolwt6Qx+3wUXVBUDO naTb9QYj+9lPtFDzkdzG14BvgO2hFeGhh/d6FQsZXva2f71s0I2W1vEoqu6YFZyjYs mK97EUzW4VjpqVI4CGLv5M6RIPGKOfUhBPGrMTYM= X-Authentication-Warning: sendmail: Host [46.233.116.122] claimed to be fleabag.domlan.talk2dom.com Message-ID: <535E362D.1050408@talk2dom.com> Date: Mon, 28 Apr 2014 12:06:21 +0100 From: Dominic Froud User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> <535E231A.1050800@netfence.it> <535E293C.5050705@freebsd.org> <535E2A2F.3030505@freebsd.org> In-Reply-To: <535E2A2F.3030505@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 11:06:32 -0000 On 28/04/2014 11:15, Julian Elischer wrote: > replying to myself.. > > On 4/28/14, 6:11 PM, Julian Elischer wrote: >> On 4/28/14, 5:44 PM, Andrea Venturoli wrote: >>> On 04/28/14 11:18, Andreas Nilsson wrote: >>> >>>> You could put all the services which are on 2.0.0.2 in a separate >>>> fib and >>>> there have another default-route. >>> >>> Thanks, but unfortunately I can't, since some services must be able >>> to answer on both addresses. >> >> the answer is to use the ipfw setfib rule for incoming packets on the >> second interface. >> setfib 1 ip from any to any in recv em0 >> In new freebsd kernels you can do this with ifconfig em0 fib 1 (I >> think that's the syntax) without involving ipfw. >> >> then the session will inherit that fib. Outgoing packets from that >> session will use fib 1 while other outgoing packets will use fib0. > from the ifconfig man page. (FreeBSD 11 but I think it's in 10 too.) > > fib fib_number > Specify interface FIB. A FIB fib_number is assigned to all > frames or packets received on that interface. The FIB is > not > inherited, e.g., vlans or other sub-interfaces will use the > default FIB (0) irrespective of the parent interface's > FIB. The > kernel needs to be tuned to support more than the default > FIB > using the ROUTETABLES kernel configuration option, or the > net.fibs tunable. > > this can be simulated using ipfw setfib should you not have it in the > release you are running. > "Outgoing packets from that session will use fib 1 while other outgoing packets will use fib0." I haven't tried this but outgoing packets not associated with any existing fib1 session (e.g. new TCP connections, UDP, etc.) could also be attached to fib1 with a rule like this? setfib 1 ip from 2.0.0.0/29 to any out xmit vlan2 Keeping all the rules in ipfw is one advantage but then you have to maintain 2 sets of routing tables - one for each fib. Doing source-routing with pf means two firewalls to manage but just one routing table. You could argue that the routing table is obscured by rules in pf though so doing "netstat -rnf inet" wouldn't be authorititative. I'd like to do something like this: route add -srcnet 2.0.0.0/29 2.0.0.1 (kernel uses arp to translate 2.0.0.1 to an interface address like vlan2) Dom From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 11:06:51 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13748492 for ; Mon, 28 Apr 2014 11:06:51 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3C6E1AAB for ; Mon, 28 Apr 2014 11:06:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3SB6oJn086206 for ; Mon, 28 Apr 2014 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3SB6ope086204 for freebsd-net@FreeBSD.org; Mon, 28 Apr 2014 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 28 Apr 2014 11:06:50 GMT Message-Id: <201404281106.s3SB6ope086204@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 11:06:51 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/188899 net [cas] cas ethernet driver seems to have issues with so o kern/188897 net [dc] dc ethernet driver seems to prevent the detection o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 476 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 14:01:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A794E96A for ; Mon, 28 Apr 2014 14:01:10 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 83D1E10E2 for ; Mon, 28 Apr 2014 14:01:10 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3SE14F3069040 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 28 Apr 2014 07:01:07 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <535E5F1B.7080302@freebsd.org> Date: Mon, 28 Apr 2014 22:00:59 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Dominic Froud , freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> <535E231A.1050800@netfence.it> <535E293C.5050705@freebsd.org> <535E2A2F.3030505@freebsd.org> <535E362D.1050408@talk2dom.com> In-Reply-To: <535E362D.1050408@talk2dom.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 14:01:10 -0000 On 4/28/14, 7:06 PM, Dominic Froud wrote: > > On 28/04/2014 11:15, Julian Elischer wrote: >> replying to myself.. >> >> On 4/28/14, 6:11 PM, Julian Elischer wrote: >>> On 4/28/14, 5:44 PM, Andrea Venturoli wrote: >>>> On 04/28/14 11:18, Andreas Nilsson wrote: >>>> >>>>> You could put all the services which are on 2.0.0.2 in a >>>>> separate fib and >>>>> there have another default-route. >>>> >>>> Thanks, but unfortunately I can't, since some services must be >>>> able to answer on both addresses. >>> >>> the answer is to use the ipfw setfib rule for incoming packets on >>> the second interface. >>> setfib 1 ip from any to any in recv em0 >>> In new freebsd kernels you can do this with ifconfig em0 fib 1 (I >>> think that's the syntax) without involving ipfw. >>> >>> then the session will inherit that fib. Outgoing packets from that >>> session will use fib 1 while other outgoing packets will use fib0. >> from the ifconfig man page. (FreeBSD 11 but I think it's in 10 too.) >> >> fib fib_number >> Specify interface FIB. A FIB fib_number is assigned >> to all >> frames or packets received on that interface. The FIB >> is not >> inherited, e.g., vlans or other sub-interfaces will >> use the >> default FIB (0) irrespective of the parent interface's >> FIB. The >> kernel needs to be tuned to support more than the >> default FIB >> using the ROUTETABLES kernel configuration option, or the >> net.fibs tunable. >> >> this can be simulated using ipfw setfib should you not have it in >> the release you are running. >> > > "Outgoing packets from that session will use fib 1 while other > outgoing packets will use fib0." > > I haven't tried this but outgoing packets not associated with any > existing fib1 session (e.g. new TCP connections, UDP, etc.) could > also be attached to fib1 with a rule like this? > > setfib 1 ip from 2.0.0.0/29 to any out xmit vlan2 > > Keeping all the rules in ipfw is one advantage but then you have to > maintain 2 sets of routing tables - one for each fib. > > Doing source-routing with pf means two firewalls to manage but just > one routing table. You could argue that the routing table is > obscured by rules in pf though so doing "netstat -rnf inet" wouldn't > be authorititative. > > I'd like to do something like this: > > route add -srcnet 2.0.0.0/29 2.0.0.1 > > (kernel uses arp to translate 2.0.0.1 to an interface address like > vlan2) > ok so you have to make a decision on each session: "which link should I use for outgoing packets?" (You can't control incoming packets for sessions initiated remotely). answers are: (1) the one that the request came in on, so the session is symetrical. (2) one selected for some other reason. In this case the session may be asymetrical, and you need to pass out packets that appear to come from the other interface... Your ISP may hate you for this and block them as they are effectively "spoofed packets" from their perspective and it may break you service agreement. For outgoing sessions option 2 works fine and you can just use a single routing table to to this, or select which FIB to use with some "undisclosed" decision, but for incoming sessions you really would do best to make your outgoing packets go out the way the request came in, and this is what option 1 does.. by setting up two fibs, each with a separate default route, and just selecting the fib to use depending on the incoming interface. Some people (me) also sometimes use NAT as well so that the outgoing packet is NAT'd to the correct address as it exits (has to be an outgoing session though), but it can be from a computer behind you for which you are routing.. that is how you can put a whole intranet out on multiple ISPs. But you need a different NAT instance on each external interface. > Dom > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 14:31:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 149716B4 for ; Mon, 28 Apr 2014 14:31:07 +0000 (UTC) Received: from mail-oa0-x22a.google.com (mail-oa0-x22a.google.com [IPv6:2607:f8b0:4003:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D2DF51464 for ; Mon, 28 Apr 2014 14:31:06 +0000 (UTC) Received: by mail-oa0-f42.google.com with SMTP id i4so7272676oah.15 for ; Mon, 28 Apr 2014 07:31:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=qIve3IrYxENMv9DvoWFHEXMaBMqxr7JHEB3Ui7QfjY8=; b=L49n28PgeaFW4kSwmYr0uTtsRqfAdXPq2npXrBVumKJ0yJvQao4OPogBV2k1/8tot7 RiBdZk9ZMXqJ1kSis2uvCmqGl7GOFWy6TPKWDSPlwiDPbiAw1lmhs0eDyG6947Lx99Zc poq9EAOeQU+0WJuDVbcL/cNSKpfsNfQWhSwKHhhCIyiZGKJkoDDqLRPcID3Zf/by2mpB ahYfMQwRyUys+BOyHDachbQr1KypDeVsB9f4YjBDtpLxdJSNEaPDnCQMXA9NBH7VnboA kmg/hKmokRUjq/yD4ZJlo3/zhmNz2EzPMOkWhtOWDCi6oHVjhsADcD+64ip4BWQg0W6v Peig== X-Received: by 10.60.176.39 with SMTP id cf7mr7587353oec.45.1398695466116; Mon, 28 Apr 2014 07:31:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.232.72 with HTTP; Mon, 28 Apr 2014 07:30:46 -0700 (PDT) In-Reply-To: References: From: Raimundo Santos Date: Mon, 28 Apr 2014 11:30:46 -0300 Message-ID: Subject: Re: running netmap-ipfw with real NICs To: Mahnaz Talebi Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 14:31:07 -0000 On 28 April 2014 01:58, Mahnaz Talebi wrote: > I am trying to run netmap-based ipfw with real NICs Hello, there are some drivers that does not support netmap yet. Raimundo Santos From owner-freebsd-net@FreeBSD.ORG Mon Apr 28 14:58:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06692257 for ; Mon, 28 Apr 2014 14:58:10 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 807251854 for ; Mon, 28 Apr 2014 14:58:09 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id s7so5062122lbd.37 for ; Mon, 28 Apr 2014 07:58:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=INEXQ9jiWFKXFW2Kta6W55vqemga3pgvEMi2EUSyPoU=; b=pFlCRysPaDgJgzkXUD7toTfPJV42uoyzCKUY+Et1CX/HKVcJe/Yf4LPEBeD5pwCqwH gMd7ze7HLYd0jgI8eL3eVsX+qgDLZNDIjQZyg+4pWgrqqWYIBAfJTS6tmUF8ZzSHiOzO ZA8SmZZs4RySiKXc7goRrI1Q3EquXXfB3gacHwbgRlER9ff8aMgmYehrPhUVV2IQeq4D WbUNLVwJcMT6m621SQcpTwkFnxinRGbrBY9CQuNYzdo5yhbjFZxL8/7ikZJ93zWjrQib 4uQGisz9IXBG+Jcnm30wwxXweggto88P/GiPNPVsc6YYxVyQROPXfrdMkOEqc04Qv8rj eF6A== MIME-Version: 1.0 X-Received: by 10.152.8.161 with SMTP id s1mr201173laa.67.1398697087405; Mon, 28 Apr 2014 07:58:07 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Mon, 28 Apr 2014 07:58:07 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Apr 2014 16:58:07 +0200 X-Google-Sender-Auth: oCfLXaYZlaLQ6Ml7lSC4D3zDJck Message-ID: Subject: Re: running netmap-ipfw with real NICs From: Luigi Rizzo To: Raimundo Santos Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , Mahnaz Talebi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 14:58:10 -0000 On Mon, Apr 28, 2014 at 4:30 PM, Raimundo Santos wrote: > On 28 April 2014 01:58, Mahnaz Talebi wrote: > > > I am trying to run netmap-based ipfw with real NICs > > > Hello, > > there are some drivers that does not support netmap yet. > =E2=80=8Bthanks for the answer but it wasn't that, i spoke to Mahnaz and he was just running out of memory, as dmesg showed. cheers luigi =E2=80=8B > > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Thu May 1 06:37:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 902F92FF for ; Thu, 1 May 2014 06:37:47 +0000 (UTC) Received: from nm18-vm5.bullet.mail.ir2.yahoo.com (nm18-vm5.bullet.mail.ir2.yahoo.com [212.82.96.229]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC0301360 for ; Thu, 1 May 2014 06:37:46 +0000 (UTC) Received: from [212.82.98.48] by nm18.bullet.mail.ir2.yahoo.com with NNFMP; 01 May 2014 06:35:37 -0000 Received: from [46.228.39.81] by tm1.bullet.mail.ir2.yahoo.com with NNFMP; 01 May 2014 06:35:37 -0000 Received: from [127.0.0.1] by smtp118.mail.ir2.yahoo.com with NNFMP; 01 May 2014 06:35:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1398926137; bh=neJRh554hy6iWQza2AkZ6JQZR/+NtXltSbXL58T8qak=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Content-Transfer-Encoding:Subject:From:Message-Id:Date:To:Mime-Version:X-Mailer; b=Ovm/fxIJ5jGcKyyDwAEL0e/jqs6qsyS9KWYayWGjOaseNC7q2kR2/d7waW9X85arFu01Un3krcTekOfQxKwbPfVdezTfCsLpgb+CZD1JI1bHh8vMzmutJbEcT2XSdCbIGMJbh/hhuSXgZZA5jgcTUYEmrtscGhyZwwQW5bxnCq8= X-Yahoo-Newman-Id: 661075.66406.bm@smtp118.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 3_0lmLsVM1kfGF3DMoLPOu.KnMbrYrFI8reL.P5AnWn0vo. yeNpVYGfCMx9gAk8b_IaI3DzDcokW44knbJn1GZxyFO357rYP.p1kekFC.oO MYxyWs2tIlLTaUKgFEDZKJ0d04Yilshf9dUKWGQseKO5pqGr1UyqlTUpU8hY FcAcyi7.KiINe7i5.q5VqMg65sqpJal5tolWhRC49cglPzUqiy96llHOQdgu w31cjEzROSH5CZJFxbIgyG.BysHSsRDc4ychVTka00iMTOrEnhzd.US6fxi. UN8tpP2wKPE6V.3MOyB1Zx2ootkYzNqx3tcJ5F89J5zqZVjJrl4jpC2oJAiz 6jD8qtyiL3TWCOZEcLO9rbYWe9y0PZnDMXqnLpEKM5UmaQP.U8duR_L3TYU2 UapjBNRZP2O9tVPqqLA95OCzrplsJ0FxI0WEZS5dG.a1vp635rZxBesGOKfL qX9QlC.JVL3waoQhloQ9J9U7O3x.2XiCIHLIKDYZBKHnvVygR1G1MG2h6x94 ycYmrPs5Gg46_GFA3sWCaj6oE8N8NWGUnzS90Sz4YfQVS X-Yahoo-SMTP: qB.Ljp.swBBJHuhONLPUz0naahUL1gQ0gTo- X-Rocket-Received: from [192.168.1.106] (multani.nicky@151.62.10.119 with xymcookie [46.228.39.225]) by smtp118.mail.ir2.yahoo.com with SMTP; 01 May 2014 06:35:37 +0000 UTC Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: ifconfig alias: same subnet netmask question From: Amrit Message-Id: Date: Thu, 1 May 2014 08:35:35 +0200 To: "freebsd-net@freebsd.org" Mime-Version: 1.0 (1.0) X-Mailer: iPod Mail (10B500) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 06:37:47 -0000 Zx Sent from my iPod From owner-freebsd-net@FreeBSD.ORG Thu May 1 09:27:00 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F0F9F13; Thu, 1 May 2014 09:27:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3525F1397; Thu, 1 May 2014 09:27:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s419R03x056774; Thu, 1 May 2014 09:27:00 GMT (envelope-from thomas@freefall.freebsd.org) Received: (from thomas@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s419R0mg056773; Thu, 1 May 2014 09:27:00 GMT (envelope-from thomas) Date: Thu, 1 May 2014 09:27:00 GMT Message-Id: <201405010927.s419R0mg056773@freefall.freebsd.org> To: thomas@cuivre.fr.eu.org, thomas@FreeBSD.org, freebsd-net@FreeBSD.org From: thomas@FreeBSD.org Subject: Re: kern/149117: [inet] [patch] in_pcbbind: redundant test X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 09:27:00 -0000 Synopsis: [inet] [patch] in_pcbbind: redundant test State-Changed-From-To: open->closed State-Changed-By: thomas State-Changed-When: Thu May 1 09:26:05 UTC 2014 State-Changed-Why: Fixed by np in rev. r245914 on 2013-01-25 http://www.freebsd.org/cgi/query-pr.cgi?pr=149117 From owner-freebsd-net@FreeBSD.ORG Thu May 1 12:49:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7763C128 for ; Thu, 1 May 2014 12:49:53 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BDC51758 for ; Thu, 1 May 2014 12:49:52 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s41CnG6h007673 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 1 May 2014 14:49:26 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s41CnADG064188 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 1 May 2014 14:49:10 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s41CnAFe051158; Thu, 1 May 2014 14:49:10 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s41Cn9Ms051157; Thu, 1 May 2014 14:49:09 +0200 (CEST) (envelope-from ticso) Date: Thu, 1 May 2014 14:49:09 +0200 From: Bernd Walter To: freebsd-net@freebsd.org Subject: How to create an SCTP association Message-ID: <20140501124908.GA50185@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 12:49:53 -0000 I have an SOCK_SEQPACKET socket and want to setup an association without sending a message. The background is that I want to keep the round-trip times of peer servers. I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO to get the RTT. But in case a host is down or services are restarted I don't have an association to ask, so I will need some way to (re)establish associations. This is done in a different thread as the normal socket operation, which uses EOR to write. If I send a (dummy)message to the socket I would have to add a mutex to not disturb the sending thread. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Thu May 1 13:13:38 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D114A44 for ; Thu, 1 May 2014 13:13:38 +0000 (UTC) Received: from mail.agsec.de (mail.kts.org [IPv6:2a00:14b0:f000:1::222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDC941A20 for ; Thu, 1 May 2014 13:13:37 +0000 (UTC) Received: from hh01.agsec.de (localhost [127.0.0.1]) by mail.agsec.de (Postfix) with ESMTP id 049947E29E8 for ; Thu, 1 May 2014 15:13:25 +0200 (CEST) X-Virus-Scanned-AGSEC: MailMurkxDeScraCxler at agsec.de Received: from mail.agsec.de ([194.55.156.222]) by hh01.agsec.de (hh01.agsec.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O2VOsALbDqxW for ; Thu, 1 May 2014 15:13:22 +0200 (CEST) Received: from ernie.int.kts.org (ernie.xnet.kts.org [IPv6:2001:6f8:1c56:200::1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.agsec.de (Postfix) with ESMTPS id 179B57E22F5 for ; Thu, 1 May 2014 15:13:22 +0200 (CEST) X-Virus-Scanned-KTS: Mail-UnWroks-U-Laksler at KTS.ORG Received: from frazzle.int.kts.org (frazzle.int.kts.org [172.31.42.11]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ernie.int.kts.org (Postfix) with ESMTPSA id B9532D4C49 for ; Thu, 1 May 2014 14:53:00 +0200 (CEST) From: Hellmuth Michaelis Content-Type: multipart/signed; boundary="Apple-Mail=_EBFE6B1B-F36F-49AE-8FF3-1865D0E9E2D2"; protocol="application/pgp-signature"; micalg=pgp-sha1 Subject: pf and ipv6 pmtu discovery not working Message-Id: <5B8E8711-24A5-46AD-893A-12C087117672@hellmuth-michaelis.de> Date: Thu, 1 May 2014 14:52:47 +0200 To: freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 13:13:38 -0000 --Apple-Mail=_EBFE6B1B-F36F-49AE-8FF3-1865D0E9E2D2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, for some time now i'm trying to nail down a phenomenon of hanging ipv6 transfers, it looks to me like there is a problem of getting ipv6 path=20= mtu discovery right when using FreeBSD and pf. My setup is an outside router (pisix, an rpi running freebsd connecting to sixxs) connected via a firewall to a "freebsd server" (ernie). The "outside router" has an MTU of 1280 bytes, everything after it to the=20 inside has an MTU of 1500 bytes. On the server i have pf running.=20 Its a 10-stable as of 28th april amd64 box.=20 Now when the "freebsd server" sends a packet via the "outside router"=20 which is larger than 1280 Bytes to the internet, the "outside router"=20 sends an ICMP packet too big to the server, which in turn lowers its MTU as shown in this tcpdump (which was taken with "set skip on fxp0" in /etc/pf.conf, where fxp0 is the interface where this takes place): 10:45:16.071634 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags = [P.], seq 518:1849, ack 167, win 2053, options [nop,nop,TS val 915142 = ecr 2428082038], length 1331 10:45:16.073732 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, = length 1240 10:45:16.073989 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags = [.], seq 518:1726, ack 167, win 2053, options [nop,nop,TS val 915144 ecr = 2428082038], length 1208 10:45:16.074119 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags = [P.], seq 1726:1849, ack 167, win 2053, options [nop,nop,TS val 915144 = ecr 2428082038], length 123 When the "set skip on fxp0" line in pf.conf is removed, the above described pmtu scenario does not work anymore: 14:02:11.106566 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags = [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val = 12730177 ecr 2439884647], length 1398 14:02:11.109301 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, = length 1240 14:02:11.788529 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags = [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val = 12730859 ecr 2439884647], length 1398 14:02:11.791271 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, = length 1240 14:02:12.965534 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags = [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val = 12732036 ecr 2439884647], length 1398 14:02:12.968302 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, = length 1240 14:02:15.093370 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags = [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val = 12734164 ecr 2439884647], length 1398 I found a similar thread of a FreeBSD bug in this area and tried to disable net.inet.tcp.rfc1323, this makes no difference. All sorts of rearranging rules or stripping ruled down to a bare minimum in pf.conf made no difference. Either disable pf at all or add "set skip on fxp0" make pmtu work. Finally i added some printf's in /usr/src/sys/netinet6/icmp6.c, but when pf is enabled, the icmp's don't even get there, so they must be filtered out earlier. At this point i found out about pfctl -x loud. For the pf enabled=20 state, the console said for a similar situation: pf: BAD ICMP 2:0 2001:xxxx:yyyy:100::253 -> 2001:xxxx:yyyy:200::1 state: = TCP out wire: 2a00:1450:4008:801::101f[443] 2001:xxxx:yyyy:200::1[58987] = stack: - [lo=3D2065898190 high=3D2065912306 win=3D2053 modulator=3D0 = wscale=3D7] [lo=3D2904051605 high=3D2904314389 win=3D240 modulator=3D0 = wscale=3D6] 4:4 seq=3D1059571332 Then i made the following modification to the file /usr/src/sys/netpfil/pf/pf.c: 4665c4665 < if (!((*state)->state_flags & PFSTATE_SLOPPY) && --- > if ((icmptype !=3D ICMP6_PACKET_TOO_BIG) && = (!((*state)->state_flags & PFSTATE_SLOPPY) && 4667c4667 < !SEQ_GEQ(seq, src->seqlo - (dst->max_win << = dws)))) { --- > !SEQ_GEQ(seq, src->seqlo - (dst->max_win << = dws))))) { and voila, pmtu works with pf enabled. It seems the behaviour must have somthing to do with the if statement if (!((*state)->state_flags & PFSTATE_SLOPPY) && (!SEQ_GEQ(src->seqhi, seq) || !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws))))=20 in /usr/src/sys/netpfil/pf/pf.c Setting "keep state (sloppy)" in a line in pf.conf letting v6 icmp=92s thru made no difference. Same with adding =84no state=93. I'm a bit stuck here right now. Is there perhaps someone with a bit more insight in this type of pf-mechanics who might want to have a look at this ? Hellmuth --=20 Hellmuth Michaelis Hallstra=DFe 20 25462 Rellingen Mobil: +49-160-9645 5696 Telefon: +49-4101-85299-20 Telefax: +49-4101-85299-21 web: www.hellmuth-michaelis.de mail: hm@hellmuth-michaelis.de --Apple-Mail=_EBFE6B1B-F36F-49AE-8FF3-1865D0E9E2D2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTYkOfAAoJEDc16e0fOQ8BGQAH/0yRpq+W4Fq08ZRnN0+YkO5Y b+03XkW80DpOg5rYDPkPJ8TgkN7EmpUM/n8VeJhnndc8L+gtZpaBOx6s/7qnYI1o vzuEog4e9fI0abGdME6WEocGExRUQMLtmGwj//rBAAtg7QbNQMqSmm1LFDbsXa+x vtMc3yYqp45CI5GfWsAmdVdNYfCiTqLeIqL+evJB4/2U9el1yuKU1L3yKTveLAns 0un3cI0iCO0qVAIOTduQQwSxNxRrcPHRaZn8ctbDI0x0CYc5/rW8z1XS3QzQ6LMR 7YIPirc23HlWS5Hwnv+0VKE3W5xmVkcawHB62MlHWYCTH5UdhabfpMmL5fmrino= =5v0D -----END PGP SIGNATURE----- --Apple-Mail=_EBFE6B1B-F36F-49AE-8FF3-1865D0E9E2D2-- From owner-freebsd-net@FreeBSD.ORG Thu May 1 14:59:25 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59FD4FAE; Thu, 1 May 2014 14:59:25 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2CDD914CE; Thu, 1 May 2014 14:59:25 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s41ExOoq087130; Thu, 1 May 2014 14:59:24 GMT (envelope-from melifaro@freefall.freebsd.org) Received: (from melifaro@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s41ExOAq087129; Thu, 1 May 2014 14:59:24 GMT (envelope-from melifaro) Date: Thu, 1 May 2014 14:59:24 GMT Message-Id: <201405011459.s41ExOAq087129@freefall.freebsd.org> To: melifaro@FreeBSD.org, freebsd-net@FreeBSD.org, melifaro@FreeBSD.org From: melifaro@FreeBSD.org Subject: Re: kern/174959: [net] [patch] rnh_walktree_from visits spurious nodes X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 14:59:25 -0000 Synopsis: [net] [patch] rnh_walktree_from visits spurious nodes Responsible-Changed-From-To: freebsd-net->melifaro Responsible-Changed-By: melifaro Responsible-Changed-When: Thu May 1 14:59:13 UTC 2014 Responsible-Changed-Why: Take. http://www.freebsd.org/cgi/query-pr.cgi?pr=174959 From owner-freebsd-net@FreeBSD.ORG Thu May 1 15:20:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 259D2BA4 for ; Thu, 1 May 2014 15:20:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12BF116E6 for ; Thu, 1 May 2014 15:20:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s41FK0TM095784 for ; Thu, 1 May 2014 15:20:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s41FK0ID095782; Thu, 1 May 2014 15:20:00 GMT (envelope-from gnats) Date: Thu, 1 May 2014 15:20:00 GMT Message-Id: <201405011520.s41FK0ID095782@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Alexander V. Chernikov" Subject: Re: kern/174958: [net] [patch] rnh_walktree_from makes unreasonable assumptions X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: "Alexander V. Chernikov" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 15:20:01 -0000 The following reply was made to PR kern/174958; it has been noted by GNATS. From: "Alexander V. Chernikov" To: bug-followup@FreeBSD.org, sklower@cs.berkeley.edu Cc: Subject: Re: kern/174958: [net] [patch] rnh_walktree_from makes unreasonable assumptions Date: Thu, 01 May 2014 19:18:41 +0400 Hello! Better late than never. I'm a bit unsure how patch&test case in this PR relates to kern/174959. Problems described here are the same as in 174959, however it looks like given patch addresses some other problem. It does not touch problem function at all, but introduces a bunch of new ones not used in test case. Can you please provide some more details? From owner-freebsd-net@FreeBSD.ORG Thu May 1 18:07:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2A725D1 for ; Thu, 1 May 2014 18:07:25 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8433E1989 for ; Thu, 1 May 2014 18:07:25 +0000 (UTC) Received: from [192.168.1.102] (p54819FCD.dip0.t-ipconnect.de [84.129.159.205]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 465F21C104626; Thu, 1 May 2014 20:07:21 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: How to create an SCTP association From: Michael Tuexen In-Reply-To: <20140501124908.GA50185@cicely7.cicely.de> Date: Thu, 1 May 2014 20:07:19 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: <20140501124908.GA50185@cicely7.cicely.de> To: ticso@cicely.de X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 18:07:25 -0000 On 01 May 2014, at 14:49, Bernd Walter wrote: > I have an SOCK_SEQPACKET socket and want to setup an association > without sending a message. Just call connect() or sctp_connectx(). > > The background is that I want to keep the round-trip times of peer > servers. > I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO > to get the RTT. > But in case a host is down or services are restarted I don't have > an association to ask, so I will need some way to (re)establish > associations. > This is done in a different thread as the normal socket operation, > which uses EOR to write. > If I send a (dummy)message to the socket I would have to add a mutex > to not disturb the sending thread. Does the above solve your issue? Best regards Michael > > -- > B.Walter http://www.bwct.de > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu May 1 20:43:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10512FE6 for ; Thu, 1 May 2014 20:43:13 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 96D5C1A19 for ; Thu, 1 May 2014 20:43:11 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s41KgsTv011478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 1 May 2014 22:42:54 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s41KgncL067804 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 1 May 2014 22:42:49 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s41KgnkR053124; Thu, 1 May 2014 22:42:49 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s41KgnuL053123; Thu, 1 May 2014 22:42:49 +0200 (CEST) (envelope-from ticso) Date: Thu, 1 May 2014 22:42:49 +0200 From: Bernd Walter To: Michael Tuexen Subject: Re: How to create an SCTP association Message-ID: <20140501204249.GB52252@cicely7.cicely.de> References: <20140501124908.GA50185@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: freebsd-net@freebsd.org, Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 20:43:13 -0000 On Thu, May 01, 2014 at 08:07:19PM +0200, Michael Tuexen wrote: > On 01 May 2014, at 14:49, Bernd Walter wrote: > > > I have an SOCK_SEQPACKET socket and want to setup an association > > without sending a message. > Just call connect() or sctp_connectx(). Cool - I wasn't sure about using *connect*() because tutorials only mention it for 1to1 sockets. > > The background is that I want to keep the round-trip times of peer > > servers. > > I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO > > to get the RTT. > > But in case a host is down or services are restarted I don't have > > an association to ask, so I will need some way to (re)establish > > associations. > > This is done in a different thread as the normal socket operation, > > which uses EOR to write. > > If I send a (dummy)message to the socket I would have to add a mutex > > to not disturb the sending thread. > Does the above solve your issue? Yes. But I have another question. Which resources will connecting an unreachable peer use on my one to many socket? My use case is a distributed hash table with many servers. Clients connect to servers via local proxy and the local proxy tries to stay in contact with all storage nodes - basicly to keep RTT values up to date. I initially plan 100 nodes per cluster, 2 cluster per location and 2 location. So such a proxy has to stay in touch with 400 peers of which 200 are on a remote location. Of course the remote location is rarely used, but in case of network failure 200 peers are unreachable. With TCP sockets the sockets are unrelated of each other. In this case I may try to sctp_connect 200 offline peers and still want to be able to reach the 200 local peers without additional latency. Is there anything special to care about to not block the traffic to the remaining peers in case a whole bunch of the others is offline? Raising socket buffer on the proxy servers is not a problem, since those have beafy RAM - the nodes however are tiny quad core ARM-A9 with 2G RAM only, which they mostly need to keep hashtable indexes. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Thu May 1 21:06:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47709761 for ; Thu, 1 May 2014 21:06:15 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A9581C2E for ; Thu, 1 May 2014 21:06:15 +0000 (UTC) Received: from [192.168.1.200] (p54819FCD.dip0.t-ipconnect.de [84.129.159.205]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id E5D311C104910; Thu, 1 May 2014 23:06:10 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: How to create an SCTP association From: Michael Tuexen In-Reply-To: <20140501204249.GB52252@cicely7.cicely.de> Date: Thu, 1 May 2014 23:06:09 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: <20140501124908.GA50185@cicely7.cicely.de> <20140501204249.GB52252@cicely7.cicely.de> To: ticso@cicely.de X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 21:06:15 -0000 On 01 May 2014, at 22:42, Bernd Walter wrote: > On Thu, May 01, 2014 at 08:07:19PM +0200, Michael Tuexen wrote: >> On 01 May 2014, at 14:49, Bernd Walter wrote: >> >>> I have an SOCK_SEQPACKET socket and want to setup an association >>> without sending a message. >> Just call connect() or sctp_connectx(). > > Cool - I wasn't sure about using *connect*() because tutorials only > mention it for 1to1 sockets. You might want to take a look at http://tools.ietf.org/html/rfc6458#section-3.1.6 http://tools.ietf.org/html/rfc6458#section-9.9 sctp_connectx() also provides you a way to get the association id, which is needed for 1-to-many style sockets. > >>> The background is that I want to keep the round-trip times of peer >>> servers. >>> I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO >>> to get the RTT. >>> But in case a host is down or services are restarted I don't have >>> an association to ask, so I will need some way to (re)establish >>> associations. >>> This is done in a different thread as the normal socket operation, >>> which uses EOR to write. >>> If I send a (dummy)message to the socket I would have to add a mutex >>> to not disturb the sending thread. >> Does the above solve your issue? > > Yes. > > But I have another question. > Which resources will connecting an unreachable peer use on my one to > many socket? Basically the resources for an association and it will run a timer for retransmitting the INITs. > > My use case is a distributed hash table with many servers. > Clients connect to servers via local proxy and the local proxy tries to > stay in contact with all storage nodes - basicly to keep RTT values up > to date. > I initially plan 100 nodes per cluster, 2 cluster per location and > 2 location. > So such a proxy has to stay in touch with 400 peers of which 200 are > on a remote location. > Of course the remote location is rarely used, but in case of network > failure 200 peers are unreachable. > With TCP sockets the sockets are unrelated of each other. > In this case I may try to sctp_connect 200 offline peers and still > want to be able to reach the 200 local peers without additional latency. > Is there anything special to care about to not block the traffic > to the remaining peers in case a whole bunch of the others is offline? > Raising socket buffer on the proxy servers is not a problem, since > those have beafy RAM - the nodes however are tiny quad core ARM-A9 with > 2G RAM only, which they mostly need to keep hashtable indexes. I don't see a problem right now. I suggest to do some testing. In case of problems, please let me know. Best regards Michael > > -- > B.Walter http://www.bwct.de > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. > From owner-freebsd-net@FreeBSD.ORG Thu May 1 22:04:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C87C963A for ; Thu, 1 May 2014 22:04:53 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 746001260 for ; Thu, 1 May 2014 22:04:53 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id s41M4N8l012026 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 2 May 2014 00:04:23 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id s41M4JBP068478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 2 May 2014 00:04:19 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id s41M4JvF053495; Fri, 2 May 2014 00:04:19 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id s41M4JOG053494; Fri, 2 May 2014 00:04:19 +0200 (CEST) (envelope-from ticso) Date: Fri, 2 May 2014 00:04:19 +0200 From: Bernd Walter To: Michael Tuexen Subject: Re: How to create an SCTP association Message-ID: <20140501220418.GC52252@cicely7.cicely.de> References: <20140501124908.GA50185@cicely7.cicely.de> <20140501204249.GB52252@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: freebsd-net@freebsd.org, Bernd Walter , ticso@cicely.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: ticso@cicely.de List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 22:04:53 -0000 On Thu, May 01, 2014 at 11:06:09PM +0200, Michael Tuexen wrote: > On 01 May 2014, at 22:42, Bernd Walter wrote: > > > On Thu, May 01, 2014 at 08:07:19PM +0200, Michael Tuexen wrote: > >> On 01 May 2014, at 14:49, Bernd Walter wrote: > >> > >>> I have an SOCK_SEQPACKET socket and want to setup an association > >>> without sending a message. > >> Just call connect() or sctp_connectx(). > > > > Cool - I wasn't sure about using *connect*() because tutorials only > > mention it for 1to1 sockets. > You might want to take a look at > http://tools.ietf.org/html/rfc6458#section-3.1.6 > http://tools.ietf.org/html/rfc6458#section-9.9 > sctp_connectx() also provides you a way to get the association id, > which is needed for 1-to-many style sockets. Ah - there it is. So many sources to get things running, SCTP still is too new to easily find good sample code beyound basics. About the association id - so far I'm always using socket_addr. I may be able to track association IDs, but are they really unique? E.g. association shuts down and a new one gets the ID? I also use it to get the RTT, which is untested code so far. > >>> The background is that I want to keep the round-trip times of peer > >>> servers. > >>> I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO > >>> to get the RTT. > >>> But in case a host is down or services are restarted I don't have > >>> an association to ask, so I will need some way to (re)establish > >>> associations. > >>> This is done in a different thread as the normal socket operation, > >>> which uses EOR to write. > >>> If I send a (dummy)message to the socket I would have to add a mutex > >>> to not disturb the sending thread. > >> Does the above solve your issue? > > > > Yes. > > > > But I have another question. > > Which resources will connecting an unreachable peer use on my one to > > many socket? > Basically the resources for an association and it will run a timer for > retransmitting the INITs. Great - that's what I was hoping for. I may have enough troubles already with data for down hosts stuck in socket buffer, but this is a one time thing until the data times out. Thank you -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-net@FreeBSD.ORG Fri May 2 07:10:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6E90F9C for ; Fri, 2 May 2014 07:10:48 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7822112DD for ; Fri, 2 May 2014 07:10:48 +0000 (UTC) Received: from [192.168.1.200] (p548190F2.dip0.t-ipconnect.de [84.129.144.242]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 7EB0D1C0AC854; Fri, 2 May 2014 09:10:43 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: How to create an SCTP association From: Michael Tuexen In-Reply-To: <20140501220418.GC52252@cicely7.cicely.de> Date: Fri, 2 May 2014 09:10:41 +0200 Content-Transfer-Encoding: 7bit Message-Id: <9597BCA1-4551-4C16-9949-534DF47775BA@lurchi.franken.de> References: <20140501124908.GA50185@cicely7.cicely.de> <20140501204249.GB52252@cicely7.cicely.de> <20140501220418.GC52252@cicely7.cicely.de> To: ticso@cicely.de X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Bernd Walter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 07:10:48 -0000 On 02 May 2014, at 00:04, Bernd Walter wrote: > On Thu, May 01, 2014 at 11:06:09PM +0200, Michael Tuexen wrote: >> On 01 May 2014, at 22:42, Bernd Walter wrote: >> >>> On Thu, May 01, 2014 at 08:07:19PM +0200, Michael Tuexen wrote: >>>> On 01 May 2014, at 14:49, Bernd Walter wrote: >>>> >>>>> I have an SOCK_SEQPACKET socket and want to setup an association >>>>> without sending a message. >>>> Just call connect() or sctp_connectx(). >>> >>> Cool - I wasn't sure about using *connect*() because tutorials only >>> mention it for 1to1 sockets. >> You might want to take a look at >> http://tools.ietf.org/html/rfc6458#section-3.1.6 >> http://tools.ietf.org/html/rfc6458#section-9.9 >> sctp_connectx() also provides you a way to get the association id, >> which is needed for 1-to-many style sockets. > > Ah - there it is. > So many sources to get things running, SCTP still is too new to > easily find good sample code beyound basics. > About the association id - so far I'm always using socket_addr. > I may be able to track association IDs, but are they really unique? > E.g. association shuts down and a new one gets the ID? http://tools.ietf.org/html/rfc6458 guarantees them to be unique between all existing associations on a given socket. So two associations on different sockets can have the same id. The spec also allows them to be reused, as indicated at the end of: http://tools.ietf.org/html/rfc6458#section-8.1.8 In FreeBSD the assoc ID is a 32-bit counter which gets incremented and wrapped around. So it is reused after a long time... > I also use it to get the RTT, which is untested code so far. > >>>>> The background is that I want to keep the round-trip times of peer >>>>> servers. >>>>> I've enabled regular heartbeat and can use SCTP_GET_PEER_ADDR_INFO >>>>> to get the RTT. >>>>> But in case a host is down or services are restarted I don't have >>>>> an association to ask, so I will need some way to (re)establish >>>>> associations. >>>>> This is done in a different thread as the normal socket operation, >>>>> which uses EOR to write. >>>>> If I send a (dummy)message to the socket I would have to add a mutex >>>>> to not disturb the sending thread. >>>> Does the above solve your issue? >>> >>> Yes. >>> >>> But I have another question. >>> Which resources will connecting an unreachable peer use on my one to >>> many socket? >> Basically the resources for an association and it will run a timer for >> retransmitting the INITs. > > Great - that's what I was hoping for. > I may have enough troubles already with data for down hosts stuck in > socket buffer, but this is a one time thing until the data times out. If you enable the SCTP_SEND_FAILED_EVENT you can get that data back if you want... Best regards Michael > > Thank you > > -- > B.Walter http://www.bwct.de > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. > From owner-freebsd-net@FreeBSD.ORG Fri May 2 10:22:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F366A753 for ; Fri, 2 May 2014 10:22:31 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B957A15C7 for ; Fri, 2 May 2014 10:22:31 +0000 (UTC) Received: from [192.168.1.200] (p548190F2.dip0.t-ipconnect.de [84.129.144.242]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id EFDA31C104913 for ; Fri, 2 May 2014 12:22:27 +0200 (CEST) From: Michael Tuexen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: RX checksum offloading problem Message-Id: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> Date: Fri, 2 May 2014 12:22:25 +0200 To: FreeBSD Net Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 10:22:32 -0000 Dear all, during testing I found that FreeBSD head (on a raspberry pi) accepts = SCTP packet with bad checksums. After debugging this I figured out that this is a = problem with the csum_flags defined in mbuf.h. The SCTP code on its input path checks for CSUM_SCTP_VALID, which is = defined in mbuf.h: #define CSUM_SCTP_VALID CSUM_L4_VALID This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is = considered to have a correct checksum. For UDP and TCP some drivers calculate the UDP/TCP checksum and set = CSUM_DATA_VALID in csum_flags to indicate that the UDP/TCP should consider csum_data to = figure out if the packet has a correct checksum. The problem is that CSUM_DATA_VALID = is defined as #define CSUM_DATA_VALID CSUM_L4_VALID In this case the semantic is not that the packet has a valid checksum, = but the csum_data field contains information. Now the following happens (on the raspberry pi the driver used is dev/usb/net/if_smsc.c 1. A packet is received and if it is not too short, the checksum = computed is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens for all IP packets, not only for UDP and TCP packets. 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as = CSUM_SCTP_VALID and accepts the packet. So no SCTP checksum check ever happened. Alternatives to fix this: 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP = packets, since it only makes sense in these cases. 2. Map CSUM_DATA_VALID to some other generic value in mbuf.h. However, = CSUM_L4_CALC, which would make perfect sense in my view, is already used for = CSUM_PSEUDO_HDR. Therefore we would have to define a new generic value. So what is the best way to fix this? Best regards Michael= From owner-freebsd-net@FreeBSD.ORG Fri May 2 14:03:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4705E18 for ; Fri, 2 May 2014 14:03:05 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 97CAA1A8C for ; Fri, 2 May 2014 14:03:05 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id C6B6025D3A92; Fri, 2 May 2014 14:02:55 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 6F663C22C24; Fri, 2 May 2014 14:02:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id HuAc5kCLlcuF; Fri, 2 May 2014 14:02:52 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (unknown [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 7BA68C22BD4; Fri, 2 May 2014 14:02:50 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: "Bjoern A. Zeeb" In-Reply-To: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> Date: Fri, 2 May 2014 14:02:49 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> To: Michael Tuexen X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 14:03:06 -0000 On 02 May 2014, at 10:22 , Michael Tuexen = wrote: > Dear all, >=20 > during testing I found that FreeBSD head (on a raspberry pi) accepts = SCTP packet > with bad checksums. After debugging this I figured out that this is a = problem with > the csum_flags defined in mbuf.h. >=20 > The SCTP code on its input path checks for CSUM_SCTP_VALID, which is = defined in mbuf.h: > #define CSUM_SCTP_VALID CSUM_L4_VALID > This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet = is considered > to have a correct checksum. >=20 > For UDP and TCP some drivers calculate the UDP/TCP checksum and set = CSUM_DATA_VALID in > csum_flags to indicate that the UDP/TCP should consider csum_data to = figure out if > the packet has a correct checksum. The problem is that CSUM_DATA_VALID = is defined as > #define CSUM_DATA_VALID CSUM_L4_VALID > In this case the semantic is not that the packet has a valid checksum, = but the csum_data > field contains information. >=20 > Now the following happens (on the raspberry pi the driver used is > dev/usb/net/if_smsc.c >=20 > 1. A packet is received and if it is not too short, the checksum = computed > is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens > for all IP packets, not only for UDP and TCP packets. > 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as = CSUM_SCTP_VALID > and accepts the packet. So no SCTP checksum check ever happened. >=20 > Alternatives to fix this: >=20 > 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or = TCP packets, since > it only makes sense in these cases. Wait, or for SCTP in cad the crc32 (I think it was) was actually = checked but not otherwise. This is how it should be imho. It seems = like a driver bug. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Fri May 2 18:04:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D42A6E1A for ; Fri, 2 May 2014 18:04:35 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7226918B9 for ; Fri, 2 May 2014 18:04:35 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id u57so4965295wes.31 for ; Fri, 02 May 2014 11:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=0szNfO+a/uo5kUZxE3BTKaVGco1oLRdtQqLzCbdQjG4=; b=X9pUzmXG4Rz3nkvtOK1HcUpNykckYbdGfF7VZaSP5OZJQwLPCV3CiYzDfoU+WCwchS GIhrTWsR8rA5YPzsa1FVT+I0gVmLOAQPoRODg28VOxPr4hfmB0Cn2OGkEJDPZ+dM/nks X7EY36oAmaNIKhw0BTwFhrrarILdES+dgLNg0DTpl5grVxImEd5RjhOYHGrPoBHLTkRs 8VXBdNV3HuHOGhYbWltaGzbavsBRznMWmA/2XMWIujsrLRSl8kztga/pjsEMtfvEng7s /Ze5zCMWvU3KwVrtq7JBkZN09hiP5RQcn+P3lzDu0Et67JNSrpGmkHVxe9b/AMG+/ny/ Bnvg== MIME-Version: 1.0 X-Received: by 10.194.88.74 with SMTP id be10mr1479840wjb.71.1399053873717; Fri, 02 May 2014 11:04:33 -0700 (PDT) Received: by 10.216.241.73 with HTTP; Fri, 2 May 2014 11:04:33 -0700 (PDT) Date: Fri, 2 May 2014 21:04:33 +0300 Message-ID: Subject: VLAN switching on freebsd From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 18:04:35 -0000 Hi, i need to create a virtual interface that forwards only defined vlan tags. Lets talk on a sample scenario : Assume that VLAN 10, 20, 30, 40 tagged on switch connected to em0 interface. create ngeth0 and ngeth1. ( i dont need netgraph interface, it can be a any virtual interface tap .. etc. ) i want to see only VLAN 10, 20 tagged on ngeth0 and VLAN 10, 30, 40 tagged on ngeth1 I tried many ways but no success. Can you suggest a way to do this? Best regards From owner-freebsd-net@FreeBSD.ORG Fri May 2 18:34:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAC90A4C for ; Fri, 2 May 2014 18:34:29 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E5ED1BBA for ; Fri, 2 May 2014 18:34:29 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s42IYNCo091164 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 2 May 2014 11:34:23 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s42IYNZA091163; Fri, 2 May 2014 11:34:23 -0700 (PDT) (envelope-from jmg) Date: Fri, 2 May 2014 11:34:23 -0700 From: John-Mark Gurney To: =?iso-8859-1?Q?=D6zkan?= KIRIK Subject: Re: VLAN switching on freebsd Message-ID: <20140502183422.GS43976@funkthat.com> Mail-Followup-To: =?iso-8859-1?Q?=D6zkan?= KIRIK , "freebsd-net@freebsd.org" References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 02 May 2014 11:34:23 -0700 (PDT) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 18:34:29 -0000 zkan KIRIK wrote this message on Fri, May 02, 2014 at 21:04 +0300: > i need to create a virtual interface that forwards only defined vlan tags. > Lets talk on a sample scenario : > > Assume that VLAN 10, 20, 30, 40 tagged on switch connected to em0 interface. > > create ngeth0 and ngeth1. ( i dont need netgraph interface, it can be a any > virtual interface tap .. etc. ) > i want to see only VLAN 10, 20 tagged on ngeth0 > and VLAN 10, 30, 40 tagged on ngeth1 > > I tried many ways but no success. > > Can you suggest a way to do this? I'm not familar w/ netgraph, but it looks like you might be able to do something simlar w/ ng_vlan and ng_bridge? Though bridge could be replaced w/ one2many, or hub depending upon requirements... Also, is this purely for snooping traffic? or do you want to be able to pass traffic both ways? em0 | ng_vlan / / \ \ 10 20 30 40 | | | | | ng_bridge | | \ / \ | | ng_vlan ng_vlan | | ngeth0 ngeth1 Not sure if this is exactly what you want, but I think it would... Though I don't know if you tried this, since you didn't describe anything you tried... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri May 2 18:59:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 998C2703 for ; Fri, 2 May 2014 18:59:41 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25FAC1DB1 for ; Fri, 2 May 2014 18:59:41 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n15so69148wiw.14 for ; Fri, 02 May 2014 11:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=iTXcWuJB07dmfacTR2Ll4/+Zv0azCpoD1CUh7Smt18U=; b=OahQrnfYubvsjxIWysqYlLDzr7W8FoHCjBdgm8lGO1P2CmJvEOXN+oNOP4YGSSg4hK BTD2itUXmXjhHuXoaBRWROjLcWmK+B85NIPyPmWIFocX2ZWd0nUMYOsPck1n8KDCxn3M AIeUPHDXY7bjS7QfYH7mW6vgAI3lQhwVBYu+QGnKbaedvfpMvkkRR2Fyr4iU+Bz229BB OHlFtSCvVKQ7BD892IuSBrN5Wsdks716flEsQoE/544PHBjTEZ7IWnKYHzJ9ZCkTk5CX YoEZZVRcMz79nDu6b7OiGu1Jz+S4F3enN4wwes785GdNEUqfmhB2gdQHHxHWAz0vC+Vn w2Qg== MIME-Version: 1.0 X-Received: by 10.194.91.175 with SMTP id cf15mr15417736wjb.5.1399057179188; Fri, 02 May 2014 11:59:39 -0700 (PDT) Received: by 10.216.241.73 with HTTP; Fri, 2 May 2014 11:59:39 -0700 (PDT) In-Reply-To: <20140502183422.GS43976@funkthat.com> References: <20140502183422.GS43976@funkthat.com> Date: Fri, 2 May 2014 21:59:39 +0300 Message-ID: Subject: Re: VLAN switching on freebsd From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 18:59:41 -0000 Thank you very much John, This approach is enlightened me. I'll try it. I couldnt think of use ng_vlan in front of ngeth nodes. I used openvswitch for this scenario, but performance is very very poor. I'm looking forward to release of the "in-kernel openvswitch on freebsd" project of Luigi Rizzo. Best Regards On Fri, May 2, 2014 at 9:34 PM, John-Mark Gurney wrote: > zkan KIRIK wrote this message on Fri, May 02, 2014 at 21:04 +0300: > > i need to create a virtual interface that forwards only defined vlan > tags. > > Lets talk on a sample scenario : > > > > Assume that VLAN 10, 20, 30, 40 tagged on switch connected to em0 > interface. > > > > create ngeth0 and ngeth1. ( i dont need netgraph interface, it can be a > any > > virtual interface tap .. etc. ) > > i want to see only VLAN 10, 20 tagged on ngeth0 > > and VLAN 10, 30, 40 tagged on ngeth1 > > > > I tried many ways but no success. > > > > Can you suggest a way to do this? > > I'm not familar w/ netgraph, but it looks like you might be able to > do something simlar w/ ng_vlan and ng_bridge? Though bridge could be > replaced w/ one2many, or hub depending upon requirements... Also, is > this purely for snooping traffic? or do you want to be able to pass > traffic both ways? > > em0 > | > ng_vlan > / / \ \ > 10 20 30 40 > | | | | > | ng_bridge | | > \ / \ | | > ng_vlan ng_vlan > | | > ngeth0 ngeth1 > > Not sure if this is exactly what you want, but I think it would... > Though I don't know if you tried this, since you didn't describe > anything you tried... > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > From owner-freebsd-net@FreeBSD.ORG Fri May 2 20:49:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7BF84CB for ; Fri, 2 May 2014 20:49:13 +0000 (UTC) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 362C4196E for ; Fri, 2 May 2014 20:49:13 +0000 (UTC) Received: by mail-we0-f175.google.com with SMTP id q58so5010633wes.20 for ; Fri, 02 May 2014 13:49:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=A8MOb0DsfZomELJXt6jP4+G6NYH6eInn3lo8F2bTPJc=; b=XC98BlYz3syTGNFUutmEOXf2AgwIpzhPPNPOr8wZck/wcJRuYKbysC7iF2RqdAZm45 sM23oNrDRc7FzfjJ9SkdNVbsj8VKctrOyeaTRT7EOp3EggD9N1n151Urp6YMUg68Eyh7 Y60uHqSDH/8nqhk2QC/1zkM040yAmEnLbj7S384B7dPiWGXNHwCpoxQ5IDwIdoRmP+7Q 5Fivyy1J3Cga2PIsDJsoses5Lcq4kjDvEcP3Y6ngL09KLHXUUNFERZdK7pbBlldBMChu 2vDWhEnmL7/uGldzFJOJjMzK8VRnTx/XL0REIJek0kzog9dVfppZzUsLtSaVUCcRHRm4 jxfA== MIME-Version: 1.0 X-Received: by 10.180.11.178 with SMTP id r18mr4647004wib.41.1399063751459; Fri, 02 May 2014 13:49:11 -0700 (PDT) Received: by 10.216.181.10 with HTTP; Fri, 2 May 2014 13:49:11 -0700 (PDT) Date: Fri, 2 May 2014 13:49:11 -0700 Message-ID: Subject: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Nick Rogers To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 20:49:13 -0000 Hello, I am hoping someone can help me debug a kernel panic I have been experiencing on one of my production systems. The system is a PF+ALTQ firewall/gateway with about 1k simultaneous devices behind it at any given time, pushing no more than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it with kgdb, but am still at a loss. I have completely replaced the hardware to rule out issues with disk/memory/etc, and it appears to be a kernel issue, likely with e1000 driver and/or PF. The panic seems to happen during times of heavier use, but the frequency is not very predictable (anywhere from a few times a day to a once a week), so I feel like its some kind of strange traffic pattern that I am unable to pinpoint. I am using a slightly custom kernel based on GENERIC, mainly to bring in ALTQ and some other options. It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 driver, although I do not believe that should affect em(4). Hoping someone can be of assistance in debugging this problem. I am willing to test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if that is advisable. Unfortunately I cannot try 10-STABLE. Thanks, -Nick Rogers uname -v FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS Here is the kernel conf... .............................................................................. include GENERIC ident RGNETS makeoptions MODULES_OVERRIDE="" options DEVICE_POLLING device crypto device cryptodev options VLAN_ARRAY device amdtemp # PF device pf device pflog options ALTQ options ALTQ_CBQ # Class Bases Queuing (CBQ) options ALTQ_RED # Random Early Detection (RED) options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) options ALTQ_PRIQ # Priority Queuing (PRIQ) options ALTQ_NOPCC # Required for SMP build # PPPoE options NETGRAPH options NETGRAPH_ETHER options NETGRAPH_PPPOE options NETGRAPH_SOCKET # IPsec device enc options IPSEC options IPSEC_FILTERTUNNEL options IPSEC_NAT_T .............................................................................. The crash dump.... .............................................................................. GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0x10 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8033d350 stack pointer = 0x28:0xffffff83545384b0 frame pointer = 0x28:0xffffff83545384c0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq262: em2:rx 0) trap number = 12 panic: page fault cpuid = 5 KDB: stack backtrace: #0 0xffffffff80956836 at kdb_backtrace+0x66 #1 0xffffffff8091c40e at panic+0x1ce #2 0xffffffff80d31e70 at trap_fatal+0x290 #3 0xffffffff80d321d1 at trap_pfault+0x211 #4 0xffffffff80d327d3 at trap+0x363 #5 0xffffffff80d1b9d3 at calltrap+0x8 #6 0xffffffff8034872d at pf_test_rule+0x17ed #7 0xffffffff8034ba12 at pf_test+0x1032 #8 0xffffffff8035112b at pf_check_in+0x2b #9 0xffffffff809e952e at pfil_run_hooks+0x9e #10 0xffffffff80a5286a at ip_input+0x2ea #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 #12 0xffffffff809df93d at ether_demux+0x14d #13 0xffffffff809dfc1e at ether_nh_input+0x1fe #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 #15 0xffffffff809df85f at ether_demux+0x6f #16 0xffffffff809dfc1e at ether_nh_input+0x1fe #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 Uptime: 17d7h20m59s Dumping 2932 out of 12256 MB: (CTRL-C to abort) ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/aio.ko...Reading symbols from /boot/kernel/aio.ko.symbols...done. done. Loaded symbols for /boot/kernel/aio.ko Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /boot/kernel/coretemp.ko.symbols...done. done. Loaded symbols for /boot/kernel/coretemp.ko Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from /boot/kernel/cc_htcp.ko.symbols...done. done. Loaded symbols for /boot/kernel/cc_htcp.ko #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:234 234 pcpu.h: No such file or directory. in pcpu.h (kgdb) list *0xffffffff8033d350 0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512). 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) 508 { 509 switch (af) { 510 #ifdef INET 511 case AF_INET: 512 dst->addr32[0] = src->addr32[0]; 513 break; 514 #endif /* INET */ 515 case AF_INET6: 516 dst->addr32[0] = src->addr32[0]; (kgdb) backtrace #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:234 #1 0xffffffff8091bee6 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:454 #2 0xffffffff8091c3e7 in panic (fmt=0x1
) at /usr/src/sys/kern/kern_shutdown.c:642 #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is not available. ) at /usr/src/sys/amd64/amd64/trap.c:878 #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at /usr/src/sys/amd64/amd64/trap.c:456 #6 0xffffffff80d1b9d3 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, pd=0xffffff83545386c0, am=0xffffff8354538790, rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at /usr/src/sys/contrib/pf/net/pf.c:3900 #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not available. ) at /usr/src/sys/contrib/pf/net/pf.c:6776 #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available. ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not available. ) at /usr/src/sys/net/pfil.c:82 #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at /usr/src/sys/netinet/ip_input.c:510 #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, source=Variable "source" is not available. ) at /usr/src/sys/net/netisr.c:1013 #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. ) at /usr/src/sys/net/if_ethersubr.c:762 #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, source=Variable "source" is not available. ) at /usr/src/sys/net/netisr.c:1013 #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. ) at /usr/src/sys/net/if_ethersubr.c:762 #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, source=Variable "source" is not available. ) at /usr/src/sys/net/netisr.c:1013 #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available. ) at /usr/src/sys/dev/e1000/if_em.c:1593 #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" is not available. ) at /usr/src/sys/kern/kern_intr.c:1272 #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at /usr/src/sys/kern/kern_intr.c:1285 #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at /usr/src/sys/kern/kern_fork.c:996 #25 0xffffffff80d1befe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #26 0x0000000000000000 in ?? () #27 0x0000000000000000 in ?? () #28 0x0000000000000001 in ?? () #29 0x0000000000000000 in ?? () #30 0x0000000000000000 in ?? () #31 0x0000000000000000 in ?? () #32 0x0000000000000000 in ?? () #33 0x0000000000000000 in ?? () #34 0x0000000000000000 in ?? () #35 0x0000000000000000 in ?? () #36 0x0000000000000000 in ?? () #37 0x0000000000000000 in ?? () #38 0x0000000000000000 in ?? () #39 0x0000000000000000 in ?? () #40 0x0000000000000000 in ?? () #41 0x0000000000000000 in ?? () #42 0x0000000000000000 in ?? () #43 0x0000000000000000 in ?? () #44 0x0000000000000000 in ?? () #45 0x0000000000000000 in ?? () #46 0x0000000000000000 in ?? () #47 0x0000000000000000 in ?? () #48 0x0000000000000000 in ?? () ---Type to continue, or q to quit--- #49 0x0000000000000000 in ?? () #50 0x0000000000000000 in ?? () #51 0xfffffe0007304920 in ?? () #52 0xfffffe0003afd000 in ?? () #53 0xfffffe0007304920 in ?? () #54 0xffffff8354538b40 in ?? () #55 0xffffff8354538ae8 in ?? () #56 0xfffffe0003b00920 in ?? () #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, newtd=0xfffffe000730c300, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/sched_ule.c:1898 Previous frame inner to this frame (corrupt stack?) .............................................................................. Relevant pciconf output... .............................................................................. em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled .............................................................................. dev.em sysctl.... .............................................................................. dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 dev.em.0.%driver: em dev.em.0.%location: slot=0 function=0 dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 subdevice=0x10d3 class=0x020000 dev.em.0.%parent: pci1 dev.em.0.nvm: -1 dev.em.0.debug: -1 dev.em.0.fc: 3 dev.em.0.rx_int_delay: 0 dev.em.0.tx_int_delay: 66 dev.em.0.rx_abs_int_delay: 66 dev.em.0.tx_abs_int_delay: 66 dev.em.0.itr: 488 dev.em.0.rx_processing_limit: -1 dev.em.0.eee_control: 1 dev.em.0.link_irq: 2 dev.em.0.mbuf_alloc_fail: 0 dev.em.0.cluster_alloc_fail: 0 dev.em.0.dropped: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.rx_overruns: 0 dev.em.0.watchdog_timeouts: 0 dev.em.0.device_control: 1477444168 dev.em.0.rx_control: 67141634 dev.em.0.fc_high_water: 18432 dev.em.0.fc_low_water: 16932 dev.em.0.queue0.txd_head: 3265 dev.em.0.queue0.txd_tail: 3265 dev.em.0.queue0.tx_irq: 81153071 dev.em.0.queue0.no_desc_avail: 0 dev.em.0.queue0.rxd_head: 388 dev.em.0.queue0.rxd_tail: 387 dev.em.0.queue0.rx_irq: 79015024 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.missed_packets: 0 dev.em.0.mac_stats.recv_no_buff: 0 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.xon_recvd: 6 dev.em.0.mac_stats.xon_txd: 0 dev.em.0.mac_stats.xoff_recvd: 6 dev.em.0.mac_stats.xoff_txd: 0 dev.em.0.mac_stats.total_pkts_recvd: 122072630 dev.em.0.mac_stats.good_pkts_recvd: 122072618 dev.em.0.mac_stats.bcast_pkts_recvd: 257 dev.em.0.mac_stats.mcast_pkts_recvd: 0 dev.em.0.mac_stats.rx_frames_64: 8634 dev.em.0.mac_stats.rx_frames_65_127: 67656673 dev.em.0.mac_stats.rx_frames_128_255: 714152 dev.em.0.mac_stats.rx_frames_256_511: 609615 dev.em.0.mac_stats.rx_frames_512_1023: 8646536 dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 dev.em.0.mac_stats.good_octets_recvd: 82940411216 dev.em.0.mac_stats.good_octets_txd: 25718335997 dev.em.0.mac_stats.total_pkts_txd: 99833592 dev.em.0.mac_stats.good_pkts_txd: 99833592 dev.em.0.mac_stats.bcast_pkts_txd: 13 dev.em.0.mac_stats.mcast_pkts_txd: 0 dev.em.0.mac_stats.tx_frames_64: 2193 dev.em.0.mac_stats.tx_frames_65_127: 29089783 dev.em.0.mac_stats.tx_frames_128_255: 54412030 dev.em.0.mac_stats.tx_frames_256_511: 9565246 dev.em.0.mac_stats.tx_frames_512_1023: 1080398 dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 dev.em.0.mac_stats.tso_txd: 1468623 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.interrupts.asserts: 2 dev.em.0.interrupts.rx_pkt_timer: 0 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.tx_pkt_timer: 0 dev.em.0.interrupts.tx_abs_timer: 0 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_queue_min_thresh: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.rx_overrun: 0 dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 dev.em.1.%driver: em dev.em.1.%location: slot=0 function=0 dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 subdevice=0x10d3 class=0x020000 dev.em.1.%parent: pci2 dev.em.1.nvm: -1 dev.em.1.debug: -1 dev.em.1.fc: 3 dev.em.1.rx_int_delay: 0 dev.em.1.tx_int_delay: 66 dev.em.1.rx_abs_int_delay: 66 dev.em.1.tx_abs_int_delay: 66 dev.em.1.itr: 488 dev.em.1.rx_processing_limit: -1 dev.em.1.eee_control: 1 dev.em.1.link_irq: 2 dev.em.1.mbuf_alloc_fail: 0 dev.em.1.cluster_alloc_fail: 0 dev.em.1.dropped: 0 dev.em.1.tx_dma_fail: 1 dev.em.1.rx_overruns: 0 dev.em.1.watchdog_timeouts: 0 dev.em.1.device_control: 1477444168 dev.em.1.rx_control: 67141634 dev.em.1.fc_high_water: 18432 dev.em.1.fc_low_water: 16932 dev.em.1.queue0.txd_head: 2451 dev.em.1.queue0.txd_tail: 2453 dev.em.1.queue0.tx_irq: 143904807 dev.em.1.queue0.no_desc_avail: 0 dev.em.1.queue0.rxd_head: 342 dev.em.1.queue0.rxd_tail: 341 dev.em.1.queue0.rx_irq: 159303310 dev.em.1.mac_stats.excess_coll: 0 dev.em.1.mac_stats.single_coll: 0 dev.em.1.mac_stats.multiple_coll: 0 dev.em.1.mac_stats.late_coll: 0 dev.em.1.mac_stats.collision_count: 0 dev.em.1.mac_stats.symbol_errors: 0 dev.em.1.mac_stats.sequence_errors: 0 dev.em.1.mac_stats.defer_count: 0 dev.em.1.mac_stats.missed_packets: 0 dev.em.1.mac_stats.recv_no_buff: 0 dev.em.1.mac_stats.recv_undersize: 0 dev.em.1.mac_stats.recv_fragmented: 0 dev.em.1.mac_stats.recv_oversize: 0 dev.em.1.mac_stats.recv_jabber: 0 dev.em.1.mac_stats.recv_errs: 0 dev.em.1.mac_stats.crc_errs: 0 dev.em.1.mac_stats.alignment_errs: 0 dev.em.1.mac_stats.coll_ext_errs: 0 dev.em.1.mac_stats.xon_recvd: 1 dev.em.1.mac_stats.xon_txd: 0 dev.em.1.mac_stats.xoff_recvd: 1 dev.em.1.mac_stats.xoff_txd: 0 dev.em.1.mac_stats.total_pkts_recvd: 331901758 dev.em.1.mac_stats.good_pkts_recvd: 331901756 dev.em.1.mac_stats.bcast_pkts_recvd: 13467 dev.em.1.mac_stats.mcast_pkts_recvd: 0 dev.em.1.mac_stats.rx_frames_64: 13905035 dev.em.1.mac_stats.rx_frames_65_127: 22315178 dev.em.1.mac_stats.rx_frames_128_255: 8343368 dev.em.1.mac_stats.rx_frames_256_511: 8602323 dev.em.1.mac_stats.rx_frames_512_1023: 8170288 dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 dev.em.1.mac_stats.good_octets_recvd: 420794715670 dev.em.1.mac_stats.good_octets_txd: 45361473880 dev.em.1.mac_stats.total_pkts_txd: 217852588 dev.em.1.mac_stats.good_pkts_txd: 217852588 dev.em.1.mac_stats.bcast_pkts_txd: 6 dev.em.1.mac_stats.mcast_pkts_txd: 0 dev.em.1.mac_stats.tx_frames_64: 64102191 dev.em.1.mac_stats.tx_frames_65_127: 120705475 dev.em.1.mac_stats.tx_frames_128_255: 6009336 dev.em.1.mac_stats.tx_frames_256_511: 4593595 dev.em.1.mac_stats.tx_frames_512_1023: 4295623 dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 dev.em.1.mac_stats.tso_txd: 291134 dev.em.1.mac_stats.tso_ctx_fail: 0 dev.em.1.interrupts.asserts: 2 dev.em.1.interrupts.rx_pkt_timer: 0 dev.em.1.interrupts.rx_abs_timer: 0 dev.em.1.interrupts.tx_pkt_timer: 0 dev.em.1.interrupts.tx_abs_timer: 0 dev.em.1.interrupts.tx_queue_empty: 0 dev.em.1.interrupts.tx_queue_min_thresh: 0 dev.em.1.interrupts.rx_desc_min_thresh: 0 dev.em.1.interrupts.rx_overrun: 0 dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 dev.em.2.%driver: em dev.em.2.%location: slot=0 function=0 dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 subdevice=0x10d3 class=0x020000 dev.em.2.%parent: pci7 dev.em.2.nvm: -1 dev.em.2.debug: -1 dev.em.2.fc: 3 dev.em.2.rx_int_delay: 0 dev.em.2.tx_int_delay: 66 dev.em.2.rx_abs_int_delay: 66 dev.em.2.tx_abs_int_delay: 66 dev.em.2.itr: 488 dev.em.2.rx_processing_limit: -1 dev.em.2.eee_control: 1 dev.em.2.link_irq: 1 dev.em.2.mbuf_alloc_fail: 0 dev.em.2.cluster_alloc_fail: 0 dev.em.2.dropped: 0 dev.em.2.tx_dma_fail: 6823 dev.em.2.rx_overruns: 0 dev.em.2.watchdog_timeouts: 0 dev.em.2.device_control: 1477444168 dev.em.2.rx_control: 67141634 dev.em.2.fc_high_water: 18432 dev.em.2.fc_low_water: 16932 dev.em.2.queue0.txd_head: 3977 dev.em.2.queue0.txd_tail: 3977 dev.em.2.queue0.tx_irq: 220950699 dev.em.2.queue0.no_desc_avail: 0 dev.em.2.queue0.rxd_head: 83 dev.em.2.queue0.rxd_tail: 82 dev.em.2.queue0.rx_irq: 125920607 dev.em.2.mac_stats.excess_coll: 0 dev.em.2.mac_stats.single_coll: 0 dev.em.2.mac_stats.multiple_coll: 0 dev.em.2.mac_stats.late_coll: 0 dev.em.2.mac_stats.collision_count: 0 dev.em.2.mac_stats.symbol_errors: 0 dev.em.2.mac_stats.sequence_errors: 0 dev.em.2.mac_stats.defer_count: 0 dev.em.2.mac_stats.missed_packets: 0 dev.em.2.mac_stats.recv_no_buff: 0 dev.em.2.mac_stats.recv_undersize: 0 dev.em.2.mac_stats.recv_fragmented: 0 dev.em.2.mac_stats.recv_oversize: 0 dev.em.2.mac_stats.recv_jabber: 0 dev.em.2.mac_stats.recv_errs: 0 dev.em.2.mac_stats.crc_errs: 0 dev.em.2.mac_stats.alignment_errs: 0 dev.em.2.mac_stats.coll_ext_errs: 0 dev.em.2.mac_stats.xon_recvd: 14123 dev.em.2.mac_stats.xon_txd: 1 dev.em.2.mac_stats.xoff_recvd: 14127 dev.em.2.mac_stats.xoff_txd: 1 dev.em.2.mac_stats.total_pkts_recvd: 229919303 dev.em.2.mac_stats.good_pkts_recvd: 229891053 dev.em.2.mac_stats.bcast_pkts_recvd: 909450 dev.em.2.mac_stats.mcast_pkts_recvd: 19452 dev.em.2.mac_stats.rx_frames_64: 1477808 dev.em.2.mac_stats.rx_frames_65_127: 195114744 dev.em.2.mac_stats.rx_frames_128_255: 6579690 dev.em.2.mac_stats.rx_frames_256_511: 5137387 dev.em.2.mac_stats.rx_frames_512_1023: 4223090 dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 dev.em.2.mac_stats.good_octets_recvd: 46129102134 dev.em.2.mac_stats.good_octets_txd: 419293159496 dev.em.2.mac_stats.total_pkts_txd: 332661584 dev.em.2.mac_stats.good_pkts_txd: 332661582 dev.em.2.mac_stats.bcast_pkts_txd: 48506 dev.em.2.mac_stats.mcast_pkts_txd: 78 dev.em.2.mac_stats.tx_frames_64: 14598198 dev.em.2.mac_stats.tx_frames_65_127: 22287108 dev.em.2.mac_stats.tx_frames_128_255: 8897511 dev.em.2.mac_stats.tx_frames_256_511: 9623000 dev.em.2.mac_stats.tx_frames_512_1023: 8325033 dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 dev.em.2.mac_stats.tso_txd: 24357891 dev.em.2.mac_stats.tso_ctx_fail: 0 dev.em.2.interrupts.asserts: 2 dev.em.2.interrupts.rx_pkt_timer: 0 dev.em.2.interrupts.rx_abs_timer: 0 dev.em.2.interrupts.tx_pkt_timer: 0 dev.em.2.interrupts.tx_abs_timer: 0 dev.em.2.interrupts.tx_queue_empty: 0 dev.em.2.interrupts.tx_queue_min_thresh: 0 dev.em.2.interrupts.rx_desc_min_thresh: 0 dev.em.2.interrupts.rx_overrun: 0 dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 dev.em.3.%driver: em dev.em.3.%location: slot=0 function=0 dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 subdevice=0x10d3 class=0x020000 dev.em.3.%parent: pci8 dev.em.3.nvm: -1 dev.em.3.debug: -1 dev.em.3.fc: 3 dev.em.3.rx_int_delay: 0 dev.em.3.tx_int_delay: 66 dev.em.3.rx_abs_int_delay: 66 dev.em.3.tx_abs_int_delay: 66 dev.em.3.itr: 488 dev.em.3.rx_processing_limit: -1 dev.em.3.eee_control: 1 dev.em.3.link_irq: 0 dev.em.3.mbuf_alloc_fail: 0 dev.em.3.cluster_alloc_fail: 0 dev.em.3.dropped: 0 dev.em.3.tx_dma_fail: 0 dev.em.3.rx_overruns: 0 dev.em.3.watchdog_timeouts: 0 dev.em.3.device_control: 1074790984 dev.em.3.rx_control: 67141634 dev.em.3.fc_high_water: 18432 dev.em.3.fc_low_water: 16932 dev.em.3.queue0.txd_head: 0 dev.em.3.queue0.txd_tail: 0 dev.em.3.queue0.tx_irq: 0 dev.em.3.queue0.no_desc_avail: 0 dev.em.3.queue0.rxd_head: 0 dev.em.3.queue0.rxd_tail: 4095 dev.em.3.queue0.rx_irq: 0 dev.em.3.mac_stats.excess_coll: 0 dev.em.3.mac_stats.single_coll: 0 dev.em.3.mac_stats.multiple_coll: 0 dev.em.3.mac_stats.late_coll: 0 dev.em.3.mac_stats.collision_count: 0 dev.em.3.mac_stats.symbol_errors: 0 dev.em.3.mac_stats.sequence_errors: 0 dev.em.3.mac_stats.defer_count: 0 dev.em.3.mac_stats.missed_packets: 0 dev.em.3.mac_stats.recv_no_buff: 0 dev.em.3.mac_stats.recv_undersize: 0 dev.em.3.mac_stats.recv_fragmented: 0 dev.em.3.mac_stats.recv_oversize: 0 dev.em.3.mac_stats.recv_jabber: 0 dev.em.3.mac_stats.recv_errs: 0 dev.em.3.mac_stats.crc_errs: 0 dev.em.3.mac_stats.alignment_errs: 0 dev.em.3.mac_stats.coll_ext_errs: 0 dev.em.3.mac_stats.xon_recvd: 0 dev.em.3.mac_stats.xon_txd: 0 dev.em.3.mac_stats.xoff_recvd: 0 dev.em.3.mac_stats.xoff_txd: 0 dev.em.3.mac_stats.total_pkts_recvd: 0 dev.em.3.mac_stats.good_pkts_recvd: 0 dev.em.3.mac_stats.bcast_pkts_recvd: 0 dev.em.3.mac_stats.mcast_pkts_recvd: 0 dev.em.3.mac_stats.rx_frames_64: 0 dev.em.3.mac_stats.rx_frames_65_127: 0 dev.em.3.mac_stats.rx_frames_128_255: 0 dev.em.3.mac_stats.rx_frames_256_511: 0 dev.em.3.mac_stats.rx_frames_512_1023: 0 dev.em.3.mac_stats.rx_frames_1024_1522: 0 dev.em.3.mac_stats.good_octets_recvd: 0 dev.em.3.mac_stats.good_octets_txd: 0 dev.em.3.mac_stats.total_pkts_txd: 0 dev.em.3.mac_stats.good_pkts_txd: 0 dev.em.3.mac_stats.bcast_pkts_txd: 0 dev.em.3.mac_stats.mcast_pkts_txd: 0 dev.em.3.mac_stats.tx_frames_64: 0 dev.em.3.mac_stats.tx_frames_65_127: 0 dev.em.3.mac_stats.tx_frames_128_255: 0 dev.em.3.mac_stats.tx_frames_256_511: 0 dev.em.3.mac_stats.tx_frames_512_1023: 0 dev.em.3.mac_stats.tx_frames_1024_1522: 0 dev.em.3.mac_stats.tso_txd: 0 dev.em.3.mac_stats.tso_ctx_fail: 0 dev.em.3.interrupts.asserts: 0 dev.em.3.interrupts.rx_pkt_timer: 0 dev.em.3.interrupts.rx_abs_timer: 0 dev.em.3.interrupts.tx_pkt_timer: 0 dev.em.3.interrupts.tx_abs_timer: 0 dev.em.3.interrupts.tx_queue_empty: 0 dev.em.3.interrupts.tx_queue_min_thresh: 0 dev.em.3.interrupts.rx_desc_min_thresh: 0 dev.em.3.interrupts.rx_overrun: 0 .............................................................................. From owner-freebsd-net@FreeBSD.ORG Fri May 2 20:53:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73D4E597 for ; Fri, 2 May 2014 20:53:36 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10E7F1A0F for ; Fri, 2 May 2014 20:53:35 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hi5so2030468wib.5 for ; Fri, 02 May 2014 13:53:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=QaJffN3H2tDpyz9w8qhF480kcdsf7tCwrO2ROxD6EFo=; b=Sfjjye6MMV8cC6h11mTuio3E85TBfXjBnqBC0TuIo4SSeBJajUCY7LPZSOkHuGa5fh ChfhBPjwfKmjZh7QBXITnOp2fPmhY83WfogrviKGlYsSu7OlXvF+5X3qjcXF1d/qjxx4 xWqntLOGj9upfKkf18kEozhwnSFXWzoGYZL6i5F2CVOnAF93WFV+fUdWObKoVCOjACG9 e23phlU5/4HZej7jm0XxWXdec3SyTHL+mVqpt2RWWo5hVKxHO8GCzFN7TwDqpABpJsh0 1UgrhFUPmqSKJeMl7tiDPs9SJdX4MaSqYLThACYITjp/bqYC2LbD6JMRon6oUOCrRRuG 2a9g== MIME-Version: 1.0 X-Received: by 10.194.1.242 with SMTP id 18mr15888134wjp.22.1399064014019; Fri, 02 May 2014 13:53:34 -0700 (PDT) Received: by 10.216.241.73 with HTTP; Fri, 2 May 2014 13:53:33 -0700 (PDT) Date: Fri, 2 May 2014 23:53:33 +0300 Message-ID: Subject: bridge Untagged packets on an interface From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 20:53:36 -0000 Hi, Assume that default vlan untagged and VLAN 10, 20, 30, 40 tagged on switch connected to em0 interface. i am trying to bridge only untagged frames on em0 with em1. Does if_vlan handle untagged frames? # ifconfig em0.0 create ifconfig: SIOCIFCREATE2: Invalid argument Any ideas? Best regards, From owner-freebsd-net@FreeBSD.ORG Fri May 2 20:59:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5896E795 for ; Fri, 2 May 2014 20:59:44 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD7711A37 for ; Fri, 2 May 2014 20:59:43 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id hm4so2980199wib.5 for ; Fri, 02 May 2014 13:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/r43hhrV+AYcERhPYxiq2dedVTWLBuXcCuHnsreY3cQ=; b=xEYGu41pQD/YgJUdEKtdrotVcBpwi9NwGMDxfNrWE0oKdkyPc8Q6uyran5y5xd8h8u FeIkoW+YqskJPcKnVwKViyBdMRkUMkwgYTJbihhhW0Zr0cEgJ9oCGifrhgyymOBL7T9w sttkZmh81Hidpx+dapREXAKdadOAfDNOy4bjUdlu2LBO7j0VgySBPYf/9aX/fPkW4A90 AISmUXRWHXVgVigY3rLiZmKjUYuU2NIH67ArmRh3bkpbqmvZPiM0TqQhLg8UETibKglF XLzGA4+ViM77JsmBjhBZhx4gT2AzFTuD44WLG7jU3CdRjUEdD0vEtHTsXzAyowUQwab/ l5fA== MIME-Version: 1.0 X-Received: by 10.180.207.10 with SMTP id ls10mr4694499wic.22.1399064382173; Fri, 02 May 2014 13:59:42 -0700 (PDT) Received: by 10.216.241.73 with HTTP; Fri, 2 May 2014 13:59:42 -0700 (PDT) In-Reply-To: References: Date: Fri, 2 May 2014 23:59:42 +0300 Message-ID: Subject: Re: bridge Untagged packets on an interface From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 20:59:44 -0000 And also i tried ng_bpf + ng_eiface conjuction, but ng_bpf doesnt match "vlan" filter. With the script below, ng_bpf always calls ifNotMatch hook. When the pattern is "ip", ng_bpf matches frames have both vlan and ip headers. I think ng_bpf doesnt process ethernet header. Is there a way to process ethernet header with ng_bpf ? Script is below: #!/bin/sh ETHER_IF=3Dem0 PATTERN=3D"vlan" BPFPROG=3D$( tcpdump -s 8192 -ddd ${PATTERN} | \ ( read len ; \ echo -n "bpf_prog_len=3D$len " ; \ echo -n "bpf_prog=3D[" ; \ while read code jt jf k ; do \ echo -n " { code=3D$code jt=3D$jt jf=3D$jf k=3D$k= }" ; \ done ; \ echo " ]" ) ) echo $BPFPROG # Shutdown nodes if exists ngctl shutdown ${ETHER_IF}: ngctl shutdown vlan_filter: ngctl shutdown tag0: ngctl shutdown untag0: ngctl -f- < w= rote: > Hi, > > Assume that default vlan untagged and VLAN 10, 20, 30, 40 tagged on switc= h > connected to em0 interface. > > i am trying to bridge only untagged frames on em0 with em1. > > Does if_vlan handle untagged frames? > > # ifconfig em0.0 create > ifconfig: SIOCIFCREATE2: Invalid argument > > Any ideas? > > Best regards, > From owner-freebsd-net@FreeBSD.ORG Sat May 3 02:37:22 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B4D46F9; Sat, 3 May 2014 02:37:22 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1036515E7; Sat, 3 May 2014 02:37:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s432bLjX090426; Sat, 3 May 2014 02:37:21 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s432bL9T090425; Sat, 3 May 2014 02:37:21 GMT (envelope-from linimon) Date: Sat, 3 May 2014 02:37:21 GMT Message-Id: <201405030237.s432bL9T090425@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/189234: [em] Big lag with Ethernet Connection I217-V X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 02:37:22 -0000 Synopsis: [em] Big lag with Ethernet Connection I217-V Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sat May 3 02:37:06 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=189234 From owner-freebsd-net@FreeBSD.ORG Sat May 3 06:44:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7E2D31E for ; Sat, 3 May 2014 06:44:58 +0000 (UTC) Received: from mail-ob0-x242.google.com (mail-ob0-x242.google.com [IPv6:2607:f8b0:4003:c01::242]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B56221A8D for ; Sat, 3 May 2014 06:44:58 +0000 (UTC) Received: by mail-ob0-f194.google.com with SMTP id va2so799210obc.1 for ; Fri, 02 May 2014 23:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=EMccv6TqMMrH67sDJzFzV6XE8CLGKbnfcaaLW82VTPc=; b=m1kGLl313yLG75lZ7PeEojqbnoVZ0N2W8W2bJMVLtbIab3i85oe/L3lvABUyoNPAPa db1oOZxjBlecnnPzvh7U4Oz2PzrjrZtV+Gl6ZNoEMr/k0yDZiXK77NroFynGEB32TNcQ InGizg13wuKdHoBA0p8yxZfffK3frGIuDzgUmgkTzUUivLmkDLMF6cVN3ygAhceyd8nw GMNckxlKxcvKXHj/5al/iTjTD054VuOz4dJKb+Bb0YbIozU0kZeWfD/fXYpTD9vHmRm2 QGDuQ4XskFXX/wX3uy4FKPnHma3NK5sNNG3Yasm3E0pdHbNZ+r72EQCOz9iVHSG4He9z nQLg== MIME-Version: 1.0 X-Received: by 10.182.33.99 with SMTP id q3mr19631950obi.33.1399099497689; Fri, 02 May 2014 23:44:57 -0700 (PDT) Received: by 10.76.101.18 with HTTP; Fri, 2 May 2014 23:44:57 -0700 (PDT) Date: Sat, 3 May 2014 14:44:57 +0800 Message-ID: Subject: netmap: how to bridge 2 eth? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 06:44:59 -0000 Hi all I'm doing to implement a simple openflow-based software switch(not same as OVS, much simpler to OVS) and I choose netmap for the network framework Now I try to do a simplest thing: there's 3 VMs, all are Ubuntu 12.04.4 x64 2 act as hosts, other 1 act as switch topo: [host1] eth0 ----- eth0 [switch] eth1 ----- eth0 [host2] I want to bridge switch's eth0ð1 (like "brctl" in linux), then host1 ping host2 the problem is, how to bridge using netmap? I tried example "bridge" in netmap project(git clone https://code.google.com/p/netmap/): ./bridge -i netmap:eth0 -i netmap:eth1 then host1 ping to host2 but pinging result is "Destination Host Unreachable" (if using brctl, pinging is fine) Then I tried vale-ctl but i dont know how to use it... e.g. ./vale-ctl -a eth0 show "eth0: Invalid argument" could anyone help me? From owner-freebsd-net@FreeBSD.ORG Sat May 3 08:11:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4FB6CF76 for ; Sat, 3 May 2014 08:11:02 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23E0311D8 for ; Sat, 3 May 2014 08:11:02 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id kp14so2140382pab.40 for ; Sat, 03 May 2014 01:11:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ys5uifsD9hSPqG4V6mghchnjOVImPkxCp5e76yZ2Z1U=; b=XZvCEW5IUIOIU+9NVXpT9cRTOOU7AXpui5IPcb2hP0kwq3RXgNf5W7/wE6yOFo/aQe +q2enn5hAqbIj2edl2xm1wRLo/MR/nzkvBegTvzUJ4RbyZeA25u5vLDnA+S11P2P3zo+ cT/wNzoLa0+Z7PtXdRIa6QeXu/q+61pyDcgzXRosbqS2PRpWKGoS3JhJeWUDmxtMSYUC 2Uj66hB2ubCUQEzS2qxDdLOdGDaldQdX93Q3igXyPZkIbapmwFCGsYc2kYDWH3oTwbTX Zt1BoJ7dD1n9y0Rt+pya7ucecTvtw0NbzHOTgwimY6duZ5m2KDRjhGed8HR5liSzmg/k B+tg== MIME-Version: 1.0 X-Received: by 10.67.1.39 with SMTP id bd7mr881642pad.15.1399104661410; Sat, 03 May 2014 01:11:01 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.70.88.103 with HTTP; Sat, 3 May 2014 01:11:01 -0700 (PDT) In-Reply-To: References: Date: Sat, 3 May 2014 10:11:01 +0200 X-Google-Sender-Auth: KxIEW31nNW2PMJqO_16-81KVZV4 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= To: Nick Rogers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 08:11:02 -0000 >From experience LEGACY_TX + ALTQ is not usable and it will panic similar to what you have shown above. I had to fix this for pfSense and the only way to get a stable driver was to have both if_transmit and if_start model activated in the driver. Finding the paths that needs this 'hybrid' is a bit time consuming but a strong candidate is altq interaction with the stack. There is work sponosred by the FreeBSD Foundation about to clarify/clean this up but i am not sure the staus of that so far. On Fri, May 2, 2014 at 10:49 PM, Nick Rogers wrote: > Hello, > > I am hoping someone can help me debug a kernel panic I have been > experiencing > on one of my production systems. The system is a PF+ALTQ firewall/gateway > with > about 1k simultaneous devices behind it at any given time, pushing no more > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it > with kgdb, but am still at a loss. > > I have completely replaced the hardware to rule out issues with > disk/memory/etc, and it appears to be a kernel issue, likely with e1000 > driver > and/or PF. > > The panic seems to happen during times of heavier use, but the frequency is > not very predictable (anywhere from a few times a day to a once a week), > so I > feel like its some kind of strange traffic pattern that I am unable to > pinpoint. > > I am using a slightly custom kernel based on GENERIC, mainly to bring in > ALTQ > and some other options. > > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 > driver, although I do not believe that should affect em(4). > > Hoping someone can be of assistance in debugging this problem. I am > willing to > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if > that > is advisable. Unfortunately I cannot try 10-STABLE. > > Thanks, > > -Nick Rogers > > > uname -v > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS > > > > Here is the kernel conf... > > .............................................................................. > include GENERIC > > ident RGNETS > > makeoptions MODULES_OVERRIDE="" > > options DEVICE_POLLING > > device crypto > device cryptodev > > options VLAN_ARRAY > > device amdtemp > > # PF > device pf > device pflog > options ALTQ > options ALTQ_CBQ # Class Bases Queuing (CBQ) > options ALTQ_RED # Random Early Detection (RED) > options ALTQ_RIO # RED In/Out > options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) > options ALTQ_PRIQ # Priority Queuing (PRIQ) > options ALTQ_NOPCC # Required for SMP build > > # PPPoE > options NETGRAPH > options NETGRAPH_ETHER > options NETGRAPH_PPPOE > options NETGRAPH_SOCKET > > # IPsec > device enc > options IPSEC > options IPSEC_FILTERTUNNEL > options IPSEC_NAT_T > > .............................................................................. > > > The crash dump.... > > .............................................................................. > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 5; apic id = 05 > fault virtual address = 0x10 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8033d350 > stack pointer = 0x28:0xffffff83545384b0 > frame pointer = 0x28:0xffffff83545384c0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (irq262: em2:rx 0) > trap number = 12 > panic: page fault > cpuid = 5 > KDB: stack backtrace: > #0 0xffffffff80956836 at kdb_backtrace+0x66 > #1 0xffffffff8091c40e at panic+0x1ce > #2 0xffffffff80d31e70 at trap_fatal+0x290 > #3 0xffffffff80d321d1 at trap_pfault+0x211 > #4 0xffffffff80d327d3 at trap+0x363 > #5 0xffffffff80d1b9d3 at calltrap+0x8 > #6 0xffffffff8034872d at pf_test_rule+0x17ed > #7 0xffffffff8034ba12 at pf_test+0x1032 > #8 0xffffffff8035112b at pf_check_in+0x2b > #9 0xffffffff809e952e at pfil_run_hooks+0x9e > #10 0xffffffff80a5286a at ip_input+0x2ea > #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #12 0xffffffff809df93d at ether_demux+0x14d > #13 0xffffffff809dfc1e at ether_nh_input+0x1fe > #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #15 0xffffffff809df85f at ether_demux+0x6f > #16 0xffffffff809dfc1e at ether_nh_input+0x1fe > #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 > Uptime: 17d7h20m59s > Dumping 2932 out of 12256 MB: (CTRL-C to abort) > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/aio.ko...Reading symbols from > /boot/kernel/aio.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/aio.ko > Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from > /boot/kernel/coretemp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/coretemp.ko > Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from > /boot/kernel/cc_htcp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/cc_htcp.ko > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > 234 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) list *0xffffffff8033d350 > 0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512). > 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) > 508 { > 509 switch (af) { > 510 #ifdef INET > 511 case AF_INET: > 512 dst->addr32[0] = src->addr32[0]; > 513 break; > 514 #endif /* INET */ > 515 case AF_INET6: > 516 dst->addr32[0] = src->addr32[0]; > (kgdb) backtrace > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > #1 0xffffffff8091bee6 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:454 > #2 0xffffffff8091c3e7 in panic (fmt=0x1
) > at /usr/src/sys/kern/kern_shutdown.c:642 > #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is > not available. > ) at /usr/src/sys/amd64/amd64/trap.c:878 > #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 > #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at > /usr/src/sys/amd64/amd64/trap.c:456 > #6 0xffffffff80d1b9d3 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:232 > #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, > src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 > #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, > sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, > m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, > pd=0xffffff83545386c0, am=0xffffff8354538790, > rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at > /usr/src/sys/contrib/pf/net/pf.c:3900 > #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not > available. > ) at /usr/src/sys/contrib/pf/net/pf.c:6776 > #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available. > ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 > #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not > available. > ) at /usr/src/sys/net/pfil.c:82 > #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at > /usr/src/sys/netinet/ip_input.c:510 > #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 > #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 > #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, > done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 > #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available. > ) at /usr/src/sys/dev/e1000/if_em.c:1593 > #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" > is not available. > ) at /usr/src/sys/kern/kern_intr.c:1272 > #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at > /usr/src/sys/kern/kern_intr.c:1285 > #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 > , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at > /usr/src/sys/kern/kern_fork.c:996 > #25 0xffffffff80d1befe in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #26 0x0000000000000000 in ?? () > #27 0x0000000000000000 in ?? () > #28 0x0000000000000001 in ?? () > #29 0x0000000000000000 in ?? () > #30 0x0000000000000000 in ?? () > #31 0x0000000000000000 in ?? () > #32 0x0000000000000000 in ?? () > #33 0x0000000000000000 in ?? () > #34 0x0000000000000000 in ?? () > #35 0x0000000000000000 in ?? () > #36 0x0000000000000000 in ?? () > #37 0x0000000000000000 in ?? () > #38 0x0000000000000000 in ?? () > #39 0x0000000000000000 in ?? () > #40 0x0000000000000000 in ?? () > #41 0x0000000000000000 in ?? () > #42 0x0000000000000000 in ?? () > #43 0x0000000000000000 in ?? () > #44 0x0000000000000000 in ?? () > #45 0x0000000000000000 in ?? () > #46 0x0000000000000000 in ?? () > #47 0x0000000000000000 in ?? () > #48 0x0000000000000000 in ?? () > ---Type to continue, or q to quit--- > #49 0x0000000000000000 in ?? () > #50 0x0000000000000000 in ?? () > #51 0xfffffe0007304920 in ?? () > #52 0xfffffe0003afd000 in ?? () > #53 0xfffffe0007304920 in ?? () > #54 0xffffff8354538b40 in ?? () > #55 0xffffff8354538ae8 in ?? () > #56 0xfffffe0003b00920 in ?? () > #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, > newtd=0xfffffe000730c300, flags=Variable "flags" is not available. > ) at /usr/src/sys/kern/sched_ule.c:1898 > Previous frame inner to this frame (corrupt stack?) > > > .............................................................................. > > > > Relevant pciconf output... > > .............................................................................. > > em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > > .............................................................................. > > > dev.em sysctl.... > > .............................................................................. > > dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.0.%driver: em > dev.em.0.%location: slot=0 function=0 > dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.0.%parent: pci1 > dev.em.0.nvm: -1 > dev.em.0.debug: -1 > dev.em.0.fc: 3 > dev.em.0.rx_int_delay: 0 > dev.em.0.tx_int_delay: 66 > dev.em.0.rx_abs_int_delay: 66 > dev.em.0.tx_abs_int_delay: 66 > dev.em.0.itr: 488 > dev.em.0.rx_processing_limit: -1 > dev.em.0.eee_control: 1 > dev.em.0.link_irq: 2 > dev.em.0.mbuf_alloc_fail: 0 > dev.em.0.cluster_alloc_fail: 0 > dev.em.0.dropped: 0 > dev.em.0.tx_dma_fail: 0 > dev.em.0.rx_overruns: 0 > dev.em.0.watchdog_timeouts: 0 > dev.em.0.device_control: 1477444168 > dev.em.0.rx_control: 67141634 > dev.em.0.fc_high_water: 18432 > dev.em.0.fc_low_water: 16932 > dev.em.0.queue0.txd_head: 3265 > dev.em.0.queue0.txd_tail: 3265 > dev.em.0.queue0.tx_irq: 81153071 > dev.em.0.queue0.no_desc_avail: 0 > dev.em.0.queue0.rxd_head: 388 > dev.em.0.queue0.rxd_tail: 387 > dev.em.0.queue0.rx_irq: 79015024 > dev.em.0.mac_stats.excess_coll: 0 > dev.em.0.mac_stats.single_coll: 0 > dev.em.0.mac_stats.multiple_coll: 0 > dev.em.0.mac_stats.late_coll: 0 > dev.em.0.mac_stats.collision_count: 0 > dev.em.0.mac_stats.symbol_errors: 0 > dev.em.0.mac_stats.sequence_errors: 0 > dev.em.0.mac_stats.defer_count: 0 > dev.em.0.mac_stats.missed_packets: 0 > dev.em.0.mac_stats.recv_no_buff: 0 > dev.em.0.mac_stats.recv_undersize: 0 > dev.em.0.mac_stats.recv_fragmented: 0 > dev.em.0.mac_stats.recv_oversize: 0 > dev.em.0.mac_stats.recv_jabber: 0 > dev.em.0.mac_stats.recv_errs: 0 > dev.em.0.mac_stats.crc_errs: 0 > dev.em.0.mac_stats.alignment_errs: 0 > dev.em.0.mac_stats.coll_ext_errs: 0 > dev.em.0.mac_stats.xon_recvd: 6 > dev.em.0.mac_stats.xon_txd: 0 > dev.em.0.mac_stats.xoff_recvd: 6 > dev.em.0.mac_stats.xoff_txd: 0 > dev.em.0.mac_stats.total_pkts_recvd: 122072630 > dev.em.0.mac_stats.good_pkts_recvd: 122072618 > dev.em.0.mac_stats.bcast_pkts_recvd: 257 > dev.em.0.mac_stats.mcast_pkts_recvd: 0 > dev.em.0.mac_stats.rx_frames_64: 8634 > dev.em.0.mac_stats.rx_frames_65_127: 67656673 > dev.em.0.mac_stats.rx_frames_128_255: 714152 > dev.em.0.mac_stats.rx_frames_256_511: 609615 > dev.em.0.mac_stats.rx_frames_512_1023: 8646536 > dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 > dev.em.0.mac_stats.good_octets_recvd: 82940411216 > dev.em.0.mac_stats.good_octets_txd: 25718335997 > dev.em.0.mac_stats.total_pkts_txd: 99833592 > dev.em.0.mac_stats.good_pkts_txd: 99833592 > dev.em.0.mac_stats.bcast_pkts_txd: 13 > dev.em.0.mac_stats.mcast_pkts_txd: 0 > dev.em.0.mac_stats.tx_frames_64: 2193 > dev.em.0.mac_stats.tx_frames_65_127: 29089783 > dev.em.0.mac_stats.tx_frames_128_255: 54412030 > dev.em.0.mac_stats.tx_frames_256_511: 9565246 > dev.em.0.mac_stats.tx_frames_512_1023: 1080398 > dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 > dev.em.0.mac_stats.tso_txd: 1468623 > dev.em.0.mac_stats.tso_ctx_fail: 0 > dev.em.0.interrupts.asserts: 2 > dev.em.0.interrupts.rx_pkt_timer: 0 > dev.em.0.interrupts.rx_abs_timer: 0 > dev.em.0.interrupts.tx_pkt_timer: 0 > dev.em.0.interrupts.tx_abs_timer: 0 > dev.em.0.interrupts.tx_queue_empty: 0 > dev.em.0.interrupts.tx_queue_min_thresh: 0 > dev.em.0.interrupts.rx_desc_min_thresh: 0 > dev.em.0.interrupts.rx_overrun: 0 > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.1.%driver: em > dev.em.1.%location: slot=0 function=0 > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.1.%parent: pci2 > dev.em.1.nvm: -1 > dev.em.1.debug: -1 > dev.em.1.fc: 3 > dev.em.1.rx_int_delay: 0 > dev.em.1.tx_int_delay: 66 > dev.em.1.rx_abs_int_delay: 66 > dev.em.1.tx_abs_int_delay: 66 > dev.em.1.itr: 488 > dev.em.1.rx_processing_limit: -1 > dev.em.1.eee_control: 1 > dev.em.1.link_irq: 2 > dev.em.1.mbuf_alloc_fail: 0 > dev.em.1.cluster_alloc_fail: 0 > dev.em.1.dropped: 0 > dev.em.1.tx_dma_fail: 1 > dev.em.1.rx_overruns: 0 > dev.em.1.watchdog_timeouts: 0 > dev.em.1.device_control: 1477444168 > dev.em.1.rx_control: 67141634 > dev.em.1.fc_high_water: 18432 > dev.em.1.fc_low_water: 16932 > dev.em.1.queue0.txd_head: 2451 > dev.em.1.queue0.txd_tail: 2453 > dev.em.1.queue0.tx_irq: 143904807 > dev.em.1.queue0.no_desc_avail: 0 > dev.em.1.queue0.rxd_head: 342 > dev.em.1.queue0.rxd_tail: 341 > dev.em.1.queue0.rx_irq: 159303310 > dev.em.1.mac_stats.excess_coll: 0 > dev.em.1.mac_stats.single_coll: 0 > dev.em.1.mac_stats.multiple_coll: 0 > dev.em.1.mac_stats.late_coll: 0 > dev.em.1.mac_stats.collision_count: 0 > dev.em.1.mac_stats.symbol_errors: 0 > dev.em.1.mac_stats.sequence_errors: 0 > dev.em.1.mac_stats.defer_count: 0 > dev.em.1.mac_stats.missed_packets: 0 > dev.em.1.mac_stats.recv_no_buff: 0 > dev.em.1.mac_stats.recv_undersize: 0 > dev.em.1.mac_stats.recv_fragmented: 0 > dev.em.1.mac_stats.recv_oversize: 0 > dev.em.1.mac_stats.recv_jabber: 0 > dev.em.1.mac_stats.recv_errs: 0 > dev.em.1.mac_stats.crc_errs: 0 > dev.em.1.mac_stats.alignment_errs: 0 > dev.em.1.mac_stats.coll_ext_errs: 0 > dev.em.1.mac_stats.xon_recvd: 1 > dev.em.1.mac_stats.xon_txd: 0 > dev.em.1.mac_stats.xoff_recvd: 1 > dev.em.1.mac_stats.xoff_txd: 0 > dev.em.1.mac_stats.total_pkts_recvd: 331901758 > dev.em.1.mac_stats.good_pkts_recvd: 331901756 > dev.em.1.mac_stats.bcast_pkts_recvd: 13467 > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > dev.em.1.mac_stats.rx_frames_64: 13905035 > dev.em.1.mac_stats.rx_frames_65_127: 22315178 > dev.em.1.mac_stats.rx_frames_128_255: 8343368 > dev.em.1.mac_stats.rx_frames_256_511: 8602323 > dev.em.1.mac_stats.rx_frames_512_1023: 8170288 > dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 > dev.em.1.mac_stats.good_octets_recvd: 420794715670 > dev.em.1.mac_stats.good_octets_txd: 45361473880 > dev.em.1.mac_stats.total_pkts_txd: 217852588 > dev.em.1.mac_stats.good_pkts_txd: 217852588 > dev.em.1.mac_stats.bcast_pkts_txd: 6 > dev.em.1.mac_stats.mcast_pkts_txd: 0 > dev.em.1.mac_stats.tx_frames_64: 64102191 > dev.em.1.mac_stats.tx_frames_65_127: 120705475 > dev.em.1.mac_stats.tx_frames_128_255: 6009336 > dev.em.1.mac_stats.tx_frames_256_511: 4593595 > dev.em.1.mac_stats.tx_frames_512_1023: 4295623 > dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 > dev.em.1.mac_stats.tso_txd: 291134 > dev.em.1.mac_stats.tso_ctx_fail: 0 > dev.em.1.interrupts.asserts: 2 > dev.em.1.interrupts.rx_pkt_timer: 0 > dev.em.1.interrupts.rx_abs_timer: 0 > dev.em.1.interrupts.tx_pkt_timer: 0 > dev.em.1.interrupts.tx_abs_timer: 0 > dev.em.1.interrupts.tx_queue_empty: 0 > dev.em.1.interrupts.tx_queue_min_thresh: 0 > dev.em.1.interrupts.rx_desc_min_thresh: 0 > dev.em.1.interrupts.rx_overrun: 0 > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.2.%driver: em > dev.em.2.%location: slot=0 function=0 > dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.2.%parent: pci7 > dev.em.2.nvm: -1 > dev.em.2.debug: -1 > dev.em.2.fc: 3 > dev.em.2.rx_int_delay: 0 > dev.em.2.tx_int_delay: 66 > dev.em.2.rx_abs_int_delay: 66 > dev.em.2.tx_abs_int_delay: 66 > dev.em.2.itr: 488 > dev.em.2.rx_processing_limit: -1 > dev.em.2.eee_control: 1 > dev.em.2.link_irq: 1 > dev.em.2.mbuf_alloc_fail: 0 > dev.em.2.cluster_alloc_fail: 0 > dev.em.2.dropped: 0 > dev.em.2.tx_dma_fail: 6823 > dev.em.2.rx_overruns: 0 > dev.em.2.watchdog_timeouts: 0 > dev.em.2.device_control: 1477444168 > dev.em.2.rx_control: 67141634 > dev.em.2.fc_high_water: 18432 > dev.em.2.fc_low_water: 16932 > dev.em.2.queue0.txd_head: 3977 > dev.em.2.queue0.txd_tail: 3977 > dev.em.2.queue0.tx_irq: 220950699 > dev.em.2.queue0.no_desc_avail: 0 > dev.em.2.queue0.rxd_head: 83 > dev.em.2.queue0.rxd_tail: 82 > dev.em.2.queue0.rx_irq: 125920607 > dev.em.2.mac_stats.excess_coll: 0 > dev.em.2.mac_stats.single_coll: 0 > dev.em.2.mac_stats.multiple_coll: 0 > dev.em.2.mac_stats.late_coll: 0 > dev.em.2.mac_stats.collision_count: 0 > dev.em.2.mac_stats.symbol_errors: 0 > dev.em.2.mac_stats.sequence_errors: 0 > dev.em.2.mac_stats.defer_count: 0 > dev.em.2.mac_stats.missed_packets: 0 > dev.em.2.mac_stats.recv_no_buff: 0 > dev.em.2.mac_stats.recv_undersize: 0 > dev.em.2.mac_stats.recv_fragmented: 0 > dev.em.2.mac_stats.recv_oversize: 0 > dev.em.2.mac_stats.recv_jabber: 0 > dev.em.2.mac_stats.recv_errs: 0 > dev.em.2.mac_stats.crc_errs: 0 > dev.em.2.mac_stats.alignment_errs: 0 > dev.em.2.mac_stats.coll_ext_errs: 0 > dev.em.2.mac_stats.xon_recvd: 14123 > dev.em.2.mac_stats.xon_txd: 1 > dev.em.2.mac_stats.xoff_recvd: 14127 > dev.em.2.mac_stats.xoff_txd: 1 > dev.em.2.mac_stats.total_pkts_recvd: 229919303 > dev.em.2.mac_stats.good_pkts_recvd: 229891053 > dev.em.2.mac_stats.bcast_pkts_recvd: 909450 > dev.em.2.mac_stats.mcast_pkts_recvd: 19452 > dev.em.2.mac_stats.rx_frames_64: 1477808 > dev.em.2.mac_stats.rx_frames_65_127: 195114744 > dev.em.2.mac_stats.rx_frames_128_255: 6579690 > dev.em.2.mac_stats.rx_frames_256_511: 5137387 > dev.em.2.mac_stats.rx_frames_512_1023: 4223090 > dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 > dev.em.2.mac_stats.good_octets_recvd: 46129102134 > dev.em.2.mac_stats.good_octets_txd: 419293159496 > dev.em.2.mac_stats.total_pkts_txd: 332661584 > dev.em.2.mac_stats.good_pkts_txd: 332661582 > dev.em.2.mac_stats.bcast_pkts_txd: 48506 > dev.em.2.mac_stats.mcast_pkts_txd: 78 > dev.em.2.mac_stats.tx_frames_64: 14598198 > dev.em.2.mac_stats.tx_frames_65_127: 22287108 > dev.em.2.mac_stats.tx_frames_128_255: 8897511 > dev.em.2.mac_stats.tx_frames_256_511: 9623000 > dev.em.2.mac_stats.tx_frames_512_1023: 8325033 > dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 > dev.em.2.mac_stats.tso_txd: 24357891 > dev.em.2.mac_stats.tso_ctx_fail: 0 > dev.em.2.interrupts.asserts: 2 > dev.em.2.interrupts.rx_pkt_timer: 0 > dev.em.2.interrupts.rx_abs_timer: 0 > dev.em.2.interrupts.tx_pkt_timer: 0 > dev.em.2.interrupts.tx_abs_timer: 0 > dev.em.2.interrupts.tx_queue_empty: 0 > dev.em.2.interrupts.tx_queue_min_thresh: 0 > dev.em.2.interrupts.rx_desc_min_thresh: 0 > dev.em.2.interrupts.rx_overrun: 0 > dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.3.%driver: em > dev.em.3.%location: slot=0 function=0 > dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.3.%parent: pci8 > dev.em.3.nvm: -1 > dev.em.3.debug: -1 > dev.em.3.fc: 3 > dev.em.3.rx_int_delay: 0 > dev.em.3.tx_int_delay: 66 > dev.em.3.rx_abs_int_delay: 66 > dev.em.3.tx_abs_int_delay: 66 > dev.em.3.itr: 488 > dev.em.3.rx_processing_limit: -1 > dev.em.3.eee_control: 1 > dev.em.3.link_irq: 0 > dev.em.3.mbuf_alloc_fail: 0 > dev.em.3.cluster_alloc_fail: 0 > dev.em.3.dropped: 0 > dev.em.3.tx_dma_fail: 0 > dev.em.3.rx_overruns: 0 > dev.em.3.watchdog_timeouts: 0 > dev.em.3.device_control: 1074790984 > dev.em.3.rx_control: 67141634 > dev.em.3.fc_high_water: 18432 > dev.em.3.fc_low_water: 16932 > dev.em.3.queue0.txd_head: 0 > dev.em.3.queue0.txd_tail: 0 > dev.em.3.queue0.tx_irq: 0 > dev.em.3.queue0.no_desc_avail: 0 > dev.em.3.queue0.rxd_head: 0 > dev.em.3.queue0.rxd_tail: 4095 > dev.em.3.queue0.rx_irq: 0 > dev.em.3.mac_stats.excess_coll: 0 > dev.em.3.mac_stats.single_coll: 0 > dev.em.3.mac_stats.multiple_coll: 0 > dev.em.3.mac_stats.late_coll: 0 > dev.em.3.mac_stats.collision_count: 0 > dev.em.3.mac_stats.symbol_errors: 0 > dev.em.3.mac_stats.sequence_errors: 0 > dev.em.3.mac_stats.defer_count: 0 > dev.em.3.mac_stats.missed_packets: 0 > dev.em.3.mac_stats.recv_no_buff: 0 > dev.em.3.mac_stats.recv_undersize: 0 > dev.em.3.mac_stats.recv_fragmented: 0 > dev.em.3.mac_stats.recv_oversize: 0 > dev.em.3.mac_stats.recv_jabber: 0 > dev.em.3.mac_stats.recv_errs: 0 > dev.em.3.mac_stats.crc_errs: 0 > dev.em.3.mac_stats.alignment_errs: 0 > dev.em.3.mac_stats.coll_ext_errs: 0 > dev.em.3.mac_stats.xon_recvd: 0 > dev.em.3.mac_stats.xon_txd: 0 > dev.em.3.mac_stats.xoff_recvd: 0 > dev.em.3.mac_stats.xoff_txd: 0 > dev.em.3.mac_stats.total_pkts_recvd: 0 > dev.em.3.mac_stats.good_pkts_recvd: 0 > dev.em.3.mac_stats.bcast_pkts_recvd: 0 > dev.em.3.mac_stats.mcast_pkts_recvd: 0 > dev.em.3.mac_stats.rx_frames_64: 0 > dev.em.3.mac_stats.rx_frames_65_127: 0 > dev.em.3.mac_stats.rx_frames_128_255: 0 > dev.em.3.mac_stats.rx_frames_256_511: 0 > dev.em.3.mac_stats.rx_frames_512_1023: 0 > dev.em.3.mac_stats.rx_frames_1024_1522: 0 > dev.em.3.mac_stats.good_octets_recvd: 0 > dev.em.3.mac_stats.good_octets_txd: 0 > dev.em.3.mac_stats.total_pkts_txd: 0 > dev.em.3.mac_stats.good_pkts_txd: 0 > dev.em.3.mac_stats.bcast_pkts_txd: 0 > dev.em.3.mac_stats.mcast_pkts_txd: 0 > dev.em.3.mac_stats.tx_frames_64: 0 > dev.em.3.mac_stats.tx_frames_65_127: 0 > dev.em.3.mac_stats.tx_frames_128_255: 0 > dev.em.3.mac_stats.tx_frames_256_511: 0 > dev.em.3.mac_stats.tx_frames_512_1023: 0 > dev.em.3.mac_stats.tx_frames_1024_1522: 0 > dev.em.3.mac_stats.tso_txd: 0 > dev.em.3.mac_stats.tso_ctx_fail: 0 > dev.em.3.interrupts.asserts: 0 > dev.em.3.interrupts.rx_pkt_timer: 0 > dev.em.3.interrupts.rx_abs_timer: 0 > dev.em.3.interrupts.tx_pkt_timer: 0 > dev.em.3.interrupts.tx_abs_timer: 0 > dev.em.3.interrupts.tx_queue_empty: 0 > dev.em.3.interrupts.tx_queue_min_thresh: 0 > dev.em.3.interrupts.rx_desc_min_thresh: 0 > dev.em.3.interrupts.rx_overrun: 0 > > > .............................................................................. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Ermal From owner-freebsd-net@FreeBSD.ORG Sat May 3 08:30:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D0D32BA; Sat, 3 May 2014 08:30:29 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 61241135C; Sat, 3 May 2014 08:30:29 +0000 (UTC) Received: from [192.168.1.200] (p548196A1.dip0.t-ipconnect.de [84.129.150.161]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 0412D1C10492E; Sat, 3 May 2014 10:30:24 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: Date: Sat, 3 May 2014 10:30:23 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 08:30:29 -0000 On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >=20 > On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >=20 >> Dear all, >>=20 >> during testing I found that FreeBSD head (on a raspberry pi) accepts = SCTP packet >> with bad checksums. After debugging this I figured out that this is a = problem with >> the csum_flags defined in mbuf.h. >>=20 >> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is = defined in mbuf.h: >> #define CSUM_SCTP_VALID CSUM_L4_VALID >> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet = is considered >> to have a correct checksum. >>=20 >> For UDP and TCP some drivers calculate the UDP/TCP checksum and set = CSUM_DATA_VALID in >> csum_flags to indicate that the UDP/TCP should consider csum_data to = figure out if >> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >> #define CSUM_DATA_VALID CSUM_L4_VALID >> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >> field contains information. >>=20 >> Now the following happens (on the raspberry pi the driver used is >> dev/usb/net/if_smsc.c >>=20 >> 1. A packet is received and if it is not too short, the checksum = computed >> is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens >> for all IP packets, not only for UDP and TCP packets. >> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as = CSUM_SCTP_VALID >> and accepts the packet. So no SCTP checksum check ever happened. >>=20 >> Alternatives to fix this: >>=20 >> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or = TCP packets, since >> it only makes sense in these cases. >=20 > Wait, or for SCTP in cad the crc32 (I think it was) was actually = checked but not otherwise. This is how it should be imho. It seems = like a driver bug. I'm not sure what you want to say with the first sentence. However, SCTP uses a CRC32C and most of the NICs don't support it (some = do and they do it right, as far as I know). I can go through the drivers listed in http://fxr.watson.org/fxr/ident?i=3DCSUM_DATA_VALID and make sure they only set CSUM_DATA_VALID for UDP and TCP packets... Best regards Michael >=20 >=20 > =97=20 > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, = 1983 >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sat May 3 09:52:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5451CC78; Sat, 3 May 2014 09:52:52 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 178331A9D; Sat, 3 May 2014 09:52:52 +0000 (UTC) Received: from [192.168.1.200] (p548196A1.dip0.t-ipconnect.de [84.129.150.161]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 935D41C104914; Sat, 3 May 2014 11:52:49 +0200 (CEST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: Date: Sat, 3 May 2014 11:52:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 09:52:52 -0000 On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >=20 > On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >=20 >> Dear all, >>=20 >> during testing I found that FreeBSD head (on a raspberry pi) accepts = SCTP packet >> with bad checksums. After debugging this I figured out that this is a = problem with >> the csum_flags defined in mbuf.h. >>=20 >> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is = defined in mbuf.h: >> #define CSUM_SCTP_VALID CSUM_L4_VALID >> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet = is considered >> to have a correct checksum. >>=20 >> For UDP and TCP some drivers calculate the UDP/TCP checksum and set = CSUM_DATA_VALID in >> csum_flags to indicate that the UDP/TCP should consider csum_data to = figure out if >> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >> #define CSUM_DATA_VALID CSUM_L4_VALID >> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >> field contains information. >>=20 >> Now the following happens (on the raspberry pi the driver used is >> dev/usb/net/if_smsc.c >>=20 >> 1. A packet is received and if it is not too short, the checksum = computed >> is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens >> for all IP packets, not only for UDP and TCP packets. >> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as = CSUM_SCTP_VALID >> and accepts the packet. So no SCTP checksum check ever happened. >>=20 >> Alternatives to fix this: >>=20 >> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or = TCP packets, since >> it only makes sense in these cases. >=20 > Wait, or for SCTP in cad the crc32 (I think it was) was actually = checked but not otherwise. This is how it should be imho. It seems = like a driver bug. I went through the list of drivers and you are right, it seems to be a = bug in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I = can't tell. Best regards Michael=20 >=20 >=20 > =97=20 > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, = 1983 >=20 >=20 From owner-freebsd-net@FreeBSD.ORG Sat May 3 20:47:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E2AF374 for ; Sat, 3 May 2014 20:47:32 +0000 (UTC) Received: from web01.jbserver.net (web01.jbserver.net [IPv6:2a00:8240:6:a::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 419451435 for ; Sat, 3 May 2014 20:47:31 +0000 (UTC) Received: from [187.185.71.234] (helo=[172.17.65.68]) by web01.jbserver.net with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WggqF-0001gZ-ER; Sat, 03 May 2014 22:47:28 +0200 Message-ID: <5364C049.8020302@gont.com.ar> Date: Sat, 03 May 2014 05:09:13 -0500 From: Fernando Gont User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Fwd: RFC 7217 on A Method for Generating Semantically Opaque Interface Identifiers with IPv6 Stateless Address Autoconfiguration (SLAAC) References: <20140430210539.E0D4318000D@rfc-editor.org> In-Reply-To: <20140430210539.E0D4318000D@rfc-editor.org> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 20:47:32 -0000 Folks, FYI, RFC 7217 has been published: It is meant to address the issues discussed in: and Thanks! Best regards, Fernando -------- Original Message --------Subject: RFC 7217 on A Method for Generating Semantically Opaque Interface Identifiers with IPv6 Stateless Address Autoconfiguration (SLAAC) Date: Wed, 30 Apr 2014 14:05:39 -0700 (PDT) From: rfc-editor@rfc-editor.org Reply-To: ietf@ietf.org To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org CC: drafts-update-ref@iana.org, ipv6@ietf.org, rfc-editor@rfc-editor.org A new Request for Comments is now available in online RFC libraries. RFC 7217 Title: A Method for Generating Semantically Opaque Interface Identifiers with IPv6 Stateless Address Autoconfiguration (SLAAC) Author: F. Gont Status: Standards Track Stream: IETF Date: April 2014 Mailbox: fgont@si6networks.com Pages: 19 Characters: 48497 Updates/Obsoletes/SeeAlso: None I-D Tag: draft-ietf-6man-stable-privacy-addresses-17.txt URL: http://www.rfc-editor.org/rfc/rfc7217.txt This document specifies a method for generating IPv6 Interface Identifiers to be used with IPv6 Stateless Address Autoconfiguration (SLAAC), such that an IPv6 address configured using this method is stable within each subnet, but the corresponding Interface Identifier changes when the host moves from one network to another. This method is meant to be an alternative to generating Interface Identifiers based on hardware addresses (e.g., IEEE LAN Media Access Control (MAC) addresses), such that the benefits of stable addresses can be achieved without sacrificing the security and privacy of users. The method specified in this document applies to all prefixes a host may be employing, including link-local, global, and unique-local prefixes (and their corresponding addresses). This document is a product of the IPv6 Maintenance Working Group of the IETF. This is now a Proposed Standard. STANDARDS TRACK: This document specifies an Internet standards track protocol for the Internet community,and requests discussion and suggestions for improvements. Please refer to the current edition of the Internet Official Protocol Standards (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see http://www.ietf.org/mailman/listinfo/ietf-announce http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see http://www.rfc-editor.org/search For downloading RFCs, see http://www.rfc-editor.org/rfc.html Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC -- Fernando Gont e-mail: fernando@gont.com.ar || fgont@si6networks.com PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1 -- Fernando Gont e-mail: fernando@gont.com.ar || fgont@si6networks.com PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1 From owner-freebsd-net@FreeBSD.ORG Sat May 3 21:46:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9444FFF for ; Sat, 3 May 2014 21:46:39 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BB091B15 for ; Sat, 3 May 2014 21:46:38 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 2F42F25D3810; Sat, 3 May 2014 21:46:29 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 1CF79C22BEC; Sat, 3 May 2014 21:46:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id AXwBVwa0Fzdw; Sat, 3 May 2014 21:46:27 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (unknown [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 0CE16C22BD1; Sat, 3 May 2014 21:46:24 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RFC 7217 on A Method for Generating Semantically Opaque Interface Identifiers with IPv6 Stateless Address Autoconfiguration (SLAAC) From: "Bjoern A. Zeeb" In-Reply-To: <5364C049.8020302@gont.com.ar> Date: Sat, 3 May 2014 21:46:22 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140430210539.E0D4318000D@rfc-editor.org> <5364C049.8020302@gont.com.ar> To: Fernando Gont X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 21:46:39 -0000 On 03 May 2014, at 10:09 , Fernando Gont wrote: > Folks, >=20 > FYI, RFC 7217 has been published: > >=20 > It is meant to address the issues discussed in: > and > = you forgot to attach the source code path for the implementation. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Sun May 4 02:29:08 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD28A227; Sun, 4 May 2014 02:29:08 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9106B1063; Sun, 4 May 2014 02:29:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s442T8Xs027037; Sun, 4 May 2014 02:29:08 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s442T8jL027036; Sun, 4 May 2014 02:29:08 GMT (envelope-from linimon) Date: Sun, 4 May 2014 02:29:08 GMT Message-Id: <201405040229.s442T8jL027036@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/189219: [dummynet] [patch] using dummynet on sparc64 and configuring a pipe is an insta-panic X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 02:29:08 -0000 Old Synopsis: using dummynet on sparc64 and configuring a pipe is an insta-panic New Synopsis: [dummynet] [patch] using dummynet on sparc64 and configuring a pipe is an insta-panic Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 02:28:32 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=189219 From owner-freebsd-net@FreeBSD.ORG Sun May 4 04:07:36 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 801DB9D1; Sun, 4 May 2014 04:07:36 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5481D18D5; Sun, 4 May 2014 04:07:36 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4447aQZ065499; Sun, 4 May 2014 04:07:36 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4447a9R065498; Sun, 4 May 2014 04:07:36 GMT (envelope-from linimon) Date: Sun, 4 May 2014 04:07:36 GMT Message-Id: <201405040407.s4447a9R065498@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/171550: [ndis] [patch] "no match for InterlockedCompareExchange" and then kernel panic upon kldload ndisgen module for RTL8188/8192CU X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 04:07:36 -0000 Old Synopsis: patch: "no match for InterlockedCompareExchange" and then kernel panic upon kldload ndisgen module for RTL8188/8192CU New Synopsis: [ndis] [patch] "no match for InterlockedCompareExchange" and then kernel panic upon kldload ndisgen module for RTL8188/8192CU Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 04:06:07 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=171550 From owner-freebsd-net@FreeBSD.ORG Sun May 4 04:51:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20C11418; Sun, 4 May 2014 04:51:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E248C1C1F; Sun, 4 May 2014 04:51:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s444p0e3083652; Sun, 4 May 2014 04:51:00 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s444p0CO083651; Sun, 4 May 2014 04:51:00 GMT (envelope-from linimon) Date: Sun, 4 May 2014 04:51:00 GMT Message-Id: <201405040451.s444p0CO083651@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-net@FreeBSD.org, freebsd-pf@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/169620: [ng] [pf] ng_l2tp incoming packet bypass pf firewall X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 04:51:01 -0000 Synopsis: [ng] [pf] ng_l2tp incoming packet bypass pf firewall Responsible-Changed-From-To: freebsd-net->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 04:50:27 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=169620 From owner-freebsd-net@FreeBSD.ORG Sun May 4 04:56:04 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F02F693; Sun, 4 May 2014 04:56:04 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5FDA81C4B; Sun, 4 May 2014 04:56:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s444u43d084154; Sun, 4 May 2014 04:56:04 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s444u4u9084153; Sun, 4 May 2014 04:56:04 GMT (envelope-from linimon) Date: Sun, 4 May 2014 04:56:04 GMT Message-Id: <201405040456.s444u4u9084153@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/186304: [bmc] watchdog service causes BMC controller reset every 20-30 minutes on Supermicro X9DRW-3F X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 04:56:04 -0000 Old Synopsis: watchdog service causes BMC controller reset every 20-30 minutes on Supermicro X9DRW-3F New Synopsis: [bmc] watchdog service causes BMC controller reset every 20-30 minutes on Supermicro X9DRW-3F Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 04:55:48 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=186304 From owner-freebsd-net@FreeBSD.ORG Sun May 4 05:15:23 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A886E909; Sun, 4 May 2014 05:15:23 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D9F91D9D; Sun, 4 May 2014 05:15:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s445FNRT092451; Sun, 4 May 2014 05:15:23 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s445FNxp092450; Sun, 4 May 2014 05:15:23 GMT (envelope-from linimon) Date: Sun, 4 May 2014 05:15:23 GMT Message-Id: <201405040515.s445FNxp092450@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/184389: [libalias] libalias fails to adjust MTU from jails X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 05:15:23 -0000 Old Synopsis: libalias fails to adjust MTU from jails New Synopsis: [libalias] libalias fails to adjust MTU from jails Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 05:14:51 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=184389 From owner-freebsd-net@FreeBSD.ORG Sun May 4 05:22:46 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28CDAC78; Sun, 4 May 2014 05:22:46 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F16FA1FAA; Sun, 4 May 2014 05:22:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s445MjPU095864; Sun, 4 May 2014 05:22:45 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s445MjAd095863; Sun, 4 May 2014 05:22:45 GMT (envelope-from linimon) Date: Sun, 4 May 2014 05:22:45 GMT Message-Id: <201405040522.s445MjAd095863@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/183920: [ixgbe] [patch] Incorrect ifconfig media on INTEL X520-T2 10G Dual-port Ethernet Server Adapter, RJ45/2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 05:22:46 -0000 Old Synopsis: Incorrect ifconfig media on INTEL X520-T2 10G Dual-port Ethernet Server Adapter, RJ45/2 New Synopsis: [ixgbe] [patch] Incorrect ifconfig media on INTEL X520-T2 10G Dual-port Ethernet Server Adapter, RJ45/2 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 05:21:46 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=183920 From owner-freebsd-net@FreeBSD.ORG Sun May 4 05:25:20 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A817D69; Sun, 4 May 2014 05:25:20 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F3AE1FBC; Sun, 4 May 2014 05:25:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s445PKfm096091; Sun, 4 May 2014 05:25:20 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s445PK0t096090; Sun, 4 May 2014 05:25:20 GMT (envelope-from linimon) Date: Sun, 4 May 2014 05:25:20 GMT Message-Id: <201405040525.s445PK0t096090@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/183381: [em] [patch] Use of 9k buffers in if_em.c hangs with resource starvation X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 05:25:20 -0000 Old Synopsis: Use of 9k buffers in if_em.c hangs with resource starvation New Synopsis: [em] [patch] Use of 9k buffers in if_em.c hangs with resource starvation Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 05:24:11 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=183381 From owner-freebsd-net@FreeBSD.ORG Sun May 4 05:26:15 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C28F1E42; Sun, 4 May 2014 05:26:15 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97D151FCB; Sun, 4 May 2014 05:26:15 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s445QFnl096191; Sun, 4 May 2014 05:26:15 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s445QFv2096190; Sun, 4 May 2014 05:26:15 GMT (envelope-from linimon) Date: Sun, 4 May 2014 05:26:15 GMT Message-Id: <201405040526.s445QFv2096190@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/183271: [em] statistic not updated on em in netmap mode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 05:26:15 -0000 Old Synopsis: statistic not updated on em in netmap mode New Synopsis: [em] statistic not updated on em in netmap mode Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 05:25:38 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=183271 From owner-freebsd-net@FreeBSD.ORG Sun May 4 05:36:04 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96AD6226; Sun, 4 May 2014 05:36:04 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 68EE710B0; Sun, 4 May 2014 05:36:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s445a4PH099815; Sun, 4 May 2014 05:36:04 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s445a4r4099813; Sun, 4 May 2014 05:36:04 GMT (envelope-from linimon) Date: Sun, 4 May 2014 05:36:04 GMT Message-Id: <201405040536.s445a4r4099813@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/179473: [new driver] if_vether.c: Source code contribution of implementation about virtual ethernet interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 05:36:04 -0000 Old Synopsis: Source code contribution of implementation about virtual ethernet interface New Synopsis: [new driver] if_vether.c: Source code contribution of implementation about virtual ethernet interface Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun May 4 05:34:19 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=179473 From owner-freebsd-net@FreeBSD.ORG Sun May 4 06:18:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF9BE625 for ; Sun, 4 May 2014 06:18:51 +0000 (UTC) Received: from mail-oa0-x243.google.com (mail-oa0-x243.google.com [IPv6:2607:f8b0:4003:c02::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8506131A for ; Sun, 4 May 2014 06:18:51 +0000 (UTC) Received: by mail-oa0-f67.google.com with SMTP id m1so1335532oag.2 for ; Sat, 03 May 2014 23:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=PNY34tiiixPd6807V2Hy6CMnUUlwU0RTxBP/3p/nGOA=; b=kUbJnICyhfDZNolC8ahQgVnPK3Lrr9MK6s//bNkYvNKWXofAr+Vt1uKCKcN47QaOlD OzLRpClL8Ax8p6BVs85mNrvn5SHmPN4p1kNdHYtUUL3vNHey76tB3tcfpN8MLoxk1o/M MrLZqJAYAlR8Le5N3LUzdHlMhAkX8oNuT3eQ6io2wAKIkUG26NctzalG278MDFFJWLjC z3SgWwPn7NGp1MROaF6nWII1m+Wv8WHve4+Oy4jEBJ2d23G5sFe71EI4e5hcqx0gZeIM QZ6u03Yt01Ny82iAKFohoorjh9zz2C36keXBi2GuLoOp1HTZbsIGvgfFy6wf4yIlqG/A XUAA== MIME-Version: 1.0 X-Received: by 10.182.128.36 with SMTP id nl4mr58519obb.63.1399184331020; Sat, 03 May 2014 23:18:51 -0700 (PDT) Received: by 10.76.101.18 with HTTP; Sat, 3 May 2014 23:18:50 -0700 (PDT) In-Reply-To: References: Date: Sun, 4 May 2014 14:18:50 +0800 Message-ID: Subject: Re: netmap: how to bridge 2 eth? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 06:18:51 -0000 add: tried ./vale-ctl -h vale0:eth0 ./vale-ctl -h vale0:eth1 and pinging get error: May 4 22:02:01 ubuntu kernel: [ 854.827834] 121.198610 [ 425] netmap_update_config configuration changed (but fine) May 4 22:02:01 ubuntu kernel: [ 854.828080] 121.198860 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 May 4 22:02:01 ubuntu kernel: [ 854.828330] 121.199111 [1049] netmap_mem_global_config reconfiguring May 4 22:02:14 ubuntu kernel: [ 868.034578] 134.402766 [ 425] netmap_update_config configuration changed (but fine) May 4 22:02:14 ubuntu kernel: [ 868.034818] 134.403019 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 May 4 22:02:44 ubuntu kernel: [ 897.779633] 164.142010 [1298] nm_txsync_prologue eth0 TX1 kring error: hwcur 0 rcur 0 hwtail 511 cur 1 tail 511 May 4 22:02:44 ubuntu kernel: [ 897.780009] 164.142403 [1298] nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 cur 0 tail 511 May 4 22:02:44 ubuntu kernel: [ 897.780284] 164.142679 [1373] nm_rxsync_prologue kring error: hwcur 0 rcur 0 hwtail 0 head 512 cur 0 tail 0 May 4 22:02:44 ubuntu kernel: [ 897.780563] 164.142944 [1549] netmap_vp_rxsync_locked ouch dangerous reset!!! May 4 22:02:44 ubuntu kernel: [ 897.780782] 164.143178 [1398] netmap_ring_reinit called for vale0:eth1 May 4 22:02:44 ubuntu kernel: [ 897.780969] 164.143365 [1423] netmap_ring_reinit total 1 errors May 4 22:02:44 ubuntu kernel: [ 897.781145] 164.143541 [1427] netmap_ring_reinit vale0:eth1 RX0 reinit, cur 0 -> 0 tail 0 -> 0 May 4 22:02:44 ubuntu kernel: [ 897.781393] 164.143789 [1298] nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 cur 1 tail 511 May 4 22:02:44 ubuntu kernel: [ 897.781664] 164.144060 [1373] nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur 1 tail 1 May 4 22:02:44 ubuntu kernel: [ 897.781947] 164.144343 [1549] netmap_vp_rxsync_locked ouch dangerous reset!!! May 4 22:02:44 ubuntu kernel: [ 897.782165] 164.144561 [1398] netmap_ring_reinit called for vale0:eth1 May 4 22:02:44 ubuntu kernel: [ 897.782355] 164.144751 [1423] netmap_ring_reinit total 1 errors May 4 22:02:44 ubuntu kernel: [ 897.782533] 164.144929 [1427] netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 May 4 22:02:44 ubuntu kernel: [ 897.782783] 164.145178 [1298] nm_txsync_prologue eth1 TX1 kring error: hwcur 0 rcur 0 hwtail 511 cur 1 tail 511 May 4 22:02:45 ubuntu kernel: [ 898.776184] 165.138378 [1298] nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 cur 1 tail 511 May 4 22:02:45 ubuntu kernel: [ 898.776493] 165.138694 [1373] nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur 1 tail 1 May 4 22:02:45 ubuntu kernel: [ 898.776765] 165.138966 [1549] netmap_vp_rxsync_locked ouch dangerous reset!!! May 4 22:02:45 ubuntu kernel: [ 898.776987] 165.139189 [1398] netmap_ring_reinit called for vale0:eth1 May 4 22:02:45 ubuntu kernel: [ 898.777177] 165.139379 [1423] netmap_ring_reinit total 1 errors May 4 22:02:45 ubuntu kernel: [ 898.777357] 165.139558 [1427] netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 May 4 22:02:45 ubuntu kernel: [ 898.777624] 165.139825 [1298] nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 cur 2 tail 511 May 4 22:02:45 ubuntu kernel: [ 898.777896] 165.140097 [1373] nm_rxsync_prologue kring error: hwcur 0 rcur 2 hwtail 2 head 512 cur 2 tail 2 May 4 22:02:45 ubuntu kernel: [ 898.778159] 165.140361 [1549] netmap_vp_rxsync_locked ouch dangerous reset!!! then finally kernel panic/crash... 2014-05-03 14:44 GMT+08:00 upyzl : > Hi all > > I'm doing to implement a simple openflow-based software switch(not same as > OVS, much simpler to OVS) > and I choose netmap for the network framework > > Now I try to do a simplest thing: > > there's 3 VMs, all are Ubuntu 12.04.4 x64 > 2 act as hosts, other 1 act as switch > > topo: [host1] eth0 ----- eth0 [switch] eth1 ----- eth0 [host2] > > I want to bridge switch's eth0ð1 (like "brctl" in linux), then host1 > ping host2 > > the problem is, how to bridge using netmap? > > I tried example "bridge" in netmap project(git clone > https://code.google.com/p/netmap/): > ./bridge -i netmap:eth0 -i netmap:eth1 > then host1 ping to host2 > but pinging result is "Destination Host Unreachable" (if using brctl, > pinging is fine) > > Then I tried vale-ctl > but i dont know how to use it... > e.g. > ./vale-ctl -a eth0 > show > "eth0: Invalid argument" > > could anyone help me? > From owner-freebsd-net@FreeBSD.ORG Sun May 4 15:25:09 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A27CBA5 for ; Sun, 4 May 2014 15:25:09 +0000 (UTC) Received: from nm1-vm1.bullet.mail.bf1.yahoo.com (nm1-vm1.bullet.mail.bf1.yahoo.com [98.139.213.163]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 382EB1DB0 for ; Sun, 4 May 2014 15:25:08 +0000 (UTC) Received: from [66.196.81.170] by nm1.bullet.mail.bf1.yahoo.com with NNFMP; 04 May 2014 15:21:56 -0000 Received: from [98.139.212.249] by tm16.bullet.mail.bf1.yahoo.com with NNFMP; 04 May 2014 15:21:56 -0000 Received: from [127.0.0.1] by omp1058.mail.bf1.yahoo.com with NNFMP; 04 May 2014 15:21:56 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 550982.81243.bm@omp1058.mail.bf1.yahoo.com Received: (qmail 67038 invoked by uid 60001); 4 May 2014 15:21:56 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1399216916; bh=Bzj8j3QJ9rPOnzgl2TAZgfEZ9ciQa+A39+KxrpCX4c8=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=TEG4U6a5oY8gYJcJoIrcDnshG7NcVCVf34CPIwOEsdqQV1brtVniTWtQOi6MNpPKtwB+J/CSbXOJ3J4+zo6U50EwgdjyK/EgVa3CN42sk0cDAhFWS9VY/B1PiM77qbLl603zK4rCHPu90pkwGGfNV8pFJOsYDvouWhNtHOV0qE0= X-YMail-OSG: ZS9uDhsVM1m1C6bYt4LALSUlshb0Z6MaEHLFYWSmdXnpCqB fU7AGVSd3iXFkX2mqpDWQAm0dXBi8MWl5Y4YCLYzoFZ08Adcq9PRbfg3OWmb g6fAVzoSBvjnrlsnjjv0cy0cwPmDoTvtolIzQ8_WyJU7YpB8Km_d8apiJYM4 KDfkRzdZX3g7mx.YsCvdXphJ8w53Y_1WFejQukjL7iE4T2gk2RLlmlp8dl.s 8aYZ4ml8cDDgZ6CLSB8pE36O33XJsJTbWwfRz4dUdtbgJl4KpjNbPYLGGlO6 4AE.xHwgOLjLKqHMDusn6b1c8.agGss_PyKBnPMtnYSS9fX1L.CHt7IXPTNv l.gPDBSt01PllDxa0rdoNhfz6pfE6M0zTvUBdJrRmjEADP5568A73Atbjl3_ QoF.hyEbo84R3aXab4Gw8Yogo6Lb4S4fJt9AlnNQ_fn_nz8LnXgpJzqpIgUW 5eyg7CFxWkBQn30yZXBHWqohV6Fzhm0EfHBE0hsl9Msl4p_aitlZM13d07mY QYCcAkkbneKGZlhVw013kxUtj48lqfaU4R4Xp6TkrqWAhTt3kEi5jI2WmY00 - Received: from [39.32.238.171] by web162704.mail.bf1.yahoo.com via HTTP; Sun, 04 May 2014 08:21:56 PDT X-Rocket-MIMEInfo: 002.001, SEksCgoJSSB3YW50IHRvIGFzayBob3cgY2FuIGkgaW1wbGVtZW50IGJhdGNoIHByb2Nlc3NpbmcgdXNpbmcgcG9sbCgpOwoJaG93IHdlIHRlbGwgTklDIGFib3V0IGJhdGNoZXMuwqAKwqAgwqAgwqAgwqAgQWxzbyBoYXZlIHlvdSBpbXBsZW1lbnRlZCBCYXRjaCBwcm9jZXNzaW5nIGluIHBrdC1nZW4uYyBpbiB0aGUgRXhhbXBsZSBEaXJlY3RvcnkuwqAKwqAgwqAgwqAgwqAgSWYgeWVzIGhvdyBjYW4gaSBjb250cm9sIGJhdGNoIHNpemUgaW4gcGt0LWdlbi5jLgoKcmVnYXJkcwptYXRpIHVyIHJhaG1hbsKgATABAQEB X-Mailer: YahooMailWebService/0.8.188.663 Message-ID: <1399216916.47516.YahooMailNeo@web162704.mail.bf1.yahoo.com> Date: Sun, 4 May 2014 08:21:56 -0700 (PDT) From: mati ur-rahman Subject: Batch processing To: "net@freebsd.org" MIME-Version: 1.0 X-Mailman-Approved-At: Sun, 04 May 2014 16:37:53 +0000 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: mati ur-rahman List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 15:25:09 -0000 HI,=0A=0A=09I want to ask how can i implement batch processing using poll()= ;=0A=09how we tell NIC about batches.=A0=0A=A0 =A0 =A0 =A0 Also have you im= plemented Batch processing in pkt-gen.c in the Example Directory.=A0=0A=A0 = =A0 =A0 =A0 If yes how can i control batch size in pkt-gen.c.=0A=0Aregards= =0Amati ur rahman=A0 From owner-freebsd-net@FreeBSD.ORG Sun May 4 19:27:51 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37B09BA9 for ; Sun, 4 May 2014 19:27:51 +0000 (UTC) Received: from nm22-vm1.bullet.mail.bf1.yahoo.com (nm22-vm1.bullet.mail.bf1.yahoo.com [98.139.212.127]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DBB3B16D7 for ; Sun, 4 May 2014 19:27:50 +0000 (UTC) Received: from [98.139.212.153] by nm22.bullet.mail.bf1.yahoo.com with NNFMP; 04 May 2014 19:27:43 -0000 Received: from [98.139.212.247] by tm10.bullet.mail.bf1.yahoo.com with NNFMP; 04 May 2014 19:27:43 -0000 Received: from [127.0.0.1] by omp1056.mail.bf1.yahoo.com with NNFMP; 04 May 2014 19:27:43 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 342761.90602.bm@omp1056.mail.bf1.yahoo.com Received: (qmail 70320 invoked by uid 60001); 4 May 2014 19:27:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1399231662; bh=CAwCFzV+ksBhREBA6FwYtn9OQsF+TYrcw7K8CDExeiQ=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=nBWtcErXi5dFpfITrKK0ZBOMFE2dmuGknTVWz5CdE4KcMmLkic0PWO6eo05WiY7g6vMacg+cjTryPmfmga5hSzt/t8qxQGTwej6rUCfcb/ZcdBFfI9CR95aI6DPmSG/WYmskxYV/1oH6b57BgQs4BWEdmjUWaxuw31mBlF5Kpxg= X-YMail-OSG: bHCYKskVM1kh44u0SkSbYrsXHQuUgChPON8arcHBWSiOoFV f37pNaAR36BPPelUpiOv.GjR4D.9_ibHQyHx8b17oB3ItFYpJov6vqkhudpj BIjMOQuZhAfsblI2vXBGv_I0Xp5w8C7e4FYATbKHA_d1XYkXbKvRekOSIKh7 lM3G0rfapVwWO5Iohpyr7bWz2WFoHonrIDZ3XK.mT4augzmrZNKEv6Y9gFI3 kiJBYinhhCyne66oxIh0j9s_D078hWEJxTfBA.Vi3fr.i4fLVXvyeYZZ9J8T AjS1EHNCuR_XArPtNsZoIyrNNq1AUhpFf9E7NRk9fgPBeV0wDAdsBgbzQh2H iDMyD8L8TBrpk6wLEiG3YWBWCpXiHTaKIJH4PdHK1d3PxwAsDjL2Z2F66zqn tQTvt.G5AgQwaVKiQfmFwnFK6vWJzFJoXzxNEME6fRSrJsxjwiGkS3x_4SoE olZqCgcy8o49t8nqZnvlr8i0g_.UQh0MYV2Yi36EAkX4BJ08KOqZBJfzmPOE uwuZT7jbO1frZejFq2g2to4kdjIhhC1mTXGgOazLmMGS.JW.TzCRxsaGy0gg - Received: from [39.32.238.171] by web162702.mail.bf1.yahoo.com via HTTP; Sun, 04 May 2014 12:27:42 PDT X-Rocket-MIMEInfo: 002.001, SEksCgpJIHdhbnQgdG8gYXNrIGhvdyBjYW4gaSBpbXBsZW1lbnQgYmF0Y2ggcHJvY2Vzc2luZyB1c2luZyBwb2xsKCk7CmhvdyB3ZSB0ZWxsIE5JQyBhYm91dCBiYXRjaGVzLsKgCsKgIMKgIMKgIMKgIEFsc28gaGF2ZSB5b3UgaW1wbGVtZW50ZWQgQmF0Y2ggcHJvY2Vzc2luZyBpbiBwa3QtZ2VuLmMgaW4gdGhlIEV4YW1wbGUgRGlyZWN0b3J5LsKgCsKgIMKgIMKgIMKgIElmIHllcyBob3cgY2FuIGkgY29udHJvbCBiYXRjaCBzaXplIGluIHBrdC1nZW4uYy4KCnJlZ2FyZHMKbWF0aSB1ciByYWhtYW7CoAEwAQEBAQ-- X-Mailer: YahooMailWebService/0.8.188.663 Message-ID: <1399231662.36183.YahooMailNeo@web162702.mail.bf1.yahoo.com> Date: Sun, 4 May 2014 12:27:42 -0700 (PDT) From: mati ur-rahman Subject: netmap help To: "net@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: mati ur-rahman List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 19:27:51 -0000 HI,=0A=0AI want to ask how can i implement batch processing using poll();= =0Ahow we tell NIC about batches.=A0=0A=A0 =A0 =A0 =A0 Also have you implem= ented Batch processing in pkt-gen.c in the Example Directory.=A0=0A=A0 =A0 = =A0 =A0 If yes how can i control batch size in pkt-gen.c.=0A=0Aregards=0Ama= ti ur rahman=A0 From owner-freebsd-net@FreeBSD.ORG Mon May 5 03:42:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BD3C123 for ; Mon, 5 May 2014 03:42:22 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 081AE1806 for ; Mon, 5 May 2014 03:42:21 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s453g0EG095595 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 4 May 2014 20:42:12 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53670883.9060809@freebsd.org> Date: Mon, 05 May 2014 11:41:55 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= , "freebsd-net@freebsd.org" Subject: Re: bridge Untagged packets on an interface References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 03:42:22 -0000 On 5/3/14, 4:59 AM, Özkan KIRIK wrote: > And also i tried ng_bpf + ng_eiface conjuction, but ng_bpf doesnt match > "vlan" filter. > > With the script below, ng_bpf always calls ifNotMatch hook. When the > pattern is "ip", ng_bpf matches frames have both vlan and ip headers. > I think ng_bpf doesnt process ethernet header. Is there a way to process > ethernet header with ng_bpf ? once you have worked out what the framework is doing then it is very easy to read the code of netgraph modules because each module does only one thing. read the sample module for descriptions of what the parts do and then look at the modules you are trying to use. it should be pretty quickly obvious what they are doing. Julian > > Script is below: > > #!/bin/sh > ETHER_IF=em0 > PATTERN="vlan" > BPFPROG=$( tcpdump -s 8192 -ddd ${PATTERN} | \ > ( read len ; \ > echo -n "bpf_prog_len=$len " ; \ > echo -n "bpf_prog=[" ; \ > while read code jt jf k ; do \ > echo -n " { code=$code jt=$jt jf=$jf k=$k }" ; \ > done ; \ > echo " ]" ) ) > > echo $BPFPROG > > # Shutdown nodes if exists > ngctl shutdown ${ETHER_IF}: > ngctl shutdown vlan_filter: > ngctl shutdown tag0: > ngctl shutdown untag0: > > ngctl -f- < mkpeer ${ETHER_IF}: bpf lower filter_in > name ${ETHER_IF}:lower vlan_filter > mkpeer vlan_filter: eiface taggedPacket ether > mkpeer vlan_filter: eiface untaggedPacket ether > name vlan_filter:taggedPacket tag0 > name vlan_filter:untaggedPacket untag0 > msg vlan_filter: setprogram { thisHook="filter_in" ifMatch="taggedPacket" > ifNotMatch="untaggedPacket" $BPFPROG } > EOF > ifconfig ngeth0 up > ifconfig ngeth1 up > > > > On Fri, May 2, 2014 at 11:53 PM, Özkan KIRIK wrote: > >> Hi, >> >> Assume that default vlan untagged and VLAN 10, 20, 30, 40 tagged on switch >> connected to em0 interface. >> >> i am trying to bridge only untagged frames on em0 with em1. >> >> Does if_vlan handle untagged frames? >> >> # ifconfig em0.0 create >> ifconfig: SIOCIFCREATE2: Invalid argument >> >> Any ideas? >> >> Best regards, >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon May 5 08:15:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B75CB4AA for ; Mon, 5 May 2014 08:15:37 +0000 (UTC) Received: from mail-oa0-x244.google.com (mail-oa0-x244.google.com [IPv6:2607:f8b0:4003:c02::244]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FC541CCA for ; Mon, 5 May 2014 08:15:37 +0000 (UTC) Received: by mail-oa0-f68.google.com with SMTP id i7so860066oag.11 for ; Mon, 05 May 2014 01:15:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=kwuVqMS9fUQssL+5EQJBg6o3Qe28pzKxfviLXhwqZAQ=; b=S532MMOYxMatm0X6t5fJJzP2LeglwST4Oz8fKAHZDGpQecZ3YBY6t4Wyvnnm+7tPrG YbHU2+/USY17gukTexlaNZ12oydgqOrpEBLdOBsNucVx1NgZd0F2qdvxDDqVZ2NqDyjj WMuFdiO2HNgX4GsCthX0u+NfHwuVI+d9BjfvzyhFo1yoo861TzicplsYmwrJiw9nLky2 y74YYpmIOEtOrHO/I0/FgdvtN21u4CwLgxUwS7u2mszJizGeavmqNHUDtSXHKPt1wBaJ WyE4cLrbfBmBVGflCXXmtBr1HCiV9Xf3HvCoIYLZ5aGIo74uNhWqQPnmT7Kq3X2ZSmZB aHrA== MIME-Version: 1.0 X-Received: by 10.182.225.163 with SMTP id rl3mr273689obc.79.1399277736802; Mon, 05 May 2014 01:15:36 -0700 (PDT) Received: by 10.76.101.18 with HTTP; Mon, 5 May 2014 01:15:36 -0700 (PDT) In-Reply-To: References: Date: Mon, 5 May 2014 16:15:36 +0800 Message-ID: Subject: Re: netmap: how to bridge 2 eth? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 08:15:37 -0000 solved I changed Ubuntu to CentOS 6.5 and use kernel 3.4.88 then netmap/vale works OK...(but this time brctl & ping cannot work...well that's off-topic) 2014-05-04 14:18 GMT+08:00 upyzl : > add: > > tried > ./vale-ctl -h vale0:eth0 > ./vale-ctl -h vale0:eth1 > and pinging > > get error: > > May 4 22:02:01 ubuntu kernel: [ 854.827834] 121.198610 [ 425] > netmap_update_config configuration changed (but fine) > May 4 22:02:01 ubuntu kernel: [ 854.828080] 121.198860 [1456] > netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 > May 4 22:02:01 ubuntu kernel: [ 854.828330] 121.199111 [1049] > netmap_mem_global_config reconfiguring > May 4 22:02:14 ubuntu kernel: [ 868.034578] 134.402766 [ 425] > netmap_update_config configuration changed (but fine) > May 4 22:02:14 ubuntu kernel: [ 868.034818] 134.403019 [1456] > netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 > May 4 22:02:44 ubuntu kernel: [ 897.779633] 164.142010 [1298] > nm_txsync_prologue eth0 TX1 kring error: hwcur 0 rcur 0 hwtail 511 > cur 1 tail 511 > May 4 22:02:44 ubuntu kernel: [ 897.780009] 164.142403 [1298] > nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 > cur 0 tail 511 > May 4 22:02:44 ubuntu kernel: [ 897.780284] 164.142679 [1373] > nm_rxsync_prologue kring error: hwcur 0 rcur 0 hwtail 0 head 512 cur > 0 tail 0 > May 4 22:02:44 ubuntu kernel: [ 897.780563] 164.142944 [1549] > netmap_vp_rxsync_locked ouch dangerous reset!!! > May 4 22:02:44 ubuntu kernel: [ 897.780782] 164.143178 [1398] > netmap_ring_reinit called for vale0:eth1 > May 4 22:02:44 ubuntu kernel: [ 897.780969] 164.143365 [1423] > netmap_ring_reinit total 1 errors > May 4 22:02:44 ubuntu kernel: [ 897.781145] 164.143541 [1427] > netmap_ring_reinit vale0:eth1 RX0 reinit, cur 0 -> 0 tail 0 -> 0 > May 4 22:02:44 ubuntu kernel: [ 897.781393] 164.143789 [1298] > nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 > cur 1 tail 511 > May 4 22:02:44 ubuntu kernel: [ 897.781664] 164.144060 [1373] > nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur > 1 tail 1 > May 4 22:02:44 ubuntu kernel: [ 897.781947] 164.144343 [1549] > netmap_vp_rxsync_locked ouch dangerous reset!!! > May 4 22:02:44 ubuntu kernel: [ 897.782165] 164.144561 [1398] > netmap_ring_reinit called for vale0:eth1 > May 4 22:02:44 ubuntu kernel: [ 897.782355] 164.144751 [1423] > netmap_ring_reinit total 1 errors > May 4 22:02:44 ubuntu kernel: [ 897.782533] 164.144929 [1427] > netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 > May 4 22:02:44 ubuntu kernel: [ 897.782783] 164.145178 [1298] > nm_txsync_prologue eth1 TX1 kring error: hwcur 0 rcur 0 hwtail 511 > cur 1 tail 511 > May 4 22:02:45 ubuntu kernel: [ 898.776184] 165.138378 [1298] > nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 > cur 1 tail 511 > May 4 22:02:45 ubuntu kernel: [ 898.776493] 165.138694 [1373] > nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur > 1 tail 1 > May 4 22:02:45 ubuntu kernel: [ 898.776765] 165.138966 [1549] > netmap_vp_rxsync_locked ouch dangerous reset!!! > May 4 22:02:45 ubuntu kernel: [ 898.776987] 165.139189 [1398] > netmap_ring_reinit called for vale0:eth1 > May 4 22:02:45 ubuntu kernel: [ 898.777177] 165.139379 [1423] > netmap_ring_reinit total 1 errors > May 4 22:02:45 ubuntu kernel: [ 898.777357] 165.139558 [1427] > netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 > May 4 22:02:45 ubuntu kernel: [ 898.777624] 165.139825 [1298] > nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 > cur 2 tail 511 > May 4 22:02:45 ubuntu kernel: [ 898.777896] 165.140097 [1373] > nm_rxsync_prologue kring error: hwcur 0 rcur 2 hwtail 2 head 512 cur > 2 tail 2 > May 4 22:02:45 ubuntu kernel: [ 898.778159] 165.140361 [1549] > netmap_vp_rxsync_locked ouch dangerous reset!!! > > > then finally kernel panic/crash... > > > 2014-05-03 14:44 GMT+08:00 upyzl : > > Hi all >> >> I'm doing to implement a simple openflow-based software switch(not same >> as OVS, much simpler to OVS) >> and I choose netmap for the network framework >> >> Now I try to do a simplest thing: >> >> there's 3 VMs, all are Ubuntu 12.04.4 x64 >> 2 act as hosts, other 1 act as switch >> >> topo: [host1] eth0 ----- eth0 [switch] eth1 ----- eth0 [host2] >> >> I want to bridge switch's eth0ð1 (like "brctl" in linux), then host1 >> ping host2 >> >> the problem is, how to bridge using netmap? >> >> I tried example "bridge" in netmap project(git clone >> https://code.google.com/p/netmap/): >> ./bridge -i netmap:eth0 -i netmap:eth1 >> then host1 ping to host2 >> but pinging result is "Destination Host Unreachable" (if using brctl, >> pinging is fine) >> >> Then I tried vale-ctl >> but i dont know how to use it... >> e.g. >> ./vale-ctl -a eth0 >> show >> "eth0: Invalid argument" >> >> could anyone help me? >> > > From owner-freebsd-net@FreeBSD.ORG Mon May 5 08:47:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F02E827 for ; Mon, 5 May 2014 08:47:33 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D53BA106E for ; Mon, 5 May 2014 08:47:32 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id dc16so5149375qab.14 for ; Mon, 05 May 2014 01:47:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6bKF5wMD1M2aYRj9QDJxLA0jZB+eFSEEQjxlFTiyUWA=; b=BkuRU4tbS0oBhiiMuUCUs18v9h4eUikYgkSOvpyNvllt35ARK6u64fhDcglqvKXizG dAM+8OrbkM0YvO86d0PuDQlsc5WBWIHzI3TxD9kAvad0/TVFYefX7OLOUwbxnddSrVBK mr6vHTD2b3Fj/GX79Jjud/SX8aqQPwcAmrkhhDvbriqttsaWgpgS4NWY3yaLDF+ZUj1b xKxTEZ5uTcLh6Z1EUyrBo4W7PTBEHfVHu1IxpnjMqj2W5v9wEXjU12wX1WgiDM1PgZEy 13pV7WqykL9isaRhrzggk4G/Ph2hHUtaE1I9CROgnq68349wTmCE39QQvChOa5KPVKFI LOeQ== MIME-Version: 1.0 X-Received: by 10.224.51.2 with SMTP id b2mr42591481qag.49.1399279651628; Mon, 05 May 2014 01:47:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Mon, 5 May 2014 01:47:31 -0700 (PDT) In-Reply-To: References: Date: Mon, 5 May 2014 01:47:31 -0700 X-Google-Sender-Auth: 7B9CHmS3OOdSnCh1MN-UtgTle-o Message-ID: Subject: Re: netmap: how to bridge 2 eth? From: Adrian Chadd To: upyzl Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 08:47:33 -0000 Hi, I think you're asking the wrong group for help. This is for FreeBSD support, not Netmap-on-Linux support. Good luck though! -a On 2 May 2014 23:44, upyzl wrote: > Hi all > > I'm doing to implement a simple openflow-based software switch(not same as > OVS, much simpler to OVS) > and I choose netmap for the network framework > > Now I try to do a simplest thing: > > there's 3 VMs, all are Ubuntu 12.04.4 x64 > 2 act as hosts, other 1 act as switch > > topo: [host1] eth0 ----- eth0 [switch] eth1 ----- eth0 [host2] > > I want to bridge switch's eth0ð1 (like "brctl" in linux), then host1 > ping host2 > > the problem is, how to bridge using netmap? > > I tried example "bridge" in netmap project(git clone > https://code.google.com/p/netmap/): > ./bridge -i netmap:eth0 -i netmap:eth1 > then host1 ping to host2 > but pinging result is "Destination Host Unreachable" (if using brctl, > pinging is fine) > > Then I tried vale-ctl > but i dont know how to use it... > e.g. > ./vale-ctl -a eth0 > show > "eth0: Invalid argument" > > could anyone help me? > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon May 5 11:06:47 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6E3FDFF for ; Mon, 5 May 2014 11:06:47 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C33F21CF1 for ; Mon, 5 May 2014 11:06:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s45B6lJ4083186 for ; Mon, 5 May 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s45B6lUN083184 for freebsd-net@FreeBSD.org; Mon, 5 May 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 5 May 2014 11:06:47 GMT Message-Id: <201405051106.s45B6lUN083184@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 11:06:47 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/189234 net [em] Big lag with Ethernet Connection I217-V o kern/189219 net [dummynet] [patch] using dummynet on sparc64 and confi o kern/188899 net [cas] cas ethernet driver seems to have issues with so o kern/188897 net [dc] dc ethernet driver seems to prevent the detection o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/186304 net [bmc] watchdog service causes BMC controller reset eve o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184389 net [libalias] libalias fails to adjust MTU from jails o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o kern/183920 net [ixgbe] [patch] Incorrect ifconfig media on INTEL X520 o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183381 net [em] [patch] Use of 9k buffers in if_em.c hangs with r o kern/183271 net [em] statistic not updated on em in netmap mode o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179473 net [new driver] if_vether.c: Source code contribution of o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171550 net [ndis] [patch] "no match for InterlockedCompareExchang o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [netinet] [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 482 problems total. From owner-freebsd-net@FreeBSD.ORG Mon May 5 14:57:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97F9E7F4; Mon, 5 May 2014 14:57:34 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43D731666; Mon, 5 May 2014 14:57:34 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id db11so4287499veb.1 for ; Mon, 05 May 2014 07:57:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=KzFDCBRqlW1CfxYYerbS+s71L3IeDHdzYVK6i5BpoR0=; b=qOJDL/ZfmNmYGA4y293omcqyZ9soWHRQgNtvb/TnCmazaD0XfOorlrtP5Elmj6PORN XEqMRFraiOIraxGyRK5UWxi0+TPL55JewA4NmL0rGpkqDu702aRG/rCuF25hN4i62i40 etdlcygxeQxHMkCnGDZ0Jurl1cLqO22dfe/DuBhqBWYu77mbwX4xfpOlCTr4dWZ48adG /4T474O2HY4o4HFGOHfcbzR1U2G/S7nCEo8NGslwNtTiyypTG+lLBdGbaUNy0rAp6/ZC dYLoi7xLSKkWP1yzxLsYN8nJTzZePd2/7jv44WzF+sGdFDqsl0/X9eDeLJdYlrpZQ6wK NQGA== MIME-Version: 1.0 X-Received: by 10.220.183.4 with SMTP id ce4mr578827vcb.54.1399301853162; Mon, 05 May 2014 07:57:33 -0700 (PDT) Received: by 10.52.106.170 with HTTP; Mon, 5 May 2014 07:57:33 -0700 (PDT) In-Reply-To: References: Date: Mon, 5 May 2014 07:57:33 -0700 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Nick Rogers To: =?UTF-8?Q?Ermal_Lu=C3=A7i?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 14:57:34 -0000 On Sat, May 3, 2014 at 1:11 AM, Ermal Lu=C3=A7i wrote: > From experience LEGACY_TX + ALTQ is not usable and it will panic similar = to > what you have shown above. Do you believe this applies to em(4) interfaces as well? From my understanding, I had to add IGB_LEGACY_TX to get ALTQ working for igb interfaces post FreeBSD 9.X. em does and has always worked with ALTQ without modification of the e1000 driver. I also have a few hundred other systems running the same kernel with a similar PF ruleset and traffic load. Something about this particular site is causing a panic that I was hoping the dump would reveal as something obvious. > > I had to fix this for pfSense and the only way to get a stable driver was= to > have both if_transmit and if_start model activated in the driver. > Finding the paths that needs this 'hybrid' is a bit time consuming but a > strong candidate is altq interaction with the stack. That sounds promising. Would it be possible to get something like this committed in a more official capacity? Is the modified pfSense driver available somewhere that I could try? Thanks. > > There is work sponosred by the FreeBSD Foundation about to clarify/clean > this up but i am not sure the staus of that so far. > > > On Fri, May 2, 2014 at 10:49 PM, Nick Rogers wrote: >> >> Hello, >> >> I am hoping someone can help me debug a kernel panic I have been >> experiencing >> on one of my production systems. The system is a PF+ALTQ firewall/gatewa= y >> with >> about 1k simultaneous devices behind it at any given time, pushing no mo= re >> than 100Mb/s of traffic. I have obtained a crash dump and tried to debug >> it >> with kgdb, but am still at a loss. >> >> I have completely replaced the hardware to rule out issues with >> disk/memory/etc, and it appears to be a kernel issue, likely with e1000 >> driver >> and/or PF. >> >> The panic seems to happen during times of heavier use, but the frequency >> is >> not very predictable (anywhere from a few times a day to a once a week), >> so I >> feel like its some kind of strange traffic pattern that I am unable to >> pinpoint. >> >> I am using a slightly custom kernel based on GENERIC, mainly to bring in >> ALTQ >> and some other options. >> >> It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 >> driver, although I do not believe that should affect em(4). >> >> Hoping someone can be of assistance in debugging this problem. I am >> willing to >> test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if >> that >> is advisable. Unfortunately I cannot try 10-STABLE. >> >> Thanks, >> >> -Nick Rogers >> >> >> uname -v >> FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 >> root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS >> >> >> >> Here is the kernel conf... >> >> ........................................................................= ...... >> include GENERIC >> >> ident RGNETS >> >> makeoptions MODULES_OVERRIDE=3D"" >> >> options DEVICE_POLLING >> >> device crypto >> device cryptodev >> >> options VLAN_ARRAY >> >> device amdtemp >> >> # PF >> device pf >> device pflog >> options ALTQ >> options ALTQ_CBQ # Class Bases Queuing (CBQ) >> options ALTQ_RED # Random Early Detection (RED) >> options ALTQ_RIO # RED In/Out >> options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) >> options ALTQ_PRIQ # Priority Queuing (PRIQ) >> options ALTQ_NOPCC # Required for SMP build >> >> # PPPoE >> options NETGRAPH >> options NETGRAPH_ETHER >> options NETGRAPH_PPPOE >> options NETGRAPH_SOCKET >> >> # IPsec >> device enc >> options IPSEC >> options IPSEC_FILTERTUNNEL >> options IPSEC_NAT_T >> >> ........................................................................= ...... >> >> >> The crash dump.... >> >> ........................................................................= ...... >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you >> are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid =3D 5; apic id =3D 05 >> fault virtual address =3D 0x10 >> fault code =3D supervisor read data, page not present >> instruction pointer =3D 0x20:0xffffffff8033d350 >> stack pointer =3D 0x28:0xffffff83545384b0 >> frame pointer =3D 0x28:0xffffff83545384c0 >> code segment =3D base 0x0, limit 0xfffff, type 0x1b >> =3D DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags =3D interrupt enabled, resume, IOPL =3D 0 >> current process =3D 12 (irq262: em2:rx 0) >> trap number =3D 12 >> panic: page fault >> cpuid =3D 5 >> KDB: stack backtrace: >> #0 0xffffffff80956836 at kdb_backtrace+0x66 >> #1 0xffffffff8091c40e at panic+0x1ce >> #2 0xffffffff80d31e70 at trap_fatal+0x290 >> #3 0xffffffff80d321d1 at trap_pfault+0x211 >> #4 0xffffffff80d327d3 at trap+0x363 >> #5 0xffffffff80d1b9d3 at calltrap+0x8 >> #6 0xffffffff8034872d at pf_test_rule+0x17ed >> #7 0xffffffff8034ba12 at pf_test+0x1032 >> #8 0xffffffff8035112b at pf_check_in+0x2b >> #9 0xffffffff809e952e at pfil_run_hooks+0x9e >> #10 0xffffffff80a5286a at ip_input+0x2ea >> #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> #12 0xffffffff809df93d at ether_demux+0x14d >> #13 0xffffffff809dfc1e at ether_nh_input+0x1fe >> #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> #15 0xffffffff809df85f at ether_demux+0x6f >> #16 0xffffffff809dfc1e at ether_nh_input+0x1fe >> #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> Uptime: 17d7h20m59s >> Dumping 2932 out of 12256 MB: (CTRL-C to abort) >> ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> Reading symbols from /boot/kernel/aio.ko...Reading symbols from >> /boot/kernel/aio.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/aio.ko >> Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from >> /boot/kernel/coretemp.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/coretemp.ko >> Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from >> /boot/kernel/cc_htcp.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/cc_htcp.ko >> #0 doadump (textdump=3DVariable "textdump" is not available. >> ) at pcpu.h:234 >> 234 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) list *0xffffffff8033d350 >> 0xffffffff8033d350 is in pf_addrcpy >> (/usr/src/sys/contrib/pf/net/pf.c:512). >> 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) >> 508 { >> 509 switch (af) { >> 510 #ifdef INET >> 511 case AF_INET: >> 512 dst->addr32[0] =3D src->addr32[0]; >> 513 break; >> 514 #endif /* INET */ >> 515 case AF_INET6: >> 516 dst->addr32[0] =3D src->addr32[0]; >> (kgdb) backtrace >> #0 doadump (textdump=3DVariable "textdump" is not available. >> ) at pcpu.h:234 >> #1 0xffffffff8091bee6 in kern_reboot (howto=3D260) at >> /usr/src/sys/kern/kern_shutdown.c:454 >> #2 0xffffffff8091c3e7 in panic (fmt=3D0x1
) >> at /usr/src/sys/kern/kern_shutdown.c:642 >> #3 0xffffffff80d31e70 in trap_fatal (frame=3D0xc, eva=3DVariable "eva" = is >> not available. >> ) at /usr/src/sys/amd64/amd64/trap.c:878 >> #4 0xffffffff80d321d1 in trap_pfault (frame=3D0xffffff8354538400, >> usermode=3D0) at /usr/src/sys/amd64/amd64/trap.c:794 >> #5 0xffffffff80d327d3 in trap (frame=3D0xffffff8354538400) at >> /usr/src/sys/amd64/amd64/trap.c:456 >> #6 0xffffffff80d1b9d3 in calltrap () at >> /usr/src/sys/amd64/amd64/exception.S:232 >> #7 0xffffffff8033d350 in pf_addrcpy (dst=3D0xfffffe010c6416b8, >> src=3D0x10, af=3D2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 >> #8 0xffffffff8034872d in pf_test_rule (rm=3D0xffffff8354538788, >> sm=3D0xffffff8354538780, direction=3D1, kif=3D0xfffffe0007d08100, >> m=3D0xfffffe0030555d00, off=3D20, h=3D0xfffffe0030bad00e, >> pd=3D0xffffff83545386c0, am=3D0xffffff8354538790, >> rsm=3D0xffffff8354538778, ifq=3D0x0, inp=3D0x0) at >> /usr/src/sys/contrib/pf/net/pf.c:3900 >> #9 0xffffffff8034ba12 in pf_test (dir=3D1, ifp=3DVariable "ifp" is not >> available. >> ) at /usr/src/sys/contrib/pf/net/pf.c:6776 >> #10 0xffffffff8035112b in pf_check_in (arg=3DVariable "arg" is not >> available. >> ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 >> #11 0xffffffff809e952e in pfil_run_hooks (ph=3DVariable "ph" is not >> available. >> ) at /usr/src/sys/net/pfil.c:82 >> #12 0xffffffff80a5286a in ip_input (m=3D0xfffffe0030555d00) at >> /usr/src/sys/netinet/ip_input.c:510 >> #13 0xffffffff809e8858 in netisr_dispatch_src (proto=3D1, >> source=3DVariable "source" is not available. >> ) at /usr/src/sys/net/netisr.c:1013 >> #14 0xffffffff809df93d in ether_demux (ifp=3D0xfffffe0030239000, >> m=3D0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 >> #15 0xffffffff809dfc1e in ether_nh_input (m=3DVariable "m" is not availa= ble. >> ) at /usr/src/sys/net/if_ethersubr.c:762 >> #16 0xffffffff809e8858 in netisr_dispatch_src (proto=3D9, >> source=3DVariable "source" is not available. >> ) at /usr/src/sys/net/netisr.c:1013 >> #17 0xffffffff809df85f in ether_demux (ifp=3D0xfffffe0003f0c800, >> m=3D0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 >> #18 0xffffffff809dfc1e in ether_nh_input (m=3DVariable "m" is not availa= ble. >> ) at /usr/src/sys/net/if_ethersubr.c:762 >> #19 0xffffffff809e8858 in netisr_dispatch_src (proto=3D9, >> source=3DVariable "source" is not available. >> ) at /usr/src/sys/net/netisr.c:1013 >> #20 0xffffffff804ccb58 in em_rxeof (rxr=3D0xfffffe0007308200, count=3D-2= , >> done=3D0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 >> #21 0xffffffff804cceb6 in em_msix_rx (arg=3DVariable "arg" is not availa= ble. >> ) at /usr/src/sys/dev/e1000/if_em.c:1593 >> #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=3DVariable "p" >> is not available. >> ) at /usr/src/sys/kern/kern_intr.c:1272 >> #23 0xffffffff808ee30d in ithread_loop (arg=3D0xfffffe000730c300) at >> /usr/src/sys/kern/kern_intr.c:1285 >> #24 0xffffffff808e951f in fork_exit (callout=3D0xffffffff808ee270 >> , arg=3D0xfffffe000730c300, frame=3D0xffffff8354538c40) at >> /usr/src/sys/kern/kern_fork.c:996 >> #25 0xffffffff80d1befe in fork_trampoline () at >> /usr/src/sys/amd64/amd64/exception.S:606 >> #26 0x0000000000000000 in ?? () >> #27 0x0000000000000000 in ?? () >> #28 0x0000000000000001 in ?? () >> #29 0x0000000000000000 in ?? () >> #30 0x0000000000000000 in ?? () >> #31 0x0000000000000000 in ?? () >> #32 0x0000000000000000 in ?? () >> #33 0x0000000000000000 in ?? () >> #34 0x0000000000000000 in ?? () >> #35 0x0000000000000000 in ?? () >> #36 0x0000000000000000 in ?? () >> #37 0x0000000000000000 in ?? () >> #38 0x0000000000000000 in ?? () >> #39 0x0000000000000000 in ?? () >> #40 0x0000000000000000 in ?? () >> #41 0x0000000000000000 in ?? () >> #42 0x0000000000000000 in ?? () >> #43 0x0000000000000000 in ?? () >> #44 0x0000000000000000 in ?? () >> #45 0x0000000000000000 in ?? () >> #46 0x0000000000000000 in ?? () >> #47 0x0000000000000000 in ?? () >> #48 0x0000000000000000 in ?? () >> ---Type to continue, or q to quit--- >> #49 0x0000000000000000 in ?? () >> #50 0x0000000000000000 in ?? () >> #51 0xfffffe0007304920 in ?? () >> #52 0xfffffe0003afd000 in ?? () >> #53 0xfffffe0007304920 in ?? () >> #54 0xffffff8354538b40 in ?? () >> #55 0xffffff8354538ae8 in ?? () >> #56 0xfffffe0003b00920 in ?? () >> #57 0xffffffff80948646 in sched_switch (td=3D0xffffffff808ee270, >> newtd=3D0xfffffe000730c300, flags=3DVariable "flags" is not available. >> ) at /usr/src/sys/kern/sched_ule.c:1898 >> Previous frame inner to this frame (corrupt stack?) >> >> >> ........................................................................= ...... >> >> >> >> Relevant pciconf output... >> >> ........................................................................= ...... >> >> em0@pci0:1:0:0: class=3D0x020000 card=3D0x10d315d9 chip=3D0x10d38086 rev= =3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82574L Gigabit Network Connection' >> class =3D network >> subclass =3D ethernet >> cap 01[c8] =3D powerspec 2 supports D0 D3 current D0 >> cap 05[d0] =3D MSI supports 1 message, 64 bit >> cap 10[e0] =3D PCI-Express 1 endpoint max data 128(256) link x1(x1) >> cap 11[a0] =3D MSI-X supports 5 messages in map 0x1c enabled >> ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected >> em1@pci0:2:0:0: class=3D0x020000 card=3D0x10d315d9 chip=3D0x10d38086 rev= =3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82574L Gigabit Network Connection' >> class =3D network >> subclass =3D ethernet >> cap 01[c8] =3D powerspec 2 supports D0 D3 current D0 >> cap 05[d0] =3D MSI supports 1 message, 64 bit >> cap 10[e0] =3D PCI-Express 1 endpoint max data 128(256) link x1(x1) >> cap 11[a0] =3D MSI-X supports 5 messages in map 0x1c enabled >> ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected >> em2@pci0:7:0:0: class=3D0x020000 card=3D0x10d315d9 chip=3D0x10d38086 rev= =3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82574L Gigabit Network Connection' >> class =3D network >> subclass =3D ethernet >> cap 01[c8] =3D powerspec 2 supports D0 D3 current D0 >> cap 05[d0] =3D MSI supports 1 message, 64 bit >> cap 10[e0] =3D PCI-Express 1 endpoint max data 128(256) link x1(x1) >> cap 11[a0] =3D MSI-X supports 5 messages in map 0x1c enabled >> ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected >> em3@pci0:8:0:0: class=3D0x020000 card=3D0x10d315d9 chip=3D0x10d38086 rev= =3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D '82574L Gigabit Network Connection' >> class =3D network >> subclass =3D ethernet >> cap 01[c8] =3D powerspec 2 supports D0 D3 current D0 >> cap 05[d0] =3D MSI supports 1 message, 64 bit >> cap 10[e0] =3D PCI-Express 1 endpoint max data 128(256) link x1(x1) >> cap 11[a0] =3D MSI-X supports 5 messages in map 0x1c enabled >> >> >> ........................................................................= ...... >> >> >> dev.em sysctl.... >> >> ........................................................................= ...... >> >> dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> dev.em.0.%driver: em >> dev.em.0.%location: slot=3D0 function=3D0 >> dev.em.0.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 >> subdevice=3D0x10d3 class=3D0x020000 >> dev.em.0.%parent: pci1 >> dev.em.0.nvm: -1 >> dev.em.0.debug: -1 >> dev.em.0.fc: 3 >> dev.em.0.rx_int_delay: 0 >> dev.em.0.tx_int_delay: 66 >> dev.em.0.rx_abs_int_delay: 66 >> dev.em.0.tx_abs_int_delay: 66 >> dev.em.0.itr: 488 >> dev.em.0.rx_processing_limit: -1 >> dev.em.0.eee_control: 1 >> dev.em.0.link_irq: 2 >> dev.em.0.mbuf_alloc_fail: 0 >> dev.em.0.cluster_alloc_fail: 0 >> dev.em.0.dropped: 0 >> dev.em.0.tx_dma_fail: 0 >> dev.em.0.rx_overruns: 0 >> dev.em.0.watchdog_timeouts: 0 >> dev.em.0.device_control: 1477444168 >> dev.em.0.rx_control: 67141634 >> dev.em.0.fc_high_water: 18432 >> dev.em.0.fc_low_water: 16932 >> dev.em.0.queue0.txd_head: 3265 >> dev.em.0.queue0.txd_tail: 3265 >> dev.em.0.queue0.tx_irq: 81153071 >> dev.em.0.queue0.no_desc_avail: 0 >> dev.em.0.queue0.rxd_head: 388 >> dev.em.0.queue0.rxd_tail: 387 >> dev.em.0.queue0.rx_irq: 79015024 >> dev.em.0.mac_stats.excess_coll: 0 >> dev.em.0.mac_stats.single_coll: 0 >> dev.em.0.mac_stats.multiple_coll: 0 >> dev.em.0.mac_stats.late_coll: 0 >> dev.em.0.mac_stats.collision_count: 0 >> dev.em.0.mac_stats.symbol_errors: 0 >> dev.em.0.mac_stats.sequence_errors: 0 >> dev.em.0.mac_stats.defer_count: 0 >> dev.em.0.mac_stats.missed_packets: 0 >> dev.em.0.mac_stats.recv_no_buff: 0 >> dev.em.0.mac_stats.recv_undersize: 0 >> dev.em.0.mac_stats.recv_fragmented: 0 >> dev.em.0.mac_stats.recv_oversize: 0 >> dev.em.0.mac_stats.recv_jabber: 0 >> dev.em.0.mac_stats.recv_errs: 0 >> dev.em.0.mac_stats.crc_errs: 0 >> dev.em.0.mac_stats.alignment_errs: 0 >> dev.em.0.mac_stats.coll_ext_errs: 0 >> dev.em.0.mac_stats.xon_recvd: 6 >> dev.em.0.mac_stats.xon_txd: 0 >> dev.em.0.mac_stats.xoff_recvd: 6 >> dev.em.0.mac_stats.xoff_txd: 0 >> dev.em.0.mac_stats.total_pkts_recvd: 122072630 >> dev.em.0.mac_stats.good_pkts_recvd: 122072618 >> dev.em.0.mac_stats.bcast_pkts_recvd: 257 >> dev.em.0.mac_stats.mcast_pkts_recvd: 0 >> dev.em.0.mac_stats.rx_frames_64: 8634 >> dev.em.0.mac_stats.rx_frames_65_127: 67656673 >> dev.em.0.mac_stats.rx_frames_128_255: 714152 >> dev.em.0.mac_stats.rx_frames_256_511: 609615 >> dev.em.0.mac_stats.rx_frames_512_1023: 8646536 >> dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 >> dev.em.0.mac_stats.good_octets_recvd: 82940411216 >> dev.em.0.mac_stats.good_octets_txd: 25718335997 >> dev.em.0.mac_stats.total_pkts_txd: 99833592 >> dev.em.0.mac_stats.good_pkts_txd: 99833592 >> dev.em.0.mac_stats.bcast_pkts_txd: 13 >> dev.em.0.mac_stats.mcast_pkts_txd: 0 >> dev.em.0.mac_stats.tx_frames_64: 2193 >> dev.em.0.mac_stats.tx_frames_65_127: 29089783 >> dev.em.0.mac_stats.tx_frames_128_255: 54412030 >> dev.em.0.mac_stats.tx_frames_256_511: 9565246 >> dev.em.0.mac_stats.tx_frames_512_1023: 1080398 >> dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 >> dev.em.0.mac_stats.tso_txd: 1468623 >> dev.em.0.mac_stats.tso_ctx_fail: 0 >> dev.em.0.interrupts.asserts: 2 >> dev.em.0.interrupts.rx_pkt_timer: 0 >> dev.em.0.interrupts.rx_abs_timer: 0 >> dev.em.0.interrupts.tx_pkt_timer: 0 >> dev.em.0.interrupts.tx_abs_timer: 0 >> dev.em.0.interrupts.tx_queue_empty: 0 >> dev.em.0.interrupts.tx_queue_min_thresh: 0 >> dev.em.0.interrupts.rx_desc_min_thresh: 0 >> dev.em.0.interrupts.rx_overrun: 0 >> dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> dev.em.1.%driver: em >> dev.em.1.%location: slot=3D0 function=3D0 >> dev.em.1.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 >> subdevice=3D0x10d3 class=3D0x020000 >> dev.em.1.%parent: pci2 >> dev.em.1.nvm: -1 >> dev.em.1.debug: -1 >> dev.em.1.fc: 3 >> dev.em.1.rx_int_delay: 0 >> dev.em.1.tx_int_delay: 66 >> dev.em.1.rx_abs_int_delay: 66 >> dev.em.1.tx_abs_int_delay: 66 >> dev.em.1.itr: 488 >> dev.em.1.rx_processing_limit: -1 >> dev.em.1.eee_control: 1 >> dev.em.1.link_irq: 2 >> dev.em.1.mbuf_alloc_fail: 0 >> dev.em.1.cluster_alloc_fail: 0 >> dev.em.1.dropped: 0 >> dev.em.1.tx_dma_fail: 1 >> dev.em.1.rx_overruns: 0 >> dev.em.1.watchdog_timeouts: 0 >> dev.em.1.device_control: 1477444168 >> dev.em.1.rx_control: 67141634 >> dev.em.1.fc_high_water: 18432 >> dev.em.1.fc_low_water: 16932 >> dev.em.1.queue0.txd_head: 2451 >> dev.em.1.queue0.txd_tail: 2453 >> dev.em.1.queue0.tx_irq: 143904807 >> dev.em.1.queue0.no_desc_avail: 0 >> dev.em.1.queue0.rxd_head: 342 >> dev.em.1.queue0.rxd_tail: 341 >> dev.em.1.queue0.rx_irq: 159303310 >> dev.em.1.mac_stats.excess_coll: 0 >> dev.em.1.mac_stats.single_coll: 0 >> dev.em.1.mac_stats.multiple_coll: 0 >> dev.em.1.mac_stats.late_coll: 0 >> dev.em.1.mac_stats.collision_count: 0 >> dev.em.1.mac_stats.symbol_errors: 0 >> dev.em.1.mac_stats.sequence_errors: 0 >> dev.em.1.mac_stats.defer_count: 0 >> dev.em.1.mac_stats.missed_packets: 0 >> dev.em.1.mac_stats.recv_no_buff: 0 >> dev.em.1.mac_stats.recv_undersize: 0 >> dev.em.1.mac_stats.recv_fragmented: 0 >> dev.em.1.mac_stats.recv_oversize: 0 >> dev.em.1.mac_stats.recv_jabber: 0 >> dev.em.1.mac_stats.recv_errs: 0 >> dev.em.1.mac_stats.crc_errs: 0 >> dev.em.1.mac_stats.alignment_errs: 0 >> dev.em.1.mac_stats.coll_ext_errs: 0 >> dev.em.1.mac_stats.xon_recvd: 1 >> dev.em.1.mac_stats.xon_txd: 0 >> dev.em.1.mac_stats.xoff_recvd: 1 >> dev.em.1.mac_stats.xoff_txd: 0 >> dev.em.1.mac_stats.total_pkts_recvd: 331901758 >> dev.em.1.mac_stats.good_pkts_recvd: 331901756 >> dev.em.1.mac_stats.bcast_pkts_recvd: 13467 >> dev.em.1.mac_stats.mcast_pkts_recvd: 0 >> dev.em.1.mac_stats.rx_frames_64: 13905035 >> dev.em.1.mac_stats.rx_frames_65_127: 22315178 >> dev.em.1.mac_stats.rx_frames_128_255: 8343368 >> dev.em.1.mac_stats.rx_frames_256_511: 8602323 >> dev.em.1.mac_stats.rx_frames_512_1023: 8170288 >> dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 >> dev.em.1.mac_stats.good_octets_recvd: 420794715670 >> dev.em.1.mac_stats.good_octets_txd: 45361473880 >> dev.em.1.mac_stats.total_pkts_txd: 217852588 >> dev.em.1.mac_stats.good_pkts_txd: 217852588 >> dev.em.1.mac_stats.bcast_pkts_txd: 6 >> dev.em.1.mac_stats.mcast_pkts_txd: 0 >> dev.em.1.mac_stats.tx_frames_64: 64102191 >> dev.em.1.mac_stats.tx_frames_65_127: 120705475 >> dev.em.1.mac_stats.tx_frames_128_255: 6009336 >> dev.em.1.mac_stats.tx_frames_256_511: 4593595 >> dev.em.1.mac_stats.tx_frames_512_1023: 4295623 >> dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 >> dev.em.1.mac_stats.tso_txd: 291134 >> dev.em.1.mac_stats.tso_ctx_fail: 0 >> dev.em.1.interrupts.asserts: 2 >> dev.em.1.interrupts.rx_pkt_timer: 0 >> dev.em.1.interrupts.rx_abs_timer: 0 >> dev.em.1.interrupts.tx_pkt_timer: 0 >> dev.em.1.interrupts.tx_abs_timer: 0 >> dev.em.1.interrupts.tx_queue_empty: 0 >> dev.em.1.interrupts.tx_queue_min_thresh: 0 >> dev.em.1.interrupts.rx_desc_min_thresh: 0 >> dev.em.1.interrupts.rx_overrun: 0 >> dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> dev.em.2.%driver: em >> dev.em.2.%location: slot=3D0 function=3D0 >> dev.em.2.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 >> subdevice=3D0x10d3 class=3D0x020000 >> dev.em.2.%parent: pci7 >> dev.em.2.nvm: -1 >> dev.em.2.debug: -1 >> dev.em.2.fc: 3 >> dev.em.2.rx_int_delay: 0 >> dev.em.2.tx_int_delay: 66 >> dev.em.2.rx_abs_int_delay: 66 >> dev.em.2.tx_abs_int_delay: 66 >> dev.em.2.itr: 488 >> dev.em.2.rx_processing_limit: -1 >> dev.em.2.eee_control: 1 >> dev.em.2.link_irq: 1 >> dev.em.2.mbuf_alloc_fail: 0 >> dev.em.2.cluster_alloc_fail: 0 >> dev.em.2.dropped: 0 >> dev.em.2.tx_dma_fail: 6823 >> dev.em.2.rx_overruns: 0 >> dev.em.2.watchdog_timeouts: 0 >> dev.em.2.device_control: 1477444168 >> dev.em.2.rx_control: 67141634 >> dev.em.2.fc_high_water: 18432 >> dev.em.2.fc_low_water: 16932 >> dev.em.2.queue0.txd_head: 3977 >> dev.em.2.queue0.txd_tail: 3977 >> dev.em.2.queue0.tx_irq: 220950699 >> dev.em.2.queue0.no_desc_avail: 0 >> dev.em.2.queue0.rxd_head: 83 >> dev.em.2.queue0.rxd_tail: 82 >> dev.em.2.queue0.rx_irq: 125920607 >> dev.em.2.mac_stats.excess_coll: 0 >> dev.em.2.mac_stats.single_coll: 0 >> dev.em.2.mac_stats.multiple_coll: 0 >> dev.em.2.mac_stats.late_coll: 0 >> dev.em.2.mac_stats.collision_count: 0 >> dev.em.2.mac_stats.symbol_errors: 0 >> dev.em.2.mac_stats.sequence_errors: 0 >> dev.em.2.mac_stats.defer_count: 0 >> dev.em.2.mac_stats.missed_packets: 0 >> dev.em.2.mac_stats.recv_no_buff: 0 >> dev.em.2.mac_stats.recv_undersize: 0 >> dev.em.2.mac_stats.recv_fragmented: 0 >> dev.em.2.mac_stats.recv_oversize: 0 >> dev.em.2.mac_stats.recv_jabber: 0 >> dev.em.2.mac_stats.recv_errs: 0 >> dev.em.2.mac_stats.crc_errs: 0 >> dev.em.2.mac_stats.alignment_errs: 0 >> dev.em.2.mac_stats.coll_ext_errs: 0 >> dev.em.2.mac_stats.xon_recvd: 14123 >> dev.em.2.mac_stats.xon_txd: 1 >> dev.em.2.mac_stats.xoff_recvd: 14127 >> dev.em.2.mac_stats.xoff_txd: 1 >> dev.em.2.mac_stats.total_pkts_recvd: 229919303 >> dev.em.2.mac_stats.good_pkts_recvd: 229891053 >> dev.em.2.mac_stats.bcast_pkts_recvd: 909450 >> dev.em.2.mac_stats.mcast_pkts_recvd: 19452 >> dev.em.2.mac_stats.rx_frames_64: 1477808 >> dev.em.2.mac_stats.rx_frames_65_127: 195114744 >> dev.em.2.mac_stats.rx_frames_128_255: 6579690 >> dev.em.2.mac_stats.rx_frames_256_511: 5137387 >> dev.em.2.mac_stats.rx_frames_512_1023: 4223090 >> dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 >> dev.em.2.mac_stats.good_octets_recvd: 46129102134 >> dev.em.2.mac_stats.good_octets_txd: 419293159496 >> dev.em.2.mac_stats.total_pkts_txd: 332661584 >> dev.em.2.mac_stats.good_pkts_txd: 332661582 >> dev.em.2.mac_stats.bcast_pkts_txd: 48506 >> dev.em.2.mac_stats.mcast_pkts_txd: 78 >> dev.em.2.mac_stats.tx_frames_64: 14598198 >> dev.em.2.mac_stats.tx_frames_65_127: 22287108 >> dev.em.2.mac_stats.tx_frames_128_255: 8897511 >> dev.em.2.mac_stats.tx_frames_256_511: 9623000 >> dev.em.2.mac_stats.tx_frames_512_1023: 8325033 >> dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 >> dev.em.2.mac_stats.tso_txd: 24357891 >> dev.em.2.mac_stats.tso_ctx_fail: 0 >> dev.em.2.interrupts.asserts: 2 >> dev.em.2.interrupts.rx_pkt_timer: 0 >> dev.em.2.interrupts.rx_abs_timer: 0 >> dev.em.2.interrupts.tx_pkt_timer: 0 >> dev.em.2.interrupts.tx_abs_timer: 0 >> dev.em.2.interrupts.tx_queue_empty: 0 >> dev.em.2.interrupts.tx_queue_min_thresh: 0 >> dev.em.2.interrupts.rx_desc_min_thresh: 0 >> dev.em.2.interrupts.rx_overrun: 0 >> dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> dev.em.3.%driver: em >> dev.em.3.%location: slot=3D0 function=3D0 >> dev.em.3.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 >> subdevice=3D0x10d3 class=3D0x020000 >> dev.em.3.%parent: pci8 >> dev.em.3.nvm: -1 >> dev.em.3.debug: -1 >> dev.em.3.fc: 3 >> dev.em.3.rx_int_delay: 0 >> dev.em.3.tx_int_delay: 66 >> dev.em.3.rx_abs_int_delay: 66 >> dev.em.3.tx_abs_int_delay: 66 >> dev.em.3.itr: 488 >> dev.em.3.rx_processing_limit: -1 >> dev.em.3.eee_control: 1 >> dev.em.3.link_irq: 0 >> dev.em.3.mbuf_alloc_fail: 0 >> dev.em.3.cluster_alloc_fail: 0 >> dev.em.3.dropped: 0 >> dev.em.3.tx_dma_fail: 0 >> dev.em.3.rx_overruns: 0 >> dev.em.3.watchdog_timeouts: 0 >> dev.em.3.device_control: 1074790984 >> dev.em.3.rx_control: 67141634 >> dev.em.3.fc_high_water: 18432 >> dev.em.3.fc_low_water: 16932 >> dev.em.3.queue0.txd_head: 0 >> dev.em.3.queue0.txd_tail: 0 >> dev.em.3.queue0.tx_irq: 0 >> dev.em.3.queue0.no_desc_avail: 0 >> dev.em.3.queue0.rxd_head: 0 >> dev.em.3.queue0.rxd_tail: 4095 >> dev.em.3.queue0.rx_irq: 0 >> dev.em.3.mac_stats.excess_coll: 0 >> dev.em.3.mac_stats.single_coll: 0 >> dev.em.3.mac_stats.multiple_coll: 0 >> dev.em.3.mac_stats.late_coll: 0 >> dev.em.3.mac_stats.collision_count: 0 >> dev.em.3.mac_stats.symbol_errors: 0 >> dev.em.3.mac_stats.sequence_errors: 0 >> dev.em.3.mac_stats.defer_count: 0 >> dev.em.3.mac_stats.missed_packets: 0 >> dev.em.3.mac_stats.recv_no_buff: 0 >> dev.em.3.mac_stats.recv_undersize: 0 >> dev.em.3.mac_stats.recv_fragmented: 0 >> dev.em.3.mac_stats.recv_oversize: 0 >> dev.em.3.mac_stats.recv_jabber: 0 >> dev.em.3.mac_stats.recv_errs: 0 >> dev.em.3.mac_stats.crc_errs: 0 >> dev.em.3.mac_stats.alignment_errs: 0 >> dev.em.3.mac_stats.coll_ext_errs: 0 >> dev.em.3.mac_stats.xon_recvd: 0 >> dev.em.3.mac_stats.xon_txd: 0 >> dev.em.3.mac_stats.xoff_recvd: 0 >> dev.em.3.mac_stats.xoff_txd: 0 >> dev.em.3.mac_stats.total_pkts_recvd: 0 >> dev.em.3.mac_stats.good_pkts_recvd: 0 >> dev.em.3.mac_stats.bcast_pkts_recvd: 0 >> dev.em.3.mac_stats.mcast_pkts_recvd: 0 >> dev.em.3.mac_stats.rx_frames_64: 0 >> dev.em.3.mac_stats.rx_frames_65_127: 0 >> dev.em.3.mac_stats.rx_frames_128_255: 0 >> dev.em.3.mac_stats.rx_frames_256_511: 0 >> dev.em.3.mac_stats.rx_frames_512_1023: 0 >> dev.em.3.mac_stats.rx_frames_1024_1522: 0 >> dev.em.3.mac_stats.good_octets_recvd: 0 >> dev.em.3.mac_stats.good_octets_txd: 0 >> dev.em.3.mac_stats.total_pkts_txd: 0 >> dev.em.3.mac_stats.good_pkts_txd: 0 >> dev.em.3.mac_stats.bcast_pkts_txd: 0 >> dev.em.3.mac_stats.mcast_pkts_txd: 0 >> dev.em.3.mac_stats.tx_frames_64: 0 >> dev.em.3.mac_stats.tx_frames_65_127: 0 >> dev.em.3.mac_stats.tx_frames_128_255: 0 >> dev.em.3.mac_stats.tx_frames_256_511: 0 >> dev.em.3.mac_stats.tx_frames_512_1023: 0 >> dev.em.3.mac_stats.tx_frames_1024_1522: 0 >> dev.em.3.mac_stats.tso_txd: 0 >> dev.em.3.mac_stats.tso_ctx_fail: 0 >> dev.em.3.interrupts.asserts: 0 >> dev.em.3.interrupts.rx_pkt_timer: 0 >> dev.em.3.interrupts.rx_abs_timer: 0 >> dev.em.3.interrupts.tx_pkt_timer: 0 >> dev.em.3.interrupts.tx_abs_timer: 0 >> dev.em.3.interrupts.tx_queue_empty: 0 >> dev.em.3.interrupts.tx_queue_min_thresh: 0 >> dev.em.3.interrupts.rx_desc_min_thresh: 0 >> dev.em.3.interrupts.rx_overrun: 0 >> >> >> ........................................................................= ...... >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > -- > Ermal From owner-freebsd-net@FreeBSD.ORG Tue May 6 16:07:25 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 486217B5; Tue, 6 May 2014 16:07:25 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D538C28A; Tue, 6 May 2014 16:07:24 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Whe03-000HBg-IU; Tue, 06 May 2014 15:57:31 +0400 Message-ID: <53690885.1010704@FreeBSD.org> Date: Tue, 06 May 2014 20:06:29 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: FreeBSD Net , hackers@freebsd.org Subject: Use of contiguous physical memory in ixgbe driver Content-Type: multipart/mixed; boundary="------------060607030109030205060500" Cc: jfv@FreeBSD.org, Adrian Chadd , wollman@freebsd.org, jhb@freebsd.org, nparhar@gmail.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 16:07:25 -0000 This is a multi-part message in MIME format. --------------060607030109030205060500 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hello guys. (bootstrapping people involved in previous version of this topic, sorry for that) There were several problem descriptions/discussions on using 9k+ mbufs with current allocator in: if_em: kern/183381 cxgbe: http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037834.html general one: http://lists.freebsd.org/pipermail/freebsd-net/2014-January/037673.html I'd like to add ixgbe (and i40e with igb) to the list. We're facing the same problem for a long time. As far as I can understand, a) everyone (tm) is aware of current 9/16k allocation problems leading to sudden network failures. b) such mbufs sizes are not absolute evil and can be useful on 40/100G and for TSO cases. c) however, no one is able to / willing to fix our allocator to pre-allocate special arena for mbufs >= 4k page size. d) so most people have written their own local hacks to disable 9k mbufs and use 4k ones. e) our list is not full, people with mellanox/solarflare/broadcom/emulex/etc are still not there (and most if not all 10g NICs support scatter/gather). Can we add more generic hack moving default mbuf size decision from NIC driver to OS and make it tunable for user? Example path for Intel ones is attached. --------------060607030109030205060500 Content-Type: text/x-patch; name="mbuf_sizes.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="mbuf_sizes.diff" Index: sys/kern/kern_mbuf.c =================================================================== --- sys/kern/kern_mbuf.c (revision 265236) +++ sys/kern/kern_mbuf.c (working copy) @@ -103,6 +103,11 @@ int nmbjumbop; /* limits number of page size jum int nmbjumbo9; /* limits number of 9k jumbo clusters */ int nmbjumbo16; /* limits number of 16k jumbo clusters */ +static int nojumbobuf; /* Use MCLBYTES mbufs */ +static int nojumbo9buf; /* Use either MCLBYTES or MJUMPAGESIZE */ +static int nojumbo16buf; /* Use any mbuf size less than MJUM16BYTES */ + + static quad_t maxmbufmem; /* overall real memory limit for all mbufs */ SYSCTL_QUAD(_kern_ipc, OID_AUTO, maxmbufmem, CTLFLAG_RDTUN, &maxmbufmem, 0, @@ -151,6 +156,17 @@ tunable_mbinit(void *dummy) if (nmbufs < nmbclusters + nmbjumbop + nmbjumbo9 + nmbjumbo16) nmbufs = lmax(maxmbufmem / MSIZE / 5, nmbclusters + nmbjumbop + nmbjumbo9 + nmbjumbo16); + + /* + * Defaults to disable 9/16-kbyte pages + */ + nojumbobuf = 0; + nojumbo9buf = 1; + nojumbo16buf = 1; + + TUNABLE_INT_FETCH("kern.ipc.nojumbobuf", &nojumbobuf); + TUNABLE_INT_FETCH("kern.ipc.nojumbo9buf", &nojumbo9buf); + TUNABLE_INT_FETCH("kern.ipc.nojumbo16buf", &nojumbo16buf); } SYSINIT(tunable_mbinit, SI_SUB_KMEM, SI_ORDER_MIDDLE, tunable_mbinit, NULL); @@ -261,6 +277,27 @@ SYSCTL_PROC(_kern_ipc, OID_AUTO, nmbufs, CTLTYPE_I "Maximum number of mbufs allowed"); /* + * Determine the correct mbuf pool + * for given mtu size + */ +int +m_preferredsize(int mtu) +{ + int size; + + if (mtu <= 2048 || nojumbobuf != 0) + size = MCLBYTES; + else if (mtu <= 4096 || nojumbo9buf != 0) + size = MJUMPAGESIZE; + else if (mtu <= 9216 || nojumbo16buf != 0) + size = MJUM9BYTES; + else + size = MJUM16BYTES; + + return (size); +} + +/* * Zones from which we allocate. */ uma_zone_t zone_mbuf; Index: sys/dev/ixgbe/ixgbe.c =================================================================== --- sys/dev/ixgbe/ixgbe.c (revision 265236) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -1138,14 +1138,7 @@ ixgbe_init_locked(struct adapter *adapter) ** Determine the correct mbuf pool ** for doing jumbo frames */ - if (adapter->max_frame_size <= 2048) - adapter->rx_mbuf_sz = MCLBYTES; - else if (adapter->max_frame_size <= 4096) - adapter->rx_mbuf_sz = MJUMPAGESIZE; - else if (adapter->max_frame_size <= 9216) - adapter->rx_mbuf_sz = MJUM9BYTES; - else - adapter->rx_mbuf_sz = MJUM16BYTES; + adapter->rx_mbuf_sz = m_preferredsize(adapter->max_frame_size); /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { Index: sys/dev/e1000/if_em.c =================================================================== --- sys/dev/e1000/if_em.c (revision 265236) +++ sys/dev/e1000/if_em.c (working copy) @@ -1342,12 +1342,7 @@ em_init_locked(struct adapter *adapter) ** Figure out the desired mbuf ** pool for doing jumbos */ - if (adapter->hw.mac.max_frame_size <= 2048) - adapter->rx_mbuf_sz = MCLBYTES; - else if (adapter->hw.mac.max_frame_size <= 4096) - adapter->rx_mbuf_sz = MJUMPAGESIZE; - else - adapter->rx_mbuf_sz = MJUM9BYTES; + adapter->rx_mbuf_sz = m_preferredsize(adapter->hw.mac.max_frame_size); /* Prepare receive descriptors and buffers */ if (em_setup_receive_structures(adapter)) { Index: sys/dev/e1000/if_igb.c =================================================================== --- sys/dev/e1000/if_igb.c (revision 265236) +++ sys/dev/e1000/if_igb.c (working copy) @@ -1335,12 +1335,7 @@ igb_init_locked(struct adapter *adapter) ** Figure out the desired mbuf pool ** for doing jumbo/packetsplit */ - if (adapter->max_frame_size <= 2048) - adapter->rx_mbuf_sz = MCLBYTES; - else if (adapter->max_frame_size <= 4096) - adapter->rx_mbuf_sz = MJUMPAGESIZE; - else - adapter->rx_mbuf_sz = MJUM9BYTES; + adapter->rx_mbuf_sz = m_preferredsize(adapter->max_frame_size); /* Prepare receive descriptors and buffers */ if (igb_setup_receive_structures(adapter)) { --------------060607030109030205060500-- From owner-freebsd-net@FreeBSD.ORG Tue May 6 17:21:14 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08589D9B; Tue, 6 May 2014 17:21:14 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B15F6C50; Tue, 6 May 2014 17:21:13 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id kx10so11669281pab.11 for ; Tue, 06 May 2014 10:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=DBOIOLc28ZVtP9lpldzVmDzwus52Q8TsixH0m2Xf1Mk=; b=t8wzi9NEjG1M4ICwnFLhy1A8ySo1Vxn/M5uSfrg5mIu/IyGxtGRnQIrRVmBwJvNW0n igQsvdxyOr7yPD2FDnVwOupZIM8WBYb1p+riu3Vd7Fnlun17+CpPtRI3kp002rclgJvo lqCqNKytG/wXwHZFEnYI96uiQu3IEAecj59AjINhNZwwrly4UAYn0n36SmVkAdC32XNl F6OKhrMYOFaThg6k9UyEe9hE2gyXLHY0QIGZ88JogyUI9xF+n19NQt49q67kVmGmh+o3 HY9i/+AF1+mHhmz3RwdwyS07jTZ60IOenFjrQiRcZ03XfvL4YSUtbh1tundw7zg344wG PF5A== X-Received: by 10.66.191.134 with SMTP id gy6mr8576016pac.27.1399396873130; Tue, 06 May 2014 10:21:13 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id pl10sm1539591pbb.56.2014.05.06.10.21.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 06 May 2014 10:21:12 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <53691A07.2060304@FreeBSD.org> Date: Tue, 06 May 2014 10:21:11 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , FreeBSD Net , hackers@freebsd.org Subject: Re: Use of contiguous physical memory in ixgbe driver References: <53690885.1010704@FreeBSD.org> In-Reply-To: <53690885.1010704@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: jfv@FreeBSD.org, Adrian Chadd , wollman@freebsd.org, jhb@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 17:21:14 -0000 On 05/06/14 09:06, Alexander V. Chernikov wrote: > Hello guys. > (bootstrapping people involved in previous version of this topic, sorry > for that) > > There were several problem descriptions/discussions on using 9k+ mbufs > with current allocator in: > if_em: kern/183381 > cxgbe: > http://lists.freebsd.org/pipermail/freebsd-net/2014-February/037834.html > > general one: > http://lists.freebsd.org/pipermail/freebsd-net/2014-January/037673.html I changed cxgbe(4) in response to those discussions (see r263317 for details; r263451 has the man page updates). It still prefers large clusters (if MTU is > 4K) by default but will happily fall back to the 4K zone if it encounters failures when allocating from the larger zones. I also added a knob that you can use to forbid cxgbe(4) from even attempting to allocate from the large zones. So the latest cxgbe(4) should be able to cope just fine at whatever MTU even when the 9K, 16K zones are depleted. Regards, Navdeep From owner-freebsd-net@FreeBSD.ORG Wed May 7 07:52:29 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22989B0D for ; Wed, 7 May 2014 07:52:29 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id A8DA8292 for ; Wed, 7 May 2014 07:52:28 +0000 (UTC) Received: from study64.tdx.co.uk (study64.tdx.co.uk [62.13.130.231]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s477qQ2b003764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 7 May 2014 08:52:27 +0100 (BST) Date: Wed, 07 May 2014 08:52:27 +0100 From: Karl Pielorz To: net@freebsd.org Subject: NIC congestion - indicators? Message-ID: <6846364B6813CBB557D4D8D5@study64.tdx.co.uk> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 07:52:29 -0000 Hi, I've a couple of boxes which *might* be hitting the ceiling for their NIC's (i.e. using all the bandwidth on the link). Other than looking at 'systat -if' - is there anything in netstat's output (or anywhere else) that would indicate the system is either heavily queuing outgoing packets - or binning them off? Thanks, -Karl From owner-freebsd-net@FreeBSD.ORG Wed May 7 07:56:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72B71E42; Wed, 7 May 2014 07:56:14 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD962C8; Wed, 7 May 2014 07:56:14 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id y10so735632pdj.25 for ; Wed, 07 May 2014 00:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=NcQnkaqDYAbL4O52JFBe/nvQHR+QqtYnNjBNlZLThCE=; b=VTGeW3azwbIJgrj4Lho854PM9jANncHynB8Qsp4+VbCc0gXnWlY+707BL9IezYPQOP Bp+ugv8yxKM6ahmC3lU7JwgMu4tYXX2DVOpem5+qgy+oPpbhyNwD9qDGtty1C9jUqtr0 0IkBTJpjEMKIa150gw326lggCiGHvwhJ0+E9l1wOkpFk07Kz5ISy6g9z1KXy8Sm6atCU +tDscrineQ/PsEAQq4su1cjaSeGBaghPKoohjDM/yCeTJJy7SWh6jweQb2zuN1QFGLiE IeEdvQfPYKdtvTG0eBVl3CkO0UmrrgCHRWDWXREPJKMLvdM+zYJV7QmKwap8UhQoU/FW 4uKA== X-Received: by 10.66.121.131 with SMTP id lk3mr16058157pab.61.1399449373546; Wed, 07 May 2014 00:56:13 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id io8sm1655920pbc.96.2014.05.07.00.56.10 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 07 May 2014 00:56:12 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 07 May 2014 16:56:12 +0900 From: Yonghyeon PYUN Date: Wed, 7 May 2014 16:56:12 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140507075612.GA1376@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 07:56:14 -0000 On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: > On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: > > > > > On 02 May 2014, at 10:22 , Michael Tuexen wrote: > > > >> Dear all, > >> > >> during testing I found that FreeBSD head (on a raspberry pi) accepts SCTP packet > >> with bad checksums. After debugging this I figured out that this is a problem with > >> the csum_flags defined in mbuf.h. > >> > >> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is defined in mbuf.h: > >> #define CSUM_SCTP_VALID CSUM_L4_VALID > >> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is considered > >> to have a correct checksum. > >> > >> For UDP and TCP some drivers calculate the UDP/TCP checksum and set CSUM_DATA_VALID in > >> csum_flags to indicate that the UDP/TCP should consider csum_data to figure out if > >> the packet has a correct checksum. The problem is that CSUM_DATA_VALID is defined as > >> #define CSUM_DATA_VALID CSUM_L4_VALID > >> In this case the semantic is not that the packet has a valid checksum, but the csum_data > >> field contains information. > >> > >> Now the following happens (on the raspberry pi the driver used is > >> dev/usb/net/if_smsc.c > >> > >> 1. A packet is received and if it is not too short, the checksum computed > >> is stored in csum_data and the flag CSUM_DATA_VALID is set. This happens > >> for all IP packets, not only for UDP and TCP packets. > >> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as CSUM_SCTP_VALID > >> and accepts the packet. So no SCTP checksum check ever happened. > >> > >> Alternatives to fix this: > >> > >> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP packets, since > >> it only makes sense in these cases. > > > > Wait, or for SCTP in cad the crc32 (I think it was) was actually checked but not otherwise. This is how it should be imho. It seems like a driver bug. > I went through the list of drivers and you are right, it seems to be a bug > in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I can't tell. > I'm not sure how the controller computes TCP/UDP checksum values. It seems the publicly available data sheet was highly sanitized so it was useless to me. The comment in the driver says that the controller computes RX checksum after the IPv4 header to the end of ethernet frame. After seeing that comment, three questions popped up: 1. Is the controller smart enough to skip IP options header in TCP/UDP checksum offloading? 2. How controller handles UDP checksum value 0x0000(i.e. sender didn't compute UDP checksum)? 3. How the controller can compute TCP checksum of fragmented packets? Since you have the controller I guess it's easy to verify all cases. For case 3, I believe the controller can't handle fragmented frames so driver should have to explicitly check ip_off field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) handle it. > Best regards > Michael > > > > > > ? > > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 > > > > From owner-freebsd-net@FreeBSD.ORG Wed May 7 08:07:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1FB6BC40; Wed, 7 May 2014 08:07:14 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1C983F0; Wed, 7 May 2014 08:07:13 +0000 (UTC) Received: from [10.225.7.42] (unknown [194.95.73.101]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 4B41A1C104904; Wed, 7 May 2014 10:07:10 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140507075612.GA1376@michelle.cdnetworks.com> Date: Wed, 7 May 2014 10:07:09 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 08:07:14 -0000 On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: > On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >>=20 >>>=20 >>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>=20 >>>> Dear all, >>>>=20 >>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>> with bad checksums. After debugging this I figured out that this is = a problem with >>>> the csum_flags defined in mbuf.h. >>>>=20 >>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which = is defined in mbuf.h: >>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the = packet is considered >>>> to have a correct checksum. >>>>=20 >>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and set = CSUM_DATA_VALID in >>>> csum_flags to indicate that the UDP/TCP should consider csum_data = to figure out if >>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>> field contains information. >>>>=20 >>>> Now the following happens (on the raspberry pi the driver used is >>>> dev/usb/net/if_smsc.c >>>>=20 >>>> 1. A packet is received and if it is not too short, the checksum = computed >>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens >>>> for all IP packets, not only for UDP and TCP packets. >>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as = CSUM_SCTP_VALID >>>> and accepts the packet. So no SCTP checksum check ever happened. >>>>=20 >>>> Alternatives to fix this: >>>>=20 >>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or = TCP packets, since >>>> it only makes sense in these cases. >>>=20 >>> Wait, or for SCTP in cad the crc32 (I think it was) was actually = checked but not otherwise. This is how it should be imho. It seems = like a driver bug. >> I went through the list of drivers and you are right, it seems to be = a bug >> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small = set I can't tell. >>=20 >=20 > I'm not sure how the controller computes TCP/UDP checksum values. > It seems the publicly available data sheet was highly sanitized so > it was useless to me. The comment in the driver says that the Same for me... > controller computes RX checksum after the IPv4 header to the end of > ethernet frame. After seeing that comment, three questions popped > up: >=20 > 1. Is the controller smart enough to skip IP options header in > TCP/UDP checksum offloading? > 2. How controller handles UDP checksum value 0x0000(i.e. sender > didn't compute UDP checksum)? > 3. How the controller can compute TCP checksum of fragmented > packets? >=20 > Since you have the controller I guess it's easy to verify all > cases. For case 3, I believe the controller can't handle > fragmented frames so driver should have to explicitly check ip_off > field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) > handle it. Let me check this. Is there a tool to send UDP/TCP with IP level options or do I need to write a small test program myself? Best regards Michael >=20 >> Best regards >> Michael=20 >>>=20 >>>=20 >>> ?=20 >>> Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, = 1983 >>>=20 >>>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Wed May 7 08:37:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AADBE901; Wed, 7 May 2014 08:37:52 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 761DC982; Wed, 7 May 2014 08:37:52 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id lj1so868560pab.28 for ; Wed, 07 May 2014 01:37:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=rTkUFd3928kYv/gHQ8imi5fd+k1T/7Csd54nknLvZ4s=; b=hElT+KmLCS2yrHrYMIFIxom44MS+iKwEsRgoqw87N6ZRo2up2PyYe+CfqnCzfj9WBT sdeq7q83HvtXINmFydxFz0RxQE1kj+aPgGCuW4BMGbVy0eAdGSNQq8QxkfCSuiKFZGi8 u6CWpKpD7u+OPsSSsoV0wzI9z8t0EtgGWd3d6kpWQa++roNEx8dIBVZHbZNHitBEwV6t YRIxPIlVEjWTKCnbt29INTcbnaj3FX5C3cCPL9jYWy2cPw92VMivTbhQw3NCAZ/yHT1a dGEuJ98jGwo6Bjp92GhTkJVJi+kKRIx8KQi9gFZJ6QTis2IOjyG290bHN1gbeNmeD9g9 p5gw== X-Received: by 10.66.148.70 with SMTP id tq6mr16550538pab.56.1399451871838; Wed, 07 May 2014 01:37:51 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id gj9sm1908440pbc.7.2014.05.07.01.37.49 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 07 May 2014 01:37:51 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 07 May 2014 17:37:51 +0900 From: Yonghyeon PYUN Date: Wed, 7 May 2014 17:37:51 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140507083751.GB1376@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 08:37:52 -0000 On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: > On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: > > > On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: > >> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: > >> > >>> > >>> On 02 May 2014, at 10:22 , Michael Tuexen wrote: > >>> > >>>> Dear all, > >>>> > >>>> during testing I found that FreeBSD head (on a raspberry pi) accepts SCTP packet > >>>> with bad checksums. After debugging this I figured out that this is a problem with > >>>> the csum_flags defined in mbuf.h. > >>>> > >>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is defined in mbuf.h: > >>>> #define CSUM_SCTP_VALID CSUM_L4_VALID > >>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is considered > >>>> to have a correct checksum. > >>>> > >>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and set CSUM_DATA_VALID in > >>>> csum_flags to indicate that the UDP/TCP should consider csum_data to figure out if > >>>> the packet has a correct checksum. The problem is that CSUM_DATA_VALID is defined as > >>>> #define CSUM_DATA_VALID CSUM_L4_VALID > >>>> In this case the semantic is not that the packet has a valid checksum, but the csum_data > >>>> field contains information. > >>>> > >>>> Now the following happens (on the raspberry pi the driver used is > >>>> dev/usb/net/if_smsc.c > >>>> > >>>> 1. A packet is received and if it is not too short, the checksum computed > >>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This happens > >>>> for all IP packets, not only for UDP and TCP packets. > >>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as CSUM_SCTP_VALID > >>>> and accepts the packet. So no SCTP checksum check ever happened. > >>>> > >>>> Alternatives to fix this: > >>>> > >>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP packets, since > >>>> it only makes sense in these cases. > >>> > >>> Wait, or for SCTP in cad the crc32 (I think it was) was actually checked but not otherwise. This is how it should be imho. It seems like a driver bug. > >> I went through the list of drivers and you are right, it seems to be a bug > >> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I can't tell. > >> > > > > I'm not sure how the controller computes TCP/UDP checksum values. > > It seems the publicly available data sheet was highly sanitized so > > it was useless to me. The comment in the driver says that the > Same for me... > > controller computes RX checksum after the IPv4 header to the end of > > ethernet frame. After seeing that comment, three questions popped > > up: > > > > 1. Is the controller smart enough to skip IP options header in > > TCP/UDP checksum offloading? > > 2. How controller handles UDP checksum value 0x0000(i.e. sender > > didn't compute UDP checksum)? > > 3. How the controller can compute TCP checksum of fragmented > > packets? > > > > Since you have the controller I guess it's easy to verify all > > cases. For case 3, I believe the controller can't handle > > fragmented frames so driver should have to explicitly check ip_off > > field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) > > handle it. > Let me check this. Is there a tool to send UDP/TCP with IP level options > or do I need to write a small test program myself? > I recall I used buggy ipsend of ipfilter package in the past but it would be more easy to write a simple test program or patch driver to generate those frames. From owner-freebsd-net@FreeBSD.ORG Wed May 7 12:30:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A0EE11A for ; Wed, 7 May 2014 12:30:34 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08C15111 for ; Wed, 7 May 2014 12:30:33 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id cm18so895630qab.36 for ; Wed, 07 May 2014 05:30:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=z+4NdTL8eaCgXc894fOM0rOWsPi/42JKgLh7eJAKfHU=; b=Q3YT+dnciTFv+VIc1FjAnoJUJ6qibeasyBkvKOHNDJrx1SgtIadiZ4Moiq26D6axan nH5/+QQZS4os+2EwxkdiFvf7xi/9+iKgGP34/4Cx+dS+4vtFWmBpdEEYr8m/+7FeEHGQ B4pzbwh0Fc2WAg7PSwJAxrkPj9XETZl0Mdmr3k6beJn3NgPB9FLbeT2jtqhmDDeqcr34 fYdVoCPQ3ezXl8uUiRmlLOu+GDAJBCkT0zQE1UXPzW0StRWYAKVJ0QDMgdzJeqH8us0U 3GO2FPWaL2A6leKX55Em4KXzshRmrDbi3egUw6lA62q5fFC8xoUUEW5Lxaq5b+4Ea4sR hXIw== MIME-Version: 1.0 X-Received: by 10.140.29.34 with SMTP id a31mr58595684qga.95.1399465832817; Wed, 07 May 2014 05:30:32 -0700 (PDT) Received: by 10.229.232.74 with HTTP; Wed, 7 May 2014 05:30:32 -0700 (PDT) In-Reply-To: References: Date: Wed, 7 May 2014 17:00:32 +0430 Message-ID: Subject: Re: running netmap-ipfw with real NICs From: Mahnaz Talebi To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 12:30:34 -0000 I use this scenario for test netmap-based ipfw with real NICs. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D sender (eth0:10.10.1.1) DUT (eth0:10.10.1.2) (eth1:10.10.2.2) receiver(10.10.2.3:eth0) ./pkt-gen -i eth0 -f rx -------------> ./kipfw netmap:eth0 netmap:eth1 --------------> ./pkt-gen -i eth0 -f tx where sender and receiver connect to DUT directly. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D I think that kipfw capture packets from DUT's eth0 (which come from sender's eth0) and forward them to it's eth1 (which connect directly to receiver's eth0), after apply policies from ipfw/ipfw. I expect that see packets that pass the ipfw's policies in receiver. Is this scenario and my expects true?! On Mon, Apr 28, 2014 at 7:28 PM, Luigi Rizzo wrote: > > > > On Mon, Apr 28, 2014 at 4:30 PM, Raimundo Santos wrote= : > >> On 28 April 2014 01:58, Mahnaz Talebi wrote: >> >> > I am trying to run netmap-based ipfw with real NICs >> >> >> Hello, >> >> there are some drivers that does not support netmap yet. >> > > =E2=80=8Bthanks for the answer but it wasn't that, i spoke to Mahnaz > and he was just running out of memory, as dmesg showed. > > cheers > luigi > =E2=80=8B > >> >> > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > From owner-freebsd-net@FreeBSD.ORG Wed May 7 12:35:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A07E221 for ; Wed, 7 May 2014 12:35:39 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07C3D1B9 for ; Wed, 7 May 2014 12:35:38 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id i17so985620qcy.11 for ; Wed, 07 May 2014 05:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Tmv74I20bh0GXu5VgnW7onAlY0Hj/7p4mbDaOUvolEQ=; b=kwUL217EV8rnCEWhaVQgRCwidXeZHGfVgn32oX9WMXZK/uy2p8XjhJ9RGxrZAfDyO1 p69Mr1kbS2m54SRqX3qO+ynwadSphawtrnleCV7oSE7AOFSwKwt2/KUREPU9ntK6Fazx VGJ7OPkKs7usILamIo+O/JtuKuBSRo2/sj9TLeuPpjemxhfuqeatb7qMdvBB2Qy/c+hI oHfLLkoNIZLF6Bz50iYHPYVyIBzIGaZ9uMLnwf9PcAZ3nkFHK4X+1rH4RLyLEBvi3u0+ wNc2+3PGZ36tAsjrYGubFlNhxFpBENXo71hmwfGR6cLT1tuUiP9NX4nmRKCnnZ3kDVB+ kq7Q== MIME-Version: 1.0 X-Received: by 10.140.91.161 with SMTP id z30mr59532647qgd.65.1399466138181; Wed, 07 May 2014 05:35:38 -0700 (PDT) Received: by 10.229.232.74 with HTTP; Wed, 7 May 2014 05:35:38 -0700 (PDT) In-Reply-To: References: Date: Wed, 7 May 2014 17:05:38 +0430 Message-ID: Subject: Re: running netmap-ipfw with real NICs From: Mahnaz Talebi To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 12:35:39 -0000 Sorry for my inaccuracy! I use "./pkt-gen -i eth0 -f tx" in sender and " ./pkt-gen -i eth0 -f rx" in receiver. On Wed, May 7, 2014 at 5:00 PM, Mahnaz Talebi wrote= : > I use this scenario for test netmap-based ipfw with real NICs. > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > sender (eth0:10.10.1.1) DUT (eth0:10.10.1.2) > (eth1:10.10.2.2) receiver(10.10.2.3:eth0) > ./pkt-gen -i eth0 -f rx -------------> ./kipfw netmap:eth0 > netmap:eth1 --------------> ./pkt-gen -i eth0 -f tx > where sender and receiver connect to DUT directly. > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > I think that kipfw capture packets from DUT's eth0 (which come from > sender's eth0) and forward them to it's eth1 (which connect directly to > receiver's eth0), after apply policies from ipfw/ipfw. > I expect that see packets that pass the ipfw's policies in receiver. > > Is this scenario and my expects true?! > > > > > > > > On Mon, Apr 28, 2014 at 7:28 PM, Luigi Rizzo wrote: > >> >> >> >> On Mon, Apr 28, 2014 at 4:30 PM, Raimundo Santos wrot= e: >> >>> On 28 April 2014 01:58, Mahnaz Talebi wrote: >>> >>> > I am trying to run netmap-based ipfw with real NICs >>> >>> >>> Hello, >>> >>> there are some drivers that does not support netmap yet. >>> >> >> =E2=80=8Bthanks for the answer but it wasn't that, i spoke to Mahnaz >> and he was just running out of memory, as dmesg showed. >> >> cheers >> luigi >> =E2=80=8B >> >>> >>> >> -- >> -----------------------------------------+------------------------------= - >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------= - >> > > From owner-freebsd-net@FreeBSD.ORG Wed May 7 15:55:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC614C14 for ; Wed, 7 May 2014 15:55:56 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B63DA62 for ; Wed, 7 May 2014 15:55:56 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id D292E139E4 for ; Wed, 7 May 2014 12:47:24 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1399477640; x=1400341641; bh=EwW7yNDlCEHL0I3KDyQD5AIldxN59ym3OA5 /LHqlFBA=; b=awyzKqh9saCcbWTImNgnfygzoSyTTOwNvmzwSfx+OGX1zyy4IAm XZpm7tshRQ4sW+Uzjsbi2y4VNVA9T4GWHEBrLo8m19ITRrNe9qRSILle1D24rSoy xTIeS1PVZiJeyZiUrJ8spo6LA4Qxw7LZ+aUPgZeYGZKRTXNgNNpColI8= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jAxNDViCGomy for ; Wed, 7 May 2014 12:47:20 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id B714B139C3 for ; Wed, 7 May 2014 12:47:19 -0300 (BRT) Message-ID: <536A5580.6020502@bsdinfo.com.br> Date: Wed, 07 May 2014 12:47:12 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Problem with removing mac address from arptable on 10-stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 15:55:56 -0000 Hi all, I'm having this problemon my FreeBSD 10-STABLE: (root@rt01)[~]# arp -an|grep 187.xxx.216.252 ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] (root@rt01)[~]# arp -d 187.xxx.216.252 delete: cannot locate 187.xxx.216.252 FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r265409: Tue May 6 01:14:05 BRT 2014 root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM amd64 This is a real problem or am I missing something? Thanks and best regards, Gondim From owner-freebsd-net@FreeBSD.ORG Wed May 7 18:18:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7AFBEC27 for ; Wed, 7 May 2014 18:18:16 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16879ADD for ; Wed, 7 May 2014 18:18:15 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id u56so1417677wes.28 for ; Wed, 07 May 2014 11:18:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=bzujWEbiuwzSyBB5btYKewGNSJnWtAz3Kq/7NfURplM=; b=FZsi5LqVu11IR27BqC5VucGp1gSt4IriDVz5tpRi8/9LpV+dR16imdt85NHCeZqJoC TbjDI9gfEgtcUxeEucfXq4xl5b845aOtu+QjVZ7LL9EEaHjYJfQVuvr2E+b1sfnLU2Jp MQVonxL35n2lVifnbbunsBPdgtJDr1NSddFcQ7B4OF5fEvEUGQh4tFNAsxBlnBTyxVzP R0gt4Zvs6hOojSy14o/nORZiYQ8StTuPFNWHJG4Hi3d+jxl9WCmq2KxcsF+XHoGrFUy/ Gz77oIOLn/+asUKqs3S1lrbgqf3Lu5wiN8V0FUO/NThVwhkNzimTWhUTPRvPALp+7J6K gtrQ== MIME-Version: 1.0 X-Received: by 10.180.36.66 with SMTP id o2mr9166508wij.40.1399486694307; Wed, 07 May 2014 11:18:14 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Wed, 7 May 2014 11:18:14 -0700 (PDT) In-Reply-To: <536A5580.6020502@bsdinfo.com.br> References: <536A5580.6020502@bsdinfo.com.br> Date: Wed, 7 May 2014 12:18:14 -0600 X-Google-Sender-Auth: AKayPlbY_W3SrDR6BpVfCf9d3V0 Message-ID: Subject: Re: Problem with removing mac address from arptable on 10-stable From: Alan Somers To: Marcelo Gondim Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 18:18:16 -0000 On Wed, May 7, 2014 at 9:47 AM, Marcelo Gondim wrote: > Hi all, > > I'm having this problemon my FreeBSD 10-STABLE: > > (root@rt01)[~]# arp -an|grep 187.xxx.216.252 > ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] > > (root@rt01)[~]# arp -d 187.xxx.216.252 > delete: cannot locate 187.xxx.216.252 > > FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r265409: Tue > May 6 01:14:05 BRT 2014 root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM > amd64 > > This is a real problem or am I missing something? Are you using multiple FIBs? There are been bugs relating to FIBs and the ARP table. If you're unsure, check "sysctl net.fibs" > > Thanks and best regards, > Gondim > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed May 7 18:57:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE89EA63 for ; Wed, 7 May 2014 18:57:55 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C378EC9 for ; Wed, 7 May 2014 18:57:55 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 0AF42139E4 for ; Wed, 7 May 2014 15:57:59 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1399489074; x=1400353075; bh=fDw/dJe0lWykCh+a+2QtJT/7bxr3fVy3LqH wUta6TBk=; b=K1VAUKSbxJ24kPvykpncDeXWqaovEZq5Hlsi9X0XZh6xdOro+BV nnd5DdNHug9vY1cVDNwQi3+FETLpzNapu9ELJng1mmQ8t8r3bDuarjunqEuidMbt V+1InnB+YozkTMgSuxufuj/2+L+lQaUGwSZCZ/dxsTvSQw/AHwdDObCA= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mgsxbLzkdZ3l for ; Wed, 7 May 2014 15:57:54 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 7FD6C139C3 for ; Wed, 7 May 2014 15:57:54 -0300 (BRT) Message-ID: <536A822A.7040404@bsdinfo.com.br> Date: Wed, 07 May 2014 15:57:46 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Problem with removing mac address from arptable on 10-stable References: <536A5580.6020502@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 18:57:55 -0000 Em 07/05/14 15:18, Alan Somers escreveu: > On Wed, May 7, 2014 at 9:47 AM, Marcelo Gondim wrote: >> Hi all, >> >> I'm having this problemon my FreeBSD 10-STABLE: >> >> (root@rt01)[~]# arp -an|grep 187.xxx.216.252 >> ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] >> >> (root@rt01)[~]# arp -d 187.xxx.216.252 >> delete: cannot locate 187.xxx.216.252 >> >> FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r265409: Tue >> May 6 01:14:05 BRT 2014 root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM >> amd64 >> >> This is a real problem or am I missing something? > Are you using multiple FIBs? There are been bugs relating to FIBs and > the ARP table. If you're unsure, check "sysctl net.fibs" Hi Alan, Thanks for your reply. # sysctl net.fibs net.fibs: 2 I'll change it to 1 and see if the problem will happen. Thanks! From owner-freebsd-net@FreeBSD.ORG Wed May 7 23:43:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D12519D for ; Wed, 7 May 2014 23:43:10 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A3F4E01 for ; Wed, 7 May 2014 23:43:10 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id DDC92139E4 for ; Wed, 7 May 2014 20:43:13 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1399506189; x=1400370190; bh=ioaUtr8/b/PNM3mLg7xJNOseb08vVXH2MUK /zjttHLk=; b=UlTjt28Qvl3mxQPBQILA87dM5h6t9ckq9LouC5QDpwU6NiE6PQK +deOthoeL8169pCWDEN1bFhxArZPJDi9DR7MSqTEN8I0S06XxO1m9mit3Qz2a5N6 xTXpQorCrMyGufSptvZczg1Bhdg2bnpARcwXX/8mgEYqEYELQCG82PBc= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KVjrBHzOGlOD for ; Wed, 7 May 2014 20:43:09 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 73CE2139C3 for ; Wed, 7 May 2014 20:43:09 -0300 (BRT) Message-ID: <536AC505.80107@bsdinfo.com.br> Date: Wed, 07 May 2014 20:43:01 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Problem with removing mac address from arptable on 10-stable References: <536A5580.6020502@bsdinfo.com.br> <536A822A.7040404@bsdinfo.com.br> In-Reply-To: <536A822A.7040404@bsdinfo.com.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 23:43:10 -0000 Em 07/05/14 15:57, Marcelo Gondim escreveu: > Em 07/05/14 15:18, Alan Somers escreveu: >> On Wed, May 7, 2014 at 9:47 AM, Marcelo Gondim >> wrote: >>> Hi all, >>> >>> I'm having this problemon my FreeBSD 10-STABLE: >>> >>> (root@rt01)[~]# arp -an|grep 187.xxx.216.252 >>> ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] >>> >>> (root@rt01)[~]# arp -d 187.xxx.216.252 >>> delete: cannot locate 187.xxx.216.252 >>> >>> FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 >>> r265409: Tue >>> May 6 01:14:05 BRT 2014 >>> root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM >>> amd64 >>> >>> This is a real problem or am I missing something? >> Are you using multiple FIBs? There are been bugs relating to FIBs and >> the ARP table. If you're unsure, check "sysctl net.fibs" > Hi Alan, > > Thanks for your reply. > > # sysctl net.fibs > net.fibs: 2 > > I'll change it to 1 and see if the problem will happen. > > Thanks! Hi Alan, did not work :( It also happens the errors below when I try to add a host in the arp table: # arp -an|grep 187.xxx ? (187.xxx.219.28) at 00:1e:67:77:de:62 on vlan4 permanent [vlan] # arp -s 187.xxx.223.254 5c:e0:f6:00:12:8e cannot intuit interface index and type for 187.xxx.223.254 # arp -s 187.xxx.216.253 5c:e0:f6:00:11:2c cannot intuit interface index and type for 187.xxx.216.253 # arp -s 187.xxx.223.253 5c:e0:f6:00:12:8b cannot intuit interface index and type for 187.xxx.223.253 # arp -s 187.xxx.216.254 5c:e0:f6:00:11:2f cannot intuit interface index and type for 187.xxx.216.254 # sysctl net.fibs net.fibs: 1 dmesg: arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 arpresolve: can't allocate llinfo for 187.xxx.216.254 Thanks and best regards From owner-freebsd-net@FreeBSD.ORG Thu May 8 02:05:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2472C1F4 for ; Thu, 8 May 2014 02:05:08 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B205DAF5 for ; Thu, 8 May 2014 02:05:07 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id w61so1811346wes.40 for ; Wed, 07 May 2014 19:05:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ddXWLYBOnJmsk5hM4P3K/2cymVUKeoX7qVs67ExUHow=; b=cH2s5W73G8nsb1DRSKDZSD9ZqP1gzBcXzZJ/Vj/whaHLE+urprGIxu1dFV3X4+eEhn tmDZes5xopUeRqjK9h9bR5kqLk44O3jmwwFwYYcVqokCNARMNTkEBFjJ5dTjxHNs8vDn 6sMrr5qCn8DVSALVmGEklQG8YHd+PXHubo+mASyoU5II7SvRxDie5Ek0dSzzRAmqJY09 Pm7PcxjAJ/behqvdSa83qdQlDiiSKVKnTPs3HQZBU7xqrXqCvEs0uHdiMGh9zvG1KQpY Lnmd4SFcfK1enmn+KR0/J4h1QzgzOUKBzImUKS0VigU7JmMJq/Hm+ya4F6OyTsPo9Wn7 W0Qg== MIME-Version: 1.0 X-Received: by 10.180.212.107 with SMTP id nj11mr1017690wic.40.1399514704967; Wed, 07 May 2014 19:05:04 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Wed, 7 May 2014 19:05:04 -0700 (PDT) In-Reply-To: <536AC505.80107@bsdinfo.com.br> References: <536A5580.6020502@bsdinfo.com.br> <536A822A.7040404@bsdinfo.com.br> <536AC505.80107@bsdinfo.com.br> Date: Wed, 7 May 2014 20:05:04 -0600 X-Google-Sender-Auth: EsqwiWldkhBAr8K7YYJOn9c53gQ Message-ID: Subject: Re: Problem with removing mac address from arptable on 10-stable From: Alan Somers To: Marcelo Gondim Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 02:05:08 -0000 On Wed, May 7, 2014 at 5:43 PM, Marcelo Gondim wrote: > Em 07/05/14 15:57, Marcelo Gondim escreveu: > >> Em 07/05/14 15:18, Alan Somers escreveu: >>> >>> On Wed, May 7, 2014 at 9:47 AM, Marcelo Gondim >>> wrote: >>>> >>>> Hi all, >>>> >>>> I'm having this problemon my FreeBSD 10-STABLE: >>>> >>>> (root@rt01)[~]# arp -an|grep 187.xxx.216.252 >>>> ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] >>>> >>>> (root@rt01)[~]# arp -d 187.xxx.216.252 >>>> delete: cannot locate 187.xxx.216.252 >>>> >>>> FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r265409: >>>> Tue >>>> May 6 01:14:05 BRT 2014 >>>> root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM >>>> amd64 >>>> >>>> This is a real problem or am I missing something? >>> >>> Are you using multiple FIBs? There are been bugs relating to FIBs and >>> the ARP table. If you're unsure, check "sysctl net.fibs" >> >> Hi Alan, >> >> Thanks for your reply. >> >> # sysctl net.fibs >> net.fibs: 2 >> >> I'll change it to 1 and see if the problem will happen. net.fibs defaults to 1. Why was it set to 2? It must've been deliberately set to 2 on your system, and changing it to 1 will probably break something. >> >> Thanks! > > Hi Alan, > > did not work :( > > It also happens the errors below when I try to add a host in the arp table: > > # arp -an|grep 187.xxx > ? (187.xxx.219.28) at 00:1e:67:77:de:62 on vlan4 permanent [vlan] > > # arp -s 187.xxx.223.254 5c:e0:f6:00:12:8e > cannot intuit interface index and type for 187.xxx.223.254 > # arp -s 187.xxx.216.253 5c:e0:f6:00:11:2c > cannot intuit interface index and type for 187.xxx.216.253 > # arp -s 187.xxx.223.253 5c:e0:f6:00:12:8b > cannot intuit interface index and type for 187.xxx.223.253 > # arp -s 187.xxx.216.254 5c:e0:f6:00:11:2f > cannot intuit interface index and type for 187.xxx.216.254 > > # sysctl net.fibs > net.fibs: 1 > > dmesg: > > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 > arpresolve: can't allocate llinfo for 187.xxx.216.254 Please show me your network settings from /etc/rc.conf. It also might be helpful to see the output of "netstat -rn" and "setfib 1 netstat -rn". One guess as to your problem would be that your subnet mask is configured wrong, or that other hosts on your vlan are configured with a different subnet mask or even an entirely different network prefix. > > Thanks and best regards > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu May 8 04:17:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ECFE5FCE for ; Thu, 8 May 2014 04:17:57 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B38AC79F for ; Thu, 8 May 2014 04:17:57 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id E2F50139E5 for ; Thu, 8 May 2014 01:18:00 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1399522677; x=1400386678; bh=4iyQIAeHvYhT GiRs0kAL4BlRaCLKVIVJe8Gioq//snY=; b=jtg+1UjI5fm8HWUGO+4NNdzoAKYg Blm0h2KO6CzVUXedkYR45Lyhx3N+phb9Ga4HZyGawwbsWcKrpHsZJE4a0pEhQbml q5sIU0U7FxtFMp+JPyBNwpzrf9p1L1KNllCNRp2exCyGdQknDJHJbUyPHdNkPzor Y55niW70/OBdcQU= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2YOaaek7qb0T for ; Thu, 8 May 2014 01:17:57 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 2A2CF139C3; Thu, 8 May 2014 01:17:55 -0300 (BRT) Message-ID: <536B056B.1080009@bsdinfo.com.br> Date: Thu, 08 May 2014 01:17:47 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Alan Somers Subject: Re: Problem with removing mac address from arptable on 10-stable References: <536A5580.6020502@bsdinfo.com.br> <536A822A.7040404@bsdinfo.com.br> <536AC505.80107@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 04:17:58 -0000 Em 07/05/14 23:05, Alan Somers escreveu: > On Wed, May 7, 2014 at 5:43 PM, Marcelo Gondim wrote: >> Em 07/05/14 15:57, Marcelo Gondim escreveu: >> >>> Em 07/05/14 15:18, Alan Somers escreveu: >>>> On Wed, May 7, 2014 at 9:47 AM, Marcelo Gondim >>>> wrote: >>>>> Hi all, >>>>> >>>>> I'm having this problemon my FreeBSD 10-STABLE: >>>>> >>>>> (root@rt01)[~]# arp -an|grep 187.xxx.216.252 >>>>> ? (187.xxx.216.252) at 5c:e0:f6:00:11:29 on vlan4 permanent [vlan] >>>>> >>>>> (root@rt01)[~]# arp -d 187.xxx.216.252 >>>>> delete: cannot locate 187.xxx.216.252 >>>>> >>>>> FreeBSD rt01.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r265409: >>>>> Tue >>>>> May 6 01:14:05 BRT 2014 >>>>> root@rt01.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM >>>>> amd64 >>>>> >>>>> This is a real problem or am I missing something? >>>> Are you using multiple FIBs? There are been bugs relating to FIBs and >>>> the ARP table. If you're unsure, check "sysctl net.fibs" >>> Hi Alan, >>> >>> Thanks for your reply. >>> >>> # sysctl net.fibs >>> net.fibs: 2 >>> >>> I'll change it to 1 and see if the problem will happen. > net.fibs defaults to 1. Why was it set to 2? It must've been > deliberately set to 2 on your system, and changing it to 1 will > probably break something. I had defined in the kernel but never used. Right now is with value 1. > >>> Thanks! >> Hi Alan, >> >> did not work :( >> >> It also happens the errors below when I try to add a host in the arp table: >> >> # arp -an|grep 187.xxx >> ? (187.xxx.219.28) at 00:1e:67:77:de:62 on vlan4 permanent [vlan] >> >> # arp -s 187.xxx.223.254 5c:e0:f6:00:12:8e >> cannot intuit interface index and type for 187.xxx.223.254 >> # arp -s 187.xxx.216.253 5c:e0:f6:00:11:2c >> cannot intuit interface index and type for 187.xxx.216.253 >> # arp -s 187.xxx.223.253 5c:e0:f6:00:12:8b >> cannot intuit interface index and type for 187.xxx.223.253 >> # arp -s 187.xxx.216.254 5c:e0:f6:00:11:2f >> cannot intuit interface index and type for 187.xxx.216.254 >> >> # sysctl net.fibs >> net.fibs: 1 >> >> dmesg: >> >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 >> arpresolve: can't allocate llinfo for 187.xxx.216.254 > Please show me your network settings from /etc/rc.conf. It also might > be helpful to see the output of "netstat -rn" and "setfib 1 netstat > -rn". One guess as to your problem would be that your subnet mask is > configured wrong, or that other hosts on your vlan are configured with > a different subnet mask or even an entirely different network prefix. Thanks again for your help. :) Instead of using rc.conf I use the /etc/start_if.xxx. /etc/start_if.em0: /sbin/ifconfig disc0 create /sbin/ifconfig em0 159.XXX.51.98/30 /sbin/ifconfig em0 inet6 2804:XXXX:FFFF:FFD8::2/64 /sbin/ifconfig disc0 127.0.0.254 /etc/start_if.em1: /sbin/ifconfig em1 64.XXX.195.70/30 /sbin/ifconfig em1 inet6 2001:XXXX:2001:1001::96/126 /etc/start_if.em2: /sbin/ifconfig lagg1 create /sbin/ifconfig em2 up /sbin/ifconfig em5 up /sbin/ifconfig lagg1 laggproto lacp laggport em2 laggport em5 /sbin/ifconfig lagg1 up /sbin/ifconfig vlan0 create /sbin/ifconfig vlan1 create /sbin/ifconfig vlan2 create /sbin/ifconfig vlan3 create /sbin/ifconfig vlan0 186.XXX.48.1/27 vlan 3081 vlandev lagg1 /sbin/ifconfig vlan0 inet6 2804:XXXX:DEAD::1/64 /sbin/ifconfig vlan1 177.XXX.240.254/27 vlan 3082 vlandev lagg1 /sbin/ifconfig vlan1 inet6 2804:XXXX:CAFE::1/64 /sbin/ifconfig vlan2 186.XXX.54.1/27 vlan 2126 vlandev lagg1 /sbin/ifconfig vlan2 inet6 2804:XXXX:CADE::1/64 /sbin/ifconfig vlan3 186.XXX.61.1/27 vlan 3088 vlandev lagg1 /sbin/ifconfig vlan3 inet6 2804:XXXX:BAD::1/64 /etc/start_if.em3: /sbin/ifconfig em3 186.YYY.1.150/30 /etc/start_if.em6: /sbin/ifconfig em6 177.XXX.255.1/29 /etc/start_if.em7: /sbin/ifconfig em7 up /sbin/ifconfig vlan4 create /sbin/ifconfig vlan5 create /sbin/ifconfig vlan4 187.XXX.219.28/21 vlan 1441 vlandev em7 /sbin/ifconfig vlan5 inet6 2001:XXXX::219:28/64 vlan 1442 vlandev em7 There are over 487,000 routes on this server. The list would be too big to put here in this email. Is a BGP router. My /etc/rc.conf: hostname="rt01.xxxxxx.com.br" keymap="br275.iso.acc" ipv6_activate_all_interfaces="YES" sshd_enable="YES" syslogd_flags="-s -s" ntpdate_enable="YES" ntpdate_hosts="0.freebsd.pool.ntp.org" firewall_enable="YES" firewall_script="/etc/beastiefrw/beastiefrw" snmpd_enable="YES" snmpd_flags="-a" snmpd_conffile="/usr/local/share/snmp/snmpd.conf" fsck_y_enable="YES" From owner-freebsd-net@FreeBSD.ORG Thu May 8 08:10:53 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20C3024D; Thu, 8 May 2014 08:10:53 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6A0BC9C; Thu, 8 May 2014 08:10:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:Content-Type:MIME-Version:Message-ID:Subject:Cc:To:From:Date; bh=uO1LXJR3hUhfnw1FDOHlOJUEwkUhD/soodyWwig1wnQ=; b=uRQV+HtjOnAMQZ8YtIOK/TOvu/hZFTIqnhu1oYCQWwzBXSdUKiwUgxAE1ZDVP4pFqRZI56RLLnGZXQsb1iLM7lKaR/nQ+EqOPD9CzQu0VWA9UU3oIriezWVeoQTa/nvjzzS+rU9zUZc6ZtaIqKs/0ykjxaI57v5yKGtzxb00AdaV18PPGePuYyUPGCTVErQ3/o2b37umQQMfBppQJzRFcQWMoxQXOcnuICbT1mD2o792auFtJx0VJyZly7ef9sKsmj0/Z0hw6z3KzqEL15YK21ldKoD9lLOHiHopoApSAgjuTLIrSUX8rpAmvQRwXZJKESB8viUDCNfpG0srL/yhceratAFFtaBzYl75sw5C0mA6CYuTzeXJXjIYaMfOrc8InnuhHMPCEQaU/lpIrSo5zq5GZizGmuKjf0JXQSYTIxUCjUqA6/PxMoVCO0IG3GiL+MHtFdRooKuZukKQIYiUel51KK3pBPZwNdMDzEA5qBFsml25ioCIuzp4F5119t5ojx1s0nhynRqVzqAZqIb0GN+gMZC7YWOuqNZ/jsBNsustc2B5VAvFJUaHMDvxtTZoTEgprl/g9I65ztWArdwJy/12QQfbKnlQkGLHc3wZyiEhq74TxvxlvbEO1CIPmjy2SpXWPN8NBPRGgqPaobivYGhRROy7UdoUULGf8lOJjm0=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WiJPm-000CSu-Fe; Thu, 08 May 2014 12:10:50 +0400 Date: Thu, 8 May 2014 12:10:48 +0400 From: Eygene Ryabinkin To: net@freebsd.org Subject: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2fHTh5uZTiUOsy+g" Content-Disposition: inline Sender: rea@codelabs.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 08:10:53 -0000 --2fHTh5uZTiUOsy+g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Good day. As current CARP implementation somewhat hijacks OUI MAC space for VRRP (00:00:5e:00:01:$VRID) and this sometimes create problems, because routing people tend to be different from the ones that run CARP clusters, so their VRID/VHID can clash inside single L2 domain, and this often leads to breakages (because of same MAC values for the different clustered instances), I have two proposals: - I'll do a patch for carp(4) that will allow it to use configurable OUI from a sysctl knob (first 5 bytes of OUI); - assign 8 bit block from current FreeBSD OUI (58-9c-fc), http://standards.ieee.org/develop/regauth/oui/oui.txt for exclusive use for CARP at FreeBSD to avoid VRRP clashes (and therefore Ethernet MAC wars). There will be, of course, other activites, mostly for extending manual pages, but I'm up to it. Any thoughts about this? By the way, does anyone knows where FreeBSD's OUI space division is further documented? Like IANA's one, http://www.iana.org/assignments/ethernet-numbers/ethernet-numbers.xhtml --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] Please, CC me: I am not subscribed to this list. --2fHTh5uZTiUOsy+g Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNrPAhfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PvTAwD9Hhp+toF83JFYVzyQbAx5C7EH yr+Vv4jJkL3ejGGuO30A/ioD/m2OdyNnuvTMLv8yptxlNSxr4dBxlp0nk3BVnwWv =Y2eE -----END PGP SIGNATURE----- --2fHTh5uZTiUOsy+g-- From owner-freebsd-net@FreeBSD.ORG Thu May 8 09:37:52 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A03E7FD9; Thu, 8 May 2014 09:37:52 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 53BDE6D5; Thu, 8 May 2014 09:37:51 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 325D225D3894; Thu, 8 May 2014 09:37:43 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 54CBDC22BDA; Thu, 8 May 2014 09:37:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id ARAy1YwkttbK; Thu, 8 May 2014 09:37:40 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (unknown [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id AD254C22BCF; Thu, 8 May 2014 09:37:39 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that From: "Bjoern A. Zeeb" In-Reply-To: Date: Thu, 8 May 2014 09:37:37 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> References: To: Eygene Ryabinkin X-Mailer: Apple Mail (2.1874) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 09:37:52 -0000 On 08 May 2014, at 08:10 , Eygene Ryabinkin wrote: > As current CARP implementation somewhat hijacks OUI MAC space for VRRP > (00:00:5e:00:01:$VRID) and this sometimes create problems, because > routing people tend to be different from the ones that run CARP > clusters, so their VRID/VHID can clash inside single L2 domain, and > this often leads to breakages (because of same MAC values for the > different clustered instances), It often leads to a bit of logging about =93hey I don=92t know this = =91version' of VRRP=94 (well yeah) on some $vendor devices who should = know better by now. Apart from that I thought the different version number was sufficient = (as it is for other protocols, and so have others who actually started = to write a draft for an independent submission early last year and = stalled on it). I am actually not in the loop on what we ended up with = in 10 but I guess given the new CARP started to understand the old stuff = glebius did not end up bumping it finally in FreeBSD? So the problem = might remain that we are on a conflicting =93VRRP/CARP version=94? In addition you should, of course, use secrets with the VRRP/CARP as = otherwise you deserve to have real clashes that do unexpected things to = your deployment. Just my -1cts /bz =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Thu May 8 09:51:03 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 206A6484; Thu, 8 May 2014 09:51:03 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B5ED1817; Thu, 8 May 2014 09:51:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=wW9bdyRZiGyKe+op0mhlaS1KgFDidDgAiksSHYLvzSo=; b=kR3yOLUxlV9WNVjsraZPGaV0FT9cNiMlo4BNr206pTRjHnJ8C9GuTQdS+tplXq9vMgBnKSHXerptOx380fk6qrT0+iNYyiDnCCssFKVqW+hVwFYDV3vjgANo031RKYw4LePediCXNeU7BK5AxdB5nRdQQa152OTKzilZMh61XZxIuzuVNCE/fnvT6Uef63T3ywzmc1rnQBArGjgIhGIFzrk0J5DEfEir8k9BN2bbxsP3Q75BqoMHhIzP7v9kfv91fLPs1MZl4fKP/bH7Q0POMVAIk1JSj5VyN3YhUI393V89CzSP82WVJXtQCY80bZFtdDaNT8L1ZvB536T/csXt0w0KmL5jahBKacRtmpGNZn7mMf4Hwqwtmh9bPtw8k97+iRah6n+HQTVSbTdGAUww3Z2aO1akH1P5kRiIE7L2AsWBPl1ACjBgAs82OdkepGJKrscQnkTJ6tjWClYfNs71/cTdR8J8JaKVWt7IevyEWugc0Eab/bA3L2UI8HZj+F8hpuGiLVm517vd2nmzp5Ox7dnAXaydjaCmKDNCRI9BkTAAi5/v0G62FGBDw4rIZx5oWnqP5rmJCoM6G6Ql+y4No/oUwA3JU+8vDVLGZnONBq9sR+dcLLjmq4YGDY5XWt7+/A9ddNOUSmJQhobNDQ1dhYdVci+Ip7AMQWJWo8VV2aA=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WiKyi-000EVv-Kg; Thu, 08 May 2014 13:51:00 +0400 Date: Thu, 8 May 2014 13:50:58 +0400 From: Eygene Ryabinkin To: "Bjoern A. Zeeb" Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="s/l3CgOIzMHHjg/5" Content-Disposition: inline In-Reply-To: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> Sender: rea@codelabs.ru Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 09:51:03 -0000 --s/l3CgOIzMHHjg/5 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Bjoern, good day. Thu, May 08, 2014 at 09:37:37AM +0000, Bjoern A. Zeeb wrote: > On 08 May 2014, at 08:10 , Eygene Ryabinkin wrote: > > As current CARP implementation somewhat hijacks OUI MAC space for VRRP > > (00:00:5e:00:01:$VRID) and this sometimes create problems, because > > routing people tend to be different from the ones that run CARP > > clusters, so their VRID/VHID can clash inside single L2 domain, and > > this often leads to breakages (because of same MAC values for the > > different clustered instances), >=20 > It often leads to a bit of logging about =E2=80=9Chey I don=E2=80=99t kno= w this > =E2=80=98version' of VRRP=E2=80=9D (well yeah) on some $vendor devices wh= o should > know better by now. Here you're talking about protocol (112) and version number that CARP packets use (version number is one higher than VRRP's one). This was fixed by most vendors, most notably Cisco. > Apart from that I thought the different version number was sufficient The thing is that both VRRP and CARP packets use MAC address (on Ethernet at least) that equals to 00:00:5e:00:01:$VRID. So in case that $VRID is the same and VRRP and CARP admins aren't aware of each other, there will be MAC conflict, so L2 packets will be switched in a "funny" manner. So, it isn't about the "control plane" messages that carry CARP/VRRP protocol type and numbers, but rather than the "data plane" messages =66rom CARP/VRRP nodes saying "hi, I am here" in replies for ARP requests and switches caching MACs in their FDB on the ports into which replies ingress. > (as it is for other protocols, and so have others who actually > started to write a draft for an independent submission early last > year and stalled on it). I am actually not in the loop on what we > ended up with in 10 but I guess given the new CARP started to > understand the old stuff glebius did not end up bumping it finally > in FreeBSD? So the problem might remain that we are on a > conflicting =E2=80=9CVRRP/CARP version=E2=80=9D? No, we're conflicting with VRRP on the MAC address space. And, as I understand, CARP in 10 hadn't changed protocol in any way, it just refurbished now CARP instances are configured and attached to the interfaces. Could be wrong here, though. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] Please, CC me: I am not subscribed to this list. --s/l3CgOIzMHHjg/5 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNrU4JfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PvK8AD8CX6NSJ31vc2dqm1ox+PUFakQ 5uJhezXz134Vp1BHUYABAJLoH3gLYMULjgDIEYEIB9xxBXJcIXjuVgspIJnan3Dv =EK2n -----END PGP SIGNATURE----- --s/l3CgOIzMHHjg/5-- From owner-freebsd-net@FreeBSD.ORG Thu May 8 10:28:26 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C68D19CB; Thu, 8 May 2014 10:28:26 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 78E69B02; Thu, 8 May 2014 10:28:26 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 22E1D25D3A00; Thu, 8 May 2014 10:28:23 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id E433CC22BDA; Thu, 8 May 2014 10:28:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id HdoBrQHGYCUQ; Thu, 8 May 2014 10:28:21 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (unknown [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id BFB4EC22BCF; Thu, 8 May 2014 10:28:20 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that From: "Bjoern A. Zeeb" In-Reply-To: Date: Thu, 8 May 2014 10:28:19 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <03D03A86-A5C4-4C0D-822E-D151AB272F35@lists.zabbadoz.net> References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> To: Eygene Ryabinkin X-Mailer: Apple Mail (2.1874) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 10:28:26 -0000 On 08 May 2014, at 09:50 , Eygene Ryabinkin wrote: >> Apart from that I thought the different version number was sufficient >=20 > The thing is that both VRRP and CARP packets use MAC address (on > Ethernet at least) that equals to 00:00:5e:00:01:$VRID. So in case > that $VRID is the same and VRRP and CARP admins aren't aware of each > other, there will be MAC conflict, so L2 packets will be switched > in a "funny" manner. How=92s that different routing guys running VRRP on the routers and server guys running vrrp on the servers and conflicting on the ID? It=92s a management problem in the administrative broadcast domain not a CARP vs. VRRP problem. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Thu May 8 10:40:56 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3AA0CED; Thu, 8 May 2014 10:40:56 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F0C4BFA; Thu, 8 May 2014 10:40:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=LcqYptTvRlB9iIh2lAHX5QNTe+boNAHTK7W6ysHoGOQ=; b=i7m13m3kU+kRw3zJnCkWHYoAJVmtaKBdnHgIAi16e5vo21HltH7OAEHo1ieA0nPhd8l1Ir82S23PG1LR+w4upY4+TYqS3uCCnqOa7jGpYQVDeJUEcmajd8jZFsTSo2qq6suVcEsfnAAnL9Yg1mqnrEbwT8K9EIci7tLQhSRoM04tyCm7BE1sZxeyWFA91u4Se8DP9cKZjoDs6qV+5Puhi2c0CU7P/IuD/glMbCzl06ygx2vFUxfyQ5SYHiVA2sTL+N8gB/TAID+yQ7wDao5hbb9LyiDdBnSMI1EPs28nFpB/czCjLNFCjerwXqeMw0oPfuZHLRZT+jMLuyWBp2p3UmdSCEScTsqNHu2KghhwEqf4Vbd4dT4yDzQH+sfsj82qVg3PyJP+MOTAKLT4jBcBSHpuieqF4kVEo5IE80dLg0rBDE1ox8Q6vpL3GXrBV5m7YZYLTPuTre6c2skPWhAeFqcPGf3Agouc/Bjk8X3qKqN9z4vzudIuOP9TG8/7RKxkNfFS1HnexfCNEqo5psDytiQC0i/l56XPkzUwFj0mMMl0Gw0P/RKBN+h8BslH2mwGpUxOMgfbIJPpnWjayJDyO+sVvdKcLstsgSJYhPWPWkqyKqqWFbYRnWjJ2GaVZRoEFSwuSHg++QXdAJMVyaAA0jbJLNN9xo3XWZveUp2Pzt4=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WiLku-000FzD-8B; Thu, 08 May 2014 14:40:48 +0400 Date: Thu, 8 May 2014 14:40:45 +0400 From: Eygene Ryabinkin To: "Bjoern A. Zeeb" Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> <03D03A86-A5C4-4C0D-822E-D151AB272F35@lists.zabbadoz.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="E/DnYTRukya0zdZ1" Content-Disposition: inline In-Reply-To: <03D03A86-A5C4-4C0D-822E-D151AB272F35@lists.zabbadoz.net> Sender: rea@codelabs.ru Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 10:40:57 -0000 --E/DnYTRukya0zdZ1 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thu, May 08, 2014 at 10:28:19AM +0000, Bjoern A. Zeeb wrote: > On 08 May 2014, at 09:50 , Eygene Ryabinkin wrote: >=20 > >> Apart from that I thought the different version number was sufficient > >=20 > > The thing is that both VRRP and CARP packets use MAC address (on > > Ethernet at least) that equals to 00:00:5e:00:01:$VRID. So in case > > that $VRID is the same and VRRP and CARP admins aren't aware of each > > other, there will be MAC conflict, so L2 packets will be switched > > in a "funny" manner. >=20 > How=E2=80=99s that different routing guys running VRRP on the routers and > server guys running vrrp on the servers and conflicting on the ID? Server guys run CARP, not VRRP. And It Happens (TM). Cisco-heads choose VRRP ID of 1 and *BSD-heads take VHID 1 for their CARP. Bang! > It=E2=80=99s a management problem in the administrative broadcast domain > not a CARP vs. VRRP problem. It hits real networks; moreover, CARP is mostly undocumented stuff that router admins aren't very much aware of, so this administrative problem is created by the clashing MAC space that is used by different protocols. In a perfect world where everyone knows every possible bit about protocols it won't happen without poor management. But the reality is different, to my regret. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --E/DnYTRukya0zdZ1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNrXy1fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PvS8gD/V5aTESQDKm9ZeK9PBljnZsnx 3DNi8Z64FtXW9c8YdgAA/277vp19tj1ytpDUNMWpN9QT2S4wAXT7gZy421Stx2lQ =Y6sx -----END PGP SIGNATURE----- --E/DnYTRukya0zdZ1-- From owner-freebsd-net@FreeBSD.ORG Thu May 8 12:08:36 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAB281A4; Thu, 8 May 2014 12:08:36 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 72C327B1; Thu, 8 May 2014 12:08:35 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 54F7725D3A00; Thu, 8 May 2014 12:08:33 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 89456C22BDB; Thu, 8 May 2014 12:08:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id BOt5qnjZaXdU; Thu, 8 May 2014 12:08:30 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (unknown [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 6E854C22BCF; Thu, 8 May 2014 12:08:29 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that From: "Bjoern A. Zeeb" In-Reply-To: Date: Thu, 8 May 2014 12:08:28 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <1B71A1AC-8A85-415D-A413-CD01635B3123@lists.zabbadoz.net> References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> To: Eygene Ryabinkin X-Mailer: Apple Mail (2.1874) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 12:08:37 -0000 On 08 May 2014, at 09:50 , Eygene Ryabinkin wrote: > No, we're conflicting with VRRP on the MAC address space. >=20 > And, as I understand, CARP in 10 hadn't changed protocol in any way, > it just refurbished now CARP instances are configured and attached to > the interfaces. Could be wrong here, though. Yes, that is why the problem remains. = http://svnweb.freebsd.org/base/head/sys/netinet/ip_carp.h?annotate=3D25308= 7#l86 #define CARP_VERSION 2 vs. RFC 3768, Virtual Router Redundancy Protocol (VRRP), 5.3.1. Version The version field specifies the VRRP protocol version of this packet. This document defines version 2. *boom* And the world is moving on ... RFC 5798, Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 = and IPv6, 5.2.1. Version The version field specifies the VRRP protocol version of this packet. This document defines version 3. So, document CARP as Version 4 and then you have your own version of the = protocol and a good reason to change the EUI-48 assignment within the = IANA OUI maybe, maybe not. = http://www.iana.org/assignments/ethernet-numbers/ethernet-numbers.xhtml#et= hernet-numbers-1 00-01-00 to 00-01-FF VRRP (Virtual Router Redundancy Protocol) = [RFC5798] 00-02-00 to 00-02-FF VRRP IPv6 (Virtual Router Redundancy Protocol = IPv6) [RFC5798] Currently we are on Version 2 and VRRP (3768) is Version 2 and we share = the OUI but speak a different language. *boom* Currently you are worried that =93CARP" !=3D =93VRRP" and still uses the = same EUI-64. But that=92s a management problem. Server guys run = Solaris and VRRP[1] in the Solaris group, and Linux and VRRP in the = Linux Group, or FreeBSD and VRRP (yes people do) in the group that tries = to talk to the other two. If they don=92t talk to each other and the = networking guys put the servers in the same subnet, they probably = conflict. *boom* Needless to say that if they don=92t tell the = networking guys they conflict with the routers as well *boom*boom* Two different networking groups do redundancy failover and years later = connect their routers; 4 routers run VRRP, same VRID by default. = *boom* The samples you can find are plenty. People need to talk. The fact that your server guys use a non-unique = Ethernet address for CARP without talking to their local authority who=92s= in charge of the network first is nothing you can fix changing the = number. The fact that multiple deployments on the same subnet might = exist is nothing a number change will fix. I think the RFC uses the = word =93coordinate=94. The thing you can change is to fix the version number for CARP, document = the protocol (so your network guys become more aware of it though they = probably won=92t anyway); then you can make sure it doesn=92t conflict = on as much as is possible with it---just that you cannot always (as = described above) without talking. So it=92s about minimising the = impact, reading your log files, and talking to people. [1] = http://docs.oracle.com/cd/E23824_01/html/821-1453/gkfjq.html#scrolltoc =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Thu May 8 13:32:33 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E1F87C4; Thu, 8 May 2014 13:32:33 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA3B0FC7; Thu, 8 May 2014 13:32:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=ltWWJjiJOIhQbfmSN2ymgduDJYOOR2ZWfBfT9tfTYyg=; b=XohEX+B/TcVCeF+sV3ugnIZzDoYLRtjW5shcjl+fhyCW0CSTD9h1cb7K/SO6qDsRMCTICPYrVx0oHskK+rOlYbtTIH8DoB7QxjRS/XJBqcjmkKYHc7CCjaMCOG+i2qYWWl2jHrkC0n0W9sq01dVdJvoLvZhfEc1gkcwbYdkoOYvtQ0KNZPyEMHlUI3lrf+JtTjnd0nFPt/6N8VHDjP4qHBUWbH+hkBJmP8C9pJfbQSMlUFYdgod21YUed2S65CeX1QKapObHspyHMlDK72xEnfJsLRaFjXY8eY7/Y9pWmwScKcYBbDDcNh5CvN3FIb2HGorHQNHZLZbbT959bMt7FcYzin9O61AhrIGBGGDyJN67iKyAyXH2e5GJ7o5YQurQgPTWQI8pMTqpzGzoYnvF4HgRX+8KMh3qDr7JBqXN6UzSW9OC0+008rTVSf4IjqSa6mbh4wgArvSK7WzvsA25EIwIcfir0qz6DVkk5LJ0vNO7PaJIuwcdWfirTCCj43DB5LB3DUFozT9qjmZJP0VXbTPYmoZgl/wfL6AyZEYhiaTz990jKWOa+4HTwuGoUwCnKr2tn5okBEHgIywpWlYLS8X5bZwPIO0pidl16xxPV6Ew2VTkucef+ApD+WNML6VYgNlps3vmFZajrXRkCcTALaQ/zW+5YMr1+OA6TyUopzk=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WiOR4-000JLJ-KW; Thu, 08 May 2014 17:32:30 +0400 Date: Thu, 8 May 2014 17:32:28 +0400 From: Eygene Ryabinkin To: "Bjoern A. Zeeb" Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> <1B71A1AC-8A85-415D-A413-CD01635B3123@lists.zabbadoz.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="DiL7RhKs8rK9YGuF" Content-Disposition: inline In-Reply-To: <1B71A1AC-8A85-415D-A413-CD01635B3123@lists.zabbadoz.net> Sender: rea@codelabs.ru Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 13:32:33 -0000 --DiL7RhKs8rK9YGuF Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thu, May 08, 2014 at 12:08:28PM +0000, Bjoern A. Zeeb wrote: > On 08 May 2014, at 09:50 , Eygene Ryabinkin wrote: >=20 > > No, we're conflicting with VRRP on the MAC address space. > >=20 > > And, as I understand, CARP in 10 hadn't changed protocol in any way, > > it just refurbished now CARP instances are configured and attached to > > the interfaces. Could be wrong here, though. >=20 > Yes, that is why the problem remains. >=20 > http://svnweb.freebsd.org/base/head/sys/netinet/ip_carp.h?annotate=3D2530= 87#l86 > #define CARP_VERSION 2 >=20 > vs. >=20 > RFC 3768, Virtual Router Redundancy Protocol (VRRP), 5.3.1. Version >=20 > The version field specifies the VRRP protocol version of this packet. > This document defines version 2. >=20 > *boom* >=20 > And the world is moving on ... >=20 > RFC 5798, Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 an= d IPv6, 5.2.1. Version >=20 > The version field specifies the VRRP protocol version of this packet. > This document defines version 3. Oh, shit :( > So, document CARP as Version 4 and then you have your own version of > the protocol and a good reason to change the EUI-48 assignment > within the IANA OUI maybe, maybe not. >=20 > http://www.iana.org/assignments/ethernet-numbers/ethernet-numbers.xhtml#e= thernet-numbers-1 >=20 > 00-01-00 to 00-01-FF VRRP (Virtual Router Redundancy Protocol) [RFC5798] > 00-02-00 to 00-02-FF VRRP IPv6 (Virtual Router Redundancy Protocol IPv6) = [RFC5798] >=20 >=20 >=20 > Currently we are on Version 2 and VRRP (3768) is Version 2 and we > share the OUI but speak a different language. *boom* More properly, we're on v2, VRRP 3768 is v2 and we share protocol number (112). And our data planes share the same OUI, but that's has nothing to do with being on version X or Y on any side, that's a different problem. > Currently you are worried that =E2=80=9CCARP" !=3D =E2=80=9CVRRP" and sti= ll uses the > same EUI-64. But that=E2=80=99s a management problem. Server guys run > Solaris and VRRP[1] in the Solaris group, and Linux and VRRP in the > Linux Group, or FreeBSD and VRRP (yes people do) in the group that > tries to talk to the other two. If they don=E2=80=99t talk to each other > and the networking guys put the servers in the same subnet, they > probably conflict. *boom* Needless to say that if they don=E2=80=99t t= ell > the networking guys they conflict with the routers as well > *boom*boom* >=20 > Two different networking groups do redundancy failover and years > later connect their routers; 4 routers run VRRP, same VRID by > default. *boom* >=20 > The samples you can find are plenty. Multiple deployments of the same protocol need coordination, but that's expected. Multiple deployments of a different protocols might (and as our case shows, do) need coordination, but that's an unexpected thing. Change of used OUI space for CARP will prevent at least the part of the breakage, the most unexpected part of the whole drama. > People need to talk. The fact that your server guys use a > non-unique Ethernet address for CARP without talking to their local > authority who=E2=80=99s in charge of the network first is nothing you can > fix changing the number. The fact that multiple deployments on the > same subnet might exist is nothing a number change will fix. I > think the RFC uses the word =E2=80=9Ccoordinate=E2=80=9D. People need to talk, yes. But do BGP heads will announce the technical details of what they are doing to the Squid cluster guys (picking protocol and software names arbitrary)? Not sure. Hell, I am sure they won't. And if we speak about RFCs: we need protocol specifications at all not only because people should know how they work, but also to avoid messing (perhaps unintentionally) some technical details of the protocols making different ones to bring havoc if used together. That's why VRRP for IPv6 has its own OUI space and that's why CARP needs its own OUI space: it is a different protocol from VRRP, so we should minimize clashes at all levels. IANA/IETF could have said "oh dear, L2 domain admins must coordinate their IPv4 and IPv6 VRRP things, so let's reuse 00-00-5E-00-01 space". But they minimised confusion where they could and that's a good thing to do. What you're saying is that "all involved people must be educated enough to grok all details and share them with others". What I am saying is that "when we can eliminate potential for breakage, we should". My assertion doesn't contradict with your, but it tries to make already complicated networking world to be slightly more error-prone (at least, I hope so and that's my intention here). > The thing you can change is to fix the version number for CARP, > document the protocol (so your network guys become more aware of it > though they probably won=E2=80=99t anyway); then you can make sure it > doesn=E2=80=99t conflict on as much as is possible with it---just that you > cannot always (as described above) without talking. So it=E2=80=99s ab= out > minimising the impact, reading your log files, and talking to > people. That is also a good thing to do, but it is a longer-term goal, because putting CARP into RFC is, well, a loooong story. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --DiL7RhKs8rK9YGuF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNrh2xfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PtwCwD/QZ0rn6QgOqY/uEOKyB6E7N5w 4UIhfemhOPo9gbDg3HYBAIy4lbJDabPVj5zVIevudvx46ntWClmA3LUIgJBTQUfH =VSIJ -----END PGP SIGNATURE----- --DiL7RhKs8rK9YGuF-- From owner-freebsd-net@FreeBSD.ORG Thu May 8 13:50:10 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75C5A11D; Thu, 8 May 2014 13:50:10 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 15F1E17B; Thu, 8 May 2014 13:50:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=vh25NrVnyyfTqScTIKrbm7EeBDJKlcS10xU8DDIphCI=; b=FhzGmWQj4bAY2/JGoj7iUGiR2EYnw5qU0hoYEs24qWndTTvwmRCn2RgMhC3g13EEvjtxzH8j2gGTNw3f0+1VA61TjAlmcbH6KUc3INq3W7WEHnJqNB8D/aBXfsp0EI6xTxLSJmvNqBkjtx1VscncrNaXsQ6pIbyrJe8oS5Ed4x3CoAXqTAWQgShzU++R4nOlq6AaEwAPaCWLE1yiYbdZkRjlVsMnWvl5oqRXI07q7ScRN2TcJtzFPJNl1mNQotRgFQkRmveE5eRhvKM7padQs0K4IdTiuo1Na2k00x9wCLG4WZFmcgk6ESjh4dA9eWg1CpknaYT6Mc/NAGiyK+3ezmULQt4X7TW9pm3cCuasvWeUfUsK1J+6mbC1lYRoHsC6xyT/ON2RopCdNf4CUZPzoTEBcx4B+pr+B2cAAUxSabalUnTSkcAnmMPaRcaIla0bcb7KRKn7EH3NDfeViWvCMTfz67j9F8Tuv9Zx+2z7hJrNo5OHJmqQHMLUYKiFOY6T7IDKVWY7cVQLuhhdtHnvmtIygIOUFSkCXaHG/mZ0yAn+kTOAYmYLGosMrTa+iMEO3m0a8COtH5xv4LpwkSSzxC9OnHhF5LNCLyQk4e45i7Yn2duTV3taPXivJqqdvl4xuETr7sLFvrWcBg43eYMztMpS7AiNTNmSJTMGoUpE+vE=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WiOi8-000Jd6-0z; Thu, 08 May 2014 17:50:08 +0400 Date: Thu, 8 May 2014 17:50:05 +0400 From: Eygene Ryabinkin To: "Bjoern A. Zeeb" Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: <0rxH4JJknMOY34ungRbP4jrPfLM@dHhGgwofm7uNfL6/X5+bGIkDUYs> References: <97B3C7CB-3E64-4FE0-81C8-F1FE6FB456A2@lists.zabbadoz.net> <1B71A1AC-8A85-415D-A413-CD01635B3123@lists.zabbadoz.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="SNIs70sCzqvszXB4" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 13:50:10 -0000 --SNIs70sCzqvszXB4 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thu, May 08, 2014 at 05:32:28PM +0400, Eygene Ryabinkin wrote: > Thu, May 08, 2014 at 12:08:28PM +0000, Bjoern A. Zeeb wrote: > > People need to talk. The fact that your server guys use a > > non-unique Ethernet address for CARP without talking to their local > > authority who=E2=80=99s in charge of the network first is nothing you c= an > > fix changing the number. The fact that multiple deployments on the > > same subnet might exist is nothing a number change will fix. I > > think the RFC uses the word =E2=80=9Ccoordinate=E2=80=9D. >=20 > People need to talk, yes. But do BGP heads will announce the > technical details of what they are doing to the Squid cluster guys > (picking protocol and software names arbitrary)? Not sure. Hell, > I am sure they won't. >=20 > And if we speak about RFCs: we need protocol specifications at all not > only because people should know how they work, but also to avoid > messing (perhaps unintentionally) some technical details of the > protocols making different ones to bring havoc if used together. >=20 > That's why VRRP for IPv6 has its own OUI space and that's why CARP > needs its own OUI space: it is a different protocol from VRRP, so we > should minimize clashes at all levels. IANA/IETF could have said "oh > dear, L2 domain admins must coordinate their IPv4 and IPv6 VRRP > things, so let's reuse 00-00-5E-00-01 space". But they minimised > confusion where they could and that's a good thing to do. By the way, http://marc.info/?l=3Dopenbsd-tech&m=3D139955603603070&w=3D2 So the new OUI assignment is already in place, it will be interesting what will be done next and if this patch will be accepted by OpenBSD. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --SNIs70sCzqvszXB4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNri41fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PtOTgD/QCrLDTft14DmjabRKzpoYbP5 KLtrYvbewu+E0BwtKucA/Agqmiy6poj2A8aiD7MuL2ZET5SauKqePc+SfdqMwkL6 =s6+x -----END PGP SIGNATURE----- --SNIs70sCzqvszXB4-- From owner-freebsd-net@FreeBSD.ORG Thu May 8 18:40:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25CC21B4; Thu, 8 May 2014 18:40:35 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8A502BD; Thu, 8 May 2014 18:40:34 +0000 (UTC) Received: from [192.168.1.103] (p508F057E.dip0.t-ipconnect.de [80.143.5.126]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id DB5A31C10493D; Thu, 8 May 2014 20:40:29 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140507083751.GB1376@michelle.cdnetworks.com> Date: Thu, 8 May 2014 20:40:22 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 18:40:35 -0000 On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: > On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: >> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: >>=20 >>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >>>>=20 >>>>>=20 >>>>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>>>=20 >>>>>> Dear all, >>>>>>=20 >>>>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>>>> with bad checksums. After debugging this I figured out that this = is a problem with >>>>>> the csum_flags defined in mbuf.h. >>>>>>=20 >>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which = is defined in mbuf.h: >>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the = packet is considered >>>>>> to have a correct checksum. >>>>>>=20 >>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and = set CSUM_DATA_VALID in >>>>>> csum_flags to indicate that the UDP/TCP should consider csum_data = to figure out if >>>>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>>>> field contains information. >>>>>>=20 >>>>>> Now the following happens (on the raspberry pi the driver used is >>>>>> dev/usb/net/if_smsc.c >>>>>>=20 >>>>>> 1. A packet is received and if it is not too short, the checksum = computed >>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This = happens >>>>>> for all IP packets, not only for UDP and TCP packets. >>>>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID = as CSUM_SCTP_VALID >>>>>> and accepts the packet. So no SCTP checksum check ever happened. >>>>>>=20 >>>>>> Alternatives to fix this: >>>>>>=20 >>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP = or TCP packets, since >>>>>> it only makes sense in these cases. >>>>>=20 >>>>> Wait, or for SCTP in cad the crc32 (I think it was) was actually = checked but not otherwise. This is how it should be imho. It seems = like a driver bug. >>>> I went through the list of drivers and you are right, it seems to = be a bug >>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small = set I can't tell. >>>>=20 >>>=20 >>> I'm not sure how the controller computes TCP/UDP checksum values. >>> It seems the publicly available data sheet was highly sanitized so >>> it was useless to me. The comment in the driver says that the >> Same for me... >>> controller computes RX checksum after the IPv4 header to the end of >>> ethernet frame. After seeing that comment, three questions popped >>> up: >>>=20 OK, I did some testing. It looks like the card is just computing the checksum over the IP payload taking the correct IP header length into = account. >>> 1. Is the controller smart enough to skip IP options header in >>> TCP/UDP checksum offloading? Yes, I can send fragmented and un-fragmented UDP packets with IP options and they are handled correctly. Even if the last fragment is too short. >>> 2. How controller handles UDP checksum value 0x0000(i.e. sender >>> didn't compute UDP checksum)? This case isn't handled. However, udp_input() looks first for zero = checksums and only after that in the csum_flags. So it doesn't result in any = problems. Would you prefer not to set CSUM_DATA_VALID in this case? >>> 3. How the controller can compute TCP checksum of fragmented >>> packets? At least it does it right for UDP... Best regards Michael >>>=20 >>> Since you have the controller I guess it's easy to verify all >>> cases. For case 3, I believe the controller can't handle >>> fragmented frames so driver should have to explicitly check ip_off >>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) >>> handle it. >> Let me check this. Is there a tool to send UDP/TCP with IP level = options >> or do I need to write a small test program myself? >>=20 >=20 > I recall I used buggy ipsend of ipfilter package in the past but it > would be more easy to write a simple test program or patch driver > to generate those frames. >=20 From owner-freebsd-net@FreeBSD.ORG Thu May 8 18:50:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEB6E5A9 for ; Thu, 8 May 2014 18:50:59 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73A07636 for ; Thu, 8 May 2014 18:50:59 +0000 (UTC) Received: from [192.168.1.103] (p508F057E.dip0.t-ipconnect.de [80.143.5.126]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 228D71C104981 for ; Thu, 8 May 2014 20:50:56 +0200 (CEST) From: Michael Tuexen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: TX Checksum offloading issue with re interfaces Message-Id: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> Date: Thu, 8 May 2014 20:50:48 +0200 To: FreeBSD Net Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 18:50:59 -0000 Dear all, while testing checksum offloading of UDP packets over IP with IP = options, I figured out that my card dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit = Ethernet dev.re.1.%driver: re dev.re.1.%location: slot=3D0 function=3D0 handle=3D\_SB_.PCI0.PE1F.LAN2 dev.re.1.%pnpinfo: vendor=3D0x10ec device=3D0x8168 subvendor=3D0x1734 = subdevice=3D0x1159 class=3D0x020000 dev.re.1.%parent: pci13 dev.re.1.stats: -1 dev.re.1.int_rx_mod: 65 computes the UDP checksum, but stores it in the packet at the place, = where it would be, if there are no IP options. So it corrupts the options in the packet... I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix it. = Any idea? Best regards Michael= From owner-freebsd-net@FreeBSD.ORG Thu May 8 20:04:08 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FA849F8; Thu, 8 May 2014 20:04:08 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C9B4D74; Thu, 8 May 2014 20:04:06 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s48K44qX050519 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 9 May 2014 00:04:04 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s48K44jx050518; Fri, 9 May 2014 00:04:04 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 9 May 2014 00:04:04 +0400 From: Gleb Smirnoff To: Eygene Ryabinkin Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: <20140508200404.GA50446@glebius.int.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 20:04:08 -0000 Eygene, On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: E> - I'll do a patch for carp(4) that will allow it to use configurable E> OUI from a sysctl knob (first 5 bytes of OUI); Please no sysctl knobs. This should be configurable via ifconfig(8) per vhid. P.S. Sorry for being silent on all legal and administrative questions. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Fri May 9 01:35:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D876EFE9; Fri, 9 May 2014 01:35:59 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2009D7A; Fri, 9 May 2014 01:35:59 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id fb1so3617619pad.37 for ; Thu, 08 May 2014 18:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=F2QNhCtbhZqsmSW6jwKEEDtvV456frR+jrs9aCjlUEI=; b=vK2dFANHKYzjpTepjl55k856XKULk/a3XJHh7skunBZMHQijMjws6j4VbnSk2Sjmlb lJjhOBU3pIGpQjg9xzGzH/pErOftutkyWbWUqiXSlRdUsW5U6ymK8Tx5Dy1x/+QxwkBW ai1g2FXJEhsa/ZUKw5HCAC7D+j+DudYzT2FReQ3vvemE9xjqnTaHbglnNZFAVAwqEJrG v7MnP/u69Em92Y7EFjXx6p7SxoqreVgzBs1HaVFNOfnZCKSAfG5LCwMana6We12AdgIs /ZYC/jjCeOHLqN9Svnj3Odyiw8TbKs0Smyr+nYeb9xaFCJMOOMsYlg3vxCDa/QhpRMoW IMKQ== X-Received: by 10.66.240.70 with SMTP id vy6mr14162747pac.80.1399599359100; Thu, 08 May 2014 18:35:59 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id px10sm507768pac.41.2014.05.08.18.35.56 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 08 May 2014 18:35:58 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 09 May 2014 10:35:56 +0900 From: Yonghyeon PYUN Date: Fri, 9 May 2014 10:35:56 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140509013556.GA3014@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 01:35:59 -0000 On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: > On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: > > > On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: > >> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: > >> > >>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: > >>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: > >>>> > >>>>> > >>>>> On 02 May 2014, at 10:22 , Michael Tuexen wrote: > >>>>> > >>>>>> Dear all, > >>>>>> > >>>>>> during testing I found that FreeBSD head (on a raspberry pi) accepts SCTP packet > >>>>>> with bad checksums. After debugging this I figured out that this is a problem with > >>>>>> the csum_flags defined in mbuf.h. > >>>>>> > >>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is defined in mbuf.h: > >>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID > >>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is considered > >>>>>> to have a correct checksum. > >>>>>> > >>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and set CSUM_DATA_VALID in > >>>>>> csum_flags to indicate that the UDP/TCP should consider csum_data to figure out if > >>>>>> the packet has a correct checksum. The problem is that CSUM_DATA_VALID is defined as > >>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID > >>>>>> In this case the semantic is not that the packet has a valid checksum, but the csum_data > >>>>>> field contains information. > >>>>>> > >>>>>> Now the following happens (on the raspberry pi the driver used is > >>>>>> dev/usb/net/if_smsc.c > >>>>>> > >>>>>> 1. A packet is received and if it is not too short, the checksum computed > >>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This happens > >>>>>> for all IP packets, not only for UDP and TCP packets. > >>>>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as CSUM_SCTP_VALID > >>>>>> and accepts the packet. So no SCTP checksum check ever happened. > >>>>>> > >>>>>> Alternatives to fix this: > >>>>>> > >>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP packets, since > >>>>>> it only makes sense in these cases. > >>>>> > >>>>> Wait, or for SCTP in cad the crc32 (I think it was) was actually checked but not otherwise. This is how it should be imho. It seems like a driver bug. > >>>> I went through the list of drivers and you are right, it seems to be a bug > >>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I can't tell. > >>>> > >>> > >>> I'm not sure how the controller computes TCP/UDP checksum values. > >>> It seems the publicly available data sheet was highly sanitized so > >>> it was useless to me. The comment in the driver says that the > >> Same for me... > >>> controller computes RX checksum after the IPv4 header to the end of > >>> ethernet frame. After seeing that comment, three questions popped > >>> up: > >>> > OK, I did some testing. It looks like the card is just computing the > checksum over the IP payload taking the correct IP header length into account. > >>> 1. Is the controller smart enough to skip IP options header in > >>> TCP/UDP checksum offloading? > Yes, I can send fragmented and un-fragmented UDP packets with IP options > and they are handled correctly. Even if the last fragment is too short. I'm assuming you're taking about receiving fragmented UDP packets with RX checksum offloading, right? > >>> 2. How controller handles UDP checksum value 0x0000(i.e. sender > >>> didn't compute UDP checksum)? > This case isn't handled. However, udp_input() looks first for zero checksums > and only after that in the csum_flags. So it doesn't result in any problems. > Would you prefer not to set CSUM_DATA_VALID in this case? At least, it correctly updates UDP stats of netstat(1). > >>> 3. How the controller can compute TCP checksum of fragmented > >>> packets? > At least it does it right for UDP... > Hmm, CSUM_DATA_VALID indicates driver computed RX TCP/UDP checksum without pseudo header. As you know, controller can't compute TCP/UDP checksum until all its fragmented payload are read from wire. Packets may arrive out of order and may be mixed with other packets. Advanced controllers with enough memory may be able to compute TCP/UDP checksums by tracking each connections(e.g LRO) but low-end controllers may be not. It seems the controller does not even support RX TCP/UDP pseudo header checksum offloading so I wonder how this controller can support RX TCP/UDP checksum offloading for fragmented TCP/UDP packets. Some controllers maintain two bits for TCP/UDP checksum offloading result in status word. One bit is used to indicate whether controller performed TCP/UDP checksum offloading and the other bit is used to indicate whether the computed checksum is correct or not. For UDP checksum value 0x0000 and fragmented TCP/UDP packets, these controllers do not attempt to compute TCP/UDP checksum. > Best regards > Michael > >>> > >>> Since you have the controller I guess it's easy to verify all > >>> cases. For case 3, I believe the controller can't handle > >>> fragmented frames so driver should have to explicitly check ip_off > >>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) > >>> handle it. > >> Let me check this. Is there a tool to send UDP/TCP with IP level options > >> or do I need to write a small test program myself? > >> > > > > I recall I used buggy ipsend of ipfilter package in the past but it > > would be more easy to write a simple test program or patch driver > > to generate those frames. > > > > From owner-freebsd-net@FreeBSD.ORG Fri May 9 01:47:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0288D33E for ; Fri, 9 May 2014 01:47:34 +0000 (UTC) Received: from mail-pd0-x236.google.com (mail-pd0-x236.google.com [IPv6:2607:f8b0:400e:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9D79E58 for ; Fri, 9 May 2014 01:47:33 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id v10so2988944pde.41 for ; Thu, 08 May 2014 18:47:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=gad3YIsDp/K8B+emcLv7DxV7YaNK5o/yv2GrRtwTO9M=; b=JF8gj2NOJdpxgQH4I5ayJ6dcsjnuNVQQ8Jf6ngt/xOX3FKDZUauxgWHrWjFTGT/eZp qeII0n6YVoLFmhZm7fjY6IBnQrxvCJ9J4swS/+sbjE9w8xTEiklaz/dUGDW1RN1qig/u FU2te3yMCC3/r8CqxT+mBrUvxbtX8DrVxrQLVWyK+RLb6NLVVcC3rjY1qdJN07OM+XGW KxcQU3iV2IVCPAAp1IBY5rkbzF/UqNlj5zNqqp2Smxk6aHQ/cGMvZYxN7VopBH3eR8m1 s18rYH/yClDOIW9fnSKnGVJ9Njepfqmv8ylOwy2QjnUV/epEZ1bl3kSkFP4T7pkyrKYJ oY3w== X-Received: by 10.66.66.202 with SMTP id h10mr14408644pat.70.1399600053371; Thu, 08 May 2014 18:47:33 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id nx12sm686009pab.6.2014.05.08.18.47.30 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 08 May 2014 18:47:32 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 09 May 2014 10:47:33 +0900 From: Yonghyeon PYUN Date: Fri, 9 May 2014 10:47:33 +0900 To: Michael Tuexen Subject: Re: TX Checksum offloading issue with re interfaces Message-ID: <20140509014733.GB3014@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 01:47:34 -0000 On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: > Dear all, > > while testing checksum offloading of UDP packets over IP with IP options, I figured > out that my card > > dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet > dev.re.1.%driver: re > dev.re.1.%location: slot=0 function=0 handle=\_SB_.PCI0.PE1F.LAN2 > dev.re.1.%pnpinfo: vendor=0x10ec device=0x8168 subvendor=0x1734 subdevice=0x1159 class=0x020000 > dev.re.1.%parent: pci13 > dev.re.1.stats: -1 > dev.re.1.int_rx_mod: 65 > > computes the UDP checksum, but stores it in the packet at the place, where it would be, > if there are no IP options. So it corrupts the options in the packet... > > I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix it. Any idea? > re(4) has a very long history on its broken TX checksum offloading. So re(4) has many workarounds for known issues on several variants. re(4) controllers support TX IPv4/TCP/UDP checksum offloading. For 8168C/8168CP, TX IPv4 checksum offloading was disabled due to generation of corrupted frames. Could you show me the dmesg output(only re(4)/rgephy(4))? The vendor uses the same PCI id for its RTL8168/8111 family chips so dmesg output is necessary to know exact controller revision. From owner-freebsd-net@FreeBSD.ORG Fri May 9 10:33:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 010D858B for ; Fri, 9 May 2014 10:33:41 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8AFD35B for ; Fri, 9 May 2014 10:33:41 +0000 (UTC) Received: from [192.168.1.103] (p508F035A.dip0.t-ipconnect.de [80.143.3.90]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 567BE1C104991; Fri, 9 May 2014 12:33:37 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: TX Checksum offloading issue with re interfaces From: Michael Tuexen In-Reply-To: <20140509014733.GB3014@michelle.cdnetworks.com> Date: Fri, 9 May 2014 12:33:24 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> <20140509014733.GB3014@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 10:33:42 -0000 On 09 May 2014, at 03:47, Yonghyeon PYUN wrote: > On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: >> Dear all, >>=20 >> while testing checksum offloading of UDP packets over IP with IP = options, I figured >> out that my card >>=20 >> dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit = Ethernet >> dev.re.1.%driver: re >> dev.re.1.%location: slot=3D0 function=3D0 handle=3D\_SB_.PCI0.PE1F.LAN2= >> dev.re.1.%pnpinfo: vendor=3D0x10ec device=3D0x8168 subvendor=3D0x1734 = subdevice=3D0x1159 class=3D0x020000 >> dev.re.1.%parent: pci13 >> dev.re.1.stats: -1 >> dev.re.1.int_rx_mod: 65 >>=20 >> computes the UDP checksum, but stores it in the packet at the place, = where it would be, >> if there are no IP options. So it corrupts the options in the = packet... >>=20 >> I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix = it. Any idea? >>=20 >=20 > re(4) has a very long history on its broken TX checksum offloading. > So re(4) has many workarounds for known issues on several variants. > re(4) controllers support TX IPv4/TCP/UDP checksum offloading. For > 8168C/8168CP, TX IPv4 checksum offloading was disabled due to > generation of corrupted frames. > Could you show me the dmesg output(only re(4)/rgephy(4))? > The vendor uses the same PCI id for its RTL8168/8111 family chips > so dmesg output is necessary to know exact controller revision. Sure (re1 was used during the test): re0: port = 0x8000-0x80ff mem 0xf6104000-0xf6104fff,0xf6100000-0xf6103fff irq 16 at = device 0.0 on pci12 re0: Using 1 MSI-X message re0: Chip rev. 0x28800000 re0: MAC rev. 0x00200000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow re0: Ethernet address: 00:19:99:85:31:d9 re1: port = 0x9000-0x90ff mem 0xf5c20000-0xf5c20fff,0xf6200000-0xf620ffff irq 17 at = device 0.0 on pci13 re1: Using 1 MSI-X message re1: Chip rev. 0x3c800000 re1: MAC rev. 0x00300000 miibus1: on re1 rgephy1: PHY 1 on = miibus1 rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow re1: Ethernet address: 00:19:99:7e:c7:46 Best regards Michael >=20 From owner-freebsd-net@FreeBSD.ORG Fri May 9 10:47:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02DF075E; Fri, 9 May 2014 10:47:04 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86F2E639; Fri, 9 May 2014 10:47:03 +0000 (UTC) Received: from [192.168.1.103] (p508F035A.dip0.t-ipconnect.de [80.143.3.90]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id BE3891C104991; Fri, 9 May 2014 12:47:00 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140509013556.GA3014@michelle.cdnetworks.com> Date: Fri, 9 May 2014 12:46:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 10:47:04 -0000 On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: > On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: >> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: >>=20 >>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: >>>> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: >>>>=20 >>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >>>>>>=20 >>>>>>>=20 >>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>>>>>=20 >>>>>>>> Dear all, >>>>>>>>=20 >>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>>>>>> with bad checksums. After debugging this I figured out that = this is a problem with >>>>>>>> the csum_flags defined in mbuf.h. >>>>>>>>=20 >>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, = which is defined in mbuf.h: >>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the = packet is considered >>>>>>>> to have a correct checksum. >>>>>>>>=20 >>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and = set CSUM_DATA_VALID in >>>>>>>> csum_flags to indicate that the UDP/TCP should consider = csum_data to figure out if >>>>>>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>>>>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>>>>>> field contains information. >>>>>>>>=20 >>>>>>>> Now the following happens (on the raspberry pi the driver used = is >>>>>>>> dev/usb/net/if_smsc.c >>>>>>>>=20 >>>>>>>> 1. A packet is received and if it is not too short, the = checksum computed >>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. = This happens >>>>>>>> for all IP packets, not only for UDP and TCP packets. >>>>>>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID = as CSUM_SCTP_VALID >>>>>>>> and accepts the packet. So no SCTP checksum check ever = happened. >>>>>>>>=20 >>>>>>>> Alternatives to fix this: >>>>>>>>=20 >>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of = UDP or TCP packets, since >>>>>>>> it only makes sense in these cases. >>>>>>>=20 >>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was = actually checked but not otherwise. This is how it should be imho. It = seems like a driver bug. >>>>>> I went through the list of drivers and you are right, it seems to = be a bug >>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a = small set I can't tell. >>>>>>=20 >>>>>=20 >>>>> I'm not sure how the controller computes TCP/UDP checksum values. >>>>> It seems the publicly available data sheet was highly sanitized so >>>>> it was useless to me. The comment in the driver says that the >>>> Same for me... >>>>> controller computes RX checksum after the IPv4 header to the end = of >>>>> ethernet frame. After seeing that comment, three questions popped >>>>> up: >>>>>=20 >> OK, I did some testing. It looks like the card is just computing the >> checksum over the IP payload taking the correct IP header length into = account. >>>>> 1. Is the controller smart enough to skip IP options header in >>>>> TCP/UDP checksum offloading? >> Yes, I can send fragmented and un-fragmented UDP packets with IP = options >> and they are handled correctly. Even if the last fragment is too = short. >=20 > I'm assuming you're taking about receiving fragmented UDP packets > with RX checksum offloading, right? Correct. >=20 >>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender >>>>> didn't compute UDP checksum)? >> This case isn't handled. However, udp_input() looks first for zero = checksums >> and only after that in the csum_flags. So it doesn't result in any = problems. >> Would you prefer not to set CSUM_DATA_VALID in this case? >=20 > At least, it correctly updates UDP stats of netstat(1). Let me double check that... >=20 >>>>> 3. How the controller can compute TCP checksum of fragmented >>>>> packets? >> At least it does it right for UDP... >>=20 >=20 > Hmm, CSUM_DATA_VALID indicates driver computed RX TCP/UDP checksum > without pseudo header. As you know, controller can't compute > TCP/UDP checksum until all its fragmented payload are read from > wire. Packets may arrive out of order and may be mixed with other I'm not sure I understand this... Please note that the pseudo header is not taken into account. So the card can compute the checksum over the payload of IP for each fragment. This is stored in the csum_data field. During reassembly the csum_data fields of the fragments are combined in = http://svnweb.freebsd.org/base/head/sys/netinet/ip_input.c?view=3Dmarkup#l= 1075 This looks OK to me. I'm not sure why you think the cards needs to keep state for this. I understand it needs state if it wants to take the pseudoheader into account, but this is not done here. > packets. Advanced controllers with enough memory may be able to > compute TCP/UDP checksums by tracking each connections(e.g LRO) but > low-end controllers may be not. It seems the controller does not > even support RX TCP/UDP pseudo header checksum offloading so I > wonder how this controller can support RX TCP/UDP checksum > offloading for fragmented TCP/UDP packets. I'm not sure what I'm missing here... You compute the sum over the parts and add the sums of the parts. That should work for the UDP/TCP checksum. >=20 > Some controllers maintain two bits for TCP/UDP checksum offloading > result in status word. One bit is used to indicate whether > controller performed TCP/UDP checksum offloading and the other bit > is used to indicate whether the computed checksum is correct or For SCTP we only get a bit that the checksum was computed and is = correct... > not. For UDP checksum value 0x0000 and fragmented TCP/UDP packets, > these controllers do not attempt to compute TCP/UDP checksum. I think it "just" computes it and leaves it up to the upper layer to use the result or not... Best regards Michael >=20 >> Best regards >> Michael >>>>>=20 >>>>> Since you have the controller I guess it's easy to verify all >>>>> cases. For case 3, I believe the controller can't handle >>>>> fragmented frames so driver should have to explicitly check ip_off >>>>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) >>>>> handle it. >>>> Let me check this. Is there a tool to send UDP/TCP with IP level = options >>>> or do I need to write a small test program myself? >>>>=20 >>>=20 >>> I recall I used buggy ipsend of ipfilter package in the past but it >>> would be more easy to write a simple test program or patch driver >>> to generate those frames. >>>=20 >>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Fri May 9 14:22:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A763369D; Fri, 9 May 2014 14:22:54 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39511B55; Fri, 9 May 2014 14:22:54 +0000 (UTC) Received: from [192.168.1.103] (p508F035A.dip0.t-ipconnect.de [80.143.3.90]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id F36BA1C104996; Fri, 9 May 2014 16:22:49 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: Date: Fri, 9 May 2014 16:22:36 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 14:22:54 -0000 On 09 May 2014, at 12:46, Michael Tuexen = wrote: > On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: >=20 >> On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: >>> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: >>>=20 >>>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: >>>>> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: >>>>>=20 >>>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >>>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: >>>>>>>=20 >>>>>>>>=20 >>>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>>>>>>=20 >>>>>>>>> Dear all, >>>>>>>>>=20 >>>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>>>>>>> with bad checksums. After debugging this I figured out that = this is a problem with >>>>>>>>> the csum_flags defined in mbuf.h. >>>>>>>>>=20 >>>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, = which is defined in mbuf.h: >>>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the = packet is considered >>>>>>>>> to have a correct checksum. >>>>>>>>>=20 >>>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum = and set CSUM_DATA_VALID in >>>>>>>>> csum_flags to indicate that the UDP/TCP should consider = csum_data to figure out if >>>>>>>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>>>>>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>>>>>>> field contains information. >>>>>>>>>=20 >>>>>>>>> Now the following happens (on the raspberry pi the driver used = is >>>>>>>>> dev/usb/net/if_smsc.c >>>>>>>>>=20 >>>>>>>>> 1. A packet is received and if it is not too short, the = checksum computed >>>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. = This happens >>>>>>>>> for all IP packets, not only for UDP and TCP packets. >>>>>>>>> 2. In case of SCTP packets, the SCTP interprets = CSUM_DATA_VALID as CSUM_SCTP_VALID >>>>>>>>> and accepts the packet. So no SCTP checksum check ever = happened. >>>>>>>>>=20 >>>>>>>>> Alternatives to fix this: >>>>>>>>>=20 >>>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of = UDP or TCP packets, since >>>>>>>>> it only makes sense in these cases. >>>>>>>>=20 >>>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was = actually checked but not otherwise. This is how it should be imho. It = seems like a driver bug. >>>>>>> I went through the list of drivers and you are right, it seems = to be a bug >>>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a = small set I can't tell. >>>>>>>=20 >>>>>>=20 >>>>>> I'm not sure how the controller computes TCP/UDP checksum values. >>>>>> It seems the publicly available data sheet was highly sanitized = so >>>>>> it was useless to me. The comment in the driver says that the >>>>> Same for me... >>>>>> controller computes RX checksum after the IPv4 header to the end = of >>>>>> ethernet frame. After seeing that comment, three questions popped >>>>>> up: >>>>>>=20 >>> OK, I did some testing. It looks like the card is just computing the >>> checksum over the IP payload taking the correct IP header length = into account. >>>>>> 1. Is the controller smart enough to skip IP options header in >>>>>> TCP/UDP checksum offloading? >>> Yes, I can send fragmented and un-fragmented UDP packets with IP = options >>> and they are handled correctly. Even if the last fragment is too = short. >>=20 >> I'm assuming you're taking about receiving fragmented UDP packets >> with RX checksum offloading, right? > Correct. >>=20 >>>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender >>>>>> didn't compute UDP checksum)? >>> This case isn't handled. However, udp_input() looks first for zero = checksums >>> and only after that in the csum_flags. So it doesn't result in any = problems. >>> Would you prefer not to set CSUM_DATA_VALID in this case? >>=20 >> At least, it correctly updates UDP stats of netstat(1). > Let me double check that... I double checked it. The statistic counters are incremented. Please note that we had a bug in the sending code of head, which made it impossible to send UDP packets with 0 checksum. That is fixed in http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265776 So any preference whether to report CSUM_DATA_VALID if a UDP packet with checksum 0 is received or not? I'm pretty open, since it does not have any effect right now... Best regards Michael >>=20 >>>>>> 3. How the controller can compute TCP checksum of fragmented >>>>>> packets? >>> At least it does it right for UDP... >>>=20 >>=20 >> Hmm, CSUM_DATA_VALID indicates driver computed RX TCP/UDP checksum >> without pseudo header. As you know, controller can't compute >> TCP/UDP checksum until all its fragmented payload are read from >> wire. Packets may arrive out of order and may be mixed with other > I'm not sure I understand this... Please note that the pseudo header > is not taken into account. So the card can compute the checksum over > the payload of IP for each fragment. This is stored in the csum_data > field. During reassembly the csum_data fields of the fragments are > combined in > = http://svnweb.freebsd.org/base/head/sys/netinet/ip_input.c?view=3Dmarkup#l= 1075 > This looks OK to me. I'm not sure why you think the cards needs > to keep state for this. I understand it needs state if it wants > to take the pseudoheader into account, but this is not done here. >> packets. Advanced controllers with enough memory may be able to >> compute TCP/UDP checksums by tracking each connections(e.g LRO) but >> low-end controllers may be not. It seems the controller does not >> even support RX TCP/UDP pseudo header checksum offloading so I >> wonder how this controller can support RX TCP/UDP checksum >> offloading for fragmented TCP/UDP packets. > I'm not sure what I'm missing here... You compute the sum over > the parts and add the sums of the parts. That should work for > the UDP/TCP checksum. >>=20 >> Some controllers maintain two bits for TCP/UDP checksum offloading >> result in status word. One bit is used to indicate whether >> controller performed TCP/UDP checksum offloading and the other bit >> is used to indicate whether the computed checksum is correct or > For SCTP we only get a bit that the checksum was computed and is = correct... >> not. For UDP checksum value 0x0000 and fragmented TCP/UDP packets, >> these controllers do not attempt to compute TCP/UDP checksum. > I think it "just" computes it and leaves it up to the upper layer > to use the result or not... >=20 > Best regards > Michael >>=20 >>> Best regards >>> Michael >>>>>>=20 >>>>>> Since you have the controller I guess it's easy to verify all >>>>>> cases. For case 3, I believe the controller can't handle >>>>>> fragmented frames so driver should have to explicitly check = ip_off >>>>>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) >>>>>> handle it. >>>>> Let me check this. Is there a tool to send UDP/TCP with IP level = options >>>>> or do I need to write a small test program myself? >>>>>=20 >>>>=20 >>>> I recall I used buggy ipsend of ipfilter package in the past but it >>>> would be more easy to write a simple test program or patch driver >>>> to generate those frames. >>>>=20 >>>=20 >>>=20 >>=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Sat May 10 01:00:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45C1DCB1 for ; Sat, 10 May 2014 01:00:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 32CA8E02 for ; Sat, 10 May 2014 01:00:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4A100Wn014261 for ; Sat, 10 May 2014 01:00:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4A100lt014260; Sat, 10 May 2014 01:00:00 GMT (envelope-from gnats) Date: Sat, 10 May 2014 01:00:00 GMT Message-Id: <201405100100.s4A100lt014260@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Keith Sklower Subject: Re: kern/174958: [net] [patch] rnh_walktree_from makes unreasonable assumptions Reply-To: Keith Sklower X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 01:00:01 -0000 The following reply was made to PR kern/174958; it has been noted by GNATS. From: Keith Sklower To: bug-followup@FreeBSD.org, melifaro@FreeBSD.org, sklower@cs.berkeley.edu Cc: zec@fer.hr Subject: Re: kern/174958: [net] [patch] rnh_walktree_from makes unreasonable assumptions Date: Fri, 9 May 2014 17:38:53 -0700 (PDT) Hi, I had many, many exchanges of email with Marko Zec concerning this issue; please keep it open. We're both agreed there is a problem, and we came up with a patch that he's OK with, but we didn't reach full agreement on some other issues which have implications for locking, which have subsequently addressed by somebody else. I would like to come up with a patch for FreeBSD 10 and revive the discussion, but I'm busy for the next couple of weeks. (and also a test harness which demonstrates casses where the distributed code fails, which can be run at user level). Keith > Date: Thu, 01 May 2014 19:18:41 +0400 > From: "Alexander V. Chernikov" > Subject: Re: kern/174958: [net] [patch] rnh_walktree_from makes unreasonable > assumptions > Hello! > Better late than never. > I'm a bit unsure how patch&test case in this PR relates to kern/174959. > Problems described here are the same as in 174959, however it looks like > given patch > addresses some other problem. It does not touch problem function at all, > but introduces a bunch > of new ones not used in test case. > Can you please provide some more details? From owner-freebsd-net@FreeBSD.ORG Sat May 10 11:59:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A398B30 for ; Sat, 10 May 2014 11:59:06 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39440661 for ; Sat, 10 May 2014 11:59:05 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1Wj5ve-0006Ob-RN for freebsd-net@freebsd.org; Sat, 10 May 2014 04:58:58 -0700 Date: Sat, 10 May 2014 04:58:58 -0700 (PDT) From: kariz To: freebsd-net@freebsd.org Message-ID: <1399723138841-5911165.post@n5.nabble.com> In-Reply-To: References: Subject: Re: running netmap-ipfw with real NICs MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 11:59:06 -0000 Is there anyone to help me?? -- View this message in context: http://freebsd.1045724.n5.nabble.com/running-netmap-ipfw-with-real-NICs-tp5906992p5911165.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sat May 10 15:30:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D029BC6 for ; Sat, 10 May 2014 15:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 38C9F8C7 for ; Sat, 10 May 2014 15:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4AFU1xB081644 for ; Sat, 10 May 2014 15:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4AFU0Eq081642; Sat, 10 May 2014 15:30:00 GMT (envelope-from gnats) Date: Sat, 10 May 2014 15:30:00 GMT Message-Id: <201405101530.s4AFU0Eq081642@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Mr. Clif" Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Reply-To: "Mr. Clif" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 15:30:01 -0000 The following reply was made to PR kern/188897; it has been noted by GNATS. From: "Mr. Clif" To: bug-followup@FreeBSD.org, clif@eugeneweb.com Cc: Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Date: Sat, 10 May 2014 08:29:03 -0700 This is a multi-part message in MIME format. --------------030302030805050404010504 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 04/30/2014 09:12 AM, John Baldwin wrote: > On Tuesday, April 29, 2014 7:42:29 pm Mr. Clif wrote: >> Hi Guys, >> >> I've been playing with the new patch, and found a few problems that I >> posted in PR 188897: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=188897 >> >> and this one is about another quad port card in the same Atom MoBo: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=188899 >> >> I would be happy to test any new fixes you want me to try. > Can you disable agp and see if that fixes things? (hint.agp.0.disabled=1 at > the loader prompt) > On 05/01/2014 09:27 AM, Mr. Clif wrote: > Ok, > > sorry about the delay, now that I'm set up again the turnaround time > should be a lot faster. > > At the loader prompt I typed: > > set hint.agp.0.disabled=1 > boot > > After a lot of testing, the only possible difference I can see is that > the re0 interface is always? there when the agp is disabled and not > always there by default. By the way, this board doesn't have an AGP > port. Oh and I only checked the dc board, since it didn't work I > didn't go on to the sun board. :-) > > Clif > On 05/08/2014 09:10 AM, John Baldwin wrote: > On Wednesday, May 07, 2014 6:37:59 pm Mr. Clif wrote: >> Ok sorry I didn't get anywhere with the AGP experiment. I guess I'll put >> the machine away until you have other things for me to try. > Sorry, I have been focusing on another PCI bug people were running into that > was fixed earlier this week. However, that one involved resources failing to > allocate at all, so I'm not sure it is related to your case. Hmm, I see your > last issue was related to the special ISA decoding bits. Can you pick one > of the cases to focus on for now and capture a verbose dmesg from a HEAD > kernel? 'devinfo -r' and 'devinfo -u' output from the non-working kernel > would also be good. > On 05/10/2014 08:18 AM, Mr. Clif wrote: > Opps I almost didn't see this email, too much spam I guess :-( > > Sounds like you want me to get the latest from the FreeBSD repo. Could > you please give me the URL and commands for that so I don't get the > wrong code? > > By last issue do you mean PR 179033? > Since that one was > closed I opened 188897 but they are both basically the same. > I'm not sure what symtoms the ISA decoding bits cause, to me it's just > that some of the NICs are unusable, but sure I can run those commands, > If you give me the git commands to get the correct code. I'll get the > non-working kernel now too. > > Clif --------------030302030805050404010504 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
On 04/30/2014 09:12 AM, John Baldwin wrote:
On Tuesday, April 29, 2014 7:42:29 pm Mr. Clif wrote:
 
Hi Guys,
 
 I've been playing with the new patch, and found a few problems that I 
 posted in PR 188897:
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=188897
 
 and this one is about another quad port card in the same Atom MoBo:
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=188899
 
 I would be happy to test any new fixes you want me to try.
 
Can you disable agp and see if that fixes things?  (hint.agp.0.disabled=1 at 
 the loader prompt)

On 05/01/2014 09:27 AM, Mr. Clif wrote:
Ok,

sorry about the delay, now that I'm set up again the turnaround time should be a lot faster.

At the loader prompt I typed:

set hint.agp.0.disabled=1
boot

After a lot of testing, the only possible difference I can see is that the re0 interface is always? there when the agp is disabled and not always there by default. By the way, this board doesn't have an AGP port. Oh and I only checked the dc board, since it didn't work I didn't go on to the sun board. :-)

    Clif


On 05/08/2014 09:10 AM, John Baldwin wrote:
On Wednesday, May 07, 2014 6:37:59 pm Mr. Clif wrote:
 
Ok sorry I didn't get anywhere with the AGP experiment. I guess I'll put 
 the machine away until you have other things for me to try.
 
Sorry, I have been focusing on another PCI bug people were running into that 
 was fixed earlier this week.  However, that one involved resources failing to 
 allocate at all, so I'm not sure it is related to your case.  Hmm, I see your
 last issue was related to the special ISA decoding bits.  Can you pick one
 of the cases to focus on for now and capture a verbose dmesg from a HEAD
 kernel?  'devinfo -r' and 'devinfo -u' output from the non-working kernel 
 would also be good.
On 05/10/2014 08:18 AM, Mr. Clif wrote:
Opps I almost didn't see this email, too much spam I guess :-(

Sounds like you want me to get the latest from the FreeBSD repo. Could you please give me the URL and commands for that so I don't get the wrong code?

By last issue do you mean PR 179033? Since that one was closed I opened 188897 but they are both basically the same.
I'm not sure what symtoms the ISA decoding bits cause, to me it's just that some of the NICs are unusable, but sure I can run those commands, If you give me the git commands to get the correct code. I'll get the non-working kernel now too.

    Clif
--------------030302030805050404010504-- From owner-freebsd-net@FreeBSD.ORG Sat May 10 15:50:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D23E8C2 for ; Sat, 10 May 2014 15:50:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E6D7DFD for ; Sat, 10 May 2014 15:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4AFo0dG094749 for ; Sat, 10 May 2014 15:50:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4AFo0AR094730; Sat, 10 May 2014 15:50:00 GMT (envelope-from gnats) Date: Sat, 10 May 2014 15:50:00 GMT Message-Id: <201405101550.s4AFo0AR094730@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Mr. Clif" Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Reply-To: "Mr. Clif" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 15:50:01 -0000 The following reply was made to PR kern/188897; it has been noted by GNATS. From: "Mr. Clif" To: bug-followup@FreeBSD.org, clif@eugeneweb.com Cc: Subject: Re: kern/188897: [dc] dc ethernet driver seems to prevent the detection of other NIC chipsets Date: Sat, 10 May 2014 08:49:24 -0700 Here is the devinfo output for the failed system: root@BSD10:~ # uname -a FreeBSD BSD10 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Fri Jan 17 01:46:25 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC i386 *************************************************************************** root@BSD10:~ # devinfo -r nexus0 npx0 apic0 I/O memory addresses: 0xfee00000-0xfee003ff ram0 I/O memory addresses: 0x0-0x8efff 0x90000-0x9ebff 0x100000-0xcee97fff 0xceebf000-0xcef47fff 0xcefbf000-0xceff0fff 0xcefff000-0xceffffff acpi0 Interrupt request lines: 0x9 I/O ports: 0x10-0x1f 0x72-0x73 0x80 0x84-0x86 0x88 0x8c-0x8e 0x90-0x9f 0x295-0x296 0x400-0x47f 0x500-0x53f 0x680-0x6ff I/O memory addresses: 0xc0000-0xdffff 0xe0000-0xfffff 0xf8000000-0xfbffffff 0xfed14000-0xfed17fff 0xfed18000-0xfed18fff 0xfed19000-0xfed19fff 0xfed1c000-0xfed1ffff 0xfff00000-0xffffffff cpu0 p4tcc0 cpufreq0 cpu1 p4tcc1 cpufreq1 cpu2 p4tcc2 cpufreq2 cpu3 p4tcc3 cpufreq3 acpi_button0 pcib0 I/O ports: 0xcf8-0xcff pci0 hostb0 vgapci0 I/O ports: 0x30c0-0x30c7 I/O memory addresses: 0xd0000000-0xdfffffff 0xe0200000-0xe02fffff 0xe0300000-0xe037ffff agp0 I/O memory addresses: 0xe0000000-0xe0000fff pcib1 I/O ports: 0x2000-0x20ff 0x2400-0x24ff 0x2800-0x28ff 0x2c00-0x2cff I/O memory addresses: 0xe0100000-0xe01fffff pci1 re0 Interrupt request lines: 0x100 pcib1 I/O port window: 0x2000-0x20ff pcib1 prefetch window: 0xe0100000-0xe0100fff 0xe0104000-0xe0107fff miibus0 rgephy0 pcib2 pci2 pcib3 pci3 pcib4 pci4 uhci0 Interrupt request lines: 0x17 I/O ports: 0x3080-0x309f usbus0 uhub0 uhci1 Interrupt request lines: 0x13 I/O ports: 0x3060-0x307f usbus1 uhub2 uhci2 Interrupt request lines: 0x12 I/O ports: 0x3040-0x305f usbus2 uhub1 uhci3 Interrupt request lines: 0x10 I/O ports: 0x3020-0x303f usbus3 uhub3 ehci0 Interrupt request lines: 0x17 I/O memory addresses: 0xe0380400-0xe03807ff usbus4 uhub4 pcib5 I/O ports: 0x1000-0x10ff 0x1400-0x14ff 0x1800-0x18ff 0x1c00-0x1cff pci5 pcib6 pcib5 I/O port window: 0x1000-0x10ff 0x1400-0x14ff 0x1800-0x18ff 0x1c00-0x1cff pci6 dc0 Interrupt request lines: 0x15 pcib6 I/O port window: 0x1400-0x147f miibus1 lxtphy0 dc1 Interrupt request lines: 0x16 pcib6 I/O port window: 0x1480-0x14ff miibus2 lxtphy1 dc2 Interrupt request lines: 0x17 pcib6 I/O port window: 0x1080-0x10ff miibus3 lxtphy2 dc3 Interrupt request lines: 0x14 pcib6 I/O port window: 0x1000-0x107f miibus4 lxtphy3 isab0 isa0 pmtimer0 sc0 vga0 I/O ports: 0x3c0-0x3df I/O memory addresses: 0xa0000-0xbffff ata0 Interrupt request lines: 0xe I/O ports: 0x1f0-0x1f7 0x3f6 ata1 Interrupt request lines: 0xf I/O ports: 0x170-0x177 0x376 atapci0 Interrupt request lines: 0x13 I/O ports: 0x30a0-0x30af 0x30b0-0x30b7 0x30b8-0x30bf 0x30c8-0x30cb 0x30cc-0x30cf I/O memory addresses: 0xe0380000-0xe03803ff ata2 ata3 acpi_sysresource0 pci_link0 pci_link1 pci_link2 pci_link3 pci_link4 pci_link5 pci_link6 pci_link7 atdma0 DMA request lines: 4 I/O ports: 0x0-0xf 0x81-0x83 0x87 0x89-0x8b 0x8f 0xc0-0xdf atrtc0 Interrupt request lines: 0x8 I/O ports: 0x70-0x71 0x74-0x77 atpic0 I/O ports: 0x20-0x3d 0xa0-0xbd 0x4d0-0x4d1 npxisa0 I/O ports: 0xf0 attimer0 Interrupt request lines: 0x0 I/O ports: 0x40-0x43 0x50-0x53 acpi_sysresource1 atkbdc0 Interrupt request lines: 0x1 I/O ports: 0x60 0x64 atkbd0 uart0 Interrupt request lines: 0x4 I/O ports: 0x3f8-0x3ff uart1 Interrupt request lines: 0x3 I/O ports: 0x2f8-0x2ff hpet0 Interrupt request lines: 0x14 I/O memory addresses: 0xfed00000-0xfed03fff acpi_timer0 ACPI I/O ports: 0x408-0x40b *************************************************************************** root@BSD10:~ # devinfo -u Interrupt request lines: 0x0 (attimer0) 0x1 (atkbdc0) 0x3 (uart1) 0x4 (uart0) 0x5-0x7 (root0) 0x8 (atrtc0) 0x9 (acpi0) 0xa-0xd (root0) 0xe (ata0) 0xf (ata1) 0x10 (uhci3) 0x11 (root0) 0x12 (uhci2) 0x13 (atapci0) 0x13 (uhci1) 0x14 (dc3) 0x14 (hpet0) 0x15 (dc0) 0x16 (dc1) 0x17 (dc2) 0x17 (ehci0) 0x17 (uhci0) 0x100 (re0) DMA request lines: 0-3 (root0) 4 (atdma0) 5-7 (root0) I/O ports: 0x0-0xf (atdma0) 0x10-0x1f (acpi0) 0x20-0x3d (atpic0) 0x3e-0x3f (root0) 0x40-0x43 (attimer0) 0x44-0x4f (root0) 0x50-0x53 (attimer0) 0x54-0x5f (root0) 0x60 (atkbdc0) 0x61 ---- 0x62-0x63 (root0) 0x64 (atkbdc0) 0x65-0x6f (root0) 0x70-0x71 (atrtc0) 0x72-0x73 (acpi0) 0x74-0x77 (atrtc0) 0x78-0x7f (root0) 0x80 (acpi0) 0x81-0x83 (atdma0) 0x84-0x86 (acpi0) 0x87 (atdma0) 0x88 (acpi0) 0x89-0x8b (atdma0) 0x8c-0x8e (acpi0) 0x8f (atdma0) 0x90-0x9f (acpi0) 0xa0-0xbd (atpic0) 0xbe-0xbf (root0) 0xc0-0xdf (atdma0) 0xe0-0xef (root0) 0xf0 (npxisa0) 0xf1-0x16f (root0) 0x170-0x177 (ata1) 0x178-0x1ef (root0) 0x1f0-0x1f7 (ata0) 0x1f8-0x294 (root0) 0x295-0x296 (acpi0) 0x297-0x2f7 (root0) 0x2f8-0x2ff (uart1) 0x300-0x375 (root0) 0x376 (ata1) 0x377-0x3bf (root0) 0x3c0-0x3df (vga0) 0x3e0-0x3f5 (root0) 0x3f6 (ata0) 0x3f7 (root0) 0x3f8-0x3ff (uart0) 0x400-0x47f (acpi0) 0x480-0x4cf (root0) 0x4d0-0x4d1 (atpic0) 0x4d2-0x4ff (root0) 0x500-0x53f (acpi0) 0x540-0x67f (root0) 0x680-0x6ff (acpi0) 0x700-0xcf7 (root0) 0xcf8-0xcff (pcib0) 0xd00-0xfff (root0) 0x1000-0x10ff (pcib5) 0x1100-0x13ff (root0) 0x1400-0x14ff (pcib5) 0x1500-0x17ff (root0) 0x1800-0x18ff (pcib5) 0x1900-0x1bff (root0) 0x1c00-0x1cff (pcib5) 0x1d00-0x1fff (root0) 0x2000-0x20ff (pcib1) 0x2100-0x23ff (root0) 0x2400-0x24ff (pcib1) 0x2500-0x27ff (root0) 0x2800-0x28ff (pcib1) 0x2900-0x2bff (root0) 0x2c00-0x2cff (pcib1) 0x2d00-0x2fff (root0) 0x3000-0x301f ---- 0x3020-0x303f (uhci3) 0x3040-0x305f (uhci2) 0x3060-0x307f (uhci1) 0x3080-0x309f (uhci0) 0x30a0-0x30af (atapci0) 0x30b0-0x30b7 (atapci0) 0x30b8-0x30bf (atapci0) 0x30c0-0x30c7 (vgapci0) 0x30c8-0x30cb (atapci0) 0x30cc-0x30cf (atapci0) 0x30d0-0xffff (root0) I/O memory addresses: 0x0-0x8efff (ram0) 0x8f000-0x8ffff (root0) 0x90000-0x9ebff (ram0) 0x9ec00-0x9ffff (root0) 0xa0000-0xbffff (vga0) 0xc0000-0xdffff (acpi0) 0xe0000-0xfffff (acpi0) 0x100000-0xcee97fff (ram0) 0xcee98000-0xceebefff (root0) 0xceebf000-0xcef47fff (ram0) 0xcef48000-0xcefbefff (root0) 0xcefbf000-0xceff0fff (ram0) 0xceff1000-0xceffefff (root0) 0xcefff000-0xceffffff (ram0) 0xcf000000-0xcfffffff (root0) 0xd0000000-0xdfffffff (vgapci0) 0xe0000000-0xe0000fff (agp0) 0xe0001000-0xe00fffff (root0) 0xe0100000-0xe01fffff (pcib1) 0xe0200000-0xe02fffff (vgapci0) 0xe0300000-0xe037ffff (vgapci0) 0xe0380000-0xe03803ff (atapci0) 0xe0380400-0xe03807ff (ehci0) 0xe0380800-0xf7ffffff (root0) 0xf8000000-0xfbffffff (acpi0) 0xfc000000-0xfebfffff (root0) 0xfec00000-0xfec000ff ---- 0xfec00100-0xfecfffff (root0) 0xfed00000-0xfed03fff (hpet0) 0xfed04000-0xfed13fff (root0) 0xfed14000-0xfed17fff (acpi0) 0xfed18000-0xfed18fff (acpi0) 0xfed19000-0xfed19fff (acpi0) 0xfed1a000-0xfed1bfff (root0) 0xfed1c000-0xfed1ffff (acpi0) 0xfed20000-0xfedfffff (root0) 0xfee00000-0xfee003ff (apic0) 0xfee00400-0xffefffff (root0) 0xfff00000-0xffffffff (acpi0) ACPI I/O ports: 0x10-0x1f (root0) 0x72-0x73 (root0) 0x80 (root0) 0x84-0x86 (root0) 0x88 (root0) 0x8c-0x8e (root0) 0x90-0x9f (root0) 0x295-0x296 (root0) 0x400-0x407 (root0) 0x408-0x40b (acpi_timer0) 0x40c-0x47f (root0) 0x500-0x53f (root0) 0x680-0x6ff (root0) ACPI I/O memory addresses: 0xc0000-0xfffff (root0) 0xf8000000-0xfbffffff (root0) 0xfed14000-0xfed19fff (root0) 0xfed1c000-0xfed1ffff (root0) 0xfff00000-0xffffffff (root0) pcib1 I/O port window: 0x2000-0x20ff (re0) 0x2400-0x24ff (root0) 0x2800-0x28ff (root0) 0x2c00-0x2cff (root0) pcib1 memory window: pcib1 prefetch window: 0xe0100000-0xe0100fff (re0) 0xe0101000-0xe0103fff (root0) 0xe0104000-0xe0107fff (re0) 0xe0108000-0xe01fffff (root0) pcib2 I/O port window: pcib2 memory window: pcib2 prefetch window: pcib3 I/O port window: pcib3 memory window: pcib3 prefetch window: pcib4 I/O port window: pcib4 memory window: pcib4 prefetch window: pcib5 I/O port window: 0x1000-0x10ff (pcib6) 0x1400-0x14ff (pcib6) 0x1800-0x18ff (pcib6) 0x1c00-0x1cff (pcib6) pcib5 memory window: pcib5 prefetch window: pcib6 I/O port window: 0x1000-0x107f (dc3) 0x1080-0x10ff (dc2) 0x1400-0x147f (dc0) 0x1480-0x14ff (dc1) 0x1800-0x18ff (root0) 0x1c00-0x1cff (root0) pcib6 memory window: pcib6 prefetch window: From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:25:45 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9AE3979 for ; Sun, 11 May 2014 20:25:45 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9ED082359 for ; Sun, 11 May 2014 20:25:45 +0000 (UTC) Received: from [75.98.19.132] (port=53425 helo=[10.104.61.82]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WjaJW-0002Hx-AU; Sun, 11 May 2014 16:25:39 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH 2/3] sfxge: TXQ index (not label) comes from FW in flush done event From: George Neville-Neil In-Reply-To: <5349126E.2060209@oktetlabs.ru> Date: Sun, 11 May 2014 16:25:36 -0400 X-Mao-Original-Outgoing-Id: 421532736.514549-92fea73e4e02b35fe0cfc2ff30e9a0a1 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5349126E.2060209@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:25:45 -0000 On Apr 12, 2014, at 6:16 , Andrew Rybchenko = wrote: > Change the second argument name of the efx_txq_flush_done_ev_t = prototype to > highlight that TXQ index (not label) comes from FW in flush done = event. > <2-sfxge-txq_index.patch> Tested, committed to HEAD, and MFC=92d. Thanks, George= From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:25:45 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC26C97A for ; Sun, 11 May 2014 20:25:45 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9ECAF2358 for ; Sun, 11 May 2014 20:25:45 +0000 (UTC) Received: from [75.98.19.132] (port=53425 helo=[10.104.61.82]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WjaJS-0002Hx-D3; Sun, 11 May 2014 16:25:35 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH 3/3] sfxge: use TXQ type as label to support more than 32 TXQs From: George Neville-Neil In-Reply-To: <534912B4.106@oktetlabs.ru> Date: Sun, 11 May 2014 16:25:31 -0400 X-Mao-Original-Outgoing-Id: 421532731.173966-b787ec6d9e2d08bfff3036b797a5b41d Content-Transfer-Encoding: quoted-printable Message-Id: References: <534912B4.106@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:25:45 -0000 On Apr 12, 2014, at 6:17 , Andrew Rybchenko = wrote: > There are 3 TXQs in event queue 0 and 1 TXQ (with TCP/UDP checksum = offload) > in all other event queues. > <3-sfxge-txq_type_as_label.patch> Tested, committed to HEAD, and MFC=92d. Thanks, George= From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:25:58 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3ECAAA87 for ; Sun, 11 May 2014 20:25:58 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 144AE235D for ; Sun, 11 May 2014 20:25:58 +0000 (UTC) Received: from [75.98.19.132] (port=53425 helo=[10.104.61.82]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WjaJn-0002Hx-Gf; Sun, 11 May 2014 16:25:56 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH 1/3] sfxge: RXQ index (not label) comes from FW in flush done/failed events From: George Neville-Neil In-Reply-To: <5349117B.4050505@oktetlabs.ru> Date: Sun, 11 May 2014 16:25:53 -0400 X-Mao-Original-Outgoing-Id: 421532753.584009-5377507d6055cbd165b9d205d31a7b17 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5349117B.4050505@oktetlabs.ru> To: Andrew Rybchenko X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:25:58 -0000 On Apr 12, 2014, at 6:12 , Andrew Rybchenko = wrote: > sfxge: RXQ index (not label) comes from FW in flush done/failed events >=20 > Change the second argument name of the efx_rxq_flush_done_ev_t and > efx_rxq_flush_failed_ev_t prototypes to highlight that RXQ index (not = label) > comes from FW in flush done and failed events. >=20 > Submitted by: Andrew Rybchenko > Sponsored by: Solarflare Communications, Inc. > <1-sfxge-rxq_index.patch> Tested, committed to HEAD, and MFC=92d. Thanks, George= From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:27:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3766EB3A for ; Sun, 11 May 2014 20:27:25 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B21F2391 for ; Sun, 11 May 2014 20:27:25 +0000 (UTC) Received: from [75.98.19.132] (port=53475 helo=[10.104.61.82]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WjaL9-0002Rg-KC; Sun, 11 May 2014 16:27:21 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Solarflare LACP bug? From: George Neville-Neil In-Reply-To: <0424FE83-E147-4F76-ACC8-D838140499F4@gmail.com> Date: Sun, 11 May 2014 16:27:14 -0400 X-Mao-Original-Outgoing-Id: 421532834.175612-93f23c038ef6486006eb4a4b48aa7091 Content-Transfer-Encoding: quoted-printable Message-Id: <921B83A5-D736-4D07-BDDA-2C092F75DA66@neville-neil.com> References: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> <53525B43.9050602@oktetlabs.ru> <0424FE83-E147-4F76-ACC8-D838140499F4@gmail.com> To: aurfalien X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: freebsd-net@freebsd.org, Andrew Rybchenko X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:27:25 -0000 On Apr 19, 2014, at 9:58 , aurfalien wrote: > On Apr 19, 2014, at 4:17 AM, Andrew Rybchenko = wrote: >=20 >> Hi, >>=20 >> On 04/16/2014 11:00 PM, aurfalien wrote: >>> Hi, >>>=20 >>> I=92ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. >>>=20 >>> While the card works fine as individual ports, upon configuring LACP = the machine suddenly reboots. >>>=20 >>> Here are my commands; >>>=20 >>> ifconfig sfxge0 up >>> ifconfig sfxge1 up >>> ifconfig lagg0 create >>> * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 = 10.0.10.99/16 >>>=20 >>> * This is were the system reboots. >> please, find patch attached. It solves the problem for me. >>=20 >> I'll discuss it with Solarflare and then submit patch to be pushed to = subversion. >>=20 >> Regards, >> Andrew. >=20 > Wow, that was very fast actually, many many thanks Andrew! >=20 Jsut to close the loop. Andrew did indeed send a patch which I tested = and committed. It has been MFC=92d to 10 today. Best, George From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:30:37 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE6BABE4; Sun, 11 May 2014 20:30:37 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1F132405; Sun, 11 May 2014 20:30:37 +0000 (UTC) Received: from [75.98.19.132] (port=53522 helo=[10.104.61.82]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1WjaOJ-0002ry-QT; Sun, 11 May 2014 16:30:36 -0400 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that From: George Neville-Neil In-Reply-To: <20140508200404.GA50446@glebius.int.ru> Date: Sun, 11 May 2014 16:30:32 -0400 X-Mao-Original-Outgoing-Id: 421533032.383534-3ca538796f8c228f87090c434c817a61 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140508200404.GA50446@glebius.int.ru> To: Gleb Smirnoff X-Mailer: Apple Mail (2.1874) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: Eygene Ryabinkin , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:30:37 -0000 On May 8, 2014, at 16:04 , Gleb Smirnoff wrote: > Eygene,=20 >=20 >=20 > On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: > E> - I'll do a patch for carp(4) that will allow it to use = configurable > E> OUI from a sysctl knob (first 5 bytes of OUI); >=20 > Please no sysctl knobs. This should be configurable via ifconfig(8) > per vhid. >=20 Agree, please do this via ifconfig. > P.S. Sorry for being silent on all legal and administrative questions. >=20 Whereas I don=92t have to be silen on this. Several of us are in = discussions about how to address the OUI allocation issue. Hopefully we=92ll have = something soon. Best, George= From owner-freebsd-net@FreeBSD.ORG Sun May 11 20:39:57 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CB63E82; Sun, 11 May 2014 20:39:57 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFCF0243C; Sun, 11 May 2014 20:39:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=WtrMijPtrb5uJKo+B/p69wzmxymOeiBxV11SOmcnSoE=; b=Yhhk8zg+LM49gVp7RSc78DS38yCLvC4T69F4KTBAcBHn1uxuaKUJfEdL6bs3RmvutZ6KPGLlI+uDRhMwMxfokXjv2kPxDpUNwheevz3wptUfLjiSiIt4mbbwW5g2FDwPOJwhRfztBpOGcty59dMOkcN3OmmHdNPV4SuVCiRrQTOwvwJQPYVBr+Z9CCbQV5HwyTehidqIJM72l45TxLjyxnI/0XXnnTjF4nvbiLc35lVkvCxomVttRJosoxS4rPIEr+0irfDSVJki5wPb8mAME+qxmglAUxU+REmksIL0CZw/J60a+Pj0eqs4u+D3egs7QCVYU55dCNPIguWD7hJQnnB02xSwx0T5sdUk9v+vZ93ikDqZkLXI0HCYh9a99zWcZyle1oPDUvLHFQon+JsKLtYsHZVWznP2efY4cpZRoEMzXyasTLuSNbwVkgYb4GCLbBFO1PfTHg6ga/0dIiJHMMZygwac/fvMm9teXox+pStueZEk+9QvvIQTmOX3h4y7vGEeXkESWY0OEhoznQH8KD/KO5OQdUzdpf7z718JqpkV+Gl9CNqZ8NKtmVKnxI2Fhb5ecE9MdJDZjX625ZBXeKOP+IiLcHKmd9xgViV7dTETdavrwd83/wSBOd/jABzZxLqJhTmZGoUjSnLrIcmPR8mcepeaO+3Aix22duv1YSQ=; Received: from light.codelabs.ru (v-light.codelabs.ru [144.206.233.83]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WjaXJ-000F3S-Ic; Mon, 12 May 2014 00:39:53 +0400 Date: Mon, 12 May 2014 00:39:49 +0400 From: Eygene Ryabinkin To: George Neville-Neil Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: References: <20140508200404.GA50446@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="cHMo6Wbp1wrKhbfi" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 20:39:57 -0000 --cHMo6Wbp1wrKhbfi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Sun, May 11, 2014 at 04:30:32PM -0400, George Neville-Neil wrote: > On May 8, 2014, at 16:04 , Gleb Smirnoff wrote: > > On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: > > E> - I'll do a patch for carp(4) that will allow it to use configurable > > E> OUI from a sysctl knob (first 5 bytes of OUI); > >=20 > > Please no sysctl knobs. This should be configurable via ifconfig(8) > > per vhid. >=20 > Agree, please do this via ifconfig. http://codelabs.ru/fbsd/carp-ouibase.diff --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --cHMo6Wbp1wrKhbfi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlNv4BRfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PtLTAD/Zd2wk8aoUpEKJUi+HAfBDWRU PDXAZv6ybsgVTdejN8QA+QFM5VY2/NPyQx1WVNvqF3HbN35VoKEgE672jHYAXxrf =yfH+ -----END PGP SIGNATURE----- --cHMo6Wbp1wrKhbfi-- From owner-freebsd-net@FreeBSD.ORG Sun May 11 23:42:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D9014CF for ; Sun, 11 May 2014 23:42:58 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70B552107 for ; Sun, 11 May 2014 23:42:58 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id fa1so2914799pad.25 for ; Sun, 11 May 2014 16:42:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=P4oJRARHZFznwcejltn9hjP4jk+0irVv1pUmrjs4Ptg=; b=Mo4+phwLhBqpHmZTZwsplO1bH2ZZKNL+4LVks7mS/iPzwB/jEyzFd5lzgTa7c+r6rI RW3kIjO0+sKRZXqSbwNqNprbe5C5+5iTMNc0zlZM4di+K4KnJPsoooDSv8rvhPRL1KUX 9a5jtdMixsx3PlXsHurSTU150V4vFSCpvxbWIrlFTMFg8/QdxMZJzW2JRxx5uY69Rc1Q j0ltFg2HR+C93cYOgQAY3UMDzgBB12YJyLqWrUm6UUkzs2zaBNVkjwkBzT2avvmnTUrH Lxdr8E7Mvcpbvg96b2jtPUtKKVxWa2EY+kqqbDaObPaU5/9SXjm2igjvdsQt7dQ5ScsR MH2Q== X-Received: by 10.67.5.233 with SMTP id cp9mr48144393pad.147.1399851778083; Sun, 11 May 2014 16:42:58 -0700 (PDT) Received: from [192.168.1.76] (75-63-29-182.lightspeed.irvnca.sbcglobal.net. [75.63.29.182]) by mx.google.com with ESMTPSA id gr10sm19406527pbc.84.2014.05.11.16.42.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 11 May 2014 16:42:56 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Solarflare LACP bug? From: aurfalien In-Reply-To: <921B83A5-D736-4D07-BDDA-2C092F75DA66@neville-neil.com> Date: Sun, 11 May 2014 16:42:54 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <354B61DF-E0E4-4F0F-86F2-095D38F31BE2@gmail.com> References: <2818B48D-3A2E-416A-875D-36DFD982D58A@gmail.com> <53525B43.9050602@oktetlabs.ru> <0424FE83-E147-4F76-ACC8-D838140499F4@gmail.com> <921B83A5-D736-4D07-BDDA-2C092F75DA66@neville-neil.com> To: George Neville-Neil X-Mailer: Apple Mail (2.1874) Cc: freebsd-net@freebsd.org, Andrew Rybchenko X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 23:42:58 -0000 On May 11, 2014, at 1:27 PM, George Neville-Neil = wrote: >=20 > On Apr 19, 2014, at 9:58 , aurfalien wrote: >=20 >> On Apr 19, 2014, at 4:17 AM, Andrew Rybchenko = wrote: >>=20 >>> Hi, >>>=20 >>> On 04/16/2014 11:00 PM, aurfalien wrote: >>>> Hi, >>>>=20 >>>> I=92ve a Solarflare SFN5162F dual port 10Gb ethernet adapter. >>>>=20 >>>> While the card works fine as individual ports, upon configuring = LACP the machine suddenly reboots. >>>>=20 >>>> Here are my commands; >>>>=20 >>>> ifconfig sfxge0 up >>>> ifconfig sfxge1 up >>>> ifconfig lagg0 create >>>> * ifconfig lagg0 up laggproto lacp laggport sfxge0 laggport sfxge1 = 10.0.10.99/16 >>>>=20 >>>> * This is were the system reboots. >>> please, find patch attached. It solves the problem for me. >>>=20 >>> I'll discuss it with Solarflare and then submit patch to be pushed = to subversion. >>>=20 >>> Regards, >>> Andrew. >>=20 >> Wow, that was very fast actually, many many thanks Andrew! >>=20 >=20 > Jsut to close the loop. Andrew did indeed send a patch which I tested = and committed. > It has been MFC=92d to 10 today. >=20 > Best, > George Hi and thanks for the update. Much appreciated. - aurf "Janitorial Services" From owner-freebsd-net@FreeBSD.ORG Mon May 12 01:36:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94C9ADC1; Mon, 12 May 2014 01:36:19 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D2E92926; Mon, 12 May 2014 01:36:19 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id fa1so3067795pad.39 for ; Sun, 11 May 2014 18:36:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=QWhrLmopZkjXi5YmQ7vxFmQtOdXksIqBLMSJ8DD/ZtU=; b=C4zqo7RZQKyIepzdHzy648c4f/K2plcfi2WUhGdLISH6VQjnfGY7ekuzT9XUOfozVI NZcmSJHq0dA3EnJgglzkfE4iRnBbFbIz6nmoILDEalMvEht60PAb4f54vS+Q/uffy3ao SHxQ4schrDHmnimx4znCQLz1VMEDprs9sdw9gZe95KUNicZnQgSn9V+EyaJ1jFhhfcBc HNeWQwoj4UMzu0EmP+5z3LGcNeigyRIOuPL3iPaPA2pKlcvZq9gd9L4bRB/XIbWdUa3l lbL5B1yQ/OMDBZSfVBgkdlbvXkJby81h0CDLOSNGsaug9ValOQEgbAsNokchczNlBMhW qtuA== X-Received: by 10.66.66.72 with SMTP id d8mr49207812pat.8.1399858579018; Sun, 11 May 2014 18:36:19 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id bz4sm19776431pbb.12.2014.05.11.18.36.15 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 11 May 2014 18:36:17 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 12 May 2014 10:36:12 +0900 From: Yonghyeon PYUN Date: Mon, 12 May 2014 10:36:12 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140512013612.GA4085@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 01:36:19 -0000 On Fri, May 09, 2014 at 12:46:48PM +0200, Michael Tuexen wrote: > On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: > > > On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: > >> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: > >> > >>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: > >>>> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: > >>>> > >>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: > >>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: > >>>>>> > >>>>>>> > >>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen wrote: > >>>>>>> > >>>>>>>> Dear all, > >>>>>>>> > >>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) accepts SCTP packet > >>>>>>>> with bad checksums. After debugging this I figured out that this is a problem with > >>>>>>>> the csum_flags defined in mbuf.h. > >>>>>>>> > >>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is defined in mbuf.h: > >>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID > >>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is considered > >>>>>>>> to have a correct checksum. > >>>>>>>> > >>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and set CSUM_DATA_VALID in > >>>>>>>> csum_flags to indicate that the UDP/TCP should consider csum_data to figure out if > >>>>>>>> the packet has a correct checksum. The problem is that CSUM_DATA_VALID is defined as > >>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID > >>>>>>>> In this case the semantic is not that the packet has a valid checksum, but the csum_data > >>>>>>>> field contains information. > >>>>>>>> > >>>>>>>> Now the following happens (on the raspberry pi the driver used is > >>>>>>>> dev/usb/net/if_smsc.c > >>>>>>>> > >>>>>>>> 1. A packet is received and if it is not too short, the checksum computed > >>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This happens > >>>>>>>> for all IP packets, not only for UDP and TCP packets. > >>>>>>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as CSUM_SCTP_VALID > >>>>>>>> and accepts the packet. So no SCTP checksum check ever happened. > >>>>>>>> > >>>>>>>> Alternatives to fix this: > >>>>>>>> > >>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP packets, since > >>>>>>>> it only makes sense in these cases. > >>>>>>> > >>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was actually checked but not otherwise. This is how it should be imho. It seems like a driver bug. > >>>>>> I went through the list of drivers and you are right, it seems to be a bug > >>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I can't tell. > >>>>>> > >>>>> > >>>>> I'm not sure how the controller computes TCP/UDP checksum values. > >>>>> It seems the publicly available data sheet was highly sanitized so > >>>>> it was useless to me. The comment in the driver says that the > >>>> Same for me... > >>>>> controller computes RX checksum after the IPv4 header to the end of > >>>>> ethernet frame. After seeing that comment, three questions popped > >>>>> up: > >>>>> > >> OK, I did some testing. It looks like the card is just computing the > >> checksum over the IP payload taking the correct IP header length into account. > >>>>> 1. Is the controller smart enough to skip IP options header in > >>>>> TCP/UDP checksum offloading? > >> Yes, I can send fragmented and un-fragmented UDP packets with IP options > >> and they are handled correctly. Even if the last fragment is too short. > > > > I'm assuming you're taking about receiving fragmented UDP packets > > with RX checksum offloading, right? > Correct. > > > >>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender > >>>>> didn't compute UDP checksum)? > >> This case isn't handled. However, udp_input() looks first for zero checksums > >> and only after that in the csum_flags. So it doesn't result in any problems. > >> Would you prefer not to set CSUM_DATA_VALID in this case? > > > > At least, it correctly updates UDP stats of netstat(1). > Let me double check that... > > > >>>>> 3. How the controller can compute TCP checksum of fragmented > >>>>> packets? > >> At least it does it right for UDP... > >> > > > > Hmm, CSUM_DATA_VALID indicates driver computed RX TCP/UDP checksum > > without pseudo header. As you know, controller can't compute > > TCP/UDP checksum until all its fragmented payload are read from > > wire. Packets may arrive out of order and may be mixed with other > I'm not sure I understand this... Please note that the pseudo header > is not taken into account. So the card can compute the checksum over > the payload of IP for each fragment. This is stored in the csum_data > field. During reassembly the csum_data fields of the fragments are > combined in > http://svnweb.freebsd.org/base/head/sys/netinet/ip_input.c?view=markup#l1075 > This looks OK to me. I'm not sure why you think the cards needs > to keep state for this. I understand it needs state if it wants > to take the pseudoheader into account, but this is not done here. Oops, sorry. You're right. Probably I was confused with old memory when I worked on that area. I've quickly read IP reassembly code again and as you said, it should work. However it seems there is a checksumming bug here. /* * In order to do checksumming faster we do 'end-around carry' here * (and not in for{} loop), though it implies we are not going to * reassemble more than 64k fragments. */ m->m_pkthdr.csum_data = (m->m_pkthdr.csum_data & 0xffff) + (m->m_pkthdr.csum_data >> 16); I guess the line above didn't account possible carry happened after the computation. Probably it could be rewritten as the following. while (m->m_pkthdr.csum_data & 0xffff0000) m->m_pkthdr.csum_data = (m->m_pkthdr.csum_data & 0xffff) + (m->m_pkthdr.csum_data >> 16); > > packets. Advanced controllers with enough memory may be able to > > compute TCP/UDP checksums by tracking each connections(e.g LRO) but > > low-end controllers may be not. It seems the controller does not > > even support RX TCP/UDP pseudo header checksum offloading so I > > wonder how this controller can support RX TCP/UDP checksum > > offloading for fragmented TCP/UDP packets. > I'm not sure what I'm missing here... You compute the sum over > the parts and add the sums of the parts. That should work for > the UDP/TCP checksum. > > > > Some controllers maintain two bits for TCP/UDP checksum offloading > > result in status word. One bit is used to indicate whether > > controller performed TCP/UDP checksum offloading and the other bit > > is used to indicate whether the computed checksum is correct or > For SCTP we only get a bit that the checksum was computed and is correct... > > not. For UDP checksum value 0x0000 and fragmented TCP/UDP packets, > > these controllers do not attempt to compute TCP/UDP checksum. > I think it "just" computes it and leaves it up to the upper layer > to use the result or not... > Yes, I stand corrected. Thanks. :-) > Best regards > Michael > > > >> Best regards > >> Michael > >>>>> > >>>>> Since you have the controller I guess it's easy to verify all > >>>>> cases. For case 3, I believe the controller can't handle > >>>>> fragmented frames so driver should have to explicitly check ip_off > >>>>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) > >>>>> handle it. > >>>> Let me check this. Is there a tool to send UDP/TCP with IP level options > >>>> or do I need to write a small test program myself? > >>>> > >>> > >>> I recall I used buggy ipsend of ipfilter package in the past but it > >>> would be more easy to write a simple test program or patch driver > >>> to generate those frames. > >>> > >> > >> > > > > From owner-freebsd-net@FreeBSD.ORG Mon May 12 01:44:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 678541FA for ; Mon, 12 May 2014 01:44:25 +0000 (UTC) Received: from mail-oa0-x242.google.com (mail-oa0-x242.google.com [IPv6:2607:f8b0:4003:c02::242]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FA9929F5 for ; Mon, 12 May 2014 01:44:25 +0000 (UTC) Received: by mail-oa0-f66.google.com with SMTP id l6so2304891oag.1 for ; Sun, 11 May 2014 18:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ieGGkq6P44WqU7Zt07s5n7KCrP6lEYZ+jyEPmQiSWL8=; b=yvf+WzBJLA0cdYrhuhJ1F1s2rLmOo1QXjjcM1S1HcPTpySiEL40rHyAB3mrPXkide4 w1iBuOn9rKm2qG6QhldlVsucJv7PSUsMrBZoJfoLUX/9RQw06xEkrFAlQs45EBsKguSZ NKB68/wciUdZkmvJBaETh3M433rj39xVBISOXbSHKSxpl8ACOOPFa4vwt+DQJAv5rJEp +FT4T7KLiXUyxmnpjyPuwsN+j32Hw6R/l/cFI3EQx1MQnV4Qgf8fDxILnnN57v/KlAea 1wRvCVSmbzWDOXslNNiVxbDE7Ll42Ly8p97Kv7/sTWuGF4ktQG9KZqTIGlKL/iGEbSMa 5+pw== MIME-Version: 1.0 X-Received: by 10.182.74.234 with SMTP id x10mr29450054obv.1.1399859064398; Sun, 11 May 2014 18:44:24 -0700 (PDT) Received: by 10.76.101.18 with HTTP; Sun, 11 May 2014 18:44:24 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 May 2014 09:44:24 +0800 Message-ID: Subject: Re: netmap: how to bridge 2 eth? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 01:44:25 -0000 seems disturbance in CentOS for netmap bridge... still not work after that... I'll try freebsd instead :) p.s. not showing my reply for Adrian here...? 2014-05-05 16:15 GMT+08:00 upyzl : > solved > I changed Ubuntu to CentOS 6.5 and use kernel 3.4.88 then netmap/vale > works OK...(but this time brctl & ping cannot work...well that's off-topic) > > > 2014-05-04 14:18 GMT+08:00 upyzl : > > add: >> >> tried >> ./vale-ctl -h vale0:eth0 >> ./vale-ctl -h vale0:eth1 >> and pinging >> >> get error: >> >> May 4 22:02:01 ubuntu kernel: [ 854.827834] 121.198610 [ 425] >> netmap_update_config configuration changed (but fine) >> May 4 22:02:01 ubuntu kernel: [ 854.828080] 121.198860 [1456] >> netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 >> May 4 22:02:01 ubuntu kernel: [ 854.828330] 121.199111 [1049] >> netmap_mem_global_config reconfiguring >> May 4 22:02:14 ubuntu kernel: [ 868.034578] 134.402766 [ 425] >> netmap_update_config configuration changed (but fine) >> May 4 22:02:14 ubuntu kernel: [ 868.034818] 134.403019 [1456] >> netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 >> May 4 22:02:44 ubuntu kernel: [ 897.779633] 164.142010 [1298] >> nm_txsync_prologue eth0 TX1 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 1 tail 511 >> May 4 22:02:44 ubuntu kernel: [ 897.780009] 164.142403 [1298] >> nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 0 tail 511 >> May 4 22:02:44 ubuntu kernel: [ 897.780284] 164.142679 [1373] >> nm_rxsync_prologue kring error: hwcur 0 rcur 0 hwtail 0 head 512 cur >> 0 tail 0 >> May 4 22:02:44 ubuntu kernel: [ 897.780563] 164.142944 [1549] >> netmap_vp_rxsync_locked ouch dangerous reset!!! >> May 4 22:02:44 ubuntu kernel: [ 897.780782] 164.143178 [1398] >> netmap_ring_reinit called for vale0:eth1 >> May 4 22:02:44 ubuntu kernel: [ 897.780969] 164.143365 [1423] >> netmap_ring_reinit total 1 errors >> May 4 22:02:44 ubuntu kernel: [ 897.781145] 164.143541 [1427] >> netmap_ring_reinit vale0:eth1 RX0 reinit, cur 0 -> 0 tail 0 -> 0 >> May 4 22:02:44 ubuntu kernel: [ 897.781393] 164.143789 [1298] >> nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 1 tail 511 >> May 4 22:02:44 ubuntu kernel: [ 897.781664] 164.144060 [1373] >> nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur >> 1 tail 1 >> May 4 22:02:44 ubuntu kernel: [ 897.781947] 164.144343 [1549] >> netmap_vp_rxsync_locked ouch dangerous reset!!! >> May 4 22:02:44 ubuntu kernel: [ 897.782165] 164.144561 [1398] >> netmap_ring_reinit called for vale0:eth1 >> May 4 22:02:44 ubuntu kernel: [ 897.782355] 164.144751 [1423] >> netmap_ring_reinit total 1 errors >> May 4 22:02:44 ubuntu kernel: [ 897.782533] 164.144929 [1427] >> netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 >> May 4 22:02:44 ubuntu kernel: [ 897.782783] 164.145178 [1298] >> nm_txsync_prologue eth1 TX1 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 1 tail 511 >> May 4 22:02:45 ubuntu kernel: [ 898.776184] 165.138378 [1298] >> nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 1 tail 511 >> May 4 22:02:45 ubuntu kernel: [ 898.776493] 165.138694 [1373] >> nm_rxsync_prologue kring error: hwcur 0 rcur 1 hwtail 1 head 512 cur >> 1 tail 1 >> May 4 22:02:45 ubuntu kernel: [ 898.776765] 165.138966 [1549] >> netmap_vp_rxsync_locked ouch dangerous reset!!! >> May 4 22:02:45 ubuntu kernel: [ 898.776987] 165.139189 [1398] >> netmap_ring_reinit called for vale0:eth1 >> May 4 22:02:45 ubuntu kernel: [ 898.777177] 165.139379 [1423] >> netmap_ring_reinit total 1 errors >> May 4 22:02:45 ubuntu kernel: [ 898.777357] 165.139558 [1427] >> netmap_ring_reinit vale0:eth1 RX0 reinit, cur 1 -> 0 tail 1 -> 1 >> May 4 22:02:45 ubuntu kernel: [ 898.777624] 165.139825 [1298] >> nm_txsync_prologue eth1 TX0 kring error: hwcur 0 rcur 0 hwtail 511 >> cur 2 tail 511 >> May 4 22:02:45 ubuntu kernel: [ 898.777896] 165.140097 [1373] >> nm_rxsync_prologue kring error: hwcur 0 rcur 2 hwtail 2 head 512 cur >> 2 tail 2 >> May 4 22:02:45 ubuntu kernel: [ 898.778159] 165.140361 [1549] >> netmap_vp_rxsync_locked ouch dangerous reset!!! >> >> >> then finally kernel panic/crash... >> >> >> 2014-05-03 14:44 GMT+08:00 upyzl : >> >> Hi all >>> >>> I'm doing to implement a simple openflow-based software switch(not same >>> as OVS, much simpler to OVS) >>> and I choose netmap for the network framework >>> >>> Now I try to do a simplest thing: >>> >>> there's 3 VMs, all are Ubuntu 12.04.4 x64 >>> 2 act as hosts, other 1 act as switch >>> >>> topo: [host1] eth0 ----- eth0 [switch] eth1 ----- eth0 [host2] >>> >>> I want to bridge switch's eth0ð1 (like "brctl" in linux), then host1 >>> ping host2 >>> >>> the problem is, how to bridge using netmap? >>> >>> I tried example "bridge" in netmap project(git clone >>> https://code.google.com/p/netmap/): >>> ./bridge -i netmap:eth0 -i netmap:eth1 >>> then host1 ping to host2 >>> but pinging result is "Destination Host Unreachable" (if using brctl, >>> pinging is fine) >>> >>> Then I tried vale-ctl >>> but i dont know how to use it... >>> e.g. >>> ./vale-ctl -a eth0 >>> show >>> "eth0: Invalid argument" >>> >>> could anyone help me? >>> >> >> > From owner-freebsd-net@FreeBSD.ORG Mon May 12 01:45:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0CD92AF; Mon, 12 May 2014 01:45:08 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B90452A0A; Mon, 12 May 2014 01:45:08 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id hz1so7173440pad.16 for ; Sun, 11 May 2014 18:45:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=h9u5zSGzUctVOzYMZzQFzS9KrTvGITLvf+UBJAeOU+Y=; b=wvbW36oeVGFDhfHg8xJqflK1aM12XHtVjj58hnoLCLMLjihnWHYvsdiE5ajb4WmBxc wIa14QGUuME0cocFAud6QfJFXTTgqT7SFNG22iqCstqscy2kBBpTLvcCOwP07Y29Rz0Z YkUQKF5qdm1LsEwQDB8QzncdrS4KHq0aHdbfWCYKmplhZ69+OyFmxmDWv0EKVh6oUDvC MfASW9POGExG/42njARcL7wM2g73p6+cRTUwTSQUAJlTyezB9a2HDE618vY9BhpvZzG6 XiepyTSPhQKmTgNOFF9eyvHlIKouXt8n6n4AxNXwkCItjXH53wYJYftgjvRi6hSUYMcl V4LA== X-Received: by 10.66.189.201 with SMTP id gk9mr49041804pac.25.1399859108400; Sun, 11 May 2014 18:45:08 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id pr4sm19763434pbb.53.2014.05.11.18.45.05 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 11 May 2014 18:45:07 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 12 May 2014 10:45:02 +0900 From: Yonghyeon PYUN Date: Mon, 12 May 2014 10:45:02 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140512014502.GB4085@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 01:45:09 -0000 On Fri, May 09, 2014 at 04:22:36PM +0200, Michael Tuexen wrote: > > On 09 May 2014, at 12:46, Michael Tuexen wrote: > > > On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: > > > >> On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: > >>> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: > >>> > >>>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: > >>>>> On 07 May 2014, at 09:56, Yonghyeon PYUN wrote: > >>>>> > >>>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: > >>>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb wrote: > >>>>>>> > >>>>>>>> > >>>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen wrote: > >>>>>>>> > >>>>>>>>> Dear all, > >>>>>>>>> > >>>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) accepts SCTP packet > >>>>>>>>> with bad checksums. After debugging this I figured out that this is a problem with > >>>>>>>>> the csum_flags defined in mbuf.h. > >>>>>>>>> > >>>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, which is defined in mbuf.h: > >>>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID > >>>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, the packet is considered > >>>>>>>>> to have a correct checksum. > >>>>>>>>> > >>>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum and set CSUM_DATA_VALID in > >>>>>>>>> csum_flags to indicate that the UDP/TCP should consider csum_data to figure out if > >>>>>>>>> the packet has a correct checksum. The problem is that CSUM_DATA_VALID is defined as > >>>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID > >>>>>>>>> In this case the semantic is not that the packet has a valid checksum, but the csum_data > >>>>>>>>> field contains information. > >>>>>>>>> > >>>>>>>>> Now the following happens (on the raspberry pi the driver used is > >>>>>>>>> dev/usb/net/if_smsc.c > >>>>>>>>> > >>>>>>>>> 1. A packet is received and if it is not too short, the checksum computed > >>>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. This happens > >>>>>>>>> for all IP packets, not only for UDP and TCP packets. > >>>>>>>>> 2. In case of SCTP packets, the SCTP interprets CSUM_DATA_VALID as CSUM_SCTP_VALID > >>>>>>>>> and accepts the packet. So no SCTP checksum check ever happened. > >>>>>>>>> > >>>>>>>>> Alternatives to fix this: > >>>>>>>>> > >>>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of UDP or TCP packets, since > >>>>>>>>> it only makes sense in these cases. > >>>>>>>> > >>>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was actually checked but not otherwise. This is how it should be imho. It seems like a driver bug. > >>>>>>> I went through the list of drivers and you are right, it seems to be a bug > >>>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a small set I can't tell. > >>>>>>> > >>>>>> > >>>>>> I'm not sure how the controller computes TCP/UDP checksum values. > >>>>>> It seems the publicly available data sheet was highly sanitized so > >>>>>> it was useless to me. The comment in the driver says that the > >>>>> Same for me... > >>>>>> controller computes RX checksum after the IPv4 header to the end of > >>>>>> ethernet frame. After seeing that comment, three questions popped > >>>>>> up: > >>>>>> > >>> OK, I did some testing. It looks like the card is just computing the > >>> checksum over the IP payload taking the correct IP header length into account. > >>>>>> 1. Is the controller smart enough to skip IP options header in > >>>>>> TCP/UDP checksum offloading? > >>> Yes, I can send fragmented and un-fragmented UDP packets with IP options > >>> and they are handled correctly. Even if the last fragment is too short. > >> > >> I'm assuming you're taking about receiving fragmented UDP packets > >> with RX checksum offloading, right? > > Correct. > >> > >>>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender > >>>>>> didn't compute UDP checksum)? > >>> This case isn't handled. However, udp_input() looks first for zero checksums > >>> and only after that in the csum_flags. So it doesn't result in any problems. > >>> Would you prefer not to set CSUM_DATA_VALID in this case? > >> > >> At least, it correctly updates UDP stats of netstat(1). > > Let me double check that... > I double checked it. The statistic counters are incremented. > Please note that we had a bug in the sending code of head, which > made it impossible to send UDP packets with 0 checksum. That is > fixed in > http://svnweb.freebsd.org/base?view=revision&revision=265776 Thanks. > > So any preference whether to report CSUM_DATA_VALID if a UDP packet > with checksum 0 is received or not? I'm pretty open, since it does > not have any effect right now... > Because upper stack correctly counts for these packets, your change (report CSUM_DATA_VALID for UDP packet with checksum value 0) looks fine. I don't remember how pf/ipf handles that case though but we can easily fix pf/ipf once we see breakage. > Best regards > Michael From owner-freebsd-net@FreeBSD.ORG Mon May 12 04:38:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E317EAC for ; Mon, 12 May 2014 04:38:42 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D76A266A for ; Mon, 12 May 2014 04:38:42 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id hz1so7576333pad.2 for ; Sun, 11 May 2014 21:38:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=wONzRumHtYe4xvpC+onlEH8HybX82eNnJNSvs1dOT3M=; b=fFGZMHkUK0gID9RaHSYZ+pMl9GQUL1AmDZBPe0CEVFWHAiUXP3eeGrKAiXJqUIIBW1 WlH40kD5gsK0sfmVCk/qHfCxNIC5m2Tllx71O89I6RsFYYDZ3DMjh4f+0lAl4vE0BTTz w/EUiaX3dZp4+b2HquGoQ5guX1AWlJNtgVkFHD5KgeziacD5ImJaQrGwKwtEFpaSTrxe urVS1pvtDaBz3180d9XxfwulnnlGuy/s4FBeT3YMagAEqfKEDJtn7mL2SqEq9BCuVjg9 5pWGVqzfJqMZhS3ObDOca+VTRLtMBm5aqKXjiCcg8jqfsrdQhE2P8hTnAsI+zGx8sY9l RDBQ== X-Received: by 10.66.156.34 with SMTP id wb2mr43411368pab.83.1399869521928; Sun, 11 May 2014 21:38:41 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id se3sm20329096pbb.80.2014.05.11.21.38.39 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 11 May 2014 21:38:41 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 12 May 2014 13:38:37 +0900 From: Yonghyeon PYUN Date: Mon, 12 May 2014 13:38:37 +0900 To: Michael Tuexen Subject: Re: TX Checksum offloading issue with re interfaces Message-ID: <20140512043837.GA1364@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> <20140509014733.GB3014@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 04:38:42 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, May 09, 2014 at 12:33:24PM +0200, Michael Tuexen wrote: > On 09 May 2014, at 03:47, Yonghyeon PYUN wrote: > > > On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: > >> Dear all, > >> > >> while testing checksum offloading of UDP packets over IP with IP options, I figured > >> out that my card > >> > >> dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet > >> dev.re.1.%driver: re > >> dev.re.1.%location: slot=0 function=0 handle=\_SB_.PCI0.PE1F.LAN2 > >> dev.re.1.%pnpinfo: vendor=0x10ec device=0x8168 subvendor=0x1734 subdevice=0x1159 class=0x020000 > >> dev.re.1.%parent: pci13 > >> dev.re.1.stats: -1 > >> dev.re.1.int_rx_mod: 65 > >> > >> computes the UDP checksum, but stores it in the packet at the place, where it would be, > >> if there are no IP options. So it corrupts the options in the packet... > >> > >> I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix it. Any idea? > >> > > > > re(4) has a very long history on its broken TX checksum offloading. > > So re(4) has many workarounds for known issues on several variants. > > re(4) controllers support TX IPv4/TCP/UDP checksum offloading. For > > 8168C/8168CP, TX IPv4 checksum offloading was disabled due to > > generation of corrupted frames. > > Could you show me the dmesg output(only re(4)/rgephy(4))? > > > The vendor uses the same PCI id for its RTL8168/8111 family chips > > so dmesg output is necessary to know exact controller revision. > Sure (re1 was used during the test): > re0: port 0x8000-0x80ff mem 0xf6104000-0xf6104fff,0xf6100000-0xf6103fff irq 16 at device 0.0 on pci12 > re0: Using 1 MSI-X message > re0: Chip rev. 0x28800000 > re0: MAC rev. 0x00200000 > miibus0: on re0 > rgephy0: PHY 1 on miibus0 > rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow > re0: Ethernet address: 00:19:99:85:31:d9 > re1: port 0x9000-0x90ff mem 0xf5c20000-0xf5c20fff,0xf6200000-0xf620ffff irq 17 at device 0.0 on pci13 > re1: Using 1 MSI-X message > re1: Chip rev. 0x3c800000 > re1: MAC rev. 0x00300000 > miibus1: on re1 > rgephy1: PHY 1 on miibus1 > rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow > re1: Ethernet address: 00:19:99:7e:c7:46 > It seems you have two variants. re0 is RTL8168DP and re1 is RTL8168CP. Do you see the issue on both controllers? I guess you may see the issue on re1 only since you've posted dev.re.1 output. I've attached a diff which may address the issue on re1 interface. If you see the issue on re0, I have to change the diff to include RTL8168D. --9jxsPFA5p3P2qPhR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="re.txcsum.diff" Index: sys/dev/re/if_re.c =================================================================== --- sys/dev/re/if_re.c (revision 265899) +++ sys/dev/re/if_re.c (working copy) @@ -1619,16 +1619,18 @@ re_attach(device_t dev) ifp->if_start = re_start; /* * RTL8168/8111C generates wrong IP checksummed frame if the - * packet has IP options so disable TX IP checksum offloading. + * packet has IP options so disable TX checksum offloading. */ if (sc->rl_hwrev->rl_rev == RL_HWREV_8168C || sc->rl_hwrev->rl_rev == RL_HWREV_8168C_SPIN2 || - sc->rl_hwrev->rl_rev == RL_HWREV_8168CP) - ifp->if_hwassist = CSUM_TCP | CSUM_UDP; - else + sc->rl_hwrev->rl_rev == RL_HWREV_8168CP) { + ifp->if_hwassist = 0; + ifp->if_capabilities = IFCAP_RXCSUM | IFCAP_TSO4; + } else { ifp->if_hwassist = CSUM_IP | CSUM_TCP | CSUM_UDP; + ifp->if_capabilities = IFCAP_HWCSUM | IFCAP_TSO4; + } ifp->if_hwassist |= CSUM_TSO; - ifp->if_capabilities = IFCAP_HWCSUM | IFCAP_TSO4; ifp->if_capenable = ifp->if_capabilities; ifp->if_init = re_init; IFQ_SET_MAXLEN(&ifp->if_snd, RL_IFQ_MAXLEN); @@ -3364,7 +3366,6 @@ re_ioctl(struct ifnet *ifp, u_long command, caddr_ struct rl_softc *sc = ifp->if_softc; struct ifreq *ifr = (struct ifreq *) data; struct mii_data *mii; - uint32_t rev; int error = 0; switch (command) { @@ -3453,15 +3454,9 @@ re_ioctl(struct ifnet *ifp, u_long command, caddr_ if ((mask & IFCAP_TXCSUM) != 0 && (ifp->if_capabilities & IFCAP_TXCSUM) != 0) { ifp->if_capenable ^= IFCAP_TXCSUM; - if ((ifp->if_capenable & IFCAP_TXCSUM) != 0) { - rev = sc->rl_hwrev->rl_rev; - if (rev == RL_HWREV_8168C || - rev == RL_HWREV_8168C_SPIN2 || - rev == RL_HWREV_8168CP) - ifp->if_hwassist |= CSUM_TCP | CSUM_UDP; - else - ifp->if_hwassist |= RE_CSUM_FEATURES; - } else + if ((ifp->if_capenable & IFCAP_TXCSUM) != 0) + ifp->if_hwassist |= RE_CSUM_FEATURES; + else ifp->if_hwassist &= ~RE_CSUM_FEATURES; reinit = 1; } --9jxsPFA5p3P2qPhR-- From owner-freebsd-net@FreeBSD.ORG Mon May 12 07:49:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5880D55 for ; Mon, 12 May 2014 07:49:50 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A55062512 for ; Mon, 12 May 2014 07:49:50 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WjkzX-0004Mu-AL for freebsd-net@freebsd.org; Mon, 12 May 2014 00:49:43 -0700 Date: Mon, 12 May 2014 00:49:43 -0700 (PDT) From: kariz To: freebsd-net@freebsd.org Message-ID: <1399880983187-5911668.post@n5.nabble.com> In-Reply-To: References: Subject: Re: running netmap-ipfw with real NICs MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 07:49:50 -0000 Hi, I tested my scenario. it is true. I could not receive packets in receiver and thought that my scenario is wrong!! My problem in this test solved by using latest version of netmap and netmap-based ipfw. Thanks from Luigi for his great work. -- View this message in context: http://freebsd.1045724.n5.nabble.com/running-netmap-ipfw-with-real-NICs-tp5906992p5911668.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Mon May 12 09:54:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12120685; Mon, 12 May 2014 09:54:52 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 967452092; Mon, 12 May 2014 09:54:51 +0000 (UTC) Received: from [10.225.7.42] (unknown [194.95.73.101]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 61F4E1C10491D; Mon, 12 May 2014 11:54:49 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140512014502.GB4085@michelle.cdnetworks.com> Date: Mon, 12 May 2014 11:54:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> <20140512014502.GB4085@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 09:54:52 -0000 On 12 May 2014, at 03:45, Yonghyeon PYUN wrote: > On Fri, May 09, 2014 at 04:22:36PM +0200, Michael Tuexen wrote: >>=20 >> On 09 May 2014, at 12:46, Michael Tuexen = wrote: >>=20 >>> On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: >>>=20 >>>> On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: >>>>> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: >>>>>=20 >>>>>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: >>>>>>> On 07 May 2014, at 09:56, Yonghyeon PYUN = wrote: >>>>>>>=20 >>>>>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >>>>>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb = wrote: >>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>>>>>>>>=20 >>>>>>>>>>> Dear all, >>>>>>>>>>>=20 >>>>>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>>>>>>>>> with bad checksums. After debugging this I figured out that = this is a problem with >>>>>>>>>>> the csum_flags defined in mbuf.h. >>>>>>>>>>>=20 >>>>>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, = which is defined in mbuf.h: >>>>>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>>>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, = the packet is considered >>>>>>>>>>> to have a correct checksum. >>>>>>>>>>>=20 >>>>>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum = and set CSUM_DATA_VALID in >>>>>>>>>>> csum_flags to indicate that the UDP/TCP should consider = csum_data to figure out if >>>>>>>>>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>>>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>>>>>>>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>>>>>>>>> field contains information. >>>>>>>>>>>=20 >>>>>>>>>>> Now the following happens (on the raspberry pi the driver = used is >>>>>>>>>>> dev/usb/net/if_smsc.c >>>>>>>>>>>=20 >>>>>>>>>>> 1. A packet is received and if it is not too short, the = checksum computed >>>>>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. = This happens >>>>>>>>>>> for all IP packets, not only for UDP and TCP packets. >>>>>>>>>>> 2. In case of SCTP packets, the SCTP interprets = CSUM_DATA_VALID as CSUM_SCTP_VALID >>>>>>>>>>> and accepts the packet. So no SCTP checksum check ever = happened. >>>>>>>>>>>=20 >>>>>>>>>>> Alternatives to fix this: >>>>>>>>>>>=20 >>>>>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of = UDP or TCP packets, since >>>>>>>>>>> it only makes sense in these cases. >>>>>>>>>>=20 >>>>>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was = actually checked but not otherwise. This is how it should be imho. It = seems like a driver bug. >>>>>>>>> I went through the list of drivers and you are right, it seems = to be a bug >>>>>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a = small set I can't tell. >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> I'm not sure how the controller computes TCP/UDP checksum = values. >>>>>>>> It seems the publicly available data sheet was highly sanitized = so >>>>>>>> it was useless to me. The comment in the driver says that the >>>>>>> Same for me... >>>>>>>> controller computes RX checksum after the IPv4 header to the = end of >>>>>>>> ethernet frame. After seeing that comment, three questions = popped >>>>>>>> up: >>>>>>>>=20 >>>>> OK, I did some testing. It looks like the card is just computing = the >>>>> checksum over the IP payload taking the correct IP header length = into account. >>>>>>>> 1. Is the controller smart enough to skip IP options header in >>>>>>>> TCP/UDP checksum offloading? >>>>> Yes, I can send fragmented and un-fragmented UDP packets with IP = options >>>>> and they are handled correctly. Even if the last fragment is too = short. >>>>=20 >>>> I'm assuming you're taking about receiving fragmented UDP packets >>>> with RX checksum offloading, right? >>> Correct. >>>>=20 >>>>>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender >>>>>>>> didn't compute UDP checksum)? >>>>> This case isn't handled. However, udp_input() looks first for zero = checksums >>>>> and only after that in the csum_flags. So it doesn't result in any = problems. >>>>> Would you prefer not to set CSUM_DATA_VALID in this case? >>>>=20 >>>> At least, it correctly updates UDP stats of netstat(1). >>> Let me double check that... >> I double checked it. The statistic counters are incremented. >> Please note that we had a bug in the sending code of head, which >> made it impossible to send UDP packets with 0 checksum. That is >> fixed in >> http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265776 >=20 > Thanks. >=20 >>=20 >> So any preference whether to report CSUM_DATA_VALID if a UDP packet >> with checksum 0 is received or not? I'm pretty open, since it does >> not have any effect right now... >>=20 >=20 > Because upper stack correctly counts for these packets, your change > (report CSUM_DATA_VALID for UDP packet with checksum value 0) looks > fine. I don't remember how pf/ipf handles that case though but we > can easily fix pf/ipf once we see breakage. OK. Best regards Michael >=20 >> Best regards >> Michael >=20 From owner-freebsd-net@FreeBSD.ORG Mon May 12 11:06:48 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FF95B55 for ; Mon, 12 May 2014 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F0B5826CE for ; Mon, 12 May 2014 11:06:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4CB6lYg067891 for ; Mon, 12 May 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4CB6lVD067889 for freebsd-net@FreeBSD.org; Mon, 12 May 2014 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 12 May 2014 11:06:47 GMT Message-Id: <201405121106.s4CB6lVD067889@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/189234 net [em] Big lag with Ethernet Connection I217-V o kern/189219 net [dummynet] [patch] using dummynet on sparc64 and confi o kern/188899 net [cas] cas ethernet driver seems to have issues with so o kern/188897 net [dc] dc ethernet driver seems to prevent the detection o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/186304 net [bmc] watchdog service causes BMC controller reset eve o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184389 net [libalias] libalias fails to adjust MTU from jails o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o kern/183920 net [ixgbe] [patch] Incorrect ifconfig media on INTEL X520 o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183381 net [em] [patch] Use of 9k buffers in if_em.c hangs with r o kern/183271 net [em] statistic not updated on em in netmap mode o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179473 net [new driver] if_vether.c: Source code contribution of o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171550 net [ndis] [patch] "no match for InterlockedCompareExchang o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [netinet] [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 482 problems total. From owner-freebsd-net@FreeBSD.ORG Mon May 12 11:09:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DCC83D2 for ; Mon, 12 May 2014 11:09:22 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E3C412876 for ; Mon, 12 May 2014 11:09:21 +0000 (UTC) Received: from [10.225.7.42] (unknown [194.95.73.101]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id C0B481C10491E; Mon, 12 May 2014 13:09:19 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: TX Checksum offloading issue with re interfaces From: Michael Tuexen In-Reply-To: <20140512043837.GA1364@michelle.cdnetworks.com> Date: Mon, 12 May 2014 13:09:18 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4DCF8CB0-220E-4BC4-AF71-E84CCDA88315@lurchi.franken.de> References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> <20140509014733.GB3014@michelle.cdnetworks.com> <20140512043837.GA1364@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 11:09:22 -0000 On 12 May 2014, at 06:38, Yonghyeon PYUN wrote: > On Fri, May 09, 2014 at 12:33:24PM +0200, Michael Tuexen wrote: >> On 09 May 2014, at 03:47, Yonghyeon PYUN wrote: >>=20 >>> On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: >>>> Dear all, >>>>=20 >>>> while testing checksum offloading of UDP packets over IP with IP = options, I figured >>>> out that my card >>>>=20 >>>> dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit = Ethernet >>>> dev.re.1.%driver: re >>>> dev.re.1.%location: slot=3D0 function=3D0 = handle=3D\_SB_.PCI0.PE1F.LAN2 >>>> dev.re.1.%pnpinfo: vendor=3D0x10ec device=3D0x8168 subvendor=3D0x1734= subdevice=3D0x1159 class=3D0x020000 >>>> dev.re.1.%parent: pci13 >>>> dev.re.1.stats: -1 >>>> dev.re.1.int_rx_mod: 65 >>>>=20 >>>> computes the UDP checksum, but stores it in the packet at the = place, where it would be, >>>> if there are no IP options. So it corrupts the options in the = packet... >>>>=20 >>>> I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix = it. Any idea? >>>>=20 >>>=20 >>> re(4) has a very long history on its broken TX checksum offloading. >>> So re(4) has many workarounds for known issues on several variants. >>> re(4) controllers support TX IPv4/TCP/UDP checksum offloading. For >>> 8168C/8168CP, TX IPv4 checksum offloading was disabled due to >>> generation of corrupted frames. >>> Could you show me the dmesg output(only re(4)/rgephy(4))? >>=20 >>> The vendor uses the same PCI id for its RTL8168/8111 family chips >>> so dmesg output is necessary to know exact controller revision. >> Sure (re1 was used during the test): >> re0: port = 0x8000-0x80ff mem 0xf6104000-0xf6104fff,0xf6100000-0xf6103fff irq 16 at = device 0.0 on pci12 >> re0: Using 1 MSI-X message >> re0: Chip rev. 0x28800000 >> re0: MAC rev. 0x00200000 >> miibus0: on re0 >> rgephy0: PHY 1 on miibus0 >> rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow >> re0: Ethernet address: 00:19:99:85:31:d9 >> re1: port = 0x9000-0x90ff mem 0xf5c20000-0xf5c20fff,0xf6200000-0xf620ffff irq 17 at = device 0.0 on pci13 >> re1: Using 1 MSI-X message >> re1: Chip rev. 0x3c800000 >> re1: MAC rev. 0x00300000 >> miibus1: on re1 >> rgephy1: PHY 1 on = miibus1 >> rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow >> re1: Ethernet address: 00:19:99:7e:c7:46 >>=20 > It seems you have two variants. You are right, I didn't know. Both are on-board interfaces... > re0 is RTL8168DP and re1 is RTL8168CP. Do you see the issue on > both controllers? I guess you may see the issue on re1 only since > you've posted dev.re.1 output. I've attached a diff which may It wasn't intentionally, but by accident, based on the addresses I was using. However, I now tested both interfaces and re0 works without any patch, but re1 needs your patch. > address the issue on re1 interface. If you see the issue on re0, > I have to change the diff to include RTL8168D. Your patch looks good. Please go ahead and commit it. Thanks for your help! Best regards Michael > From owner-freebsd-net@FreeBSD.ORG Mon May 12 11:22:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E8A2893; Mon, 12 May 2014 11:22:08 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4FD62A37; Mon, 12 May 2014 11:22:07 +0000 (UTC) Received: from [10.225.7.42] (unknown [194.95.73.101]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 8A48E1C104906; Mon, 12 May 2014 13:22:05 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140512013612.GA4085@michelle.cdnetworks.com> Date: Mon, 12 May 2014 13:22:03 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <72BEDE7B-C1F1-4EE2-BA68-49AED7209E8A@lurchi.franken.de> References: <0EB8F4F6-65C2-4B90-8101-FCC53A15C6F9@lurchi.franken.de> <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> <20140512013612.GA4085@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 11:22:08 -0000 On 12 May 2014, at 03:36, Yonghyeon PYUN wrote: > On Fri, May 09, 2014 at 12:46:48PM +0200, Michael Tuexen wrote: >> On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: >>=20 >>> On Thu, May 08, 2014 at 08:40:22PM +0200, Michael Tuexen wrote: >>>> On 07 May 2014, at 10:37, Yonghyeon PYUN wrote: >>>>=20 >>>>> On Wed, May 07, 2014 at 10:07:09AM +0200, Michael Tuexen wrote: >>>>>> On 07 May 2014, at 09:56, Yonghyeon PYUN = wrote: >>>>>>=20 >>>>>>> On Sat, May 03, 2014 at 11:52:47AM +0200, Michael Tuexen wrote: >>>>>>>> On 02 May 2014, at 16:02, Bjoern A. Zeeb = wrote: >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On 02 May 2014, at 10:22 , Michael Tuexen = wrote: >>>>>>>>>=20 >>>>>>>>>> Dear all, >>>>>>>>>>=20 >>>>>>>>>> during testing I found that FreeBSD head (on a raspberry pi) = accepts SCTP packet >>>>>>>>>> with bad checksums. After debugging this I figured out that = this is a problem with >>>>>>>>>> the csum_flags defined in mbuf.h. >>>>>>>>>>=20 >>>>>>>>>> The SCTP code on its input path checks for CSUM_SCTP_VALID, = which is defined in mbuf.h: >>>>>>>>>> #define CSUM_SCTP_VALID CSUM_L4_VALID >>>>>>>>>> This makes sense: If CSUM_SCTP_VALID is set in csum_flags, = the packet is considered >>>>>>>>>> to have a correct checksum. >>>>>>>>>>=20 >>>>>>>>>> For UDP and TCP some drivers calculate the UDP/TCP checksum = and set CSUM_DATA_VALID in >>>>>>>>>> csum_flags to indicate that the UDP/TCP should consider = csum_data to figure out if >>>>>>>>>> the packet has a correct checksum. The problem is that = CSUM_DATA_VALID is defined as >>>>>>>>>> #define CSUM_DATA_VALID CSUM_L4_VALID >>>>>>>>>> In this case the semantic is not that the packet has a valid = checksum, but the csum_data >>>>>>>>>> field contains information. >>>>>>>>>>=20 >>>>>>>>>> Now the following happens (on the raspberry pi the driver = used is >>>>>>>>>> dev/usb/net/if_smsc.c >>>>>>>>>>=20 >>>>>>>>>> 1. A packet is received and if it is not too short, the = checksum computed >>>>>>>>>> is stored in csum_data and the flag CSUM_DATA_VALID is set. = This happens >>>>>>>>>> for all IP packets, not only for UDP and TCP packets. >>>>>>>>>> 2. In case of SCTP packets, the SCTP interprets = CSUM_DATA_VALID as CSUM_SCTP_VALID >>>>>>>>>> and accepts the packet. So no SCTP checksum check ever = happened. >>>>>>>>>>=20 >>>>>>>>>> Alternatives to fix this: >>>>>>>>>>=20 >>>>>>>>>> 1. Change all drivers to set CSUM_DATA_VALID only in case of = UDP or TCP packets, since >>>>>>>>>> it only makes sense in these cases. >>>>>>>>>=20 >>>>>>>>> Wait, or for SCTP in cad the crc32 (I think it was) was = actually checked but not otherwise. This is how it should be imho. It = seems like a driver bug. >>>>>>>> I went through the list of drivers and you are right, it seems = to be a bug >>>>>>>> in if_smsc.c. Most of the other drivers check for UDP/TCP, a = small set I can't tell. >>>>>>>>=20 >>>>>>>=20 >>>>>>> I'm not sure how the controller computes TCP/UDP checksum = values. >>>>>>> It seems the publicly available data sheet was highly sanitized = so >>>>>>> it was useless to me. The comment in the driver says that the >>>>>> Same for me... >>>>>>> controller computes RX checksum after the IPv4 header to the end = of >>>>>>> ethernet frame. After seeing that comment, three questions = popped >>>>>>> up: >>>>>>>=20 >>>> OK, I did some testing. It looks like the card is just computing = the >>>> checksum over the IP payload taking the correct IP header length = into account. >>>>>>> 1. Is the controller smart enough to skip IP options header in >>>>>>> TCP/UDP checksum offloading? >>>> Yes, I can send fragmented and un-fragmented UDP packets with IP = options >>>> and they are handled correctly. Even if the last fragment is too = short. >>>=20 >>> I'm assuming you're taking about receiving fragmented UDP packets >>> with RX checksum offloading, right? >> Correct. >>>=20 >>>>>>> 2. How controller handles UDP checksum value 0x0000(i.e. sender >>>>>>> didn't compute UDP checksum)? >>>> This case isn't handled. However, udp_input() looks first for zero = checksums >>>> and only after that in the csum_flags. So it doesn't result in any = problems. >>>> Would you prefer not to set CSUM_DATA_VALID in this case? >>>=20 >>> At least, it correctly updates UDP stats of netstat(1). >> Let me double check that... >>>=20 >>>>>>> 3. How the controller can compute TCP checksum of fragmented >>>>>>> packets? >>>> At least it does it right for UDP... >>>>=20 >>>=20 >>> Hmm, CSUM_DATA_VALID indicates driver computed RX TCP/UDP checksum >>> without pseudo header. As you know, controller can't compute >>> TCP/UDP checksum until all its fragmented payload are read from >>> wire. Packets may arrive out of order and may be mixed with other >> I'm not sure I understand this... Please note that the pseudo header >> is not taken into account. So the card can compute the checksum over >> the payload of IP for each fragment. This is stored in the csum_data >> field. During reassembly the csum_data fields of the fragments are >> combined in >> = http://svnweb.freebsd.org/base/head/sys/netinet/ip_input.c?view=3Dmarkup#l= 1075 >> This looks OK to me. I'm not sure why you think the cards needs >> to keep state for this. I understand it needs state if it wants >> to take the pseudoheader into account, but this is not done here. >=20 > Oops, sorry. You're right. Probably I was confused with old memory > when I worked on that area. I've quickly read IP reassembly code > again and as you said, it should work. However it seems there is a > checksumming bug here. >=20 > /* > * In order to do checksumming faster we do 'end-around carry' = here > * (and not in for{} loop), though it implies we are not going = to > * reassemble more than 64k fragments. > */ > m->m_pkthdr.csum_data =3D > (m->m_pkthdr.csum_data & 0xffff) + (m->m_pkthdr.csum_data >> = 16); >=20 > I guess the line above didn't account possible carry happened after > the computation. Probably it could be rewritten as the following. >=20 > while (m->m_pkthdr.csum_data & 0xffff0000) > m->m_pkthdr.csum_data =3D (m->m_pkthdr.csum_data & = 0xffff) + > (m->m_pkthdr.csum_data >> 16); I think you are right here. Good catch. Will you fix it? Best regards Michael >=20 >>> packets. Advanced controllers with enough memory may be able to >>> compute TCP/UDP checksums by tracking each connections(e.g LRO) but >>> low-end controllers may be not. It seems the controller does not >>> even support RX TCP/UDP pseudo header checksum offloading so I >>> wonder how this controller can support RX TCP/UDP checksum >>> offloading for fragmented TCP/UDP packets. >> I'm not sure what I'm missing here... You compute the sum over >> the parts and add the sums of the parts. That should work for >> the UDP/TCP checksum. >>>=20 >>> Some controllers maintain two bits for TCP/UDP checksum offloading >>> result in status word. One bit is used to indicate whether >>> controller performed TCP/UDP checksum offloading and the other bit >>> is used to indicate whether the computed checksum is correct or >> For SCTP we only get a bit that the checksum was computed and is = correct... >>> not. For UDP checksum value 0x0000 and fragmented TCP/UDP packets, >>> these controllers do not attempt to compute TCP/UDP checksum. >> I think it "just" computes it and leaves it up to the upper layer >> to use the result or not... >>=20 >=20 > Yes, I stand corrected. Thanks. :-) >=20 >> Best regards >> Michael >>>=20 >>>> Best regards >>>> Michael >>>>>>>=20 >>>>>>> Since you have the controller I guess it's easy to verify all >>>>>>> cases. For case 3, I believe the controller can't handle >>>>>>> fragmented frames so driver should have to explicitly check = ip_off >>>>>>> field of IPv4 header. See how gem(4)/sk(4)/hme(4) and fxp(4) >>>>>>> handle it. >>>>>> Let me check this. Is there a tool to send UDP/TCP with IP level = options >>>>>> or do I need to write a small test program myself? >>>>>>=20 >>>>>=20 >>>>> I recall I used buggy ipsend of ipfilter package in the past but = it >>>>> would be more easy to write a simple test program or patch driver >>>>> to generate those frames. >>>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Mon May 12 11:28:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AAB5DAB1 for ; Mon, 12 May 2014 11:28:19 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 462A92A99 for ; Mon, 12 May 2014 11:28:19 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id r20so4291671wiv.3 for ; Mon, 12 May 2014 04:28:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=vUqtUZ3G3qsH/Ue3tLnE2Ry8CFbd1AVK9/Osi88xm9c=; b=YUr6/ptKCxXcHSoG2RnWk1GRanCdw8LsSh7Dpkl8V9xmP/STvIrYLPUakvgeIwcrnJ 1Pq7/cyJT2hIYJavaCKh7x/N/pPSJtk/sMZagVBgf4rx4z+JlXnLlBINeg1PGxbTfl0e /AfgyeVSZAqwlKEm+7yDiq2zXqEdWwgb8gviXTTll+8BissmfCdHzsEna8u6LeYYFDlg i3vGlaExCLovmPJUb4mWLp/zX9KORkdUlVlwcGsfur4AmLiFucDkFQnywbiKFiVL5HjT u99iEeMOU1gHza70ZioCKZAWoO2y0e/wrDh11xISXDs8lGP/cDIBJ4w+e+VHcgVGqDVh R+Gg== MIME-Version: 1.0 X-Received: by 10.194.187.50 with SMTP id fp18mr91752wjc.89.1399894097482; Mon, 12 May 2014 04:28:17 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Mon, 12 May 2014 04:28:17 -0700 (PDT) Reply-To: araujo@FreeBSD.org Date: Mon, 12 May 2014 19:28:17 +0800 Message-ID: Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that (Eygene Ryabinkin) From: Marcelo Araujo To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 11:28:19 -0000 2014-05-12 19:22 GMT+08:00 : > > 6. Re: Allowing CARP to use arbitrary OUI prefix and allocating > block from FreeBSD's OUI space assignment for that (Eygene Ryabinkin) > > > Message: 6 > Date: Mon, 12 May 2014 00:39:49 +0400 > From: Eygene Ryabinkin > To: George Neville-Neil > Cc: net@freebsd.org > Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating > block from FreeBSD's OUI space assignment for that > Message-ID: > Content-Type: text/plain; charset="us-ascii" > > Sun, May 11, 2014 at 04:30:32PM -0400, George Neville-Neil wrote: > > On May 8, 2014, at 16:04 , Gleb Smirnoff wrote: > > > On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: > > > E> - I'll do a patch for carp(4) that will allow it to use > configurable > > > E> OUI from a sysctl knob (first 5 bytes of OUI); > > > > > > Please no sysctl knobs. This should be configurable via ifconfig(8) > > > per vhid. > > > > Agree, please do this via ifconfig. > > http://codelabs.ru/fbsd/carp-ouibase.diff > > Dear Eygene, Just hold a bit your patch, till I provide mine version too as we spoke in another email. Thanks, I will keep you updated. -- Marcelo Araujo araujo@FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Mon May 12 17:43:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A21890C for ; Mon, 12 May 2014 17:43:58 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AD602E07 for ; Mon, 12 May 2014 17:43:57 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 97C35139C4 for ; Mon, 12 May 2014 14:43:52 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1399916628; x=1400780629; bh=bv9SZ6gdCYSu L2FaodVGuug5PagtYlWhgjSzmVMoLyA=; b=Uq9JRmSVhByNyiOmGD+R5mm+6yhA nnyxPsZWInaxHtkS35WvcwWPzJrV83cobaWhEqljMAeorC7PR6hZnWAqisg7vcXC XahMgthvVy6BmCcDzt/cRu3Q0CulnYVVn4Rwh32vo4UJcnn+tHRPm1Y0eAfTUIFp gVVJ5qqk9Wv9yRA= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9UojUKjoHmcu for ; Mon, 12 May 2014 14:43:48 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id B27F6139C3 for ; Mon, 12 May 2014 14:43:47 -0300 (BRT) Message-ID: <5371084F.1060009@bsdinfo.com.br> Date: Mon, 12 May 2014 14:43:43 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Problem with ipfw table add 0.0.0.0/8 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 17:43:58 -0000 Hi all, Today I discovered a likely problem: # ipfw table 99 add 0.0.0.0/8 # ipfw table 99 list ::/8 0 Is this correct? IPv6? # uname -a FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #6 r265408: Fri May 9 12:00:40 BRT 2014 root@mail.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM amd64 Cheers, Gondim From owner-freebsd-net@FreeBSD.ORG Mon May 12 18:02:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 897EA66C for ; Mon, 12 May 2014 18:02:07 +0000 (UTC) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47CD62FBC for ; Mon, 12 May 2014 18:02:07 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id rd18so2462936iec.38 for ; Mon, 12 May 2014 11:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=yy9wgFQhZuqihHO/aR37oI93eeWWi2L1x/bTbMw6ZvA=; b=efrEnxHEOLxLX7hktvc/HqXMLvwh51kQ6Z9aa/Vl+TwyeWKeTpbafBrEmtH2I24Dpr +RtEq9KeXk0LqDhW9De6R50sjLDKl3ihhwb0G4cPfQikK5eYYI9qI/YonQifO/ohQHr4 f/nzgsLJEexVuY+T9/JKBQCuVb/KHvVKiE75g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=yy9wgFQhZuqihHO/aR37oI93eeWWi2L1x/bTbMw6ZvA=; b=OxEn8L9wTpCcCo1kny2Ud/9LzEK7WbTa2Jyj6ctfQWh+MhQ7Q1tZwmG82p92AP7G7e lfheKAxGUw1W/it5mFvisWFXfBuI9ORH1kL/gZfRuGigCTg0+XBQN/gvT97jvAGBChAn q77uREzKSH3OiUyfaEMDAgYlH241Fh8TmPrI5lSp1+4ifpb3lMQhURJmCtk465RNYYZ5 zDZJrbUFKAEPOcRwiK8FmiblVXP2OCDC0TnjA0lmVA6iskEM6jZ2kATesqtIupA8ZVln QQDQc11ktgvr61+EWogMIE+v8oL77wDZHq8qvk6vZQ20qfNI05Bl1IbJFCA8BLdzSL9P HROQ== X-Gm-Message-State: ALoCoQlTklp0MKZ6QpfVZtKEvl10vCX6B4sZ5y4xgpKTbFQfLESsjBv8O9K3S4qtS10jHMZiKr7S X-Received: by 10.50.147.9 with SMTP id tg9mr48040492igb.31.1399917726508; Mon, 12 May 2014 11:02:06 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id i16sm23874007igf.11.2014.05.12.11.02.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 12 May 2014 11:02:04 -0700 (PDT) References: <5371084F.1060009@bsdinfo.com.br> Mime-Version: 1.0 (1.0) In-Reply-To: <5371084F.1060009@bsdinfo.com.br> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-66B94B0A-99D8-402D-8654-40B94754CBCD; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Problem with ipfw table add 0.0.0.0/8 Date: Mon, 12 May 2014 14:02:00 -0400 To: Marcelo Gondim X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 18:02:07 -0000 --Apple-Mail-66B94B0A-99D8-402D-8654-40B94754CBCD Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cute. Same this happen when there are paren around the quad ? --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On May 12, 2014, at 13:43, Marcelo Gondim wrote: >=20 > Hi all, >=20 > Today I discovered a likely problem: >=20 > # ipfw table 99 add 0.0.0.0/8 >=20 > # ipfw table 99 list > ::/8 0 >=20 > Is this correct? IPv6? >=20 > # uname -a > FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #6 r265408: Fri= May 9 12:00:40 BRT 2014 root@mail.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDI= M amd64 >=20 > Cheers, > Gondim > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --Apple-Mail-66B94B0A-99D8-402D-8654-40B94754CBCD Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDUxMjE4MDIwMVowIwYJKoZIhvcN AQkEMRYEFJjQUxeI2Frz5u4xDk8RH4sf6bVsMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAUwkVKLzOFbJiQ1/mQo9R 2mRxFtBPMwb0avQQOtTZ0Z1lBc+CuQKMuQlY/xWNNqZQ/nd5dETgc6RuEeNoPxui8ogNURkbKKlm fm1iR8i9mHZ01L2JIvFvzhFKLAolH1N8n+f1AyAIZQePFZY2nxBi9p8Tfpwv1a9I3aN7Nn35pgXL vT0K5O/nM43rQnZqQ6TNxzMcT8p2DNTt121bmp1iEbLGe2Zc5csBEobxRzf6L7F52Aobh9lfSqx6 aM7ZPJXL2qT7hvmUmcY0OOygg8CUCBO5Xtjc5rZbg4EHGHXMxhZ+oqjRQfoqxgEumAwEeAjfiiic 24xyypSFTP4Vf5UzxwAAAAAAAA== --Apple-Mail-66B94B0A-99D8-402D-8654-40B94754CBCD-- From owner-freebsd-net@FreeBSD.ORG Mon May 12 18:21:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07031AA4 for ; Mon, 12 May 2014 18:21:39 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8C94217A for ; Mon, 12 May 2014 18:21:38 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id E91BD139C4 for ; Mon, 12 May 2014 15:21:39 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1399918895; x=1400782896; bh=GcJlNSTvUZiT 3W0b0YT1xfMcdPO7DN3SwDtYbkJzF1M=; b=FeREXwrZgdBMV/BEJCw8D9NQOYxo gj3fg1xmX21nv0h+Jta8pt1fMcf/F9SjOnKvNVd7kLbbBJZUbdwEeLDNsWGBpIDu MPTHc27FiBRCTYNxVY6wgF8AhK8inaJ61QCMzxvejDhlx/DS2I+fyb/y+tDY0KaW KQomwgd5bE1Zuhw= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eHWd70vbfBE7 for ; Mon, 12 May 2014 15:21:35 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 4220C139C3 for ; Mon, 12 May 2014 15:21:35 -0300 (BRT) Message-ID: <5371112B.2030209@bsdinfo.com.br> Date: Mon, 12 May 2014 15:21:31 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 18:21:39 -0000 Hi Jason, Same problem. Em 12/05/14 15:02, Jason Hellenthal escreveu: > Cute. Same this happen when there are paren around the quad ? > -- Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On May 12, 2014, at 13:43, Marcelo Gondim wrote: > > Hi all, > > Today I discovered a likely problem: > > # ipfw table 99 add 0.0.0.0/8 > > # ipfw table 99 list > ::/8 0 > > Is this correct? IPv6? > > # uname -a > FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #6 r265408: Fri May 9 12:00:40 BRT 2014root@mail.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM amd64 > > Cheers, > Gondim > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to"freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon May 12 22:54:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1704E904 for ; Mon, 12 May 2014 22:54:17 +0000 (UTC) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CA3362EC0 for ; Mon, 12 May 2014 22:54:16 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 1256828423 for ; Tue, 13 May 2014 00:54:08 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 3BCEA28422 for ; Tue, 13 May 2014 00:54:07 +0200 (CEST) Message-ID: <5371510E.40302@quip.cz> Date: Tue, 13 May 2014 00:54:06 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: FreeBSD Net Subject: Best practices with network settings for virtualization Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 22:54:17 -0000 I originaly posted this to virtualization@ list week ago. I didn't recieved any answer, so maybe this list is better for questions like the following. I would like to ask some really experienced person - what is the best way to run virtual guests connected to network with public IPs? I think many people run unsecure setup with guests with simple bridged network. I know there are many options with tun, bridge, epair, VDE, Open vSwitch etc., my main concern is the setup of network where each guest can use only predefined MAC and predefined IP(s). If some malicious user or malware in guest OS tried to change MAC od IP, I would like to disallow that or do not allow any offending traffic to reach outside network or any other guest running on the same machine. Guests can be VirtualBox, Bhyve or anything else. I really appreciate any help or ideas. -- Miroslav Lachman From owner-freebsd-net@FreeBSD.ORG Mon May 12 23:56:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5921453A for ; Mon, 12 May 2014 23:56:49 +0000 (UTC) Received: from mail-ve0-x22c.google.com (mail-ve0-x22c.google.com [IPv6:2607:f8b0:400c:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 185142433 for ; Mon, 12 May 2014 23:56:49 +0000 (UTC) Received: by mail-ve0-f172.google.com with SMTP id oz11so9806775veb.31 for ; Mon, 12 May 2014 16:56:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Ldt3riQrHWPRPsKKlcFkUbSMkTmiEoAfRiCM+P22uWo=; b=tNi0Ezl/SB5LUpx/kSMxm2obnNaNpYxOkNMqyv8zgQ77cLtB4asIr1Q/X5BGAY+BKS UpTihi2GA1WyZseIM1dVkOIBAsYfgdBD5OkiAhPJ3ZmP9YL3EUby5DeopDswKBSrqz2a TBQFgLq4P1dciUrfdBP/iv63jX9XLwT8i3BE3IZsncZGP+wl3OY7WewDIvGMUylg4pOM lRJs5qFn0L4nDgJ8puI75LMEoRIw+hGMVxsN1f968bwkUp4HQoA7gQEjvWsTTigU17Aw UaCVhcCflkgH+RVDHXb/eTy4ks4bo2p4BeoRFr+c4LH7am6wPjznS9HNXAMX9TRvTpY6 DcaQ== MIME-Version: 1.0 X-Received: by 10.58.23.6 with SMTP id i6mr25423881vef.12.1399939008153; Mon, 12 May 2014 16:56:48 -0700 (PDT) Received: by 10.52.98.130 with HTTP; Mon, 12 May 2014 16:56:48 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 May 2014 16:56:48 -0700 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Nick Rogers To: "freebsd-net@freebsd.org" , "jfvogel@gmail.com" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 23:56:49 -0000 On Fri, May 2, 2014 at 1:49 PM, Nick Rogers wrote: > Hello, > > I am hoping someone can help me debug a kernel panic I have been experiencing > on one of my production systems. The system is a PF+ALTQ firewall/gateway with > about 1k simultaneous devices behind it at any given time, pushing no more > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it > with kgdb, but am still at a loss. > > I have completely replaced the hardware to rule out issues with > disk/memory/etc, and it appears to be a kernel issue, likely with e1000 driver > and/or PF. > > The panic seems to happen during times of heavier use, but the frequency is > not very predictable (anywhere from a few times a day to a once a week), so I > feel like its some kind of strange traffic pattern that I am unable to > pinpoint. > > I am using a slightly custom kernel based on GENERIC, mainly to bring in ALTQ > and some other options. > > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 > driver, although I do not believe that should affect em(4). > > Hoping someone can be of assistance in debugging this problem. I am willing to > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if that > is advisable. Unfortunately I cannot try 10-STABLE. > > Thanks, > > -Nick Rogers > > > uname -v > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS > > > > Here is the kernel conf... > .............................................................................. > include GENERIC > > ident RGNETS > > makeoptions MODULES_OVERRIDE="" > > options DEVICE_POLLING > > device crypto > device cryptodev > > options VLAN_ARRAY > > device amdtemp > > # PF > device pf > device pflog > options ALTQ > options ALTQ_CBQ # Class Bases Queuing (CBQ) > options ALTQ_RED # Random Early Detection (RED) > options ALTQ_RIO # RED In/Out > options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) > options ALTQ_PRIQ # Priority Queuing (PRIQ) > options ALTQ_NOPCC # Required for SMP build > > # PPPoE > options NETGRAPH > options NETGRAPH_ETHER > options NETGRAPH_PPPOE > options NETGRAPH_SOCKET > > # IPsec > device enc > options IPSEC > options IPSEC_FILTERTUNNEL > options IPSEC_NAT_T > .............................................................................. > > > The crash dump.... > .............................................................................. > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 5; apic id = 05 > fault virtual address = 0x10 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8033d350 > stack pointer = 0x28:0xffffff83545384b0 > frame pointer = 0x28:0xffffff83545384c0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (irq262: em2:rx 0) > trap number = 12 > panic: page fault > cpuid = 5 > KDB: stack backtrace: > #0 0xffffffff80956836 at kdb_backtrace+0x66 > #1 0xffffffff8091c40e at panic+0x1ce > #2 0xffffffff80d31e70 at trap_fatal+0x290 > #3 0xffffffff80d321d1 at trap_pfault+0x211 > #4 0xffffffff80d327d3 at trap+0x363 > #5 0xffffffff80d1b9d3 at calltrap+0x8 > #6 0xffffffff8034872d at pf_test_rule+0x17ed > #7 0xffffffff8034ba12 at pf_test+0x1032 > #8 0xffffffff8035112b at pf_check_in+0x2b > #9 0xffffffff809e952e at pfil_run_hooks+0x9e > #10 0xffffffff80a5286a at ip_input+0x2ea > #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #12 0xffffffff809df93d at ether_demux+0x14d > #13 0xffffffff809dfc1e at ether_nh_input+0x1fe > #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #15 0xffffffff809df85f at ether_demux+0x6f > #16 0xffffffff809dfc1e at ether_nh_input+0x1fe > #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 > Uptime: 17d7h20m59s > Dumping 2932 out of 12256 MB: (CTRL-C to abort) > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/aio.ko...Reading symbols from > /boot/kernel/aio.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/aio.ko > Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from > /boot/kernel/coretemp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/coretemp.ko > Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from > /boot/kernel/cc_htcp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/cc_htcp.ko > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > 234 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) list *0xffffffff8033d350 > 0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512). > 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) > 508 { > 509 switch (af) { > 510 #ifdef INET > 511 case AF_INET: > 512 dst->addr32[0] = src->addr32[0]; > 513 break; > 514 #endif /* INET */ > 515 case AF_INET6: > 516 dst->addr32[0] = src->addr32[0]; > (kgdb) backtrace > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > #1 0xffffffff8091bee6 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:454 > #2 0xffffffff8091c3e7 in panic (fmt=0x1
) > at /usr/src/sys/kern/kern_shutdown.c:642 > #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is > not available. > ) at /usr/src/sys/amd64/amd64/trap.c:878 > #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 > #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at > /usr/src/sys/amd64/amd64/trap.c:456 > #6 0xffffffff80d1b9d3 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:232 > #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, > src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 > #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, > sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, > m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, > pd=0xffffff83545386c0, am=0xffffff8354538790, > rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at > /usr/src/sys/contrib/pf/net/pf.c:3900 > #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not available. > ) at /usr/src/sys/contrib/pf/net/pf.c:6776 > #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available. > ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 > #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not available. > ) at /usr/src/sys/net/pfil.c:82 > #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at > /usr/src/sys/netinet/ip_input.c:510 > #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 > #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 > #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, > done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 > #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available. > ) at /usr/src/sys/dev/e1000/if_em.c:1593 It seems like the panic is in the MSIX receive handler? I guess I will try disabling MSIX for em. Jack, can you comment on this please? > #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" > is not available. > ) at /usr/src/sys/kern/kern_intr.c:1272 > #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at > /usr/src/sys/kern/kern_intr.c:1285 > #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 > , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at > /usr/src/sys/kern/kern_fork.c:996 > #25 0xffffffff80d1befe in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #26 0x0000000000000000 in ?? () > #27 0x0000000000000000 in ?? () > #28 0x0000000000000001 in ?? () > #29 0x0000000000000000 in ?? () > #30 0x0000000000000000 in ?? () > #31 0x0000000000000000 in ?? () > #32 0x0000000000000000 in ?? () > #33 0x0000000000000000 in ?? () > #34 0x0000000000000000 in ?? () > #35 0x0000000000000000 in ?? () > #36 0x0000000000000000 in ?? () > #37 0x0000000000000000 in ?? () > #38 0x0000000000000000 in ?? () > #39 0x0000000000000000 in ?? () > #40 0x0000000000000000 in ?? () > #41 0x0000000000000000 in ?? () > #42 0x0000000000000000 in ?? () > #43 0x0000000000000000 in ?? () > #44 0x0000000000000000 in ?? () > #45 0x0000000000000000 in ?? () > #46 0x0000000000000000 in ?? () > #47 0x0000000000000000 in ?? () > #48 0x0000000000000000 in ?? () > ---Type to continue, or q to quit--- > #49 0x0000000000000000 in ?? () > #50 0x0000000000000000 in ?? () > #51 0xfffffe0007304920 in ?? () > #52 0xfffffe0003afd000 in ?? () > #53 0xfffffe0007304920 in ?? () > #54 0xffffff8354538b40 in ?? () > #55 0xffffff8354538ae8 in ?? () > #56 0xfffffe0003b00920 in ?? () > #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, > newtd=0xfffffe000730c300, flags=Variable "flags" is not available. > ) at /usr/src/sys/kern/sched_ule.c:1898 > Previous frame inner to this frame (corrupt stack?) > > .............................................................................. > > > > Relevant pciconf output... > .............................................................................. > > em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > .............................................................................. > > > dev.em sysctl.... > .............................................................................. > > dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.0.%driver: em > dev.em.0.%location: slot=0 function=0 > dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.0.%parent: pci1 > dev.em.0.nvm: -1 > dev.em.0.debug: -1 > dev.em.0.fc: 3 > dev.em.0.rx_int_delay: 0 > dev.em.0.tx_int_delay: 66 > dev.em.0.rx_abs_int_delay: 66 > dev.em.0.tx_abs_int_delay: 66 > dev.em.0.itr: 488 > dev.em.0.rx_processing_limit: -1 > dev.em.0.eee_control: 1 > dev.em.0.link_irq: 2 > dev.em.0.mbuf_alloc_fail: 0 > dev.em.0.cluster_alloc_fail: 0 > dev.em.0.dropped: 0 > dev.em.0.tx_dma_fail: 0 > dev.em.0.rx_overruns: 0 > dev.em.0.watchdog_timeouts: 0 > dev.em.0.device_control: 1477444168 > dev.em.0.rx_control: 67141634 > dev.em.0.fc_high_water: 18432 > dev.em.0.fc_low_water: 16932 > dev.em.0.queue0.txd_head: 3265 > dev.em.0.queue0.txd_tail: 3265 > dev.em.0.queue0.tx_irq: 81153071 > dev.em.0.queue0.no_desc_avail: 0 > dev.em.0.queue0.rxd_head: 388 > dev.em.0.queue0.rxd_tail: 387 > dev.em.0.queue0.rx_irq: 79015024 > dev.em.0.mac_stats.excess_coll: 0 > dev.em.0.mac_stats.single_coll: 0 > dev.em.0.mac_stats.multiple_coll: 0 > dev.em.0.mac_stats.late_coll: 0 > dev.em.0.mac_stats.collision_count: 0 > dev.em.0.mac_stats.symbol_errors: 0 > dev.em.0.mac_stats.sequence_errors: 0 > dev.em.0.mac_stats.defer_count: 0 > dev.em.0.mac_stats.missed_packets: 0 > dev.em.0.mac_stats.recv_no_buff: 0 > dev.em.0.mac_stats.recv_undersize: 0 > dev.em.0.mac_stats.recv_fragmented: 0 > dev.em.0.mac_stats.recv_oversize: 0 > dev.em.0.mac_stats.recv_jabber: 0 > dev.em.0.mac_stats.recv_errs: 0 > dev.em.0.mac_stats.crc_errs: 0 > dev.em.0.mac_stats.alignment_errs: 0 > dev.em.0.mac_stats.coll_ext_errs: 0 > dev.em.0.mac_stats.xon_recvd: 6 > dev.em.0.mac_stats.xon_txd: 0 > dev.em.0.mac_stats.xoff_recvd: 6 > dev.em.0.mac_stats.xoff_txd: 0 > dev.em.0.mac_stats.total_pkts_recvd: 122072630 > dev.em.0.mac_stats.good_pkts_recvd: 122072618 > dev.em.0.mac_stats.bcast_pkts_recvd: 257 > dev.em.0.mac_stats.mcast_pkts_recvd: 0 > dev.em.0.mac_stats.rx_frames_64: 8634 > dev.em.0.mac_stats.rx_frames_65_127: 67656673 > dev.em.0.mac_stats.rx_frames_128_255: 714152 > dev.em.0.mac_stats.rx_frames_256_511: 609615 > dev.em.0.mac_stats.rx_frames_512_1023: 8646536 > dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 > dev.em.0.mac_stats.good_octets_recvd: 82940411216 > dev.em.0.mac_stats.good_octets_txd: 25718335997 > dev.em.0.mac_stats.total_pkts_txd: 99833592 > dev.em.0.mac_stats.good_pkts_txd: 99833592 > dev.em.0.mac_stats.bcast_pkts_txd: 13 > dev.em.0.mac_stats.mcast_pkts_txd: 0 > dev.em.0.mac_stats.tx_frames_64: 2193 > dev.em.0.mac_stats.tx_frames_65_127: 29089783 > dev.em.0.mac_stats.tx_frames_128_255: 54412030 > dev.em.0.mac_stats.tx_frames_256_511: 9565246 > dev.em.0.mac_stats.tx_frames_512_1023: 1080398 > dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 > dev.em.0.mac_stats.tso_txd: 1468623 > dev.em.0.mac_stats.tso_ctx_fail: 0 > dev.em.0.interrupts.asserts: 2 > dev.em.0.interrupts.rx_pkt_timer: 0 > dev.em.0.interrupts.rx_abs_timer: 0 > dev.em.0.interrupts.tx_pkt_timer: 0 > dev.em.0.interrupts.tx_abs_timer: 0 > dev.em.0.interrupts.tx_queue_empty: 0 > dev.em.0.interrupts.tx_queue_min_thresh: 0 > dev.em.0.interrupts.rx_desc_min_thresh: 0 > dev.em.0.interrupts.rx_overrun: 0 > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.1.%driver: em > dev.em.1.%location: slot=0 function=0 > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.1.%parent: pci2 > dev.em.1.nvm: -1 > dev.em.1.debug: -1 > dev.em.1.fc: 3 > dev.em.1.rx_int_delay: 0 > dev.em.1.tx_int_delay: 66 > dev.em.1.rx_abs_int_delay: 66 > dev.em.1.tx_abs_int_delay: 66 > dev.em.1.itr: 488 > dev.em.1.rx_processing_limit: -1 > dev.em.1.eee_control: 1 > dev.em.1.link_irq: 2 > dev.em.1.mbuf_alloc_fail: 0 > dev.em.1.cluster_alloc_fail: 0 > dev.em.1.dropped: 0 > dev.em.1.tx_dma_fail: 1 > dev.em.1.rx_overruns: 0 > dev.em.1.watchdog_timeouts: 0 > dev.em.1.device_control: 1477444168 > dev.em.1.rx_control: 67141634 > dev.em.1.fc_high_water: 18432 > dev.em.1.fc_low_water: 16932 > dev.em.1.queue0.txd_head: 2451 > dev.em.1.queue0.txd_tail: 2453 > dev.em.1.queue0.tx_irq: 143904807 > dev.em.1.queue0.no_desc_avail: 0 > dev.em.1.queue0.rxd_head: 342 > dev.em.1.queue0.rxd_tail: 341 > dev.em.1.queue0.rx_irq: 159303310 > dev.em.1.mac_stats.excess_coll: 0 > dev.em.1.mac_stats.single_coll: 0 > dev.em.1.mac_stats.multiple_coll: 0 > dev.em.1.mac_stats.late_coll: 0 > dev.em.1.mac_stats.collision_count: 0 > dev.em.1.mac_stats.symbol_errors: 0 > dev.em.1.mac_stats.sequence_errors: 0 > dev.em.1.mac_stats.defer_count: 0 > dev.em.1.mac_stats.missed_packets: 0 > dev.em.1.mac_stats.recv_no_buff: 0 > dev.em.1.mac_stats.recv_undersize: 0 > dev.em.1.mac_stats.recv_fragmented: 0 > dev.em.1.mac_stats.recv_oversize: 0 > dev.em.1.mac_stats.recv_jabber: 0 > dev.em.1.mac_stats.recv_errs: 0 > dev.em.1.mac_stats.crc_errs: 0 > dev.em.1.mac_stats.alignment_errs: 0 > dev.em.1.mac_stats.coll_ext_errs: 0 > dev.em.1.mac_stats.xon_recvd: 1 > dev.em.1.mac_stats.xon_txd: 0 > dev.em.1.mac_stats.xoff_recvd: 1 > dev.em.1.mac_stats.xoff_txd: 0 > dev.em.1.mac_stats.total_pkts_recvd: 331901758 > dev.em.1.mac_stats.good_pkts_recvd: 331901756 > dev.em.1.mac_stats.bcast_pkts_recvd: 13467 > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > dev.em.1.mac_stats.rx_frames_64: 13905035 > dev.em.1.mac_stats.rx_frames_65_127: 22315178 > dev.em.1.mac_stats.rx_frames_128_255: 8343368 > dev.em.1.mac_stats.rx_frames_256_511: 8602323 > dev.em.1.mac_stats.rx_frames_512_1023: 8170288 > dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 > dev.em.1.mac_stats.good_octets_recvd: 420794715670 > dev.em.1.mac_stats.good_octets_txd: 45361473880 > dev.em.1.mac_stats.total_pkts_txd: 217852588 > dev.em.1.mac_stats.good_pkts_txd: 217852588 > dev.em.1.mac_stats.bcast_pkts_txd: 6 > dev.em.1.mac_stats.mcast_pkts_txd: 0 > dev.em.1.mac_stats.tx_frames_64: 64102191 > dev.em.1.mac_stats.tx_frames_65_127: 120705475 > dev.em.1.mac_stats.tx_frames_128_255: 6009336 > dev.em.1.mac_stats.tx_frames_256_511: 4593595 > dev.em.1.mac_stats.tx_frames_512_1023: 4295623 > dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 > dev.em.1.mac_stats.tso_txd: 291134 > dev.em.1.mac_stats.tso_ctx_fail: 0 > dev.em.1.interrupts.asserts: 2 > dev.em.1.interrupts.rx_pkt_timer: 0 > dev.em.1.interrupts.rx_abs_timer: 0 > dev.em.1.interrupts.tx_pkt_timer: 0 > dev.em.1.interrupts.tx_abs_timer: 0 > dev.em.1.interrupts.tx_queue_empty: 0 > dev.em.1.interrupts.tx_queue_min_thresh: 0 > dev.em.1.interrupts.rx_desc_min_thresh: 0 > dev.em.1.interrupts.rx_overrun: 0 > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.2.%driver: em > dev.em.2.%location: slot=0 function=0 > dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.2.%parent: pci7 > dev.em.2.nvm: -1 > dev.em.2.debug: -1 > dev.em.2.fc: 3 > dev.em.2.rx_int_delay: 0 > dev.em.2.tx_int_delay: 66 > dev.em.2.rx_abs_int_delay: 66 > dev.em.2.tx_abs_int_delay: 66 > dev.em.2.itr: 488 > dev.em.2.rx_processing_limit: -1 > dev.em.2.eee_control: 1 > dev.em.2.link_irq: 1 > dev.em.2.mbuf_alloc_fail: 0 > dev.em.2.cluster_alloc_fail: 0 > dev.em.2.dropped: 0 > dev.em.2.tx_dma_fail: 6823 > dev.em.2.rx_overruns: 0 > dev.em.2.watchdog_timeouts: 0 > dev.em.2.device_control: 1477444168 > dev.em.2.rx_control: 67141634 > dev.em.2.fc_high_water: 18432 > dev.em.2.fc_low_water: 16932 > dev.em.2.queue0.txd_head: 3977 > dev.em.2.queue0.txd_tail: 3977 > dev.em.2.queue0.tx_irq: 220950699 > dev.em.2.queue0.no_desc_avail: 0 > dev.em.2.queue0.rxd_head: 83 > dev.em.2.queue0.rxd_tail: 82 > dev.em.2.queue0.rx_irq: 125920607 > dev.em.2.mac_stats.excess_coll: 0 > dev.em.2.mac_stats.single_coll: 0 > dev.em.2.mac_stats.multiple_coll: 0 > dev.em.2.mac_stats.late_coll: 0 > dev.em.2.mac_stats.collision_count: 0 > dev.em.2.mac_stats.symbol_errors: 0 > dev.em.2.mac_stats.sequence_errors: 0 > dev.em.2.mac_stats.defer_count: 0 > dev.em.2.mac_stats.missed_packets: 0 > dev.em.2.mac_stats.recv_no_buff: 0 > dev.em.2.mac_stats.recv_undersize: 0 > dev.em.2.mac_stats.recv_fragmented: 0 > dev.em.2.mac_stats.recv_oversize: 0 > dev.em.2.mac_stats.recv_jabber: 0 > dev.em.2.mac_stats.recv_errs: 0 > dev.em.2.mac_stats.crc_errs: 0 > dev.em.2.mac_stats.alignment_errs: 0 > dev.em.2.mac_stats.coll_ext_errs: 0 > dev.em.2.mac_stats.xon_recvd: 14123 > dev.em.2.mac_stats.xon_txd: 1 > dev.em.2.mac_stats.xoff_recvd: 14127 > dev.em.2.mac_stats.xoff_txd: 1 > dev.em.2.mac_stats.total_pkts_recvd: 229919303 > dev.em.2.mac_stats.good_pkts_recvd: 229891053 > dev.em.2.mac_stats.bcast_pkts_recvd: 909450 > dev.em.2.mac_stats.mcast_pkts_recvd: 19452 > dev.em.2.mac_stats.rx_frames_64: 1477808 > dev.em.2.mac_stats.rx_frames_65_127: 195114744 > dev.em.2.mac_stats.rx_frames_128_255: 6579690 > dev.em.2.mac_stats.rx_frames_256_511: 5137387 > dev.em.2.mac_stats.rx_frames_512_1023: 4223090 > dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 > dev.em.2.mac_stats.good_octets_recvd: 46129102134 > dev.em.2.mac_stats.good_octets_txd: 419293159496 > dev.em.2.mac_stats.total_pkts_txd: 332661584 > dev.em.2.mac_stats.good_pkts_txd: 332661582 > dev.em.2.mac_stats.bcast_pkts_txd: 48506 > dev.em.2.mac_stats.mcast_pkts_txd: 78 > dev.em.2.mac_stats.tx_frames_64: 14598198 > dev.em.2.mac_stats.tx_frames_65_127: 22287108 > dev.em.2.mac_stats.tx_frames_128_255: 8897511 > dev.em.2.mac_stats.tx_frames_256_511: 9623000 > dev.em.2.mac_stats.tx_frames_512_1023: 8325033 > dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 > dev.em.2.mac_stats.tso_txd: 24357891 > dev.em.2.mac_stats.tso_ctx_fail: 0 > dev.em.2.interrupts.asserts: 2 > dev.em.2.interrupts.rx_pkt_timer: 0 > dev.em.2.interrupts.rx_abs_timer: 0 > dev.em.2.interrupts.tx_pkt_timer: 0 > dev.em.2.interrupts.tx_abs_timer: 0 > dev.em.2.interrupts.tx_queue_empty: 0 > dev.em.2.interrupts.tx_queue_min_thresh: 0 > dev.em.2.interrupts.rx_desc_min_thresh: 0 > dev.em.2.interrupts.rx_overrun: 0 > dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.3.%driver: em > dev.em.3.%location: slot=0 function=0 > dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.3.%parent: pci8 > dev.em.3.nvm: -1 > dev.em.3.debug: -1 > dev.em.3.fc: 3 > dev.em.3.rx_int_delay: 0 > dev.em.3.tx_int_delay: 66 > dev.em.3.rx_abs_int_delay: 66 > dev.em.3.tx_abs_int_delay: 66 > dev.em.3.itr: 488 > dev.em.3.rx_processing_limit: -1 > dev.em.3.eee_control: 1 > dev.em.3.link_irq: 0 > dev.em.3.mbuf_alloc_fail: 0 > dev.em.3.cluster_alloc_fail: 0 > dev.em.3.dropped: 0 > dev.em.3.tx_dma_fail: 0 > dev.em.3.rx_overruns: 0 > dev.em.3.watchdog_timeouts: 0 > dev.em.3.device_control: 1074790984 > dev.em.3.rx_control: 67141634 > dev.em.3.fc_high_water: 18432 > dev.em.3.fc_low_water: 16932 > dev.em.3.queue0.txd_head: 0 > dev.em.3.queue0.txd_tail: 0 > dev.em.3.queue0.tx_irq: 0 > dev.em.3.queue0.no_desc_avail: 0 > dev.em.3.queue0.rxd_head: 0 > dev.em.3.queue0.rxd_tail: 4095 > dev.em.3.queue0.rx_irq: 0 > dev.em.3.mac_stats.excess_coll: 0 > dev.em.3.mac_stats.single_coll: 0 > dev.em.3.mac_stats.multiple_coll: 0 > dev.em.3.mac_stats.late_coll: 0 > dev.em.3.mac_stats.collision_count: 0 > dev.em.3.mac_stats.symbol_errors: 0 > dev.em.3.mac_stats.sequence_errors: 0 > dev.em.3.mac_stats.defer_count: 0 > dev.em.3.mac_stats.missed_packets: 0 > dev.em.3.mac_stats.recv_no_buff: 0 > dev.em.3.mac_stats.recv_undersize: 0 > dev.em.3.mac_stats.recv_fragmented: 0 > dev.em.3.mac_stats.recv_oversize: 0 > dev.em.3.mac_stats.recv_jabber: 0 > dev.em.3.mac_stats.recv_errs: 0 > dev.em.3.mac_stats.crc_errs: 0 > dev.em.3.mac_stats.alignment_errs: 0 > dev.em.3.mac_stats.coll_ext_errs: 0 > dev.em.3.mac_stats.xon_recvd: 0 > dev.em.3.mac_stats.xon_txd: 0 > dev.em.3.mac_stats.xoff_recvd: 0 > dev.em.3.mac_stats.xoff_txd: 0 > dev.em.3.mac_stats.total_pkts_recvd: 0 > dev.em.3.mac_stats.good_pkts_recvd: 0 > dev.em.3.mac_stats.bcast_pkts_recvd: 0 > dev.em.3.mac_stats.mcast_pkts_recvd: 0 > dev.em.3.mac_stats.rx_frames_64: 0 > dev.em.3.mac_stats.rx_frames_65_127: 0 > dev.em.3.mac_stats.rx_frames_128_255: 0 > dev.em.3.mac_stats.rx_frames_256_511: 0 > dev.em.3.mac_stats.rx_frames_512_1023: 0 > dev.em.3.mac_stats.rx_frames_1024_1522: 0 > dev.em.3.mac_stats.good_octets_recvd: 0 > dev.em.3.mac_stats.good_octets_txd: 0 > dev.em.3.mac_stats.total_pkts_txd: 0 > dev.em.3.mac_stats.good_pkts_txd: 0 > dev.em.3.mac_stats.bcast_pkts_txd: 0 > dev.em.3.mac_stats.mcast_pkts_txd: 0 > dev.em.3.mac_stats.tx_frames_64: 0 > dev.em.3.mac_stats.tx_frames_65_127: 0 > dev.em.3.mac_stats.tx_frames_128_255: 0 > dev.em.3.mac_stats.tx_frames_256_511: 0 > dev.em.3.mac_stats.tx_frames_512_1023: 0 > dev.em.3.mac_stats.tx_frames_1024_1522: 0 > dev.em.3.mac_stats.tso_txd: 0 > dev.em.3.mac_stats.tso_ctx_fail: 0 > dev.em.3.interrupts.asserts: 0 > dev.em.3.interrupts.rx_pkt_timer: 0 > dev.em.3.interrupts.rx_abs_timer: 0 > dev.em.3.interrupts.tx_pkt_timer: 0 > dev.em.3.interrupts.tx_abs_timer: 0 > dev.em.3.interrupts.tx_queue_empty: 0 > dev.em.3.interrupts.tx_queue_min_thresh: 0 > dev.em.3.interrupts.rx_desc_min_thresh: 0 > dev.em.3.interrupts.rx_overrun: 0 > > .............................................................................. From owner-freebsd-net@FreeBSD.ORG Tue May 13 00:17:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 443707B1 for ; Tue, 13 May 2014 00:17:00 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4ABD25BE for ; Tue, 13 May 2014 00:16:59 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id hq16so6880952vcb.23 for ; Mon, 12 May 2014 17:16:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=07Uz8dhDZk35zGiW4obda/DdviuTSl1cj0CXTLSASy8=; b=iOPDRsiqTFLArptHDuIHB7jVEFFQEGraNrroiKpPuFa88sPm8/GoF96DWOlzpRoq8r FSJqDBFjyt1ciJLzl1BDJhL4pLg7agfo8/043F3hRwcBLhRYfedz9YiVweEcwZyntx1s VJP32D+12EyHMCl2DRfJtjr57OmP/urCdi0MWuYBS0DCvPK/PfYdLTiSPbjroCiS3IsN +gVPFFwfbnFC1iPvlZZRpwa+7WljF/t4VCu6BmXAsRJ2AjXrnD05eQPhePwkHHlGbizo 56bY0uUMxeQZZ3gMZN0/fMZm4WOPglPSAEPYyQpIbsmkDljJhmAryE62DZlitjWhGnVd BCAA== MIME-Version: 1.0 X-Received: by 10.52.12.36 with SMTP id v4mr22095280vdb.20.1399940219027; Mon, 12 May 2014 17:16:59 -0700 (PDT) Received: by 10.220.9.5 with HTTP; Mon, 12 May 2014 17:16:58 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 May 2014 17:16:58 -0700 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Jack Vogel To: Nick Rogers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 00:17:00 -0000 Nick, I'm very busy with some critical internal deadlines, I will look at this when I can come up for air, but please be patient. Jack On Mon, May 12, 2014 at 4:56 PM, Nick Rogers wrote: > On Fri, May 2, 2014 at 1:49 PM, Nick Rogers wrote: > > Hello, > > > > I am hoping someone can help me debug a kernel panic I have been > experiencing > > on one of my production systems. The system is a PF+ALTQ > firewall/gateway with > > about 1k simultaneous devices behind it at any given time, pushing no > more > > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug > it > > with kgdb, but am still at a loss. > > > > I have completely replaced the hardware to rule out issues with > > disk/memory/etc, and it appears to be a kernel issue, likely with e1000 > driver > > and/or PF. > > > > The panic seems to happen during times of heavier use, but the frequency > is > > not very predictable (anywhere from a few times a day to a once a week), > so I > > feel like its some kind of strange traffic pattern that I am unable to > > pinpoint. > > > > I am using a slightly custom kernel based on GENERIC, mainly to bring in > ALTQ > > and some other options. > > > > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 > > driver, although I do not believe that should affect em(4). > > > > Hoping someone can be of assistance in debugging this problem. I am > willing to > > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if > that > > is advisable. Unfortunately I cannot try 10-STABLE. > > > > Thanks, > > > > -Nick Rogers > > > > > > uname -v > > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 > > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS > > > > > > > > Here is the kernel conf... > > > .............................................................................. > > include GENERIC > > > > ident RGNETS > > > > makeoptions MODULES_OVERRIDE="" > > > > options DEVICE_POLLING > > > > device crypto > > device cryptodev > > > > options VLAN_ARRAY > > > > device amdtemp > > > > # PF > > device pf > > device pflog > > options ALTQ > > options ALTQ_CBQ # Class Bases Queuing (CBQ) > > options ALTQ_RED # Random Early Detection (RED) > > options ALTQ_RIO # RED In/Out > > options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) > > options ALTQ_PRIQ # Priority Queuing (PRIQ) > > options ALTQ_NOPCC # Required for SMP build > > > > # PPPoE > > options NETGRAPH > > options NETGRAPH_ETHER > > options NETGRAPH_PPPOE > > options NETGRAPH_SOCKET > > > > # IPsec > > device enc > > options IPSEC > > options IPSEC_FILTERTUNNEL > > options IPSEC_NAT_T > > > .............................................................................. > > > > > > The crash dump.... > > > .............................................................................. > > > > GNU gdb 6.1.1 [FreeBSD] > > Copyright 2004 Free Software Foundation, Inc. > > GDB is free software, covered by the GNU General Public License, and you > are > > welcome to change it and/or distribute copies of it under certain > conditions. > > Type "show copying" to see the conditions. > > There is absolutely no warranty for GDB. Type "show warranty" for > details. > > This GDB was configured as "amd64-marcel-freebsd"... > > > > Unread portion of the kernel message buffer: > > > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 5; apic id = 05 > > fault virtual address = 0x10 > > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff8033d350 > > stack pointer = 0x28:0xffffff83545384b0 > > frame pointer = 0x28:0xffffff83545384c0 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = interrupt enabled, resume, IOPL = 0 > > current process = 12 (irq262: em2:rx 0) > > trap number = 12 > > panic: page fault > > cpuid = 5 > > KDB: stack backtrace: > > #0 0xffffffff80956836 at kdb_backtrace+0x66 > > #1 0xffffffff8091c40e at panic+0x1ce > > #2 0xffffffff80d31e70 at trap_fatal+0x290 > > #3 0xffffffff80d321d1 at trap_pfault+0x211 > > #4 0xffffffff80d327d3 at trap+0x363 > > #5 0xffffffff80d1b9d3 at calltrap+0x8 > > #6 0xffffffff8034872d at pf_test_rule+0x17ed > > #7 0xffffffff8034ba12 at pf_test+0x1032 > > #8 0xffffffff8035112b at pf_check_in+0x2b > > #9 0xffffffff809e952e at pfil_run_hooks+0x9e > > #10 0xffffffff80a5286a at ip_input+0x2ea > > #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 > > #12 0xffffffff809df93d at ether_demux+0x14d > > #13 0xffffffff809dfc1e at ether_nh_input+0x1fe > > #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 > > #15 0xffffffff809df85f at ether_demux+0x6f > > #16 0xffffffff809dfc1e at ether_nh_input+0x1fe > > #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 > > Uptime: 17d7h20m59s > > Dumping 2932 out of 12256 MB: (CTRL-C to abort) > > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > > > Reading symbols from /boot/kernel/aio.ko...Reading symbols from > > /boot/kernel/aio.ko.symbols...done. > > done. > > Loaded symbols for /boot/kernel/aio.ko > > Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from > > /boot/kernel/coretemp.ko.symbols...done. > > done. > > Loaded symbols for /boot/kernel/coretemp.ko > > Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from > > /boot/kernel/cc_htcp.ko.symbols...done. > > done. > > Loaded symbols for /boot/kernel/cc_htcp.ko > > #0 doadump (textdump=Variable "textdump" is not available. > > ) at pcpu.h:234 > > 234 pcpu.h: No such file or directory. > > in pcpu.h > > (kgdb) list *0xffffffff8033d350 > > 0xffffffff8033d350 is in pf_addrcpy > (/usr/src/sys/contrib/pf/net/pf.c:512). > > 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) > > 508 { > > 509 switch (af) { > > 510 #ifdef INET > > 511 case AF_INET: > > 512 dst->addr32[0] = src->addr32[0]; > > 513 break; > > 514 #endif /* INET */ > > 515 case AF_INET6: > > 516 dst->addr32[0] = src->addr32[0]; > > (kgdb) backtrace > > #0 doadump (textdump=Variable "textdump" is not available. > > ) at pcpu.h:234 > > #1 0xffffffff8091bee6 in kern_reboot (howto=260) at > > /usr/src/sys/kern/kern_shutdown.c:454 > > #2 0xffffffff8091c3e7 in panic (fmt=0x1
) > > at /usr/src/sys/kern/kern_shutdown.c:642 > > #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is > > not available. > > ) at /usr/src/sys/amd64/amd64/trap.c:878 > > #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, > > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 > > #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at > > /usr/src/sys/amd64/amd64/trap.c:456 > > #6 0xffffffff80d1b9d3 in calltrap () at > > /usr/src/sys/amd64/amd64/exception.S:232 > > #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, > > src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 > > #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, > > sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, > > m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, > > pd=0xffffff83545386c0, am=0xffffff8354538790, > > rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at > > /usr/src/sys/contrib/pf/net/pf.c:3900 > > #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not > available. > > ) at /usr/src/sys/contrib/pf/net/pf.c:6776 > > #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not > available. > > ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 > > #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not > available. > > ) at /usr/src/sys/net/pfil.c:82 > > #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at > > /usr/src/sys/netinet/ip_input.c:510 > > #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, > > source=Variable "source" is not available. > > ) at /usr/src/sys/net/netisr.c:1013 > > #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, > > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 > > #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not > available. > > ) at /usr/src/sys/net/if_ethersubr.c:762 > > #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > > source=Variable "source" is not available. > > ) at /usr/src/sys/net/netisr.c:1013 > > #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, > > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 > > #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not > available. > > ) at /usr/src/sys/net/if_ethersubr.c:762 > > #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > > source=Variable "source" is not available. > > ) at /usr/src/sys/net/netisr.c:1013 > > #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, > > done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 > > #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not > available. > > ) at /usr/src/sys/dev/e1000/if_em.c:1593 > > It seems like the panic is in the MSIX receive handler? I guess I will > try disabling MSIX for em. > > Jack, can you comment on this please? > > > > #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" > > is not available. > > ) at /usr/src/sys/kern/kern_intr.c:1272 > > #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at > > /usr/src/sys/kern/kern_intr.c:1285 > > #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 > > , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at > > /usr/src/sys/kern/kern_fork.c:996 > > #25 0xffffffff80d1befe in fork_trampoline () at > > /usr/src/sys/amd64/amd64/exception.S:606 > > #26 0x0000000000000000 in ?? () > > #27 0x0000000000000000 in ?? () > > #28 0x0000000000000001 in ?? () > > #29 0x0000000000000000 in ?? () > > #30 0x0000000000000000 in ?? () > > #31 0x0000000000000000 in ?? () > > #32 0x0000000000000000 in ?? () > > #33 0x0000000000000000 in ?? () > > #34 0x0000000000000000 in ?? () > > #35 0x0000000000000000 in ?? () > > #36 0x0000000000000000 in ?? () > > #37 0x0000000000000000 in ?? () > > #38 0x0000000000000000 in ?? () > > #39 0x0000000000000000 in ?? () > > #40 0x0000000000000000 in ?? () > > #41 0x0000000000000000 in ?? () > > #42 0x0000000000000000 in ?? () > > #43 0x0000000000000000 in ?? () > > #44 0x0000000000000000 in ?? () > > #45 0x0000000000000000 in ?? () > > #46 0x0000000000000000 in ?? () > > #47 0x0000000000000000 in ?? () > > #48 0x0000000000000000 in ?? () > > ---Type to continue, or q to quit--- > > #49 0x0000000000000000 in ?? () > > #50 0x0000000000000000 in ?? () > > #51 0xfffffe0007304920 in ?? () > > #52 0xfffffe0003afd000 in ?? () > > #53 0xfffffe0007304920 in ?? () > > #54 0xffffff8354538b40 in ?? () > > #55 0xffffff8354538ae8 in ?? () > > #56 0xfffffe0003b00920 in ?? () > > #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, > > newtd=0xfffffe000730c300, flags=Variable "flags" is not available. > > ) at /usr/src/sys/kern/sched_ule.c:1898 > > Previous frame inner to this frame (corrupt stack?) > > > > > .............................................................................. > > > > > > > > Relevant pciconf output... > > > .............................................................................. > > > > em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > class = network > > subclass = ethernet > > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > > cap 05[d0] = MSI supports 1 message, 64 bit > > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > > em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > class = network > > subclass = ethernet > > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > > cap 05[d0] = MSI supports 1 message, 64 bit > > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > > em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > class = network > > subclass = ethernet > > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > > cap 05[d0] = MSI supports 1 message, 64 bit > > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > > em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > class = network > > subclass = ethernet > > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > > cap 05[d0] = MSI supports 1 message, 64 bit > > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > > > > .............................................................................. > > > > > > dev.em sysctl.... > > > .............................................................................. > > > > dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > > dev.em.0.%driver: em > > dev.em.0.%location: slot=0 function=0 > > dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > > subdevice=0x10d3 class=0x020000 > > dev.em.0.%parent: pci1 > > dev.em.0.nvm: -1 > > dev.em.0.debug: -1 > > dev.em.0.fc: 3 > > dev.em.0.rx_int_delay: 0 > > dev.em.0.tx_int_delay: 66 > > dev.em.0.rx_abs_int_delay: 66 > > dev.em.0.tx_abs_int_delay: 66 > > dev.em.0.itr: 488 > > dev.em.0.rx_processing_limit: -1 > > dev.em.0.eee_control: 1 > > dev.em.0.link_irq: 2 > > dev.em.0.mbuf_alloc_fail: 0 > > dev.em.0.cluster_alloc_fail: 0 > > dev.em.0.dropped: 0 > > dev.em.0.tx_dma_fail: 0 > > dev.em.0.rx_overruns: 0 > > dev.em.0.watchdog_timeouts: 0 > > dev.em.0.device_control: 1477444168 > > dev.em.0.rx_control: 67141634 > > dev.em.0.fc_high_water: 18432 > > dev.em.0.fc_low_water: 16932 > > dev.em.0.queue0.txd_head: 3265 > > dev.em.0.queue0.txd_tail: 3265 > > dev.em.0.queue0.tx_irq: 81153071 > > dev.em.0.queue0.no_desc_avail: 0 > > dev.em.0.queue0.rxd_head: 388 > > dev.em.0.queue0.rxd_tail: 387 > > dev.em.0.queue0.rx_irq: 79015024 > > dev.em.0.mac_stats.excess_coll: 0 > > dev.em.0.mac_stats.single_coll: 0 > > dev.em.0.mac_stats.multiple_coll: 0 > > dev.em.0.mac_stats.late_coll: 0 > > dev.em.0.mac_stats.collision_count: 0 > > dev.em.0.mac_stats.symbol_errors: 0 > > dev.em.0.mac_stats.sequence_errors: 0 > > dev.em.0.mac_stats.defer_count: 0 > > dev.em.0.mac_stats.missed_packets: 0 > > dev.em.0.mac_stats.recv_no_buff: 0 > > dev.em.0.mac_stats.recv_undersize: 0 > > dev.em.0.mac_stats.recv_fragmented: 0 > > dev.em.0.mac_stats.recv_oversize: 0 > > dev.em.0.mac_stats.recv_jabber: 0 > > dev.em.0.mac_stats.recv_errs: 0 > > dev.em.0.mac_stats.crc_errs: 0 > > dev.em.0.mac_stats.alignment_errs: 0 > > dev.em.0.mac_stats.coll_ext_errs: 0 > > dev.em.0.mac_stats.xon_recvd: 6 > > dev.em.0.mac_stats.xon_txd: 0 > > dev.em.0.mac_stats.xoff_recvd: 6 > > dev.em.0.mac_stats.xoff_txd: 0 > > dev.em.0.mac_stats.total_pkts_recvd: 122072630 > > dev.em.0.mac_stats.good_pkts_recvd: 122072618 > > dev.em.0.mac_stats.bcast_pkts_recvd: 257 > > dev.em.0.mac_stats.mcast_pkts_recvd: 0 > > dev.em.0.mac_stats.rx_frames_64: 8634 > > dev.em.0.mac_stats.rx_frames_65_127: 67656673 > > dev.em.0.mac_stats.rx_frames_128_255: 714152 > > dev.em.0.mac_stats.rx_frames_256_511: 609615 > > dev.em.0.mac_stats.rx_frames_512_1023: 8646536 > > dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 > > dev.em.0.mac_stats.good_octets_recvd: 82940411216 > > dev.em.0.mac_stats.good_octets_txd: 25718335997 > > dev.em.0.mac_stats.total_pkts_txd: 99833592 > > dev.em.0.mac_stats.good_pkts_txd: 99833592 > > dev.em.0.mac_stats.bcast_pkts_txd: 13 > > dev.em.0.mac_stats.mcast_pkts_txd: 0 > > dev.em.0.mac_stats.tx_frames_64: 2193 > > dev.em.0.mac_stats.tx_frames_65_127: 29089783 > > dev.em.0.mac_stats.tx_frames_128_255: 54412030 > > dev.em.0.mac_stats.tx_frames_256_511: 9565246 > > dev.em.0.mac_stats.tx_frames_512_1023: 1080398 > > dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 > > dev.em.0.mac_stats.tso_txd: 1468623 > > dev.em.0.mac_stats.tso_ctx_fail: 0 > > dev.em.0.interrupts.asserts: 2 > > dev.em.0.interrupts.rx_pkt_timer: 0 > > dev.em.0.interrupts.rx_abs_timer: 0 > > dev.em.0.interrupts.tx_pkt_timer: 0 > > dev.em.0.interrupts.tx_abs_timer: 0 > > dev.em.0.interrupts.tx_queue_empty: 0 > > dev.em.0.interrupts.tx_queue_min_thresh: 0 > > dev.em.0.interrupts.rx_desc_min_thresh: 0 > > dev.em.0.interrupts.rx_overrun: 0 > > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > > dev.em.1.%driver: em > > dev.em.1.%location: slot=0 function=0 > > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > > subdevice=0x10d3 class=0x020000 > > dev.em.1.%parent: pci2 > > dev.em.1.nvm: -1 > > dev.em.1.debug: -1 > > dev.em.1.fc: 3 > > dev.em.1.rx_int_delay: 0 > > dev.em.1.tx_int_delay: 66 > > dev.em.1.rx_abs_int_delay: 66 > > dev.em.1.tx_abs_int_delay: 66 > > dev.em.1.itr: 488 > > dev.em.1.rx_processing_limit: -1 > > dev.em.1.eee_control: 1 > > dev.em.1.link_irq: 2 > > dev.em.1.mbuf_alloc_fail: 0 > > dev.em.1.cluster_alloc_fail: 0 > > dev.em.1.dropped: 0 > > dev.em.1.tx_dma_fail: 1 > > dev.em.1.rx_overruns: 0 > > dev.em.1.watchdog_timeouts: 0 > > dev.em.1.device_control: 1477444168 > > dev.em.1.rx_control: 67141634 > > dev.em.1.fc_high_water: 18432 > > dev.em.1.fc_low_water: 16932 > > dev.em.1.queue0.txd_head: 2451 > > dev.em.1.queue0.txd_tail: 2453 > > dev.em.1.queue0.tx_irq: 143904807 > > dev.em.1.queue0.no_desc_avail: 0 > > dev.em.1.queue0.rxd_head: 342 > > dev.em.1.queue0.rxd_tail: 341 > > dev.em.1.queue0.rx_irq: 159303310 > > dev.em.1.mac_stats.excess_coll: 0 > > dev.em.1.mac_stats.single_coll: 0 > > dev.em.1.mac_stats.multiple_coll: 0 > > dev.em.1.mac_stats.late_coll: 0 > > dev.em.1.mac_stats.collision_count: 0 > > dev.em.1.mac_stats.symbol_errors: 0 > > dev.em.1.mac_stats.sequence_errors: 0 > > dev.em.1.mac_stats.defer_count: 0 > > dev.em.1.mac_stats.missed_packets: 0 > > dev.em.1.mac_stats.recv_no_buff: 0 > > dev.em.1.mac_stats.recv_undersize: 0 > > dev.em.1.mac_stats.recv_fragmented: 0 > > dev.em.1.mac_stats.recv_oversize: 0 > > dev.em.1.mac_stats.recv_jabber: 0 > > dev.em.1.mac_stats.recv_errs: 0 > > dev.em.1.mac_stats.crc_errs: 0 > > dev.em.1.mac_stats.alignment_errs: 0 > > dev.em.1.mac_stats.coll_ext_errs: 0 > > dev.em.1.mac_stats.xon_recvd: 1 > > dev.em.1.mac_stats.xon_txd: 0 > > dev.em.1.mac_stats.xoff_recvd: 1 > > dev.em.1.mac_stats.xoff_txd: 0 > > dev.em.1.mac_stats.total_pkts_recvd: 331901758 > > dev.em.1.mac_stats.good_pkts_recvd: 331901756 > > dev.em.1.mac_stats.bcast_pkts_recvd: 13467 > > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > > dev.em.1.mac_stats.rx_frames_64: 13905035 > > dev.em.1.mac_stats.rx_frames_65_127: 22315178 > > dev.em.1.mac_stats.rx_frames_128_255: 8343368 > > dev.em.1.mac_stats.rx_frames_256_511: 8602323 > > dev.em.1.mac_stats.rx_frames_512_1023: 8170288 > > dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 > > dev.em.1.mac_stats.good_octets_recvd: 420794715670 > > dev.em.1.mac_stats.good_octets_txd: 45361473880 > > dev.em.1.mac_stats.total_pkts_txd: 217852588 > > dev.em.1.mac_stats.good_pkts_txd: 217852588 > > dev.em.1.mac_stats.bcast_pkts_txd: 6 > > dev.em.1.mac_stats.mcast_pkts_txd: 0 > > dev.em.1.mac_stats.tx_frames_64: 64102191 > > dev.em.1.mac_stats.tx_frames_65_127: 120705475 > > dev.em.1.mac_stats.tx_frames_128_255: 6009336 > > dev.em.1.mac_stats.tx_frames_256_511: 4593595 > > dev.em.1.mac_stats.tx_frames_512_1023: 4295623 > > dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 > > dev.em.1.mac_stats.tso_txd: 291134 > > dev.em.1.mac_stats.tso_ctx_fail: 0 > > dev.em.1.interrupts.asserts: 2 > > dev.em.1.interrupts.rx_pkt_timer: 0 > > dev.em.1.interrupts.rx_abs_timer: 0 > > dev.em.1.interrupts.tx_pkt_timer: 0 > > dev.em.1.interrupts.tx_abs_timer: 0 > > dev.em.1.interrupts.tx_queue_empty: 0 > > dev.em.1.interrupts.tx_queue_min_thresh: 0 > > dev.em.1.interrupts.rx_desc_min_thresh: 0 > > dev.em.1.interrupts.rx_overrun: 0 > > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > > dev.em.2.%driver: em > > dev.em.2.%location: slot=0 function=0 > > dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > > subdevice=0x10d3 class=0x020000 > > dev.em.2.%parent: pci7 > > dev.em.2.nvm: -1 > > dev.em.2.debug: -1 > > dev.em.2.fc: 3 > > dev.em.2.rx_int_delay: 0 > > dev.em.2.tx_int_delay: 66 > > dev.em.2.rx_abs_int_delay: 66 > > dev.em.2.tx_abs_int_delay: 66 > > dev.em.2.itr: 488 > > dev.em.2.rx_processing_limit: -1 > > dev.em.2.eee_control: 1 > > dev.em.2.link_irq: 1 > > dev.em.2.mbuf_alloc_fail: 0 > > dev.em.2.cluster_alloc_fail: 0 > > dev.em.2.dropped: 0 > > dev.em.2.tx_dma_fail: 6823 > > dev.em.2.rx_overruns: 0 > > dev.em.2.watchdog_timeouts: 0 > > dev.em.2.device_control: 1477444168 > > dev.em.2.rx_control: 67141634 > > dev.em.2.fc_high_water: 18432 > > dev.em.2.fc_low_water: 16932 > > dev.em.2.queue0.txd_head: 3977 > > dev.em.2.queue0.txd_tail: 3977 > > dev.em.2.queue0.tx_irq: 220950699 > > dev.em.2.queue0.no_desc_avail: 0 > > dev.em.2.queue0.rxd_head: 83 > > dev.em.2.queue0.rxd_tail: 82 > > dev.em.2.queue0.rx_irq: 125920607 > > dev.em.2.mac_stats.excess_coll: 0 > > dev.em.2.mac_stats.single_coll: 0 > > dev.em.2.mac_stats.multiple_coll: 0 > > dev.em.2.mac_stats.late_coll: 0 > > dev.em.2.mac_stats.collision_count: 0 > > dev.em.2.mac_stats.symbol_errors: 0 > > dev.em.2.mac_stats.sequence_errors: 0 > > dev.em.2.mac_stats.defer_count: 0 > > dev.em.2.mac_stats.missed_packets: 0 > > dev.em.2.mac_stats.recv_no_buff: 0 > > dev.em.2.mac_stats.recv_undersize: 0 > > dev.em.2.mac_stats.recv_fragmented: 0 > > dev.em.2.mac_stats.recv_oversize: 0 > > dev.em.2.mac_stats.recv_jabber: 0 > > dev.em.2.mac_stats.recv_errs: 0 > > dev.em.2.mac_stats.crc_errs: 0 > > dev.em.2.mac_stats.alignment_errs: 0 > > dev.em.2.mac_stats.coll_ext_errs: 0 > > dev.em.2.mac_stats.xon_recvd: 14123 > > dev.em.2.mac_stats.xon_txd: 1 > > dev.em.2.mac_stats.xoff_recvd: 14127 > > dev.em.2.mac_stats.xoff_txd: 1 > > dev.em.2.mac_stats.total_pkts_recvd: 229919303 > > dev.em.2.mac_stats.good_pkts_recvd: 229891053 > > dev.em.2.mac_stats.bcast_pkts_recvd: 909450 > > dev.em.2.mac_stats.mcast_pkts_recvd: 19452 > > dev.em.2.mac_stats.rx_frames_64: 1477808 > > dev.em.2.mac_stats.rx_frames_65_127: 195114744 > > dev.em.2.mac_stats.rx_frames_128_255: 6579690 > > dev.em.2.mac_stats.rx_frames_256_511: 5137387 > > dev.em.2.mac_stats.rx_frames_512_1023: 4223090 > > dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 > > dev.em.2.mac_stats.good_octets_recvd: 46129102134 > > dev.em.2.mac_stats.good_octets_txd: 419293159496 > > dev.em.2.mac_stats.total_pkts_txd: 332661584 > > dev.em.2.mac_stats.good_pkts_txd: 332661582 > > dev.em.2.mac_stats.bcast_pkts_txd: 48506 > > dev.em.2.mac_stats.mcast_pkts_txd: 78 > > dev.em.2.mac_stats.tx_frames_64: 14598198 > > dev.em.2.mac_stats.tx_frames_65_127: 22287108 > > dev.em.2.mac_stats.tx_frames_128_255: 8897511 > > dev.em.2.mac_stats.tx_frames_256_511: 9623000 > > dev.em.2.mac_stats.tx_frames_512_1023: 8325033 > > dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 > > dev.em.2.mac_stats.tso_txd: 24357891 > > dev.em.2.mac_stats.tso_ctx_fail: 0 > > dev.em.2.interrupts.asserts: 2 > > dev.em.2.interrupts.rx_pkt_timer: 0 > > dev.em.2.interrupts.rx_abs_timer: 0 > > dev.em.2.interrupts.tx_pkt_timer: 0 > > dev.em.2.interrupts.tx_abs_timer: 0 > > dev.em.2.interrupts.tx_queue_empty: 0 > > dev.em.2.interrupts.tx_queue_min_thresh: 0 > > dev.em.2.interrupts.rx_desc_min_thresh: 0 > > dev.em.2.interrupts.rx_overrun: 0 > > dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > > dev.em.3.%driver: em > > dev.em.3.%location: slot=0 function=0 > > dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > > subdevice=0x10d3 class=0x020000 > > dev.em.3.%parent: pci8 > > dev.em.3.nvm: -1 > > dev.em.3.debug: -1 > > dev.em.3.fc: 3 > > dev.em.3.rx_int_delay: 0 > > dev.em.3.tx_int_delay: 66 > > dev.em.3.rx_abs_int_delay: 66 > > dev.em.3.tx_abs_int_delay: 66 > > dev.em.3.itr: 488 > > dev.em.3.rx_processing_limit: -1 > > dev.em.3.eee_control: 1 > > dev.em.3.link_irq: 0 > > dev.em.3.mbuf_alloc_fail: 0 > > dev.em.3.cluster_alloc_fail: 0 > > dev.em.3.dropped: 0 > > dev.em.3.tx_dma_fail: 0 > > dev.em.3.rx_overruns: 0 > > dev.em.3.watchdog_timeouts: 0 > > dev.em.3.device_control: 1074790984 > > dev.em.3.rx_control: 67141634 > > dev.em.3.fc_high_water: 18432 > > dev.em.3.fc_low_water: 16932 > > dev.em.3.queue0.txd_head: 0 > > dev.em.3.queue0.txd_tail: 0 > > dev.em.3.queue0.tx_irq: 0 > > dev.em.3.queue0.no_desc_avail: 0 > > dev.em.3.queue0.rxd_head: 0 > > dev.em.3.queue0.rxd_tail: 4095 > > dev.em.3.queue0.rx_irq: 0 > > dev.em.3.mac_stats.excess_coll: 0 > > dev.em.3.mac_stats.single_coll: 0 > > dev.em.3.mac_stats.multiple_coll: 0 > > dev.em.3.mac_stats.late_coll: 0 > > dev.em.3.mac_stats.collision_count: 0 > > dev.em.3.mac_stats.symbol_errors: 0 > > dev.em.3.mac_stats.sequence_errors: 0 > > dev.em.3.mac_stats.defer_count: 0 > > dev.em.3.mac_stats.missed_packets: 0 > > dev.em.3.mac_stats.recv_no_buff: 0 > > dev.em.3.mac_stats.recv_undersize: 0 > > dev.em.3.mac_stats.recv_fragmented: 0 > > dev.em.3.mac_stats.recv_oversize: 0 > > dev.em.3.mac_stats.recv_jabber: 0 > > dev.em.3.mac_stats.recv_errs: 0 > > dev.em.3.mac_stats.crc_errs: 0 > > dev.em.3.mac_stats.alignment_errs: 0 > > dev.em.3.mac_stats.coll_ext_errs: 0 > > dev.em.3.mac_stats.xon_recvd: 0 > > dev.em.3.mac_stats.xon_txd: 0 > > dev.em.3.mac_stats.xoff_recvd: 0 > > dev.em.3.mac_stats.xoff_txd: 0 > > dev.em.3.mac_stats.total_pkts_recvd: 0 > > dev.em.3.mac_stats.good_pkts_recvd: 0 > > dev.em.3.mac_stats.bcast_pkts_recvd: 0 > > dev.em.3.mac_stats.mcast_pkts_recvd: 0 > > dev.em.3.mac_stats.rx_frames_64: 0 > > dev.em.3.mac_stats.rx_frames_65_127: 0 > > dev.em.3.mac_stats.rx_frames_128_255: 0 > > dev.em.3.mac_stats.rx_frames_256_511: 0 > > dev.em.3.mac_stats.rx_frames_512_1023: 0 > > dev.em.3.mac_stats.rx_frames_1024_1522: 0 > > dev.em.3.mac_stats.good_octets_recvd: 0 > > dev.em.3.mac_stats.good_octets_txd: 0 > > dev.em.3.mac_stats.total_pkts_txd: 0 > > dev.em.3.mac_stats.good_pkts_txd: 0 > > dev.em.3.mac_stats.bcast_pkts_txd: 0 > > dev.em.3.mac_stats.mcast_pkts_txd: 0 > > dev.em.3.mac_stats.tx_frames_64: 0 > > dev.em.3.mac_stats.tx_frames_65_127: 0 > > dev.em.3.mac_stats.tx_frames_128_255: 0 > > dev.em.3.mac_stats.tx_frames_256_511: 0 > > dev.em.3.mac_stats.tx_frames_512_1023: 0 > > dev.em.3.mac_stats.tx_frames_1024_1522: 0 > > dev.em.3.mac_stats.tso_txd: 0 > > dev.em.3.mac_stats.tso_ctx_fail: 0 > > dev.em.3.interrupts.asserts: 0 > > dev.em.3.interrupts.rx_pkt_timer: 0 > > dev.em.3.interrupts.rx_abs_timer: 0 > > dev.em.3.interrupts.tx_pkt_timer: 0 > > dev.em.3.interrupts.tx_abs_timer: 0 > > dev.em.3.interrupts.tx_queue_empty: 0 > > dev.em.3.interrupts.tx_queue_min_thresh: 0 > > dev.em.3.interrupts.rx_desc_min_thresh: 0 > > dev.em.3.interrupts.rx_overrun: 0 > > > > > .............................................................................. > From owner-freebsd-net@FreeBSD.ORG Tue May 13 00:24:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D03D5998 for ; Tue, 13 May 2014 00:24:10 +0000 (UTC) Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E3B8266F for ; Tue, 13 May 2014 00:24:10 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id ld13so9167619vcb.26 for ; Mon, 12 May 2014 17:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hTz8uAoP311vcketaWlMhH/O5xt7y/TBPBEN63L7be0=; b=xnyeVYpXL10OryI2iIqRqhFFvFdQjCaCzqUB4MwE9zItw/k5y9F/igd9Tyd7KxOUHZ q+CNyq6k15+GRkAnsUEdg0kIg35qbKJ/ttF/EGli0EpmR7PGJsG6XhSoBi71NnLLG0oa yMDlnEZl2pMS57kAzsDgyLJsjMh4jTtxBLD7x3/Io7klF588vpPgxjnYvHTmrPjIKANG seBv0uuxw75h9/gM7HRaWG3j+eMuCeUzRwwvolum49+8bqWGUvQYW4OogItGJ7YUWJ6T mC6f+H2IBMyBgcJfqgbwE3k9J5T1mhl4diGybFIV2tKfu4JSom4XLZY2bzsvH76dFw8G LNrw== MIME-Version: 1.0 X-Received: by 10.52.99.168 with SMTP id er8mr21822199vdb.26.1399940649588; Mon, 12 May 2014 17:24:09 -0700 (PDT) Received: by 10.52.98.130 with HTTP; Mon, 12 May 2014 17:24:09 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 May 2014 17:24:09 -0700 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Nick Rogers To: Jack Vogel Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 00:24:10 -0000 On Mon, May 12, 2014 at 5:16 PM, Jack Vogel wrote: > Nick, > > I'm very busy with some critical internal deadlines, I will look at this > when I can > come up for air, but please be patient. Thank you. > > Jack > > > > On Mon, May 12, 2014 at 4:56 PM, Nick Rogers wrote: >> >> On Fri, May 2, 2014 at 1:49 PM, Nick Rogers wrote: >> > Hello, >> > >> > I am hoping someone can help me debug a kernel panic I have been >> > experiencing >> > on one of my production systems. The system is a PF+ALTQ >> > firewall/gateway with >> > about 1k simultaneous devices behind it at any given time, pushing no >> > more >> > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug >> > it >> > with kgdb, but am still at a loss. >> > >> > I have completely replaced the hardware to rule out issues with >> > disk/memory/etc, and it appears to be a kernel issue, likely with e1000 >> > driver >> > and/or PF. >> > >> > The panic seems to happen during times of heavier use, but the frequency >> > is >> > not very predictable (anywhere from a few times a day to a once a week), >> > so I >> > feel like its some kind of strange traffic pattern that I am unable to >> > pinpoint. >> > >> > I am using a slightly custom kernel based on GENERIC, mainly to bring in >> > ALTQ >> > and some other options. >> > >> > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 >> > driver, although I do not believe that should affect em(4). >> > >> > Hoping someone can be of assistance in debugging this problem. I am >> > willing to >> > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if >> > that >> > is advisable. Unfortunately I cannot try 10-STABLE. >> > >> > Thanks, >> > >> > -Nick Rogers >> > >> > >> > uname -v >> > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 >> > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS >> > >> > >> > >> > Here is the kernel conf... >> > >> > .............................................................................. >> > include GENERIC >> > >> > ident RGNETS >> > >> > makeoptions MODULES_OVERRIDE="" >> > >> > options DEVICE_POLLING >> > >> > device crypto >> > device cryptodev >> > >> > options VLAN_ARRAY >> > >> > device amdtemp >> > >> > # PF >> > device pf >> > device pflog >> > options ALTQ >> > options ALTQ_CBQ # Class Bases Queuing (CBQ) >> > options ALTQ_RED # Random Early Detection (RED) >> > options ALTQ_RIO # RED In/Out >> > options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) >> > options ALTQ_PRIQ # Priority Queuing (PRIQ) >> > options ALTQ_NOPCC # Required for SMP build >> > >> > # PPPoE >> > options NETGRAPH >> > options NETGRAPH_ETHER >> > options NETGRAPH_PPPOE >> > options NETGRAPH_SOCKET >> > >> > # IPsec >> > device enc >> > options IPSEC >> > options IPSEC_FILTERTUNNEL >> > options IPSEC_NAT_T >> > >> > .............................................................................. >> > >> > >> > The crash dump.... >> > >> > .............................................................................. >> > >> > GNU gdb 6.1.1 [FreeBSD] >> > Copyright 2004 Free Software Foundation, Inc. >> > GDB is free software, covered by the GNU General Public License, and you >> > are >> > welcome to change it and/or distribute copies of it under certain >> > conditions. >> > Type "show copying" to see the conditions. >> > There is absolutely no warranty for GDB. Type "show warranty" for >> > details. >> > This GDB was configured as "amd64-marcel-freebsd"... >> > >> > Unread portion of the kernel message buffer: >> > >> > >> > Fatal trap 12: page fault while in kernel mode >> > cpuid = 5; apic id = 05 >> > fault virtual address = 0x10 >> > fault code = supervisor read data, page not present >> > instruction pointer = 0x20:0xffffffff8033d350 >> > stack pointer = 0x28:0xffffff83545384b0 >> > frame pointer = 0x28:0xffffff83545384c0 >> > code segment = base 0x0, limit 0xfffff, type 0x1b >> > = DPL 0, pres 1, long 1, def32 0, gran 1 >> > processor eflags = interrupt enabled, resume, IOPL = 0 >> > current process = 12 (irq262: em2:rx 0) >> > trap number = 12 >> > panic: page fault >> > cpuid = 5 >> > KDB: stack backtrace: >> > #0 0xffffffff80956836 at kdb_backtrace+0x66 >> > #1 0xffffffff8091c40e at panic+0x1ce >> > #2 0xffffffff80d31e70 at trap_fatal+0x290 >> > #3 0xffffffff80d321d1 at trap_pfault+0x211 >> > #4 0xffffffff80d327d3 at trap+0x363 >> > #5 0xffffffff80d1b9d3 at calltrap+0x8 >> > #6 0xffffffff8034872d at pf_test_rule+0x17ed >> > #7 0xffffffff8034ba12 at pf_test+0x1032 >> > #8 0xffffffff8035112b at pf_check_in+0x2b >> > #9 0xffffffff809e952e at pfil_run_hooks+0x9e >> > #10 0xffffffff80a5286a at ip_input+0x2ea >> > #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> > #12 0xffffffff809df93d at ether_demux+0x14d >> > #13 0xffffffff809dfc1e at ether_nh_input+0x1fe >> > #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> > #15 0xffffffff809df85f at ether_demux+0x6f >> > #16 0xffffffff809dfc1e at ether_nh_input+0x1fe >> > #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 >> > Uptime: 17d7h20m59s >> > Dumping 2932 out of 12256 MB: (CTRL-C to abort) >> > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> > >> > Reading symbols from /boot/kernel/aio.ko...Reading symbols from >> > /boot/kernel/aio.ko.symbols...done. >> > done. >> > Loaded symbols for /boot/kernel/aio.ko >> > Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from >> > /boot/kernel/coretemp.ko.symbols...done. >> > done. >> > Loaded symbols for /boot/kernel/coretemp.ko >> > Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from >> > /boot/kernel/cc_htcp.ko.symbols...done. >> > done. >> > Loaded symbols for /boot/kernel/cc_htcp.ko >> > #0 doadump (textdump=Variable "textdump" is not available. >> > ) at pcpu.h:234 >> > 234 pcpu.h: No such file or directory. >> > in pcpu.h >> > (kgdb) list *0xffffffff8033d350 >> > 0xffffffff8033d350 is in pf_addrcpy >> > (/usr/src/sys/contrib/pf/net/pf.c:512). >> > 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) >> > 508 { >> > 509 switch (af) { >> > 510 #ifdef INET >> > 511 case AF_INET: >> > 512 dst->addr32[0] = src->addr32[0]; >> > 513 break; >> > 514 #endif /* INET */ >> > 515 case AF_INET6: >> > 516 dst->addr32[0] = src->addr32[0]; >> > (kgdb) backtrace >> > #0 doadump (textdump=Variable "textdump" is not available. >> > ) at pcpu.h:234 >> > #1 0xffffffff8091bee6 in kern_reboot (howto=260) at >> > /usr/src/sys/kern/kern_shutdown.c:454 >> > #2 0xffffffff8091c3e7 in panic (fmt=0x1
) >> > at /usr/src/sys/kern/kern_shutdown.c:642 >> > #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is >> > not available. >> > ) at /usr/src/sys/amd64/amd64/trap.c:878 >> > #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, >> > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 >> > #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at >> > /usr/src/sys/amd64/amd64/trap.c:456 >> > #6 0xffffffff80d1b9d3 in calltrap () at >> > /usr/src/sys/amd64/amd64/exception.S:232 >> > #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, >> > src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 >> > #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, >> > sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, >> > m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, >> > pd=0xffffff83545386c0, am=0xffffff8354538790, >> > rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at >> > /usr/src/sys/contrib/pf/net/pf.c:3900 >> > #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not >> > available. >> > ) at /usr/src/sys/contrib/pf/net/pf.c:6776 >> > #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not >> > available. >> > ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 >> > #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not >> > available. >> > ) at /usr/src/sys/net/pfil.c:82 >> > #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at >> > /usr/src/sys/netinet/ip_input.c:510 >> > #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, >> > source=Variable "source" is not available. >> > ) at /usr/src/sys/net/netisr.c:1013 >> > #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, >> > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 >> > #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not >> > available. >> > ) at /usr/src/sys/net/if_ethersubr.c:762 >> > #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, >> > source=Variable "source" is not available. >> > ) at /usr/src/sys/net/netisr.c:1013 >> > #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, >> > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 >> > #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not >> > available. >> > ) at /usr/src/sys/net/if_ethersubr.c:762 >> > #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, >> > source=Variable "source" is not available. >> > ) at /usr/src/sys/net/netisr.c:1013 >> > #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, >> > done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 >> > #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not >> > available. >> > ) at /usr/src/sys/dev/e1000/if_em.c:1593 >> >> It seems like the panic is in the MSIX receive handler? I guess I will >> try disabling MSIX for em. >> >> Jack, can you comment on this please? >> >> >> > #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" >> > is not available. >> > ) at /usr/src/sys/kern/kern_intr.c:1272 >> > #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at >> > /usr/src/sys/kern/kern_intr.c:1285 >> > #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 >> > , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at >> > /usr/src/sys/kern/kern_fork.c:996 >> > #25 0xffffffff80d1befe in fork_trampoline () at >> > /usr/src/sys/amd64/amd64/exception.S:606 >> > #26 0x0000000000000000 in ?? () >> > #27 0x0000000000000000 in ?? () >> > #28 0x0000000000000001 in ?? () >> > #29 0x0000000000000000 in ?? () >> > #30 0x0000000000000000 in ?? () >> > #31 0x0000000000000000 in ?? () >> > #32 0x0000000000000000 in ?? () >> > #33 0x0000000000000000 in ?? () >> > #34 0x0000000000000000 in ?? () >> > #35 0x0000000000000000 in ?? () >> > #36 0x0000000000000000 in ?? () >> > #37 0x0000000000000000 in ?? () >> > #38 0x0000000000000000 in ?? () >> > #39 0x0000000000000000 in ?? () >> > #40 0x0000000000000000 in ?? () >> > #41 0x0000000000000000 in ?? () >> > #42 0x0000000000000000 in ?? () >> > #43 0x0000000000000000 in ?? () >> > #44 0x0000000000000000 in ?? () >> > #45 0x0000000000000000 in ?? () >> > #46 0x0000000000000000 in ?? () >> > #47 0x0000000000000000 in ?? () >> > #48 0x0000000000000000 in ?? () >> > ---Type to continue, or q to quit--- >> > #49 0x0000000000000000 in ?? () >> > #50 0x0000000000000000 in ?? () >> > #51 0xfffffe0007304920 in ?? () >> > #52 0xfffffe0003afd000 in ?? () >> > #53 0xfffffe0007304920 in ?? () >> > #54 0xffffff8354538b40 in ?? () >> > #55 0xffffff8354538ae8 in ?? () >> > #56 0xfffffe0003b00920 in ?? () >> > #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, >> > newtd=0xfffffe000730c300, flags=Variable "flags" is not available. >> > ) at /usr/src/sys/kern/sched_ule.c:1898 >> > Previous frame inner to this frame (corrupt stack?) >> > >> > >> > .............................................................................. >> > >> > >> > >> > Relevant pciconf output... >> > >> > .............................................................................. >> > >> > em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 >> > hdr=0x00 >> > vendor = 'Intel Corporation' >> > device = '82574L Gigabit Network Connection' >> > class = network >> > subclass = ethernet >> > cap 01[c8] = powerspec 2 supports D0 D3 current D0 >> > cap 05[d0] = MSI supports 1 message, 64 bit >> > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) >> > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled >> > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected >> > em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 >> > hdr=0x00 >> > vendor = 'Intel Corporation' >> > device = '82574L Gigabit Network Connection' >> > class = network >> > subclass = ethernet >> > cap 01[c8] = powerspec 2 supports D0 D3 current D0 >> > cap 05[d0] = MSI supports 1 message, 64 bit >> > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) >> > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled >> > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected >> > em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 >> > hdr=0x00 >> > vendor = 'Intel Corporation' >> > device = '82574L Gigabit Network Connection' >> > class = network >> > subclass = ethernet >> > cap 01[c8] = powerspec 2 supports D0 D3 current D0 >> > cap 05[d0] = MSI supports 1 message, 64 bit >> > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) >> > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled >> > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected >> > em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 >> > hdr=0x00 >> > vendor = 'Intel Corporation' >> > device = '82574L Gigabit Network Connection' >> > class = network >> > subclass = ethernet >> > cap 01[c8] = powerspec 2 supports D0 D3 current D0 >> > cap 05[d0] = MSI supports 1 message, 64 bit >> > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) >> > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled >> > >> > >> > .............................................................................. >> > >> > >> > dev.em sysctl.... >> > >> > .............................................................................. >> > >> > dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> > dev.em.0.%driver: em >> > dev.em.0.%location: slot=0 function=0 >> > dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 >> > subdevice=0x10d3 class=0x020000 >> > dev.em.0.%parent: pci1 >> > dev.em.0.nvm: -1 >> > dev.em.0.debug: -1 >> > dev.em.0.fc: 3 >> > dev.em.0.rx_int_delay: 0 >> > dev.em.0.tx_int_delay: 66 >> > dev.em.0.rx_abs_int_delay: 66 >> > dev.em.0.tx_abs_int_delay: 66 >> > dev.em.0.itr: 488 >> > dev.em.0.rx_processing_limit: -1 >> > dev.em.0.eee_control: 1 >> > dev.em.0.link_irq: 2 >> > dev.em.0.mbuf_alloc_fail: 0 >> > dev.em.0.cluster_alloc_fail: 0 >> > dev.em.0.dropped: 0 >> > dev.em.0.tx_dma_fail: 0 >> > dev.em.0.rx_overruns: 0 >> > dev.em.0.watchdog_timeouts: 0 >> > dev.em.0.device_control: 1477444168 >> > dev.em.0.rx_control: 67141634 >> > dev.em.0.fc_high_water: 18432 >> > dev.em.0.fc_low_water: 16932 >> > dev.em.0.queue0.txd_head: 3265 >> > dev.em.0.queue0.txd_tail: 3265 >> > dev.em.0.queue0.tx_irq: 81153071 >> > dev.em.0.queue0.no_desc_avail: 0 >> > dev.em.0.queue0.rxd_head: 388 >> > dev.em.0.queue0.rxd_tail: 387 >> > dev.em.0.queue0.rx_irq: 79015024 >> > dev.em.0.mac_stats.excess_coll: 0 >> > dev.em.0.mac_stats.single_coll: 0 >> > dev.em.0.mac_stats.multiple_coll: 0 >> > dev.em.0.mac_stats.late_coll: 0 >> > dev.em.0.mac_stats.collision_count: 0 >> > dev.em.0.mac_stats.symbol_errors: 0 >> > dev.em.0.mac_stats.sequence_errors: 0 >> > dev.em.0.mac_stats.defer_count: 0 >> > dev.em.0.mac_stats.missed_packets: 0 >> > dev.em.0.mac_stats.recv_no_buff: 0 >> > dev.em.0.mac_stats.recv_undersize: 0 >> > dev.em.0.mac_stats.recv_fragmented: 0 >> > dev.em.0.mac_stats.recv_oversize: 0 >> > dev.em.0.mac_stats.recv_jabber: 0 >> > dev.em.0.mac_stats.recv_errs: 0 >> > dev.em.0.mac_stats.crc_errs: 0 >> > dev.em.0.mac_stats.alignment_errs: 0 >> > dev.em.0.mac_stats.coll_ext_errs: 0 >> > dev.em.0.mac_stats.xon_recvd: 6 >> > dev.em.0.mac_stats.xon_txd: 0 >> > dev.em.0.mac_stats.xoff_recvd: 6 >> > dev.em.0.mac_stats.xoff_txd: 0 >> > dev.em.0.mac_stats.total_pkts_recvd: 122072630 >> > dev.em.0.mac_stats.good_pkts_recvd: 122072618 >> > dev.em.0.mac_stats.bcast_pkts_recvd: 257 >> > dev.em.0.mac_stats.mcast_pkts_recvd: 0 >> > dev.em.0.mac_stats.rx_frames_64: 8634 >> > dev.em.0.mac_stats.rx_frames_65_127: 67656673 >> > dev.em.0.mac_stats.rx_frames_128_255: 714152 >> > dev.em.0.mac_stats.rx_frames_256_511: 609615 >> > dev.em.0.mac_stats.rx_frames_512_1023: 8646536 >> > dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 >> > dev.em.0.mac_stats.good_octets_recvd: 82940411216 >> > dev.em.0.mac_stats.good_octets_txd: 25718335997 >> > dev.em.0.mac_stats.total_pkts_txd: 99833592 >> > dev.em.0.mac_stats.good_pkts_txd: 99833592 >> > dev.em.0.mac_stats.bcast_pkts_txd: 13 >> > dev.em.0.mac_stats.mcast_pkts_txd: 0 >> > dev.em.0.mac_stats.tx_frames_64: 2193 >> > dev.em.0.mac_stats.tx_frames_65_127: 29089783 >> > dev.em.0.mac_stats.tx_frames_128_255: 54412030 >> > dev.em.0.mac_stats.tx_frames_256_511: 9565246 >> > dev.em.0.mac_stats.tx_frames_512_1023: 1080398 >> > dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 >> > dev.em.0.mac_stats.tso_txd: 1468623 >> > dev.em.0.mac_stats.tso_ctx_fail: 0 >> > dev.em.0.interrupts.asserts: 2 >> > dev.em.0.interrupts.rx_pkt_timer: 0 >> > dev.em.0.interrupts.rx_abs_timer: 0 >> > dev.em.0.interrupts.tx_pkt_timer: 0 >> > dev.em.0.interrupts.tx_abs_timer: 0 >> > dev.em.0.interrupts.tx_queue_empty: 0 >> > dev.em.0.interrupts.tx_queue_min_thresh: 0 >> > dev.em.0.interrupts.rx_desc_min_thresh: 0 >> > dev.em.0.interrupts.rx_overrun: 0 >> > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> > dev.em.1.%driver: em >> > dev.em.1.%location: slot=0 function=0 >> > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 >> > subdevice=0x10d3 class=0x020000 >> > dev.em.1.%parent: pci2 >> > dev.em.1.nvm: -1 >> > dev.em.1.debug: -1 >> > dev.em.1.fc: 3 >> > dev.em.1.rx_int_delay: 0 >> > dev.em.1.tx_int_delay: 66 >> > dev.em.1.rx_abs_int_delay: 66 >> > dev.em.1.tx_abs_int_delay: 66 >> > dev.em.1.itr: 488 >> > dev.em.1.rx_processing_limit: -1 >> > dev.em.1.eee_control: 1 >> > dev.em.1.link_irq: 2 >> > dev.em.1.mbuf_alloc_fail: 0 >> > dev.em.1.cluster_alloc_fail: 0 >> > dev.em.1.dropped: 0 >> > dev.em.1.tx_dma_fail: 1 >> > dev.em.1.rx_overruns: 0 >> > dev.em.1.watchdog_timeouts: 0 >> > dev.em.1.device_control: 1477444168 >> > dev.em.1.rx_control: 67141634 >> > dev.em.1.fc_high_water: 18432 >> > dev.em.1.fc_low_water: 16932 >> > dev.em.1.queue0.txd_head: 2451 >> > dev.em.1.queue0.txd_tail: 2453 >> > dev.em.1.queue0.tx_irq: 143904807 >> > dev.em.1.queue0.no_desc_avail: 0 >> > dev.em.1.queue0.rxd_head: 342 >> > dev.em.1.queue0.rxd_tail: 341 >> > dev.em.1.queue0.rx_irq: 159303310 >> > dev.em.1.mac_stats.excess_coll: 0 >> > dev.em.1.mac_stats.single_coll: 0 >> > dev.em.1.mac_stats.multiple_coll: 0 >> > dev.em.1.mac_stats.late_coll: 0 >> > dev.em.1.mac_stats.collision_count: 0 >> > dev.em.1.mac_stats.symbol_errors: 0 >> > dev.em.1.mac_stats.sequence_errors: 0 >> > dev.em.1.mac_stats.defer_count: 0 >> > dev.em.1.mac_stats.missed_packets: 0 >> > dev.em.1.mac_stats.recv_no_buff: 0 >> > dev.em.1.mac_stats.recv_undersize: 0 >> > dev.em.1.mac_stats.recv_fragmented: 0 >> > dev.em.1.mac_stats.recv_oversize: 0 >> > dev.em.1.mac_stats.recv_jabber: 0 >> > dev.em.1.mac_stats.recv_errs: 0 >> > dev.em.1.mac_stats.crc_errs: 0 >> > dev.em.1.mac_stats.alignment_errs: 0 >> > dev.em.1.mac_stats.coll_ext_errs: 0 >> > dev.em.1.mac_stats.xon_recvd: 1 >> > dev.em.1.mac_stats.xon_txd: 0 >> > dev.em.1.mac_stats.xoff_recvd: 1 >> > dev.em.1.mac_stats.xoff_txd: 0 >> > dev.em.1.mac_stats.total_pkts_recvd: 331901758 >> > dev.em.1.mac_stats.good_pkts_recvd: 331901756 >> > dev.em.1.mac_stats.bcast_pkts_recvd: 13467 >> > dev.em.1.mac_stats.mcast_pkts_recvd: 0 >> > dev.em.1.mac_stats.rx_frames_64: 13905035 >> > dev.em.1.mac_stats.rx_frames_65_127: 22315178 >> > dev.em.1.mac_stats.rx_frames_128_255: 8343368 >> > dev.em.1.mac_stats.rx_frames_256_511: 8602323 >> > dev.em.1.mac_stats.rx_frames_512_1023: 8170288 >> > dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 >> > dev.em.1.mac_stats.good_octets_recvd: 420794715670 >> > dev.em.1.mac_stats.good_octets_txd: 45361473880 >> > dev.em.1.mac_stats.total_pkts_txd: 217852588 >> > dev.em.1.mac_stats.good_pkts_txd: 217852588 >> > dev.em.1.mac_stats.bcast_pkts_txd: 6 >> > dev.em.1.mac_stats.mcast_pkts_txd: 0 >> > dev.em.1.mac_stats.tx_frames_64: 64102191 >> > dev.em.1.mac_stats.tx_frames_65_127: 120705475 >> > dev.em.1.mac_stats.tx_frames_128_255: 6009336 >> > dev.em.1.mac_stats.tx_frames_256_511: 4593595 >> > dev.em.1.mac_stats.tx_frames_512_1023: 4295623 >> > dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 >> > dev.em.1.mac_stats.tso_txd: 291134 >> > dev.em.1.mac_stats.tso_ctx_fail: 0 >> > dev.em.1.interrupts.asserts: 2 >> > dev.em.1.interrupts.rx_pkt_timer: 0 >> > dev.em.1.interrupts.rx_abs_timer: 0 >> > dev.em.1.interrupts.tx_pkt_timer: 0 >> > dev.em.1.interrupts.tx_abs_timer: 0 >> > dev.em.1.interrupts.tx_queue_empty: 0 >> > dev.em.1.interrupts.tx_queue_min_thresh: 0 >> > dev.em.1.interrupts.rx_desc_min_thresh: 0 >> > dev.em.1.interrupts.rx_overrun: 0 >> > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> > dev.em.2.%driver: em >> > dev.em.2.%location: slot=0 function=0 >> > dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 >> > subdevice=0x10d3 class=0x020000 >> > dev.em.2.%parent: pci7 >> > dev.em.2.nvm: -1 >> > dev.em.2.debug: -1 >> > dev.em.2.fc: 3 >> > dev.em.2.rx_int_delay: 0 >> > dev.em.2.tx_int_delay: 66 >> > dev.em.2.rx_abs_int_delay: 66 >> > dev.em.2.tx_abs_int_delay: 66 >> > dev.em.2.itr: 488 >> > dev.em.2.rx_processing_limit: -1 >> > dev.em.2.eee_control: 1 >> > dev.em.2.link_irq: 1 >> > dev.em.2.mbuf_alloc_fail: 0 >> > dev.em.2.cluster_alloc_fail: 0 >> > dev.em.2.dropped: 0 >> > dev.em.2.tx_dma_fail: 6823 >> > dev.em.2.rx_overruns: 0 >> > dev.em.2.watchdog_timeouts: 0 >> > dev.em.2.device_control: 1477444168 >> > dev.em.2.rx_control: 67141634 >> > dev.em.2.fc_high_water: 18432 >> > dev.em.2.fc_low_water: 16932 >> > dev.em.2.queue0.txd_head: 3977 >> > dev.em.2.queue0.txd_tail: 3977 >> > dev.em.2.queue0.tx_irq: 220950699 >> > dev.em.2.queue0.no_desc_avail: 0 >> > dev.em.2.queue0.rxd_head: 83 >> > dev.em.2.queue0.rxd_tail: 82 >> > dev.em.2.queue0.rx_irq: 125920607 >> > dev.em.2.mac_stats.excess_coll: 0 >> > dev.em.2.mac_stats.single_coll: 0 >> > dev.em.2.mac_stats.multiple_coll: 0 >> > dev.em.2.mac_stats.late_coll: 0 >> > dev.em.2.mac_stats.collision_count: 0 >> > dev.em.2.mac_stats.symbol_errors: 0 >> > dev.em.2.mac_stats.sequence_errors: 0 >> > dev.em.2.mac_stats.defer_count: 0 >> > dev.em.2.mac_stats.missed_packets: 0 >> > dev.em.2.mac_stats.recv_no_buff: 0 >> > dev.em.2.mac_stats.recv_undersize: 0 >> > dev.em.2.mac_stats.recv_fragmented: 0 >> > dev.em.2.mac_stats.recv_oversize: 0 >> > dev.em.2.mac_stats.recv_jabber: 0 >> > dev.em.2.mac_stats.recv_errs: 0 >> > dev.em.2.mac_stats.crc_errs: 0 >> > dev.em.2.mac_stats.alignment_errs: 0 >> > dev.em.2.mac_stats.coll_ext_errs: 0 >> > dev.em.2.mac_stats.xon_recvd: 14123 >> > dev.em.2.mac_stats.xon_txd: 1 >> > dev.em.2.mac_stats.xoff_recvd: 14127 >> > dev.em.2.mac_stats.xoff_txd: 1 >> > dev.em.2.mac_stats.total_pkts_recvd: 229919303 >> > dev.em.2.mac_stats.good_pkts_recvd: 229891053 >> > dev.em.2.mac_stats.bcast_pkts_recvd: 909450 >> > dev.em.2.mac_stats.mcast_pkts_recvd: 19452 >> > dev.em.2.mac_stats.rx_frames_64: 1477808 >> > dev.em.2.mac_stats.rx_frames_65_127: 195114744 >> > dev.em.2.mac_stats.rx_frames_128_255: 6579690 >> > dev.em.2.mac_stats.rx_frames_256_511: 5137387 >> > dev.em.2.mac_stats.rx_frames_512_1023: 4223090 >> > dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 >> > dev.em.2.mac_stats.good_octets_recvd: 46129102134 >> > dev.em.2.mac_stats.good_octets_txd: 419293159496 >> > dev.em.2.mac_stats.total_pkts_txd: 332661584 >> > dev.em.2.mac_stats.good_pkts_txd: 332661582 >> > dev.em.2.mac_stats.bcast_pkts_txd: 48506 >> > dev.em.2.mac_stats.mcast_pkts_txd: 78 >> > dev.em.2.mac_stats.tx_frames_64: 14598198 >> > dev.em.2.mac_stats.tx_frames_65_127: 22287108 >> > dev.em.2.mac_stats.tx_frames_128_255: 8897511 >> > dev.em.2.mac_stats.tx_frames_256_511: 9623000 >> > dev.em.2.mac_stats.tx_frames_512_1023: 8325033 >> > dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 >> > dev.em.2.mac_stats.tso_txd: 24357891 >> > dev.em.2.mac_stats.tso_ctx_fail: 0 >> > dev.em.2.interrupts.asserts: 2 >> > dev.em.2.interrupts.rx_pkt_timer: 0 >> > dev.em.2.interrupts.rx_abs_timer: 0 >> > dev.em.2.interrupts.tx_pkt_timer: 0 >> > dev.em.2.interrupts.tx_abs_timer: 0 >> > dev.em.2.interrupts.tx_queue_empty: 0 >> > dev.em.2.interrupts.tx_queue_min_thresh: 0 >> > dev.em.2.interrupts.rx_desc_min_thresh: 0 >> > dev.em.2.interrupts.rx_overrun: 0 >> > dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 >> > dev.em.3.%driver: em >> > dev.em.3.%location: slot=0 function=0 >> > dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 >> > subdevice=0x10d3 class=0x020000 >> > dev.em.3.%parent: pci8 >> > dev.em.3.nvm: -1 >> > dev.em.3.debug: -1 >> > dev.em.3.fc: 3 >> > dev.em.3.rx_int_delay: 0 >> > dev.em.3.tx_int_delay: 66 >> > dev.em.3.rx_abs_int_delay: 66 >> > dev.em.3.tx_abs_int_delay: 66 >> > dev.em.3.itr: 488 >> > dev.em.3.rx_processing_limit: -1 >> > dev.em.3.eee_control: 1 >> > dev.em.3.link_irq: 0 >> > dev.em.3.mbuf_alloc_fail: 0 >> > dev.em.3.cluster_alloc_fail: 0 >> > dev.em.3.dropped: 0 >> > dev.em.3.tx_dma_fail: 0 >> > dev.em.3.rx_overruns: 0 >> > dev.em.3.watchdog_timeouts: 0 >> > dev.em.3.device_control: 1074790984 >> > dev.em.3.rx_control: 67141634 >> > dev.em.3.fc_high_water: 18432 >> > dev.em.3.fc_low_water: 16932 >> > dev.em.3.queue0.txd_head: 0 >> > dev.em.3.queue0.txd_tail: 0 >> > dev.em.3.queue0.tx_irq: 0 >> > dev.em.3.queue0.no_desc_avail: 0 >> > dev.em.3.queue0.rxd_head: 0 >> > dev.em.3.queue0.rxd_tail: 4095 >> > dev.em.3.queue0.rx_irq: 0 >> > dev.em.3.mac_stats.excess_coll: 0 >> > dev.em.3.mac_stats.single_coll: 0 >> > dev.em.3.mac_stats.multiple_coll: 0 >> > dev.em.3.mac_stats.late_coll: 0 >> > dev.em.3.mac_stats.collision_count: 0 >> > dev.em.3.mac_stats.symbol_errors: 0 >> > dev.em.3.mac_stats.sequence_errors: 0 >> > dev.em.3.mac_stats.defer_count: 0 >> > dev.em.3.mac_stats.missed_packets: 0 >> > dev.em.3.mac_stats.recv_no_buff: 0 >> > dev.em.3.mac_stats.recv_undersize: 0 >> > dev.em.3.mac_stats.recv_fragmented: 0 >> > dev.em.3.mac_stats.recv_oversize: 0 >> > dev.em.3.mac_stats.recv_jabber: 0 >> > dev.em.3.mac_stats.recv_errs: 0 >> > dev.em.3.mac_stats.crc_errs: 0 >> > dev.em.3.mac_stats.alignment_errs: 0 >> > dev.em.3.mac_stats.coll_ext_errs: 0 >> > dev.em.3.mac_stats.xon_recvd: 0 >> > dev.em.3.mac_stats.xon_txd: 0 >> > dev.em.3.mac_stats.xoff_recvd: 0 >> > dev.em.3.mac_stats.xoff_txd: 0 >> > dev.em.3.mac_stats.total_pkts_recvd: 0 >> > dev.em.3.mac_stats.good_pkts_recvd: 0 >> > dev.em.3.mac_stats.bcast_pkts_recvd: 0 >> > dev.em.3.mac_stats.mcast_pkts_recvd: 0 >> > dev.em.3.mac_stats.rx_frames_64: 0 >> > dev.em.3.mac_stats.rx_frames_65_127: 0 >> > dev.em.3.mac_stats.rx_frames_128_255: 0 >> > dev.em.3.mac_stats.rx_frames_256_511: 0 >> > dev.em.3.mac_stats.rx_frames_512_1023: 0 >> > dev.em.3.mac_stats.rx_frames_1024_1522: 0 >> > dev.em.3.mac_stats.good_octets_recvd: 0 >> > dev.em.3.mac_stats.good_octets_txd: 0 >> > dev.em.3.mac_stats.total_pkts_txd: 0 >> > dev.em.3.mac_stats.good_pkts_txd: 0 >> > dev.em.3.mac_stats.bcast_pkts_txd: 0 >> > dev.em.3.mac_stats.mcast_pkts_txd: 0 >> > dev.em.3.mac_stats.tx_frames_64: 0 >> > dev.em.3.mac_stats.tx_frames_65_127: 0 >> > dev.em.3.mac_stats.tx_frames_128_255: 0 >> > dev.em.3.mac_stats.tx_frames_256_511: 0 >> > dev.em.3.mac_stats.tx_frames_512_1023: 0 >> > dev.em.3.mac_stats.tx_frames_1024_1522: 0 >> > dev.em.3.mac_stats.tso_txd: 0 >> > dev.em.3.mac_stats.tso_ctx_fail: 0 >> > dev.em.3.interrupts.asserts: 0 >> > dev.em.3.interrupts.rx_pkt_timer: 0 >> > dev.em.3.interrupts.rx_abs_timer: 0 >> > dev.em.3.interrupts.tx_pkt_timer: 0 >> > dev.em.3.interrupts.tx_abs_timer: 0 >> > dev.em.3.interrupts.tx_queue_empty: 0 >> > dev.em.3.interrupts.tx_queue_min_thresh: 0 >> > dev.em.3.interrupts.rx_desc_min_thresh: 0 >> > dev.em.3.interrupts.rx_overrun: 0 >> > >> > >> > .............................................................................. > > From owner-freebsd-net@FreeBSD.ORG Tue May 13 04:47:35 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54BC3F52; Tue, 13 May 2014 04:47:35 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 292B929BD; Tue, 13 May 2014 04:47:35 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4D4lZuJ099701; Tue, 13 May 2014 04:47:35 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4D4lYn9099700; Tue, 13 May 2014 04:47:34 GMT (envelope-from linimon) Date: Tue, 13 May 2014 04:47:34 GMT Message-Id: <201405130447.s4D4lYn9099700@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/189531: [fxp] Wake on Lan (WOL) enabled for Intel 82562EZ ethernet adapter, but WOL does not work X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 04:47:35 -0000 Old Synopsis: Wake on Lan (WOL) enabled for Intel 82562EZ ethernet adapter, but WOL does not work New Synopsis: [fxp] Wake on Lan (WOL) enabled for Intel 82562EZ ethernet adapter, but WOL does not work Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Tue May 13 04:47:02 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=189531 From owner-freebsd-net@FreeBSD.ORG Tue May 13 05:21:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A413CA9E; Tue, 13 May 2014 05:21:39 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C8342CA4; Tue, 13 May 2014 05:21:39 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id lj1so9717439pab.14 for ; Mon, 12 May 2014 22:21:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=mOB4bsJVlO3s+dOvIY0ngXmUV2WhaPXUMGOkVdCjiBo=; b=oMrc+Yc6bIk5kWj+C0Pt8V2uy33c+ptCDuPZGSuXrAfRdaXmx8Vp1FQJCpLc9B2kHd 2rpgiV5/3V913AqFFYS2uzlPcz16cIJcLLg8u+ZX+JDmId2yrt2tf7KytsXOPIFZEIAU Sltfgcp8jsKTNUE4HR4KYFIgf/Uk/Srmsc/Ps/hkGKeUr6KUF7LdoUpTnpO1cK88IIWG L0Q26xWjdpZK+99zzjITjk6htJ9hVvErLvmDwFsjc1C5Vaw7pMKOEfK9MACPrp7gAu1f bKUaXa2y1MZPsnFHzbL4kSiuBBsjsI75TpvAnIDE30OabiikqrBpul/dUxdjtx85Wx46 tf8Q== X-Received: by 10.66.136.131 with SMTP id qa3mr61172005pab.77.1399958499014; Mon, 12 May 2014 22:21:39 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id ek2sm25951177pbd.30.2014.05.12.22.21.36 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 12 May 2014 22:21:38 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 13 May 2014 14:21:34 +0900 From: Yonghyeon PYUN Date: Tue, 13 May 2014 14:21:34 +0900 To: Michael Tuexen Subject: Re: RX checksum offloading problem Message-ID: <20140513052134.GA1419@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> <20140512013612.GA4085@michelle.cdnetworks.com> <72BEDE7B-C1F1-4EE2-BA68-49AED7209E8A@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72BEDE7B-C1F1-4EE2-BA68-49AED7209E8A@lurchi.franken.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 05:21:39 -0000 On Mon, May 12, 2014 at 01:22:03PM +0200, Michael Tuexen wrote: > On 12 May 2014, at 03:36, Yonghyeon PYUN wrote: > > > On Fri, May 09, 2014 at 12:46:48PM +0200, Michael Tuexen wrote: > >> On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: > >> [...] > > Oops, sorry. You're right. Probably I was confused with old memory > > when I worked on that area. I've quickly read IP reassembly code > > again and as you said, it should work. However it seems there is a > > checksumming bug here. > > > > /* > > * In order to do checksumming faster we do 'end-around carry' here > > * (and not in for{} loop), though it implies we are not going to > > * reassemble more than 64k fragments. > > */ > > m->m_pkthdr.csum_data = > > (m->m_pkthdr.csum_data & 0xffff) + (m->m_pkthdr.csum_data >> 16); > > > > I guess the line above didn't account possible carry happened after > > the computation. Probably it could be rewritten as the following. > > > > while (m->m_pkthdr.csum_data & 0xffff0000) > > m->m_pkthdr.csum_data = (m->m_pkthdr.csum_data & 0xffff) + > > (m->m_pkthdr.csum_data >> 16); > I think you are right here. Good catch. Will you fix it? > Done in r265942. From owner-freebsd-net@FreeBSD.ORG Tue May 13 05:23:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB0A8C76 for ; Tue, 13 May 2014 05:23:23 +0000 (UTC) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BA072CC8 for ; Tue, 13 May 2014 05:23:23 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id rd3so9811760pab.15 for ; Mon, 12 May 2014 22:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=cHJZFF5zrvGlGGOqvOV90m6bVt9nRZFiKTZbiHYshyw=; b=dJVyIvfF7y94uHal8Lwp8/U/tnLwCSH/s3zGEbKyBNyUWLoNbGwT3RIyfvPqKd+2Wr os8n4YGLHHp6HoWSvZ7jQA/IvW1xlGqpXBPO4VGfHijBofkCgkby8gqXMv2H0q/z9vOp t+aR8wskUP44NZ/Gs9dUpqqovRptVeLHlo/xzmGfyVuuN1p3aUsqIuUD1rv/yw5Bd20p zgArd6XshFowrJifAL2apEShlNIjoYvZxChRuXKxsfZczboLkruIPU7jWPflN0LdkCBw fFKybjoVm98jD4wB82qRzr0/zD8JE7KK+TSA46Uj104WQ/GJD/E8qE8iumbiuAtrA9RV GWOg== X-Received: by 10.69.26.103 with SMTP id ix7mr2731968pbd.41.1399958603282; Mon, 12 May 2014 22:23:23 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id wp3sm25926634pbc.67.2014.05.12.22.23.20 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 12 May 2014 22:23:22 -0700 (PDT) X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 13 May 2014 14:23:19 +0900 From: Yonghyeon PYUN Date: Tue, 13 May 2014 14:23:19 +0900 To: Michael Tuexen Subject: Re: TX Checksum offloading issue with re interfaces Message-ID: <20140513052319.GB1419@michelle.cdnetworks.com> Reply-To: pyunyh@gmail.com References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> <20140509014733.GB3014@michelle.cdnetworks.com> <20140512043837.GA1364@michelle.cdnetworks.com> <4DCF8CB0-220E-4BC4-AF71-E84CCDA88315@lurchi.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4DCF8CB0-220E-4BC4-AF71-E84CCDA88315@lurchi.franken.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 05:23:23 -0000 On Mon, May 12, 2014 at 01:09:18PM +0200, Michael Tuexen wrote: > On 12 May 2014, at 06:38, Yonghyeon PYUN wrote: > > > On Fri, May 09, 2014 at 12:33:24PM +0200, Michael Tuexen wrote: > >> On 09 May 2014, at 03:47, Yonghyeon PYUN wrote: > >> > >>> On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: > >>>> Dear all, > >>>> > >>>> while testing checksum offloading of UDP packets over IP with IP options, I figured > >>>> out that my card > >>>> > >>>> dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet > >>>> dev.re.1.%driver: re > >>>> dev.re.1.%location: slot=0 function=0 handle=\_SB_.PCI0.PE1F.LAN2 > >>>> dev.re.1.%pnpinfo: vendor=0x10ec device=0x8168 subvendor=0x1734 subdevice=0x1159 class=0x020000 > >>>> dev.re.1.%parent: pci13 > >>>> dev.re.1.stats: -1 > >>>> dev.re.1.int_rx_mod: 65 > >>>> > >>>> computes the UDP checksum, but stores it in the packet at the place, where it would be, > >>>> if there are no IP options. So it corrupts the options in the packet... > >>>> > >>>> I looked at sys/dev/re/if_re.c, but couldn't figure out how to fix it. Any idea? > >>>> > >>> > >>> re(4) has a very long history on its broken TX checksum offloading. > >>> So re(4) has many workarounds for known issues on several variants. > >>> re(4) controllers support TX IPv4/TCP/UDP checksum offloading. For > >>> 8168C/8168CP, TX IPv4 checksum offloading was disabled due to > >>> generation of corrupted frames. > >>> Could you show me the dmesg output(only re(4)/rgephy(4))? > >> > >>> The vendor uses the same PCI id for its RTL8168/8111 family chips > >>> so dmesg output is necessary to know exact controller revision. > >> Sure (re1 was used during the test): > >> re0: port 0x8000-0x80ff mem 0xf6104000-0xf6104fff,0xf6100000-0xf6103fff irq 16 at device 0.0 on pci12 > >> re0: Using 1 MSI-X message > >> re0: Chip rev. 0x28800000 > >> re0: MAC rev. 0x00200000 > >> miibus0: on re0 > >> rgephy0: PHY 1 on miibus0 > >> rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow > >> re0: Ethernet address: 00:19:99:85:31:d9 > >> re1: port 0x9000-0x90ff mem 0xf5c20000-0xf5c20fff,0xf6200000-0xf620ffff irq 17 at device 0.0 on pci13 > >> re1: Using 1 MSI-X message > >> re1: Chip rev. 0x3c800000 > >> re1: MAC rev. 0x00300000 > >> miibus1: on re1 > >> rgephy1: PHY 1 on miibus1 > >> rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow > >> re1: Ethernet address: 00:19:99:7e:c7:46 > >> > > It seems you have two variants. > You are right, I didn't know. Both are on-board interfaces... > > re0 is RTL8168DP and re1 is RTL8168CP. Do you see the issue on > > both controllers? I guess you may see the issue on re1 only since > > you've posted dev.re.1 output. I've attached a diff which may > It wasn't intentionally, but by accident, based on the addresses > I was using. However, I now tested both interfaces and re0 works > without any patch, but re1 needs your patch. > > address the issue on re1 interface. If you see the issue on re0, > > I have to change the diff to include RTL8168D. > Your patch looks good. Please go ahead and commit it. > Thanks for your help! Fixed in r265943. Thanks for testing! From owner-freebsd-net@FreeBSD.ORG Tue May 13 07:01:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1EF8C51D for ; Tue, 13 May 2014 07:01:53 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A56C22416 for ; Tue, 13 May 2014 07:01:52 +0000 (UTC) Received: from [192.168.1.200] (p54819945.dip0.t-ipconnect.de [84.129.153.69]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 32E341C104644; Tue, 13 May 2014 09:01:50 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: TX Checksum offloading issue with re interfaces From: Michael Tuexen In-Reply-To: <20140513052319.GB1419@michelle.cdnetworks.com> Date: Tue, 13 May 2014 09:01:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <256D7D52-A58A-43D0-A77F-A46D523B2D5A@lurchi.franken.de> References: <95E18108-E6DD-49EA-90B3-A9F9E5F93D20@lurchi.franken.de> <20140509014733.GB3014@michelle.cdnetworks.com> <20140512043837.GA1364@michelle.cdnetworks.com> <4DCF8CB0-220E-4BC4-AF71-E84CCDA88315@lurchi.franken.de> <20140513052319.GB1419@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 07:01:53 -0000 On 13 May 2014, at 07:23, Yonghyeon PYUN wrote: > On Mon, May 12, 2014 at 01:09:18PM +0200, Michael Tuexen wrote: >> On 12 May 2014, at 06:38, Yonghyeon PYUN wrote: >>=20 >>> On Fri, May 09, 2014 at 12:33:24PM +0200, Michael Tuexen wrote: >>>> On 09 May 2014, at 03:47, Yonghyeon PYUN wrote: >>>>=20 >>>>> On Thu, May 08, 2014 at 08:50:48PM +0200, Michael Tuexen wrote: >>>>>> Dear all, >>>>>>=20 >>>>>> while testing checksum offloading of UDP packets over IP with IP = options, I figured >>>>>> out that my card >>>>>>=20 >>>>>> dev.re.1.%desc: RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit = Ethernet >>>>>> dev.re.1.%driver: re >>>>>> dev.re.1.%location: slot=3D0 function=3D0 = handle=3D\_SB_.PCI0.PE1F.LAN2 >>>>>> dev.re.1.%pnpinfo: vendor=3D0x10ec device=3D0x8168 = subvendor=3D0x1734 subdevice=3D0x1159 class=3D0x020000 >>>>>> dev.re.1.%parent: pci13 >>>>>> dev.re.1.stats: -1 >>>>>> dev.re.1.int_rx_mod: 65 >>>>>>=20 >>>>>> computes the UDP checksum, but stores it in the packet at the = place, where it would be, >>>>>> if there are no IP options. So it corrupts the options in the = packet... >>>>>>=20 >>>>>> I looked at sys/dev/re/if_re.c, but couldn't figure out how to = fix it. Any idea? >>>>>>=20 >>>>>=20 >>>>> re(4) has a very long history on its broken TX checksum = offloading. >>>>> So re(4) has many workarounds for known issues on several = variants. >>>>> re(4) controllers support TX IPv4/TCP/UDP checksum offloading. = For >>>>> 8168C/8168CP, TX IPv4 checksum offloading was disabled due to >>>>> generation of corrupted frames. >>>>> Could you show me the dmesg output(only re(4)/rgephy(4))? >>>>=20 >>>>> The vendor uses the same PCI id for its RTL8168/8111 family chips >>>>> so dmesg output is necessary to know exact controller revision. >>>> Sure (re1 was used during the test): >>>> re0: = port 0x8000-0x80ff mem 0xf6104000-0xf6104fff,0xf6100000-0xf6103fff irq = 16 at device 0.0 on pci12 >>>> re0: Using 1 MSI-X message >>>> re0: Chip rev. 0x28800000 >>>> re0: MAC rev. 0x00200000 >>>> miibus0: on re0 >>>> rgephy0: PHY 1 on miibus0 >>>> rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow >>>> re0: Ethernet address: 00:19:99:85:31:d9 >>>> re1: = port 0x9000-0x90ff mem 0xf5c20000-0xf5c20fff,0xf6200000-0xf620ffff irq = 17 at device 0.0 on pci13 >>>> re1: Using 1 MSI-X message >>>> re1: Chip rev. 0x3c800000 >>>> re1: MAC rev. 0x00300000 >>>> miibus1: on re1 >>>> rgephy1: PHY 1 on = miibus1 >>>> rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, = 1000baseT-FDX-flow-master, auto, auto-flow >>>> re1: Ethernet address: 00:19:99:7e:c7:46 >>>>=20 >>> It seems you have two variants. >> You are right, I didn't know. Both are on-board interfaces... >>> re0 is RTL8168DP and re1 is RTL8168CP. Do you see the issue on >>> both controllers? I guess you may see the issue on re1 only since >>> you've posted dev.re.1 output. I've attached a diff which may >> It wasn't intentionally, but by accident, based on the addresses >> I was using. However, I now tested both interfaces and re0 works >> without any patch, but re1 needs your patch. >>> address the issue on re1 interface. If you see the issue on re0, >>> I have to change the diff to include RTL8168D. >> Your patch looks good. Please go ahead and commit it. >> Thanks for your help! >=20 > Fixed in r265943. Great! > Thanks for testing! Thanks for fixing the issue. Best regards Michael >=20 From owner-freebsd-net@FreeBSD.ORG Tue May 13 07:02:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8FD05A4; Tue, 13 May 2014 07:02:07 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail-n.franken.de", Issuer "Thawte DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 856E42418; Tue, 13 May 2014 07:02:07 +0000 (UTC) Received: from [192.168.1.200] (p54819945.dip0.t-ipconnect.de [84.129.153.69]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 066AC1C104906; Tue, 13 May 2014 09:02:04 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: RX checksum offloading problem From: Michael Tuexen In-Reply-To: <20140513052134.GA1419@michelle.cdnetworks.com> Date: Tue, 13 May 2014 09:02:04 +0200 Content-Transfer-Encoding: 7bit Message-Id: <1E676F4A-50A7-4FB9-8EE8-08B2A4BD54CB@lurchi.franken.de> References: <20140507075612.GA1376@michelle.cdnetworks.com> <36469814-FAC8-4172-A792-487E2AB8ECB9@lurchi.franken.de> <20140507083751.GB1376@michelle.cdnetworks.com> <415C1CB5-3AF9-44E4-943A-74116037980E@lurchi.franken.de> <20140509013556.GA3014@michelle.cdnetworks.com> <20140512013612.GA4085@michelle.cdnetworks.com> <72BEDE7B-C1F1-4EE2-BA68-49AED7209E8A@lurchi.franken.de> <20140513052134.GA1419@michelle.cdnetworks.com> To: pyunyh@gmail.com X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 07:02:07 -0000 On 13 May 2014, at 07:21, Yonghyeon PYUN wrote: > On Mon, May 12, 2014 at 01:22:03PM +0200, Michael Tuexen wrote: >> On 12 May 2014, at 03:36, Yonghyeon PYUN wrote: >> >>> On Fri, May 09, 2014 at 12:46:48PM +0200, Michael Tuexen wrote: >>>> On 09 May 2014, at 03:35, Yonghyeon PYUN wrote: >>>> > > [...] > >>> Oops, sorry. You're right. Probably I was confused with old memory >>> when I worked on that area. I've quickly read IP reassembly code >>> again and as you said, it should work. However it seems there is a >>> checksumming bug here. >>> >>> /* >>> * In order to do checksumming faster we do 'end-around carry' here >>> * (and not in for{} loop), though it implies we are not going to >>> * reassemble more than 64k fragments. >>> */ >>> m->m_pkthdr.csum_data = >>> (m->m_pkthdr.csum_data & 0xffff) + (m->m_pkthdr.csum_data >> 16); >>> >>> I guess the line above didn't account possible carry happened after >>> the computation. Probably it could be rewritten as the following. >>> >>> while (m->m_pkthdr.csum_data & 0xffff0000) >>> m->m_pkthdr.csum_data = (m->m_pkthdr.csum_data & 0xffff) + >>> (m->m_pkthdr.csum_data >> 16); >> I think you are right here. Good catch. Will you fix it? >> > > Done in r265942. Great. Thanks. Michael > From owner-freebsd-net@FreeBSD.ORG Tue May 13 09:46:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D33E0E90 for ; Tue, 13 May 2014 09:46:29 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E08022DF for ; Tue, 13 May 2014 09:46:29 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1Wk9L1-0000Zi-Ef; Tue, 13 May 2014 13:49:31 +0400 Message-ID: <5371E9E7.70400@smartspb.net> Date: Tue, 13 May 2014 13:46:15 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Marcelo Gondim , FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> In-Reply-To: <5371112B.2030209@bsdinfo.com.br> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140512-4, 12.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 09:46:29 -0000 May be this will help? See answer on http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 12.05.2014 22:21, Marcelo Gondim ÐŋÐļŅˆÐĩŅ‚: > Hi Jason, > > Same problem. > > Em 12/05/14 15:02, Jason Hellenthal escreveu: >> Cute. Same this happen when there are paren around the quad ? >> > -- Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > >> On May 12, 2014, at 13:43, Marcelo Gondim wrote: >> >> Hi all, >> >> Today I discovered a likely problem: >> >> # ipfw table 99 add 0.0.0.0/8 >> >> # ipfw table 99 list >> ::/8 0 >> >> Is this correct? IPv6? >> >> # uname -a >> FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #6 >> r265408: Fri May 9 12:00:40 BRT >> 2014root@mail.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM amd64 >> >> Cheers, >> Gondim >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to"freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Tue May 13 10:34:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F687B24 for ; Tue, 13 May 2014 10:34:03 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 022622791 for ; Tue, 13 May 2014 10:34:03 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wk67v-0009nH-Sa; Tue, 13 May 2014 10:23:47 +0400 Message-ID: <5371F4C8.3080501@FreeBSD.org> Date: Tue, 13 May 2014 14:32:40 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Dennis Yusupoff , Marcelo Gondim , FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> In-Reply-To: <5371E9E7.70400@smartspb.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 10:34:03 -0000 On 13.05.2014 13:46, Dennis Yusupoff wrote: > May be this will help? See answer on > http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 I'll try to fix it within a few days. The problem itself happens due to the fact that every CIDR table address is packed into IPv6 address and IPv4 ones are encoded as deprecated IPv6-compatible ones. this leads to the problems with decoding things like 0/X or ::1 > > 12.05.2014 22:21, Marcelo Gondim ÐŋÐļŅˆÐĩŅ‚: >> Hi Jason, >> >> Same problem. >> >> Em 12/05/14 15:02, Jason Hellenthal escreveu: >>> Cute. Same this happen when there are paren around the quad ? >>> >> -- Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN >> >>> On May 12, 2014, at 13:43, Marcelo Gondim wrote: >>> >>> Hi all, >>> >>> Today I discovered a likely problem: >>> >>> # ipfw table 99 add 0.0.0.0/8 >>> >>> # ipfw table 99 list >>> ::/8 0 >>> >>> Is this correct? IPv6? >>> >>> # uname -a >>> FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #6 >>> r265408: Fri May 9 12:00:40 BRT >>> 2014root@mail.xxxxxx.com.br:/usr/obj/usr/src/sys/GONDIM amd64 >>> >>> Cheers, >>> Gondim >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to"freebsd-net-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> From owner-freebsd-net@FreeBSD.ORG Tue May 13 12:06:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44E9EBCF for ; Tue, 13 May 2014 12:06:01 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3E212F10 for ; Tue, 13 May 2014 12:05:59 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1WkBW1-000F6D-HD for freebsd-net@freebsd.org; Tue, 13 May 2014 16:09:01 +0400 Message-ID: <53720AA4.80909@smartspb.net> Date: Tue, 13 May 2014 16:05:56 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> In-Reply-To: <5371F4C8.3080501@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140512-4, 12.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 12:06:01 -0000 I think that universal table for all kind of data (ipv4, ipv6, ports, etc) is a bad idea by design. At least unless you haven't any ability to specify address family on add, to avoid attempts to guess what user meant. Something like "ipfw table X add DEEF.DE ipv6". 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: > On 13.05.2014 13:46, Dennis Yusupoff wrote: >> May be this will help? See answer on >> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 > I'll try to fix it within a few days. > > The problem itself happens due to the fact that every CIDR table > address is packed into IPv6 address and IPv4 ones are encoded as > deprecated IPv6-compatible ones. > this leads to the problems with decoding things like 0/X or ::1 -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Tue May 13 15:42:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E2D5E62A for ; Tue, 13 May 2014 15:42:07 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CB5022F2 for ; Tue, 13 May 2014 15:42:07 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4DFft5p036861 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 13 May 2014 08:41:58 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53723D3E.7030307@freebsd.org> Date: Tue, 13 May 2014 23:41:50 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz>, FreeBSD Net Subject: Re: Best practices with network settings for virtualization References: <5371510E.40302@quip.cz> In-Reply-To: <5371510E.40302@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 15:42:08 -0000 On 5/13/14, 6:54 AM, Miroslav Lachman wrote: > I originaly posted this to virtualization@ list week ago. I didn't > recieved any answer, so maybe this list is better for questions like > the following. > > I would like to ask some really experienced person - what is the > best way to run virtual guests connected to network with public IPs? > > I think many people run unsecure setup with guests with simple > bridged network. > > I know there are many options with tun, bridge, epair, VDE, Open > vSwitch etc., my main concern is the setup of network where each > guest can use only predefined MAC and predefined IP(s). If some > malicious user or malware in guest OS tried to change MAC od IP, I > would like to disallow that or do not allow any offending traffic to > reach outside network or any other guest running on the same machine. > Guests can be VirtualBox, Bhyve or anything else. Assuming you mean virtualization like bhyve and not virtualization like jails, ad that you can use private addresses for the VMs, you can still run each virtual machine inside a VNET jail, then using something like epair you can connect the jails to a central 'router' jail that runs ipfw and enforces what each jail sends out. If you want actual routable addresses on each jail (so that the jail sees the outside workd directly it's a bit more difficult because you can't act as a 'router' in the middle. Maybe others have more ideas. If you need to bridge a bunch of virtual machines so that they have addressable interfaces. you can run bhyve or VB inside a vnet jail as above but each jail would need to do its own enforcing by having its own ipfw, listenning on the virtual interface that is attaching to the bridge. I have not done htis but I'm sure it can be done. you'll need to experiment. just remember that each VNET jail can have it's own firewall and it's own interfaces. real or virtual. > > I really appreciate any help or ideas. > > -- > Miroslav Lachman > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue May 13 17:44:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 77DC5DCB; Tue, 13 May 2014 17:44:22 +0000 (UTC) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3503F2E97; Tue, 13 May 2014 17:44:21 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 7B74528430; Tue, 13 May 2014 19:44:18 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 8AC272842E; Tue, 13 May 2014 19:44:17 +0200 (CEST) Message-ID: <537259F1.7070908@quip.cz> Date: Tue, 13 May 2014 19:44:17 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: Julian Elischer Subject: Re: Best practices with network settings for virtualization References: <5371510E.40302@quip.cz> <53723D3E.7030307@freebsd.org> In-Reply-To: <53723D3E.7030307@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 17:44:22 -0000 Julian Elischer wrote: > On 5/13/14, 6:54 AM, Miroslav Lachman wrote: >> I originaly posted this to virtualization@ list week ago. I didn't >> recieved any answer, so maybe this list is better for questions like >> the following. >> >> I would like to ask some really experienced person - what is the best >> way to run virtual guests connected to network with public IPs? >> >> I think many people run unsecure setup with guests with simple bridged >> network. >> >> I know there are many options with tun, bridge, epair, VDE, Open >> vSwitch etc., my main concern is the setup of network where each guest >> can use only predefined MAC and predefined IP(s). If some malicious >> user or malware in guest OS tried to change MAC od IP, I would like to >> disallow that or do not allow any offending traffic to reach outside >> network or any other guest running on the same machine. >> Guests can be VirtualBox, Bhyve or anything else. > Assuming you mean virtualization like bhyve and not virtualization like > jails, ad that you can use private addresses for the VMs, you can still > run each virtual machine inside a VNET jail, then using something like > epair you can connect the jails to a central 'router' jail that runs > ipfw and enforces what each jail sends out. > > If you want actual routable addresses on each jail (so that the jail > sees the outside workd directly it's a bit more difficult because you > can't act as a 'router' in the middle. Maybe others have more ideas. > > If you need to bridge a bunch of virtual machines so that they have > addressable interfaces. you can run bhyve or VB inside a vnet jail as > above but each jail would need to do its own enforcing by having its own > ipfw, listenning on the virtual interface that is attaching to the > bridge. I have not done htis but I'm sure it can be done. you'll need to > experiment. > just remember that each VNET jail can have it's own firewall and it's > own interfaces. real or virtual. Thank you for your answer. I am mainly interested in to virtualization like Bhyve or VirtualBox with routable addresses in guest instances. So it is limited to some solutions with virtual network switch with IP+MAC ACL capability. But I didn't find any example of this setup on the internet. Are VNET jails of production quality? And can be Bhyve / VirtualBox guest run inside of them? (each guest in separate vnet jail) Miroslav Lachman From owner-freebsd-net@FreeBSD.ORG Wed May 14 02:56:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A685C266 for ; Wed, 14 May 2014 02:56:06 +0000 (UTC) Received: from mail-pb0-x234.google.com (mail-pb0-x234.google.com [IPv6:2607:f8b0:400e:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7996B2BE6 for ; Wed, 14 May 2014 02:56:06 +0000 (UTC) Received: by mail-pb0-f52.google.com with SMTP id rr13so1056419pbb.25 for ; Tue, 13 May 2014 19:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=5tn6xtGCVzTXbJ5CQUGtaA26yc2QDmsvGIoZJVczjBU=; b=NK3Y5n84C+VbYSMq/qy6Q5C9TxhZHm4vpz7bQGRydjy7Be+4LTqG4er0rkypu1kEzf MT0k3dvlBsSMeOOmObyIHNXScXcLXvkS4vTDqf04Me9CDiEfd+8DWsMzzhoPPuyVA0uP aPhuNHKRuzrJJogPhn6ZpjVOYv7oPIhCFELY6NOJov22Ycnca6RYs8H3GNNPO19+81ks SuHy+c8Z2BjmuPWRRKVHBfffkdf3791o+iOflBXhUcP2tWasl/SM1Go1yFkvmB2TqFtS vdjWV+sDuYI5sscCfFTDMEtvJrNyZrUw47V7h/67AOOt2sQOM2bPODwCItnvG3iIN9uc KQbA== X-Received: by 10.66.164.135 with SMTP id yq7mr1001436pab.126.1400036166013; Tue, 13 May 2014 19:56:06 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id g6sm1867910pat.2.2014.05.13.19.56.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 13 May 2014 19:56:05 -0700 (PDT) Message-ID: <5372DB3E.7000506@gmail.com> Date: Wed, 14 May 2014 10:55:58 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Christopher Forgeron Subject: Re: syslogd:sendto: no buffer available on 10-stable References: <53310C57.5070102@gmail.com> <533240B4.50809@gmail.com> In-Reply-To: <533240B4.50809@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 May 2014 02:56:06 -0000 The issue is still remained. Does it related to flowtable ? # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 109, 1, 109, 0, 0 UMA Zones: 1664, 0, 109, 1, 109, 0, 0 UMA Slabs: 80, 0, 48753, 1147,795180888, 0, 0 UMA RCntSlabs: 88, 0, 94781, 34, 94781, 0, 0 UMA Hash: 256, 0, 2, 13, 9, 0, 0 4 Bucket: 32, 0, 1, 1374, 16673, 0, 0 6 Bucket: 48, 0, 0, 1328, 68823, 0, 0 8 Bucket: 64, 0, 0, 1550, 111011, 0, 0 12 Bucket: 96, 0, 27, 1367, 635428, 0, 0 16 Bucket: 128, 0, 2985, 1882,63426594, 11, 0 32 Bucket: 256, 0, 7782, 1938,74452218, 46, 0 64 Bucket: 512, 0, 3027, 2917,23978224,4610, 0 128 Bucket: 1024, 0, 10672, 5868,47846405605, 742, 0 vmem btag: 56, 0, 26858, 36687,340295492, 895, 0 VM OBJECT: 256, 0, 210353, 17497,367582461, 0, 0 RADIX NODE: 144, 0, 347266, 41642,796163752, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 6, 273, 6, 0, 0 MAP ENTRY: 128, 0, 1638, 2113,51340019, 0, 0 VMSPACE: 448, 0, 43, 308, 1040861, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 325, 0, 325, 0, 0 16: 16, 0, 16452, 38517,35980070234, 0, 0 32: 32, 0, 40209, 27166,3888808650, 0, 0 64: 64, 0, 17213, 68719,68691410126, 0, 0 128: 128, 0, 10520, 157128,476747943, 0, 0 256: 256, 0, 7914, 87501,1264824107, 0, 0 512: 512, 0, 22137, 93663,492467476, 0, 0 1024: 1024, 0, 66, 262, 889383, 0, 0 2048: 2048, 0, 130, 796,791425021, 0, 0 4096: 4096, 0, 393, 939,207284522, 0, 0 SLEEPQUEUE: 80, 0, 289, 734, 289, 0, 0 64 pcpu: 8, 0, 132323, 669, 132323, 0, 0 Files: 80, 0, 5483, 21667,14367132928, 0, 0 rl_entry: 40, 0, 157, 843, 157, 0, 0 TURNSTILE: 136, 0, 289, 271, 289, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 ertt: 72, 0, 8469, 38831,9821340346, 0, 0 ertt_txseginfo: 40, 0, 16571, 159929,95003293642, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 68, 112, 1040885, 0, 0 THREAD: 1168, 0, 259, 29, 273, 0, 0 cpuset: 72, 0, 106, 169, 114, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 6518985, 43732, 45393,199965865255, 0, 0 mbuf: 256, 6518985, 8890, 45745,383256720839, 0, 0 mbuf_cluster: 2048, 524288, 89125, 37, 89125, 0, 0 mbuf_jumbo_page: 4096, 509294, 8444, 41756,33590222643, 0, 0 mbuf_jumbo_9k: 9216, 150902, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84882, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 7, 13113,1955534264, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 180, 570, 2955, 0, 0 ttyoutq: 256, 0, 95, 640, 1531, 0, 0 ata_request: 336, 0, 0, 88, 148, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 209164, 41156,1494569676, 0, 0 VNODEPOLL: 112, 0, 2, 278, 2, 0, 0 BUF TRIE: 144, 0, 7226, 98155,399747594, 0, 0 NAMEI: 1024, 0, 0, 340,1778005738, 0, 0 S VFS Cache: 108, 0, 104334, 130446,1097507303, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 36, 63336, 8079613, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 57266, 934,53319431, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 Mountpoints: 816, 0, 7, 33, 7, 0, 0 pipe: 744, 0, 6, 164, 836922, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0 AIOCB: 480, 0, 0, 0, 0, 0, 0 AIOL: 128, 0, 0, 0, 0, 0, 0 AIOLIO: 272, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 168, 1477, 4629801, 0, 0 itimer: 352, 0, 1, 32, 1, 0, 0 KNOTE: 128, 0, 5245, 20020,122354662379, 0, 0 flows: 40, 696322, 696322, 0, 933075,93243583709, 0 socket: 696, 523485, 11682, 57448,9822208423, 0, 0 ipq: 56, 16401, 0, 0, 0, 0, 0 udp_inpcb: 392, 523490, 53, 327, 615958, 0, 0 udpcb: 16, 523586, 53, 1453, 615958, 0, 0 tcp_inpcb: 392, 523490, 17842, 60488,9821340337, 0, 0 tcpcb: 1024, 523488, 8463, 37937,9821340336, 0, 0 tcptw: 88, 32805, 9373, 23432,1622657168,320470, 0 syncache: 160, 409600, 120, 264980,6085298712, 0, 0 hostcache: 136, 0, 0, 0, 0, 0, 0 tcpreass: 40, 32800, 27, 1773,118471998, 0, 0 sackhole: 32, 0, 117, 1758,1200394757, 0, 0 sctp_ep: 1400, 523486, 0, 0, 0, 0, 0 sctp_asoc: 2344, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 830, 42, 0, 0 sctp_raddr: 720, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 ripcb: 392, 523490, 0, 240, 237, 0, 0 unpcb: 240, 523488, 38, 698, 251845, 0, 0 rtentry: 200, 0, 768, 212, 832, 0, 0 IPFW dynamic rule: 120, 4125, 0, 0, 0, 0, 0 divcb: 392, 523490, 0, 0, 0, 0, 0 selfd: 56, 0, 245, 1388,4214189053, 0, 0 SWAPMETA: 288, 2037191, 0, 0, 0, 0, 0 FFS inode: 168, 0, 209130, 41478,1494565024, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 209130, 41490,1494564173, 0, 0 # uname -a FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r264098: Fri Apr 4 11:14:53 CST 2014 root@sq-l1-n2:/usr/obj/usr/src/sys/10-stable-r264098 amd64 Regards Simon 䚎 14-3-26 10:51, k simon 写道: > Thanks, Christopher. But I think my problem may does not related to > TSO issue. I have tried disable tso with "ifconfig igb(x) -tso" and > ovserved with "netstat -ihw 1", and found "oErrs" does not disappeared. > > Regards > Simon > > 䚎 14-3-25 22:08, Christopher Forgeron 写道: >> Hi Simon, >> >> Try checking out the "9.2 ixgbe tx queue hang' thread here, and see if >> it applies to you. >> >> >> On Tue, Mar 25, 2014 at 1:55 AM, k simon > > wrote: >> >> Hi,Lists: >> I have got lots of "no buffer available" on 10-stable with igb >> nic. >> But em and bce works well. And I tried force igb to 4 or 8 queues and >> set hw.igb.rx_process_limit="-1", but have nothing helped. >> >> >> Regards >> Simon >> >> >> >> # uname -a >> FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r262469: Tue >> Feb 25 >> 13:27:11 CST 2014 >> root@sq-l1-n2:/usr/obj/usr/src/sys/stable-10-262458 amd64 >> >> >> # netstat -mb >> 19126/73289/92415 mbufs in use >> (current/cache/total) >> 13289/46841/60130/524288 mbuf clusters in use >> (current/cache/total/max) >> 13289/46812 mbuf+clusters out of packet secondary zone in use >> (current/cache) >> 5638/22605/28243/262144 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/77672 9k jumbo clusters in use (current/cache/total/max) >> 0/0/0/43690 16k jumbo clusters in use (current/cache/total/max) >> 53914K/202424K/256338K bytes allocated to network >> (current/cache/total) >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> >> >> # netstat -di >> Name Mtu Network Address Ipkts Ierrs Idrop >> Opkts Oerrs Coll Drop >> igb0 1500 00:1b:21:70:5f:80 17212101113 1355809 0 >> 19612978862 0 0 >> igb1 1500 00:1b:21:70:5f:81 76601294282 >> 81162751 0 >> 74236432686 0 0 >> lo0 16384 20532742636 0 0 >> 20522475797 0 0 >> lo0 - your-net localhost 2736994243 - - >> 20520227166 - - >> >> >> # sysctl hw.igb >> hw.igb.rxd: 2048 >> hw.igb.txd: 2048 >> hw.igb.enable_aim: 1 >> hw.igb.enable_msix: 1 >> hw.igb.max_interrupt_rate: 12000 >> hw.igb.buf_ring_size: 4096 >> hw.igb.header_split: 0 >> hw.igb.num_queues: 1 >> hw.igb.rx_process_limit: 1000 >> >> >> # sysctl dev.igb.1 >> dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 >> dev.igb.1.%driver: igb >> dev.igb.1.%location: slot=0 function=1 >> dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 >> subdevice=0xa04c class=0x020000 >> dev.igb.1.%parent: pci8 >> dev.igb.1.nvm: -1 >> dev.igb.1.enable_aim: 1 >> dev.igb.1.fc: 0 >> dev.igb.1.rx_processing_limit: 4096 >> dev.igb.1.link_irq: 3 >> dev.igb.1.dropped: 0 >> dev.igb.1.tx_dma_fail: 0 >> dev.igb.1.rx_overruns: 0 >> dev.igb.1.watchdog_timeouts: 0 >> dev.igb.1.device_control: 1086325313 >> dev.igb.1.rx_control: 67141634 >> dev.igb.1.interrupt_mask: 4 >> dev.igb.1.extended_int_mask: 2147483651 >> dev.igb.1.tx_buf_alloc: 0 >> dev.igb.1.rx_buf_alloc: 0 >> dev.igb.1.fc_high_water: 58976 >> dev.igb.1.fc_low_water: 58960 >> dev.igb.1.queue0.no_desc_avail: 10874 >> dev.igb.1.queue0.tx_packets: 74509997338 >> dev.igb.1.queue0.rx_packets: 76837720630 >> dev.igb.1.queue0.rx_bytes: 35589607860237 >> dev.igb.1.queue0.lro_queued: 0 >> dev.igb.1.queue0.lro_flushed: 0 >> dev.igb.1.mac_stats.excess_coll: 0 >> dev.igb.1.mac_stats.single_coll: 0 >> dev.igb.1.mac_stats.multiple_coll: 0 >> dev.igb.1.mac_stats.late_coll: 0 >> dev.igb.1.mac_stats.collision_count: 0 >> dev.igb.1.mac_stats.symbol_errors: 0 >> dev.igb.1.mac_stats.sequence_errors: 0 >> dev.igb.1.mac_stats.defer_count: 0 >> dev.igb.1.mac_stats.missed_packets: 81162751 >> dev.igb.1.mac_stats.recv_no_buff: 176691324 >> dev.igb.1.mac_stats.recv_undersize: 0 >> dev.igb.1.mac_stats.recv_fragmented: 0 >> dev.igb.1.mac_stats.recv_oversize: 0 >> dev.igb.1.mac_stats.recv_jabber: 0 >> dev.igb.1.mac_stats.recv_errs: 0 >> dev.igb.1.mac_stats.crc_errs: 0 >> dev.igb.1.mac_stats.alignment_errs: 0 >> dev.igb.1.mac_stats.coll_ext_errs: 0 >> dev.igb.1.mac_stats.xon_recvd: 0 >> dev.igb.1.mac_stats.xon_txd: 0 >> dev.igb.1.mac_stats.xoff_recvd: 0 >> dev.igb.1.mac_stats.xoff_txd: 0 >> dev.igb.1.mac_stats.total_pkts_recvd: 76925709917 >> dev.igb.1.mac_stats.good_pkts_recvd: 76837704301 >> dev.igb.1.mac_stats.bcast_pkts_recvd: 49174716 >> dev.igb.1.mac_stats.mcast_pkts_recvd: 282670 >> dev.igb.1.mac_stats.rx_frames_64: 31057121854 >> dev.igb.1.mac_stats.rx_frames_65_127: 19996324498 >> dev.igb.1.mac_stats.rx_frames_128_255: 1171960837 >> dev.igb.1.mac_stats.rx_frames_256_511: 2295894674 >> dev.igb.1.mac_stats.rx_frames_512_1023: 2026241811 >> dev.igb.1.mac_stats.rx_frames_1024_1522: 20290160627 >> dev.igb.1.mac_stats.good_octets_recvd: 36204302378783 >> dev.igb.1.mac_stats.good_octets_txd: 59038220741656 >> dev.igb.1.mac_stats.total_pkts_txd: 90973292365 >> dev.igb.1.mac_stats.good_pkts_txd: 90973292359 >> dev.igb.1.mac_stats.bcast_pkts_txd: 2408182 >> dev.igb.1.mac_stats.mcast_pkts_txd: 246782 >> dev.igb.1.mac_stats.tx_frames_64: 24604769631 >> dev.igb.1.mac_stats.tx_frames_65_127: 21373976133 >> dev.igb.1.mac_stats.tx_frames_128_255: 3047554762 >> dev.igb.1.mac_stats.tx_frames_256_511: 3751820879 >> dev.igb.1.mac_stats.tx_frames_512_1023: 3481550512 >> dev.igb.1.mac_stats.tx_frames_1024_1522: 34713620448 >> dev.igb.1.mac_stats.tso_txd: 9549335102 >> dev.igb.1.mac_stats.tso_ctx_fail: 0 >> dev.igb.1.interrupts.asserts: 16929843739 >> dev.igb.1.interrupts.rx_pkt_timer: 76834839132 >> dev.igb.1.interrupts.rx_abs_timer: 0 >> dev.igb.1.interrupts.tx_pkt_timer: 0 >> dev.igb.1.interrupts.tx_abs_timer: 76835042438 >> dev.igb.1.interrupts.tx_queue_empty: 90970561829 >> dev.igb.1.interrupts.tx_queue_min_thresh: 0 >> dev.igb.1.interrupts.rx_desc_min_thresh: 0 >> dev.igb.1.interrupts.rx_overrun: 0 >> dev.igb.1.host.breaker_tx_pkt: 0 >> dev.igb.1.host.host_tx_pkt_discard: 0 >> dev.igb.1.host.rx_pkt: 2865171 >> dev.igb.1.host.breaker_rx_pkts: 0 >> dev.igb.1.host.breaker_rx_pkt_drop: 0 >> dev.igb.1.host.tx_good_pkt: 2730536 >> dev.igb.1.host.breaker_tx_pkt_drop: 0 >> dev.igb.1.host.rx_good_bytes: 36204307257585 >> dev.igb.1.host.tx_good_bytes: 59038220743158 >> dev.igb.1.host.length_errors: 0 >> dev.igb.1.host.serdes_violation_pkt: 0 >> dev.igb.1.host.header_redir_missed: 0 >> >> >> >> # cat /var/run/dmesg.boot |grep igb >> igb0: mem >> 0xd5ee0000-0xd5efffff,0xd5800000-0xd5bfffff,0xd5edc000-0xd5edffff >> irq 18 >> at device 0.0 on pci8 >> igb0: Using MSIX interrupts with 2 vectors >> igb0: Ethernet address: 00:1b:21:70:5f:80 >> igb1: mem >> 0xd5ea0000-0xd5ebffff,0xd5400000-0xd57fffff,0xd5ed8000-0xd5edbfff >> irq 19 >> at device 0.1 on pci8 >> igb1: Using MSIX interrupts with 2 vectors >> igb1: Ethernet address: 00:1b:21:70:5f:81 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org >> " >> >> From owner-freebsd-net@FreeBSD.ORG Wed May 14 06:15:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D2B0CB2 for ; Wed, 14 May 2014 06:15:30 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E45372D52 for ; Wed, 14 May 2014 06:15:29 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WkSTL-0003Ug-1b for freebsd-net@freebsd.org; Tue, 13 May 2014 23:15:23 -0700 Date: Tue, 13 May 2014 23:15:23 -0700 (PDT) From: kariz To: freebsd-net@freebsd.org Message-ID: <1400048123005-5912251.post@n5.nabble.com> In-Reply-To: References: Subject: Re: running netmap-ipfw with real NICs MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 May 2014 06:15:30 -0000 Hi Luigi, Again, My receiver can not receive packets. dmesg messages are: [ 7612.954837] 081.363279 [ 753] generic_netmap_dtor Restored native NA (null) [ 7613.054226] 081.462874 [ 753] generic_netmap_dtor Restored native NA (null) Please help me to use netmap-ipfw. Thanks in advance. -- View this message in context: http://freebsd.1045724.n5.nabble.com/running-netmap-ipfw-with-real-NICs-tp5906992p5912251.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Wed May 14 10:26:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3A05919 for ; Wed, 14 May 2014 10:26:11 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 2875C211C for ; Wed, 14 May 2014 10:26:10 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id 141AB6B9882 for ; Wed, 14 May 2014 12:05:36 +0200 (CEST) Received: from soth.ventu (adsl-ull-47-174.41-151.net24.it [151.41.174.47]) (authenticated bits=0) by ns2.biolchim.it (8.14.8/8.14.8) with ESMTP id s4EA5Wgg084761 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 14 May 2014 12:05:33 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-47-174.41-151.net24.it [151.41.174.47] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.8/8.14.7) with ESMTP id s4EA5QUo067983; Wed, 14 May 2014 12:05:26 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53733FE6.4060605@netfence.it> Date: Wed, 14 May 2014 12:05:26 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Server with multiple public IP References: <535E1842.20905@netfence.it> <535E1C66.6090004@talk2dom.com> In-Reply-To: <535E1C66.6090004@talk2dom.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Wed, 14 May 2014 12:05:33 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 Cc: dom@talk2dom.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 May 2014 10:26:11 -0000 On 04/28/14 11:16, Dominic Froud wrote: > On 28/04/2014 09:58, Andrea Venturoli wrote: >> I've got a server which has two (or more) interfaces with public IPs. >> >> Let's say, as an example (with fictional IPs): >> ifconfig_vlan1="inet 1.0.0.2 netmask 255.255.255.248..." >> ifconfig_vlan2="inet 2.0.0.2 netmask 255.255.255.248..." >> >> Of course, I can only have a default route, let's say 1.0.0.1. >> This is fine for outgoing traffic and for incoming connections on vlan1. >> However, when someone from the outside connects to 2.0.0.2, reply >> packets still go out through 1.0.0.1 (on vlan1), but they should go >> through vlan2 to 2.0.0.1 > > You want source-based routing. > > I have this situation and I used pf(4) to do it with a rule like: > > pass out quick route-to ( vlan2 ) from 2.0.0.0/29 to any no state > > As a variation you can give an optional next-hop address if you have a > static router for that vlan, e.g. if your router is 2.0.0.1: > > pass out quick route-to ( vlan2 2.0.0.1 ) from 2.0.0.0/29 to any no state > > Also, you can run pf and ipfw at the same time! > > Hope this helps, I ended up using this solution... so far so good (and so easy). Thanks a lot. bye av. From owner-freebsd-net@FreeBSD.ORG Wed May 14 17:07:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7BC7487 for ; Wed, 14 May 2014 17:07:50 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 717B62435 for ; Wed, 14 May 2014 17:07:50 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4EH7cUm040945 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 14 May 2014 10:07:42 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5373A2D5.4050303@freebsd.org> Date: Thu, 15 May 2014 01:07:33 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> Subject: Re: Best practices with network settings for virtualization References: <5371510E.40302@quip.cz> <53723D3E.7030307@freebsd.org> <537259F1.7070908@quip.cz> In-Reply-To: <537259F1.7070908@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 May 2014 17:07:50 -0000 On 5/14/14, 1:44 AM, Miroslav Lachman wrote: > Julian Elischer wrote: >> On 5/13/14, 6:54 AM, Miroslav Lachman wrote: >>> I originaly posted this to virtualization@ list week ago. I didn't >>> recieved any answer, so maybe this list is better for questions like >>> the following. >>> >>> I would like to ask some really experienced person - what is the best >>> way to run virtual guests connected to network with public IPs? >>> >>> I think many people run unsecure setup with guests with simple >>> bridged >>> network. >>> >>> I know there are many options with tun, bridge, epair, VDE, Open >>> vSwitch etc., my main concern is the setup of network where each >>> guest >>> can use only predefined MAC and predefined IP(s). If some malicious >>> user or malware in guest OS tried to change MAC od IP, I would >>> like to >>> disallow that or do not allow any offending traffic to reach outside >>> network or any other guest running on the same machine. >>> Guests can be VirtualBox, Bhyve or anything else. >> Assuming you mean virtualization like bhyve and not virtualization >> like >> jails, ad that you can use private addresses for the VMs, you can >> still >> run each virtual machine inside a VNET jail, then using something like >> epair you can connect the jails to a central 'router' jail that runs >> ipfw and enforces what each jail sends out. >> >> If you want actual routable addresses on each jail (so that the jail >> sees the outside workd directly it's a bit more difficult because you >> can't act as a 'router' in the middle. Maybe others have more ideas. >> >> If you need to bridge a bunch of virtual machines so that they have >> addressable interfaces. you can run bhyve or VB inside a vnet jail as >> above but each jail would need to do its own enforcing by having >> its own >> ipfw, listenning on the virtual interface that is attaching to the >> bridge. I have not done htis but I'm sure it can be done. you'll >> need to >> experiment. >> just remember that each VNET jail can have it's own firewall and it's >> own interfaces. real or virtual. > > Thank you for your answer. > I am mainly interested in to virtualization like Bhyve or VirtualBox > with routable addresses in guest instances. So it is limited to some > solutions with virtual network switch with IP+MAC ACL capability. > But I didn't find any example of this setup on the internet. > > Are VNET jails of production quality? And can be Bhyve / VirtualBox > guest run inside of them? (each guest in separate vnet jail) > > Miroslav Lachman > there are some incomplete features, but Bhyve and vbox are likley to use just a small subset of functionality of the stack so I'm guessing it would be stable. From owner-freebsd-net@FreeBSD.ORG Thu May 15 04:43:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16DAE833; Thu, 15 May 2014 04:43:40 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B86F32FAE; Thu, 15 May 2014 04:43:39 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id w7so917205qcr.34 for ; Wed, 14 May 2014 21:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=jhKEnfy+S3LIg3ExmQD+qvQG9QGh1+sT6wjto8q98/8=; b=ncf+edh1PaJ/r5WdUIrf7jqbfol5+jv+HsfNotJ/ogctKh/BN0EvL1HIX9eGeK+1EZ JaWBv1Yu7DADTixbB1usDWOEj3LqKw4IciZNA+pgd0N4K0JFmNeyqhB8bGmBR2d+CGVZ nSwIQHnpyw2QEzYtTVN4ecbS7mnExlY8gXzge/JcKPlkF33OBKyXNScdF3tiI6aFPdbv MJNAR03Llc8vRBo9LJCUoEEJ1Vfw/LD7Mx7SaARD6Fk3N3JwBIPN9m+vJUJnI9IDT0IO hV1Rr1qBtfFKf7+ECWBxrY541gveYsOv230wMn9EMbeoI60DaVaPPDybOoYWef9dJHu4 v9NA== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr9666950qaa.76.1400129018305; Wed, 14 May 2014 21:43:38 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Wed, 14 May 2014 21:43:38 -0700 (PDT) Date: Wed, 14 May 2014 21:43:38 -0700 X-Google-Sender-Auth: 615EBagYUG4pOjyHhtu_W3pPu-E Message-ID: Subject: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 04:43:40 -0000 Hi, here's a completely untested patch for discussion. I'm emailing it out mostly as a "is this a good idea" patch rather than a "it should just be committed" patch. The RSS stuff from Robert maps connections to pcbgroups based on the RSS hash, but it doesn't map the TCP timers the same way. So by default they're all on swi0 and the per CPU timers don't take the hash type or correctly choose the CPU. This patch: http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff does a few things: * it stores the hashtype in the inp as well as the flowid; * the rss code grows a new method that maps the flowid/hashtype to a cpuid * .. and the existing mbuf to cpuid method now uses this; * the tcp timer code now has an inline function that knows about RSS and defaults to the existing way of doing things if RSS isn't enabled. There's still a bunch of work left before all the lock contention compartmentalization and balancing of RSS is realised. I'm just chewing off the little corner bits that are easy to get done now. So, any comments? I'll give it a proper whirl on some 10G hardware in a few days but I thought I'd at least get the general idea out there for comment. Thanks, -a From owner-freebsd-net@FreeBSD.ORG Thu May 15 05:26:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2EC5E1C; Thu, 15 May 2014 05:26:27 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E4EF226B; Thu, 15 May 2014 05:26:27 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so931392qgf.35 for ; Wed, 14 May 2014 22:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=CpDsdyDMfcRI9cGao7L4svj6N3YB5E4hD50EtYZNPBE=; b=zH7gVi035I+gHEmEMJgaflF690C8o3trLsbqTY4yJENwWOL2UeR1Hzr2ti5pfh3cMs 6GO05pP+fEGqd3ONZTKbFmjO5WQfqIfT4k59yPvlmEDiM8zIDUmbN+9XP+MPUHx2CynQ KyIw305O1mSY7nvNo6MGOp0URzAKQyDDUrUxPsnZ5Uz4qAUtNjsLMXUnB3JM/gnxY9+p xoI3tgfySuU4fB4kuDRaaVYdsloRiTDo8cCFtYlxQXm1mXt8c7nCg01jBt+c8nrl+rTl jRRoO9IqQ9p/ONgNWqUzJyZgrOVIJULXuIYBZRLGnE3zUGrPLKvEGylHM942ujeyABeV p2nQ== MIME-Version: 1.0 X-Received: by 10.224.51.2 with SMTP id b2mr9844840qag.49.1400131586528; Wed, 14 May 2014 22:26:26 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Wed, 14 May 2014 22:26:26 -0700 (PDT) In-Reply-To: References: Date: Wed, 14 May 2014 22:26:26 -0700 X-Google-Sender-Auth: g60kka5dSxI-4RNDAXWB8t4jZCs Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 05:26:27 -0000 .. and i've done a little more than no testing, so far so good. http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-2.diff This adds IP_FLOWID, IP_FLOWTYPE and IP_RSSCPUID to fetch the socket flowid, flowtype and cpuid from the inp. It's mostly for debugging for now. Thanks, -a On 14 May 2014 21:43, Adrian Chadd wrote: > Hi, > > here's a completely untested patch for discussion. I'm emailing it out > mostly as a "is this a good idea" patch rather than a "it should just > be committed" patch. > > The RSS stuff from Robert maps connections to pcbgroups based on the > RSS hash, but it doesn't map the TCP timers the same way. So by > default they're all on swi0 and the per CPU timers don't take the hash > type or correctly choose the CPU. > > This patch: > > http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff > > does a few things: > > * it stores the hashtype in the inp as well as the flowid; > * the rss code grows a new method that maps the flowid/hashtype to a cpuid > * .. and the existing mbuf to cpuid method now uses this; > * the tcp timer code now has an inline function that knows about RSS > and defaults to the existing way of doing things if RSS isn't enabled. > > There's still a bunch of work left before all the lock contention > compartmentalization and balancing of RSS is realised. I'm just > chewing off the little corner bits that are easy to get done now. > > So, any comments? I'll give it a proper whirl on some 10G hardware in > a few days but I thought I'd at least get the general idea out there > for comment. > > Thanks, > > > > > -a From owner-freebsd-net@FreeBSD.ORG Fri May 16 03:07:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B337126 for ; Fri, 16 May 2014 03:07:08 +0000 (UTC) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C6462F0C for ; Fri, 16 May 2014 03:07:07 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id r20so246815wiv.13 for ; Thu, 15 May 2014 20:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=XpF6E4Y/weB5NwBqyjOzdmB1mZHXNEM0KP0FO4xGlR4=; b=qeGSdAZQYQXatUV9D0V0wxadjfhfV6/52IPGDYLy3byXxzb+bkE+L/P7HNIOq+tuZf uKWT+erUoFHX1ILe6tarSRphEONMFF45TkNBAmzQOqqF5wlNfU2qDxjTY47gMdl2T3B4 9/mcZ27tef2XU1MAJ4Ke/T9A9Ucy+cP08bjLWJ7Zk6iKTIEWZ/AbXovCkfFg43fleXAz J1Xu122A6oVszqDFqmGBZVDbFibrfT9+CSUx2aWtnm/WsScnJSAl9w+gaLbc48FrZqDp eR9GJeYmxKxwpeSQ17yvNxmrhuoZumXkBu/2bwrvG4Wq/lxP+mfPaUCRBM6WwRiyrCmr ZkRg== MIME-Version: 1.0 X-Received: by 10.180.101.6 with SMTP id fc6mr18310198wib.59.1400209625957; Thu, 15 May 2014 20:07:05 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Thu, 15 May 2014 20:07:05 -0700 (PDT) Reply-To: araujo@FreeBSD.org Date: Fri, 16 May 2014 11:07:05 +0800 Message-ID: Subject: PR: 177571 [ixgbe] Cannot select media type. From: Marcelo Araujo To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 03:07:08 -0000 Hello guys, Is there anyone besides jfv@ that could take a look on it. The change looks like be so trivial, and the problem in somehow is so annoying. PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/177571 Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Fri May 16 14:54:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 939F3C0D for ; Fri, 16 May 2014 14:54:40 +0000 (UTC) Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52E5E2799 for ; Fri, 16 May 2014 14:54:40 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id oy12so3175845veb.24 for ; Fri, 16 May 2014 07:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=QBfJgcxwT8w0DRyzDTpCDtQOTZf8RmJ69+wrEifjcXM=; b=ZjC3DIch21HskVOx+yWbvSw0mAsvtzhHLJOZziZPmXF9/zOqcZKW37BTBs/mTDIDPc wdbU5tjR4MKAiphFyPBMUhe/vfug5/I/P1LVy3YqhhBPplk7xWurfxEplUO+Wx4mGmga 45lnH/8mxLdwWwypx+uYLCZs1c/XeDKdb4W/H96FuGdOgmdUFoF40AsCdbFGjIvauXDC Pe7vfA32j+w07rPCXkXOojkPJKO4kYIkMmRnt24GbKnfuwKfFvqnQbGYK7zOEIBAivfV O6q/S32pR3FkDTTqyD6/RXxlZMD65lZ9pHfLZXM9OLC6NsVbwyzVg271iCM4ZzIDd86n htSA== MIME-Version: 1.0 X-Received: by 10.52.250.4 with SMTP id yy4mr846747vdc.56.1400252079433; Fri, 16 May 2014 07:54:39 -0700 (PDT) Received: by 10.52.98.130 with HTTP; Fri, 16 May 2014 07:54:39 -0700 (PDT) In-Reply-To: References: Date: Fri, 16 May 2014 07:54:39 -0700 Message-ID: Subject: Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF From: Nick Rogers To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 14:54:40 -0000 On Fri, May 2, 2014 at 1:49 PM, Nick Rogers wrote: > Hello, > > I am hoping someone can help me debug a kernel panic I have been experiencing > on one of my production systems. The system is a PF+ALTQ firewall/gateway with > about 1k simultaneous devices behind it at any given time, pushing no more > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it > with kgdb, but am still at a loss. > > I have completely replaced the hardware to rule out issues with > disk/memory/etc, and it appears to be a kernel issue, likely with e1000 driver > and/or PF. > > The panic seems to happen during times of heavier use, but the frequency is > not very predictable (anywhere from a few times a day to a once a week), so I > feel like its some kind of strange traffic pattern that I am unable to > pinpoint. > > I am using a slightly custom kernel based on GENERIC, mainly to bring in ALTQ > and some other options. > > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000 > driver, although I do not believe that should affect em(4). > > Hoping someone can be of assistance in debugging this problem. I am willing to > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if that > is advisable. Unfortunately I cannot try 10-STABLE. There is now a PR for this. It appears the issue may be a bug in 9.x PF based on the responses to the PR thus far. http://www.freebsd.org/cgi/query-pr.cgi?pr=189741 > > Thanks, > > -Nick Rogers > > > uname -v > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014 > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS > > > > Here is the kernel conf... > .............................................................................. > include GENERIC > > ident RGNETS > > makeoptions MODULES_OVERRIDE="" > > options DEVICE_POLLING > > device crypto > device cryptodev > > options VLAN_ARRAY > > device amdtemp > > # PF > device pf > device pflog > options ALTQ > options ALTQ_CBQ # Class Bases Queuing (CBQ) > options ALTQ_RED # Random Early Detection (RED) > options ALTQ_RIO # RED In/Out > options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) > options ALTQ_PRIQ # Priority Queuing (PRIQ) > options ALTQ_NOPCC # Required for SMP build > > # PPPoE > options NETGRAPH > options NETGRAPH_ETHER > options NETGRAPH_PPPOE > options NETGRAPH_SOCKET > > # IPsec > device enc > options IPSEC > options IPSEC_FILTERTUNNEL > options IPSEC_NAT_T > .............................................................................. > > > The crash dump.... > .............................................................................. > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 5; apic id = 05 > fault virtual address = 0x10 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8033d350 > stack pointer = 0x28:0xffffff83545384b0 > frame pointer = 0x28:0xffffff83545384c0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (irq262: em2:rx 0) > trap number = 12 > panic: page fault > cpuid = 5 > KDB: stack backtrace: > #0 0xffffffff80956836 at kdb_backtrace+0x66 > #1 0xffffffff8091c40e at panic+0x1ce > #2 0xffffffff80d31e70 at trap_fatal+0x290 > #3 0xffffffff80d321d1 at trap_pfault+0x211 > #4 0xffffffff80d327d3 at trap+0x363 > #5 0xffffffff80d1b9d3 at calltrap+0x8 > #6 0xffffffff8034872d at pf_test_rule+0x17ed > #7 0xffffffff8034ba12 at pf_test+0x1032 > #8 0xffffffff8035112b at pf_check_in+0x2b > #9 0xffffffff809e952e at pfil_run_hooks+0x9e > #10 0xffffffff80a5286a at ip_input+0x2ea > #11 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #12 0xffffffff809df93d at ether_demux+0x14d > #13 0xffffffff809dfc1e at ether_nh_input+0x1fe > #14 0xffffffff809e8858 at netisr_dispatch_src+0x218 > #15 0xffffffff809df85f at ether_demux+0x6f > #16 0xffffffff809dfc1e at ether_nh_input+0x1fe > #17 0xffffffff809e8858 at netisr_dispatch_src+0x218 > Uptime: 17d7h20m59s > Dumping 2932 out of 12256 MB: (CTRL-C to abort) > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/aio.ko...Reading symbols from > /boot/kernel/aio.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/aio.ko > Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from > /boot/kernel/coretemp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/coretemp.ko > Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from > /boot/kernel/cc_htcp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/cc_htcp.ko > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > 234 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) list *0xffffffff8033d350 > 0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512). > 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af) > 508 { > 509 switch (af) { > 510 #ifdef INET > 511 case AF_INET: > 512 dst->addr32[0] = src->addr32[0]; > 513 break; > 514 #endif /* INET */ > 515 case AF_INET6: > 516 dst->addr32[0] = src->addr32[0]; > (kgdb) backtrace > #0 doadump (textdump=Variable "textdump" is not available. > ) at pcpu.h:234 > #1 0xffffffff8091bee6 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:454 > #2 0xffffffff8091c3e7 in panic (fmt=0x1
) > at /usr/src/sys/kern/kern_shutdown.c:642 > #3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is > not available. > ) at /usr/src/sys/amd64/amd64/trap.c:878 > #4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794 > #5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at > /usr/src/sys/amd64/amd64/trap.c:456 > #6 0xffffffff80d1b9d3 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:232 > #7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8, > src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522 > #8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788, > sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100, > m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e, > pd=0xffffff83545386c0, am=0xffffff8354538790, > rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at > /usr/src/sys/contrib/pf/net/pf.c:3900 > #9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not available. > ) at /usr/src/sys/contrib/pf/net/pf.c:6776 > #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available. > ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131 > #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not available. > ) at /usr/src/sys/net/pfil.c:82 > #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at > /usr/src/sys/netinet/ip_input.c:510 > #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943 > #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800, > m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852 > #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available. > ) at /usr/src/sys/net/if_ethersubr.c:762 > #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9, > source=Variable "source" is not available. > ) at /usr/src/sys/net/netisr.c:1013 > #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2, > done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525 > #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available. > ) at /usr/src/sys/dev/e1000/if_em.c:1593 > #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p" > is not available. > ) at /usr/src/sys/kern/kern_intr.c:1272 > #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at > /usr/src/sys/kern/kern_intr.c:1285 > #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270 > , arg=0xfffffe000730c300, frame=0xffffff8354538c40) at > /usr/src/sys/kern/kern_fork.c:996 > #25 0xffffffff80d1befe in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #26 0x0000000000000000 in ?? () > #27 0x0000000000000000 in ?? () > #28 0x0000000000000001 in ?? () > #29 0x0000000000000000 in ?? () > #30 0x0000000000000000 in ?? () > #31 0x0000000000000000 in ?? () > #32 0x0000000000000000 in ?? () > #33 0x0000000000000000 in ?? () > #34 0x0000000000000000 in ?? () > #35 0x0000000000000000 in ?? () > #36 0x0000000000000000 in ?? () > #37 0x0000000000000000 in ?? () > #38 0x0000000000000000 in ?? () > #39 0x0000000000000000 in ?? () > #40 0x0000000000000000 in ?? () > #41 0x0000000000000000 in ?? () > #42 0x0000000000000000 in ?? () > #43 0x0000000000000000 in ?? () > #44 0x0000000000000000 in ?? () > #45 0x0000000000000000 in ?? () > #46 0x0000000000000000 in ?? () > #47 0x0000000000000000 in ?? () > #48 0x0000000000000000 in ?? () > ---Type to continue, or q to quit--- > #49 0x0000000000000000 in ?? () > #50 0x0000000000000000 in ?? () > #51 0xfffffe0007304920 in ?? () > #52 0xfffffe0003afd000 in ?? () > #53 0xfffffe0007304920 in ?? () > #54 0xffffff8354538b40 in ?? () > #55 0xffffff8354538ae8 in ?? () > #56 0xfffffe0003b00920 in ?? () > #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270, > newtd=0xfffffe000730c300, flags=Variable "flags" is not available. > ) at /usr/src/sys/kern/sched_ule.c:1898 > Previous frame inner to this frame (corrupt stack?) > > .............................................................................. > > > > Relevant pciconf output... > .............................................................................. > > em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled > > .............................................................................. > > > dev.em sysctl.... > .............................................................................. > > dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.0.%driver: em > dev.em.0.%location: slot=0 function=0 > dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.0.%parent: pci1 > dev.em.0.nvm: -1 > dev.em.0.debug: -1 > dev.em.0.fc: 3 > dev.em.0.rx_int_delay: 0 > dev.em.0.tx_int_delay: 66 > dev.em.0.rx_abs_int_delay: 66 > dev.em.0.tx_abs_int_delay: 66 > dev.em.0.itr: 488 > dev.em.0.rx_processing_limit: -1 > dev.em.0.eee_control: 1 > dev.em.0.link_irq: 2 > dev.em.0.mbuf_alloc_fail: 0 > dev.em.0.cluster_alloc_fail: 0 > dev.em.0.dropped: 0 > dev.em.0.tx_dma_fail: 0 > dev.em.0.rx_overruns: 0 > dev.em.0.watchdog_timeouts: 0 > dev.em.0.device_control: 1477444168 > dev.em.0.rx_control: 67141634 > dev.em.0.fc_high_water: 18432 > dev.em.0.fc_low_water: 16932 > dev.em.0.queue0.txd_head: 3265 > dev.em.0.queue0.txd_tail: 3265 > dev.em.0.queue0.tx_irq: 81153071 > dev.em.0.queue0.no_desc_avail: 0 > dev.em.0.queue0.rxd_head: 388 > dev.em.0.queue0.rxd_tail: 387 > dev.em.0.queue0.rx_irq: 79015024 > dev.em.0.mac_stats.excess_coll: 0 > dev.em.0.mac_stats.single_coll: 0 > dev.em.0.mac_stats.multiple_coll: 0 > dev.em.0.mac_stats.late_coll: 0 > dev.em.0.mac_stats.collision_count: 0 > dev.em.0.mac_stats.symbol_errors: 0 > dev.em.0.mac_stats.sequence_errors: 0 > dev.em.0.mac_stats.defer_count: 0 > dev.em.0.mac_stats.missed_packets: 0 > dev.em.0.mac_stats.recv_no_buff: 0 > dev.em.0.mac_stats.recv_undersize: 0 > dev.em.0.mac_stats.recv_fragmented: 0 > dev.em.0.mac_stats.recv_oversize: 0 > dev.em.0.mac_stats.recv_jabber: 0 > dev.em.0.mac_stats.recv_errs: 0 > dev.em.0.mac_stats.crc_errs: 0 > dev.em.0.mac_stats.alignment_errs: 0 > dev.em.0.mac_stats.coll_ext_errs: 0 > dev.em.0.mac_stats.xon_recvd: 6 > dev.em.0.mac_stats.xon_txd: 0 > dev.em.0.mac_stats.xoff_recvd: 6 > dev.em.0.mac_stats.xoff_txd: 0 > dev.em.0.mac_stats.total_pkts_recvd: 122072630 > dev.em.0.mac_stats.good_pkts_recvd: 122072618 > dev.em.0.mac_stats.bcast_pkts_recvd: 257 > dev.em.0.mac_stats.mcast_pkts_recvd: 0 > dev.em.0.mac_stats.rx_frames_64: 8634 > dev.em.0.mac_stats.rx_frames_65_127: 67656673 > dev.em.0.mac_stats.rx_frames_128_255: 714152 > dev.em.0.mac_stats.rx_frames_256_511: 609615 > dev.em.0.mac_stats.rx_frames_512_1023: 8646536 > dev.em.0.mac_stats.rx_frames_1024_1522: 44437008 > dev.em.0.mac_stats.good_octets_recvd: 82940411216 > dev.em.0.mac_stats.good_octets_txd: 25718335997 > dev.em.0.mac_stats.total_pkts_txd: 99833592 > dev.em.0.mac_stats.good_pkts_txd: 99833592 > dev.em.0.mac_stats.bcast_pkts_txd: 13 > dev.em.0.mac_stats.mcast_pkts_txd: 0 > dev.em.0.mac_stats.tx_frames_64: 2193 > dev.em.0.mac_stats.tx_frames_65_127: 29089783 > dev.em.0.mac_stats.tx_frames_128_255: 54412030 > dev.em.0.mac_stats.tx_frames_256_511: 9565246 > dev.em.0.mac_stats.tx_frames_512_1023: 1080398 > dev.em.0.mac_stats.tx_frames_1024_1522: 5683942 > dev.em.0.mac_stats.tso_txd: 1468623 > dev.em.0.mac_stats.tso_ctx_fail: 0 > dev.em.0.interrupts.asserts: 2 > dev.em.0.interrupts.rx_pkt_timer: 0 > dev.em.0.interrupts.rx_abs_timer: 0 > dev.em.0.interrupts.tx_pkt_timer: 0 > dev.em.0.interrupts.tx_abs_timer: 0 > dev.em.0.interrupts.tx_queue_empty: 0 > dev.em.0.interrupts.tx_queue_min_thresh: 0 > dev.em.0.interrupts.rx_desc_min_thresh: 0 > dev.em.0.interrupts.rx_overrun: 0 > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.1.%driver: em > dev.em.1.%location: slot=0 function=0 > dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.1.%parent: pci2 > dev.em.1.nvm: -1 > dev.em.1.debug: -1 > dev.em.1.fc: 3 > dev.em.1.rx_int_delay: 0 > dev.em.1.tx_int_delay: 66 > dev.em.1.rx_abs_int_delay: 66 > dev.em.1.tx_abs_int_delay: 66 > dev.em.1.itr: 488 > dev.em.1.rx_processing_limit: -1 > dev.em.1.eee_control: 1 > dev.em.1.link_irq: 2 > dev.em.1.mbuf_alloc_fail: 0 > dev.em.1.cluster_alloc_fail: 0 > dev.em.1.dropped: 0 > dev.em.1.tx_dma_fail: 1 > dev.em.1.rx_overruns: 0 > dev.em.1.watchdog_timeouts: 0 > dev.em.1.device_control: 1477444168 > dev.em.1.rx_control: 67141634 > dev.em.1.fc_high_water: 18432 > dev.em.1.fc_low_water: 16932 > dev.em.1.queue0.txd_head: 2451 > dev.em.1.queue0.txd_tail: 2453 > dev.em.1.queue0.tx_irq: 143904807 > dev.em.1.queue0.no_desc_avail: 0 > dev.em.1.queue0.rxd_head: 342 > dev.em.1.queue0.rxd_tail: 341 > dev.em.1.queue0.rx_irq: 159303310 > dev.em.1.mac_stats.excess_coll: 0 > dev.em.1.mac_stats.single_coll: 0 > dev.em.1.mac_stats.multiple_coll: 0 > dev.em.1.mac_stats.late_coll: 0 > dev.em.1.mac_stats.collision_count: 0 > dev.em.1.mac_stats.symbol_errors: 0 > dev.em.1.mac_stats.sequence_errors: 0 > dev.em.1.mac_stats.defer_count: 0 > dev.em.1.mac_stats.missed_packets: 0 > dev.em.1.mac_stats.recv_no_buff: 0 > dev.em.1.mac_stats.recv_undersize: 0 > dev.em.1.mac_stats.recv_fragmented: 0 > dev.em.1.mac_stats.recv_oversize: 0 > dev.em.1.mac_stats.recv_jabber: 0 > dev.em.1.mac_stats.recv_errs: 0 > dev.em.1.mac_stats.crc_errs: 0 > dev.em.1.mac_stats.alignment_errs: 0 > dev.em.1.mac_stats.coll_ext_errs: 0 > dev.em.1.mac_stats.xon_recvd: 1 > dev.em.1.mac_stats.xon_txd: 0 > dev.em.1.mac_stats.xoff_recvd: 1 > dev.em.1.mac_stats.xoff_txd: 0 > dev.em.1.mac_stats.total_pkts_recvd: 331901758 > dev.em.1.mac_stats.good_pkts_recvd: 331901756 > dev.em.1.mac_stats.bcast_pkts_recvd: 13467 > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > dev.em.1.mac_stats.rx_frames_64: 13905035 > dev.em.1.mac_stats.rx_frames_65_127: 22315178 > dev.em.1.mac_stats.rx_frames_128_255: 8343368 > dev.em.1.mac_stats.rx_frames_256_511: 8602323 > dev.em.1.mac_stats.rx_frames_512_1023: 8170288 > dev.em.1.mac_stats.rx_frames_1024_1522: 270565564 > dev.em.1.mac_stats.good_octets_recvd: 420794715670 > dev.em.1.mac_stats.good_octets_txd: 45361473880 > dev.em.1.mac_stats.total_pkts_txd: 217852588 > dev.em.1.mac_stats.good_pkts_txd: 217852588 > dev.em.1.mac_stats.bcast_pkts_txd: 6 > dev.em.1.mac_stats.mcast_pkts_txd: 0 > dev.em.1.mac_stats.tx_frames_64: 64102191 > dev.em.1.mac_stats.tx_frames_65_127: 120705475 > dev.em.1.mac_stats.tx_frames_128_255: 6009336 > dev.em.1.mac_stats.tx_frames_256_511: 4593595 > dev.em.1.mac_stats.tx_frames_512_1023: 4295623 > dev.em.1.mac_stats.tx_frames_1024_1522: 18146368 > dev.em.1.mac_stats.tso_txd: 291134 > dev.em.1.mac_stats.tso_ctx_fail: 0 > dev.em.1.interrupts.asserts: 2 > dev.em.1.interrupts.rx_pkt_timer: 0 > dev.em.1.interrupts.rx_abs_timer: 0 > dev.em.1.interrupts.tx_pkt_timer: 0 > dev.em.1.interrupts.tx_abs_timer: 0 > dev.em.1.interrupts.tx_queue_empty: 0 > dev.em.1.interrupts.tx_queue_min_thresh: 0 > dev.em.1.interrupts.rx_desc_min_thresh: 0 > dev.em.1.interrupts.rx_overrun: 0 > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.2.%driver: em > dev.em.2.%location: slot=0 function=0 > dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.2.%parent: pci7 > dev.em.2.nvm: -1 > dev.em.2.debug: -1 > dev.em.2.fc: 3 > dev.em.2.rx_int_delay: 0 > dev.em.2.tx_int_delay: 66 > dev.em.2.rx_abs_int_delay: 66 > dev.em.2.tx_abs_int_delay: 66 > dev.em.2.itr: 488 > dev.em.2.rx_processing_limit: -1 > dev.em.2.eee_control: 1 > dev.em.2.link_irq: 1 > dev.em.2.mbuf_alloc_fail: 0 > dev.em.2.cluster_alloc_fail: 0 > dev.em.2.dropped: 0 > dev.em.2.tx_dma_fail: 6823 > dev.em.2.rx_overruns: 0 > dev.em.2.watchdog_timeouts: 0 > dev.em.2.device_control: 1477444168 > dev.em.2.rx_control: 67141634 > dev.em.2.fc_high_water: 18432 > dev.em.2.fc_low_water: 16932 > dev.em.2.queue0.txd_head: 3977 > dev.em.2.queue0.txd_tail: 3977 > dev.em.2.queue0.tx_irq: 220950699 > dev.em.2.queue0.no_desc_avail: 0 > dev.em.2.queue0.rxd_head: 83 > dev.em.2.queue0.rxd_tail: 82 > dev.em.2.queue0.rx_irq: 125920607 > dev.em.2.mac_stats.excess_coll: 0 > dev.em.2.mac_stats.single_coll: 0 > dev.em.2.mac_stats.multiple_coll: 0 > dev.em.2.mac_stats.late_coll: 0 > dev.em.2.mac_stats.collision_count: 0 > dev.em.2.mac_stats.symbol_errors: 0 > dev.em.2.mac_stats.sequence_errors: 0 > dev.em.2.mac_stats.defer_count: 0 > dev.em.2.mac_stats.missed_packets: 0 > dev.em.2.mac_stats.recv_no_buff: 0 > dev.em.2.mac_stats.recv_undersize: 0 > dev.em.2.mac_stats.recv_fragmented: 0 > dev.em.2.mac_stats.recv_oversize: 0 > dev.em.2.mac_stats.recv_jabber: 0 > dev.em.2.mac_stats.recv_errs: 0 > dev.em.2.mac_stats.crc_errs: 0 > dev.em.2.mac_stats.alignment_errs: 0 > dev.em.2.mac_stats.coll_ext_errs: 0 > dev.em.2.mac_stats.xon_recvd: 14123 > dev.em.2.mac_stats.xon_txd: 1 > dev.em.2.mac_stats.xoff_recvd: 14127 > dev.em.2.mac_stats.xoff_txd: 1 > dev.em.2.mac_stats.total_pkts_recvd: 229919303 > dev.em.2.mac_stats.good_pkts_recvd: 229891053 > dev.em.2.mac_stats.bcast_pkts_recvd: 909450 > dev.em.2.mac_stats.mcast_pkts_recvd: 19452 > dev.em.2.mac_stats.rx_frames_64: 1477808 > dev.em.2.mac_stats.rx_frames_65_127: 195114744 > dev.em.2.mac_stats.rx_frames_128_255: 6579690 > dev.em.2.mac_stats.rx_frames_256_511: 5137387 > dev.em.2.mac_stats.rx_frames_512_1023: 4223090 > dev.em.2.mac_stats.rx_frames_1024_1522: 17358334 > dev.em.2.mac_stats.good_octets_recvd: 46129102134 > dev.em.2.mac_stats.good_octets_txd: 419293159496 > dev.em.2.mac_stats.total_pkts_txd: 332661584 > dev.em.2.mac_stats.good_pkts_txd: 332661582 > dev.em.2.mac_stats.bcast_pkts_txd: 48506 > dev.em.2.mac_stats.mcast_pkts_txd: 78 > dev.em.2.mac_stats.tx_frames_64: 14598198 > dev.em.2.mac_stats.tx_frames_65_127: 22287108 > dev.em.2.mac_stats.tx_frames_128_255: 8897511 > dev.em.2.mac_stats.tx_frames_256_511: 9623000 > dev.em.2.mac_stats.tx_frames_512_1023: 8325033 > dev.em.2.mac_stats.tx_frames_1024_1522: 268930732 > dev.em.2.mac_stats.tso_txd: 24357891 > dev.em.2.mac_stats.tso_ctx_fail: 0 > dev.em.2.interrupts.asserts: 2 > dev.em.2.interrupts.rx_pkt_timer: 0 > dev.em.2.interrupts.rx_abs_timer: 0 > dev.em.2.interrupts.tx_pkt_timer: 0 > dev.em.2.interrupts.tx_abs_timer: 0 > dev.em.2.interrupts.tx_queue_empty: 0 > dev.em.2.interrupts.tx_queue_min_thresh: 0 > dev.em.2.interrupts.rx_desc_min_thresh: 0 > dev.em.2.interrupts.rx_overrun: 0 > dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8 > dev.em.3.%driver: em > dev.em.3.%location: slot=0 function=0 > dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9 > subdevice=0x10d3 class=0x020000 > dev.em.3.%parent: pci8 > dev.em.3.nvm: -1 > dev.em.3.debug: -1 > dev.em.3.fc: 3 > dev.em.3.rx_int_delay: 0 > dev.em.3.tx_int_delay: 66 > dev.em.3.rx_abs_int_delay: 66 > dev.em.3.tx_abs_int_delay: 66 > dev.em.3.itr: 488 > dev.em.3.rx_processing_limit: -1 > dev.em.3.eee_control: 1 > dev.em.3.link_irq: 0 > dev.em.3.mbuf_alloc_fail: 0 > dev.em.3.cluster_alloc_fail: 0 > dev.em.3.dropped: 0 > dev.em.3.tx_dma_fail: 0 > dev.em.3.rx_overruns: 0 > dev.em.3.watchdog_timeouts: 0 > dev.em.3.device_control: 1074790984 > dev.em.3.rx_control: 67141634 > dev.em.3.fc_high_water: 18432 > dev.em.3.fc_low_water: 16932 > dev.em.3.queue0.txd_head: 0 > dev.em.3.queue0.txd_tail: 0 > dev.em.3.queue0.tx_irq: 0 > dev.em.3.queue0.no_desc_avail: 0 > dev.em.3.queue0.rxd_head: 0 > dev.em.3.queue0.rxd_tail: 4095 > dev.em.3.queue0.rx_irq: 0 > dev.em.3.mac_stats.excess_coll: 0 > dev.em.3.mac_stats.single_coll: 0 > dev.em.3.mac_stats.multiple_coll: 0 > dev.em.3.mac_stats.late_coll: 0 > dev.em.3.mac_stats.collision_count: 0 > dev.em.3.mac_stats.symbol_errors: 0 > dev.em.3.mac_stats.sequence_errors: 0 > dev.em.3.mac_stats.defer_count: 0 > dev.em.3.mac_stats.missed_packets: 0 > dev.em.3.mac_stats.recv_no_buff: 0 > dev.em.3.mac_stats.recv_undersize: 0 > dev.em.3.mac_stats.recv_fragmented: 0 > dev.em.3.mac_stats.recv_oversize: 0 > dev.em.3.mac_stats.recv_jabber: 0 > dev.em.3.mac_stats.recv_errs: 0 > dev.em.3.mac_stats.crc_errs: 0 > dev.em.3.mac_stats.alignment_errs: 0 > dev.em.3.mac_stats.coll_ext_errs: 0 > dev.em.3.mac_stats.xon_recvd: 0 > dev.em.3.mac_stats.xon_txd: 0 > dev.em.3.mac_stats.xoff_recvd: 0 > dev.em.3.mac_stats.xoff_txd: 0 > dev.em.3.mac_stats.total_pkts_recvd: 0 > dev.em.3.mac_stats.good_pkts_recvd: 0 > dev.em.3.mac_stats.bcast_pkts_recvd: 0 > dev.em.3.mac_stats.mcast_pkts_recvd: 0 > dev.em.3.mac_stats.rx_frames_64: 0 > dev.em.3.mac_stats.rx_frames_65_127: 0 > dev.em.3.mac_stats.rx_frames_128_255: 0 > dev.em.3.mac_stats.rx_frames_256_511: 0 > dev.em.3.mac_stats.rx_frames_512_1023: 0 > dev.em.3.mac_stats.rx_frames_1024_1522: 0 > dev.em.3.mac_stats.good_octets_recvd: 0 > dev.em.3.mac_stats.good_octets_txd: 0 > dev.em.3.mac_stats.total_pkts_txd: 0 > dev.em.3.mac_stats.good_pkts_txd: 0 > dev.em.3.mac_stats.bcast_pkts_txd: 0 > dev.em.3.mac_stats.mcast_pkts_txd: 0 > dev.em.3.mac_stats.tx_frames_64: 0 > dev.em.3.mac_stats.tx_frames_65_127: 0 > dev.em.3.mac_stats.tx_frames_128_255: 0 > dev.em.3.mac_stats.tx_frames_256_511: 0 > dev.em.3.mac_stats.tx_frames_512_1023: 0 > dev.em.3.mac_stats.tx_frames_1024_1522: 0 > dev.em.3.mac_stats.tso_txd: 0 > dev.em.3.mac_stats.tso_ctx_fail: 0 > dev.em.3.interrupts.asserts: 0 > dev.em.3.interrupts.rx_pkt_timer: 0 > dev.em.3.interrupts.rx_abs_timer: 0 > dev.em.3.interrupts.tx_pkt_timer: 0 > dev.em.3.interrupts.tx_abs_timer: 0 > dev.em.3.interrupts.tx_queue_empty: 0 > dev.em.3.interrupts.tx_queue_min_thresh: 0 > dev.em.3.interrupts.rx_desc_min_thresh: 0 > dev.em.3.interrupts.rx_overrun: 0 > > .............................................................................. From owner-freebsd-net@FreeBSD.ORG Fri May 16 17:47:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 527A8FCB; Fri, 16 May 2014 17:47:17 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03845276A; Fri, 16 May 2014 17:47:16 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id z60so4661956qgd.23 for ; Fri, 16 May 2014 10:47:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=3fn/7a/yqhgBNu5ZYOwoheSNoBzqrpBRWoHfCP6v9zQ=; b=D0pim9RdvganPv0jklB+MjLiRlBX0Wn5AOoAfof23uvaDU4PcGXMjCiPml4lffWdA5 GrVMWPlbYa0zgPlxidBHeDEVcKYZ0f05uPj+DvoDTOAgEQ7EbA2uiXafJi6yudkoCoaW kb7rhU2HdpvgGW+YQQCkeWisymPeKB895a6wP1ZW1Owb1fbgjwemh4jVvwF36+WKpe/r 6kb/JczjR5yVIHLlFU6ROc3XFVY7AxVBYeG4/Q/DSDTmHXn8vcLh/jjV8wmg+FhXW9dd fGQPsPD2MYprLNfgwnF8mleELKUCaq3G0b6pxv2/nqKpZn4N43/SPJGApvSz0hBFNSKH 8OUg== MIME-Version: 1.0 X-Received: by 10.224.37.10 with SMTP id v10mr21295121qad.98.1400262436155; Fri, 16 May 2014 10:47:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 16 May 2014 10:47:16 -0700 (PDT) In-Reply-To: References: Date: Fri, 16 May 2014 10:47:16 -0700 X-Google-Sender-Auth: sLjaowIUnoP9phseKygkcJhL_n4 Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 17:47:17 -0000 Ok, I've given this a whirl on a slightly larger system. There's no 10Gbit/sec in it yet, but it's stable under 64,000 sockets at 1Gbit/sec. I'm going to commit this over the next couple of days unless there are any objections. The defaults are still the same so it won't affect the rest of you. -a On 14 May 2014 22:26, Adrian Chadd wrote: > .. and i've done a little more than no testing, so far so good. > > http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-2.diff > > This adds IP_FLOWID, IP_FLOWTYPE and IP_RSSCPUID to fetch the socket > flowid, flowtype and cpuid from the inp. It's mostly for debugging for > now. > > Thanks, > > > > -a > > > On 14 May 2014 21:43, Adrian Chadd wrote: >> Hi, >> >> here's a completely untested patch for discussion. I'm emailing it out >> mostly as a "is this a good idea" patch rather than a "it should just >> be committed" patch. >> >> The RSS stuff from Robert maps connections to pcbgroups based on the >> RSS hash, but it doesn't map the TCP timers the same way. So by >> default they're all on swi0 and the per CPU timers don't take the hash >> type or correctly choose the CPU. >> >> This patch: >> >> http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff >> >> does a few things: >> >> * it stores the hashtype in the inp as well as the flowid; >> * the rss code grows a new method that maps the flowid/hashtype to a cpuid >> * .. and the existing mbuf to cpuid method now uses this; >> * the tcp timer code now has an inline function that knows about RSS >> and defaults to the existing way of doing things if RSS isn't enabled. >> >> There's still a bunch of work left before all the lock contention >> compartmentalization and balancing of RSS is realised. I'm just >> chewing off the little corner bits that are easy to get done now. >> >> So, any comments? I'll give it a proper whirl on some 10G hardware in >> a few days but I thought I'd at least get the general idea out there >> for comment. >> >> Thanks, >> >> >> >> >> -a From owner-freebsd-net@FreeBSD.ORG Sat May 17 00:13:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A2451A9; Sat, 17 May 2014 00:13:44 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE52527EA; Sat, 17 May 2014 00:13:43 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so5284361qge.8 for ; Fri, 16 May 2014 17:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=ez2nZuSIk5nTYc25Bj2BvHaR2z32pWp5UE37c+DXWBI=; b=aMucMvGfD4VztmyRoCcKu0LthF5ELKwivUULVC+tVs8mfkAy67CD6VqtmDt+tpXD5v 9t1kJB/3OClln4gt7HJl8CVdwVB1x4BUhnMujbISjDlc6FHxIZab1tL33kbilkbc+xdJ GD2q/rp3U7Jp5V6wHMXLQTCUQFqi1VMAmb27aq/a+gBZv7hoaikyQUmeabA9rIVHrA/s 7tqjCQ98g9Y59vQJQ+0LHI50eRpQUyw1P7DnGuP8fFFeI1erjBy9FnxQtoPc8usjD7PC Oe2x23pil3fWcvFwNeKcYN8+14tcflhj2cjD5nHi54WXQXb6Q8jh+vrkOCUERnnvrLim ekoQ== MIME-Version: 1.0 X-Received: by 10.140.22.209 with SMTP id 75mr28925779qgn.4.1400285622901; Fri, 16 May 2014 17:13:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 16 May 2014 17:13:42 -0700 (PDT) In-Reply-To: References: Date: Fri, 16 May 2014 17:13:42 -0700 X-Google-Sender-Auth: wYZxX76B31db0QZ5ydIri50MoGA Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 00:13:44 -0000 On 16 May 2014 10:47, Adrian Chadd wrote: > Ok, I've given this a whirl on a slightly larger system. There's no > 10Gbit/sec in it yet, but it's stable under 64,000 sockets at > 1Gbit/sec. > > I'm going to commit this over the next couple of days unless there are > any objections. The defaults are still the same so it won't affect the > rest of you. http://people.freebsd.org/~adrian/norse/20140517-tcp-rss-timers-3.diff I've committed the in.h IP_* fields for now, just to get that out of the way. This patch fixes the inp_cpu lookup code to actually be right. :-) -a From owner-freebsd-net@FreeBSD.ORG Sat May 17 13:46:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D49676E1 for ; Sat, 17 May 2014 13:46:07 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96B2222E9 for ; Sat, 17 May 2014 13:46:07 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wlb1w-000Og5-K1; Sat, 17 May 2014 13:35:48 +0400 Message-ID: <537767C5.80205@FreeBSD.org> Date: Sat, 17 May 2014 17:44:37 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> In-Reply-To: <53720AA4.80909@smartspb.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 13:46:07 -0000 On 13.05.2014 16:05, Dennis Yusupoff wrote: > I think that universal table for all kind of data (ipv4, ipv6, ports, > etc) is a bad idea by design. At least unless you haven't any ability to It is not always "universal" in kernel. Actually, different radix tables are used to store both IPv4 and IPv6 in single table. > specify address family on add, to avoid attempts to guess what user > meant. Something like "ipfw table X add DEEF.DE ipv6". I'm going to add explicit table type/naming setup soon. Idea is the following: 1) Existing table can be named and addressed by either number or name. However, you still need to assign table number manually. 2) Table type/name can be specified explicitly via one of the following commands: * ipfw table 1 create [type ] [name "table_name"] * ipfw table name "table_name" * ipfw table "table_name" type 3) ipfw(8) stops trying to guess appropriate type based on used value. Instead, it requests table type from kernel and interprets value according to returned type. Default type for all tables is cidr 4) Table(s) can be returned to default values using ipfw table destroy. Destroy means: * flush * table tries (or other structures) freed * type set to cidr > > > 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>> May be this will help? See answer on >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >> I'll try to fix it within a few days. Fixed in r266310. >> >> The problem itself happens due to the fact that every CIDR table >> address is packed into IPv6 address and IPv4 ones are encoded as >> deprecated IPv6-compatible ones. >> this leads to the problems with decoding things like 0/X or ::1 From owner-freebsd-net@FreeBSD.ORG Sat May 17 14:44:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3604F65; Sat, 17 May 2014 14:44:32 +0000 (UTC) Received: from exprod6og121.obsmtp.com (exprod6og121.obsmtp.com [64.18.1.237]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28C3028D8; Sat, 17 May 2014 14:44:31 +0000 (UTC) Received: from osprey.verisign.com ([216.168.239.75]) (using TLSv1) by exprod6ob121.postini.com ([64.18.5.12]) with SMTP ID DSNKU3d1yRQ4mfjkdO4/nYcUMTwPNb8Zx3uB@postini.com; Sat, 17 May 2014 07:44:32 PDT Received: from brn1wnexcas02.vcorp.ad.vrsn.com (brn1wnexcas02.vcorp.ad.vrsn.com [10.173.152.206]) by osprey.verisign.com (8.13.6/8.13.4) with ESMTP id s4HEiLqK012362 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 17 May 2014 10:44:24 -0400 Received: from BRN1WNEXMBX01.vcorp.ad.vrsn.com ([::1]) by brn1wnexcas02.vcorp.ad.vrsn.com ([::1]) with mapi id 14.03.0174.001; Sat, 17 May 2014 10:44:21 -0400 From: "Bentkofsky, Michael" To: "'freebsd-net@freebsd.org'" Subject: Re: [rfc] tcp timer update for RSS Thread-Topic: Re: [rfc] tcp timer update for RSS Thread-Index: Ac9x2wm0XVYCeE+tS3KGThdtBwSzYA== Date: Sat, 17 May 2014 14:44:20 +0000 Message-ID: <080FBD5B7A09F845842100A6DE79623346F2EB44@BRN1WNEXMBX01.vcorp.ad.vrsn.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.173.152.4] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "'Adrian Chadd \(adrian@freebsd.org\)'" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 14:44:32 -0000 Hi Adrian, I haven't had the chance to look this over carefully yet as we're at BSDCan= . I think I understand what you're trying to achieve by aligning the per-CP= U timer processing per core. In principal that sounds reasonable, although = I am unsure if you were trying to solve a particular performance issue with= this particular change. My sense is this is all preparatory with the goal = of all inp processing to become per core. Could you comment on the general = evolution you're considering? Do most of the PCB structures become per-core= , as in PCB groups? If you'd like us to test this change, I'm happy to do so. At the moment I d= on't know if we'd expect to see any benefit - do you have any traffic condi= tions for which this showed any difference? But we can certainly drive many= hundreds of thousands of connections at reasonably high connection rates i= f that will help. Thanks, Mike From owner-freebsd-net@FreeBSD.ORG Sat May 17 14:49:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59830169 for ; Sat, 17 May 2014 14:49:34 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AAFB2956 for ; Sat, 17 May 2014 14:49:34 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so6165310qgf.3 for ; Sat, 17 May 2014 07:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=LGI70hRahlmuTHVBA9oyG63DFLNh3aOEXjW6fqFFsUo=; b=atYybAjgebbWZIxXD6p9+Mlus/UkhcXy2gJroMmdIq6w8+euQ7qrvYxRZEvONj6/La IiktixCl8VO3QIqjFiZWhmszRrHTleDxfiVHUhzMHUl4trGzTi5FBnCu7OwekjUzQvpL /FOkKVPz+kN+DIZMdX5MN5jEonDalsCJRr5s1qG8ivTYwgjreuucS+0xXSY4dLAlCuP9 3JGyPsOR5NbEujgxmol3/C1OE1oYrDFDeMULMTEobvvgdDxuAjoeEXq53nn1Y9MJsogf bLEnfMcNwyvX5V2DGes2PQenhJUzzIPDothujvQYQnmRg/TmQ+/RFTTnTTaQ6D8kIkI8 W0eA== MIME-Version: 1.0 X-Received: by 10.229.198.2 with SMTP id em2mr25059252qcb.21.1400338173323; Sat, 17 May 2014 07:49:33 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sat, 17 May 2014 07:49:33 -0700 (PDT) In-Reply-To: <080FBD5B7A09F845842100A6DE79623346F2EB44@BRN1WNEXMBX01.vcorp.ad.vrsn.com> References: <080FBD5B7A09F845842100A6DE79623346F2EB44@BRN1WNEXMBX01.vcorp.ad.vrsn.com> Date: Sat, 17 May 2014 07:49:33 -0700 X-Google-Sender-Auth: ojuNHsG-CKArJNxHqN1db7oeaK4 Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Bentkofsky, Michael" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 14:49:34 -0000 On 17 May 2014 07:44, Bentkofsky, Michael wrote: > Hi Adrian, > > > > I haven=E2=80=99t had the chance to look this over carefully yet as we=E2= =80=99re at BSDCan. > I think I understand what you=E2=80=99re trying to achieve by aligning th= e per-CPU > timer processing per core. In principal that sounds reasonable, although = I > am unsure if you were trying to solve a particular performance issue with > this particular change. My sense is this is all preparatory with the goal= of > all inp processing to become per core. Could you comment on the general > evolution you=E2=80=99re considering? Do most of the PCB structures becom= e per-core, > as in PCB groups? > > > > If you=E2=80=99d like us to test this change, I=E2=80=99m happy to do so.= At the moment I > don=E2=80=99t know if we=E2=80=99d expect to see any benefit =E2=80=93 do= you have any traffic > conditions for which this showed any difference? But we can certainly dri= ve > many hundreds of thousands of connections at reasonably high connection > rates if that will help. Hi! Yes, the aim is to provide RSS support in the RSS model of "align everything to a specific core." The goal for RSS is to remove both lock contention between cores and keep data CPU/cache local to improve efficiency there. There's nothing obvious that'll be beneficial right now. The items at https://wiki.freebsd.org/NetworkRSS have to occur before it is beneficial to anyone. -a From owner-freebsd-net@FreeBSD.ORG Sat May 17 14:52:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E275F304 for ; Sat, 17 May 2014 14:52:29 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2B632A19 for ; Sat, 17 May 2014 14:52:29 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id w7so6319990qcr.34 for ; Sat, 17 May 2014 07:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=CnKzEtxzU0L9vrZ04elOL6rYhwkxXwQAz1TD8A4we1g=; b=wKJLz0t7bcmc9CH+X5Vr6qb3esmagB7rAEKjs7+F5B4IG3Kq7uolyRnbQUKXej3moy mTvHaeA2Xgn6EQziZ1ZwzHvc2kq+VyYxf3RAo+CHJlvo5JP29RhgXgeRPES0cFnAotpu Wllrqrjttvv3VE/k+ybFSvDxdjPPOSNZ+WFW2JxYqw4jmcO1F7m38ycGPV/kjGK0o30c q33avZE8aBV8GqTykhFEQvVIUHbQcNnSXdfvXzs3ongnxDf80QYOUsRVNmFspW3uswKX +pAFmg/kURMUN/p+CJJWWUdBFcVP8/EeZ8ek3wlvS+fmgqTTHEk/+aNHOYcg7NO+KE8p FXzw== MIME-Version: 1.0 X-Received: by 10.224.37.10 with SMTP id v10mr28438397qad.98.1400338348530; Sat, 17 May 2014 07:52:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sat, 17 May 2014 07:52:28 -0700 (PDT) In-Reply-To: References: <080FBD5B7A09F845842100A6DE79623346F2EB44@BRN1WNEXMBX01.vcorp.ad.vrsn.com> Date: Sat, 17 May 2014 07:52:28 -0700 X-Google-Sender-Auth: Rwvjp_npmufsaWci2ekNI5wApJE Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Bentkofsky, Michael" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 14:52:30 -0000 .. and as a note - it'll all be behind #ifdef RSS. -a On 17 May 2014 07:49, Adrian Chadd wrote: > On 17 May 2014 07:44, Bentkofsky, Michael wrot= e: >> Hi Adrian, >> >> >> >> I haven=E2=80=99t had the chance to look this over carefully yet as we= =E2=80=99re at BSDCan. >> I think I understand what you=E2=80=99re trying to achieve by aligning t= he per-CPU >> timer processing per core. In principal that sounds reasonable, although= I >> am unsure if you were trying to solve a particular performance issue wit= h >> this particular change. My sense is this is all preparatory with the goa= l of >> all inp processing to become per core. Could you comment on the general >> evolution you=E2=80=99re considering? Do most of the PCB structures beco= me per-core, >> as in PCB groups? >> >> >> >> If you=E2=80=99d like us to test this change, I=E2=80=99m happy to do so= . At the moment I >> don=E2=80=99t know if we=E2=80=99d expect to see any benefit =E2=80=93 d= o you have any traffic >> conditions for which this showed any difference? But we can certainly dr= ive >> many hundreds of thousands of connections at reasonably high connection >> rates if that will help. > > Hi! > > Yes, the aim is to provide RSS support in the RSS model of "align > everything to a specific core." The goal for RSS is to remove both > lock contention between cores and keep data CPU/cache local to improve > efficiency there. > > There's nothing obvious that'll be beneficial right now. The items at > https://wiki.freebsd.org/NetworkRSS have to occur before it is > beneficial to anyone. > > > > -a From owner-freebsd-net@FreeBSD.ORG Sat May 17 15:14:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89AFBB3D; Sat, 17 May 2014 15:14:16 +0000 (UTC) Received: from mail-oa0-x22a.google.com (mail-oa0-x22a.google.com [IPv6:2607:f8b0:4003:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43F682C0C; Sat, 17 May 2014 15:14:16 +0000 (UTC) Received: by mail-oa0-f42.google.com with SMTP id j17so4370359oag.15 for ; Sat, 17 May 2014 08:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YZxpt0Tp13VBU7GowUSSDc1V/Yzs3x8oo2tQcdIJFrk=; b=ifczQNYDVhjlx30+CFhRLHAcDcNSB6ozjmTMq7mTMSgzlq+N0YbG/3DCc4zD9IuSfl iiQF4IR1EXe6iyDPVmWqEobqGykQI3eLzXW/O5b7+YqfvU0x6tpaCLu3CMBiVNRjrIN0 Xwz1QnzXOhd7N8/dSSI3B7UvKVZdG1mbpGxdTY3TRkzRBmVxmIDK+hsAaADK/Ppwt4Pv 9bxtx5Wkj3S+jvcP4Aezd1crcfDljCIWGM4sbwWDZ5rM/nk0dxLtjt2IN2mCa7K190rR svTc7YsY2AodQq6RbhwUV48prP++e73D+mUcToK9YtaaokWoq89+3v4D9sI+QjwJo0Vo YMyw== MIME-Version: 1.0 X-Received: by 10.182.213.168 with SMTP id nt8mr23885021obc.7.1400339655561; Sat, 17 May 2014 08:14:15 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Sat, 17 May 2014 08:14:15 -0700 (PDT) In-Reply-To: <537767C5.80205@FreeBSD.org> References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> Date: Sat, 17 May 2014 17:14:15 +0200 Message-ID: Subject: Re: Problem with ipfw table add 0.0.0.0/8 From: Andreas Nilsson To: "Alexander V. Chernikov" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 15:14:16 -0000 On Sat, May 17, 2014 at 3:44 PM, Alexander V. Chernikov < melifaro@freebsd.org> wrote: > On 13.05.2014 16:05, Dennis Yusupoff wrote: > >> I think that universal table for all kind of data (ipv4, ipv6, ports, >> etc) is a bad idea by design. At least unless you haven't any ability to >> > It is not always "universal" in kernel. > Actually, different radix tables are used to store both IPv4 and IPv6 in > single table. > > specify address family on add, to avoid attempts to guess what user >> meant. Something like "ipfw table X add DEEF.DE ipv6". >> > I'm going to add explicit table type/naming setup soon. > Idea is the following: > > 1) Existing table can be named and addressed by either number or name. > However, you still need to assign table number manually. > > 2) Table type/name can be specified explicitly via one of the following > commands: > * ipfw table 1 create [type ] [name "table_name"] > * ipfw table name "table_name" > * ipfw table "table_name" type > > 3) ipfw(8) stops trying to guess appropriate type based on used value. > Instead, > it requests table type from kernel and interprets value according to > returned type. > Default type for all tables is cidr > > 4) Table(s) can be returned to default values using ipfw table > destroy. > Destroy means: > * flush > * table tries (or other structures) freed > * type set to cidr > > > > >> >> 13.05.2014 14:32, Alexander V. Chernikov =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >> >>> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>> >>>> May be this will help? See answer on >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/189471 >>>> >>> I'll try to fix it within a few days. >>> >> Fixed in r266310. > > With all of these changes, would it be possible to get tablearg to store ipv6 as well? I seem to remember it is 32bit only today. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Sat May 17 15:25:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 297CAE67 for ; Sat, 17 May 2014 15:25:24 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0B5B2CE7 for ; Sat, 17 May 2014 15:25:23 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wlca0-000PND-3e; Sat, 17 May 2014 15:15:04 +0400 Message-ID: <53777F09.5030000@FreeBSD.org> Date: Sat, 17 May 2014 19:23:53 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Andreas Nilsson Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 15:25:24 -0000 On 17.05.2014 19:14, Andreas Nilsson wrote: > > > > On Sat, May 17, 2014 at 3:44 PM, Alexander V. Chernikov > > wrote: > > On 13.05.2014 16:05, Dennis Yusupoff wrote: > > I think that universal table for all kind of data (ipv4, ipv6, > ports, > etc) is a bad idea by design. At least unless you haven't any > ability to > > It is not always "universal" in kernel. > Actually, different radix tables are used to store both IPv4 and > IPv6 in single table. > > specify address family on add, to avoid attempts to guess what > user > meant. Something like "ipfw table X add DEEF.DE > ipv6". > > I'm going to add explicit table type/naming setup soon. > Idea is the following: > > 1) Existing table can be named and addressed by either number or name. > However, you still need to assign table number manually. > > 2) Table type/name can be specified explicitly via one of the > following commands: > * ipfw table 1 create [type ] [name > "table_name"] > * ipfw table name "table_name" > * ipfw table "table_name" type > > 3) ipfw(8) stops trying to guess appropriate type based on used > value. Instead, > it requests table type from kernel and interprets value according > to returned type. > Default type for all tables is cidr > > 4) Table(s) can be returned to default values using ipfw table > destroy. > Destroy means: > * flush > * table tries (or other structures) freed > * type set to cidr > > > > > > 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: > > On 13.05.2014 13:46, Dennis Yusupoff wrote: > > May be this will help? See answer on > http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 > > I'll try to fix it within a few days. > > Fixed in r266310. > > With all of these changes, would it be possible to get tablearg to > store ipv6 as well? I seem to remember it is 32bit only today. Well, I'd prefer not to increase tablearg directly. It is probably possible to implement some kind of "nexthop" table adds additional auto-filled nexthop array. > > Best regards > Andreas From owner-freebsd-net@FreeBSD.ORG Sat May 17 19:57:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5846032B; Sat, 17 May 2014 19:57:57 +0000 (UTC) Received: from pit.databus.com (Databus-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:80b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 292982219; Sat, 17 May 2014 19:57:56 +0000 (UTC) Received: by pit.databus.com (Postfix, from userid 202) id 15B4C39D1C; Sat, 17 May 2014 15:57:55 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=databus.com; s=20140217; t=1400356675; bh=3C9bf+FuTPfP5kPz4XbijhW8970VKr+xoXpr7BuxJLo=; l=1741; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=jiMowduQsCLcJCvB55MRP80kJ5R2QYQUUzOr9U/8tYBMTwU9bpru5SUuio7kbMjh5 djvoSiUABnoM3vUbKPOneQ8zv7LZyA6nsPjsNqmopXZMcnpuL1Qpj/TNhVEaknbyD3 XadGCZqOPgE8VI2yim1H1+ao2lwJ1SEd81qamFagxr7qp8F1L/yRDpY1oj13Y6sprN mA8GAJhv4dv6LG5wGtxlUR5gTtldEmXtnXOg5sY1GMm2GMX4coyfOdJi0B6YFHAT/c 1n9zexQ9M+Lh/kZPiWkUQxFawviNhHHv9evx1qa/tPM4ErAzjJn4AtUkCyoZg1t3TC rGKk/dDvc854g== Date: Sat, 17 May 2014 15:57:55 -0400 From: Barney Wolff To: "Alexander V. Chernikov" Subject: Re: Problem with ipfw table add 0.0.0.0/8 Message-ID: <20140517195754.GA1087@pit.databus.com> References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <537767C5.80205@FreeBSD.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 19:57:57 -0000 On Sat, May 17, 2014 at 05:44:37PM +0400, Alexander V. Chernikov wrote: > On 13.05.2014 16:05, Dennis Yusupoff wrote: > > I think that universal table for all kind of data (ipv4, ipv6, ports, > > etc) is a bad idea by design. At least unless you haven't any ability to > It is not always "universal" in kernel. > Actually, different radix tables are used to store both IPv4 and IPv6 in > single table. > > specify address family on add, to avoid attempts to guess what user > > meant. Something like "ipfw table X add DEEF.DE ipv6". > I'm going to add explicit table type/naming setup soon. > Idea is the following: > > 1) Existing table can be named and addressed by either number or name. > However, you still need to assign table number manually. > > 2) Table type/name can be specified explicitly via one of the following > commands: > * ipfw table 1 create [type ] [name "table_name"] > * ipfw table name "table_name" > * ipfw table "table_name" type > > 3) ipfw(8) stops trying to guess appropriate type based on used value. > Instead, > it requests table type from kernel and interprets value according to > returned type. > Default type for all tables is cidr > > 4) Table(s) can be returned to default values using ipfw table > destroy. > Destroy means: > * flush > * table tries (or other structures) freed > * type set to cidr Please avoid violating POLA. I for one have scripts that automatically add entries and would need to be modified if separate ipfw tables become required for ipv4 and ipv6. I'd have no problem, of course, with changes to ipfw internals as long as the existing public API continues to work. From owner-freebsd-net@FreeBSD.ORG Sat May 17 20:00:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 763D8698 for ; Sat, 17 May 2014 20:00:15 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3725F2239 for ; Sat, 17 May 2014 20:00:15 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wlgrz-00014K-C2; Sat, 17 May 2014 19:49:55 +0400 Message-ID: <5377BF74.6010006@FreeBSD.org> Date: Sat, 17 May 2014 23:58:44 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Barney Wolff Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <20140517195754.GA1087@pit.databus.com> In-Reply-To: <20140517195754.GA1087@pit.databus.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 20:00:15 -0000 On 17.05.2014 23:57, Barney Wolff wrote: > On Sat, May 17, 2014 at 05:44:37PM +0400, Alexander V. Chernikov wrote: >> On 13.05.2014 16:05, Dennis Yusupoff wrote: >>> I think that universal table for all kind of data (ipv4, ipv6, ports, >>> etc) is a bad idea by design. At least unless you haven't any ability to >> It is not always "universal" in kernel. >> Actually, different radix tables are used to store both IPv4 and IPv6 in >> single table. >>> specify address family on add, to avoid attempts to guess what user >>> meant. Something like "ipfw table X add DEEF.DE ipv6". >> I'm going to add explicit table type/naming setup soon. >> Idea is the following: >> >> 1) Existing table can be named and addressed by either number or name. >> However, you still need to assign table number manually. >> >> 2) Table type/name can be specified explicitly via one of the following >> commands: >> * ipfw table 1 create [type ] [name "table_name"] >> * ipfw table name "table_name" >> * ipfw table "table_name" type >> >> 3) ipfw(8) stops trying to guess appropriate type based on used value. >> Instead, >> it requests table type from kernel and interprets value according to >> returned type. >> Default type for all tables is cidr >> >> 4) Table(s) can be returned to default values using ipfw table >> destroy. >> Destroy means: >> * flush >> * table tries (or other structures) freed >> * type set to cidr > Please avoid violating POLA. I for one have scripts that automatically add > entries and would need to be modified if separate ipfw tables become required > for ipv4 and ipv6. I'd have no problem, of course, with changes to ipfw I haven't said anything about splitting IPv4 and IPv6 tables. > internals as long as the existing public API continues to work. > From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:17:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35775195 for ; Sat, 17 May 2014 23:17:23 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 18A7620F0 for ; Sat, 17 May 2014 23:17:22 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 384CC3AC90 for ; Sat, 17 May 2014 16:17:12 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-net@freebsd.org Subject: Gateway? Date: Sat, 17 May 2014 16:17:12 -0700 Message-ID: <73288.1400368632@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:17:23 -0000 Forgive me, please for such a rudimentary sort of question. I've been doing IP networking for more than 15 years, but I never really plumbed the depths, and thus I only know the basics. Quite simply, I'd like to know if the defaultrouter= IPv4 address specified in my /etc/rc.conf file should be the same as whatever I normally see as the first hop in an outgoing traceroute. I ask because for me these two are apparently not the same. Here's what I have: defaultrouter="69.62.255.254" and here is one example of a recent outgoing traceroute: % traceroute 74.125.239.148 traceroute to 74.125.239.148 (74.125.239.148), 64 hops max, 52 byte packets 1 86.255-62-69.res.dyn.surewest.net (69.62.255.86) 28.884 ms 31.395 ms 30.024 ms 2 216.0.55.209 (216.0.55.209) 26.486 ms 26.024 ms 25.850 ms 3 ae1d0.mcr1.roseville-ca.us.xo.net (216.156.1.77) 25.384 ms 27.298 ms 27.060 ms 4 vb1510.rar3.sanjose-ca.us.xo.net (216.156.0.153) 27.289 ms 34.022 ms 36.213 ms 5 207.88.14.226.ptr.us.xo.net (207.88.14.226) 26.993 ms 26.567 ms 25.568 ms 6 216.156.84.30.ptr.us.xo.net (216.156.84.30) 24.800 ms 26.432 ms 25.845 ms 7 209.85.249.5 (209.85.249.5) 26.033 ms 110.066 ms 28.663 ms 8 66.249.95.31 (66.249.95.31) 26.985 ms 25.285 ms 28.066 ms 9 nuq05s02-in-f20.1e100.net (74.125.239.148) 26.895 ms 27.434 ms 27.063 ms P.S. I don't know if it makes any difference or not, but I'm on a residential DSL line, I have two assigned static IP addresses, and I have been told that the router in my closet is set to "bridged" mode. P.P.S. I am exploring all this because I have been having increasingly serious but maddeningly intermittent connectivity problems for some several weeks now. The timing when these problems began appeared, perhaps only coincidently, to closely coincide with the date on which I was "upgraded" from a 3Mbps DSL line to a 6Mbps DSL line, at my request. (And I do wonder now if that change may have somehow produced my current intermittent non-connectivity. Unfortunately, my ISP was recently bought out by a bigger company, and it now appears that the new bigger company no longer employs any persons in either their first or second level tech support teams who even know what a gateway is.) From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:29:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6898716 for ; Sat, 17 May 2014 23:29:14 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FB9321A2 for ; Sat, 17 May 2014 23:29:14 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id B90CD139CB for ; Sat, 17 May 2014 20:29:09 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1400369345; x=1401233346; bh=fqu46juQIyFK JTU6MFw3+Gd4V0ANTczjQUZmvtvfN/c=; b=Phfq9M9IBEcud2TVkQMAtq/o9TLG HPXnHdEb7AupHsbdjIm27VH2IagZnzK1NKYRdmtDhyG3LT3vQtzJ9+0yCUiQZEIu g+ld4rxNairx/OATHpAQ2uV/T24gR2pVn7EpIC6fCEUQwnM9UrfrYerYtqDToJIy R2pJjZQvL4IP+gg= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pFIQipOYn4UB for ; Sat, 17 May 2014 20:29:05 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id BE275139C9; Sat, 17 May 2014 20:29:04 -0300 (BRT) Message-ID: <5377F0BB.1040501@bsdinfo.com.br> Date: Sat, 17 May 2014 20:28:59 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , Dennis Yusupoff , FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> In-Reply-To: <537767C5.80205@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:29:14 -0000 Em 17/05/14 10:44, Alexander V. Chernikov escreveu: > On 13.05.2014 16:05, Dennis Yusupoff wrote: >> I think that universal table for all kind of data (ipv4, ipv6, ports, >> etc) is a bad idea by design. At least unless you haven't any ability to > It is not always "universal" in kernel. > Actually, different radix tables are used to store both IPv4 and IPv6 > in single table. >> specify address family on add, to avoid attempts to guess what user >> meant. Something like "ipfw table X add DEEF.DE ipv6". > I'm going to add explicit table type/naming setup soon. > Idea is the following: > > 1) Existing table can be named and addressed by either number or name. > However, you still need to assign table number manually. > > 2) Table type/name can be specified explicitly via one of the > following commands: > * ipfw table 1 create [type ] [name "table_name"] > * ipfw table name "table_name" > * ipfw table "table_name" type > > 3) ipfw(8) stops trying to guess appropriate type based on used value. > Instead, > it requests table type from kernel and interprets value according to > returned type. > Default type for all tables is cidr > > 4) Table(s) can be returned to default values using ipfw table > destroy. > Destroy means: > * flush > * table tries (or other structures) freed > * type set to cidr > > >> >> >> 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >>> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>>> May be this will help? See answer on >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >>> I'll try to fix it within a few days. > Fixed in r266310. The problem still exists. # ipfw table 99 add 0.0.0.0/8 # ipfw table 99 list ::/8 0 # uname -a FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r266370: Sat May 17 19:57:23 BRT 2014 root@mail.xxxxxx.br:/usr/obj/usr/src/sys/GONDIM amd64 Cheers, From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:30:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44F128D3 for ; Sat, 17 May 2014 23:30:17 +0000 (UTC) Received: from mail-oa0-f44.google.com (mail-oa0-f44.google.com [209.85.219.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 118B221B0 for ; Sat, 17 May 2014 23:30:16 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id o6so4673636oag.31 for ; Sat, 17 May 2014 16:30:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=MaLWX9Aj0W3L6CO3zZUZuAKkTp5vMeyeI7LYKzrdQ4Y=; b=coVf7UE+WdOycuDSzN2tIZswdp2JvsEs4DF1SYpEIWjgqeY2GthxRYyrCgtUldzJkB etDlmfiS0ZkM7qSYZBMs9yLKa2RP7pEq0gAYD/E/eMESc53ROwAKO2En5vggvUZ3YwHs FezwNjuIALjThkxmbupQ5MlJBu4WMErRIAjKibCT6vgd66dliePRPiwu5NlCQdOhGIER ENEBqLQpBClDOBRElnX5A0xr1OkMPNPHnKq3DslUsdM3RmpJ7MHdhdNx2VK3bLdFx5Kp yulf3zXASlAGwrURcX992Ox7oARsmExnjD1OKoqojz9rxFgerFtXmd06rF3xvp7pbJu0 gLrw== X-Gm-Message-State: ALoCoQnFo+rZlyLZ3+rbA1hmaBoezOia519M+7mweqerqZKTEd+LdEmwfk5NcX/XimfMGljKeRqA MIME-Version: 1.0 X-Received: by 10.60.51.39 with SMTP id h7mr26214452oeo.52.1400369409953; Sat, 17 May 2014 16:30:09 -0700 (PDT) Received: by 10.60.17.33 with HTTP; Sat, 17 May 2014 16:30:09 -0700 (PDT) In-Reply-To: <73288.1400368632@server1.tristatelogic.com> References: <73288.1400368632@server1.tristatelogic.com> Date: Sat, 17 May 2014 16:30:09 -0700 Message-ID: Subject: Re: Gateway? From: Michael Sierchio To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:30:17 -0000 On Sat, May 17, 2014 at 4:17 PM, Ronald F. Guilmette wrote: > Quite simply, I'd like to know if the defaultrouter= IPv4 address > specified in my /etc/rc.conf file should be the same as whatever > I normally see as the first hop in an outgoing traceroute. Maybe... see comments below. > defaultrouter="69.62.255.254" > > and here is one example of a recent outgoing traceroute: > > % traceroute 74.125.239.148 > traceroute to 74.125.239.148 (74.125.239.148), 64 hops max, 52 byte packets > 1 86.255-62-69.res.dyn.surewest.net (69.62.255.86) 28.884 ms 31.395 ms 30.024 ms > 2 216.0.55.209 (216.0.55.209) 26.486 ms 26.024 ms 25.850 ms Do you have a fixed IP address (statically assigned), or are you getting an address via DHCP from your ISP? If it's DHCP, your defaultrouter definition is overridden every time you get/renew a lease. netstat -r -n -f inet | grep -v link tells where your packets go next. But in any case, it's helpful to know how traceroute works. It usually sends UDP packets with increasing TTLs which are supposed to elicit an ICMP error message (TTL expired) from hops along the way. The IP address you get a response from may be different from what you expect, especially when navigating the innards of your ISPs switch fabric. It's possible that it isn't even the address of any interface on any router. On the intermittent failure issue - are you running a firewall? Do you permit 67-68/udp between your gateway and the ISP? And did Surewest get acquired by XO? - M From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:30:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 168F28D4 for ; Sat, 17 May 2014 23:30:18 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id CAC8221B1 for ; Sat, 17 May 2014 23:30:17 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 4F49F3AC90 for ; Sat, 17 May 2014 16:30:17 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-net@freebsd.org Subject: arp strangeness? Date: Sat, 17 May 2014 16:30:17 -0700 Message-ID: <73317.1400369417@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:30:18 -0000 As I mentioned in my immediately prior posting here, I have been having spurious total (100%) connectivity dropouts, quite frequently, for some several weeks now. I have no idea what might be causing this, and thus I am exploring everything. Long long ago (i.e. several years ago now), I was told by my ISP that the gateway address I should be using for my DSL line (which has two static IPs associated with it) was 69.62.255.254. Thus, since that time, the following line has been present in my /etc/rc.conf file: defaultrouter="69.62.255.254" At some time in the not very distant past... a time which may or may not have coincided with the beginning of my recent connectivity problems... I began to notice arp-related messages showing up in my /var/log/messages file on a fairly routine basis. I had never seen these kinds of messages before, at least not en mass, as they seem to be appearing now. A sample of these is attached below. As you can see, whatever machine is at 69.62.255.254... which belongs to my ISP, not me... appears to be changing its physical MAC address on a routine basis, almost exactly every 20 minutes, and then immediately changing it back again. Is this normal?? What would cause this type of behavior? And more to the point, how might these frequent abrupt changes in the MAC address of the gateway machine I use to reach the rest of the Internet affect my connectivity, or lack thereof? P.S. Yes, the name of my machine is "segfault", which explains why that word is present within the following example arp messages. May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 16 23:45:25 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 16 23:45:25 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 00:05:21 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 00:05:21 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 00:25:17 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 00:25:17 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 00:45:13 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 00:45:13 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 01:05:09 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 01:05:09 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 01:25:06 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 01:25:06 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 01:45:02 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 01:45:02 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 02:04:57 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 02:04:57 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 02:24:53 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 02:24:53 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 02:44:49 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 02:44:49 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 03:04:45 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 03:04:45 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 03:24:41 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 03:24:41 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 03:44:37 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 03:44:37 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 04:04:33 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 04:04:33 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 04:24:29 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 04:44:27 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 05:04:23 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 05:04:23 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 05:24:19 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 05:24:19 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 05:44:15 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 05:44:15 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 06:04:12 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 06:04:12 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 06:24:08 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 06:24:08 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 06:44:04 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 06:44:04 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 07:04:00 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 07:04:00 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 07:23:56 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 07:23:56 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 07:43:52 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 07:43:52 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 08:03:49 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 08:03:49 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 08:23:45 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 08:23:45 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 08:43:41 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 09:03:37 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 09:23:33 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 09:23:33 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 09:43:29 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 09:43:29 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 10:03:25 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 10:03:25 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 10:23:21 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 10:23:21 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 10:43:17 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 10:43:17 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 11:03:13 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 11:03:13 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 11:23:09 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 11:23:09 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 11:43:05 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 11:43:05 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 12:03:03 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 12:03:03 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 12:22:59 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 12:22:59 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 12:42:55 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 12:42:55 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 13:02:51 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 13:02:51 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 13:22:47 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 13:22:47 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 13:42:43 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 13:42:43 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 14:02:39 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 14:02:39 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 14:22:35 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 14:22:35 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 May 17 14:42:32 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 May 17 15:02:30 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:33:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18DD0A1A for ; Sat, 17 May 2014 23:33:04 +0000 (UTC) Received: from mail-oa0-f46.google.com (mail-oa0-f46.google.com [209.85.219.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6AD6223F for ; Sat, 17 May 2014 23:33:03 +0000 (UTC) Received: by mail-oa0-f46.google.com with SMTP id i4so4642795oah.5 for ; Sat, 17 May 2014 16:32:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=mvxi0P44L2c/PzMqwvclkCzHTAeVVJX2TaFohhrNG7g=; b=M6CuCiedi45vWJTH+S1eDIF3lvDIXOLKl+gbEgn7mr7KL+VCWjbv0O4jo6RGdm3oez Z1n23AkP+URvzPnQEfTteKopbTM3za01pvliSDzeq3FmWboEBTV080hdkyCp61VhWgT6 JQ8qFHWZkdAOAOxSSGksfBXCauSFCEMQpzmZiBkT888qKsqG0wdjbK1D1KULWW6sk4nb X64rBnjzrSxif3AWFOWkVVc2eT7YB1fqf0dGi4xrxD9NR/btW9FVzV1ugrcqrtOogAbj y+OkOb7Hr63jzL4VYi/lypGNzwreoKUwaksO9243cjXvD26o7ej6PSROXQ4VWaO2dlvG gUEA== X-Gm-Message-State: ALoCoQl94/84KMRUaWNlArWGtW7P7ZidY9ra+ZuWfnHLlF22YuRzV/mUeQ5OJo1W3f1uPLQkekRz MIME-Version: 1.0 X-Received: by 10.60.159.5 with SMTP id wy5mr26303164oeb.58.1400369577642; Sat, 17 May 2014 16:32:57 -0700 (PDT) Received: by 10.60.17.33 with HTTP; Sat, 17 May 2014 16:32:57 -0700 (PDT) In-Reply-To: <73317.1400369417@server1.tristatelogic.com> References: <73317.1400369417@server1.tristatelogic.com> Date: Sat, 17 May 2014 16:32:57 -0700 Message-ID: Subject: Re: arp strangeness? From: Michael Sierchio To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:33:04 -0000 On Sat, May 17, 2014 at 4:30 PM, Ronald F. Guilmette wrote: > May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 > May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 > May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 > May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 Yeah, the router address may be a synthetic address shared by multiple physical interfaces, or it may be fictional and handled via multiple interfaces/routers/etc. in your ISPs fabric running some HA routing (via OSPF for example). It's normal. - M From owner-freebsd-net@FreeBSD.ORG Sat May 17 23:37:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DE37AD7 for ; Sat, 17 May 2014 23:37:55 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5684C2267 for ; Sat, 17 May 2014 23:37:54 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 023BC139CA for ; Sat, 17 May 2014 20:37:57 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1400369872; x=1401233873; bh=ZX7bo30XoCY9 Pq/ydFjajeEaB60rpgI50Xi2IxHoKbA=; b=EMudYWa+kFNBJciT89ecRRZWkNny hIiI+lYPFyc0u8Kk/I3a4GqgnaI0pdr5SvW2sOltkEKgSvE1WpDOZj5uD3jYUj8Y BLb+uO1cuJWYxAJqbDiEsA3kHQptCtkkkPDFHBgl0sldpSe6iFzPLgqDdXOKpbHQ EdV0V45wbvHUnD0= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G10_L9jkk0Nj for ; Sat, 17 May 2014 20:37:52 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 198E3139C9; Sat, 17 May 2014 20:37:51 -0300 (BRT) Message-ID: <5377F2CB.5010406@bsdinfo.com.br> Date: Sat, 17 May 2014 20:37:47 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , Dennis Yusupoff , FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <5377F0BB.1040501@bsdinfo.com.br> In-Reply-To: <5377F0BB.1040501@bsdinfo.com.br> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 23:37:55 -0000 Em 17/05/14 20:28, Marcelo Gondim escreveu: > Em 17/05/14 10:44, Alexander V. Chernikov escreveu: >> On 13.05.2014 16:05, Dennis Yusupoff wrote: >>> I think that universal table for all kind of data (ipv4, ipv6, ports, >>> etc) is a bad idea by design. At least unless you haven't any >>> ability to >> It is not always "universal" in kernel. >> Actually, different radix tables are used to store both IPv4 and IPv6 >> in single table. >>> specify address family on add, to avoid attempts to guess what user >>> meant. Something like "ipfw table X add DEEF.DE ipv6". >> I'm going to add explicit table type/naming setup soon. >> Idea is the following: >> >> 1) Existing table can be named and addressed by either number or name. >> However, you still need to assign table number manually. >> >> 2) Table type/name can be specified explicitly via one of the >> following commands: >> * ipfw table 1 create [type ] [name >> "table_name"] >> * ipfw table name "table_name" >> * ipfw table "table_name" type >> >> 3) ipfw(8) stops trying to guess appropriate type based on used >> value. Instead, >> it requests table type from kernel and interprets value according to >> returned type. >> Default type for all tables is cidr >> >> 4) Table(s) can be returned to default values using ipfw table >> destroy. >> Destroy means: >> * flush >> * table tries (or other structures) freed >> * type set to cidr >> >> >>> >>> >>> 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >>>> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>>>> May be this will help? See answer on >>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >>>> I'll try to fix it within a few days. >> Fixed in r266310. > The problem still exists. > > # ipfw table 99 add 0.0.0.0/8 > # ipfw table 99 list > ::/8 0 > > # uname -a > FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 r266370: > Sat May 17 19:57:23 BRT 2014 > root@mail.xxxxxx.br:/usr/obj/usr/src/sys/GONDIM amd64 Ah! Sorry! Is still in the head. From owner-freebsd-net@FreeBSD.ORG Sun May 18 00:38:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F435611 for ; Sun, 18 May 2014 00:38:40 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 6BB1B26A8 for ; Sun, 18 May 2014 00:38:39 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id B98833AC90 for ; Sat, 17 May 2014 17:38:38 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-net@freebsd.org Subject: Re: Gateway? Date: Sat, 17 May 2014 17:38:38 -0700 Message-ID: <73579.1400373518@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 00:38:40 -0000 Michael Sierchio kudzu at tenebras.com wrote: >On Sat, May 17, 2014 at 4:17 PM, Ronald F. Guilmette > wrote: > >> Quite simply, I'd like to know if the defaultrouter= IPv4 address >> specified in my /etc/rc.conf file should be the same as whatever >> I normally see as the first hop in an outgoing traceroute. > >Maybe... see comments below. > >> defaultrouter="69.62.255.254" >> >> and here is one example of a recent outgoing traceroute: >> >> % traceroute 74.125.239.148 >> traceroute to 74.125.239.148 (74.125.239.148), 64 hops max, 52 byte packets >> 1 86.255-62-69.res.dyn.surewest.net (69.62.255.86) 28.884 ms 31.395 ms 30.024 ms >> 2 216.0.55.209 (216.0.55.209) 26.486 ms 26.024 ms 25.850 ms > >Do you have a fixed IP address (statically assigned), Yes, I do. two of them in fact... 69.62.255.118 and 69.62.255.119. >or are you getting an address via DHCP from your ISP? No. I do use DHCP within my local network, but my connection(s) to my ISP are from my two static IPs. >If it's DHCP, your >defaultrouter definition is overridden every time you get/renew a lease. Right. Actually, I did know at least that much. >netstat -r -n -f inet | grep -v link > >tells where your packets go next. Yes, and my default route is definitely pointed at 69.62.255.254: ========================================================================== Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 69.62.255.254 UGS 0 1615581 rl0 69.62.255.0/24 link#3 U 0 32 rl0 69.62.255.118 link#3 UHS 0 98 lo0 127.0.0.1 link#7 UH 0 18465739 lo0 192.168.1.0/24 link#4 U 0 7248 nfe0 192.168.1.2 link#4 UHS 0 0 lo0 ========================================================================== >But in any case, it's helpful to >know how traceroute works. It usually sends UDP packets with >increasing TTLs which are supposed to elicit an ICMP error message >(TTL expired) from hops along the way. I did actually know that much (but admittedly not much more). >The IP address you get a >response from may be different from what you expect, especially when >navigating the innards of your ISPs switch fabric. It's possible that >it isn't even the address of any interface on any router. If you could explain all of what you just said a bit more for me, I'm sure that it would be both enlightening and educational for me. (But if you don't have time, that's OK too. I understand that I should probably go buy a nice thick book or enroll in a two-semester course if I seriously wanted to understand this stuff in depth.) >On the intermittent failure issue - are you running a firewall? Yes. >Do you permit 67-68/udp between your gateway and the ISP? I'm not sure I underatand the question. The gateway machine belongs to the ISP. I do not have any control over which packets they allow to pass between one of their own machines and any other. Off list I'll send you output from "ipfw -a list" and you can tell me what you think. In general, traceroute _does_ appear to work OK on this system. >And did Surewest get acquired by XO? No, a company called "Consolidated Communications": http://www.consolidated.com/ (It used to be that when I would call tech support for Surewest, formerly Roseville Telephone... a little tiny island of helpful friendliness bounded on all sides by a veritable sea of PacBell unhelpfulness... I could actually talk to somebody local here in Roseville who actually knew something about networking... even when I called on the weekends. Today however, when I called tech support the first level ``support'' girl... yes, I'm sexist, and I said ``girl''... told me that she was located in Florida, and she quite clearly knew absolutely nothing about nothing, so I asked to speak to a second level person. This time it was a guy, who said he was in Texas, and he also quite clearly knew absolutely nothing. The only even vaguely useful thing he told me was that he ran some test on my line and the result came back at 9.6, which he said was borderline, adding that a really good number would be between 15 and 20. When I asked him to what these numbers referred, and what they were measurements of, he confessed that he had no idea, adding that "I'm not an engineer." Geeezzzz Louise! This caged monkey was looking at numbers and giving out numbers, and he doesn't even have the vaguest idea what they even mean or what they measure! Oh well, neither do I. But then again I'm not being *paid* to do ISP tech support. Geeezzzzzzz! Well, so he said that he'd open a ticket and send a trouble request to some "other department"... presumably the people who actually do know something about ISP operations... and then, after making me wait another 20 minutes on hold, he FINALLY gave me the trouble ticket number I had insisted upon obtaining. In short, things have definitely gone downhill in a big way. I'm just about ready to chuck it all and jump into the open arms of the Last Mile Evil Empire, aka Comcast.) From owner-freebsd-net@FreeBSD.ORG Sun May 18 01:39:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6E4913A for ; Sun, 18 May 2014 01:39:53 +0000 (UTC) Received: from mail.in-addr.com (noop.in-addr.com [208.58.23.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7F442B99 for ; Sun, 18 May 2014 01:39:53 +0000 (UTC) Received: from gjp by mail.in-addr.com with local (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WlpcL-000JDP-4F; Sat, 17 May 2014 21:10:21 -0400 Date: Sat, 17 May 2014 21:10:21 -0400 From: Gary Palmer To: "Ronald F. Guilmette" Subject: Re: Gateway? Message-ID: <20140518011020.GB56618@in-addr.com> References: <73288.1400368632@server1.tristatelogic.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <73288.1400368632@server1.tristatelogic.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 01:39:54 -0000 On Sat, May 17, 2014 at 04:17:12PM -0700, Ronald F. Guilmette wrote: > > Forgive me, please for such a rudimentary sort of question. I've > been doing IP networking for more than 15 years, but I never really > plumbed the depths, and thus I only know the basics. > > Quite simply, I'd like to know if the defaultrouter= IPv4 address > specified in my /etc/rc.conf file should be the same as whatever > I normally see as the first hop in an outgoing traceroute. I ask > because for me these two are apparently not the same. Here's what > I have: > > defaultrouter="69.62.255.254" > > and here is one example of a recent outgoing traceroute: > > % traceroute 74.125.239.148 > traceroute to 74.125.239.148 (74.125.239.148), 64 hops max, 52 byte packets > 1 86.255-62-69.res.dyn.surewest.net (69.62.255.86) 28.884 ms 31.395 ms 30.024 ms > 2 216.0.55.209 (216.0.55.209) 26.486 ms 26.024 ms 25.850 ms > 3 ae1d0.mcr1.roseville-ca.us.xo.net (216.156.1.77) 25.384 ms 27.298 ms 27.060 ms > 4 vb1510.rar3.sanjose-ca.us.xo.net (216.156.0.153) 27.289 ms 34.022 ms 36.213 ms > 5 207.88.14.226.ptr.us.xo.net (207.88.14.226) 26.993 ms 26.567 ms 25.568 ms > 6 216.156.84.30.ptr.us.xo.net (216.156.84.30) 24.800 ms 26.432 ms 25.845 ms > 7 209.85.249.5 (209.85.249.5) 26.033 ms 110.066 ms 28.663 ms > 8 66.249.95.31 (66.249.95.31) 26.985 ms 25.285 ms 28.066 ms > 9 nuq05s02-in-f20.1e100.net (74.125.239.148) 26.895 ms 27.434 ms 27.063 ms They could be using some kind of HSRP or VRRP, and the above behaviour would be normal (at least it was the last time I ran HSRP). You set the route to the failover address, but traceroute sees the response from the actual interface address on whichever gateway handles the packet. e.g. router 1 interface 0 has IP x.x.x.2 and is the HSRP master router 2 interface 0 has IP x.x.x.3 and is the HSRP backup HSRP IP is x.x.x.1 Traceroute out and you normally see the answer from the first hop as x.x.x.2 There are other tricks they could be doing as well which would also give the above behaviour. Regards, Gary From owner-freebsd-net@FreeBSD.ORG Sun May 18 04:09:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F974229 for ; Sun, 18 May 2014 04:09:39 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1DD182591 for ; Sun, 18 May 2014 04:09:38 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4I49RFw053478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 17 May 2014 21:09:30 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53783271.6090409@freebsd.org> Date: Sun, 18 May 2014 12:09:21 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Michael Sierchio , "freebsd-net@freebsd.org" Subject: Re: arp strangeness? References: <73317.1400369417@server1.tristatelogic.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 04:09:39 -0000 On 5/18/14, 7:32 AM, Michael Sierchio wrote: > On Sat, May 17, 2014 at 4:30 PM, Ronald F. Guilmette > wrote: > >> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from > 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from > 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 >> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from > 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from > 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 > > Yeah, the router address may be a synthetic address shared by multiple > physical interfaces, or > it may be fictional and handled via multiple interfaces/routers/etc. in > your ISPs fabric running some HA > routing (via OSPF for example). but check with your ISP that your information is current. It may be that you should be using another address and this one is just working by accident. > > It's normal. > > - M > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun May 18 04:13:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F112B2C7; Sun, 18 May 2014 04:13:29 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A666B2611; Sun, 18 May 2014 04:13:29 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4I4CeRF053502 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 17 May 2014 21:12:43 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53783333.3010205@freebsd.org> Date: Sun, 18 May 2014 12:12:35 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , Dennis Yusupoff , FreeBSD Net , Marcelo Gondim Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> In-Reply-To: <537767C5.80205@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 04:13:30 -0000 On 5/17/14, 9:44 PM, Alexander V. Chernikov wrote: > On 13.05.2014 16:05, Dennis Yusupoff wrote: >> I think that universal table for all kind of data (ipv4, ipv6, ports, >> etc) is a bad idea by design. At least unless you haven't any >> ability to > It is not always "universal" in kernel. > Actually, different radix tables are used to store both IPv4 and > IPv6 in single table. >> specify address family on add, to avoid attempts to guess what user >> meant. Something like "ipfw table X add DEEF.DE ipv6". > I'm going to add explicit table type/naming setup soon. > Idea is the following: > > 1) Existing table can be named and addressed by either number or name. > However, you still need to assign table number manually. > > 2) Table type/name can be specified explicitly via one of the > following commands: > * ipfw table 1 create [type ] [name > "table_name"] type "ports" would be nice but tricky to do right. > * ipfw table name "table_name" > * ipfw table "table_name" type > > 3) ipfw(8) stops trying to guess appropriate type based on used > value. Instead, > it requests table type from kernel and interprets value according to > returned type. > Default type for all tables is cidr the guessing was a hack for compatibilty. Its time to stop doing that has definitely come. (I did it.. sorry) > > 4) Table(s) can be returned to default values using ipfw table > destroy. > Destroy means: > * flush > * table tries (or other structures) freed > * type set to cidr > > >> >> >> 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >>> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>>> May be this will help? See answer on >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >>> I'll try to fix it within a few days. > Fixed in r266310. >>> >>> The problem itself happens due to the fact that every CIDR table >>> address is packed into IPv6 address and IPv4 ones are encoded as >>> deprecated IPv6-compatible ones. >>> this leads to the problems with decoding things like 0/X or ::1 > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Sun May 18 04:47:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F35F1853 for ; Sun, 18 May 2014 04:47:32 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 99BA8284B for ; Sun, 18 May 2014 04:47:32 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id tp5so1157300ieb.39 for ; Sat, 17 May 2014 21:47:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=nA119dHfM8Hy0sPyjX3eJA5nUaWxqZl9Fwk1mv1o7dA=; b=ZGlhQvtbWm7jlMesLnxCBcuYKth6Q3+wkV7CUrjQJIVhCdYljApo18/lOf0Pk01Ge/ YFdJp/cPitRVqKJQ+rLpjqPKn4G16GTxj4n3R07iha9EzskxEuldxOZHFYhGeUtg3/Ke EEW3Ys4CR6F7Dklw9WmJW1OrqeeFWcfCPFbz0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=nA119dHfM8Hy0sPyjX3eJA5nUaWxqZl9Fwk1mv1o7dA=; b=Frqtx7QdS0v5bez1GXIG4mt0Vk7/J33rO7XKwpYP+/pLb+UtAWFL8YG+dPdTAq0XEN cErlgWGGahKm/KDRHaX9BRghTkTNHxk5uDvOyjSenNr4OSO3ja30bocAOG4kKhVB8ilR MwLRz+tKuaMYwfrYsW3qLmlJCtDDASRNimNFpSs44fTJkfvP3Cp+0YF7uvdyhtKbONgl 0UwX/tfNTuQCLJrnglVXerl/M/57jRqfScJ2Fop/Jb8vqu4MDagLRRGNPHvgTHCaVuQC +IkEpl5fm8Dr99/jgu2WAk0LEjSbKTgdFOSkYF/KpP5f3WQprfWuWLUvXZwF/d7gBoBC TAjg== X-Gm-Message-State: ALoCoQmteYtrFTSUmNttdicq3Omk3H1dLiOdQ00BpyQduzotC8ysAb+JmhB8NBCHSP2XJBKGKScZ X-Received: by 10.43.93.5 with SMTP id bs5mr25488975icc.11.1400388451365; Sat, 17 May 2014 21:47:31 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id d6sm3253552igr.12.2014.05.17.21.47.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 17 May 2014 21:47:30 -0700 (PDT) References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> Mime-Version: 1.0 (1.0) In-Reply-To: <53783333.3010205@freebsd.org> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-5F80BD8F-633C-4C3E-829A-4C7958CB0A2E; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: Problem with ipfw table add 0.0.0.0/8 Date: Sun, 18 May 2014 00:47:28 -0400 To: Julian Elischer Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim , "Alexander V. Chernikov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 04:47:33 -0000 --Apple-Mail-5F80BD8F-633C-4C3E-829A-4C7958CB0A2E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable > On May 18, 2014, at 0:12, Julian Elischer wrote: >> 2) Table type/name can be specified explicitly via one of the following c= ommands: >> * ipfw table 1 create [type ] [name "table_name"]= > type "ports" would be nice but tricky to do right. That . . . would be a great addition and have me switching from pf to ipfw. Pullllease do! :-)= --Apple-Mail-5F80BD8F-633C-4C3E-829A-4C7958CB0A2E Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDUxODA0NDcyOVowIwYJKoZIhvcN AQkEMRYEFFVzAldksK8so4SsvjU0Z4CnhMTAMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAiZIW7wXuPRqORO0m9IrE tvOjeH8J+BOP1pwzscMdztSm/JqqPA1NWiTPrhCEg6EnRfMQbSvySnrXE5xRYR5X6/5Kip046k11 8erTkUTIdcI8eQp5qfJTUrijss5OTqV6Bo5w0vJLB9+yvlzfvHhAKGy4wPrFrre8UJw2mGpBkIDt cOyJ6CChEsPVlBTyaxI3Jr77cMGsZjalrnt6lgCd023aWlf/I2AygYuYDBB/Wt9HpcF1DuYZnhaD N9YG8YZWn2keHAHjjOQzHX7CvIGkiK6pbJgDMSjUYmiMDJ87pFNuDcGzQ09I8Un3deQ2sdBY+ipA Jq2Vh13mR2wuGh/fiQAAAAAAAA== --Apple-Mail-5F80BD8F-633C-4C3E-829A-4C7958CB0A2E-- From owner-freebsd-net@FreeBSD.ORG Sun May 18 11:29:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C56A1870 for ; Sun, 18 May 2014 11:29:35 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 635E621BA for ; Sun, 18 May 2014 11:29:35 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n15so2872305wiw.8 for ; Sun, 18 May 2014 04:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=8wOTCNL7d9F2ve+iUJ4XYP7isrXEoiKrwtJYm1mBhFU=; b=xe+S4j30Lh4VGGnSy1Ljpx1ZMMrc2jrsmQjaY93fMo7BQVBgS3kQISzUinLGUWjDBk RNxYjhXq1xBnN04odMYMLptVyic9czXkO7b62XEO9x5LFZOMfKJSn24xO8hlzgVtTYE8 r/xwOacjJBO+JScgkIK3wMv6rfyMbhbZhTV196wB4LkUApjaOm7ATwT4l/QiJy+twSgw VBgA6Iq/USwJvgiRW08uro6fvcVhpJaDjVSHwMmlLagvpxk3IY0VuAXNC9xq1zlPYPj6 EkbdKDSVBXBdoEVQUnxLTCLreCZqg2OCbe3acFQpvcKSudypifi9TkaAybLZ1mlSXstC Ek3g== MIME-Version: 1.0 X-Received: by 10.180.183.5 with SMTP id ei5mr4318665wic.54.1400412573214; Sun, 18 May 2014 04:29:33 -0700 (PDT) Received: by 10.194.92.144 with HTTP; Sun, 18 May 2014 04:29:33 -0700 (PDT) Date: Sun, 18 May 2014 16:59:33 +0530 Message-ID: Subject: Regarding Netmap on Fedora From: Prashant Upadhyaya To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 11:29:35 -0000 Hi, I am using Fedora 18 with Intel 82599 NIC. I am new to netmap and want to experiment with it in the above environment. Can somebody please advise what is the easiest way to go about the above (installation etc.) so that I can concentrate on writing the userspace programs using netmap quickly. Regards -Prashant From owner-freebsd-net@FreeBSD.ORG Sun May 18 12:24:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C237437C for ; Sun, 18 May 2014 12:24:06 +0000 (UTC) Received: from land.berklix.org (land.berklix.org [144.76.10.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5161525CA for ; Sun, 18 May 2014 12:24:05 +0000 (UTC) Received: from mart.js.berklix.net (p57BCF036.dip0.t-ipconnect.de [87.188.240.54]) (authenticated bits=128) by land.berklix.org (8.14.5/8.14.5) with ESMTP id s4ICN8Nm025405; Sun, 18 May 2014 12:23:09 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id s4ICNlmY052790; Sun, 18 May 2014 14:23:47 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.7/8.14.7) with ESMTP id s4ICNTAf097378; Sun, 18 May 2014 14:23:41 +0200 (CEST) (envelope-from jhs@berklix.com) Message-Id: <201405181223.s4ICNTAf097378@fire.js.berklix.net> To: Prashant Upadhyaya Subject: Re: Regarding Netmap on Fedora From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultants, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sun, 18 May 2014 16:59:33 +0530." Date: Sun, 18 May 2014 14:23:29 +0200 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 12:24:06 -0000 > I am using Fedora 18 with Intel 82599 NIC. .............^^^^^^^^^ Ask on a Linux/Fedora list then. This is a FreeBSD list. > I am new to netmap and want to experiment with it in the above environment. > Can somebody please advise what is the easiest way to go about the above > (installation etc.) so that I can concentrate on writing the userspace > programs using netmap quickly. Cheers, Julian -- Julian Stacey, BSD Linux Unix C Sys Eng Consultant Munich http://berklix.com Interleave reply paragraphs like a play script. http://berklix.eu/pirates/ - A daft name but good ideas, read before voting. From owner-freebsd-net@FreeBSD.ORG Sun May 18 20:00:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA124625; Sun, 18 May 2014 20:00:31 +0000 (UTC) Received: from zimbra.nitronet.pl (zimbra.nitronet.pl [79.98.150.2]) by mx1.freebsd.org (Postfix) with ESMTP id 6F1F626CF; Sun, 18 May 2014 20:00:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id 14FE460CC3; Sun, 18 May 2014 21:54:14 +0200 (CEST) Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 8Y2bjIIXeopw; Sun, 18 May 2014 21:54:13 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id 0C071616FE; Sun, 18 May 2014 21:54:13 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.8.4 zimbra.nitronet.pl 0C071616FE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nitronet.pl; s=C6C9846A-6E8C-11E3-A3F3-239CBF155B7D; t=1400442853; bh=BCnotBZf2sSZnieCc+h8AePozmJFcAGqkVCSXlq6UBI=; h=Date:From:Message-ID:To:Subject:MIME-Version:Content-Type: Content-Transfer-Encoding; b=wvqGHPMlBrTRyJFQZ/8I6ba743V6FZONUue8ivp3W1VTfRuraH5haxMRtkxPsdTwo gPYUl54r1s6CAzr7hOke9qMmVQdXwfiG1PzuVrvphUX+T63qDT0CJeRwRozFqFUMec z323wCzhuH0LbrITCXHlBB/GWlQ10UQh8GVlKb0E= X-Virus-Scanned: amavisd-new at zimbra.nitronet.pl Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id EQTkXIBola1w; Sun, 18 May 2014 21:54:12 +0200 (CEST) Received: from hC35A6B23.cli.nitronet.pl (hC35A6B23.cli.nitronet.pl [195.90.107.35]) by zimbra.nitronet.pl (Postfix) with ESMTPSA id E393C60CC3; Sun, 18 May 2014 21:54:12 +0200 (CEST) Date: Sun, 18 May 2014 21:52:43 +0200 From: =?windows-1250?Q?Pawe=B3_Tyll?= X-Priority: 3 (Normal) Message-ID: <119041575.20140518215243@ofca.me> To: freebsd-stable@freebsd.org, freebsd-net@freebsd.org Subject: Periodic panics (again) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 20:00:32 -0000 Hi guys, I'm experiencing (somewhat random) panics every few days. FreeBSD 10.0-STABLE #0 r265785 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x378 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff808ef4f9 stack pointer = 0x28:0xfffffe034a534960 frame pointer = 0x28:0xfffffe034a5349f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (dummynet) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff8092ba30 at kdb_backtrace+0x60 #1 0xffffffff808f0f85 at panic+0x155 #2 0xffffffff80d01a5f at trap_fatal+0x38f #3 0xffffffff80d01d78 at trap_pfault+0x308 #4 0xffffffff80d01430 at trap+0x4a0 #5 0xffffffff80ce7f32 at calltrap+0x8 #6 0xffffffff80a1ca77 at ip_input+0x4a7 #7 0xffffffff809bbae2 at netisr_dispatch_src+0x62 #8 0xffffffff80adf96c at dummynet_send+0x10c #9 0xffffffff80adf584 at dummynet_task+0x2c4 #10 0xffffffff8093a095 at taskqueue_run_locked+0xe5 #11 0xffffffff8093ab28 at taskqueue_thread_loop+0xa8 #12 0xffffffff808c1b1a at fork_exit+0x9a #13 0xffffffff80ce846e at fork_trampoline+0xe (kgdb) #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff808f0c02 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452 #2 0xffffffff808f0fc4 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff80d01a5f in trap_fatal (frame=, eva=) at /usr/src/sys/amd64/amd64/trap.c:881 #4 0xffffffff80d01d78 in trap_pfault (frame=0xfffffe034a5348b0, usermode=) at /usr/src/sys/amd64/amd64/trap.c:692 #5 0xffffffff80d01430 in trap (frame=0xfffffe034a5348b0) at /usr/src/sys/amd64/amd64/trap.c:456 #6 0xffffffff80ce7f32 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff808ef4f9 in __rw_rlock (c=0xfffff80063753438, file=0x0, line=0) at /usr/src/sys/kern/kern_rwlock.c:420 #8 0xffffffff80a1ca77 in ip_input (m=0xfffff8010d7fc800) at /usr/src/sys/netinet/ip_input.c:593 #9 0xffffffff809bbae2 in netisr_dispatch_src (proto=, source=, m=0x0) at /usr/src/sys/net/netisr.c:972 #10 0xffffffff80adf96c in dummynet_send (m=) at /usr/src/sys/netpfil/ipfw/ip_dn_io.c:665 #11 0xffffffff80adf584 in dummynet_task (context=, pending=) at /usr/src/sys/netpfil/ipfw/ip_dn_io.c:625 #12 0xffffffff8093a095 in taskqueue_run_locked (queue=0xfffff8005d7a5e00) at /usr/src/sys/kern/subr_taskqueue.c:342 #13 0xffffffff8093ab28 in taskqueue_thread_loop (arg=) at /usr/src/sys/kern/subr_taskqueue.c:563 #14 0xffffffff808c1b1a in fork_exit ( callout=0xffffffff8093aa80 , arg=0xffffffff8151b758, frame=0xfffffe034a534c00) at /usr/src/sys/kern/kern_fork.c:995 #15 0xffffffff80ce846e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #16 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) Full dump is available (actually four of them at this time) if more info is needed. Any ideas? Kind regards. From owner-freebsd-net@FreeBSD.ORG Sun May 18 22:36:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACC26383 for ; Sun, 18 May 2014 22:36:57 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 9192F223E for ; Sun, 18 May 2014 22:36:56 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id BBC093AD48 for ; Sun, 18 May 2014 15:36:52 -0700 (PDT) From: "Ronald F. Guilmette" To: "freebsd-net@freebsd.org" Subject: Re: arp strangeness? In-Reply-To: <53783271.6090409@freebsd.org> Date: Sun, 18 May 2014 15:36:52 -0700 Message-ID: <1737.1400452612@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 22:36:57 -0000 In message <53783271.6090409@freebsd.org>, Julian Elischer wrote: >On 5/18/14, 7:32 AM, Michael Sierchio wrote: >> On Sat, May 17, 2014 at 4:30 PM, Ronald F. Guilmette >> wrote: >> >>> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from >> 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >>> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from >> 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 >>> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from >> 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >>> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from >> 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 >> >> Yeah, the router address may be a synthetic address shared by multiple >> physical interfaces, or >> it may be fictional and handled via multiple interfaces/routers/etc. in >> your ISPs fabric running some HA >> routing (via OSPF for example). > >but check with your ISP that your information is current. >It may be that you should be using another address and this one is >just working by accident. Mostly because I am having ongoing connectivity issues, I did in fact have a phone conversion, at last, with some actually knowledgable person(s) at my ISP (Surewest aka Consolidated Communications), and among the many questions I asked, I did also ask if the old (ancient?) gateway address I had been using was still the current and proper one, and sure enough, no, the current proper one is now the .1 address within the /24 I happen to be located in. So I've changed that now. (I do thank you for the suggestion, but I had already planned to ask them if I was using the correct gateway address.) Oddly, even though I now have the defaultrouter address in my /etc/rc.conf file set to the .1 address, _now_ my traceroutes are showing the .2 address in this same /24 as the first hop. Oh well, I'm not going to worry about it. I have bigger fish to fry. (Specifically, my ISP now informs me that there are definite problems with either my ADSL2+ router, or my line, or both, and I will be working with them to correct those issues.) From owner-freebsd-net@FreeBSD.ORG Sun May 18 23:06:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE3D5CFA for ; Sun, 18 May 2014 23:06:22 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 8059524A4 for ; Sun, 18 May 2014 23:06:22 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id CBBC77300B; Mon, 19 May 2014 01:02:42 +0200 (CEST) Date: Mon, 19 May 2014 01:02:42 +0200 From: Luigi Rizzo To: freebsd-net@freebsd.org Subject: netmap and other discussions on freebsd-net: please be open minded. Message-ID: <20140518230242.GA28117@onelab2.iet.unipi.it> References: <201405181223.s4ICNTAf097378@fire.js.berklix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201405181223.s4ICNTAf097378@fire.js.berklix.net> User-Agent: Mutt/1.5.20 (2009-06-14) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 23:06:22 -0000 Folks, i have two requests for you: 1. please do not complain about questions on this list related to a core network-related FreeBSD subsystem (netmap, dummynet, netgraph, tcp stack...) even if they are concerned with ports to Linux or other OSes or to userspace. At least in principle, these topics _are_ relevant here precisely because they relate to code whose main home is in the FreeBSD source tree, and bugs or feature suggestions etc that may emerge will directly benefit FreeBSD. 2. on the other hand, it would be good if people could avoid questions like "give me step by step instructions on how to install/run/use X". There are notes in README and Makefiles, something on the web pages, and quick web searches should point you to previous mailing list discussions on the same topic _before asking_. So in the specific case below i can understand a reply (perhaps a private one) saying "have you done your homework (read documentation, do a web search) ?", but "we only do FreeBSD here" does not sound right to me. cheers luigi On Sun, May 18, 2014 at 02:23:29PM +0200, XXX wrote: > > I am using Fedora 18 with Intel 82599 NIC. > .............^^^^^^^^^ > > Ask on a Linux/Fedora list then. This is a FreeBSD list. > > > I am new to netmap and want to experiment with it in the above environment. > > Can somebody please advise what is the easiest way to go about the above > > (installation etc.) so that I can concentrate on writing the userspace > > programs using netmap quickly. > > > Cheers, > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun May 18 23:44:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 960862B8; Sun, 18 May 2014 23:44:24 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A30F2774; Sun, 18 May 2014 23:44:24 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so7529598qge.36 for ; Sun, 18 May 2014 16:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=adycisgBehZMJaGd7jW/wmspl8OHPL4fhmk6fxrpFBM=; b=ix3fyDLVicTJn3vfd/V4+JYshmwuN1Nj5W0YYFpUmrzW+Kf/Tw4Ogi34Qsc5RdHH9G UA4ee3QpRnwvGEm8QBQxX/JWgbK+oeiNIk062dUgxZ2retGwlR4OB/F0LvYZ+kR/nV7P wl6fk9uVthG7bVPwv+KNBxzumc9LEBbYai/CPJxrLXFATy4RQMqDSHZYUUQi4LQfkUxc 6izNA/bwoYRMW9DOxhqXNntV/M3LcUkkb2PSsNMWYP/1kiquF3LOIjPN+1co8gvxeQFD gjtGoJQBWoC40BpaxJQoLHcF4wRnBSlUUDLUWgFt/JQMtH4NTVrjzs2vLc1JXSSbLGGY o/9w== MIME-Version: 1.0 X-Received: by 10.224.137.1 with SMTP id u1mr42220050qat.73.1400456663494; Sun, 18 May 2014 16:44:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 18 May 2014 16:44:23 -0700 (PDT) Date: Sun, 18 May 2014 16:44:23 -0700 X-Google-Sender-Auth: wWW3g-T1NLhWgsHe8cBIux6F_Vs Message-ID: Subject: [rfc] teach netstat about flowid/flowtype From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 23:44:24 -0000 Hi! My next hack - adding flowid/flowtype to netstat. It's netstat -R. It's like -x, but listing RSS/flowid stuff. http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff This is useful for both RSS and non-RSS debugging - NICs that populate the mbuf flowid will start to show flowid's up in netstat. It doesn't currently show the rss CPUID as that isn't cached in the inpcb. I'll worry about adding that later on. Any issues with me just committing this? I promise I'll email doc@ to get it documented. Thanks, -a From owner-freebsd-net@FreeBSD.ORG Sun May 18 23:49:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2FA341F for ; Sun, 18 May 2014 23:49:17 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6399727A0 for ; Sun, 18 May 2014 23:49:17 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so7786746qgf.31 for ; Sun, 18 May 2014 16:49:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=DydVqbZxBxneHtDELvqQ6HPez/q4In4lPWHknQoMKZ8=; b=Xem2IKD5Fh54APF8mbte+X7c1AusL3DCxSBLDUg9HeTXWYlLZr64FgYODOh7ZqBBAB BZrinYOCtmQY3Fe33xJJWpWbXG4KYLuMlUCQn+C1G3nh8/q2CD6+teSEf3BB6O7CMsDU BDIIGQc/udCb/BToyLO25nfQ+GN27QuR6njyn4c8FJa1D4SI+8P0gKSq2hv8Wiac9taR aYZdWznxZckFgqvbKXsK25yoXJnrH2U7OR93fE10ho9ObozFhFK8jiLtUnyZH2s2Csjf HjD1Zluq4E1OeEDmx0TMz9TyHc/ppB6Ws8YJlvJ/mV2yQaP/ukuaeBYirhVTmXQRGo3n BZrA== MIME-Version: 1.0 X-Received: by 10.229.97.71 with SMTP id k7mr45042205qcn.4.1400456956555; Sun, 18 May 2014 16:49:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 18 May 2014 16:49:16 -0700 (PDT) In-Reply-To: <20140518230242.GA28117@onelab2.iet.unipi.it> References: <201405181223.s4ICNTAf097378@fire.js.berklix.net> <20140518230242.GA28117@onelab2.iet.unipi.it> Date: Sun, 18 May 2014 16:49:16 -0700 X-Google-Sender-Auth: wODOpEP216PMrTwKSUE5V0Bhr_I Message-ID: Subject: Re: netmap and other discussions on freebsd-net: please be open minded. From: Adrian Chadd To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 23:49:17 -0000 Hi! Is there a netmap list that these questions (regardless of OS) could go to? -a On 18 May 2014 16:02, Luigi Rizzo wrote: > Folks, i have two requests for you: > > 1. please do not complain about questions on this list related > to a core network-related FreeBSD subsystem (netmap, dummynet, > netgraph, tcp stack...) even if they are concerned with ports > to Linux or other OSes or to userspace. > > At least in principle, these topics _are_ relevant here precisely > because they relate to code whose main home is in the FreeBSD > source tree, and bugs or feature suggestions etc that may emerge > will directly benefit FreeBSD. > > 2. on the other hand, it would be good if people could avoid questions like > "give me step by step instructions on how to install/run/use X". > There are notes in README and Makefiles, something on the > web pages, and quick web searches should point you to previous > mailing list discussions on the same topic _before asking_. > > So in the specific case below i can understand a reply (perhaps a > private one) saying "have you done your homework (read documentation, > do a web search) ?", but "we only do FreeBSD here" does not sound > right to me. > > cheers > luigi > > On Sun, May 18, 2014 at 02:23:29PM +0200, XXX wrote: >> > I am using Fedora 18 with Intel 82599 NIC. >> .............^^^^^^^^^ >> >> Ask on a Linux/Fedora list then. This is a FreeBSD list. >> >> > I am new to netmap and want to experiment with it in the above environment. >> > Can somebody please advise what is the easiest way to go about the above >> > (installation etc.) so that I can concentrate on writing the userspace >> > programs using netmap quickly. >> >> >> Cheers, >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon May 19 01:36:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B82D98AD for ; Mon, 19 May 2014 01:36:11 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6EEE32F2F for ; Mon, 19 May 2014 01:36:11 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4J1a7xQ057096 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 18 May 2014 18:36:09 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53796003.9030306@freebsd.org> Date: Mon, 19 May 2014 09:36:03 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Ronald F. Guilmette" , "freebsd-net@freebsd.org" Subject: Re: arp strangeness? References: <1737.1400452612@server1.tristatelogic.com> In-Reply-To: <1737.1400452612@server1.tristatelogic.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 01:36:11 -0000 On 5/19/14, 6:36 AM, Ronald F. Guilmette wrote: > In message <53783271.6090409@freebsd.org>, > Julian Elischer wrote: > >> On 5/18/14, 7:32 AM, Michael Sierchio wrote: >>> On Sat, May 17, 2014 at 4:30 PM, Ronald F. Guilmette >>> wrote: >>> >>>> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from >>> 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >>>> May 16 23:05:33 segfault kernel: arp: 69.62.255.254 moved from >>> 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 >>>> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from >>> 00:1e:13:22:eb:51 to 00:00:0e:07:ac:00 on rl0 >>>> May 16 23:25:29 segfault kernel: arp: 69.62.255.254 moved from >>> 00:00:0e:07:ac:00 to 00:1e:13:22:eb:51 on rl0 >>> >>> Yeah, the router address may be a synthetic address shared by multiple >>> physical interfaces, or >>> it may be fictional and handled via multiple interfaces/routers/etc. in >>> your ISPs fabric running some HA >>> routing (via OSPF for example). >> but check with your ISP that your information is current. >> It may be that you should be using another address and this one is >> just working by accident. > Mostly because I am having ongoing connectivity issues, I did in fact > have a phone conversion, at last, with some actually knowledgable > person(s) at my ISP (Surewest aka Consolidated Communications), and > among the many questions I asked, I did also ask if the old (ancient?) > gateway address I had been using was still the current and proper one, > and sure enough, no, the current proper one is now the .1 address within > the /24 I happen to be located in. So I've changed that now. (I do > thank you for the suggestion, but I had already planned to ask them > if I was using the correct gateway address.) > > Oddly, even though I now have the defaultrouter address in my /etc/rc.conf > file set to the .1 address, _now_ my traceroutes are showing the .2 address > in this same /24 as the first hop. Oh well, I'm not going to worry about it. > I have bigger fish to fry. it is possible that the router replies with "ONE" of its addreses.. not necessarily the one closest to you. I've seen that before. > (Specifically, my ISP now informs me that there are definite problems > with either my ADSL2+ router, or my line, or both, and I will be working > with them to correct those issues.) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon May 19 01:38:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D142099A; Mon, 19 May 2014 01:38:47 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A11002F3E; Mon, 19 May 2014 01:38:47 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4J1cgcW057102 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 18 May 2014 18:38:45 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5379609E.6000702@freebsd.org> Date: Mon, 19 May 2014 09:38:38 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Adrian Chadd , "freebsd-arch@freebsd.org" , FreeBSD Net Subject: Re: [rfc] teach netstat about flowid/flowtype References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 01:38:47 -0000 On 5/19/14, 7:44 AM, Adrian Chadd wrote: > Hi! > > My next hack - adding flowid/flowtype to netstat. It's netstat -R. > It's like -x, but listing RSS/flowid stuff. > > http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff > > This is useful for both RSS and non-RSS debugging - NICs that populate > the mbuf flowid will start to show flowid's up in netstat. > > It doesn't currently show the rss CPUID as that isn't cached in the > inpcb. I'll worry about adding that later on. > > Any issues with me just committing this? I promise I'll email doc@ to > get it documented. If you don't I'll get "you-know-who" to nag you.. I need a good Italian meal some time.. > > Thanks, > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon May 19 05:12:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAA344B5; Mon, 19 May 2014 05:12:34 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE862FD1; Mon, 19 May 2014 05:12:34 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so7877734qgf.7 for ; Sun, 18 May 2014 22:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XGbmWea/pHBL1Y2cyazgKn0kbT41OrKge0U6tO7daH0=; b=ZXN0eAbCtBDBCbn+tNan9oJ813Me2RxryZedKH9+zNc+I1YmGE9Me+DzOZw6mslL23 vs3WTou1tvaHjsVBik6uOLZ5+lQF8coNOKBPcRziNT4XlbVaqO8i6XJXFjYZbAHAfH2x WXfd/JEva/LX1fOjV7B+alpf8XFz5KPIEBJGJ0Cs15cPONzz+CzSv5UzJr2273JUNadn OY6sT0vmtC57O22vsBbpiicLqJBZAyAm5zuyhNtdG2JifjDy8kfUvzvpJtMJnpPT3yx9 ob39u1EYxgj7BW6LUdpnCndofGh3B2o8oEi/TPyBQXtrGvNAw4u1QRCdC0M2hWWyS932 Nbag== MIME-Version: 1.0 X-Received: by 10.224.37.10 with SMTP id v10mr39749773qad.98.1400476353722; Sun, 18 May 2014 22:12:33 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 18 May 2014 22:12:33 -0700 (PDT) In-Reply-To: <5379609E.6000702@freebsd.org> References: <5379609E.6000702@freebsd.org> Date: Sun, 18 May 2014 22:12:33 -0700 X-Google-Sender-Auth: yfWCFhb7l-MyLK3lD725Nu8VIgU Message-ID: Subject: Re: [rfc] teach netstat about flowid/flowtype From: Adrian Chadd To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 05:12:35 -0000 *laugh* On 18 May 2014 18:38, Julian Elischer wrote: > On 5/19/14, 7:44 AM, Adrian Chadd wrote: >> >> Hi! >> >> My next hack - adding flowid/flowtype to netstat. It's netstat -R. >> It's like -x, but listing RSS/flowid stuff. >> >> http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff >> >> This is useful for both RSS and non-RSS debugging - NICs that populate >> the mbuf flowid will start to show flowid's up in netstat. >> >> It doesn't currently show the rss CPUID as that isn't cached in the >> inpcb. I'll worry about adding that later on. >> >> Any issues with me just committing this? I promise I'll email doc@ to >> get it documented. > > If you don't I'll get "you-know-who" to nag you.. I need a good Italian meal > some time.. You should just visit her. She can show you what your "oh that's a crystal set" introduction spawned into. :-) -a From owner-freebsd-net@FreeBSD.ORG Mon May 19 07:51:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7C35D22; Mon, 19 May 2014 07:51:36 +0000 (UTC) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2EB492B98; Mon, 19 May 2014 07:51:36 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id w62so5137852wes.16 for ; Mon, 19 May 2014 00:51:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WM0LBd5QUeZyWbL58/zzO0XKbOOeUKBTazv7c7FdTnI=; b=ZMfxDtRWk/0zT+WxcIxDBLVDEl1uVPW3AGmz9rDm0/TAPNul3ZH5bK8CBS9SMSxKR1 qG00HL4GH4tIn4Lasr3Ss0A/S107ptsaXITSeYXk5GyG4pIs5FPdUYIB/x7tpzm5k9YV NX4WoJOh25FpdVYGS+kk63Py8v/T8PExRt8IWvJhy0WFNyhuV4rB6aB9utg7LdLy0VvP WLUS56XGxwSxB5blkgeE919IdUaNa9W7ijDtap5imZuQOV6I+/30fisTcLpgLuCm+AeZ KW66DLOQOSuqS485KT68YShcKZQA6QZwDP+QLdOzqHlrzNGcq4zaLX2WCeCF2dwBrfl5 xZ9w== MIME-Version: 1.0 X-Received: by 10.180.107.97 with SMTP id hb1mr11472898wib.20.1400485894419; Mon, 19 May 2014 00:51:34 -0700 (PDT) Received: by 10.216.116.136 with HTTP; Mon, 19 May 2014 00:51:34 -0700 (PDT) In-Reply-To: References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> Date: Mon, 19 May 2014 15:51:34 +0800 Message-ID: Subject: Re: Problem with ipfw table add 0.0.0.0/8 From: Bill Yuan To: Jason Hellenthal Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , FreeBSD Net , "Alexander V. Chernikov" , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 07:51:37 -0000 Hi Alex, You guys are chatting here! I agree with you, the table is the place should be enhanced, and I am working in this way as described below 1. Support more types. ip : cidr ipv4 : same as ip ipv6 : ip addr v6 mac : mac address iface : interface name interface : same as iface port : it is Alex's idea, I dont know how it works. 2. Setup the table type ipfw table type it will setup the type of the table, and flush the table 3. Get table type ipfw table type show 4. Add item into the table ipfw table add a. get the type of table b. if the type is not defined yet, that also means the table is new or empty, then guess the type based on the c. format the and insert into the table. In this way so call "back compatible" 5. how to use table case 1 ipfw add [line] allow icmp from "table(1)" to "table(2)" in the ipfw userland command, it should check the table1 and table 2 should be ipv4 or ipv6 type case 2 ipfw add allow icmp from any to any MAC "table(3)" "table(4)" in this case, the table(3) and table(4) should be a table of MAC addresses. case 3 ipfw add allow icmp from any to any via table(5) in this case, the table 5 should be table of interface names. currently I am working on the mac type. :) On Sun, May 18, 2014 at 12:47 PM, Jason Hellenthal wrote: > > > > On May 18, 2014, at 0:12, Julian Elischer wrote: > >> 2) Table type/name can be specified explicitly via one of the following > commands: > >> * ipfw table 1 create [type ] [name > "table_name"] > > type "ports" would be nice but tricky to do right. > > That . . . would be a great addition and have me switching from pf to ipfw. > > Pullllease do! :-) From owner-freebsd-net@FreeBSD.ORG Mon May 19 08:28:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42BF0F7C; Mon, 19 May 2014 08:28:26 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C0F92F53; Mon, 19 May 2014 08:28:25 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id z11so3501083lbi.41 for ; Mon, 19 May 2014 01:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=1Tmha219LeMy5LWAtFXffnVAo5AbCR9GJ5619M3SP18=; b=IDMpIri30Bk27cLU5VDHMPaq7Qvl6WnZ1Wy9jKcAvZLPkjRFhY3hiv/rG5NJrWwlM8 Rxj8gq4QG9y5eMtpna0kt4ukkH1I8eruyhaeJyVLMjRNskMisKZOSK4ricf0OMBIjS8P Jrvk6jqOsjQeqSGwB9z3PMyV4Y6oStPXmVo2bWBOizP1ehxnPMg3sO1SKLCvoDgcyBpX L3TVJYAOebEnaVCr7ZE1M6niO9X0X0ZSqOQhyrAUQC+IDgUFwrI7vWG1PBvrEZmEW7PO hl5J2v403GicEFTaCT7bZs/B4SuX3J/EHFEY5MdYYMhY2WNrUaccIOABql3mPguRsWQA +ajw== MIME-Version: 1.0 X-Received: by 10.112.50.241 with SMTP id f17mr19319881lbo.7.1400488103223; Mon, 19 May 2014 01:28:23 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Mon, 19 May 2014 01:28:23 -0700 (PDT) In-Reply-To: References: <201405181223.s4ICNTAf097378@fire.js.berklix.net> <20140518230242.GA28117@onelab2.iet.unipi.it> Date: Mon, 19 May 2014 04:28:23 -0400 X-Google-Sender-Auth: wDtd_X2424Go0HQevaW5jZSgKPA Message-ID: Subject: Re: netmap and other discussions on freebsd-net: please be open minded. From: Luigi Rizzo To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 08:28:26 -0000 On Sun, May 18, 2014 at 7:49 PM, Adrian Chadd wrote: > Hi! > > Is there a netmap list that these questions (regardless of OS) could go t= o? > =E2=80=8Bno there isn't one. At the moment there isn't enough traffic to suggest that, and creating one is likely to generate traffic to both the specific list and freebsd-net. Note, the problem is not only for netmap: we have ipfw+dummynet which is already cross platform, and as we bring pieces of the kernel to userspace (see=E2=80=8B e.g. the various efforts related to bringing up the network stack) we lose the tight bind to the os. cheers luigi (context below) > On 18 May 2014 16:02, Luigi Rizzo wrote: > > Folks, i have two requests for you: > > > > 1. please do not complain about questions on this list related > > to a core network-related FreeBSD subsystem (netmap, dummynet, > > netgraph, tcp stack...) even if they are concerned with ports > > to Linux or other OSes or to userspace. > > > > At least in principle, these topics _are_ relevant here precisely > > because they relate to code whose main home is in the FreeBSD > > source tree, and bugs or feature suggestions etc that may emerge > > will directly benefit FreeBSD. > > > > 2. on the other hand, it would be good if people could avoid questions > like > > "give me step by step instructions on how to install/run/use X". > > There are notes in README and Makefiles, something on the > > web pages, and quick web searches should point you to previous > > mailing list discussions on the same topic _before asking_. > > > > So in the specific case below i can understand a reply (perhaps a > > private one) saying "have you done your homework (read documentation, > > do a web search) ?", but "we only do FreeBSD here" does not sound > > right to me. > > > > cheers > > luigi > > > > On Sun, May 18, 2014 at 02:23:29PM +0200, XXX wrote: > >> > I am using Fedora 18 with Intel 82599 NIC. > >> .............^^^^^^^^^ > >> > >> Ask on a Linux/Fedora list then. This is a FreeBSD list. > >> > >> > I am new to netmap and want to experiment with it in the above > environment. > >> > Can somebody please advise what is the easiest way to go about the > above > >> > (installation etc.) so that I can concentrate on writing the userspa= ce > >> > programs using netmap quickly. > >> > >> > >> Cheers, > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Mon May 19 08:54:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 332BF780 for ; Mon, 19 May 2014 08:54:26 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DF398218F for ; Mon, 19 May 2014 08:54:25 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1WmJOM-000JMP-G5 for freebsd-net@freebsd.org; Mon, 19 May 2014 12:57:54 +0400 Message-ID: <5379C6B6.4030105@smartspb.net> Date: Mon, 19 May 2014 12:54:14 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140518-1, 18.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 08:54:26 -0000 Alex, Bill, it's a good news, glad to hear it. Let me ask even more functionality: 6. Test if entry exist in table: ipfw table test It extremely useful in case of big, unordered data in the table - for example different networks with different mask. Now it's almost impossible to find out is checked IP occurs in the table or not. 7. Are the any reason to keep use numbers only as table names? The more tables uses, the harder to distinct tables in quick look at rules. Compare: ipfw add [line] allow icmp from "table(1)" to "table(2)" and something like ipfw add [line] allow icmp from "table(trusted)" to "table(backbone)" Any comments are welcome. 19.05.2014 11:51, Bill Yuan ÐŋÐļŅˆÐĩŅ‚: > Hi Alex, > > You guys are chatting here! I agree with you, the table is the place > should be enhanced, and I am working in this way as described below > > 1. Support more types. > ip : cidr > ipv4 : same as ip > ipv6 : ip addr v6 > mac : mac address > iface : interface name > interface : same as iface > port : it is Alex's idea, I dont know how it works. > > 2. Setup the table type > ipfw table type > it will setup the type of the table, and flush the table > > 3. Get table type > ipfw table type show > > 4. Add item into the table > ipfw table add > > a. get the type of table > b. if the type is not defined yet, that also means the table is new or > empty, > then guess the type based on the > c. format the and insert into the table. > > In this way so call "back compatible" > > 5. how to use table > > case 1 > ipfw add [line] allow icmp from "table(1)" to "table(2)" > in the ipfw userland command, it should check the table1 and table 2 > should be ipv4 or ipv6 type > > case 2 > ipfw add allow icmp from any to any MAC "table(3)" "table(4)" > in this case, the table(3) and table(4) should be a table of MAC > addresses. > > case 3 > ipfw add allow icmp from any to any via table(5) > in this case, the table 5 should be table of interface names. > -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Mon May 19 09:11:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6357DCF7 for ; Mon, 19 May 2014 09:11:30 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F071322B4 for ; Mon, 19 May 2014 09:11:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s4J8gIAB097858; Mon, 19 May 2014 18:42:19 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 19 May 2014 18:42:18 +1000 (EST) From: Ian Smith To: Luigi Rizzo Subject: Re: netmap and other discussions on freebsd-net: please be open minded. In-Reply-To: <20140518230242.GA28117@onelab2.iet.unipi.it> Message-ID: <20140519174117.G89611@sola.nimnet.asn.au> References: <201405181223.s4ICNTAf097378@fire.js.berklix.net> <20140518230242.GA28117@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 09:11:30 -0000 On Mon, 19 May 2014 01:02:42 _0200, Luigi Rizzo wrote: > Folks, i have two requests for you: > > 1. please do not complain about questions on this list related > to a core network-related FreeBSD subsystem (netmap, dummynet, > netgraph, tcp stack...) even if they are concerned with ports > to Linux or other OSes or to userspace. > > At least in principle, these topics _are_ relevant here precisely > because they relate to code whose main home is in the FreeBSD > source tree, and bugs or feature suggestions etc that may emerge > will directly benefit FreeBSD. I'm glad I waited before responding to the OP+1 or I may have been somewhat less polite, let alone authoritative .. so thanks. I can only add that projects such as netmap - and ipfw & dummynet for Linux for that matter - can only serve to lower the wall, in perception at least, between the merits of either OS for purpose, and further increase the exposure of the quality of FreeBSD networking to the Linux masses; ie, I believe this can only be a net gain for FreeBSD overall. > 2. on the other hand, it would be good if people could avoid questions like > "give me step by step instructions on how to install/run/use X". > There are notes in README and Makefiles, something on the > web pages, and quick web searches should point you to previous > mailing list discussions on the same topic _before asking_. googling 'netmap linux' seems to point to most everything relevant, including some discussions on Linux lists, and perhaps illustrative of (1) is http://www.ntop.org/pf_ring/dna-vs-netmap/ cheers, Ian (no, I haven't played with netmap, but it sure smells good!) From owner-freebsd-net@FreeBSD.ORG Mon May 19 09:15:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A7F1EB1 for ; Mon, 19 May 2014 09:15:49 +0000 (UTC) Received: from mail-oa0-x22b.google.com (mail-oa0-x22b.google.com [IPv6:2607:f8b0:4003:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3ED22234C for ; Mon, 19 May 2014 09:15:49 +0000 (UTC) Received: by mail-oa0-f43.google.com with SMTP id l6so5881221oag.16 for ; Mon, 19 May 2014 02:15:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3qiPZveMcl7y7lnvC+Z9alX69hPaumZkG2dHJOZh0Ms=; b=oW7abyQ5satS8hDw/DIfYKE3z5LvqluH4YQP8WUFwBfGhkKd5OQjnfHzly0LoIO8ED m3JmuzmV8EKyHF6eYqjSScfcSkgnqEcskD7UrUqFqjhy3AyfmW9lWu+u8qGsvzTmFOVP UGSkdpyg/bb5tSxkgQ9n7pRr88kZ5SLSpDQUFbqvpboSJ7Tobd+nqPyYiUYIdpOaYsmK 9XmewJTpP8A65OJz79/nRq/mHFpAnVXgTDVe08H80jGWxOs/WKDaQb1hyantm/AYnhB7 hyebPOLvgPFrPF7C7uRJg0K1U03kRvHKridor2bD07rlXezVZ2oH0a+lYC5HuYlw8F6M /LFQ== MIME-Version: 1.0 X-Received: by 10.60.62.211 with SMTP id a19mr2255924oes.71.1400490948202; Mon, 19 May 2014 02:15:48 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Mon, 19 May 2014 02:15:48 -0700 (PDT) In-Reply-To: <5379C6B6.4030105@smartspb.net> References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> Date: Mon, 19 May 2014 11:15:48 +0200 Message-ID: Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 From: Andreas Nilsson To: Dennis Yusupoff Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 09:15:49 -0000 On Mon, May 19, 2014 at 10:54 AM, Dennis Yusupoff wrote: > Alex, Bill, it's a good news, glad to hear it. > > Let me ask even more functionality: > > 6. Test if entry exist in table: > ipfw table test > It extremely useful in case of big, unordered data in the table - for > example different networks with different mask. Now it's almost > impossible to find out is checked IP occurs in the table or not. > So having 10.0.1.1/16 in table and looking for 10.0.240.15 would say in table? That would be nice. > > 7. Are the any reason to keep use numbers only as table names? The more > tables uses, the harder to distinct tables in quick look at rules. Compar= e: > ipfw add [line] allow icmp from "table(1)" to "table(2)" > and something like > ipfw add [line] allow icmp from "table(trusted)" to "table(backbone)" > > Any comments are welcome. > > If table can have names, the above would be really nice as well. /A > > 19.05.2014 11:51, Bill Yuan =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > > Hi Alex, > > > > You guys are chatting here! I agree with you, the table is the place > > should be enhanced, and I am working in this way as described below > > > > 1. Support more types. > > ip : cidr > > ipv4 : same as ip > > ipv6 : ip addr v6 > > mac : mac address > > iface : interface name > > interface : same as iface > > port : it is Alex's idea, I dont know how it works. > > > > 2. Setup the table type > > ipfw table type > > it will setup the type of the table, and flush the table > > > > 3. Get table type > > ipfw table type show > > > > 4. Add item into the table > > ipfw table add > > > > a. get the type of table > > b. if the type is not defined yet, that also means the table is new or > > empty, > > then guess the type based on the > > c. format the and insert into the table. > > > > In this way so call "back compatible" > > > > 5. how to use table > > > > case 1 > > ipfw add [line] allow icmp from "table(1)" to "table(2)" > > in the ipfw userland command, it should check the table1 and table 2 > > should be ipv4 or ipv6 type > > > > case 2 > > ipfw add allow icmp from any to any MAC "table(3)" "table(4)" > > in this case, the table(3) and table(4) should be a table of MAC > > addresses. > > > > case 3 > > ipfw add allow icmp from any to any via table(5) > > in this case, the table 5 should be table of interface names. > > > > -- > Best regards, > Dennis Yusupoff, > network engineer of > Smart-Telecom ISP > Russia, Saint-Petersburg > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon May 19 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5924840D for ; Mon, 19 May 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 451A82DB7 for ; Mon, 19 May 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4JB6nV5080091 for ; Mon, 19 May 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4JB6mbT080089 for freebsd-net@FreeBSD.org; Mon, 19 May 2014 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 19 May 2014 11:06:48 GMT Message-Id: <201405191106.s4JB6mbT080089@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/189531 net [fxp] Wake on Lan (WOL) enabled for Intel 82562EZ ethe o kern/189234 net [em] Big lag with Ethernet Connection I217-V o kern/189219 net [dummynet] [patch] using dummynet on sparc64 and confi o kern/188899 net [cas] cas ethernet driver seems to have issues with so o kern/188897 net [dc] dc ethernet driver seems to prevent the detection o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/186304 net [bmc] watchdog service causes BMC controller reset eve o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184389 net [libalias] libalias fails to adjust MTU from jails o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o kern/183920 net [ixgbe] [patch] Incorrect ifconfig media on INTEL X520 o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183381 net [em] [patch] Use of 9k buffers in if_em.c hangs with r o kern/183271 net [em] statistic not updated on em in netmap mode o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179473 net [new driver] if_vether.c: Source code contribution of o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171550 net [ndis] [patch] "no match for InterlockedCompareExchang o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [netinet] [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 483 problems total. From owner-freebsd-net@FreeBSD.ORG Mon May 19 12:52:45 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F4FA4CD; Mon, 19 May 2014 12:52:45 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A553C2861; Mon, 19 May 2014 12:52:44 +0000 (UTC) Received: from 95.108.170.210-red.dhcp.yndx.net ([95.108.170.210] helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WmJ9J-000HCD-FW; Mon, 19 May 2014 12:42:21 +0400 Message-ID: <5379FE3C.6060501@FreeBSD.org> Date: Mon, 19 May 2014 16:51:08 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: FreeBSD Net Subject: [CFT]: ipfw named tables / different tabletypes Content-Type: multipart/mixed; boundary="------------040508090907050000010802" Cc: Luigi Rizzo X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 12:52:45 -0000 This is a multi-part message in MIME format. --------------040508090907050000010802 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hello list! This patch adds ability to name tables / reference them by name. Additionally, it simplifies adding new table types. Change list: Kernel: 1) Add new IP_FW_TABLE_XGETCFG / IP_FW_TABLE_XSETCFG opcodes to permit table reconfiguration 2) Tables data is now opaque to main ipfw code: use 1 pointer in first ip_fw_chain cache line for lookups and another one for config state. 3) Do not assume radix is the one and only lookup mechasim for doing lookups (more changes following) 4) Table data layout is changed to the following: +struct table_info { + void *state; /* IPv4 tables */ + void *xstate;/* extended tables */ + table_lookup_t *lookup;/* lookup function */ + struct table_config *cfg; /* Additional data, can be NULL */ +}; Array of size "table_max * sizeof(struct table_info)" is allocated on startup (very much like in current code in term of memory). 5) State holds any additional info table may need for configuration purposes and is allocated on demand. 6) By default, all tables are CIDR (IPv4+IPv6) and does not hold *cfg state. 7) State is allocated when: * table is referenced in some rules * type is non-default * table is named 8) Tables can be named and referenced by their names, but it is still needed to explicitly select table number. 8) Table references are now explicitly tracked by kernel checking if opcode lookup type and table type are the same 9) Do not assume tbl is uint16_t 10) Change locking model: use both IPFW and IPFW_UH locks, as main ipfw code does 11) While here, fix possible panic in case of adding new table entry while reallocating tables_max Some more on table types: +struct table_config { + uint8_t tabletype; /* lookup table types */ + uint8_t ftype; /* format table type */ + uint8_t atype; /* algorithm type */ + uint8_t spare0; + uint32_t refcnt; /* Number of references */ + uint32_t count; /* Number of records */ + struct table_info *ti; + TAILQ_ENTRY(table_config) next; /* namehash */ + char tablename[64]; /* table name */ +}; "tabletype" is basically type of the key we're looking for (e.g. IPv4/IPv6 address, interface name, port/uid, etc..). "ftype" is pure userland field helping to format keys in appropriate way (like shown DSCP values in hex or binary). "atype" permits to use different algorithm for the same lookup key ( currently not implemented, but planned). Good example can be CIDR table consisting only of host routes. User: Nothing changes for people using tables for IPv4/IPv6 address matching. New cmds: ipfw table type Changes table type to different one. Permitted IFF: * table is not referenced in ruleset * table is empy ipfw table name XXX Names (or renames) table. Not the name has to be unique. ipfw table flush (Not a new command, actually). Flushes all table records leaving configuration intact. ipfw table destroy Flushes table state AND configuration. Tables becomes unnamed IPFW_TABLE_CIDR one. Next changes: * Further rework add/del table entry to permit adding non-radix tables more easily * Change "iface" table type implementation to uint32_t iflist[65536] to permit O(1) interface matching * Add general u32 lookup method for dealing with ports/uids/jails/dscp and other such consumers I'm planning to commit this one (actually, a bit improved version) in the beginning of next week if no objections received. --------------040508090907050000010802 Content-Type: text/x-patch; name="ipfw_tables3.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ipfw_tables3.diff" Index: sbin/ipfw/ipfw2.c =================================================================== --- sbin/ipfw/ipfw2.c (revision 266310) +++ sbin/ipfw/ipfw2.c (working copy) @@ -204,6 +204,12 @@ static struct _s_x limit_masks[] = { {NULL, 0} }; +static struct _s_x f_tabletypes[] = { + {"cidr", IPFW_TABLE_CIDR}, + {"iface", IPFW_TABLE_INTERFACE}, + {NULL, 0} +}; + /* * we use IPPROTO_ETHERTYPE as a fake protocol id to call the print routines * This is only used in this code. @@ -4124,6 +4130,9 @@ ipfw_flush(int force) static void table_list(uint16_t num, int need_header); static void table_fill_xentry(char *arg, ipfw_table_xentry *xent); +static void table_get_info(uint16_t num, char *name, int need_header); +static uint32_t table_get_num(char *name); +static int table_check_name(char *name); /* * Retrieve maximum number of tables supported by ipfw(4) module. @@ -4158,6 +4167,7 @@ ipfw_get_tables_max() * ipfw table N delete addr[/masklen] * ipfw table {N | all} flush * ipfw table {N | all} list + * ipfw table {M | all} info */ void ipfw_table_handler(int ac, char *av[]) @@ -4167,6 +4177,7 @@ ipfw_table_handler(int ac, char *av[]) int is_all; uint32_t a; uint32_t tables_max; + int l, type; tables_max = ipfw_get_tables_max(); @@ -4181,13 +4192,18 @@ ipfw_table_handler(int ac, char *av[]) xent.tbl = 0; is_all = 1; ac--; av++; - } else - errx(EX_USAGE, "table number or 'all' keyword required"); + } else { + xent.tbl = table_get_num(*av); + is_all = 0; + ac--; av++; + } + if (xent.tbl >= tables_max) errx(EX_USAGE, "The table number exceeds the maximum allowed " "value (%d)", tables_max - 1); NEED1("table needs command"); if (is_all && _substrcmp(*av, "list") != 0 + && _substrcmp(*av, "info") != 0 && _substrcmp(*av, "flush") != 0) errx(EX_USAGE, "table number required"); @@ -4243,6 +4259,59 @@ ipfw_table_handler(int ac, char *av[]) do { table_list(xent.tbl, is_all); } while (++xent.tbl < a); + } else if (_substrcmp(*av, "info") == 0) { + a = is_all ? tables_max : (uint32_t)(xent.tbl + 1); + do { + table_get_info(xent.tbl, NULL, is_all); + } while (++xent.tbl < a); + } else if (_substrcmp(*av, "type") == 0) { + ac--; av++; + if (!ac) + errx(EX_USAGE, "table type required"); + + if ((type = match_token(f_tabletypes, *av)) == -1) + errx(EX_DATAERR, "Unknown table type"); + + ipfw_xtable_cfg cfg; + + memset(&cfg, 0, sizeof(cfg)); + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; + cfg.tbl = xent.tbl; + cfg.mask = IPFW_XCFG_TYPE; + cfg.type = type; + l = sizeof(cfg); + + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { + /* + * XXX: Provide human-readable error here + * by using IP_FW_TABLE_XGETCFG + */ + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); + } + } else if (_substrcmp(*av, "name") == 0) { + ac--; av++; + if (!ac) + errx(EX_USAGE, "table name required"); + + if (table_check_name(*av) != 0) + errx(EX_USAGE, "bad table name"); + + ipfw_xtable_cfg cfg; + + memset(&cfg, 0, sizeof(cfg)); + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; + cfg.tbl = xent.tbl; + cfg.mask = IPFW_XCFG_NAME; + strlcpy(cfg.tablename, *av, sizeof(cfg.tablename)); + l = sizeof(cfg); + + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { + /* + * XXX: Provide human-readable error here + * by using IP_FW_TABLE_XGETCFG + */ + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); + } } else errx(EX_USAGE, "invalid table command %s", *av); } @@ -4423,3 +4492,107 @@ table_list(uint16_t num, int need_header) free(tbl); } + +static uint32_t +table_get_num(char *name) +{ + ipfw_xtable_cfg cfg; + socklen_t l; + + memset(&cfg, 0, sizeof(cfg)); + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; + l = sizeof(cfg); + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); + cfg.mask |= IPFW_XCFG_NAMEID; + + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) + err(EX_OSERR, "table name '%s' not found", name); + + return (cfg.tbl); +} + +static void +table_get_info(uint16_t num, char *name, int is_all) +{ + ipfw_xtable_cfg cfg; + socklen_t l; + char *t; + + memset(&cfg, 0, sizeof(cfg)); + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; + cfg.mask = IPFW_XCFG_NAME | IPFW_XCFG_TYPE | IPFW_XCFG_REFS | + IPFW_XCFG_CNT; + l = sizeof(cfg); + + if (name == NULL) { + cfg.tbl = num; + cfg.mask |= IPFW_XCFG_NUMID; + } else { + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); + cfg.mask |= IPFW_XCFG_NAMEID; + } + + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETCFG)"); + + if (is_all != 0) { + /* + * Skip unreferenced tables with default type + */ + if (cfg.type == IPFW_TABLE_CIDR && cfg.refcnt == 0) + return; + printf("---table(%d)---\n", num); + } + if (cfg.mask & IPFW_XCFG_NAME) + printf("Name: %s\n", cfg.tablename); + switch (cfg.type) { + case IPFW_TABLE_CIDR: + t = "cidr"; + break; + case IPFW_TABLE_INTERFACE: + t = "cidr"; + break; +/* + case IPFW_TABLE_U32: + t = "u32"; + break; + case IPFW_TABLE_U16: + t = "u16"; + break; +*/ + default: + t = "unknown"; + break; + }; + + printf("Type: %s\tFormatting: %s\n", t, "default"); + printf("References: %u\tEntries: %u\n", cfg.refcnt, cfg.count); +} + +static int +table_check_name(char *tablename) +{ + int c, i, l; + + /* + * Check if tablename is null-terminated and contains + * valid symbols only. Valid mask is: + * [a-zA-Z\-\.][a-zA-Z0-9\-_\.]{0,62} + */ + l = strlen(tablename); + if (l == 0 || l >= 64) + return (EINVAL); + /* Restrict first symbol to non-digit */ + if (isdigit(tablename[0])) + return (EINVAL); + for (i = 0; i < l; i++) { + c = tablename[i]; + if (isalpha(c) || isdigit(c) || c == '_' || + c == '-' || c == '.') + continue; + return (EINVAL); + } + + return (0); +} + Index: sys/netinet/ip_fw.h =================================================================== --- sys/netinet/ip_fw.h (revision 266310) +++ sys/netinet/ip_fw.h (working copy) @@ -74,6 +74,8 @@ typedef struct _ip_fw3_opheader { #define IP_FW_TABLE_XDEL 87 /* delete entry */ #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ #define IP_FW_TABLE_XLIST 89 /* list table contents */ +#define IP_FW_TABLE_XGETCFG 90 /* configure table */ +#define IP_FW_TABLE_XSETCFG 91 /* configure table */ /* * The kernel representation of ipfw rules is made of a list of @@ -632,7 +634,7 @@ typedef struct _ipfw_table { } ipfw_table; typedef struct _ipfw_xtable { - ip_fw3_opheader opheader; /* eXtended tables are controlled via IP_FW3 */ + ip_fw3_opheader opheader; /* IP_FW3 opcode */ uint32_t size; /* size of entries in bytes */ uint32_t cnt; /* # of entries */ uint16_t tbl; /* table number */ @@ -640,4 +642,31 @@ typedef struct _ipfw_xtable { ipfw_table_xentry xent[0]; /* entries */ } ipfw_xtable; +#define IPFW_XCFG_NAME 0x01 /* get/set name */ +#define IPFW_XCFG_TYPE 0x02 /* get/set type */ +#define IPFW_XCFG_FTYPE 0x04 /* get/set format type */ +#define IPFW_XCFG_REFS 0x08 /* get/set number of references */ +#define IPFW_XCFG_CNT 0x10 /* ge number of records */ +#define IPFW_XCFG_NUMID 0x20 /* table identified by num */ +#define IPFW_XCFG_NAMEID 0x40 /* table identified by name */ +typedef struct _ipfw_xtable_cfg { + ip_fw3_opheader opheader; /* IP_FW3 opcode */ + uint32_t size; /* size of structure in bytes */ + uint32_t mask; /* data mask to set/retrieve */ + uint32_t tlvmask; /* data mask to set/retrieve */ + uint32_t tbl; /* table id */ + uint16_t type; /* table type */ + uint16_t ftype; /* table format type */ + uint32_t count; /* number of records */ + uint32_t refcnt; /* number of references */ + char tablename[64]; /* table name */ +} ipfw_xtable_cfg; + +typedef struct _ipfw_xtable_tlv { + uint16_t type; + uint16_t length; +} ipfw_xtable_tlv; + + + #endif /* _IPFW2_H */ Index: sys/netpfil/ipfw/ip_fw2.c =================================================================== --- sys/netpfil/ipfw/ip_fw2.c (revision 266306) +++ sys/netpfil/ipfw/ip_fw2.c (working copy) @@ -352,15 +352,16 @@ tcpopts_match(struct tcphdr *tcp, ipfw_insn *cmd) } static int -iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, uint32_t *tablearg) +iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, + uint32_t *tablearg) { if (ifp == NULL) /* no iface with this packet, match fails */ return 0; /* Check by name or by IP address */ if (cmd->name[0] != '\0') { /* match by name */ if (cmd->name[0] == '\1') /* use tablearg to match */ - return ipfw_lookup_table_extended(chain, cmd->p.glob, - ifp->if_xname, tablearg, IPFW_TABLE_INTERFACE); + return (ipfw_lookup_table(chain, cmd->p.glob, IFNAMSIZ, + ifp, tablearg)); /* Check name */ if (cmd->p.glob) { if (fnmatch(cmd->name, ifp->if_xname, 0) == 0) @@ -1492,7 +1493,7 @@ do { \ break; } match = ipfw_lookup_table(chain, - cmd->arg1, key, &v); + cmd->arg1, sizeof(uint32_t), &key, &v); if (!match) break; if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) @@ -1504,9 +1505,9 @@ do { \ uint32_t v = 0; void *pkey = (cmd->opcode == O_IP_DST_LOOKUP) ? &args->f_id.dst_ip6: &args->f_id.src_ip6; - match = ipfw_lookup_table_extended(chain, - cmd->arg1, pkey, &v, - IPFW_TABLE_CIDR); + match = ipfw_lookup_table(chain, + cmd->arg1, sizeof(uint32_t), + pkey, &v); if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) match = ((ipfw_insn_u32 *)cmd)->d[0] == v; if (match) Index: sys/netpfil/ipfw/ip_fw_private.h =================================================================== --- sys/netpfil/ipfw/ip_fw_private.h (revision 266306) +++ sys/netpfil/ipfw/ip_fw_private.h (working copy) @@ -217,9 +217,7 @@ struct ip_fw_chain { uint32_t id; /* ruleset id */ int n_rules; /* number of static rules */ LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ - struct radix_node_head **tables; /* IPv4 tables */ - struct radix_node_head **xtables; /* extended tables */ - uint8_t *tabletype; /* Array of table types */ + void *tablestate; /* Array of table data */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t rwmtx; #else @@ -229,6 +227,7 @@ struct ip_fw_chain { uint32_t gencnt; /* NAT generation count */ struct ip_fw *reap; /* list of rules to reap */ struct ip_fw *default_rule; + void *tablecfg; /* tables module configuration */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t uh_lock; #else @@ -303,9 +302,8 @@ int ipfw_chk(struct ip_fw_args *args); void ipfw_reap_rules(struct ip_fw *head); /* In ip_fw_table.c */ -struct radix_node; -int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, - uint32_t *val); +int ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t keylen, + void *key, uint32_t *val); int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, uint32_t *val, int type); int ipfw_init_tables(struct ip_fw_chain *ch); @@ -316,11 +314,18 @@ int ipfw_add_table_entry(struct ip_fw_chain *ch, u int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, uint8_t plen, uint8_t mlen, uint8_t type); int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); -int ipfw_dump_table_entry(struct radix_node *rn, void *arg); int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); +int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, + uint32_t *cnt); int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); +int ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl); +int ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype); +int ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl); +int ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, + size_t *sz); +int ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, + size_t *sz); /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ Index: sys/netpfil/ipfw/ip_fw_sockopt.c =================================================================== --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266306) +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy) @@ -146,6 +146,119 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * } /* + * In @del==0 mode: + * Checks is opcode is referencing table of appropriate type. + * Adds reference count for found table if true. + * In del==1 mode + * Decrements refcount for given table. + * + * Returns 0 on success and appropriate error code otherwise. + */ +static int +bind_tables(struct ip_fw_chain *chain, struct ip_fw *rule, int del) +{ + int cmdlen, error, ftype, l, skip, type, v; + ipfw_insn *cmd; + ipfw_insn_if *cmdif; + uint32_t tbl; + + l = rule->cmd_len; + cmd = rule->cmd; + cmdlen = 0; + tbl = 0; + type = 0; + ftype = 0; + + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + skip = 0; + + switch (cmd->opcode) { + case O_IP_SRC_LOOKUP: + case O_IP_DST_LOOKUP: + /* Basic IPv4/IPv6 or u32 lookups */ + tbl = cmd->arg1; + /* Assume CIDR by default */ + type = IPFW_TABLE_CIDR; + ftype = 0; + + if (cmdlen > F_INSN_SIZE(ipfw_insn_u32)) { + /* generic lookup. The key must be + * in 32bit big-endian format. + */ + v = ((ipfw_insn_u32 *)cmd)->d[1]; + switch (v) { + case 0: + case 1: + /* IPv4 src/dst */ + break; + case 2: + case 3: + /* src/dst port */ + //type = IPFW_TABLE_U16; + break; + case 4: + /* uid/gid */ + //type = IPFW_TABLE_U32; + case 5: + //type = IPFW_TABLE_U32; + /* jid */ + case 6: + //type = IPFW_TABLE_U16; + /* dscp */ + break; + } + } + break; + case O_XMIT: + case O_RECV: + case O_VIA: + /* Interface table, possibly */ + cmdif = (ipfw_insn_if *)cmd; + if (cmdif->name[0] != '\1') { + skip = 1; + break; + } + + type = IPFW_TABLE_INTERFACE; + ftype = 0; + tbl = cmdif->p.glob; + break; + default: + skip = 1; + } + + if (skip != 0) + continue; + + /* ref/unref given table */ + if (del != 0) + error = ipfw_unref_table(chain, tbl); + else + error = ipfw_ref_table(chain, tbl, type, ftype); + + if (error != 0) + return (error); + } + + return (0); +} + +/* + * Removes table bindings for every rule in rule chain @head. + */ +static void +unbind_tables(struct ip_fw_chain *chain, struct ip_fw *head) +{ + struct ip_fw *rule; + + while ((rule = head) != NULL) { + head = head->x_next; + bind_tables(chain, rule, 1); + } +} + +/* * Add a new rule to the list. Copy the rule into a malloc'ed area, then * possibly create a rule number and add the rule to the list. * Update the rule_number in the input struct so the caller knows it as well. @@ -157,6 +270,7 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip { struct ip_fw *rule; int i, l, insert_before; + int error; struct ip_fw **map; /* the new array of pointers */ if (chain->map == NULL || input_rule->rulenum > IPFW_DEFAULT_RULE - 1) @@ -168,9 +282,16 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip map = get_map(chain, 1, 0 /* not locked */); if (map == NULL) { free(rule, M_IPFW); - return ENOSPC; + return (ENOSPC); } + /* Reference tables, if any */ + if ((error = bind_tables(chain, input_rule, 0)) != 0) { + IPFW_UH_WUNLOCK(chain); + free(rule, M_IPFW); + return (error); + } + bcopy(input_rule, rule, l); /* clear fields not settable from userland */ rule->x_next = NULL; @@ -421,6 +542,7 @@ del_entry(struct ip_fw_chain *chain, uint32_t arg) rule = chain->reap; chain->reap = NULL; + unbind_tables(chain, rule); IPFW_UH_WUNLOCK(chain); ipfw_reap_rules(rule); if (map) @@ -934,6 +1056,7 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf #define IP_FW3_OPLENGTH(x) ((x)->sopt_valsize - sizeof(ip_fw3_opheader)) + /** * {set|get}sockopt parser. */ @@ -1113,7 +1236,7 @@ ipfw_ctl(struct sockopt *sopt) sopt->sopt_name == IP_FW_RESETLOG); break; - /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ + /*--- TABLE locking is described in ip_fw_table.c ---*/ case IP_FW_TABLE_ADD: { ipfw_table_entry ent; @@ -1237,7 +1360,7 @@ ipfw_ctl(struct sockopt *sopt) tbl = (uint32_t *)(op3 + 1); IPFW_RLOCK(chain); - error = ipfw_count_xtable(chain, *tbl, tbl); + error = ipfw_count_xtable(chain, *tbl, tbl, NULL); IPFW_RUNLOCK(chain); if (error) break; @@ -1281,6 +1404,39 @@ ipfw_ctl(struct sockopt *sopt) } break; + case IP_FW_TABLE_XGETCFG: /* IP_FW3 */ + case IP_FW_TABLE_XSETCFG: /* IP_FW3 */ + { + ipfw_xtable_cfg *xcfg; + + if (sopt->sopt_valsize < sizeof(xcfg)) { + error = EINVAL; + break; + } + if ((size = sopt->sopt_valsize) > RULE_MAXSIZE) { + error = E2BIG; + break; + } + + xcfg = malloc(size, M_TEMP, M_WAITOK); + error = sooptcopyin(sopt, xcfg, size, sizeof(*xcfg)); + if (error != 0) { + free(xcfg, M_TEMP); + break; + } + + if (opt == IP_FW_TABLE_XGETCFG) { + error = ipfw_getconfig_table(chain, xcfg, &size); + if (error == 0) + error = sooptcopyout(sopt, xcfg, size); + } else + error = ipfw_setconfig_table(chain, xcfg, &size); + + free(xcfg, M_TEMP); + } + break; + + /*--- NAT operations are protected by the IPFW_LOCK ---*/ case IP_FW_NAT_CFG: if (IPFW_NAT_LOADED) Index: sys/netpfil/ipfw/ip_fw_table.c =================================================================== --- sys/netpfil/ipfw/ip_fw_table.c (revision 266310) +++ sys/netpfil/ipfw/ip_fw_table.c (working copy) @@ -35,8 +35,13 @@ __FBSDID("$FreeBSD$"); * As a degenerate case we can interpret keys as 32-bit integers * (with a /32 mask). * - * The table is protected by the IPFW lock even for manipulation coming - * from userland, because operations are typically fast. + * Locking resembles ipfw rules model: + * changes in: table entries, table count and table_info are protected by holding + * both UH and main write locks. + * changes in table_config struture are protected by UH lock. + * + * Userland readers should use UH lock if possible. + * */ #include "opt_ipfw.h" @@ -48,12 +53,14 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include #include #include #include +#include #include /* ip_fw.h requires IFNAMSIZ */ #include #include @@ -101,6 +108,65 @@ struct table_xentry { }; /* + * filled for non-CIDR or named tables + * + * Table has the following `type` concepts: + * + * `tabletype` represents lookup key type (cidr, ifp, uid, etc..) + * `ftype` is pure userland field helping to properly format table data + * `atype` represents exact lookup algorithm for given tabletype. + * For example, we can use more efficient search schemes if we plan + * to use some specific table for storing host-routes only. + * + */ +struct table_config { + uint8_t tabletype; /* lookup table types */ + uint8_t ftype; /* format table type */ + uint8_t atype; /* algorith type */ + uint8_t spare0; + uint32_t refcnt; /* Number of references */ + uint32_t count; /* Number of records */ + struct table_info *ti; + TAILQ_ENTRY(table_config) next; /* namehash */ + char tablename[64]; /* table name */ +}; + +typedef int (table_lookup_t)(void *state, void *xstate, void *key, + uint32_t keylen, uint32_t *val); + +struct table_info { + void *state; /* IPv4 tables */ + void *xstate;/* extended tables */ + table_lookup_t *lookup;/* lookup function */ + struct table_config *cfg; /* Additional data, can be NULL */ +}; + +static int lookup_cidr(void *, void *, void *, uint32_t, uint32_t *); +static int lookup_iface(void *, void *, void *, uint32_t, uint32_t *); + +static table_lookup_t (*tablehandlers[]) = { + NULL, /* type 0 unused */ + lookup_cidr, /* IPFW_TABLE_CIDR */ + lookup_iface, /* IPFW_TABLE_INTERFACE */ +}; + +#define TABLETYPE(t) ((t)->cfg != NULL ? (t)->cfg->tabletype:IPFW_TABLE_CIDR) +#define TABLEFTYPE(ti) ((ti)->cfg != NULL ? (ti)->cfg->ftype : 0) +#define TABLEREFS(ti) ((ti)->cfg != NULL ? (ti)->cfg->refcnt : 0) + +#define TABLENAME_HASH_SIZE 32 +struct tables_config { + TAILQ_HEAD(, table_config) thash[TABLENAME_HASH_SIZE]; +}; + +#define TABLENAME_HASH(n) (fnv_32_str(n,FNV1_32_INIT)%TABLENAME_HASH_SIZE) +static struct table_info *find_table(struct ip_fw_chain *ch, char *tablename); + +static void flush_table(struct ip_fw_chain *ch, struct table_info *ti); +static int change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, + struct table_info *ti2, table_lookup_t lookup); + +/* * The radix code expects addr and mask to be array of bytes, * with the first byte being the length of the array. rn_inithead * is called with the offset in bits of the lookup key within the @@ -145,13 +211,19 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 struct radix_node *rn; in_addr_t addr; int offset; - void *ent_ptr; + uintptr_t state_off; + void *ent_ptr, *tablestate; struct sockaddr *addr_ptr, *mask_ptr; + struct table_info *ti; + struct table_config *cfg; char c; if (tbl >= V_fw_tables_max) return (EINVAL); + tablestate = ch->tablestate; + ti = &((struct table_info *)tablestate)[tbl]; + switch (type) { case IPFW_TABLE_CIDR: if (plen == sizeof(in_addr_t)) { @@ -170,7 +242,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 addr = *((in_addr_t *)paddr); ent->addr.sin_addr.s_addr = addr & ent->mask.sin_addr.s_addr; /* Set pointers */ - rnh_ptr = &ch->tables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->state; ent_ptr = ent; addr_ptr = (struct sockaddr *)&ent->addr; mask_ptr = (struct sockaddr *)&ent->mask; @@ -191,7 +263,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 memcpy(&xent->a.addr6.sin6_addr, paddr, sizeof(struct in6_addr)); APPLY_MASK(&xent->a.addr6.sin6_addr, &xent->m.mask6.sin6_addr); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->xstate; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.addr6; mask_ptr = (struct sockaddr *)&xent->m.mask6; @@ -227,7 +299,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 mask_ptr = (struct sockaddr *)&xent->m.ifmask; #endif /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->xstate; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.iface; mask_ptr = NULL; @@ -237,48 +309,67 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 return (EINVAL); } + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); + /* + * Check if tablestate was reallocated. + */ + if (ch->tablestate != tablestate) { + state_off = (uintptr_t)ti - (uintptr_t)tablestate; + ti = (struct table_info *) + (state_off + (uintptr_t)ch->tablestate); + + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; + rnh_ptr = (struct radix_node_head **) + (state_off + (uintptr_t)ch->tablestate); + } + /* Check if tabletype is valid */ - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { + if (TABLETYPE(ti) != type) { IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); free(ent_ptr, M_IPFW_TBL); - return (EINVAL); + return (EFTYPE); } /* Check if radix tree exists */ if ((rnh = *rnh_ptr) == NULL) { IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); /* Create radix for a new table */ if (!rn_inithead((void **)&rnh, offset)) { free(ent_ptr, M_IPFW_TBL); return (ENOMEM); } + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); if (*rnh_ptr != NULL) { /* Tree is already attached by other thread */ rn_detachhead((void **)&rnh); rnh = *rnh_ptr; /* Check table type another time */ - if (ch->tabletype[tbl] != type) { + if (TABLETYPE(ti) != type) { IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); free(ent_ptr, M_IPFW_TBL); - return (EINVAL); + return (EFTYPE); } } else { + /* new trie */ *rnh_ptr = rnh; - /* - * Set table type. It can be set already - * (if we have IPv6-only table) but setting - * it another time does not hurt - */ - ch->tabletype[tbl] = type; } } rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); + + /* Maintain number of records */ + if (rn != NULL && (cfg = ti->cfg) != NULL) + cfg->count++; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); if (rn == NULL) { free(ent_ptr, M_IPFW_TBL); @@ -296,11 +387,18 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 in_addr_t addr; struct sockaddr_in sa, mask; struct sockaddr *sa_ptr, *mask_ptr; + struct table_info *ti; + struct table_config *cfg; + void *tablestate; + uintptr_t state_off; char c; if (tbl >= V_fw_tables_max) return (EINVAL); + tablestate = ch->tablestate; + ti = &((struct table_info *)tablestate)[tbl]; + switch (type) { case IPFW_TABLE_CIDR: if (plen == sizeof(in_addr_t)) { @@ -310,7 +408,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); addr = *((in_addr_t *)paddr); sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; - rnh_ptr = &ch->tables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->state; sa_ptr = (struct sockaddr *)&sa; mask_ptr = (struct sockaddr *)&mask; #ifdef INET6 @@ -327,7 +425,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 ipv6_writemask(&mask6.sin6_addr, mlen); memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); - rnh_ptr = &ch->xtables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->xstate; sa_ptr = (struct sockaddr *)&sa6; mask_ptr = (struct sockaddr *)&mask6; #endif @@ -362,7 +460,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 mask_ptr = NULL; memcpy(ifname.ifname, paddr, mlen); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; + rnh_ptr = (struct radix_node_head **)&ti->xstate; sa_ptr = (struct sockaddr *)&ifname; break; @@ -371,19 +469,42 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 return (EINVAL); } + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); + + /* + * Check if tablestate was reallocated. + */ + if (ch->tablestate != tablestate) { + state_off = (uintptr_t)ti - (uintptr_t)tablestate; + ti = (struct table_info *) + (state_off + (uintptr_t)ch->tablestate); + + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; + rnh_ptr = (struct radix_node_head **) + (state_off + (uintptr_t)ch->tablestate); + } + if ((rnh = *rnh_ptr) == NULL) { IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); return (ESRCH); } - if (ch->tabletype[tbl] != type) { + if (TABLETYPE(ti) != type) { IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); return (EINVAL); } ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, rnh); + + /* Maintain number of records */ + if (ent != NULL && (cfg = ti->cfg) != NULL) + cfg->count--; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); if (ent == NULL) return (ESRCH); @@ -392,7 +513,381 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 return (0); } +/* + * binds newly-created config to given table + */ static int +bind_config(struct ip_fw_chain *ch, uint32_t tbl, struct table_info *ti, + struct table_config *cfg) +{ + int error; + uint32_t sz, cnt; + + /* Set defaults */ + cfg->tabletype = IPFW_TABLE_CIDR; + + /* count current number of entries */ + if ((error = ipfw_count_xtable(ch, tbl, &sz, &cnt)) != 0) + return (error); + cfg->count = cnt; + + ti->cfg = cfg; + cfg->ti = ti; + + return (0); +} + +static struct table_info * +find_table(struct ip_fw_chain *ch, char *tablename) +{ + struct tables_config *tc; + struct table_config *cfg; + int hash; + + hash = TABLENAME_HASH(tablename); + tc = (struct tables_config *)ch->tablecfg; + + TAILQ_FOREACH(cfg, &tc->thash[hash], next) { + if (strcmp(cfg->tablename, tablename) == 0) + return (cfg->ti); + } + + return (NULL); +} + +/* + * Fills in supplied buffer with table configuration info. + */ +int +ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) +{ + struct table_info *ti; + struct table_config *cfg; + uint32_t tbl, tsz, cnt; + + tbl = xcfg->tbl; + + IPFW_UH_RLOCK(ch); + + if (xcfg->mask & IPFW_XCFG_NUMID) { + /* Table is identified by number */ + tbl = xcfg->tbl; + + if (tbl >= V_fw_tables_max) { + IPFW_UH_RUNLOCK(ch); + return (EINVAL); + } + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { + /* Let's try to find table by name */ + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { + IPFW_UH_RUNLOCK(ch); + return (EINVAL); + } + + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { + IPFW_UH_RUNLOCK(ch); + return (ESRCH); + } + + /* Guess table number based on offset */ + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); + } else { + IPFW_UH_RUNLOCK(ch); + return (ESRCH); + } + + ti = &((struct table_info *)ch->tablestate)[tbl]; + cfg = ti->cfg; + + /* Save table id anywat */ + xcfg->tbl = tbl; + + if ((xcfg->mask & IPFW_XCFG_NAME) != 0 ) { + if (cfg == NULL || cfg->tablename == NULL) { + xcfg->tablename[0] = '\0'; + xcfg->mask &= ~IPFW_XCFG_NAME; + } else + strlcpy(xcfg->tablename, cfg->tablename, + sizeof(cfg->tablename)); + } + + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) + xcfg->type = TABLETYPE(ti); + + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) + xcfg->ftype = TABLEFTYPE(ti); + + if ((xcfg->mask & IPFW_XCFG_REFS) != 0) + xcfg->refcnt = TABLEREFS(ti); + + if ((xcfg->mask & IPFW_XCFG_CNT) != 0) { + /* + * Use items count from cfg, if it exists. + * Otherwise, calculate manually. + */ + if (cfg != NULL) + xcfg->count = cfg->count; + else { + IPFW_RLOCK(ch); + ipfw_count_xtable(ch, tbl, &tsz, &cnt); + IPFW_RUNLOCK(ch); + xcfg->count = cnt; + } + } + + IPFW_UH_RUNLOCK(ch); + + return (0); +} + +/* + * Fills in supplied buffer with table configuration info. + */ +int +ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) +{ + struct table_info *ti, ti_storage, *ti2; + struct table_config *cfg; + uint32_t tbl; + int l; + int error; + + tbl = xcfg->tbl; + + /* Do some preliminary checking */ + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { + if (xcfg->type > IPFW_TABLE_MAXTYPE || xcfg->type == 0) + return (EINVAL); + } + + /* + * Check if tablename is null-terminated. + * More fine-grained checks should be done by userland. + */ + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { + l = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); + if (l == sizeof(xcfg->tablename) || l == 0) + return (EINVAL); + } + + /* Checl if we need to allocate config structure */ + IPFW_UH_RLOCK(ch); + + if (xcfg->mask & IPFW_XCFG_NUMID) { + /* Table is identified by number */ + tbl = xcfg->tbl; + + if (tbl >= V_fw_tables_max) { + IPFW_UH_RUNLOCK(ch); + return (EINVAL); + } + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { + /* Let's try to find table by name */ + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { + IPFW_UH_RUNLOCK(ch); + return (EINVAL); + } + + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { + IPFW_UH_RUNLOCK(ch); + return (ESRCH); + } + + /* Guess table number based on offset */ + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); + } + + ti = &((struct table_info *)ch->tablestate)[tbl]; + cfg = ti->cfg; + IPFW_UH_RUNLOCK(ch); + + /* + * Allocate new config structure if needed. + * cfg represents pointer to new structure or NULL + */ + if (cfg == NULL) + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_WAITOK | M_ZERO); + else + cfg = NULL; + + IPFW_UH_WLOCK(ch); + /* + * We need to check another time since there is probability that + * V_fw_tables_max was changed or ti->cfg was destroyed + */ + + if (tbl >= V_fw_tables_max) { + IPFW_UH_WUNLOCK(ch); + return (EINVAL); + } + + ti = &((struct table_info *)ch->tablestate)[tbl]; + if (ti->cfg == NULL) { + if (cfg == NULL) { + /* destroy_table() had happened before IPFW_UH_WLOCK */ + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_NOWAIT|M_ZERO); + if (cfg == NULL) { + IPFW_UH_WUNLOCK(ch); + return (ENOMEM); + } + } + + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { + IPFW_UH_WUNLOCK(ch); + free(cfg, M_IPFW_TBL); + return (error); + } + } else if (cfg != NULL) { + /* + * ti->cfg has been allocated by other thread. + * Free our allocation. + */ + free(cfg, M_IPFW_TBL); + } + + cfg = ti->cfg; + + /* + * Pretend to be atomic: check everything before doing actual job + */ + error = 0; + + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { + /* + * We can set type if + * 1) no one hold any references to our table + * 2) we have no records in given table + */ + + if ((cfg->tabletype != xcfg->type) && (cfg->refcnt > 0) && + (cfg->count > 0)) + error = EBUSY; + } + + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { + /* Check if new table name exists */ + if (find_table(ch, xcfg->tablename) != NULL) + error = EEXIST; + } + + if (error != 0) { + IPFW_UH_WUNLOCK(ch); + return (error); + } + + /* Everything checked, let's get the job done */ + ti2 = NULL; + + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { + /* we need to change table_info here */ + IPFW_WLOCK(ch); + cfg->tabletype = xcfg->type; + if (change_table_handler(ch, ti, &ti_storage, + tablehandlers[cfg->tabletype]) != 0) + ti2 = &ti_storage; + IPFW_WUNLOCK(ch); + } + + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { + int hash; + struct tables_config *tc; + tc = (struct tables_config *)ch->tablecfg; + + if (strlen(cfg->tablename) != 0) { + /* unlink from old one */ + hash = TABLENAME_HASH(cfg->tablename); + TAILQ_REMOVE(&tc->thash[hash], cfg, next); + } + strlcpy(cfg->tablename, xcfg->tablename, + sizeof(cfg->tablename)); + /* link new one */ + hash = TABLENAME_HASH(cfg->tablename); + TAILQ_INSERT_HEAD(&tc->thash[hash], cfg, next); + } + + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) + cfg->ftype = xcfg->ftype; + + IPFW_UH_WUNLOCK(ch); + + /* Free old table state if set */ + if (ti2 != NULL) + flush_table(ch, ti2); + + return (0); +} + + +/* + * Checks if given table's type is the same as @type. + * If true, increment @tbl refcount + */ +int +ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype) +{ + struct table_info *ti; + struct table_config *cfg; + int error; + + IPFW_UH_WLOCK_ASSERT(ch); + + if (tbl >= V_fw_tables_max) + return (EINVAL); + + ti = &((struct table_info *)ch->tablestate)[tbl]; + + if (TABLETYPE(ti) != type) + return (EFTYPE); + + /* Ignore ftype for now */ + + if (ti->cfg == NULL) { + /* + * Let's create confdata. + * XXX: we can fail here + */ + cfg = malloc(sizeof(struct table_config), M_IPFW_TBL, + M_NOWAIT | M_ZERO); + + if (cfg == NULL) + return (ENOMEM); + + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { + free(cfg, M_IPFW_TBL); + return (error); + } + } + + ti->cfg->refcnt++; + + return (0); +} + +int +ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl) +{ + struct table_info *ti; + + IPFW_UH_WLOCK_ASSERT(ch); + + if (tbl >= V_fw_tables_max) + return (EINVAL); + + ti = &((struct table_info *)ch->tablestate)[tbl]; + + KASSERT(ti->cfg != 0, ("ipfw: no config for table %d", tbl)); + + KASSERT(ti->cfg->refcnt > 0, ("ipfw: refcnt for table %d is %d", + tbl, ti->cfg->refcnt)); + + ti->cfg->refcnt--; + + return (0); +} + +static int flush_table_entry(struct radix_node *rn, void *arg) { struct radix_node_head * const rnh = arg; @@ -405,40 +900,138 @@ flush_table_entry(struct radix_node *rn, void *arg return (0); } +/* + * Flushes data from table state pointers. + * Frees pointers itself. + */ +static void +flush_table(struct ip_fw_chain *ch, struct table_info *ti) +{ + struct radix_node_head *rnh; + + if ((rnh = ti->state) != NULL) { + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead((void **)&rnh); + } + + if ((rnh = ti->xstate) != NULL) { + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead((void **)&rnh); + } +} + +/* + * change table handler saving previous state in @ti2. + * Returns 1 if hable had some state, 0 otherwise. + */ +static int +change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, + struct table_info *ti2, table_lookup_t lookup) +{ + + if (ti->state == NULL && ti->state == NULL) { + /* No state. Set handler and return. */ + ti->lookup = lookup; + return (0); + } + + *ti2 = *ti; + + ti->state = NULL; + ti->xstate = NULL; + ti->lookup = lookup; + + return (1); +} + +/* + * flushes all data in given table leaving table type/naming + * intact. + */ int ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) { - struct radix_node_head *rnh, *xrnh; + struct table_info *ti, tti; + struct table_config *cfg; - if (tbl >= V_fw_tables_max) + IPFW_UH_WLOCK(ch); + IPFW_WLOCK(ch); + + if (tbl >= V_fw_tables_max) { + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); return (EINVAL); + } - /* - * We free both (IPv4 and extended) radix trees and - * clear table type here to permit table to be reused - * for different type without module reload - */ + ti = &((struct table_info *)ch->tablestate)[tbl]; + tti = *ti; + /* Remove state tables from main structure */ + ti->state = NULL; + ti->xstate = NULL; + if ((cfg = ti->cfg) != NULL) + cfg->count = 0; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + flush_table(ch, &tti); + + return (0); +} + +static void +destroy_table(struct ip_fw_chain *ch, struct table_info *ti) +{ + struct table_config *cfg; + + flush_table(ch, ti); + + if ((cfg = ti->cfg) != NULL) { + /* Free configuration state */ + free(cfg, M_IPFW_TBL); + ti->cfg = NULL; + } + + /* Set lookup pointer back to CIDR */ + ti->lookup = lookup_cidr; +} + +/* + * Destroys given table iff no rules are referencing it. + * Flushes all data in tablestate, destroys any + * special configuration/naming associated with the table + * and sets its type (ti->cfg == NULL) back to CIDR. + */ +int +ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl) +{ + struct table_info *ti, tti; + struct table_config *cfg; + + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); - /* Set IPv4 table pointer to zero */ - if ((rnh = ch->tables[tbl]) != NULL) - ch->tables[tbl] = NULL; - /* Set extended table pointer to zero */ - if ((xrnh = ch->xtables[tbl]) != NULL) - ch->xtables[tbl] = NULL; - /* Zero table type */ - ch->tabletype[tbl] = 0; - IPFW_WUNLOCK(ch); - if (rnh != NULL) { - rnh->rnh_walktree(rnh, flush_table_entry, rnh); - rn_detachhead((void **)&rnh); + if (tbl >= V_fw_tables_max) { + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + return (EINVAL); } - if (xrnh != NULL) { - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); - rn_detachhead((void **)&xrnh); + ti = &((struct table_info *)ch->tablestate)[tbl]; + if (((cfg = ti->cfg) != NULL) && (cfg->refcnt != 0)) { + /* Table is referenced by some rules */ + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + return (EBUSY); } + tti = *ti; + /* Remove state tables from main structure */ + ti->state = NULL; + ti->xstate = NULL; + ti->cfg = NULL; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + + destroy_table(ch, &tti); return (0); } @@ -446,152 +1039,179 @@ ipfw_flush_table(struct ip_fw_chain *ch, uint16_t void ipfw_destroy_tables(struct ip_fw_chain *ch) { - uint16_t tbl; + uint32_t tbl; + struct table_info *ti; /* Flush all tables */ - for (tbl = 0; tbl < V_fw_tables_max; tbl++) - ipfw_flush_table(ch, tbl); + ti = (struct table_info *)ch->tablestate; + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) + destroy_table(ch, ti); /* Free pointers itself */ - free(ch->tables, M_IPFW); - free(ch->xtables, M_IPFW); - free(ch->tabletype, M_IPFW); + free(ch->tablestate, M_IPFW_TBL); + free(ch->tablecfg, M_IPFW_TBL); } int ipfw_init_tables(struct ip_fw_chain *ch) { + struct table_info *ti; + struct tables_config *tc; + uint32_t tbl; + /* Allocate pointers */ - ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + ch->tablestate = malloc(V_fw_tables_max * sizeof(struct table_info), + M_IPFW_TBL, M_WAITOK | M_ZERO); + + ch->tablecfg = malloc(sizeof(struct tables_config), M_IPFW_TBL, + M_WAITOK | M_ZERO); + + /* Set initial handlers for all tables */ + ti = (struct table_info *)ch->tablestate; + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) + ti->lookup = lookup_cidr; + + /* Set up tablenames hash */ + tc = ch->tablecfg; + for (tbl = 0; tbl < TABLENAME_HASH_SIZE; tbl++) + TAILQ_INIT(&tc->thash[tbl]); + return (0); } int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables) { - struct radix_node_head **tables, **xtables, *rnh; - struct radix_node_head **tables_old, **xtables_old; - uint8_t *tabletype, *tabletype_old; unsigned int ntables_old, tbl; + struct table_info *ti, *ti_old, *ti_new; /* Check new value for validity */ if (ntables > IPFW_TABLES_MAX) ntables = IPFW_TABLES_MAX; /* Allocate new pointers */ - tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + ti_new = malloc(ntables * sizeof(struct table_info), + M_IPFW_TBL, M_WAITOK | M_ZERO); + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; /* Copy old table pointers */ - memcpy(tables, ch->tables, sizeof(void *) * tbl); - memcpy(xtables, ch->xtables, sizeof(void *) * tbl); - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); + memcpy(ti_new, ch->tablestate, sizeof(struct table_info) * tbl); /* Change pointers and number of tables */ - tables_old = ch->tables; - xtables_old = ch->xtables; - tabletype_old = ch->tabletype; - ch->tables = tables; - ch->xtables = xtables; - ch->tabletype = tabletype; + ti_old = (struct table_info *)ch->tablestate; + ch->tablestate = ti_new; ntables_old = V_fw_tables_max; V_fw_tables_max = ntables; IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); /* Check if we need to destroy radix trees */ if (ntables < ntables_old) { - for (tbl = ntables; tbl < ntables_old; tbl++) { - if ((rnh = tables_old[tbl]) != NULL) { - rnh->rnh_walktree(rnh, flush_table_entry, rnh); - rn_detachhead((void **)&rnh); - } + ti = &ti_old[ntables]; + for (tbl = ntables; tbl < ntables_old; tbl++, ti++) + destroy_table(ch, ti); + } - if ((rnh = xtables_old[tbl]) != NULL) { - rnh->rnh_walktree(rnh, flush_table_entry, rnh); - rn_detachhead((void **)&rnh); - } - } + /* Check if we need to setup new ones */ + if (ntables > ntables_old) { + ti = &ti_new[ntables_old]; + for (tbl = ntables_old; tbl < ntables; tbl++, ti++) + ti->lookup = lookup_cidr; } /* Free old pointers */ - free(tables_old, M_IPFW); - free(xtables_old, M_IPFW); - free(tabletype_old, M_IPFW); + free(ti_old, M_IPFW_TBL); return (0); } -int -ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, - uint32_t *val) +static int +lookup_cidr(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) { struct radix_node_head *rnh; struct table_entry *ent; - struct sockaddr_in sa; + struct table_xentry *xent; - if (tbl >= V_fw_tables_max) + + if (keylen == sizeof(struct in_addr)) { + /* IPv4 lookup */ + struct sockaddr_in sa; + + if ((rnh = (struct radix_node_head *)st) == NULL) + return (0); + + KEY_LEN(sa) = KEY_LEN_INET; + sa.sin_addr.s_addr = ((struct in_addr *)key)->s_addr; + ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); + if (ent != NULL) { + *val = ent->value; + return (1); + } + return (0); - if ((rnh = ch->tables[tbl]) == NULL) + } + + /* IPv6 lookup */ + struct sockaddr_in6 sa6; + + if ((rnh = (struct radix_node_head *)xst) == NULL) return (0); - KEY_LEN(sa) = KEY_LEN_INET; - sa.sin_addr.s_addr = addr; - ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); - if (ent != NULL) { - *val = ent->value; + + KEY_LEN(sa6) = KEY_LEN_INET6; + memcpy(&sa6.sin6_addr, key, sizeof(struct in6_addr)); + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); + + if (xent != NULL) { + *val = xent->value; return (1); } + return (0); } -int -ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint32_t *val, int type) +static int +lookup_iface(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) { struct radix_node_head *rnh; + struct xaddr_iface iface; struct table_xentry *xent; - struct sockaddr_in6 sa6; - struct xaddr_iface iface; - if (tbl >= V_fw_tables_max) + if ((rnh = (struct radix_node_head *)xst) == NULL) return (0); - if ((rnh = ch->xtables[tbl]) == NULL) - return (0); - switch (type) { - case IPFW_TABLE_CIDR: - KEY_LEN(sa6) = KEY_LEN_INET6; - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); - break; - - case IPFW_TABLE_INTERFACE: - KEY_LEN(iface) = KEY_LEN_IFACE + - strlcpy(iface.ifname, (char *)paddr, IF_NAMESIZE) + 1; - /* Assume direct match */ - /* FIXME: Add interface pattern matching */ - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); - break; - - default: - return (0); - } - + /* IPv6 lookup */ + KEY_LEN(iface) = KEY_LEN_IFACE + + strlcpy(iface.ifname, (char *)key, IF_NAMESIZE) + 1; + /* Assume direct match */ + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); if (xent != NULL) { *val = xent->value; return (1); } + return (0); } +int +ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, + uint32_t keylen, void *key, uint32_t *val) +{ + struct table_info *ti; + + if (tbl >= V_fw_tables_max) + return (0); + + ti = &((struct table_info *)ch->tablestate)[tbl]; + + return (ti->lookup(ti->state, ti->xstate, key, keylen, val)); +} + static int count_table_entry(struct radix_node *rn, void *arg) { @@ -605,11 +1225,15 @@ int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) { struct radix_node_head *rnh; + struct table_info *ti; if (tbl >= V_fw_tables_max) return (EINVAL); + + ti = &((struct table_info *)ch->tablestate)[tbl]; + *cnt = 0; - if ((rnh = ch->tables[tbl]) == NULL) + if ((rnh = ti->state) == NULL) return (0); rnh->rnh_walktree(rnh, count_table_entry, cnt); return (0); @@ -640,11 +1264,15 @@ int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) { struct radix_node_head *rnh; + struct table_info *ti; if (tbl->tbl >= V_fw_tables_max) return (EINVAL); + + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; + tbl->cnt = 0; - if ((rnh = ch->tables[tbl->tbl]) == NULL) + if ((rnh = ti->state) == NULL) return (0); rnh->rnh_walktree(rnh, dump_table_entry, tbl); return (0); @@ -660,20 +1288,32 @@ count_table_xentry(struct radix_node *rn, void *ar } int -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) +ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, + uint32_t *cnt) { struct radix_node_head *rnh; + struct table_info *ti; + uint32_t count; if (tbl >= V_fw_tables_max) return (EINVAL); - *cnt = 0; - if ((rnh = ch->tables[tbl]) != NULL) - rnh->rnh_walktree(rnh, count_table_xentry, cnt); - if ((rnh = ch->xtables[tbl]) != NULL) - rnh->rnh_walktree(rnh, count_table_xentry, cnt); + + ti = &((struct table_info *)ch->tablestate)[tbl]; + + *sz = 0; + count = 0; + if ((rnh = ti->state) != NULL) + rnh->rnh_walktree(rnh, count_table_xentry, sz); + if ((rnh = ti->xstate) != NULL) + rnh->rnh_walktree(rnh, count_table_xentry, sz); + count = *sz / sizeof(ipfw_table_xentry); + + if (cnt != NULL) + *cnt = count; + /* Return zero if table is empty */ - if (*cnt > 0) - (*cnt) += sizeof(ipfw_xtable); + if (*sz > 0) + (*sz) += sizeof(ipfw_xtable); return (0); } @@ -750,14 +1390,18 @@ int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) { struct radix_node_head *rnh; + struct table_info *ti; if (tbl->tbl >= V_fw_tables_max) return (EINVAL); + + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; + tbl->cnt = 0; - tbl->type = ch->tabletype[tbl->tbl]; - if ((rnh = ch->tables[tbl->tbl]) != NULL) + tbl->type = TABLETYPE(ti); + if ((rnh = ti->state) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); - if ((rnh = ch->xtables[tbl->tbl]) != NULL) + if ((rnh = ti->xstate) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); return (0); } --------------040508090907050000010802-- From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:01:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 256018CF for ; Mon, 19 May 2014 13:01:41 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D814F28E8 for ; Mon, 19 May 2014 13:01:40 +0000 (UTC) Received: from 95.108.170.210-red.dhcp.yndx.net ([95.108.170.210] helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WmJHx-000HFo-Cq; Mon, 19 May 2014 12:51:17 +0400 Message-ID: <537A0054.5000707@FreeBSD.org> Date: Mon, 19 May 2014 17:00:04 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Bill Yuan , Jason Hellenthal Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Dennis Yusupoff , FreeBSD Net , Marcelo Gondim X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:01:41 -0000 On 19.05.2014 11:51, Bill Yuan wrote: > Hi Alex, Hello Bill! > > You guys are chatting here! I agree with you, the table is the place should > be enhanced, and I am working in this way as described below > > 1. Support more types. > ip : cidr > ipv4 : same as ip > ipv6 : ip addr v6 > mac : mac address > iface : interface name > interface : same as iface > port : it is Alex's idea, I dont know how it works. Well, actually that's not mine. ipfw implement the following since long ago: + v = ((ipfw_insn_u32 *)cmd)->d[1]; + switch (v) { + case 0: + case 1: + /* IPv4 src/dst */ + break; + case 2: + case 3: + /* src/dst port */ + break; + case 4: + /* uid/gid */ + case 5: + /* jid */ + case 6: + /* dscp */ + break; + } I hope you're not using radix to implement mac addresses lookup? Anyway, it looks like we're doing similar things. Can you take a look on '[CFT]: ipfw named tables / different tabletypes' topic and see how much it conflicts with your changes? > > 2. Setup the table type > ipfw table type > it will setup the type of the table, and flush the table > > 3. Get table type > ipfw table type show > > 4. Add item into the table > ipfw table add > > a. get the type of table > b. if the type is not defined yet, that also means the table is new or > empty, > then guess the type based on the > c. format the and insert into the table. > > In this way so call "back compatible" > > 5. how to use table > > case 1 > ipfw add [line] allow icmp from "table(1)" to "table(2)" > in the ipfw userland command, it should check the table1 and table 2 should > be ipv4 or ipv6 type > > case 2 > ipfw add allow icmp from any to any MAC "table(3)" "table(4)" > in this case, the table(3) and table(4) should be a table of MAC addresses. > > case 3 > ipfw add allow icmp from any to any via table(5) > in this case, the table 5 should be table of interface names. > > > currently I am working on the mac type. :) > > > > > On Sun, May 18, 2014 at 12:47 PM, Jason Hellenthal > wrote: > >> >>> On May 18, 2014, at 0:12, Julian Elischer wrote: >>>> 2) Table type/name can be specified explicitly via one of the following >> commands: >>>> * ipfw table 1 create [type ] [name >> "table_name"] >>> type "ports" would be nice but tricky to do right. >> That . . . would be a great addition and have me switching from pf to ipfw. >> >> Pullllease do! :-) > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:03:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60735B0D for ; Mon, 19 May 2014 13:03:07 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 20F57296A for ; Mon, 19 May 2014 13:03:07 +0000 (UTC) Received: from 95.108.170.210-red.dhcp.yndx.net ([95.108.170.210] helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WmJJM-000HGZ-ID; Mon, 19 May 2014 12:52:44 +0400 Message-ID: <537A00AC.6050305@FreeBSD.org> Date: Mon, 19 May 2014 17:01:32 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Dennis Yusupoff , "freebsd-net@freebsd.org" Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> In-Reply-To: <5379C6B6.4030105@smartspb.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:03:07 -0000 On 19.05.2014 12:54, Dennis Yusupoff wrote: > Alex, Bill, it's a good news, glad to hear it. > > Let me ask even more functionality: > > 6. Test if entry exist in table: > ipfw table test > It extremely useful in case of big, unordered data in the table - for > example different networks with different mask. Now it's almost > impossible to find out is checked IP occurs in the table or not. Longest prefix match or exact match? From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:13:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E389E82; Mon, 19 May 2014 13:13:01 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3646B2A7B; Mon, 19 May 2014 13:13:01 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id rr13so5873190pbb.21 for ; Mon, 19 May 2014 06:13:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; bh=v9d+oMp+zxrytLw5hOgL9Gywuyd3EofIKeFxRXXXM/s=; b=o3nyIkapCDZVA0302LsuAfNYad2SrVahcSOxU5+40ZpsNUPIlHrVUqIQKWtdkTjizf T8QinmLF7RaILSNrdaKIf51mmJurJoewyax5/MsjQt+Ns7wYBghZpu5JdQcEh2E25L3j wV6GeUW9gyrZYZ7N5/juxCLiT5GeOpOZqriMkcwqiop4cFzzEqFttnEBSGOcf5/pXIgA hPJAcEK4O6eerXE6ScYTyHGVPANo+Yh8NwwHSTH6kJK5jtA8QCve9jC9i03XUle9Jy1X QuIy203UiRAypx1nykV3bJALWl1GUCYFGPyoA4mhQaHYKvrSkdbMHe8nnZKrvjHHU6xA J6/w== X-Received: by 10.66.193.104 with SMTP id hn8mr42547718pac.99.1400505180695; Mon, 19 May 2014 06:13:00 -0700 (PDT) Received: from [192.168.1.100] ([203.117.37.234]) by mx.google.com with ESMTPSA id sv10sm75611051pab.32.2014.05.19.06.12.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 19 May 2014 06:12:59 -0700 (PDT) Message-ID: <537A0356.7050104@gmail.com> Date: Mon, 19 May 2014 21:12:54 +0800 From: bycn82 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <537A0054.5000707@FreeBSD.org> In-Reply-To: <537A0054.5000707@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , Marcelo Gondim , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:13:01 -0000 On 5/19/14 21:00, Alexander V. Chernikov wrote: > On 19.05.2014 11:51, Bill Yuan wrote: >> Hi Alex, > Hello Bill! >> >> You guys are chatting here! I agree with you, the table is the place >> should >> be enhanced, and I am working in this way as described below >> >> 1. Support more types. >> ip : cidr >> ipv4 : same as ip >> ipv6 : ip addr v6 >> mac : mac address >> iface : interface name >> interface : same as iface >> port : it is Alex's idea, I dont know how it works. > Well, actually that's not mine. ipfw implement the following since > long ago: > + v = ((ipfw_insn_u32 *)cmd)->d[1]; > + switch (v) { > + case 0: > + case 1: > + /* IPv4 src/dst */ > + break; > + case 2: > + case 3: > + /* src/dst port */ > + break; > + case 4: > + /* uid/gid */ > + case 5: > + /* jid */ > + case 6: > + /* dscp */ > + break; > + } > > I hope you're not using radix to implement mac addresses lookup? > > Anyway, it looks like we're doing similar things. > Can you take a look on '[CFT]: ipfw named tables / different > tabletypes' topic and > see how much it conflicts with your changes? >> >> 2. Setup the table type >> ipfw table type >> it will setup the type of the table, and flush the table >> >> 3. Get table type >> ipfw table type show >> >> 4. Add item into the table >> ipfw table add >> >> a. get the type of table >> b. if the type is not defined yet, that also means the table is new or >> empty, >> then guess the type based on the >> c. format the and insert into the table. >> >> In this way so call "back compatible" >> >> 5. how to use table >> >> case 1 >> ipfw add [line] allow icmp from "table(1)" to "table(2)" >> in the ipfw userland command, it should check the table1 and table 2 >> should >> be ipv4 or ipv6 type >> >> case 2 >> ipfw add allow icmp from any to any MAC "table(3)" "table(4)" >> in this case, the table(3) and table(4) should be a table of MAC >> addresses. >> >> case 3 >> ipfw add allow icmp from any to any via table(5) >> in this case, the table 5 should be table of interface names. >> >> >> currently I am working on the mac type. :) >> >> >> >> >> On Sun, May 18, 2014 at 12:47 PM, Jason Hellenthal >> wrote: >> >>> >>>> On May 18, 2014, at 0:12, Julian Elischer wrote: >>>>> 2) Table type/name can be specified explicitly via one of the >>>>> following >>> commands: >>>>> * ipfw table 1 create [type ] [name >>> "table_name"] >>>> type "ports" would be nice but tricky to do right. >>> That . . . would be a great addition and have me switching from pf >>> to ipfw. >>> >>> Pullllease do! :-) >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > It is good to know that have company who is working in the same direction. and it is really feeling good to have user who is expecting this feature before implemented. :) You bring up the code first , I can try to add on a patch for the "mac" type or others , As a newbie here, I am not confident about how to implement is the best way. From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:21:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B322C1; Mon, 19 May 2014 13:21:42 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 348072B52; Mon, 19 May 2014 13:21:42 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bj1so5800485pad.13 for ; Mon, 19 May 2014 06:21:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; bh=u1xq1Ub7jrQWz+nSt248LxK9fhnFNj+haQWuLW0yvkg=; b=CrufwkBWeI1pFnsdV2EWyCli7KgCvr1kyZfRU5KcfvLAsi3UuJqfe3gR4DXsY2bFq4 G7cQRBR2E7KEih2RtERrfcs59S+z8e81/0jzYYieAlNtQpVtIejfQNnozEMxZxptIeWg g/r2NQe8AgnxvdOEYp4Dktj9K1t6Wkra8Zil7JXrbmU3D3gBkpHiCxvuBWbQ+qIIwtjV AUj+Vn3uKs2yPcphq11N2mtSVxmRV2leOK4Ecp27TlamBD96CGGAwIyB8Xq7F5Qa1wRV xpmfe6t9V/ravVUc5Ougpc+lYXxspydQgdOB06gfqNqU0l4TmwnjMn21y848JoK6Wdpw jvlw== X-Received: by 10.68.202.167 with SMTP id kj7mr43315712pbc.160.1400505701762; Mon, 19 May 2014 06:21:41 -0700 (PDT) Received: from [192.168.1.100] ([203.117.37.234]) by mx.google.com with ESMTPSA id is5sm30149224pbb.8.2014.05.19.06.21.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 19 May 2014 06:21:40 -0700 (PDT) Message-ID: <537A0560.2070902@gmail.com> Date: Mon, 19 May 2014 21:21:36 +0800 From: bycn82 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> <537A00AC.6050305@FreeBSD.org> In-Reply-To: <537A00AC.6050305@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:21:42 -0000 On 5/19/14 21:01, Alexander V. Chernikov wrote: > On 19.05.2014 12:54, Dennis Yusupoff wrote: >> Alex, Bill, it's a good news, glad to hear it. >> >> Let me ask even more functionality: >> >> 6. Test if entry exist in table: >> ipfw table test >> It extremely useful in case of big, unordered data in the table - for >> example different networks with different mask. Now it's almost >> impossible to find out is checked IP occurs in the table or not. > Longest prefix match or exact match? > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > It will be nice to have this feature, but since the `ipfw table list` is existing, so I think this can be implemented outside the ipfw. (personal opinion only ) bycn82 From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:26:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 825641CB for ; Mon, 19 May 2014 13:26:36 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A60F2B98 for ; Mon, 19 May 2014 13:26:35 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1WmNdj-000N7g-Ns for freebsd-net@freebsd.org; Mon, 19 May 2014 17:30:03 +0400 Message-ID: <537A0687.3050501@smartspb.net> Date: Mon, 19 May 2014 17:26:31 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> <537A00AC.6050305@FreeBSD.org> In-Reply-To: <537A00AC.6050305@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140519-0, 19.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:26:36 -0000 Longest prefix match, obviously. Doesn't see any reason to search for exact match in case of existing prefix with that ip. 19.05.2014 17:01, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: > On 19.05.2014 12:54, Dennis Yusupoff wrote: >> Alex, Bill, it's a good news, glad to hear it. >> >> Let me ask even more functionality: >> >> 6. Test if entry exist in table: >> ipfw table test >> It extremely useful in case of big, unordered data in the table - for >> example different networks with different mask. Now it's almost >> impossible to find out is checked IP occurs in the table or not. > Longest prefix match or exact match? > > -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Mon May 19 13:38:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83A1F758 for ; Mon, 19 May 2014 13:38:50 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A2462CC4 for ; Mon, 19 May 2014 13:38:50 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1WmNpb-000OF6-Mv for freebsd-net@freebsd.org; Mon, 19 May 2014 17:42:19 +0400 Message-ID: <537A0967.1000808@smartspb.net> Date: Mon, 19 May 2014 17:38:47 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 CC: "freebsd-net@freebsd.org" Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> <537A00AC.6050305@FreeBSD.org> <537A0560.2070902@gmail.com> In-Reply-To: <537A0560.2070902@gmail.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140519-0, 19.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:38:50 -0000 It's not enough, actually. Imagine what you have a table with different networks. If you'll try to find out is an IP belongs to some of that networks from the table, you should to write relatively serious "wrapper" with network range calculations in it. Or can you show differ (easier) way? So it's REALLY usefull to implement that functions "out-of-the-box". I'm risking to be annoying, but there is a good (from customers point of view) example of tables manipulation in Linux: ipset project (http://ipset.netfilter.org/ipset.man.html) 19.05.2014 17:21, bycn82 ÐŋÐļŅˆÐĩŅ‚: > It will be nice to have this feature, > but since the `ipfw table list` is existing, > so I think this can be implemented outside the ipfw. > (personal opinion only ) > -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Mon May 19 16:56:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7B79FB0 for ; Mon, 19 May 2014 16:56:11 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A80132EB2 for ; Mon, 19 May 2014 16:56:11 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id i17so9316239qcy.25 for ; Mon, 19 May 2014 09:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JUq+Xg5B9hWCyUUtSkeHwUEKJp3mSQ8BQQak66vzlOE=; b=KawNyc6mV7JoY2hln6kT1wXNrFFH3hcM1m2TLYLkF3njmxSCSKkE82WNktLHuXyJJK ysbxW393JGj8sMqfQ8CMav3t3L4l4GyraBBmETxjvxc3ZFoPeA/RlHGPVx8+0dkf7hdF NVnLghet4SkF8X43aFd7iKieC9N0Mg+ycHufA+YOdSQLg1XIgjHOBcLCuRS1mSyG6lxG ofasFfUZVoNLlLy8vwHN5HKHbemz0xFn9grmzKoGWTBDp1LRogzXPf0fxCoHrxU28Uff FLIx0DGoJGnQf4QtrR5XrbCR/Yfa/IAyfrVT9qcllMJMR3KKXY/23O4wHeHSLEcNi6Is oZ5g== MIME-Version: 1.0 X-Received: by 10.224.129.66 with SMTP id n2mr49771055qas.55.1400518570667; Mon, 19 May 2014 09:56:10 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Mon, 19 May 2014 09:56:10 -0700 (PDT) In-Reply-To: <20140519174117.G89611@sola.nimnet.asn.au> References: <201405181223.s4ICNTAf097378@fire.js.berklix.net> <20140518230242.GA28117@onelab2.iet.unipi.it> <20140519174117.G89611@sola.nimnet.asn.au> Date: Mon, 19 May 2014 09:56:10 -0700 X-Google-Sender-Auth: vzqKau-e2vvnz-X4auE6qTJ-W_0 Message-ID: Subject: Re: netmap and other discussions on freebsd-net: please be open minded. From: Adrian Chadd To: Ian Smith Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Luigi Rizzo X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 16:56:12 -0000 On 19 May 2014 01:42, Ian Smith wrote: > On Mon, 19 May 2014 01:02:42 _0200, Luigi Rizzo wrote: > > Folks, i have two requests for you: > > > > 1. please do not complain about questions on this list related > > to a core network-related FreeBSD subsystem (netmap, dummynet, > > netgraph, tcp stack...) even if they are concerned with ports > > to Linux or other OSes or to userspace. > > > > At least in principle, these topics _are_ relevant here precisely > > because they relate to code whose main home is in the FreeBSD > > source tree, and bugs or feature suggestions etc that may emerge > > will directly benefit FreeBSD. > > I'm glad I waited before responding to the OP+1 or I may have been > somewhat less polite, let alone authoritative .. so thanks. > > I can only add that projects such as netmap - and ipfw & dummynet for > Linux for that matter - can only serve to lower the wall, in perception > at least, between the merits of either OS for purpose, and further > increase the exposure of the quality of FreeBSD networking to the Linux > masses; ie, I believe this can only be a net gain for FreeBSD overall. > > > 2. on the other hand, it would be good if people could avoid questions like > > "give me step by step instructions on how to install/run/use X". > > There are notes in README and Makefiles, something on the > > web pages, and quick web searches should point you to previous > > mailing list discussions on the same topic _before asking_. > > googling 'netmap linux' seems to point to most everything relevant, > including some discussions on Linux lists, and perhaps illustrative > of (1) is http://www.ntop.org/pf_ring/dna-vs-netmap/ > > cheers, Ian (no, I haven't played with netmap, but it sure smells good!) My experience is the opposite - it makes it easier to migrate away from FreeBSD to Linux. You have to be careful. -a From owner-freebsd-net@FreeBSD.ORG Mon May 19 20:10:02 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39EAB1C4 for ; Mon, 19 May 2014 20:10:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 278A72235 for ; Mon, 19 May 2014 20:10:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4JKA1mV047693 for ; Mon, 19 May 2014 20:10:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4JKA1xN047692; Mon, 19 May 2014 20:10:01 GMT (envelope-from gnats) Date: Mon, 19 May 2014 20:10:01 GMT Message-Id: <201405192010.s4JKA1xN047692@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: hiren panchasara Subject: Re: kern/184311: [bge] [panic] kernel panic with bge(4) on SunFire X2100 Reply-To: hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 20:10:02 -0000 The following reply was made to PR kern/184311; it has been noted by GNATS. From: hiren panchasara To: bug-followup@FreeBSD.org, benjamin.stier@ub.uni-tuebingen.de Cc: Subject: Re: kern/184311: [bge] [panic] kernel panic with bge(4) on SunFire X2100 Date: Mon, 19 May 2014 13:08:14 -0700 I just hit this on FreeBSD10. vgapci0: mem 0xc9000000-0xc9ffffff,0xd0000000-0xd7ffffff irq 16 at device 5.0 on pci1 vgapci0: Boot video device pcib2: at device 12.0 on pci0 pci2: on pcib2 bge0: mem 0xca000000-0xca00ffff irq 16 at device 0.0 on pci2 bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:17:08:92:b6:e9 pcib3: at device 13.0 on pci0 pci3: on pcib3 bge1: mem 0xca100000-0xca10ffff irq 19 at device 0.0 on pci3 bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: no media present ifmedia_set: no match for 0x0/0xfffffff panic: ifmedia_set cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff81a3f1b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffffff81a3f260 vpanic() at vpanic+0x126/frame 0xffffffff81a3f2a0 panic() at panic+0x43/frame 0xffffffff81a3f300 ifmedia_set() at ifmedia_set+0x5a/frame 0xffffffff81a3f310 brgphy_attach() at brgphy_attach+0x3a4/frame 0xffffffff81a3f350 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f3b0 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f3d0 miibus_attach() at miibus_attach+0xbd/frame 0xffffffff81a3f410 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f470 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f490 mii_attach() at mii_attach+0x435/frame 0xffffffff81a3f520 bge_attach() at bge_attach+0x4151/frame 0xffffffff81a3f600 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f660 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f680 acpi_pci_attach() at acpi_pci_attach+0x15f/frame 0xffffffff81a3f6d0 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f730 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f750 acpi_pcib_attach() at acpi_pcib_attach+0x23d/frame 0xffffffff81a3f7a0 acpi_pcib_pci_attach() at acpi_pcib_pci_attach+0x9f/frame 0xffffffff81a3f7e0 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f840 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f860 acpi_pci_attach() at acpi_pci_attach+0x15f/frame 0xffffffff81a3f8b0 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3f910 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3f930 acpi_pcib_attach() at acpi_pcib_attach+0x23d/frame 0xffffffff81a3f980 acpi_pcib_acpi_attach() at acpi_pcib_acpi_attach+0x2a9/frame 0xffffffff81a3f9d0 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3fa30 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3fa50 acpi_attach() at acpi_attach+0xdd4/frame 0xffffffff81a3fb10 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3fb70 bus_generic_attach() at bus_generic_attach+0x4a/frame 0xffffffff81a3fb90 nexus_acpi_attach() at nexus_acpi_attach+0x76/frame 0xffffffff81a3fbc0 device_attach() at device_attach+0x3a2/frame 0xffffffff81a3fc20 bus_generic_new_pass() at bus_generic_new_pass+0x116/frame 0xffffffff81a3fc50 bus_set_pass() at bus_set_pass+0x8f/frame 0xffffffff81a3fc80 configure() at configure+0xa/frame 0xffffffff81a3fc90 mi_startup() at mi_startup+0x118/frame 0xffffffff81a3fcb0 btext() at btext+0x2c Uptime: 1s Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... From owner-freebsd-net@FreeBSD.ORG Tue May 20 10:09:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2957488 for ; Tue, 20 May 2014 10:09:25 +0000 (UTC) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [84.52.89.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81BC423A9 for ; Tue, 20 May 2014 10:09:25 +0000 (UTC) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 734997F4D9; Tue, 20 May 2014 14:00:31 +0400 (MSK) X-DKIM: Sendmail DKIM Filter v2.8.2 shelob.oktetlabs.ru 734997F4D9 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1400580031; bh=tRb4ANkNETRGOTXLsji1ZMVGDWmSTmZT9aUZooOswUw=; l=1523; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type; b=VgOt8Ds0qsxtMNVEBIpzfxnT4rXLXWOWPdZcHx+8e+xPcqHGfLttux62DViLQsk9P Tsp68nw79mbRkAyUwSb3YoTflA5xwCUZrdVdWOnfS3tkyUsEGUfWepTnpOVcjX4nZB EEL5QowOv9LP+bOcfenF43VuHdthOHWYb9Zpz+8M= Message-ID: <537B27BF.7000800@oktetlabs.ru> Date: Tue, 20 May 2014 14:00:31 +0400 From: Andrew Rybchenko Organization: OKTET Labs User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: sfxge: Do no allow EFSYS_MEM_ALLOC sleep Content-Type: multipart/mixed; boundary="------------030702020709090005030108" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 10:09:25 -0000 This is a multi-part message in MIME format. --------------030702020709090005030108 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit It solves locking problem when EFSYS_MEM_ALLOC is called in the context holding a mutex (not allowed to sleep). E.g. on interface bring up or multicast addresses addition. --------------030702020709090005030108 Content-Type: text/x-patch; name="alloc_nowait.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="alloc_nowait.patch" sfxge: Do no allow EFSYS_MEM_ALLOC sleep It solves locking problem when EFSYS_MEM_ALLOC is called in the context holding a mutex (not allowed to sleep). E.g. on interface bring up or multicast addresses addition. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. diff --git a/head/sys/dev/sfxge/common/efsys.h b/head/sys/dev/sfxge/common/efsys.h --- a/head/sys/dev/sfxge/common/efsys.h +++ b/head/sys/dev/sfxge/common/efsys.h @@ -701,7 +701,11 @@ #define EFSYS_KMEM_ALLOC(_esip, _size, _p) \ do { \ (_esip) = (_esip); \ - (_p) = malloc((_size), M_SFXGE, M_WAITOK|M_ZERO); \ + /* \ + * The macro is used in non-sleepable contexts, for \ + * example, holding a mutex. \ + */ \ + (_p) = malloc((_size), M_SFXGE, M_NOWAIT|M_ZERO); \ _NOTE(CONSTANTCONDITION) \ } while (B_FALSE) --------------030702020709090005030108-- From owner-freebsd-net@FreeBSD.ORG Tue May 20 17:33:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10CD5E26 for ; Tue, 20 May 2014 17:33:10 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 88E652D60 for ; Tue, 20 May 2014 17:33:09 +0000 (UTC) Received: from 95.108.170.210-red.dhcp.yndx.net ([95.108.170.210] helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wmk0A-0002Y7-FK; Tue, 20 May 2014 17:22:42 +0400 Message-ID: <537B9170.6040303@FreeBSD.org> Date: Tue, 20 May 2014 21:31:28 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: bycn82 Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <537A0054.5000707@FreeBSD.org> <537A0356.7050104@gmail.com> In-Reply-To: <537A0356.7050104@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Dennis Yusupoff , Marcelo Gondim , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 17:33:10 -0000 On 19.05.2014 17:12, bycn82 wrote: > On 5/19/14 21:00, Alexander V. Chernikov wrote: >> On 19.05.2014 11:51, Bill Yuan wrote: >>> Hi Alex, >> Hello Bill! >>> >>> You guys are chatting here! I agree with you, the table is the place >>> should >>> be enhanced, and I am working in this way as described below >>> >>> 1. Support more types. >>> ip : cidr >>> ipv4 : same as ip >>> ipv6 : ip addr v6 >>> mac : mac address >>> iface : interface name >>> interface : same as iface >>> port : it is Alex's idea, I dont know how it works. >> Well, actually that's not mine. ipfw implement the following since >> long ago: >> + v = ((ipfw_insn_u32 *)cmd)->d[1]; >> + switch (v) { >> + case 0: >> + case 1: >> + /* IPv4 src/dst */ >> + break; >> + case 2: >> + case 3: >> + /* src/dst port */ >> + break; >> + case 4: >> + /* uid/gid */ >> + case 5: >> + /* jid */ >> + case 6: >> + /* dscp */ >> + break; >> + } >> >> I hope you're not using radix to implement mac addresses lookup? >> >> Anyway, it looks like we're doing similar things. >> Can you take a look on '[CFT]: ipfw named tables / different >> tabletypes' topic and >> see how much it conflicts with your changes? >>> >>> 2. Setup the table type >>> ipfw table type >>> it will setup the type of the table, and flush the table >>> >>> 3. Get table type >>> ipfw table type show >>> >>> 4. Add item into the table >>> ipfw table add >>> >>> a. get the type of table >>> b. if the type is not defined yet, that also means the table is new or >>> empty, >>> then guess the type based on the >>> c. format the and insert into the table. >>> >>> In this way so call "back compatible" >>> >>> 5. how to use table >>> >>> case 1 >>> ipfw add [line] allow icmp from "table(1)" to "table(2)" >>> in the ipfw userland command, it should check the table1 and table 2 >>> should >>> be ipv4 or ipv6 type >>> >>> case 2 >>> ipfw add allow icmp from any to any MAC "table(3)" "table(4)" >>> in this case, the table(3) and table(4) should be a table of MAC >>> addresses. >>> >>> case 3 >>> ipfw add allow icmp from any to any via table(5) >>> in this case, the table 5 should be table of interface names. >>> >>> >>> currently I am working on the mac type. :) >>> >>> >>> >>> >>> On Sun, May 18, 2014 at 12:47 PM, Jason Hellenthal >>> wrote: >>> >>>> >>>>> On May 18, 2014, at 0:12, Julian Elischer wrote: >>>>>> 2) Table type/name can be specified explicitly via one of the >>>>>> following >>>> commands: >>>>>> * ipfw table 1 create [type ] [name >>>> "table_name"] >>>>> type "ports" would be nice but tricky to do right. >>>> That . . . would be a great addition and have me switching from pf >>>> to ipfw. >>>> >>>> Pullllease do! :-) >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> > > It is good to know that have company who is working in the same > direction. > and it is really feeling good to have user who is expecting this > feature before implemented. :) Yup. Named tables (and arbitrary tables) should have been done long time ago.. > > You bring up the code first , I can try to add on a patch for the > "mac" type or others , As a newbie here, I am not confident about how > to implement is the best way. Well, stock radix is slow and consumes a lot of memory per record (more than 3 cache lines). So it is probably better to implement either array of configurable item size or/and hash table. From owner-freebsd-net@FreeBSD.ORG Tue May 20 17:34:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14344F4E for ; Tue, 20 May 2014 17:34:28 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C684B2D70 for ; Tue, 20 May 2014 17:34:27 +0000 (UTC) Received: from 95.108.170.210-red.dhcp.yndx.net ([95.108.170.210] helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wmk1T-0002Yk-B5 for freebsd-net@freebsd.org; Tue, 20 May 2014 17:24:03 +0400 Message-ID: <537B91C2.7000609@FreeBSD.org> Date: Tue, 20 May 2014 21:32:50 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> <537A00AC.6050305@FreeBSD.org> <537A0560.2070902@gmail.com> <537A0967.1000808@smartspb.net> In-Reply-To: <537A0967.1000808@smartspb.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 17:34:28 -0000 On 19.05.2014 17:38, Dennis Yusupoff wrote: > It's not enough, actually. > Imagine what you have a table with different networks. If you'll try to > find out is an IP belongs to some of that networks from the table, you > should to write relatively serious "wrapper" with network range > calculations in it. Or can you show differ (easier) way? > So it's REALLY usefull to implement that functions "out-of-the-box". Doing "test" function is quite easy. I'll probably do this after finishing new tables code merge. > I'm risking to be annoying, but there is a good (from customers point of > view) example of tables manipulation in Linux: ipset project > (http://ipset.netfilter.org/ipset.man.html) > > 19.05.2014 17:21, bycn82 ÐŋÐļŅˆÐĩŅ‚: >> It will be nice to have this feature, >> but since the `ipfw table list` is existing, >> so I think this can be implemented outside the ipfw. >> (personal opinion only ) >> > From owner-freebsd-net@FreeBSD.ORG Wed May 21 02:04:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E778F43; Wed, 21 May 2014 02:04:31 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A37C2B2E; Wed, 21 May 2014 02:04:31 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id hz1so886401pad.16 for ; Tue, 20 May 2014 19:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=qAmaYmuyqWi7ZBQW9Zp7AbsbT/wwIpzSGo0IOPxxqPM=; b=zYXClIQSb7xJoSxwCEkLVtuPPlVN+B4kSNQIARidEKFq33nHFMAEoZKvcP8mxTEKuc 6Eyzzm3UylOcJfYdIgHFdtWwNk3MVeq+ml/hCt+Wc9i0+Xh7VxbhHEEgjFzDYn88HNtJ L0hsbiXczjfx0d4tCbz+OVMLzymYAsGT/1UM/l6pMVxRRh+AlFURS5JDLJPPwDiKMAwp BymQiI637fbQ7ZYD6zmlgYYvhq9bgd8HflbDGt0Jx9jOB6R0sIa/bk0vn0F1SacsmcnA aMFVY6RWeEsIDKL37k2cRsJZ4RdxmBv1cZNN+Sy5wzJJonL7XMPTRwgwqAH5XvyGWGmd 1JHQ== X-Received: by 10.68.110.226 with SMTP id id2mr55094524pbb.40.1400637871019; Tue, 20 May 2014 19:04:31 -0700 (PDT) Received: from [10.85.169.80] ([14.100.132.158]) by mx.google.com with ESMTPSA id sv10sm5292818pbc.74.2014.05.20.19.04.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 20 May 2014 19:04:29 -0700 (PDT) References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53783333.3010205@freebsd.org> <5379C6B6.4030105@smartspb.net> <537A00AC.6050305@FreeBSD.org> <537A0560.2070902@gmail.com> <537A0967.1000808@smartspb.net> <537B91C2.7000609@FreeBSD.org> Mime-Version: 1.0 (1.0) In-Reply-To: <537B91C2.7000609@FreeBSD.org> Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: quoted-printable Message-Id: <6BBE52AB-E405-4A2D-BFF3-6838130C8D6A@gmail.com> X-Mailer: iPhone Mail (11D201) From: bycn82 Subject: Re: [Was]: Problem with ipfw table add 0.0.0.0/8 Date: Wed, 21 May 2014 10:04:22 +0800 To: "Alexander V. Chernikov" Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 02:04:31 -0000 It will e nice to have this utility function > On 21 May, 2014, at 1:32 am, "Alexander V. Chernikov" wrote: >=20 >> On 19.05.2014 17:38, Dennis Yusupoff wrote: >> It's not enough, actually. >> Imagine what you have a table with different networks. If you'll try to >> find out is an IP belongs to some of that networks from the table, you >> should to write relatively serious "wrapper" with network range >> calculations in it. Or can you show differ (easier) way? >> So it's REALLY usefull to implement that functions "out-of-the-box". > Doing "test" function is quite easy. I'll probably do this after finishing= new tables code merge. >> I'm risking to be annoying, but there is a good (from customers point of >> view) example of tables manipulation in Linux: ipset project >> (http://ipset.netfilter.org/ipset.man.html) >>=20 >> 19.05.2014 17:21, bycn82 =A7=E1=A7=DA=A7=EA=A7=D6=A7=E4: >>> It will be nice to have this feature, >>> but since the `ipfw table list` is existing, >>> so I think this can be implemented outside the ipfw. >>> (personal opinion only ) >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed May 21 06:52:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9369FB72; Wed, 21 May 2014 06:52:29 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36A8921E0; Wed, 21 May 2014 06:52:29 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so2547897qgf.31 for ; Tue, 20 May 2014 23:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=RDG+x4DCmBYG/qjveJA5meUavBAeO68jrQV9pVfwBYg=; b=APF4heMJRPHurqhsZBMtKiRbIdTMWLp3RE73X+0AZuvAKNHXvzae547eTn8mJVf/gg IU1IHBDJfnELq9jnko2eL+XdUwhCTecnnf/LPfum0brDFBPOLRzegD9/xAM+ig19mXLX 8jQhudkOLWRBbXwA8gwItQ5XvG2Tpqw7gAHaYb3u7EdzOg1u4yfx+2pz6RZ8Wj2lOgEU sr1KIcHbuYu9SJv+CVoDgIWDIKTNyBATTCzjNTjyXlkF5OCCX2s55/+Gh3KtZ/baft+b ylSzO4pyZDweSOKO4htHuYG62kaodA5Z1eAh5taCjnEZvMR2MaNIwICKV81STuuGZFHh Z5mA== MIME-Version: 1.0 X-Received: by 10.224.129.66 with SMTP id n2mr65395338qas.55.1400655148371; Tue, 20 May 2014 23:52:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Tue, 20 May 2014 23:52:28 -0700 (PDT) Date: Tue, 20 May 2014 23:52:28 -0700 X-Google-Sender-Auth: c_9tztid7o7-85D9wxCKnQecXU8 Message-ID: Subject: [rfc] add non-contiguous CPU ID support to in_rss.c From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" , Robert Watson Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 06:52:29 -0000 Hi Robert, This patch uses CPU_FIRST() and CPU_NEXT() to iterate over the CPU IDs. Think this is alright? -a Index: sys/netinet/in_rss.c =================================================================== --- sys/netinet/in_rss.c (revision 266429) +++ sys/netinet/in_rss.c (working copy) @@ -176,6 +176,7 @@ rss_init(__unused void *arg) { u_int i; + u_int cpuid; /* * Validate tunables, coerce to sensible values. @@ -245,11 +246,12 @@ /* * Set up initial CPU assignments: round-robin by default. - * - * XXXRW: Need a mapping to non-contiguous IDs here. */ - for (i = 0; i < rss_buckets; i++) - rss_table[i].rte_cpu = i % rss_ncpus; + cpuid = CPU_FIRST(); + for (i = 0; i < rss_buckets; i++) { + rss_table[i].rte_cpu = cpuid; + cpuid = CPU_NEXT(cpuid); + } /* * Randomize rrs_key. From owner-freebsd-net@FreeBSD.ORG Wed May 21 11:05:48 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C804E57; Wed, 21 May 2014 11:05:48 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id BBE842BDA; Wed, 21 May 2014 11:05:46 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id D052A7300A; Wed, 21 May 2014 13:10:02 +0200 (CEST) Date: Wed, 21 May 2014 13:10:02 +0200 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140521111002.GB62462@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5379FE3C.6060501@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 11:05:48 -0000 On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: > Hello list! > > This patch adds ability to name tables / reference them by name. > Additionally, it simplifies adding new table types. Hi Alex, at a high level, i think your changes here are in the right direction. However i think part of the design, and also of the patch below, is not sound/correct so i would like to wait to commit at least until some of the problems below are resolved. 1. The patch as is has several issues (variable declarations in the middle of a block, assignment in conditionals, incorrect comments in cut&pasted code, missing checks on strings) that should be corrected. 2. you are introducing several distinct features that would be appropriate to commit separately. In the specific, the name-number translation is something that should go in first and independently, and possibly could be used for other features as well (pipes, nat entries, jump targets) so it might be good to have a closer look at its implementation (more later). 3. there is no explanation, here or in the code, on how the names are managed. I could try to figure out from the code but it would be the wrong way to understand things so let me ask, and please explain what you have in mind. Let me address first the name <-> table-id thing. Introducing a symbolic name for tables is a great and useful feature. However the implementation has some tricky issues: - do you want to retain the old numeric identifiers or not ? I think it is a bad idea to have two namespaces for the same thing, so if we are switching to alphanumeric strings for tables we should as well get rid of the numbers (i.e. consider them as strings as well). I am strongly in favour of not using names as aliases for numbers. It would require no changes for clients issuing ipfw commands from a script, and would not require users to to manually handle the name-id translations. - The rename command worries me a lot: > ipfw table name XXX > Names (or renames) table. Not the name has to be unique. because it is neither clear nor intuitive whether you want to 1. just rename a table (possibly breaking or creating references for rules in the firewall), or 2. modify the name-id translation preserving existing pointers. Consider the sequence ipfw table 13 name bar ipfw add 100 count dst-ip table bar ipfw table 13 name foo ipfw table 14 name bar ipfw add 200 count src-port 22 dst-ip table bar Approach #1 would detach rule 100 from table 13 and then connect to 14 (in a non-atomic way), whereas approach #2 would make rule 100 and 200 point to different tables (which is confusing). Now part of this problem goes away if we get rid of number altogether. You may legitimately want to swap tables in an atomic way (we have something similar for rulesets): ipfw set swap number number So here is what i would propose: - [gradually] introduce new commands that accept strings for every place where a number was previously accepted. Internally in the firewall, the old 16-bit number is interpreted as a string. - within the kernel create a small set of functions (invoked on insert, list, delete) that do proper checks on the string and translate it to a pointer (or short integer, i.e. an index in an array) to the proper object. Done properly, we can reuse this code also for pipes, sets, nat groups and whatnot. When a rule references an object that does not exist, you create an empty object that a subsequent table/pipe/XXX 'create' would initialize. On 'destroy', no need to touch the ruleset, just clear the object. - for renames, try to follow the approach used for sets i.e. ipfw table rename old-name new-name changes the name, preserves the object. Does not touch the ruleset, only the translation table ipfw table swap first-table second-table atomically swap the content of the two table objects (once again not touching the rulesets) cheers luigi I had a similar issue when implementing you can point to an empty one) and then to the new one, whereas #2 would repoint rule 100 to table 'foo' ( because i do not know whether names are stored In the details, however, I would really prefer like that you > Change list: > Kernel: > 1) Add new IP_FW_TABLE_XGETCFG / IP_FW_TABLE_XSETCFG opcodes to permit > table reconfiguration > 2) Tables data is now opaque to main ipfw code: use 1 pointer in first > ip_fw_chain cache line for lookups and another one for config state. > 3) Do not assume radix is the one and only lookup mechasim for doing > lookups (more changes following) > 4) Table data layout is changed to the following: > +struct table_info { > + void *state; /* IPv4 tables */ > + void *xstate;/* extended tables */ > + table_lookup_t *lookup;/* lookup function */ > + struct table_config *cfg; /* Additional data, can be NULL */ > +}; > Array of size "table_max * sizeof(struct table_info)" is allocated on > startup (very much like in current code in term of memory). > 5) State holds any additional info table may need for configuration > purposes and is allocated on demand. > > 6) By default, all tables are CIDR (IPv4+IPv6) and does not hold *cfg state. > 7) State is allocated when: > * table is referenced in some rules > * type is non-default > * table is named > 8) Tables can be named and referenced by their names, but it is still > needed to explicitly select table number. > 8) Table references are now explicitly tracked by kernel checking if > opcode lookup type and table type are the same > > 9) Do not assume tbl is uint16_t > 10) Change locking model: use both IPFW and IPFW_UH locks, as main ipfw > code does > 11) While here, fix possible panic in case of adding new table entry > while reallocating tables_max > > Some more on table types: > +struct table_config { > + uint8_t tabletype; /* lookup table types */ > + uint8_t ftype; /* format table type */ > + uint8_t atype; /* algorithm type */ > + uint8_t spare0; > + uint32_t refcnt; /* Number of references */ > + uint32_t count; /* Number of records */ > + struct table_info *ti; > + TAILQ_ENTRY(table_config) next; /* namehash */ > + char tablename[64]; /* table name */ > +}; > > "tabletype" is basically type of the key we're looking for (e.g. > IPv4/IPv6 address, interface name, port/uid, etc..). > "ftype" is pure userland field helping to format keys in appropriate way > (like shown DSCP values in hex or binary). > "atype" permits to use different algorithm for the same lookup key ( > currently not implemented, but planned). > Good example can be CIDR table consisting only of host routes. > > User: > Nothing changes for people using tables for IPv4/IPv6 address matching. > New cmds: > ipfw table type > Changes table type to different one. Permitted IFF: > * table is not referenced in ruleset > * table is empy > > ipfw table name XXX > Names (or renames) table. Not the name has to be unique. > > ipfw table flush > (Not a new command, actually). > Flushes all table records leaving configuration intact. > > ipfw table destroy > Flushes table state AND configuration. > Tables becomes unnamed IPFW_TABLE_CIDR one. > > > Next changes: > * Further rework add/del table entry to permit adding non-radix tables > more easily > * Change "iface" table type implementation to uint32_t iflist[65536] to > permit O(1) interface matching > * Add general u32 lookup method for dealing with ports/uids/jails/dscp > and other such consumers > > > > I'm planning to commit this one (actually, a bit improved version) in > the beginning of next week if no objections received. > > > > Index: sbin/ipfw/ipfw2.c > =================================================================== > --- sbin/ipfw/ipfw2.c (revision 266310) > +++ sbin/ipfw/ipfw2.c (working copy) > @@ -204,6 +204,12 @@ static struct _s_x limit_masks[] = { > {NULL, 0} > }; > > +static struct _s_x f_tabletypes[] = { > + {"cidr", IPFW_TABLE_CIDR}, > + {"iface", IPFW_TABLE_INTERFACE}, > + {NULL, 0} > +}; > + > /* > * we use IPPROTO_ETHERTYPE as a fake protocol id to call the print routines > * This is only used in this code. > @@ -4124,6 +4130,9 @@ ipfw_flush(int force) > > static void table_list(uint16_t num, int need_header); > static void table_fill_xentry(char *arg, ipfw_table_xentry *xent); > +static void table_get_info(uint16_t num, char *name, int need_header); > +static uint32_t table_get_num(char *name); > +static int table_check_name(char *name); > > /* > * Retrieve maximum number of tables supported by ipfw(4) module. > @@ -4158,6 +4167,7 @@ ipfw_get_tables_max() > * ipfw table N delete addr[/masklen] > * ipfw table {N | all} flush > * ipfw table {N | all} list > + * ipfw table {M | all} info > */ > void > ipfw_table_handler(int ac, char *av[]) > @@ -4167,6 +4177,7 @@ ipfw_table_handler(int ac, char *av[]) > int is_all; > uint32_t a; > uint32_t tables_max; > + int l, type; > > tables_max = ipfw_get_tables_max(); > > @@ -4181,13 +4192,18 @@ ipfw_table_handler(int ac, char *av[]) > xent.tbl = 0; > is_all = 1; > ac--; av++; > - } else > - errx(EX_USAGE, "table number or 'all' keyword required"); > + } else { > + xent.tbl = table_get_num(*av); > + is_all = 0; > + ac--; av++; > + } > + > if (xent.tbl >= tables_max) > errx(EX_USAGE, "The table number exceeds the maximum allowed " > "value (%d)", tables_max - 1); > NEED1("table needs command"); > if (is_all && _substrcmp(*av, "list") != 0 > + && _substrcmp(*av, "info") != 0 > && _substrcmp(*av, "flush") != 0) > errx(EX_USAGE, "table number required"); > > @@ -4243,6 +4259,59 @@ ipfw_table_handler(int ac, char *av[]) > do { > table_list(xent.tbl, is_all); > } while (++xent.tbl < a); > + } else if (_substrcmp(*av, "info") == 0) { > + a = is_all ? tables_max : (uint32_t)(xent.tbl + 1); > + do { > + table_get_info(xent.tbl, NULL, is_all); > + } while (++xent.tbl < a); > + } else if (_substrcmp(*av, "type") == 0) { > + ac--; av++; > + if (!ac) > + errx(EX_USAGE, "table type required"); > + > + if ((type = match_token(f_tabletypes, *av)) == -1) > + errx(EX_DATAERR, "Unknown table type"); > + > + ipfw_xtable_cfg cfg; > + > + memset(&cfg, 0, sizeof(cfg)); > + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; > + cfg.tbl = xent.tbl; > + cfg.mask = IPFW_XCFG_TYPE; > + cfg.type = type; > + l = sizeof(cfg); > + > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { > + /* > + * XXX: Provide human-readable error here > + * by using IP_FW_TABLE_XGETCFG > + */ > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); > + } > + } else if (_substrcmp(*av, "name") == 0) { > + ac--; av++; > + if (!ac) > + errx(EX_USAGE, "table name required"); > + > + if (table_check_name(*av) != 0) > + errx(EX_USAGE, "bad table name"); > + > + ipfw_xtable_cfg cfg; > + > + memset(&cfg, 0, sizeof(cfg)); > + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; > + cfg.tbl = xent.tbl; > + cfg.mask = IPFW_XCFG_NAME; > + strlcpy(cfg.tablename, *av, sizeof(cfg.tablename)); > + l = sizeof(cfg); > + > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { > + /* > + * XXX: Provide human-readable error here > + * by using IP_FW_TABLE_XGETCFG > + */ > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); > + } > } else > errx(EX_USAGE, "invalid table command %s", *av); > } > @@ -4423,3 +4492,107 @@ table_list(uint16_t num, int need_header) > > free(tbl); > } > + > +static uint32_t > +table_get_num(char *name) > +{ > + ipfw_xtable_cfg cfg; > + socklen_t l; > + > + memset(&cfg, 0, sizeof(cfg)); > + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; > + l = sizeof(cfg); > + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); > + cfg.mask |= IPFW_XCFG_NAMEID; > + > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) > + err(EX_OSERR, "table name '%s' not found", name); > + > + return (cfg.tbl); > +} > + > +static void > +table_get_info(uint16_t num, char *name, int is_all) > +{ > + ipfw_xtable_cfg cfg; > + socklen_t l; > + char *t; > + > + memset(&cfg, 0, sizeof(cfg)); > + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; > + cfg.mask = IPFW_XCFG_NAME | IPFW_XCFG_TYPE | IPFW_XCFG_REFS | > + IPFW_XCFG_CNT; > + l = sizeof(cfg); > + > + if (name == NULL) { > + cfg.tbl = num; > + cfg.mask |= IPFW_XCFG_NUMID; > + } else { > + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); > + cfg.mask |= IPFW_XCFG_NAMEID; > + } > + > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETCFG)"); > + > + if (is_all != 0) { > + /* > + * Skip unreferenced tables with default type > + */ > + if (cfg.type == IPFW_TABLE_CIDR && cfg.refcnt == 0) > + return; > + printf("---table(%d)---\n", num); > + } > + if (cfg.mask & IPFW_XCFG_NAME) > + printf("Name: %s\n", cfg.tablename); > + switch (cfg.type) { > + case IPFW_TABLE_CIDR: > + t = "cidr"; > + break; > + case IPFW_TABLE_INTERFACE: > + t = "cidr"; > + break; > +/* > + case IPFW_TABLE_U32: > + t = "u32"; > + break; > + case IPFW_TABLE_U16: > + t = "u16"; > + break; > +*/ > + default: > + t = "unknown"; > + break; > + }; > + > + printf("Type: %s\tFormatting: %s\n", t, "default"); > + printf("References: %u\tEntries: %u\n", cfg.refcnt, cfg.count); > +} > + > +static int > +table_check_name(char *tablename) > +{ > + int c, i, l; > + > + /* > + * Check if tablename is null-terminated and contains > + * valid symbols only. Valid mask is: > + * [a-zA-Z\-\.][a-zA-Z0-9\-_\.]{0,62} > + */ > + l = strlen(tablename); > + if (l == 0 || l >= 64) > + return (EINVAL); > + /* Restrict first symbol to non-digit */ > + if (isdigit(tablename[0])) > + return (EINVAL); > + for (i = 0; i < l; i++) { > + c = tablename[i]; > + if (isalpha(c) || isdigit(c) || c == '_' || > + c == '-' || c == '.') > + continue; > + return (EINVAL); > + } > + > + return (0); > +} > + > Index: sys/netinet/ip_fw.h > =================================================================== > --- sys/netinet/ip_fw.h (revision 266310) > +++ sys/netinet/ip_fw.h (working copy) > @@ -74,6 +74,8 @@ typedef struct _ip_fw3_opheader { > #define IP_FW_TABLE_XDEL 87 /* delete entry */ > #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ > #define IP_FW_TABLE_XLIST 89 /* list table contents */ > +#define IP_FW_TABLE_XGETCFG 90 /* configure table */ > +#define IP_FW_TABLE_XSETCFG 91 /* configure table */ > > /* > * The kernel representation of ipfw rules is made of a list of > @@ -632,7 +634,7 @@ typedef struct _ipfw_table { > } ipfw_table; > > typedef struct _ipfw_xtable { > - ip_fw3_opheader opheader; /* eXtended tables are controlled via IP_FW3 */ > + ip_fw3_opheader opheader; /* IP_FW3 opcode */ > uint32_t size; /* size of entries in bytes */ > uint32_t cnt; /* # of entries */ > uint16_t tbl; /* table number */ > @@ -640,4 +642,31 @@ typedef struct _ipfw_xtable { > ipfw_table_xentry xent[0]; /* entries */ > } ipfw_xtable; > > +#define IPFW_XCFG_NAME 0x01 /* get/set name */ > +#define IPFW_XCFG_TYPE 0x02 /* get/set type */ > +#define IPFW_XCFG_FTYPE 0x04 /* get/set format type */ > +#define IPFW_XCFG_REFS 0x08 /* get/set number of references */ > +#define IPFW_XCFG_CNT 0x10 /* ge number of records */ > +#define IPFW_XCFG_NUMID 0x20 /* table identified by num */ > +#define IPFW_XCFG_NAMEID 0x40 /* table identified by name */ > +typedef struct _ipfw_xtable_cfg { > + ip_fw3_opheader opheader; /* IP_FW3 opcode */ > + uint32_t size; /* size of structure in bytes */ > + uint32_t mask; /* data mask to set/retrieve */ > + uint32_t tlvmask; /* data mask to set/retrieve */ > + uint32_t tbl; /* table id */ > + uint16_t type; /* table type */ > + uint16_t ftype; /* table format type */ > + uint32_t count; /* number of records */ > + uint32_t refcnt; /* number of references */ > + char tablename[64]; /* table name */ > +} ipfw_xtable_cfg; > + > +typedef struct _ipfw_xtable_tlv { > + uint16_t type; > + uint16_t length; > +} ipfw_xtable_tlv; > + > + > + > #endif /* _IPFW2_H */ > Index: sys/netpfil/ipfw/ip_fw2.c > =================================================================== > --- sys/netpfil/ipfw/ip_fw2.c (revision 266306) > +++ sys/netpfil/ipfw/ip_fw2.c (working copy) > @@ -352,15 +352,16 @@ tcpopts_match(struct tcphdr *tcp, ipfw_insn *cmd) > } > > static int > -iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, uint32_t *tablearg) > +iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, > + uint32_t *tablearg) > { > if (ifp == NULL) /* no iface with this packet, match fails */ > return 0; > /* Check by name or by IP address */ > if (cmd->name[0] != '\0') { /* match by name */ > if (cmd->name[0] == '\1') /* use tablearg to match */ > - return ipfw_lookup_table_extended(chain, cmd->p.glob, > - ifp->if_xname, tablearg, IPFW_TABLE_INTERFACE); > + return (ipfw_lookup_table(chain, cmd->p.glob, IFNAMSIZ, > + ifp, tablearg)); > /* Check name */ > if (cmd->p.glob) { > if (fnmatch(cmd->name, ifp->if_xname, 0) == 0) > @@ -1492,7 +1493,7 @@ do { \ > break; > } > match = ipfw_lookup_table(chain, > - cmd->arg1, key, &v); > + cmd->arg1, sizeof(uint32_t), &key, &v); > if (!match) > break; > if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) > @@ -1504,9 +1505,9 @@ do { \ > uint32_t v = 0; > void *pkey = (cmd->opcode == O_IP_DST_LOOKUP) ? > &args->f_id.dst_ip6: &args->f_id.src_ip6; > - match = ipfw_lookup_table_extended(chain, > - cmd->arg1, pkey, &v, > - IPFW_TABLE_CIDR); > + match = ipfw_lookup_table(chain, > + cmd->arg1, sizeof(uint32_t), > + pkey, &v); > if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) > match = ((ipfw_insn_u32 *)cmd)->d[0] == v; > if (match) > Index: sys/netpfil/ipfw/ip_fw_private.h > =================================================================== > --- sys/netpfil/ipfw/ip_fw_private.h (revision 266306) > +++ sys/netpfil/ipfw/ip_fw_private.h (working copy) > @@ -217,9 +217,7 @@ struct ip_fw_chain { > uint32_t id; /* ruleset id */ > int n_rules; /* number of static rules */ > LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ > - struct radix_node_head **tables; /* IPv4 tables */ > - struct radix_node_head **xtables; /* extended tables */ > - uint8_t *tabletype; /* Array of table types */ > + void *tablestate; /* Array of table data */ > #if defined( __linux__ ) || defined( _WIN32 ) > spinlock_t rwmtx; > #else > @@ -229,6 +227,7 @@ struct ip_fw_chain { > uint32_t gencnt; /* NAT generation count */ > struct ip_fw *reap; /* list of rules to reap */ > struct ip_fw *default_rule; > + void *tablecfg; /* tables module configuration */ > #if defined( __linux__ ) || defined( _WIN32 ) > spinlock_t uh_lock; > #else > @@ -303,9 +302,8 @@ int ipfw_chk(struct ip_fw_args *args); > void ipfw_reap_rules(struct ip_fw *head); > > /* In ip_fw_table.c */ > -struct radix_node; > -int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, > - uint32_t *val); > +int ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t keylen, > + void *key, uint32_t *val); > int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, > uint32_t *val, int type); > int ipfw_init_tables(struct ip_fw_chain *ch); > @@ -316,11 +314,18 @@ int ipfw_add_table_entry(struct ip_fw_chain *ch, u > int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, > uint8_t plen, uint8_t mlen, uint8_t type); > int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); > -int ipfw_dump_table_entry(struct radix_node *rn, void *arg); > int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); > -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); > +int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, > + uint32_t *cnt); > int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); > int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); > +int ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl); > +int ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype); > +int ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl); > +int ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > + size_t *sz); > +int ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > + size_t *sz); > > /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ > > Index: sys/netpfil/ipfw/ip_fw_sockopt.c > =================================================================== > --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266306) > +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy) > @@ -146,6 +146,119 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * > } > > /* > + * In @del==0 mode: > + * Checks is opcode is referencing table of appropriate type. > + * Adds reference count for found table if true. > + * In del==1 mode > + * Decrements refcount for given table. > + * > + * Returns 0 on success and appropriate error code otherwise. > + */ > +static int > +bind_tables(struct ip_fw_chain *chain, struct ip_fw *rule, int del) > +{ > + int cmdlen, error, ftype, l, skip, type, v; > + ipfw_insn *cmd; > + ipfw_insn_if *cmdif; > + uint32_t tbl; > + > + l = rule->cmd_len; > + cmd = rule->cmd; > + cmdlen = 0; > + tbl = 0; > + type = 0; > + ftype = 0; > + > + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { > + cmdlen = F_LEN(cmd); > + skip = 0; > + > + switch (cmd->opcode) { > + case O_IP_SRC_LOOKUP: > + case O_IP_DST_LOOKUP: > + /* Basic IPv4/IPv6 or u32 lookups */ > + tbl = cmd->arg1; > + /* Assume CIDR by default */ > + type = IPFW_TABLE_CIDR; > + ftype = 0; > + > + if (cmdlen > F_INSN_SIZE(ipfw_insn_u32)) { > + /* generic lookup. The key must be > + * in 32bit big-endian format. > + */ > + v = ((ipfw_insn_u32 *)cmd)->d[1]; > + switch (v) { > + case 0: > + case 1: > + /* IPv4 src/dst */ > + break; > + case 2: > + case 3: > + /* src/dst port */ > + //type = IPFW_TABLE_U16; > + break; > + case 4: > + /* uid/gid */ > + //type = IPFW_TABLE_U32; > + case 5: > + //type = IPFW_TABLE_U32; > + /* jid */ > + case 6: > + //type = IPFW_TABLE_U16; > + /* dscp */ > + break; > + } > + } > + break; > + case O_XMIT: > + case O_RECV: > + case O_VIA: > + /* Interface table, possibly */ > + cmdif = (ipfw_insn_if *)cmd; > + if (cmdif->name[0] != '\1') { > + skip = 1; > + break; > + } > + > + type = IPFW_TABLE_INTERFACE; > + ftype = 0; > + tbl = cmdif->p.glob; > + break; > + default: > + skip = 1; > + } > + > + if (skip != 0) > + continue; > + > + /* ref/unref given table */ > + if (del != 0) > + error = ipfw_unref_table(chain, tbl); > + else > + error = ipfw_ref_table(chain, tbl, type, ftype); > + > + if (error != 0) > + return (error); > + } > + > + return (0); > +} > + > +/* > + * Removes table bindings for every rule in rule chain @head. > + */ > +static void > +unbind_tables(struct ip_fw_chain *chain, struct ip_fw *head) > +{ > + struct ip_fw *rule; > + > + while ((rule = head) != NULL) { > + head = head->x_next; > + bind_tables(chain, rule, 1); > + } > +} > + > +/* > * Add a new rule to the list. Copy the rule into a malloc'ed area, then > * possibly create a rule number and add the rule to the list. > * Update the rule_number in the input struct so the caller knows it as well. > @@ -157,6 +270,7 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip > { > struct ip_fw *rule; > int i, l, insert_before; > + int error; > struct ip_fw **map; /* the new array of pointers */ > > if (chain->map == NULL || input_rule->rulenum > IPFW_DEFAULT_RULE - 1) > @@ -168,9 +282,16 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip > map = get_map(chain, 1, 0 /* not locked */); > if (map == NULL) { > free(rule, M_IPFW); > - return ENOSPC; > + return (ENOSPC); > } > > + /* Reference tables, if any */ > + if ((error = bind_tables(chain, input_rule, 0)) != 0) { > + IPFW_UH_WUNLOCK(chain); > + free(rule, M_IPFW); > + return (error); > + } > + > bcopy(input_rule, rule, l); > /* clear fields not settable from userland */ > rule->x_next = NULL; > @@ -421,6 +542,7 @@ del_entry(struct ip_fw_chain *chain, uint32_t arg) > > rule = chain->reap; > chain->reap = NULL; > + unbind_tables(chain, rule); > IPFW_UH_WUNLOCK(chain); > ipfw_reap_rules(rule); > if (map) > @@ -934,6 +1056,7 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf > > > #define IP_FW3_OPLENGTH(x) ((x)->sopt_valsize - sizeof(ip_fw3_opheader)) > + > /** > * {set|get}sockopt parser. > */ > @@ -1113,7 +1236,7 @@ ipfw_ctl(struct sockopt *sopt) > sopt->sopt_name == IP_FW_RESETLOG); > break; > > - /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ > + /*--- TABLE locking is described in ip_fw_table.c ---*/ > case IP_FW_TABLE_ADD: > { > ipfw_table_entry ent; > @@ -1237,7 +1360,7 @@ ipfw_ctl(struct sockopt *sopt) > tbl = (uint32_t *)(op3 + 1); > > IPFW_RLOCK(chain); > - error = ipfw_count_xtable(chain, *tbl, tbl); > + error = ipfw_count_xtable(chain, *tbl, tbl, NULL); > IPFW_RUNLOCK(chain); > if (error) > break; > @@ -1281,6 +1404,39 @@ ipfw_ctl(struct sockopt *sopt) > } > break; > > + case IP_FW_TABLE_XGETCFG: /* IP_FW3 */ > + case IP_FW_TABLE_XSETCFG: /* IP_FW3 */ > + { > + ipfw_xtable_cfg *xcfg; > + > + if (sopt->sopt_valsize < sizeof(xcfg)) { > + error = EINVAL; > + break; > + } > + if ((size = sopt->sopt_valsize) > RULE_MAXSIZE) { > + error = E2BIG; > + break; > + } > + > + xcfg = malloc(size, M_TEMP, M_WAITOK); > + error = sooptcopyin(sopt, xcfg, size, sizeof(*xcfg)); > + if (error != 0) { > + free(xcfg, M_TEMP); > + break; > + } > + > + if (opt == IP_FW_TABLE_XGETCFG) { > + error = ipfw_getconfig_table(chain, xcfg, &size); > + if (error == 0) > + error = sooptcopyout(sopt, xcfg, size); > + } else > + error = ipfw_setconfig_table(chain, xcfg, &size); > + > + free(xcfg, M_TEMP); > + } > + break; > + > + > /*--- NAT operations are protected by the IPFW_LOCK ---*/ > case IP_FW_NAT_CFG: > if (IPFW_NAT_LOADED) > Index: sys/netpfil/ipfw/ip_fw_table.c > =================================================================== > --- sys/netpfil/ipfw/ip_fw_table.c (revision 266310) > +++ sys/netpfil/ipfw/ip_fw_table.c (working copy) > @@ -35,8 +35,13 @@ __FBSDID("$FreeBSD$"); > * As a degenerate case we can interpret keys as 32-bit integers > * (with a /32 mask). > * > - * The table is protected by the IPFW lock even for manipulation coming > - * from userland, because operations are typically fast. > + * Locking resembles ipfw rules model: > + * changes in: table entries, table count and table_info are protected by holding > + * both UH and main write locks. > + * changes in table_config struture are protected by UH lock. > + * > + * Userland readers should use UH lock if possible. > + * > */ > > #include "opt_ipfw.h" > @@ -48,12 +53,14 @@ __FBSDID("$FreeBSD$"); > > #include > #include > +#include > #include > #include > #include > #include > #include > #include > +#include > #include /* ip_fw.h requires IFNAMSIZ */ > #include > #include > @@ -101,6 +108,65 @@ struct table_xentry { > }; > > /* > + * filled for non-CIDR or named tables > + * > + * Table has the following `type` concepts: > + * > + * `tabletype` represents lookup key type (cidr, ifp, uid, etc..) > + * `ftype` is pure userland field helping to properly format table data > + * `atype` represents exact lookup algorithm for given tabletype. > + * For example, we can use more efficient search schemes if we plan > + * to use some specific table for storing host-routes only. > + * > + */ > +struct table_config { > + uint8_t tabletype; /* lookup table types */ > + uint8_t ftype; /* format table type */ > + uint8_t atype; /* algorith type */ > + uint8_t spare0; > + uint32_t refcnt; /* Number of references */ > + uint32_t count; /* Number of records */ > + struct table_info *ti; > + TAILQ_ENTRY(table_config) next; /* namehash */ > + char tablename[64]; /* table name */ > +}; > + > +typedef int (table_lookup_t)(void *state, void *xstate, void *key, > + uint32_t keylen, uint32_t *val); > + > +struct table_info { > + void *state; /* IPv4 tables */ > + void *xstate;/* extended tables */ > + table_lookup_t *lookup;/* lookup function */ > + struct table_config *cfg; /* Additional data, can be NULL */ > +}; > + > +static int lookup_cidr(void *, void *, void *, uint32_t, uint32_t *); > +static int lookup_iface(void *, void *, void *, uint32_t, uint32_t *); > + > +static table_lookup_t (*tablehandlers[]) = { > + NULL, /* type 0 unused */ > + lookup_cidr, /* IPFW_TABLE_CIDR */ > + lookup_iface, /* IPFW_TABLE_INTERFACE */ > +}; > + > +#define TABLETYPE(t) ((t)->cfg != NULL ? (t)->cfg->tabletype:IPFW_TABLE_CIDR) > +#define TABLEFTYPE(ti) ((ti)->cfg != NULL ? (ti)->cfg->ftype : 0) > +#define TABLEREFS(ti) ((ti)->cfg != NULL ? (ti)->cfg->refcnt : 0) > + > +#define TABLENAME_HASH_SIZE 32 > +struct tables_config { > + TAILQ_HEAD(, table_config) thash[TABLENAME_HASH_SIZE]; > +}; > + > +#define TABLENAME_HASH(n) (fnv_32_str(n,FNV1_32_INIT)%TABLENAME_HASH_SIZE) > +static struct table_info *find_table(struct ip_fw_chain *ch, char *tablename); > + > +static void flush_table(struct ip_fw_chain *ch, struct table_info *ti); > +static int change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, > + struct table_info *ti2, table_lookup_t lookup); > + > +/* > * The radix code expects addr and mask to be array of bytes, > * with the first byte being the length of the array. rn_inithead > * is called with the offset in bits of the lookup key within the > @@ -145,13 +211,19 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > struct radix_node *rn; > in_addr_t addr; > int offset; > - void *ent_ptr; > + uintptr_t state_off; > + void *ent_ptr, *tablestate; > struct sockaddr *addr_ptr, *mask_ptr; > + struct table_info *ti; > + struct table_config *cfg; > char c; > > if (tbl >= V_fw_tables_max) > return (EINVAL); > > + tablestate = ch->tablestate; > + ti = &((struct table_info *)tablestate)[tbl]; > + > switch (type) { > case IPFW_TABLE_CIDR: > if (plen == sizeof(in_addr_t)) { > @@ -170,7 +242,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > addr = *((in_addr_t *)paddr); > ent->addr.sin_addr.s_addr = addr & ent->mask.sin_addr.s_addr; > /* Set pointers */ > - rnh_ptr = &ch->tables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->state; > ent_ptr = ent; > addr_ptr = (struct sockaddr *)&ent->addr; > mask_ptr = (struct sockaddr *)&ent->mask; > @@ -191,7 +263,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > memcpy(&xent->a.addr6.sin6_addr, paddr, sizeof(struct in6_addr)); > APPLY_MASK(&xent->a.addr6.sin6_addr, &xent->m.mask6.sin6_addr); > /* Set pointers */ > - rnh_ptr = &ch->xtables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > ent_ptr = xent; > addr_ptr = (struct sockaddr *)&xent->a.addr6; > mask_ptr = (struct sockaddr *)&xent->m.mask6; > @@ -227,7 +299,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > mask_ptr = (struct sockaddr *)&xent->m.ifmask; > #endif > /* Set pointers */ > - rnh_ptr = &ch->xtables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > ent_ptr = xent; > addr_ptr = (struct sockaddr *)&xent->a.iface; > mask_ptr = NULL; > @@ -237,48 +309,67 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > return (EINVAL); > } > > + IPFW_UH_WLOCK(ch); > IPFW_WLOCK(ch); > > + /* > + * Check if tablestate was reallocated. > + */ > + if (ch->tablestate != tablestate) { > + state_off = (uintptr_t)ti - (uintptr_t)tablestate; > + ti = (struct table_info *) > + (state_off + (uintptr_t)ch->tablestate); > + > + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; > + rnh_ptr = (struct radix_node_head **) > + (state_off + (uintptr_t)ch->tablestate); > + } > + > /* Check if tabletype is valid */ > - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { > + if (TABLETYPE(ti) != type) { > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > free(ent_ptr, M_IPFW_TBL); > - return (EINVAL); > + return (EFTYPE); > } > > /* Check if radix tree exists */ > if ((rnh = *rnh_ptr) == NULL) { > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > /* Create radix for a new table */ > if (!rn_inithead((void **)&rnh, offset)) { > free(ent_ptr, M_IPFW_TBL); > return (ENOMEM); > } > > + IPFW_UH_WLOCK(ch); > IPFW_WLOCK(ch); > if (*rnh_ptr != NULL) { > /* Tree is already attached by other thread */ > rn_detachhead((void **)&rnh); > rnh = *rnh_ptr; > /* Check table type another time */ > - if (ch->tabletype[tbl] != type) { > + if (TABLETYPE(ti) != type) { > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > free(ent_ptr, M_IPFW_TBL); > - return (EINVAL); > + return (EFTYPE); > } > } else { > + /* new trie */ > *rnh_ptr = rnh; > - /* > - * Set table type. It can be set already > - * (if we have IPv6-only table) but setting > - * it another time does not hurt > - */ > - ch->tabletype[tbl] = type; > } > } > > rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); > + > + /* Maintain number of records */ > + if (rn != NULL && (cfg = ti->cfg) != NULL) > + cfg->count++; > + > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > > if (rn == NULL) { > free(ent_ptr, M_IPFW_TBL); > @@ -296,11 +387,18 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > in_addr_t addr; > struct sockaddr_in sa, mask; > struct sockaddr *sa_ptr, *mask_ptr; > + struct table_info *ti; > + struct table_config *cfg; > + void *tablestate; > + uintptr_t state_off; > char c; > > if (tbl >= V_fw_tables_max) > return (EINVAL); > > + tablestate = ch->tablestate; > + ti = &((struct table_info *)tablestate)[tbl]; > + > switch (type) { > case IPFW_TABLE_CIDR: > if (plen == sizeof(in_addr_t)) { > @@ -310,7 +408,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); > addr = *((in_addr_t *)paddr); > sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; > - rnh_ptr = &ch->tables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->state; > sa_ptr = (struct sockaddr *)&sa; > mask_ptr = (struct sockaddr *)&mask; > #ifdef INET6 > @@ -327,7 +425,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > ipv6_writemask(&mask6.sin6_addr, mlen); > memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); > APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); > - rnh_ptr = &ch->xtables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > sa_ptr = (struct sockaddr *)&sa6; > mask_ptr = (struct sockaddr *)&mask6; > #endif > @@ -362,7 +460,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > mask_ptr = NULL; > memcpy(ifname.ifname, paddr, mlen); > /* Set pointers */ > - rnh_ptr = &ch->xtables[tbl]; > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > sa_ptr = (struct sockaddr *)&ifname; > > break; > @@ -371,19 +469,42 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > return (EINVAL); > } > > + IPFW_UH_WLOCK(ch); > IPFW_WLOCK(ch); > + > + /* > + * Check if tablestate was reallocated. > + */ > + if (ch->tablestate != tablestate) { > + state_off = (uintptr_t)ti - (uintptr_t)tablestate; > + ti = (struct table_info *) > + (state_off + (uintptr_t)ch->tablestate); > + > + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; > + rnh_ptr = (struct radix_node_head **) > + (state_off + (uintptr_t)ch->tablestate); > + } > + > if ((rnh = *rnh_ptr) == NULL) { > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > return (ESRCH); > } > > - if (ch->tabletype[tbl] != type) { > + if (TABLETYPE(ti) != type) { > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > return (EINVAL); > } > > ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, rnh); > + > + /* Maintain number of records */ > + if (ent != NULL && (cfg = ti->cfg) != NULL) > + cfg->count--; > + > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > > if (ent == NULL) > return (ESRCH); > @@ -392,7 +513,381 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > return (0); > } > > +/* > + * binds newly-created config to given table > + */ > static int > +bind_config(struct ip_fw_chain *ch, uint32_t tbl, struct table_info *ti, > + struct table_config *cfg) > +{ > + int error; > + uint32_t sz, cnt; > + > + /* Set defaults */ > + cfg->tabletype = IPFW_TABLE_CIDR; > + > + /* count current number of entries */ > + if ((error = ipfw_count_xtable(ch, tbl, &sz, &cnt)) != 0) > + return (error); > + cfg->count = cnt; > + > + ti->cfg = cfg; > + cfg->ti = ti; > + > + return (0); > +} > + > +static struct table_info * > +find_table(struct ip_fw_chain *ch, char *tablename) > +{ > + struct tables_config *tc; > + struct table_config *cfg; > + int hash; > + > + hash = TABLENAME_HASH(tablename); > + tc = (struct tables_config *)ch->tablecfg; > + > + TAILQ_FOREACH(cfg, &tc->thash[hash], next) { > + if (strcmp(cfg->tablename, tablename) == 0) > + return (cfg->ti); > + } > + > + return (NULL); > +} > + > +/* > + * Fills in supplied buffer with table configuration info. > + */ > +int > +ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) > +{ > + struct table_info *ti; > + struct table_config *cfg; > + uint32_t tbl, tsz, cnt; > + > + tbl = xcfg->tbl; > + > + IPFW_UH_RLOCK(ch); > + > + if (xcfg->mask & IPFW_XCFG_NUMID) { > + /* Table is identified by number */ > + tbl = xcfg->tbl; > + > + if (tbl >= V_fw_tables_max) { > + IPFW_UH_RUNLOCK(ch); > + return (EINVAL); > + } > + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { > + /* Let's try to find table by name */ > + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { > + IPFW_UH_RUNLOCK(ch); > + return (EINVAL); > + } > + > + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { > + IPFW_UH_RUNLOCK(ch); > + return (ESRCH); > + } > + > + /* Guess table number based on offset */ > + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); > + } else { > + IPFW_UH_RUNLOCK(ch); > + return (ESRCH); > + } > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + cfg = ti->cfg; > + > + /* Save table id anywat */ > + xcfg->tbl = tbl; > + > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0 ) { > + if (cfg == NULL || cfg->tablename == NULL) { > + xcfg->tablename[0] = '\0'; > + xcfg->mask &= ~IPFW_XCFG_NAME; > + } else > + strlcpy(xcfg->tablename, cfg->tablename, > + sizeof(cfg->tablename)); > + } > + > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) > + xcfg->type = TABLETYPE(ti); > + > + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) > + xcfg->ftype = TABLEFTYPE(ti); > + > + if ((xcfg->mask & IPFW_XCFG_REFS) != 0) > + xcfg->refcnt = TABLEREFS(ti); > + > + if ((xcfg->mask & IPFW_XCFG_CNT) != 0) { > + /* > + * Use items count from cfg, if it exists. > + * Otherwise, calculate manually. > + */ > + if (cfg != NULL) > + xcfg->count = cfg->count; > + else { > + IPFW_RLOCK(ch); > + ipfw_count_xtable(ch, tbl, &tsz, &cnt); > + IPFW_RUNLOCK(ch); > + xcfg->count = cnt; > + } > + } > + > + IPFW_UH_RUNLOCK(ch); > + > + return (0); > +} > + > +/* > + * Fills in supplied buffer with table configuration info. > + */ > +int > +ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) > +{ > + struct table_info *ti, ti_storage, *ti2; > + struct table_config *cfg; > + uint32_t tbl; > + int l; > + int error; > + > + tbl = xcfg->tbl; > + > + /* Do some preliminary checking */ > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > + if (xcfg->type > IPFW_TABLE_MAXTYPE || xcfg->type == 0) > + return (EINVAL); > + } > + > + /* > + * Check if tablename is null-terminated. > + * More fine-grained checks should be done by userland. > + */ > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > + l = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > + if (l == sizeof(xcfg->tablename) || l == 0) > + return (EINVAL); > + } > + > + /* Checl if we need to allocate config structure */ > + IPFW_UH_RLOCK(ch); > + > + if (xcfg->mask & IPFW_XCFG_NUMID) { > + /* Table is identified by number */ > + tbl = xcfg->tbl; > + > + if (tbl >= V_fw_tables_max) { > + IPFW_UH_RUNLOCK(ch); > + return (EINVAL); > + } > + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { > + /* Let's try to find table by name */ > + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { > + IPFW_UH_RUNLOCK(ch); > + return (EINVAL); > + } > + > + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { > + IPFW_UH_RUNLOCK(ch); > + return (ESRCH); > + } > + > + /* Guess table number based on offset */ > + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); > + } > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + cfg = ti->cfg; > + IPFW_UH_RUNLOCK(ch); > + > + /* > + * Allocate new config structure if needed. > + * cfg represents pointer to new structure or NULL > + */ > + if (cfg == NULL) > + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_WAITOK | M_ZERO); > + else > + cfg = NULL; > + > + IPFW_UH_WLOCK(ch); > + /* > + * We need to check another time since there is probability that > + * V_fw_tables_max was changed or ti->cfg was destroyed > + */ > + > + if (tbl >= V_fw_tables_max) { > + IPFW_UH_WUNLOCK(ch); > + return (EINVAL); > + } > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + if (ti->cfg == NULL) { > + if (cfg == NULL) { > + /* destroy_table() had happened before IPFW_UH_WLOCK */ > + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_NOWAIT|M_ZERO); > + if (cfg == NULL) { > + IPFW_UH_WUNLOCK(ch); > + return (ENOMEM); > + } > + } > + > + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { > + IPFW_UH_WUNLOCK(ch); > + free(cfg, M_IPFW_TBL); > + return (error); > + } > + } else if (cfg != NULL) { > + /* > + * ti->cfg has been allocated by other thread. > + * Free our allocation. > + */ > + free(cfg, M_IPFW_TBL); > + } > + > + cfg = ti->cfg; > + > + /* > + * Pretend to be atomic: check everything before doing actual job > + */ > + error = 0; > + > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > + /* > + * We can set type if > + * 1) no one hold any references to our table > + * 2) we have no records in given table > + */ > + > + if ((cfg->tabletype != xcfg->type) && (cfg->refcnt > 0) && > + (cfg->count > 0)) > + error = EBUSY; > + } > + > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > + /* Check if new table name exists */ > + if (find_table(ch, xcfg->tablename) != NULL) > + error = EEXIST; > + } > + > + if (error != 0) { > + IPFW_UH_WUNLOCK(ch); > + return (error); > + } > + > + /* Everything checked, let's get the job done */ > + ti2 = NULL; > + > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > + /* we need to change table_info here */ > + IPFW_WLOCK(ch); > + cfg->tabletype = xcfg->type; > + if (change_table_handler(ch, ti, &ti_storage, > + tablehandlers[cfg->tabletype]) != 0) > + ti2 = &ti_storage; > + IPFW_WUNLOCK(ch); > + } > + > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > + int hash; > + struct tables_config *tc; > + tc = (struct tables_config *)ch->tablecfg; > + > + if (strlen(cfg->tablename) != 0) { > + /* unlink from old one */ > + hash = TABLENAME_HASH(cfg->tablename); > + TAILQ_REMOVE(&tc->thash[hash], cfg, next); > + } > + strlcpy(cfg->tablename, xcfg->tablename, > + sizeof(cfg->tablename)); > + /* link new one */ > + hash = TABLENAME_HASH(cfg->tablename); > + TAILQ_INSERT_HEAD(&tc->thash[hash], cfg, next); > + } > + > + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) > + cfg->ftype = xcfg->ftype; > + > + IPFW_UH_WUNLOCK(ch); > + > + /* Free old table state if set */ > + if (ti2 != NULL) > + flush_table(ch, ti2); > + > + return (0); > +} > + > + > +/* > + * Checks if given table's type is the same as @type. > + * If true, increment @tbl refcount > + */ > +int > +ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype) > +{ > + struct table_info *ti; > + struct table_config *cfg; > + int error; > + > + IPFW_UH_WLOCK_ASSERT(ch); > + > + if (tbl >= V_fw_tables_max) > + return (EINVAL); > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + > + if (TABLETYPE(ti) != type) > + return (EFTYPE); > + > + /* Ignore ftype for now */ > + > + if (ti->cfg == NULL) { > + /* > + * Let's create confdata. > + * XXX: we can fail here > + */ > + cfg = malloc(sizeof(struct table_config), M_IPFW_TBL, > + M_NOWAIT | M_ZERO); > + > + if (cfg == NULL) > + return (ENOMEM); > + > + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { > + free(cfg, M_IPFW_TBL); > + return (error); > + } > + } > + > + ti->cfg->refcnt++; > + > + return (0); > +} > + > +int > +ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl) > +{ > + struct table_info *ti; > + > + IPFW_UH_WLOCK_ASSERT(ch); > + > + if (tbl >= V_fw_tables_max) > + return (EINVAL); > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + > + KASSERT(ti->cfg != 0, ("ipfw: no config for table %d", tbl)); > + > + KASSERT(ti->cfg->refcnt > 0, ("ipfw: refcnt for table %d is %d", > + tbl, ti->cfg->refcnt)); > + > + ti->cfg->refcnt--; > + > + return (0); > +} > + > +static int > flush_table_entry(struct radix_node *rn, void *arg) > { > struct radix_node_head * const rnh = arg; > @@ -405,40 +900,138 @@ flush_table_entry(struct radix_node *rn, void *arg > return (0); > } > > +/* > + * Flushes data from table state pointers. > + * Frees pointers itself. > + */ > +static void > +flush_table(struct ip_fw_chain *ch, struct table_info *ti) > +{ > + struct radix_node_head *rnh; > + > + if ((rnh = ti->state) != NULL) { > + rnh->rnh_walktree(rnh, flush_table_entry, rnh); > + rn_detachhead((void **)&rnh); > + } > + > + if ((rnh = ti->xstate) != NULL) { > + rnh->rnh_walktree(rnh, flush_table_entry, rnh); > + rn_detachhead((void **)&rnh); > + } > +} > + > +/* > + * change table handler saving previous state in @ti2. > + * Returns 1 if hable had some state, 0 otherwise. > + */ > +static int > +change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, > + struct table_info *ti2, table_lookup_t lookup) > +{ > + > + if (ti->state == NULL && ti->state == NULL) { > + /* No state. Set handler and return. */ > + ti->lookup = lookup; > + return (0); > + } > + > + *ti2 = *ti; > + > + ti->state = NULL; > + ti->xstate = NULL; > + ti->lookup = lookup; > + > + return (1); > +} > + > +/* > + * flushes all data in given table leaving table type/naming > + * intact. > + */ > int > ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) > { > - struct radix_node_head *rnh, *xrnh; > + struct table_info *ti, tti; > + struct table_config *cfg; > > - if (tbl >= V_fw_tables_max) > + IPFW_UH_WLOCK(ch); > + IPFW_WLOCK(ch); > + > + if (tbl >= V_fw_tables_max) { > + IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > return (EINVAL); > + } > > - /* > - * We free both (IPv4 and extended) radix trees and > - * clear table type here to permit table to be reused > - * for different type without module reload > - */ > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + tti = *ti; > + /* Remove state tables from main structure */ > + ti->state = NULL; > + ti->xstate = NULL; > + if ((cfg = ti->cfg) != NULL) > + cfg->count = 0; > + IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > > + flush_table(ch, &tti); > + > + return (0); > +} > + > +static void > +destroy_table(struct ip_fw_chain *ch, struct table_info *ti) > +{ > + struct table_config *cfg; > + > + flush_table(ch, ti); > + > + if ((cfg = ti->cfg) != NULL) { > + /* Free configuration state */ > + free(cfg, M_IPFW_TBL); > + ti->cfg = NULL; > + } > + > + /* Set lookup pointer back to CIDR */ > + ti->lookup = lookup_cidr; > +} > + > +/* > + * Destroys given table iff no rules are referencing it. > + * Flushes all data in tablestate, destroys any > + * special configuration/naming associated with the table > + * and sets its type (ti->cfg == NULL) back to CIDR. > + */ > +int > +ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl) > +{ > + struct table_info *ti, tti; > + struct table_config *cfg; > + > + IPFW_UH_WLOCK(ch); > IPFW_WLOCK(ch); > - /* Set IPv4 table pointer to zero */ > - if ((rnh = ch->tables[tbl]) != NULL) > - ch->tables[tbl] = NULL; > - /* Set extended table pointer to zero */ > - if ((xrnh = ch->xtables[tbl]) != NULL) > - ch->xtables[tbl] = NULL; > - /* Zero table type */ > - ch->tabletype[tbl] = 0; > - IPFW_WUNLOCK(ch); > > - if (rnh != NULL) { > - rnh->rnh_walktree(rnh, flush_table_entry, rnh); > - rn_detachhead((void **)&rnh); > + if (tbl >= V_fw_tables_max) { > + IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > + return (EINVAL); > } > > - if (xrnh != NULL) { > - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); > - rn_detachhead((void **)&xrnh); > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + if (((cfg = ti->cfg) != NULL) && (cfg->refcnt != 0)) { > + /* Table is referenced by some rules */ > + IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > + return (EBUSY); > } > + tti = *ti; > + /* Remove state tables from main structure */ > + ti->state = NULL; > + ti->xstate = NULL; > + ti->cfg = NULL; > + IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > + > + destroy_table(ch, &tti); > > return (0); > } > @@ -446,152 +1039,179 @@ ipfw_flush_table(struct ip_fw_chain *ch, uint16_t > void > ipfw_destroy_tables(struct ip_fw_chain *ch) > { > - uint16_t tbl; > + uint32_t tbl; > + struct table_info *ti; > > /* Flush all tables */ > - for (tbl = 0; tbl < V_fw_tables_max; tbl++) > - ipfw_flush_table(ch, tbl); > + ti = (struct table_info *)ch->tablestate; > + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) > + destroy_table(ch, ti); > > /* Free pointers itself */ > - free(ch->tables, M_IPFW); > - free(ch->xtables, M_IPFW); > - free(ch->tabletype, M_IPFW); > + free(ch->tablestate, M_IPFW_TBL); > + free(ch->tablecfg, M_IPFW_TBL); > } > > int > ipfw_init_tables(struct ip_fw_chain *ch) > { > + struct table_info *ti; > + struct tables_config *tc; > + uint32_t tbl; > + > /* Allocate pointers */ > - ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); > - ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); > - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); > + ch->tablestate = malloc(V_fw_tables_max * sizeof(struct table_info), > + M_IPFW_TBL, M_WAITOK | M_ZERO); > + > + ch->tablecfg = malloc(sizeof(struct tables_config), M_IPFW_TBL, > + M_WAITOK | M_ZERO); > + > + /* Set initial handlers for all tables */ > + ti = (struct table_info *)ch->tablestate; > + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) > + ti->lookup = lookup_cidr; > + > + /* Set up tablenames hash */ > + tc = ch->tablecfg; > + for (tbl = 0; tbl < TABLENAME_HASH_SIZE; tbl++) > + TAILQ_INIT(&tc->thash[tbl]); > + > return (0); > } > > int > ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables) > { > - struct radix_node_head **tables, **xtables, *rnh; > - struct radix_node_head **tables_old, **xtables_old; > - uint8_t *tabletype, *tabletype_old; > unsigned int ntables_old, tbl; > + struct table_info *ti, *ti_old, *ti_new; > > /* Check new value for validity */ > if (ntables > IPFW_TABLES_MAX) > ntables = IPFW_TABLES_MAX; > > /* Allocate new pointers */ > - tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); > - xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); > - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); > + ti_new = malloc(ntables * sizeof(struct table_info), > + M_IPFW_TBL, M_WAITOK | M_ZERO); > > + IPFW_UH_WLOCK(ch); > IPFW_WLOCK(ch); > > tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; > > /* Copy old table pointers */ > - memcpy(tables, ch->tables, sizeof(void *) * tbl); > - memcpy(xtables, ch->xtables, sizeof(void *) * tbl); > - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); > + memcpy(ti_new, ch->tablestate, sizeof(struct table_info) * tbl); > > /* Change pointers and number of tables */ > - tables_old = ch->tables; > - xtables_old = ch->xtables; > - tabletype_old = ch->tabletype; > - ch->tables = tables; > - ch->xtables = xtables; > - ch->tabletype = tabletype; > + ti_old = (struct table_info *)ch->tablestate; > + ch->tablestate = ti_new; > > ntables_old = V_fw_tables_max; > V_fw_tables_max = ntables; > > IPFW_WUNLOCK(ch); > + IPFW_UH_WUNLOCK(ch); > > /* Check if we need to destroy radix trees */ > if (ntables < ntables_old) { > - for (tbl = ntables; tbl < ntables_old; tbl++) { > - if ((rnh = tables_old[tbl]) != NULL) { > - rnh->rnh_walktree(rnh, flush_table_entry, rnh); > - rn_detachhead((void **)&rnh); > - } > + ti = &ti_old[ntables]; > + for (tbl = ntables; tbl < ntables_old; tbl++, ti++) > + destroy_table(ch, ti); > + } > > - if ((rnh = xtables_old[tbl]) != NULL) { > - rnh->rnh_walktree(rnh, flush_table_entry, rnh); > - rn_detachhead((void **)&rnh); > - } > - } > + /* Check if we need to setup new ones */ > + if (ntables > ntables_old) { > + ti = &ti_new[ntables_old]; > + for (tbl = ntables_old; tbl < ntables; tbl++, ti++) > + ti->lookup = lookup_cidr; > } > > /* Free old pointers */ > - free(tables_old, M_IPFW); > - free(xtables_old, M_IPFW); > - free(tabletype_old, M_IPFW); > + free(ti_old, M_IPFW_TBL); > > return (0); > } > > -int > -ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, > - uint32_t *val) > +static int > +lookup_cidr(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) > { > struct radix_node_head *rnh; > struct table_entry *ent; > - struct sockaddr_in sa; > + struct table_xentry *xent; > > - if (tbl >= V_fw_tables_max) > + > + if (keylen == sizeof(struct in_addr)) { > + /* IPv4 lookup */ > + struct sockaddr_in sa; > + > + if ((rnh = (struct radix_node_head *)st) == NULL) > + return (0); > + > + KEY_LEN(sa) = KEY_LEN_INET; > + sa.sin_addr.s_addr = ((struct in_addr *)key)->s_addr; > + ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); > + if (ent != NULL) { > + *val = ent->value; > + return (1); > + } > + > return (0); > - if ((rnh = ch->tables[tbl]) == NULL) > + } > + > + /* IPv6 lookup */ > + struct sockaddr_in6 sa6; > + > + if ((rnh = (struct radix_node_head *)xst) == NULL) > return (0); > - KEY_LEN(sa) = KEY_LEN_INET; > - sa.sin_addr.s_addr = addr; > - ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); > - if (ent != NULL) { > - *val = ent->value; > + > + KEY_LEN(sa6) = KEY_LEN_INET6; > + memcpy(&sa6.sin6_addr, key, sizeof(struct in6_addr)); > + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); > + > + if (xent != NULL) { > + *val = xent->value; > return (1); > } > + > return (0); > } > > -int > -ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, > - uint32_t *val, int type) > +static int > +lookup_iface(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) > { > struct radix_node_head *rnh; > + struct xaddr_iface iface; > struct table_xentry *xent; > - struct sockaddr_in6 sa6; > - struct xaddr_iface iface; > > - if (tbl >= V_fw_tables_max) > + if ((rnh = (struct radix_node_head *)xst) == NULL) > return (0); > - if ((rnh = ch->xtables[tbl]) == NULL) > - return (0); > > - switch (type) { > - case IPFW_TABLE_CIDR: > - KEY_LEN(sa6) = KEY_LEN_INET6; > - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); > - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); > - break; > - > - case IPFW_TABLE_INTERFACE: > - KEY_LEN(iface) = KEY_LEN_IFACE + > - strlcpy(iface.ifname, (char *)paddr, IF_NAMESIZE) + 1; > - /* Assume direct match */ > - /* FIXME: Add interface pattern matching */ > - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); > - break; > - > - default: > - return (0); > - } > - > + /* IPv6 lookup */ > + KEY_LEN(iface) = KEY_LEN_IFACE + > + strlcpy(iface.ifname, (char *)key, IF_NAMESIZE) + 1; > + /* Assume direct match */ > + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); > if (xent != NULL) { > *val = xent->value; > return (1); > } > + > return (0); > } > > +int > +ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, > + uint32_t keylen, void *key, uint32_t *val) > +{ > + struct table_info *ti; > + > + if (tbl >= V_fw_tables_max) > + return (0); > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + > + return (ti->lookup(ti->state, ti->xstate, key, keylen, val)); > +} > + > static int > count_table_entry(struct radix_node *rn, void *arg) > { > @@ -605,11 +1225,15 @@ int > ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) > { > struct radix_node_head *rnh; > + struct table_info *ti; > > if (tbl >= V_fw_tables_max) > return (EINVAL); > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + > *cnt = 0; > - if ((rnh = ch->tables[tbl]) == NULL) > + if ((rnh = ti->state) == NULL) > return (0); > rnh->rnh_walktree(rnh, count_table_entry, cnt); > return (0); > @@ -640,11 +1264,15 @@ int > ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) > { > struct radix_node_head *rnh; > + struct table_info *ti; > > if (tbl->tbl >= V_fw_tables_max) > return (EINVAL); > + > + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; > + > tbl->cnt = 0; > - if ((rnh = ch->tables[tbl->tbl]) == NULL) > + if ((rnh = ti->state) == NULL) > return (0); > rnh->rnh_walktree(rnh, dump_table_entry, tbl); > return (0); > @@ -660,20 +1288,32 @@ count_table_xentry(struct radix_node *rn, void *ar > } > > int > -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) > +ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, > + uint32_t *cnt) > { > struct radix_node_head *rnh; > + struct table_info *ti; > + uint32_t count; > > if (tbl >= V_fw_tables_max) > return (EINVAL); > - *cnt = 0; > - if ((rnh = ch->tables[tbl]) != NULL) > - rnh->rnh_walktree(rnh, count_table_xentry, cnt); > - if ((rnh = ch->xtables[tbl]) != NULL) > - rnh->rnh_walktree(rnh, count_table_xentry, cnt); > + > + ti = &((struct table_info *)ch->tablestate)[tbl]; > + > + *sz = 0; > + count = 0; > + if ((rnh = ti->state) != NULL) > + rnh->rnh_walktree(rnh, count_table_xentry, sz); > + if ((rnh = ti->xstate) != NULL) > + rnh->rnh_walktree(rnh, count_table_xentry, sz); > + count = *sz / sizeof(ipfw_table_xentry); > + > + if (cnt != NULL) > + *cnt = count; > + > /* Return zero if table is empty */ > - if (*cnt > 0) > - (*cnt) += sizeof(ipfw_xtable); > + if (*sz > 0) > + (*sz) += sizeof(ipfw_xtable); > return (0); > } > > @@ -750,14 +1390,18 @@ int > ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) > { > struct radix_node_head *rnh; > + struct table_info *ti; > > if (tbl->tbl >= V_fw_tables_max) > return (EINVAL); > + > + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; > + > tbl->cnt = 0; > - tbl->type = ch->tabletype[tbl->tbl]; > - if ((rnh = ch->tables[tbl->tbl]) != NULL) > + tbl->type = TABLETYPE(ti); > + if ((rnh = ti->state) != NULL) > rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); > - if ((rnh = ch->xtables[tbl->tbl]) != NULL) > + if ((rnh = ti->xstate) != NULL) > rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); > return (0); > } From owner-freebsd-net@FreeBSD.ORG Wed May 21 14:58:55 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1292AD2B; Wed, 21 May 2014 14:58:55 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1D6B2EE0; Wed, 21 May 2014 14:58:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=OtwL0EfNWC2OWUZrcNM25ZJVmGQ5rVx/eMsHN1xMrkc=; b=Nb4Ny3UzfG/jl6dY2SfRl2FGuHt0AFa/Shmy8oEmq2K67CcWvVdd05SwjSW0b86Ox7S81SLSwtRiU/+l13rdSfkK9YFoFlK2OhzeoEX0vb7dy2+2yMXXHebTzmOK9E7yhmnGFu8SXVa6Rx29nstQzN5UE9/8+xej/Qk4KzL216ZV6WqJdSZv/ZOSGE04Dmp/LKEQvbRxF3tAz8nk7+bc0zRLf3mr2kyopIfCUlH9tw8XE862BLlZSAAYp/IuRe/MYTXu3knK1z9yHk3GWyjWr90ybnSRRJtUoIsAK+tqf2G6d+lBK4lFRJYcfrgS3ZNuQYxzCQOdo/OLjo/E8BLIEdMsxymIc8YXW3Xum8ARI2nXwvSav9pchZVannvd62d1K13UTbubJ1tol/vfgsyrJ3pjPNvBgv2AZRehQsyt4BVoz19jAqs0KPz1GieNOeRUzHFxWBenzahT+QnMwOJ7JOLPcmTfou9cYalzBx6+koL89utBHVavnMJf0JXqBuB0PAJPgPZ+SApZdgwZH2n1RZ7wYiEfNSNn1nUtCaDodZRBLiQzUXvkhb6XyT2vp8b15AuaENSKMa/0m0WQRshyGuPSu1JjHHmJ3xeKhwzjqiROU+vVHD9bDBpj9lhP7vulpZweLSGJGkUoSIN7MjipoaSvLEInoCJkFzUNuyMMZy8=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1Wn7yk-000094-Cn; Wed, 21 May 2014 18:58:50 +0400 Date: Wed, 21 May 2014 18:58:47 +0400 From: Eygene Ryabinkin To: George Neville-Neil Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Message-ID: References: <20140508200404.GA50446@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="87MiR7gHvrw39A9h" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: Marcelo Araujo , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 14:58:55 -0000 --87MiR7gHvrw39A9h Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Mon, May 12, 2014 at 12:39:49AM +0400, Eygene Ryabinkin wrote: > Sun, May 11, 2014 at 04:30:32PM -0400, George Neville-Neil wrote: > > On May 8, 2014, at 16:04 , Gleb Smirnoff wrote: > > > On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: > > > E> - I'll do a patch for carp(4) that will allow it to use configura= ble > > > E> OUI from a sysctl knob (first 5 bytes of OUI); > > >=20 > > > Please no sysctl knobs. This should be configurable via ifconfig(8) > > > per vhid. > >=20 > > Agree, please do this via ifconfig. >=20 > http://codelabs.ru/fbsd/carp-ouibase.diff Updated the patch, URL remains the same: http://codelabs.ru/fbsd/carp-ouibase.diff Changes: - full MAC is settable via ether/lladdr/link keyword, no ouibase keyword now exists; - these keywords accept "carp" and "vrrp" keywords making them to set new and old bases with the last octet set to the VHID; - network.subr was updated not to mess with any keywords that go after 'vhid' and just pass it down to ifconfig as is. I did two days of testing and hadn't yet found any problems. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] Please, CC me, I am not subscribed to this list. --87MiR7gHvrw39A9h Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlN8vydfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Ps+awEAiLw9dotBH8WuSgWEguiMl98X GQRBSj+RPhIb2X9/pR8A/Rmbbni0z3kmzFyFf8R70+PzoFFVjlu+mmP6lcU2sqRu =lx8D -----END PGP SIGNATURE----- --87MiR7gHvrw39A9h-- From owner-freebsd-net@FreeBSD.ORG Wed May 21 15:17:24 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DEDE1FF6; Wed, 21 May 2014 15:17:24 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A3532066; Wed, 21 May 2014 15:17:22 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id ty20so137598lab.26 for ; Wed, 21 May 2014 08:17:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=C9HBq5SvNYm9/PtVj17WaMnRGV2cSBUDmBcYe+pfb5Q=; b=XKVEwDyk5l07inl9abfykmGTD0kde0TWRL1gSi4ZNzfHfHsXoZWrNrG+vqTFeUcRhU aVgExakqtiSDEWRZaVtb2RSarXPYFXfgMDo496iGt38cLK/v8DwE6TqG3i/XuX9Xxf5P PyvqC4ykYtelc9U5gcP4lXuq45+RbS9PD+3X6eIVE07xWDF4abTOm92chiNFF8gBUL1e VsXrxT5nXY9f3jJs7Dtu/4qHb6tLT+sp22Z+AQYuNa+9ozcfBrCTU33Y7/hh7eMInVCv 6ENIb7sCs/ftHQkZlFKr3fbvvYMOOdkmqM6Lxt9nFv6WupgWKhZvLWS/eT+XzW60SKXg p5jA== MIME-Version: 1.0 X-Received: by 10.112.135.106 with SMTP id pr10mr36211869lbb.24.1400685440739; Wed, 21 May 2014 08:17:20 -0700 (PDT) Received: by 10.112.5.10 with HTTP; Wed, 21 May 2014 08:17:20 -0700 (PDT) In-Reply-To: <20140521111002.GB62462@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> Date: Wed, 21 May 2014 23:17:20 +0800 Message-ID: Subject: Re: [CFT]: ipfw named tables / different tabletypes From: Bill Yuan To: Luigi Rizzo X-Mailman-Approved-At: Wed, 21 May 2014 15:30:31 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Luigi Rizzo , "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 15:17:25 -0000 1. "each table can have it's own name", I like this idea. I am also working on this kind of functions, But I am in different way. In my opinion. the "name" or "type" of the table, all this are utility function for the user. in the kernel space. I want to keep all the things the same as before. and add a translate function in the user land commands, so that means we need to find a space to keep the mapping of the name<->id or type<->id, 2. As in ipfw_check() or other function. we use "switch case" sentences to keep the code clear and clean. so I re-factor the table_handler, defined some actions for "list" "flush" "add", when I have time, I will read your patch first. On Wed, May 21, 2014 at 7:10 PM, Luigi Rizzo wrote: > On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: > > Hello list! > > > > This patch adds ability to name tables / reference them by name. > > Additionally, it simplifies adding new table types. > > Hi Alex, > at a high level, i think your changes here are in the right direction. > > However i think part of the design, and also of the patch below, > is not sound/correct so i would like to wait to commit at least > until some of the problems below are resolved. > > 1. The patch as is has several issues (variable declarations in the > middle of a block, assignment in conditionals, incorrect > comments in cut&pasted code, missing checks on strings) > that should be corrected. > > 2. you are introducing several distinct features that would be > appropriate to commit separately. > > In the specific, the name-number translation is something > that should go in first and independently, and possibly > could be used for other features as well (pipes, nat entries, > jump targets) so it might be good to have a closer look at > its implementation (more later). > > 3. there is no explanation, here or in the code, on how the > names are managed. I could try to figure out from the code > but it would be the wrong way to understand things so let me > ask, and please explain what you have in mind. > > Let me address first the name <-> table-id thing. > > Introducing a symbolic name for tables is a great and useful feature. > However the implementation has some tricky issues: > > - do you want to retain the old numeric identifiers or not ? > I think it is a bad idea to have two namespaces for the same thing, > so if we are switching to alphanumeric strings for tables we should > as well get rid of the numbers (i.e. consider them as strings as well). > > I am strongly in favour of not using names as aliases for numbers. > It would require no changes for clients issuing ipfw commands > from a script, and would not require users to to manually handle > the name-id translations. > > - The rename command worries me a lot: > > > ipfw table name XXX > > Names (or renames) table. Not the name has to be unique. > > because it is neither clear nor intuitive whether you want to > 1. just rename a table (possibly breaking or creating references > for rules in the firewall), or > 2. modify the name-id translation preserving existing pointers. > > Consider the sequence > ipfw table 13 name bar > ipfw add 100 count dst-ip table bar > ipfw table 13 name foo > ipfw table 14 name bar > ipfw add 200 count src-port 22 dst-ip table bar > > Approach #1 would detach rule 100 from table 13 and then connect to 14 > (in a non-atomic way), whereas approach #2 would make rule 100 and 200 > point to different tables (which is confusing). > > Now part of this problem goes away if we get rid of number altogether. > > You may legitimately want to swap tables in an atomic way (we have > something > similar for rulesets): > ipfw set swap number number > > > So here is what i would propose: > - [gradually] introduce new commands that accept strings for every place > where > a number was previously accepted. Internally in the firewall, the old > 16-bit number is interpreted as a string. > > - within the kernel create a small set of functions (invoked on insert, > list, delete) > that do proper checks on the string and translate it to a pointer (or > short integer, > i.e. an index in an array) to the proper object. Done properly, we can > reuse > this code also for pipes, sets, nat groups and whatnot. > > When a rule references an object that does not exist, you create an empty > object that a subsequent table/pipe/XXX 'create' would initialize. > On 'destroy', no need to touch the ruleset, just clear the object. > > - for renames, try to follow the approach used for sets i.e. > ipfw table rename old-name new-name > changes the name, preserves the object. > Does not touch the ruleset, only the translation table > ipfw table swap first-table second-table > atomically swap the content of the two table objects > (once again not touching the rulesets) > > cheers > luigi > > I had a similar issue when implementing > you can point to an empty one) and then to the new one, > whereas #2 would repoint rule 100 to table 'foo' ( > > because i do not know whether > names are stored > In the details, however, I would really prefer like that you > > > > Change list: > > Kernel: > > 1) Add new IP_FW_TABLE_XGETCFG / IP_FW_TABLE_XSETCFG opcodes to permit > > table reconfiguration > > 2) Tables data is now opaque to main ipfw code: use 1 pointer in first > > ip_fw_chain cache line for lookups and another one for config state. > > 3) Do not assume radix is the one and only lookup mechasim for doing > > lookups (more changes following) > > 4) Table data layout is changed to the following: > > +struct table_info { > > + void *state; /* IPv4 tables */ > > + void *xstate;/* extended tables */ > > + table_lookup_t *lookup;/* lookup function */ > > + struct table_config *cfg; /* Additional data, can be NULL > */ > > +}; > > Array of size "table_max * sizeof(struct table_info)" is allocated on > > startup (very much like in current code in term of memory). > > 5) State holds any additional info table may need for configuration > > purposes and is allocated on demand. > > > > 6) By default, all tables are CIDR (IPv4+IPv6) and does not hold *cfg > state. > > 7) State is allocated when: > > * table is referenced in some rules > > * type is non-default > > * table is named > > 8) Tables can be named and referenced by their names, but it is still > > needed to explicitly select table number. > > 8) Table references are now explicitly tracked by kernel checking if > > opcode lookup type and table type are the same > > > > 9) Do not assume tbl is uint16_t > > 10) Change locking model: use both IPFW and IPFW_UH locks, as main ipfw > > code does > > 11) While here, fix possible panic in case of adding new table entry > > while reallocating tables_max > > > > Some more on table types: > > +struct table_config { > > + uint8_t tabletype; /* lookup table types */ > > + uint8_t ftype; /* format table type */ > > + uint8_t atype; /* algorithm type */ > > + uint8_t spare0; > > + uint32_t refcnt; /* Number of references */ > > + uint32_t count; /* Number of records */ > > + struct table_info *ti; > > + TAILQ_ENTRY(table_config) next; /* namehash */ > > + char tablename[64]; /* table name */ > > +}; > > > > "tabletype" is basically type of the key we're looking for (e.g. > > IPv4/IPv6 address, interface name, port/uid, etc..). > > "ftype" is pure userland field helping to format keys in appropriate way > > (like shown DSCP values in hex or binary). > > "atype" permits to use different algorithm for the same lookup key ( > > currently not implemented, but planned). > > Good example can be CIDR table consisting only of host routes. > > > > User: > > Nothing changes for people using tables for IPv4/IPv6 address matching. > > New cmds: > > ipfw table type > > Changes table type to different one. Permitted IFF: > > * table is not referenced in ruleset > > * table is empy > > > > ipfw table name XXX > > Names (or renames) table. Not the name has to be unique. > > > > ipfw table flush > > (Not a new command, actually). > > Flushes all table records leaving configuration intact. > > > > ipfw table destroy > > Flushes table state AND configuration. > > Tables becomes unnamed IPFW_TABLE_CIDR one. > > > > > > Next changes: > > * Further rework add/del table entry to permit adding non-radix tables > > more easily > > * Change "iface" table type implementation to uint32_t iflist[65536] to > > permit O(1) interface matching > > * Add general u32 lookup method for dealing with ports/uids/jails/dscp > > and other such consumers > > > > > > > > I'm planning to commit this one (actually, a bit improved version) in > > the beginning of next week if no objections received. > > > > > > > > > Index: sbin/ipfw/ipfw2.c > > =================================================================== > > --- sbin/ipfw/ipfw2.c (revision 266310) > > +++ sbin/ipfw/ipfw2.c (working copy) > > @@ -204,6 +204,12 @@ static struct _s_x limit_masks[] = { > > {NULL, 0} > > }; > > > > +static struct _s_x f_tabletypes[] = { > > + {"cidr", IPFW_TABLE_CIDR}, > > + {"iface", IPFW_TABLE_INTERFACE}, > > + {NULL, 0} > > +}; > > + > > /* > > * we use IPPROTO_ETHERTYPE as a fake protocol id to call the print > routines > > * This is only used in this code. > > @@ -4124,6 +4130,9 @@ ipfw_flush(int force) > > > > static void table_list(uint16_t num, int need_header); > > static void table_fill_xentry(char *arg, ipfw_table_xentry *xent); > > +static void table_get_info(uint16_t num, char *name, int need_header); > > +static uint32_t table_get_num(char *name); > > +static int table_check_name(char *name); > > > > /* > > * Retrieve maximum number of tables supported by ipfw(4) module. > > @@ -4158,6 +4167,7 @@ ipfw_get_tables_max() > > * ipfw table N delete addr[/masklen] > > * ipfw table {N | all} flush > > * ipfw table {N | all} list > > + * ipfw table {M | all} info > > */ > > void > > ipfw_table_handler(int ac, char *av[]) > > @@ -4167,6 +4177,7 @@ ipfw_table_handler(int ac, char *av[]) > > int is_all; > > uint32_t a; > > uint32_t tables_max; > > + int l, type; > > > > tables_max = ipfw_get_tables_max(); > > > > @@ -4181,13 +4192,18 @@ ipfw_table_handler(int ac, char *av[]) > > xent.tbl = 0; > > is_all = 1; > > ac--; av++; > > - } else > > - errx(EX_USAGE, "table number or 'all' keyword required"); > > + } else { > > + xent.tbl = table_get_num(*av); > > + is_all = 0; > > + ac--; av++; > > + } > > + > > if (xent.tbl >= tables_max) > > errx(EX_USAGE, "The table number exceeds the maximum > allowed " > > "value (%d)", tables_max - 1); > > NEED1("table needs command"); > > if (is_all && _substrcmp(*av, "list") != 0 > > + && _substrcmp(*av, "info") != 0 > > && _substrcmp(*av, "flush") != 0) > > errx(EX_USAGE, "table number required"); > > > > @@ -4243,6 +4259,59 @@ ipfw_table_handler(int ac, char *av[]) > > do { > > table_list(xent.tbl, is_all); > > } while (++xent.tbl < a); > > + } else if (_substrcmp(*av, "info") == 0) { > > + a = is_all ? tables_max : (uint32_t)(xent.tbl + 1); > > + do { > > + table_get_info(xent.tbl, NULL, is_all); > > + } while (++xent.tbl < a); > > + } else if (_substrcmp(*av, "type") == 0) { > > + ac--; av++; > > + if (!ac) > > + errx(EX_USAGE, "table type required"); > > + > > + if ((type = match_token(f_tabletypes, *av)) == -1) > > + errx(EX_DATAERR, "Unknown table type"); > > + > > + ipfw_xtable_cfg cfg; > > + > > + memset(&cfg, 0, sizeof(cfg)); > > + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; > > + cfg.tbl = xent.tbl; > > + cfg.mask = IPFW_XCFG_TYPE; > > + cfg.type = type; > > + l = sizeof(cfg); > > + > > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { > > + /* > > + * XXX: Provide human-readable error here > > + * by using IP_FW_TABLE_XGETCFG > > + */ > > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); > > + } > > + } else if (_substrcmp(*av, "name") == 0) { > > + ac--; av++; > > + if (!ac) > > + errx(EX_USAGE, "table name required"); > > + > > + if (table_check_name(*av) != 0) > > + errx(EX_USAGE, "bad table name"); > > + > > + ipfw_xtable_cfg cfg; > > + > > + memset(&cfg, 0, sizeof(cfg)); > > + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; > > + cfg.tbl = xent.tbl; > > + cfg.mask = IPFW_XCFG_NAME; > > + strlcpy(cfg.tablename, *av, sizeof(cfg.tablename)); > > + l = sizeof(cfg); > > + > > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { > > + /* > > + * XXX: Provide human-readable error here > > + * by using IP_FW_TABLE_XGETCFG > > + */ > > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); > > + } > > } else > > errx(EX_USAGE, "invalid table command %s", *av); > > } > > @@ -4423,3 +4492,107 @@ table_list(uint16_t num, int need_header) > > > > free(tbl); > > } > > + > > +static uint32_t > > +table_get_num(char *name) > > +{ > > + ipfw_xtable_cfg cfg; > > + socklen_t l; > > + > > + memset(&cfg, 0, sizeof(cfg)); > > + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; > > + l = sizeof(cfg); > > + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); > > + cfg.mask |= IPFW_XCFG_NAMEID; > > + > > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) > > + err(EX_OSERR, "table name '%s' not found", name); > > + > > + return (cfg.tbl); > > +} > > + > > +static void > > +table_get_info(uint16_t num, char *name, int is_all) > > +{ > > + ipfw_xtable_cfg cfg; > > + socklen_t l; > > + char *t; > > + > > + memset(&cfg, 0, sizeof(cfg)); > > + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; > > + cfg.mask = IPFW_XCFG_NAME | IPFW_XCFG_TYPE | IPFW_XCFG_REFS | > > + IPFW_XCFG_CNT; > > + l = sizeof(cfg); > > + > > + if (name == NULL) { > > + cfg.tbl = num; > > + cfg.mask |= IPFW_XCFG_NUMID; > > + } else { > > + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); > > + cfg.mask |= IPFW_XCFG_NAMEID; > > + } > > + > > + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) > > + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETCFG)"); > > + > > + if (is_all != 0) { > > + /* > > + * Skip unreferenced tables with default type > > + */ > > + if (cfg.type == IPFW_TABLE_CIDR && cfg.refcnt == 0) > > + return; > > + printf("---table(%d)---\n", num); > > + } > > + if (cfg.mask & IPFW_XCFG_NAME) > > + printf("Name: %s\n", cfg.tablename); > > + switch (cfg.type) { > > + case IPFW_TABLE_CIDR: > > + t = "cidr"; > > + break; > > + case IPFW_TABLE_INTERFACE: > > + t = "cidr"; > > + break; > > +/* > > + case IPFW_TABLE_U32: > > + t = "u32"; > > + break; > > + case IPFW_TABLE_U16: > > + t = "u16"; > > + break; > > +*/ > > + default: > > + t = "unknown"; > > + break; > > + }; > > + > > + printf("Type: %s\tFormatting: %s\n", t, "default"); > > + printf("References: %u\tEntries: %u\n", cfg.refcnt, cfg.count); > > +} > > + > > +static int > > +table_check_name(char *tablename) > > +{ > > + int c, i, l; > > + > > + /* > > + * Check if tablename is null-terminated and contains > > + * valid symbols only. Valid mask is: > > + * [a-zA-Z\-\.][a-zA-Z0-9\-_\.]{0,62} > > + */ > > + l = strlen(tablename); > > + if (l == 0 || l >= 64) > > + return (EINVAL); > > + /* Restrict first symbol to non-digit */ > > + if (isdigit(tablename[0])) > > + return (EINVAL); > > + for (i = 0; i < l; i++) { > > + c = tablename[i]; > > + if (isalpha(c) || isdigit(c) || c == '_' || > > + c == '-' || c == '.') > > + continue; > > + return (EINVAL); > > + } > > + > > + return (0); > > +} > > + > > Index: sys/netinet/ip_fw.h > > =================================================================== > > --- sys/netinet/ip_fw.h (revision 266310) > > +++ sys/netinet/ip_fw.h (working copy) > > @@ -74,6 +74,8 @@ typedef struct _ip_fw3_opheader { > > #define IP_FW_TABLE_XDEL 87 /* delete entry */ > > #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ > > #define IP_FW_TABLE_XLIST 89 /* list table contents */ > > +#define IP_FW_TABLE_XGETCFG 90 /* configure table */ > > +#define IP_FW_TABLE_XSETCFG 91 /* configure table */ > > > > /* > > * The kernel representation of ipfw rules is made of a list of > > @@ -632,7 +634,7 @@ typedef struct _ipfw_table { > > } ipfw_table; > > > > typedef struct _ipfw_xtable { > > - ip_fw3_opheader opheader; /* eXtended tables are controlled > via IP_FW3 */ > > + ip_fw3_opheader opheader; /* IP_FW3 opcode */ > > uint32_t size; /* size of entries in bytes */ > > uint32_t cnt; /* # of entries */ > > uint16_t tbl; /* table number */ > > @@ -640,4 +642,31 @@ typedef struct _ipfw_xtable { > > ipfw_table_xentry xent[0]; /* entries */ > > } ipfw_xtable; > > > > +#define IPFW_XCFG_NAME 0x01 /* get/set name > */ > > +#define IPFW_XCFG_TYPE 0x02 /* get/set type > */ > > +#define IPFW_XCFG_FTYPE 0x04 /* get/set format type > */ > > +#define IPFW_XCFG_REFS 0x08 /* get/set number of > references */ > > +#define IPFW_XCFG_CNT 0x10 /* ge number of records > */ > > +#define IPFW_XCFG_NUMID 0x20 /* table identified by num > */ > > +#define IPFW_XCFG_NAMEID 0x40 /* table identified by > name */ > > +typedef struct _ipfw_xtable_cfg { > > + ip_fw3_opheader opheader; /* IP_FW3 opcode */ > > + uint32_t size; /* size of structure in bytes */ > > + uint32_t mask; /* data mask to set/retrieve */ > > + uint32_t tlvmask; /* data mask to set/retrieve */ > > + uint32_t tbl; /* table id */ > > + uint16_t type; /* table type */ > > + uint16_t ftype; /* table format type */ > > + uint32_t count; /* number of records */ > > + uint32_t refcnt; /* number of references */ > > + char tablename[64]; /* table name */ > > +} ipfw_xtable_cfg; > > + > > +typedef struct _ipfw_xtable_tlv { > > + uint16_t type; > > + uint16_t length; > > +} ipfw_xtable_tlv; > > + > > + > > + > > #endif /* _IPFW2_H */ > > Index: sys/netpfil/ipfw/ip_fw2.c > > =================================================================== > > --- sys/netpfil/ipfw/ip_fw2.c (revision 266306) > > +++ sys/netpfil/ipfw/ip_fw2.c (working copy) > > @@ -352,15 +352,16 @@ tcpopts_match(struct tcphdr *tcp, ipfw_insn *cmd) > > } > > > > static int > > -iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain > *chain, uint32_t *tablearg) > > +iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain > *chain, > > + uint32_t *tablearg) > > { > > if (ifp == NULL) /* no iface with this packet, match fails > */ > > return 0; > > /* Check by name or by IP address */ > > if (cmd->name[0] != '\0') { /* match by name */ > > if (cmd->name[0] == '\1') /* use tablearg to match */ > > - return ipfw_lookup_table_extended(chain, > cmd->p.glob, > > - ifp->if_xname, tablearg, > IPFW_TABLE_INTERFACE); > > + return (ipfw_lookup_table(chain, cmd->p.glob, > IFNAMSIZ, > > + ifp, tablearg)); > > /* Check name */ > > if (cmd->p.glob) { > > if (fnmatch(cmd->name, ifp->if_xname, 0) == 0) > > @@ -1492,7 +1493,7 @@ do { > \ > > break; > > } > > match = ipfw_lookup_table(chain, > > - cmd->arg1, key, &v); > > + cmd->arg1, sizeof(uint32_t), &key, > &v); > > if (!match) > > break; > > if (cmdlen == > F_INSN_SIZE(ipfw_insn_u32)) > > @@ -1504,9 +1505,9 @@ do { > \ > > uint32_t v = 0; > > void *pkey = (cmd->opcode == > O_IP_DST_LOOKUP) ? > > &args->f_id.dst_ip6: > &args->f_id.src_ip6; > > - match = > ipfw_lookup_table_extended(chain, > > - cmd->arg1, pkey, > &v, > > - IPFW_TABLE_CIDR); > > + match = ipfw_lookup_table(chain, > > + cmd->arg1, sizeof(uint32_t), > > + pkey, &v); > > if (cmdlen == > F_INSN_SIZE(ipfw_insn_u32)) > > match = ((ipfw_insn_u32 > *)cmd)->d[0] == v; > > if (match) > > Index: sys/netpfil/ipfw/ip_fw_private.h > > =================================================================== > > --- sys/netpfil/ipfw/ip_fw_private.h (revision 266306) > > +++ sys/netpfil/ipfw/ip_fw_private.h (working copy) > > @@ -217,9 +217,7 @@ struct ip_fw_chain { > > uint32_t id; /* ruleset id */ > > int n_rules; /* number of static rules */ > > LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ > > - struct radix_node_head **tables; /* IPv4 tables */ > > - struct radix_node_head **xtables; /* extended tables */ > > - uint8_t *tabletype; /* Array of table types */ > > + void *tablestate; /* Array of table data */ > > #if defined( __linux__ ) || defined( _WIN32 ) > > spinlock_t rwmtx; > > #else > > @@ -229,6 +227,7 @@ struct ip_fw_chain { > > uint32_t gencnt; /* NAT generation count */ > > struct ip_fw *reap; /* list of rules to reap */ > > struct ip_fw *default_rule; > > + void *tablecfg; /* tables module configuration */ > > #if defined( __linux__ ) || defined( _WIN32 ) > > spinlock_t uh_lock; > > #else > > @@ -303,9 +302,8 @@ int ipfw_chk(struct ip_fw_args *args); > > void ipfw_reap_rules(struct ip_fw *head); > > > > /* In ip_fw_table.c */ > > -struct radix_node; > > -int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t > addr, > > - uint32_t *val); > > +int ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t > keylen, > > + void *key, uint32_t *val); > > int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, > void *paddr, > > uint32_t *val, int type); > > int ipfw_init_tables(struct ip_fw_chain *ch); > > @@ -316,11 +314,18 @@ int ipfw_add_table_entry(struct ip_fw_chain *ch, u > > int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void > *paddr, > > uint8_t plen, uint8_t mlen, uint8_t type); > > int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t > *cnt); > > -int ipfw_dump_table_entry(struct radix_node *rn, void *arg); > > int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); > > -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t > *cnt); > > +int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t > *sz, > > + uint32_t *cnt); > > int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); > > int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); > > +int ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl); > > +int ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int > ftype); > > +int ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl); > > +int ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > > + size_t *sz); > > +int ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > > + size_t *sz); > > > > /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ > > > > Index: sys/netpfil/ipfw/ip_fw_sockopt.c > > =================================================================== > > --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266306) > > +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy) > > @@ -146,6 +146,119 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * > > } > > > > /* > > + * In @del==0 mode: > > + * Checks is opcode is referencing table of appropriate type. > > + * Adds reference count for found table if true. > > + * In del==1 mode > > + * Decrements refcount for given table. > > + * > > + * Returns 0 on success and appropriate error code otherwise. > > + */ > > +static int > > +bind_tables(struct ip_fw_chain *chain, struct ip_fw *rule, int del) > > +{ > > + int cmdlen, error, ftype, l, skip, type, v; > > + ipfw_insn *cmd; > > + ipfw_insn_if *cmdif; > > + uint32_t tbl; > > + > > + l = rule->cmd_len; > > + cmd = rule->cmd; > > + cmdlen = 0; > > + tbl = 0; > > + type = 0; > > + ftype = 0; > > + > > + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { > > + cmdlen = F_LEN(cmd); > > + skip = 0; > > + > > + switch (cmd->opcode) { > > + case O_IP_SRC_LOOKUP: > > + case O_IP_DST_LOOKUP: > > + /* Basic IPv4/IPv6 or u32 lookups */ > > + tbl = cmd->arg1; > > + /* Assume CIDR by default */ > > + type = IPFW_TABLE_CIDR; > > + ftype = 0; > > + > > + if (cmdlen > F_INSN_SIZE(ipfw_insn_u32)) { > > + /* generic lookup. The key must be > > + * in 32bit big-endian format. > > + */ > > + v = ((ipfw_insn_u32 *)cmd)->d[1]; > > + switch (v) { > > + case 0: > > + case 1: > > + /* IPv4 src/dst */ > > + break; > > + case 2: > > + case 3: > > + /* src/dst port */ > > + //type = IPFW_TABLE_U16; > > + break; > > + case 4: > > + /* uid/gid */ > > + //type = IPFW_TABLE_U32; > > + case 5: > > + //type = IPFW_TABLE_U32; > > + /* jid */ > > + case 6: > > + //type = IPFW_TABLE_U16; > > + /* dscp */ > > + break; > > + } > > + } > > + break; > > + case O_XMIT: > > + case O_RECV: > > + case O_VIA: > > + /* Interface table, possibly */ > > + cmdif = (ipfw_insn_if *)cmd; > > + if (cmdif->name[0] != '\1') { > > + skip = 1; > > + break; > > + } > > + > > + type = IPFW_TABLE_INTERFACE; > > + ftype = 0; > > + tbl = cmdif->p.glob; > > + break; > > + default: > > + skip = 1; > > + } > > + > > + if (skip != 0) > > + continue; > > + > > + /* ref/unref given table */ > > + if (del != 0) > > + error = ipfw_unref_table(chain, tbl); > > + else > > + error = ipfw_ref_table(chain, tbl, type, ftype); > > + > > + if (error != 0) > > + return (error); > > + } > > + > > + return (0); > > +} > > + > > +/* > > + * Removes table bindings for every rule in rule chain @head. > > + */ > > +static void > > +unbind_tables(struct ip_fw_chain *chain, struct ip_fw *head) > > +{ > > + struct ip_fw *rule; > > + > > + while ((rule = head) != NULL) { > > + head = head->x_next; > > + bind_tables(chain, rule, 1); > > + } > > +} > > + > > +/* > > * Add a new rule to the list. Copy the rule into a malloc'ed area, then > > * possibly create a rule number and add the rule to the list. > > * Update the rule_number in the input struct so the caller knows it as > well. > > @@ -157,6 +270,7 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip > > { > > struct ip_fw *rule; > > int i, l, insert_before; > > + int error; > > struct ip_fw **map; /* the new array of pointers */ > > > > if (chain->map == NULL || input_rule->rulenum > IPFW_DEFAULT_RULE > - 1) > > @@ -168,9 +282,16 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip > > map = get_map(chain, 1, 0 /* not locked */); > > if (map == NULL) { > > free(rule, M_IPFW); > > - return ENOSPC; > > + return (ENOSPC); > > } > > > > + /* Reference tables, if any */ > > + if ((error = bind_tables(chain, input_rule, 0)) != 0) { > > + IPFW_UH_WUNLOCK(chain); > > + free(rule, M_IPFW); > > + return (error); > > + } > > + > > bcopy(input_rule, rule, l); > > /* clear fields not settable from userland */ > > rule->x_next = NULL; > > @@ -421,6 +542,7 @@ del_entry(struct ip_fw_chain *chain, uint32_t arg) > > > > rule = chain->reap; > > chain->reap = NULL; > > + unbind_tables(chain, rule); > > IPFW_UH_WUNLOCK(chain); > > ipfw_reap_rules(rule); > > if (map) > > @@ -934,6 +1056,7 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf > > > > > > #define IP_FW3_OPLENGTH(x) ((x)->sopt_valsize - > sizeof(ip_fw3_opheader)) > > + > > /** > > * {set|get}sockopt parser. > > */ > > @@ -1113,7 +1236,7 @@ ipfw_ctl(struct sockopt *sopt) > > sopt->sopt_name == IP_FW_RESETLOG); > > break; > > > > - /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ > > + /*--- TABLE locking is described in ip_fw_table.c ---*/ > > case IP_FW_TABLE_ADD: > > { > > ipfw_table_entry ent; > > @@ -1237,7 +1360,7 @@ ipfw_ctl(struct sockopt *sopt) > > tbl = (uint32_t *)(op3 + 1); > > > > IPFW_RLOCK(chain); > > - error = ipfw_count_xtable(chain, *tbl, tbl); > > + error = ipfw_count_xtable(chain, *tbl, tbl, NULL); > > IPFW_RUNLOCK(chain); > > if (error) > > break; > > @@ -1281,6 +1404,39 @@ ipfw_ctl(struct sockopt *sopt) > > } > > break; > > > > + case IP_FW_TABLE_XGETCFG: /* IP_FW3 */ > > + case IP_FW_TABLE_XSETCFG: /* IP_FW3 */ > > + { > > + ipfw_xtable_cfg *xcfg; > > + > > + if (sopt->sopt_valsize < sizeof(xcfg)) { > > + error = EINVAL; > > + break; > > + } > > + if ((size = sopt->sopt_valsize) > RULE_MAXSIZE) { > > + error = E2BIG; > > + break; > > + } > > + > > + xcfg = malloc(size, M_TEMP, M_WAITOK); > > + error = sooptcopyin(sopt, xcfg, size, > sizeof(*xcfg)); > > + if (error != 0) { > > + free(xcfg, M_TEMP); > > + break; > > + } > > + > > + if (opt == IP_FW_TABLE_XGETCFG) { > > + error = ipfw_getconfig_table(chain, xcfg, > &size); > > + if (error == 0) > > + error = sooptcopyout(sopt, xcfg, > size); > > + } else > > + error = ipfw_setconfig_table(chain, xcfg, > &size); > > + > > + free(xcfg, M_TEMP); > > + } > > + break; > > + > > + > > /*--- NAT operations are protected by the IPFW_LOCK ---*/ > > case IP_FW_NAT_CFG: > > if (IPFW_NAT_LOADED) > > Index: sys/netpfil/ipfw/ip_fw_table.c > > =================================================================== > > --- sys/netpfil/ipfw/ip_fw_table.c (revision 266310) > > +++ sys/netpfil/ipfw/ip_fw_table.c (working copy) > > @@ -35,8 +35,13 @@ __FBSDID("$FreeBSD$"); > > * As a degenerate case we can interpret keys as 32-bit integers > > * (with a /32 mask). > > * > > - * The table is protected by the IPFW lock even for manipulation coming > > - * from userland, because operations are typically fast. > > + * Locking resembles ipfw rules model: > > + * changes in: table entries, table count and table_info are protected > by holding > > + * both UH and main write locks. > > + * changes in table_config struture are protected by UH lock. > > + * > > + * Userland readers should use UH lock if possible. > > + * > > */ > > > > #include "opt_ipfw.h" > > @@ -48,12 +53,14 @@ __FBSDID("$FreeBSD$"); > > > > #include > > #include > > +#include > > #include > > #include > > #include > > #include > > #include > > #include > > +#include > > #include /* ip_fw.h requires IFNAMSIZ */ > > #include > > #include > > @@ -101,6 +108,65 @@ struct table_xentry { > > }; > > > > /* > > + * filled for non-CIDR or named tables > > + * > > + * Table has the following `type` concepts: > > + * > > + * `tabletype` represents lookup key type (cidr, ifp, uid, etc..) > > + * `ftype` is pure userland field helping to properly format table data > > + * `atype` represents exact lookup algorithm for given tabletype. > > + * For example, we can use more efficient search schemes if we plan > > + * to use some specific table for storing host-routes only. > > + * > > + */ > > +struct table_config { > > + uint8_t tabletype; /* lookup table types */ > > + uint8_t ftype; /* format table type */ > > + uint8_t atype; /* algorith type */ > > + uint8_t spare0; > > + uint32_t refcnt; /* Number of references */ > > + uint32_t count; /* Number of records */ > > + struct table_info *ti; > > + TAILQ_ENTRY(table_config) next; /* namehash */ > > + char tablename[64]; /* table name */ > > +}; > > + > > +typedef int (table_lookup_t)(void *state, void *xstate, void *key, > > + uint32_t keylen, uint32_t *val); > > + > > +struct table_info { > > + void *state; /* IPv4 tables */ > > + void *xstate;/* extended tables */ > > + table_lookup_t *lookup;/* lookup function */ > > + struct table_config *cfg; /* Additional data, can be NULL */ > > +}; > > + > > +static int lookup_cidr(void *, void *, void *, uint32_t, uint32_t *); > > +static int lookup_iface(void *, void *, void *, uint32_t, uint32_t *); > > + > > +static table_lookup_t (*tablehandlers[]) = { > > + NULL, /* type 0 unused */ > > + lookup_cidr, /* IPFW_TABLE_CIDR */ > > + lookup_iface, /* IPFW_TABLE_INTERFACE */ > > +}; > > + > > +#define TABLETYPE(t) ((t)->cfg != NULL ? > (t)->cfg->tabletype:IPFW_TABLE_CIDR) > > +#define TABLEFTYPE(ti) ((ti)->cfg != NULL ? (ti)->cfg->ftype : 0) > > +#define TABLEREFS(ti) ((ti)->cfg != NULL ? (ti)->cfg->refcnt : 0) > > + > > +#define TABLENAME_HASH_SIZE 32 > > +struct tables_config { > > + TAILQ_HEAD(, table_config) thash[TABLENAME_HASH_SIZE]; > > +}; > > + > > +#define TABLENAME_HASH(n) > (fnv_32_str(n,FNV1_32_INIT)%TABLENAME_HASH_SIZE) > > +static struct table_info *find_table(struct ip_fw_chain *ch, char > *tablename); > > + > > +static void flush_table(struct ip_fw_chain *ch, struct table_info *ti); > > +static int change_table_handler(struct ip_fw_chain *ch, struct > table_info *ti, > > + struct table_info *ti2, table_lookup_t lookup); > > + > > +/* > > * The radix code expects addr and mask to be array of bytes, > > * with the first byte being the length of the array. rn_inithead > > * is called with the offset in bits of the lookup key within the > > @@ -145,13 +211,19 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > > struct radix_node *rn; > > in_addr_t addr; > > int offset; > > - void *ent_ptr; > > + uintptr_t state_off; > > + void *ent_ptr, *tablestate; > > struct sockaddr *addr_ptr, *mask_ptr; > > + struct table_info *ti; > > + struct table_config *cfg; > > char c; > > > > if (tbl >= V_fw_tables_max) > > return (EINVAL); > > > > + tablestate = ch->tablestate; > > + ti = &((struct table_info *)tablestate)[tbl]; > > + > > switch (type) { > > case IPFW_TABLE_CIDR: > > if (plen == sizeof(in_addr_t)) { > > @@ -170,7 +242,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > > addr = *((in_addr_t *)paddr); > > ent->addr.sin_addr.s_addr = addr & > ent->mask.sin_addr.s_addr; > > /* Set pointers */ > > - rnh_ptr = &ch->tables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->state; > > ent_ptr = ent; > > addr_ptr = (struct sockaddr *)&ent->addr; > > mask_ptr = (struct sockaddr *)&ent->mask; > > @@ -191,7 +263,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > > memcpy(&xent->a.addr6.sin6_addr, paddr, > sizeof(struct in6_addr)); > > APPLY_MASK(&xent->a.addr6.sin6_addr, > &xent->m.mask6.sin6_addr); > > /* Set pointers */ > > - rnh_ptr = &ch->xtables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > > ent_ptr = xent; > > addr_ptr = (struct sockaddr *)&xent->a.addr6; > > mask_ptr = (struct sockaddr *)&xent->m.mask6; > > @@ -227,7 +299,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > > mask_ptr = (struct sockaddr *)&xent->m.ifmask; > > #endif > > /* Set pointers */ > > - rnh_ptr = &ch->xtables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > > ent_ptr = xent; > > addr_ptr = (struct sockaddr *)&xent->a.iface; > > mask_ptr = NULL; > > @@ -237,48 +309,67 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 > > return (EINVAL); > > } > > > > + IPFW_UH_WLOCK(ch); > > IPFW_WLOCK(ch); > > > > + /* > > + * Check if tablestate was reallocated. > > + */ > > + if (ch->tablestate != tablestate) { > > + state_off = (uintptr_t)ti - (uintptr_t)tablestate; > > + ti = (struct table_info *) > > + (state_off + (uintptr_t)ch->tablestate); > > + > > + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; > > + rnh_ptr = (struct radix_node_head **) > > + (state_off + (uintptr_t)ch->tablestate); > > + } > > + > > /* Check if tabletype is valid */ > > - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { > > + if (TABLETYPE(ti) != type) { > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > free(ent_ptr, M_IPFW_TBL); > > - return (EINVAL); > > + return (EFTYPE); > > } > > > > /* Check if radix tree exists */ > > if ((rnh = *rnh_ptr) == NULL) { > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > /* Create radix for a new table */ > > if (!rn_inithead((void **)&rnh, offset)) { > > free(ent_ptr, M_IPFW_TBL); > > return (ENOMEM); > > } > > > > + IPFW_UH_WLOCK(ch); > > IPFW_WLOCK(ch); > > if (*rnh_ptr != NULL) { > > /* Tree is already attached by other thread */ > > rn_detachhead((void **)&rnh); > > rnh = *rnh_ptr; > > /* Check table type another time */ > > - if (ch->tabletype[tbl] != type) { > > + if (TABLETYPE(ti) != type) { > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > free(ent_ptr, M_IPFW_TBL); > > - return (EINVAL); > > + return (EFTYPE); > > } > > } else { > > + /* new trie */ > > *rnh_ptr = rnh; > > - /* > > - * Set table type. It can be set already > > - * (if we have IPv6-only table) but setting > > - * it another time does not hurt > > - */ > > - ch->tabletype[tbl] = type; > > } > > } > > > > rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); > > + > > + /* Maintain number of records */ > > + if (rn != NULL && (cfg = ti->cfg) != NULL) > > + cfg->count++; > > + > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > > > if (rn == NULL) { > > free(ent_ptr, M_IPFW_TBL); > > @@ -296,11 +387,18 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > in_addr_t addr; > > struct sockaddr_in sa, mask; > > struct sockaddr *sa_ptr, *mask_ptr; > > + struct table_info *ti; > > + struct table_config *cfg; > > + void *tablestate; > > + uintptr_t state_off; > > char c; > > > > if (tbl >= V_fw_tables_max) > > return (EINVAL); > > > > + tablestate = ch->tablestate; > > + ti = &((struct table_info *)tablestate)[tbl]; > > + > > switch (type) { > > case IPFW_TABLE_CIDR: > > if (plen == sizeof(in_addr_t)) { > > @@ -310,7 +408,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - > mlen)) - 1) : 0); > > addr = *((in_addr_t *)paddr); > > sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; > > - rnh_ptr = &ch->tables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->state; > > sa_ptr = (struct sockaddr *)&sa; > > mask_ptr = (struct sockaddr *)&mask; > > #ifdef INET6 > > @@ -327,7 +425,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > ipv6_writemask(&mask6.sin6_addr, mlen); > > memcpy(&sa6.sin6_addr, paddr, sizeof(struct > in6_addr)); > > APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); > > - rnh_ptr = &ch->xtables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > > sa_ptr = (struct sockaddr *)&sa6; > > mask_ptr = (struct sockaddr *)&mask6; > > #endif > > @@ -362,7 +460,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > mask_ptr = NULL; > > memcpy(ifname.ifname, paddr, mlen); > > /* Set pointers */ > > - rnh_ptr = &ch->xtables[tbl]; > > + rnh_ptr = (struct radix_node_head **)&ti->xstate; > > sa_ptr = (struct sockaddr *)&ifname; > > > > break; > > @@ -371,19 +469,42 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > return (EINVAL); > > } > > > > + IPFW_UH_WLOCK(ch); > > IPFW_WLOCK(ch); > > + > > + /* > > + * Check if tablestate was reallocated. > > + */ > > + if (ch->tablestate != tablestate) { > > + state_off = (uintptr_t)ti - (uintptr_t)tablestate; > > + ti = (struct table_info *) > > + (state_off + (uintptr_t)ch->tablestate); > > + > > + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; > > + rnh_ptr = (struct radix_node_head **) > > + (state_off + (uintptr_t)ch->tablestate); > > + } > > + > > if ((rnh = *rnh_ptr) == NULL) { > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > return (ESRCH); > > } > > > > - if (ch->tabletype[tbl] != type) { > > + if (TABLETYPE(ti) != type) { > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > return (EINVAL); > > } > > > > ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, > rnh); > > + > > + /* Maintain number of records */ > > + if (ent != NULL && (cfg = ti->cfg) != NULL) > > + cfg->count--; > > + > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > > > if (ent == NULL) > > return (ESRCH); > > @@ -392,7 +513,381 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 > > return (0); > > } > > > > +/* > > + * binds newly-created config to given table > > + */ > > static int > > +bind_config(struct ip_fw_chain *ch, uint32_t tbl, struct table_info *ti, > > + struct table_config *cfg) > > +{ > > + int error; > > + uint32_t sz, cnt; > > + > > + /* Set defaults */ > > + cfg->tabletype = IPFW_TABLE_CIDR; > > + > > + /* count current number of entries */ > > + if ((error = ipfw_count_xtable(ch, tbl, &sz, &cnt)) != 0) > > + return (error); > > + cfg->count = cnt; > > + > > + ti->cfg = cfg; > > + cfg->ti = ti; > > + > > + return (0); > > +} > > + > > +static struct table_info * > > +find_table(struct ip_fw_chain *ch, char *tablename) > > +{ > > + struct tables_config *tc; > > + struct table_config *cfg; > > + int hash; > > + > > + hash = TABLENAME_HASH(tablename); > > + tc = (struct tables_config *)ch->tablecfg; > > + > > + TAILQ_FOREACH(cfg, &tc->thash[hash], next) { > > + if (strcmp(cfg->tablename, tablename) == 0) > > + return (cfg->ti); > > + } > > + > > + return (NULL); > > +} > > + > > +/* > > + * Fills in supplied buffer with table configuration info. > > + */ > > +int > > +ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > size_t *sz) > > +{ > > + struct table_info *ti; > > + struct table_config *cfg; > > + uint32_t tbl, tsz, cnt; > > + > > + tbl = xcfg->tbl; > > + > > + IPFW_UH_RLOCK(ch); > > + > > + if (xcfg->mask & IPFW_XCFG_NUMID) { > > + /* Table is identified by number */ > > + tbl = xcfg->tbl; > > + > > + if (tbl >= V_fw_tables_max) { > > + IPFW_UH_RUNLOCK(ch); > > + return (EINVAL); > > + } > > + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { > > + /* Let's try to find table by name */ > > + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > > + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { > > + IPFW_UH_RUNLOCK(ch); > > + return (EINVAL); > > + } > > + > > + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { > > + IPFW_UH_RUNLOCK(ch); > > + return (ESRCH); > > + } > > + > > + /* Guess table number based on offset */ > > + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / > sizeof(*ti); > > + } else { > > + IPFW_UH_RUNLOCK(ch); > > + return (ESRCH); > > + } > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + cfg = ti->cfg; > > + > > + /* Save table id anywat */ > > + xcfg->tbl = tbl; > > + > > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0 ) { > > + if (cfg == NULL || cfg->tablename == NULL) { > > + xcfg->tablename[0] = '\0'; > > + xcfg->mask &= ~IPFW_XCFG_NAME; > > + } else > > + strlcpy(xcfg->tablename, cfg->tablename, > > + sizeof(cfg->tablename)); > > + } > > + > > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) > > + xcfg->type = TABLETYPE(ti); > > + > > + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) > > + xcfg->ftype = TABLEFTYPE(ti); > > + > > + if ((xcfg->mask & IPFW_XCFG_REFS) != 0) > > + xcfg->refcnt = TABLEREFS(ti); > > + > > + if ((xcfg->mask & IPFW_XCFG_CNT) != 0) { > > + /* > > + * Use items count from cfg, if it exists. > > + * Otherwise, calculate manually. > > + */ > > + if (cfg != NULL) > > + xcfg->count = cfg->count; > > + else { > > + IPFW_RLOCK(ch); > > + ipfw_count_xtable(ch, tbl, &tsz, &cnt); > > + IPFW_RUNLOCK(ch); > > + xcfg->count = cnt; > > + } > > + } > > + > > + IPFW_UH_RUNLOCK(ch); > > + > > + return (0); > > +} > > + > > +/* > > + * Fills in supplied buffer with table configuration info. > > + */ > > +int > > +ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, > size_t *sz) > > +{ > > + struct table_info *ti, ti_storage, *ti2; > > + struct table_config *cfg; > > + uint32_t tbl; > > + int l; > > + int error; > > + > > + tbl = xcfg->tbl; > > + > > + /* Do some preliminary checking */ > > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > > + if (xcfg->type > IPFW_TABLE_MAXTYPE || xcfg->type == 0) > > + return (EINVAL); > > + } > > + > > + /* > > + * Check if tablename is null-terminated. > > + * More fine-grained checks should be done by userland. > > + */ > > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > > + l = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > > + if (l == sizeof(xcfg->tablename) || l == 0) > > + return (EINVAL); > > + } > > + > > + /* Checl if we need to allocate config structure */ > > + IPFW_UH_RLOCK(ch); > > + > > + if (xcfg->mask & IPFW_XCFG_NUMID) { > > + /* Table is identified by number */ > > + tbl = xcfg->tbl; > > + > > + if (tbl >= V_fw_tables_max) { > > + IPFW_UH_RUNLOCK(ch); > > + return (EINVAL); > > + } > > + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { > > + /* Let's try to find table by name */ > > + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); > > + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { > > + IPFW_UH_RUNLOCK(ch); > > + return (EINVAL); > > + } > > + > > + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { > > + IPFW_UH_RUNLOCK(ch); > > + return (ESRCH); > > + } > > + > > + /* Guess table number based on offset */ > > + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / > sizeof(*ti); > > + } > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + cfg = ti->cfg; > > + IPFW_UH_RUNLOCK(ch); > > + > > + /* > > + * Allocate new config structure if needed. > > + * cfg represents pointer to new structure or NULL > > + */ > > + if (cfg == NULL) > > + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_WAITOK | M_ZERO); > > + else > > + cfg = NULL; > > + > > + IPFW_UH_WLOCK(ch); > > + /* > > + * We need to check another time since there is probability that > > + * V_fw_tables_max was changed or ti->cfg was destroyed > > + */ > > + > > + if (tbl >= V_fw_tables_max) { > > + IPFW_UH_WUNLOCK(ch); > > + return (EINVAL); > > + } > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + if (ti->cfg == NULL) { > > + if (cfg == NULL) { > > + /* destroy_table() had happened before > IPFW_UH_WLOCK */ > > + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, > M_NOWAIT|M_ZERO); > > + if (cfg == NULL) { > > + IPFW_UH_WUNLOCK(ch); > > + return (ENOMEM); > > + } > > + } > > + > > + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { > > + IPFW_UH_WUNLOCK(ch); > > + free(cfg, M_IPFW_TBL); > > + return (error); > > + } > > + } else if (cfg != NULL) { > > + /* > > + * ti->cfg has been allocated by other thread. > > + * Free our allocation. > > + */ > > + free(cfg, M_IPFW_TBL); > > + } > > + > > + cfg = ti->cfg; > > + > > + /* > > + * Pretend to be atomic: check everything before doing actual job > > + */ > > + error = 0; > > + > > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > > + /* > > + * We can set type if > > + * 1) no one hold any references to our table > > + * 2) we have no records in given table > > + */ > > + > > + if ((cfg->tabletype != xcfg->type) && (cfg->refcnt > 0) && > > + (cfg->count > 0)) > > + error = EBUSY; > > + } > > + > > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > > + /* Check if new table name exists */ > > + if (find_table(ch, xcfg->tablename) != NULL) > > + error = EEXIST; > > + } > > + > > + if (error != 0) { > > + IPFW_UH_WUNLOCK(ch); > > + return (error); > > + } > > + > > + /* Everything checked, let's get the job done */ > > + ti2 = NULL; > > + > > + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { > > + /* we need to change table_info here */ > > + IPFW_WLOCK(ch); > > + cfg->tabletype = xcfg->type; > > + if (change_table_handler(ch, ti, &ti_storage, > > + tablehandlers[cfg->tabletype]) != 0) > > + ti2 = &ti_storage; > > + IPFW_WUNLOCK(ch); > > + } > > + > > + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { > > + int hash; > > + struct tables_config *tc; > > + tc = (struct tables_config *)ch->tablecfg; > > + > > + if (strlen(cfg->tablename) != 0) { > > + /* unlink from old one */ > > + hash = TABLENAME_HASH(cfg->tablename); > > + TAILQ_REMOVE(&tc->thash[hash], cfg, next); > > + } > > + strlcpy(cfg->tablename, xcfg->tablename, > > + sizeof(cfg->tablename)); > > + /* link new one */ > > + hash = TABLENAME_HASH(cfg->tablename); > > + TAILQ_INSERT_HEAD(&tc->thash[hash], cfg, next); > > + } > > + > > + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) > > + cfg->ftype = xcfg->ftype; > > + > > + IPFW_UH_WUNLOCK(ch); > > + > > + /* Free old table state if set */ > > + if (ti2 != NULL) > > + flush_table(ch, ti2); > > + > > + return (0); > > +} > > + > > + > > +/* > > + * Checks if given table's type is the same as @type. > > + * If true, increment @tbl refcount > > + */ > > +int > > +ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int > ftype) > > +{ > > + struct table_info *ti; > > + struct table_config *cfg; > > + int error; > > + > > + IPFW_UH_WLOCK_ASSERT(ch); > > + > > + if (tbl >= V_fw_tables_max) > > + return (EINVAL); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + > > + if (TABLETYPE(ti) != type) > > + return (EFTYPE); > > + > > + /* Ignore ftype for now */ > > + > > + if (ti->cfg == NULL) { > > + /* > > + * Let's create confdata. > > + * XXX: we can fail here > > + */ > > + cfg = malloc(sizeof(struct table_config), M_IPFW_TBL, > > + M_NOWAIT | M_ZERO); > > + > > + if (cfg == NULL) > > + return (ENOMEM); > > + > > + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { > > + free(cfg, M_IPFW_TBL); > > + return (error); > > + } > > + } > > + > > + ti->cfg->refcnt++; > > + > > + return (0); > > +} > > + > > +int > > +ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl) > > +{ > > + struct table_info *ti; > > + > > + IPFW_UH_WLOCK_ASSERT(ch); > > + > > + if (tbl >= V_fw_tables_max) > > + return (EINVAL); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + > > + KASSERT(ti->cfg != 0, ("ipfw: no config for table %d", tbl)); > > + > > + KASSERT(ti->cfg->refcnt > 0, ("ipfw: refcnt for table %d is %d", > > + tbl, ti->cfg->refcnt)); > > + > > + ti->cfg->refcnt--; > > + > > + return (0); > > +} > > + > > +static int > > flush_table_entry(struct radix_node *rn, void *arg) > > { > > struct radix_node_head * const rnh = arg; > > @@ -405,40 +900,138 @@ flush_table_entry(struct radix_node *rn, void *arg > > return (0); > > } > > > > +/* > > + * Flushes data from table state pointers. > > + * Frees pointers itself. > > + */ > > +static void > > +flush_table(struct ip_fw_chain *ch, struct table_info *ti) > > +{ > > + struct radix_node_head *rnh; > > + > > + if ((rnh = ti->state) != NULL) { > > + rnh->rnh_walktree(rnh, flush_table_entry, rnh); > > + rn_detachhead((void **)&rnh); > > + } > > + > > + if ((rnh = ti->xstate) != NULL) { > > + rnh->rnh_walktree(rnh, flush_table_entry, rnh); > > + rn_detachhead((void **)&rnh); > > + } > > +} > > + > > +/* > > + * change table handler saving previous state in @ti2. > > + * Returns 1 if hable had some state, 0 otherwise. > > + */ > > +static int > > +change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, > > + struct table_info *ti2, table_lookup_t lookup) > > +{ > > + > > + if (ti->state == NULL && ti->state == NULL) { > > + /* No state. Set handler and return. */ > > + ti->lookup = lookup; > > + return (0); > > + } > > + > > + *ti2 = *ti; > > + > > + ti->state = NULL; > > + ti->xstate = NULL; > > + ti->lookup = lookup; > > + > > + return (1); > > +} > > + > > +/* > > + * flushes all data in given table leaving table type/naming > > + * intact. > > + */ > > int > > ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) > > { > > - struct radix_node_head *rnh, *xrnh; > > + struct table_info *ti, tti; > > + struct table_config *cfg; > > > > - if (tbl >= V_fw_tables_max) > > + IPFW_UH_WLOCK(ch); > > + IPFW_WLOCK(ch); > > + > > + if (tbl >= V_fw_tables_max) { > > + IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > return (EINVAL); > > + } > > > > - /* > > - * We free both (IPv4 and extended) radix trees and > > - * clear table type here to permit table to be reused > > - * for different type without module reload > > - */ > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + tti = *ti; > > + /* Remove state tables from main structure */ > > + ti->state = NULL; > > + ti->xstate = NULL; > > + if ((cfg = ti->cfg) != NULL) > > + cfg->count = 0; > > + IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > > > + flush_table(ch, &tti); > > + > > + return (0); > > +} > > + > > +static void > > +destroy_table(struct ip_fw_chain *ch, struct table_info *ti) > > +{ > > + struct table_config *cfg; > > + > > + flush_table(ch, ti); > > + > > + if ((cfg = ti->cfg) != NULL) { > > + /* Free configuration state */ > > + free(cfg, M_IPFW_TBL); > > + ti->cfg = NULL; > > + } > > + > > + /* Set lookup pointer back to CIDR */ > > + ti->lookup = lookup_cidr; > > +} > > + > > +/* > > + * Destroys given table iff no rules are referencing it. > > + * Flushes all data in tablestate, destroys any > > + * special configuration/naming associated with the table > > + * and sets its type (ti->cfg == NULL) back to CIDR. > > + */ > > +int > > +ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl) > > +{ > > + struct table_info *ti, tti; > > + struct table_config *cfg; > > + > > + IPFW_UH_WLOCK(ch); > > IPFW_WLOCK(ch); > > - /* Set IPv4 table pointer to zero */ > > - if ((rnh = ch->tables[tbl]) != NULL) > > - ch->tables[tbl] = NULL; > > - /* Set extended table pointer to zero */ > > - if ((xrnh = ch->xtables[tbl]) != NULL) > > - ch->xtables[tbl] = NULL; > > - /* Zero table type */ > > - ch->tabletype[tbl] = 0; > > - IPFW_WUNLOCK(ch); > > > > - if (rnh != NULL) { > > - rnh->rnh_walktree(rnh, flush_table_entry, rnh); > > - rn_detachhead((void **)&rnh); > > + if (tbl >= V_fw_tables_max) { > > + IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > + return (EINVAL); > > } > > > > - if (xrnh != NULL) { > > - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); > > - rn_detachhead((void **)&xrnh); > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + if (((cfg = ti->cfg) != NULL) && (cfg->refcnt != 0)) { > > + /* Table is referenced by some rules */ > > + IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > + return (EBUSY); > > } > > + tti = *ti; > > + /* Remove state tables from main structure */ > > + ti->state = NULL; > > + ti->xstate = NULL; > > + ti->cfg = NULL; > > + IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > + > > + destroy_table(ch, &tti); > > > > return (0); > > } > > @@ -446,152 +1039,179 @@ ipfw_flush_table(struct ip_fw_chain *ch, > uint16_t > > void > > ipfw_destroy_tables(struct ip_fw_chain *ch) > > { > > - uint16_t tbl; > > + uint32_t tbl; > > + struct table_info *ti; > > > > /* Flush all tables */ > > - for (tbl = 0; tbl < V_fw_tables_max; tbl++) > > - ipfw_flush_table(ch, tbl); > > + ti = (struct table_info *)ch->tablestate; > > + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) > > + destroy_table(ch, ti); > > > > /* Free pointers itself */ > > - free(ch->tables, M_IPFW); > > - free(ch->xtables, M_IPFW); > > - free(ch->tabletype, M_IPFW); > > + free(ch->tablestate, M_IPFW_TBL); > > + free(ch->tablecfg, M_IPFW_TBL); > > } > > > > int > > ipfw_init_tables(struct ip_fw_chain *ch) > > { > > + struct table_info *ti; > > + struct tables_config *tc; > > + uint32_t tbl; > > + > > /* Allocate pointers */ > > - ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, > M_WAITOK | M_ZERO); > > - ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, > M_WAITOK | M_ZERO); > > - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, > M_WAITOK | M_ZERO); > > + ch->tablestate = malloc(V_fw_tables_max * sizeof(struct > table_info), > > + M_IPFW_TBL, M_WAITOK | M_ZERO); > > + > > + ch->tablecfg = malloc(sizeof(struct tables_config), M_IPFW_TBL, > > + M_WAITOK | M_ZERO); > > + > > + /* Set initial handlers for all tables */ > > + ti = (struct table_info *)ch->tablestate; > > + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) > > + ti->lookup = lookup_cidr; > > + > > + /* Set up tablenames hash */ > > + tc = ch->tablecfg; > > + for (tbl = 0; tbl < TABLENAME_HASH_SIZE; tbl++) > > + TAILQ_INIT(&tc->thash[tbl]); > > + > > return (0); > > } > > > > int > > ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables) > > { > > - struct radix_node_head **tables, **xtables, *rnh; > > - struct radix_node_head **tables_old, **xtables_old; > > - uint8_t *tabletype, *tabletype_old; > > unsigned int ntables_old, tbl; > > + struct table_info *ti, *ti_old, *ti_new; > > > > /* Check new value for validity */ > > if (ntables > IPFW_TABLES_MAX) > > ntables = IPFW_TABLES_MAX; > > > > /* Allocate new pointers */ > > - tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | > M_ZERO); > > - xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | > M_ZERO); > > - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | > M_ZERO); > > + ti_new = malloc(ntables * sizeof(struct table_info), > > + M_IPFW_TBL, M_WAITOK | M_ZERO); > > > > + IPFW_UH_WLOCK(ch); > > IPFW_WLOCK(ch); > > > > tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; > > > > /* Copy old table pointers */ > > - memcpy(tables, ch->tables, sizeof(void *) * tbl); > > - memcpy(xtables, ch->xtables, sizeof(void *) * tbl); > > - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); > > + memcpy(ti_new, ch->tablestate, sizeof(struct table_info) * tbl); > > > > /* Change pointers and number of tables */ > > - tables_old = ch->tables; > > - xtables_old = ch->xtables; > > - tabletype_old = ch->tabletype; > > - ch->tables = tables; > > - ch->xtables = xtables; > > - ch->tabletype = tabletype; > > + ti_old = (struct table_info *)ch->tablestate; > > + ch->tablestate = ti_new; > > > > ntables_old = V_fw_tables_max; > > V_fw_tables_max = ntables; > > > > IPFW_WUNLOCK(ch); > > + IPFW_UH_WUNLOCK(ch); > > > > /* Check if we need to destroy radix trees */ > > if (ntables < ntables_old) { > > - for (tbl = ntables; tbl < ntables_old; tbl++) { > > - if ((rnh = tables_old[tbl]) != NULL) { > > - rnh->rnh_walktree(rnh, flush_table_entry, > rnh); > > - rn_detachhead((void **)&rnh); > > - } > > + ti = &ti_old[ntables]; > > + for (tbl = ntables; tbl < ntables_old; tbl++, ti++) > > + destroy_table(ch, ti); > > + } > > > > - if ((rnh = xtables_old[tbl]) != NULL) { > > - rnh->rnh_walktree(rnh, flush_table_entry, > rnh); > > - rn_detachhead((void **)&rnh); > > - } > > - } > > + /* Check if we need to setup new ones */ > > + if (ntables > ntables_old) { > > + ti = &ti_new[ntables_old]; > > + for (tbl = ntables_old; tbl < ntables; tbl++, ti++) > > + ti->lookup = lookup_cidr; > > } > > > > /* Free old pointers */ > > - free(tables_old, M_IPFW); > > - free(xtables_old, M_IPFW); > > - free(tabletype_old, M_IPFW); > > + free(ti_old, M_IPFW_TBL); > > > > return (0); > > } > > > > -int > > -ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, > > - uint32_t *val) > > +static int > > +lookup_cidr(void *st, void *xst, void *key, uint32_t keylen, uint32_t > *val) > > { > > struct radix_node_head *rnh; > > struct table_entry *ent; > > - struct sockaddr_in sa; > > + struct table_xentry *xent; > > > > - if (tbl >= V_fw_tables_max) > > + > > + if (keylen == sizeof(struct in_addr)) { > > + /* IPv4 lookup */ > > + struct sockaddr_in sa; > > + > > + if ((rnh = (struct radix_node_head *)st) == NULL) > > + return (0); > > + > > + KEY_LEN(sa) = KEY_LEN_INET; > > + sa.sin_addr.s_addr = ((struct in_addr *)key)->s_addr; > > + ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); > > + if (ent != NULL) { > > + *val = ent->value; > > + return (1); > > + } > > + > > return (0); > > - if ((rnh = ch->tables[tbl]) == NULL) > > + } > > + > > + /* IPv6 lookup */ > > + struct sockaddr_in6 sa6; > > + > > + if ((rnh = (struct radix_node_head *)xst) == NULL) > > return (0); > > - KEY_LEN(sa) = KEY_LEN_INET; > > - sa.sin_addr.s_addr = addr; > > - ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); > > - if (ent != NULL) { > > - *val = ent->value; > > + > > + KEY_LEN(sa6) = KEY_LEN_INET6; > > + memcpy(&sa6.sin6_addr, key, sizeof(struct in6_addr)); > > + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); > > + > > + if (xent != NULL) { > > + *val = xent->value; > > return (1); > > } > > + > > return (0); > > } > > > > -int > > -ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void > *paddr, > > - uint32_t *val, int type) > > +static int > > +lookup_iface(void *st, void *xst, void *key, uint32_t keylen, uint32_t > *val) > > { > > struct radix_node_head *rnh; > > + struct xaddr_iface iface; > > struct table_xentry *xent; > > - struct sockaddr_in6 sa6; > > - struct xaddr_iface iface; > > > > - if (tbl >= V_fw_tables_max) > > + if ((rnh = (struct radix_node_head *)xst) == NULL) > > return (0); > > - if ((rnh = ch->xtables[tbl]) == NULL) > > - return (0); > > > > - switch (type) { > > - case IPFW_TABLE_CIDR: > > - KEY_LEN(sa6) = KEY_LEN_INET6; > > - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); > > - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, > rnh)); > > - break; > > - > > - case IPFW_TABLE_INTERFACE: > > - KEY_LEN(iface) = KEY_LEN_IFACE + > > - strlcpy(iface.ifname, (char *)paddr, IF_NAMESIZE) + 1; > > - /* Assume direct match */ > > - /* FIXME: Add interface pattern matching */ > > - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, > rnh)); > > - break; > > - > > - default: > > - return (0); > > - } > > - > > + /* IPv6 lookup */ > > + KEY_LEN(iface) = KEY_LEN_IFACE + > > + strlcpy(iface.ifname, (char *)key, IF_NAMESIZE) + 1; > > + /* Assume direct match */ > > + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); > > if (xent != NULL) { > > *val = xent->value; > > return (1); > > } > > + > > return (0); > > } > > > > +int > > +ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, > > + uint32_t keylen, void *key, uint32_t *val) > > +{ > > + struct table_info *ti; > > + > > + if (tbl >= V_fw_tables_max) > > + return (0); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + > > + return (ti->lookup(ti->state, ti->xstate, key, keylen, val)); > > +} > > + > > static int > > count_table_entry(struct radix_node *rn, void *arg) > > { > > @@ -605,11 +1225,15 @@ int > > ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) > > { > > struct radix_node_head *rnh; > > + struct table_info *ti; > > > > if (tbl >= V_fw_tables_max) > > return (EINVAL); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + > > *cnt = 0; > > - if ((rnh = ch->tables[tbl]) == NULL) > > + if ((rnh = ti->state) == NULL) > > return (0); > > rnh->rnh_walktree(rnh, count_table_entry, cnt); > > return (0); > > @@ -640,11 +1264,15 @@ int > > ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) > > { > > struct radix_node_head *rnh; > > + struct table_info *ti; > > > > if (tbl->tbl >= V_fw_tables_max) > > return (EINVAL); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; > > + > > tbl->cnt = 0; > > - if ((rnh = ch->tables[tbl->tbl]) == NULL) > > + if ((rnh = ti->state) == NULL) > > return (0); > > rnh->rnh_walktree(rnh, dump_table_entry, tbl); > > return (0); > > @@ -660,20 +1288,32 @@ count_table_xentry(struct radix_node *rn, void *ar > > } > > > > int > > -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) > > +ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, > > + uint32_t *cnt) > > { > > struct radix_node_head *rnh; > > + struct table_info *ti; > > + uint32_t count; > > > > if (tbl >= V_fw_tables_max) > > return (EINVAL); > > - *cnt = 0; > > - if ((rnh = ch->tables[tbl]) != NULL) > > - rnh->rnh_walktree(rnh, count_table_xentry, cnt); > > - if ((rnh = ch->xtables[tbl]) != NULL) > > - rnh->rnh_walktree(rnh, count_table_xentry, cnt); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl]; > > + > > + *sz = 0; > > + count = 0; > > + if ((rnh = ti->state) != NULL) > > + rnh->rnh_walktree(rnh, count_table_xentry, sz); > > + if ((rnh = ti->xstate) != NULL) > > + rnh->rnh_walktree(rnh, count_table_xentry, sz); > > + count = *sz / sizeof(ipfw_table_xentry); > > + > > + if (cnt != NULL) > > + *cnt = count; > > + > > /* Return zero if table is empty */ > > - if (*cnt > 0) > > - (*cnt) += sizeof(ipfw_xtable); > > + if (*sz > 0) > > + (*sz) += sizeof(ipfw_xtable); > > return (0); > > } > > > > @@ -750,14 +1390,18 @@ int > > ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) > > { > > struct radix_node_head *rnh; > > + struct table_info *ti; > > > > if (tbl->tbl >= V_fw_tables_max) > > return (EINVAL); > > + > > + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; > > + > > tbl->cnt = 0; > > - tbl->type = ch->tabletype[tbl->tbl]; > > - if ((rnh = ch->tables[tbl->tbl]) != NULL) > > + tbl->type = TABLETYPE(ti); > > + if ((rnh = ti->state) != NULL) > > rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); > > - if ((rnh = ch->xtables[tbl->tbl]) != NULL) > > + if ((rnh = ti->xstate) != NULL) > > rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); > > return (0); > > } > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed May 21 16:49:46 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D39B506; Wed, 21 May 2014 16:49:46 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70DC828C7; Wed, 21 May 2014 16:49:45 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id w7so1791607lbi.15 for ; Wed, 21 May 2014 09:49:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gZAMD1z7uZbfTTgSTmPO3adW2FUHhDmFTIVf8bcCPAQ=; b=l3vAKITRKEGVhh+eoWeyDxnPtqLfIc1cbSq0VBFhEpNxyKNVWejaM7ydZj2rXqcHNo D20w7Y92YzDCcjyOy30voLWvsQKIAc7bkMFahpu9FFkriUTAfZB+Z+335UXPNU7JAc7p 3O70Gt3LXB/sHLmFDDGum+N58tb1TT6HoBxMnW4PF5P7hLjJGQifPsaxPKDbehOGfq+J t+3gKnn1oPLMM3NkH+IlRmaL6B4xgUHQ2bQZaQDi1+gI3MmhWtexEXRkib/ju47tsvv6 mb7QmseNVo112p1n2LiECrpihhjx9QHmKx/Y8C7K4P5Ku/r2wyYnyUQlMudmB+oi9TRm 4TpQ== MIME-Version: 1.0 X-Received: by 10.112.50.241 with SMTP id f17mr31797467lbo.7.1400690983136; Wed, 21 May 2014 09:49:43 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.107 with HTTP; Wed, 21 May 2014 09:49:43 -0700 (PDT) In-Reply-To: References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> Date: Wed, 21 May 2014 12:49:43 -0400 X-Google-Sender-Auth: qtcbqsyrxpHCxecGpQPltF-yFSQ Message-ID: Subject: Re: [CFT]: ipfw named tables / different tabletypes From: Luigi Rizzo To: Bill Yuan Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Luigi Rizzo , "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 16:49:46 -0000 On Wed, May 21, 2014 at 11:17 AM, Bill Yuan wrote: > 1. "each table can have it's own name", I like this idea. I am also > working on this kind of functions, But I am in different way. In my > opinion. the "name" or "type" of the table, all this are utility function > for the user. in the kernel space. I want to keep all the things the same > as before. and add a translate function in the user land commands, so tha= t > means we need to find a space to keep the mapping of the name<->id or > type<->id, > =E2=80=8Bthis does not work, you'd have another database to keep in sync and on which you have races with modifications to the ipfw config. cheers luigi =E2=80=8B From owner-freebsd-net@FreeBSD.ORG Wed May 21 17:32:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C32C6947; Wed, 21 May 2014 17:32:59 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A49252CE3; Wed, 21 May 2014 17:32:59 +0000 (UTC) Received: from [10.12.72.220] (unknown [69.164.56.1]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id B28AB194138; Wed, 21 May 2014 17:32:57 +0000 (UTC) Subject: Re: [rfc] add non-contiguous CPU ID support to in_rss.c From: Sean Bruno Reply-To: sbruno@freebsd.org To: Adrian Chadd In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Wed, 21 May 2014 06:25:18 -0700 Message-ID: <1400678718.973.0.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 17:32:59 -0000 On Tue, 2014-05-20 at 23:52 -0700, Adrian Chadd wrote: > Hi Robert, > > This patch uses CPU_FIRST() and CPU_NEXT() to iterate over the CPU IDs. > > Think this is alright? > > -a > > > Index: sys/netinet/in_rss.c > =================================================================== > --- sys/netinet/in_rss.c (revision 266429) > +++ sys/netinet/in_rss.c (working copy) > @@ -176,6 +176,7 @@ > rss_init(__unused void *arg) > { > u_int i; > + u_int cpuid; > > /* > * Validate tunables, coerce to sensible values. > @@ -245,11 +246,12 @@ > > /* > * Set up initial CPU assignments: round-robin by default. > - * > - * XXXRW: Need a mapping to non-contiguous IDs here. > */ > - for (i = 0; i < rss_buckets; i++) > - rss_table[i].rte_cpu = i % rss_ncpus; > + cpuid = CPU_FIRST(); > + for (i = 0; i < rss_buckets; i++) { > + rss_table[i].rte_cpu = cpuid; > + cpuid = CPU_NEXT(cpuid); > + } > > /* > * Randomize rrs_key. > __ Yah, I did something similar in igb(4) to assign queues to CPU more correctly. sean From owner-freebsd-net@FreeBSD.ORG Wed May 21 18:11:40 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BD2FBE8; Wed, 21 May 2014 18:11:40 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64F202098; Wed, 21 May 2014 18:11:39 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wn74w-000CBO-ON; Wed, 21 May 2014 18:01:10 +0400 Message-ID: <537CEC12.8050404@FreeBSD.org> Date: Wed, 21 May 2014 22:10:26 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> In-Reply-To: <20140521111002.GB62462@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 18:11:40 -0000 On 21.05.2014 15:10, Luigi Rizzo wrote: > On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: >> Hello list! >> >> This patch adds ability to name tables / reference them by name. >> Additionally, it simplifies adding new table types. > Hi Alex, > at a high level, i think your changes here are in the right direction. Hello! > > However i think part of the design, and also of the patch below, > is not sound/correct so i would like to wait to commit at least > until some of the problems below are resolved. > > 1. The patch as is has several issues (variable declarations in the > middle of a block, assignment in conditionals, incorrect > comments in cut&pasted code, missing checks on strings) > that should be corrected. ..missing documentation and so on :) Of course, I'll fix all these (or most of these :)) > > 2. you are introducing several distinct features that would be > appropriate to commit separately. > > In the specific, the name-number translation is something > that should go in first and independently, and possibly > could be used for other features as well (pipes, nat entries, > jump targets) so it might be good to have a closer look at > its implementation (more later). > > 3. there is no explanation, here or in the code, on how the > names are managed. I could try to figure out from the code > but it would be the wrong way to understand things so let me > ask, and please explain what you have in mind. Currently it is very simple non-resizable hash table with fnv hash based on table name. > > Let me address first the name <-> table-id thing. > > Introducing a symbolic name for tables is a great and useful feature. > However the implementation has some tricky issues: > > - do you want to retain the old numeric identifiers or not ? > I think it is a bad idea to have two namespaces for the same thing, > so if we are switching to alphanumeric strings for tables we should > as well get rid of the numbers (i.e. consider them as strings as well). > > I am strongly in favour of not using names as aliases for numbers. > It would require no changes for clients issuing ipfw commands > from a script, and would not require users to to manually handle > the name-id translations. Well. I'd prefer not to. However, code we're discussing assumes that numeric ids are primary ones (e.g. you can't have named, but unnumbered table, you have to choose number by yourself). Switching to named-only tables can be tricky since we don't want to match them by inside rules and we don't want to loose atomicity when allocating table ids via separate cmds before adding rule. It looks like we can do the following: 1) Add another IP_FW3_ADD opcode with the following layout: { rule len; unmodified rule itself; tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} .. } Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be real values but values described in attached TLVs. After validating/parsing/checking under UH lock we replace fake values with real ones (probably, newly allocated) and return or rollback atomically. The same thing can be done for displaying ruleset, however I'd prefer client to ask for table names and caching them while displaying. Old clients will retain the ability to operate on tables but will see table ids, so nothing will change for them (except the case when new binary adds table "5" and new binary sees id which is not 5, but that shouldn't be a problem). > > - The rename command worries me a lot: > > > ipfw table name XXX > > Names (or renames) table. Not the name has to be unique. > > because it is neither clear nor intuitive whether you want to > 1. just rename a table (possibly breaking or creating references > for rules in the firewall), or > 2. modify the name-id translation preserving existing pointers. > > Consider the sequence > ipfw table 13 name bar > ipfw add 100 count dst-ip table bar > ipfw table 13 name foo > ipfw table 14 name bar > ipfw add 200 count src-port 22 dst-ip table bar > > Approach #1 would detach rule 100 from table 13 and then connect to 14 > (in a non-atomic way), whereas approach #2 would make rule 100 and 200 > point to different tables (which is confusing). > > Now part of this problem goes away if we get rid of number altogether. > > You may legitimately want to swap tables in an atomic way (we have something > similar for rulesets): > ipfw set swap number number There is some problem here: Atomic ruleset changes is a great thing, but tables have no atomic support. We, for example, are solving this by using different table range on every new configuration. It won't be possible with named-only tables (since names usually care some semantic in contrast to numbers). The only thing I can think of is separate namespace per each set. What do you think? > > > So here is what i would propose: > - [gradually] introduce new commands that accept strings for every place where > a number was previously accepted. Internally in the firewall, the old > 16-bit number is interpreted as a string. Yup. > > - within the kernel create a small set of functions (invoked on insert, list, delete) > that do proper checks on the string and translate it to a pointer (or short integer, > i.e. an index in an array) to the proper object. Done properly, we can reuse > this code also for pipes, sets, nat groups and whatnot. Yup. > > When a rule references an object that does not exist, you create an empty > object that a subsequent table/pipe/XXX 'create' would initialize. > On 'destroy', no need to touch the ruleset, just clear the object. Yes. This looks better than requiring user to create table of given type _before_ referencing it. > > - for renames, try to follow the approach used for sets i.e. > ipfw table rename old-name new-name > changes the name, preserves the object. > Does not touch the ruleset, only the translation table Well, I'm not sure about renaming. I'd prefer permitting renaming iff table is not referenced. (and why do we need to rename tables?) > ipfw table swap first-table second-table > atomically swap the content of the two table objects > (once again not touching the rulesets) > > cheers > luigi > > I had a similar issue when implementing > you can point to an empty one) and then to the new one, > whereas #2 would repoint rule 100 to table 'foo' ( > > because i do not know whether > names are stored > In the details, however, I would really prefer like that you > > >> Change list: >> Kernel: >> 1) Add new IP_FW_TABLE_XGETCFG / IP_FW_TABLE_XSETCFG opcodes to permit >> table reconfiguration >> 2) Tables data is now opaque to main ipfw code: use 1 pointer in first >> ip_fw_chain cache line for lookups and another one for config state. >> 3) Do not assume radix is the one and only lookup mechasim for doing >> lookups (more changes following) >> 4) Table data layout is changed to the following: >> +struct table_info { >> + void *state; /* IPv4 tables */ >> + void *xstate;/* extended tables */ >> + table_lookup_t *lookup;/* lookup function */ >> + struct table_config *cfg; /* Additional data, can be NULL */ >> +}; >> Array of size "table_max * sizeof(struct table_info)" is allocated on >> startup (very much like in current code in term of memory). >> 5) State holds any additional info table may need for configuration >> purposes and is allocated on demand. >> >> 6) By default, all tables are CIDR (IPv4+IPv6) and does not hold *cfg state. >> 7) State is allocated when: >> * table is referenced in some rules >> * type is non-default >> * table is named >> 8) Tables can be named and referenced by their names, but it is still >> needed to explicitly select table number. >> 8) Table references are now explicitly tracked by kernel checking if >> opcode lookup type and table type are the same >> >> 9) Do not assume tbl is uint16_t >> 10) Change locking model: use both IPFW and IPFW_UH locks, as main ipfw >> code does >> 11) While here, fix possible panic in case of adding new table entry >> while reallocating tables_max >> >> Some more on table types: >> +struct table_config { >> + uint8_t tabletype; /* lookup table types */ >> + uint8_t ftype; /* format table type */ >> + uint8_t atype; /* algorithm type */ >> + uint8_t spare0; >> + uint32_t refcnt; /* Number of references */ >> + uint32_t count; /* Number of records */ >> + struct table_info *ti; >> + TAILQ_ENTRY(table_config) next; /* namehash */ >> + char tablename[64]; /* table name */ >> +}; >> >> "tabletype" is basically type of the key we're looking for (e.g. >> IPv4/IPv6 address, interface name, port/uid, etc..). >> "ftype" is pure userland field helping to format keys in appropriate way >> (like shown DSCP values in hex or binary). >> "atype" permits to use different algorithm for the same lookup key ( >> currently not implemented, but planned). >> Good example can be CIDR table consisting only of host routes. >> >> User: >> Nothing changes for people using tables for IPv4/IPv6 address matching. >> New cmds: >> ipfw table type >> Changes table type to different one. Permitted IFF: >> * table is not referenced in ruleset >> * table is empy >> >> ipfw table name XXX >> Names (or renames) table. Not the name has to be unique. >> >> ipfw table flush >> (Not a new command, actually). >> Flushes all table records leaving configuration intact. >> >> ipfw table destroy >> Flushes table state AND configuration. >> Tables becomes unnamed IPFW_TABLE_CIDR one. >> >> >> Next changes: >> * Further rework add/del table entry to permit adding non-radix tables >> more easily >> * Change "iface" table type implementation to uint32_t iflist[65536] to >> permit O(1) interface matching >> * Add general u32 lookup method for dealing with ports/uids/jails/dscp >> and other such consumers >> >> >> >> I'm planning to commit this one (actually, a bit improved version) in >> the beginning of next week if no objections received. >> >> >> >> Index: sbin/ipfw/ipfw2.c >> =================================================================== >> --- sbin/ipfw/ipfw2.c (revision 266310) >> +++ sbin/ipfw/ipfw2.c (working copy) >> @@ -204,6 +204,12 @@ static struct _s_x limit_masks[] = { >> {NULL, 0} >> }; >> >> +static struct _s_x f_tabletypes[] = { >> + {"cidr", IPFW_TABLE_CIDR}, >> + {"iface", IPFW_TABLE_INTERFACE}, >> + {NULL, 0} >> +}; >> + >> /* >> * we use IPPROTO_ETHERTYPE as a fake protocol id to call the print routines >> * This is only used in this code. >> @@ -4124,6 +4130,9 @@ ipfw_flush(int force) >> >> static void table_list(uint16_t num, int need_header); >> static void table_fill_xentry(char *arg, ipfw_table_xentry *xent); >> +static void table_get_info(uint16_t num, char *name, int need_header); >> +static uint32_t table_get_num(char *name); >> +static int table_check_name(char *name); >> >> /* >> * Retrieve maximum number of tables supported by ipfw(4) module. >> @@ -4158,6 +4167,7 @@ ipfw_get_tables_max() >> * ipfw table N delete addr[/masklen] >> * ipfw table {N | all} flush >> * ipfw table {N | all} list >> + * ipfw table {M | all} info >> */ >> void >> ipfw_table_handler(int ac, char *av[]) >> @@ -4167,6 +4177,7 @@ ipfw_table_handler(int ac, char *av[]) >> int is_all; >> uint32_t a; >> uint32_t tables_max; >> + int l, type; >> >> tables_max = ipfw_get_tables_max(); >> >> @@ -4181,13 +4192,18 @@ ipfw_table_handler(int ac, char *av[]) >> xent.tbl = 0; >> is_all = 1; >> ac--; av++; >> - } else >> - errx(EX_USAGE, "table number or 'all' keyword required"); >> + } else { >> + xent.tbl = table_get_num(*av); >> + is_all = 0; >> + ac--; av++; >> + } >> + >> if (xent.tbl >= tables_max) >> errx(EX_USAGE, "The table number exceeds the maximum allowed " >> "value (%d)", tables_max - 1); >> NEED1("table needs command"); >> if (is_all && _substrcmp(*av, "list") != 0 >> + && _substrcmp(*av, "info") != 0 >> && _substrcmp(*av, "flush") != 0) >> errx(EX_USAGE, "table number required"); >> >> @@ -4243,6 +4259,59 @@ ipfw_table_handler(int ac, char *av[]) >> do { >> table_list(xent.tbl, is_all); >> } while (++xent.tbl < a); >> + } else if (_substrcmp(*av, "info") == 0) { >> + a = is_all ? tables_max : (uint32_t)(xent.tbl + 1); >> + do { >> + table_get_info(xent.tbl, NULL, is_all); >> + } while (++xent.tbl < a); >> + } else if (_substrcmp(*av, "type") == 0) { >> + ac--; av++; >> + if (!ac) >> + errx(EX_USAGE, "table type required"); >> + >> + if ((type = match_token(f_tabletypes, *av)) == -1) >> + errx(EX_DATAERR, "Unknown table type"); >> + >> + ipfw_xtable_cfg cfg; >> + >> + memset(&cfg, 0, sizeof(cfg)); >> + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; >> + cfg.tbl = xent.tbl; >> + cfg.mask = IPFW_XCFG_TYPE; >> + cfg.type = type; >> + l = sizeof(cfg); >> + >> + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { >> + /* >> + * XXX: Provide human-readable error here >> + * by using IP_FW_TABLE_XGETCFG >> + */ >> + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); >> + } >> + } else if (_substrcmp(*av, "name") == 0) { >> + ac--; av++; >> + if (!ac) >> + errx(EX_USAGE, "table name required"); >> + >> + if (table_check_name(*av) != 0) >> + errx(EX_USAGE, "bad table name"); >> + >> + ipfw_xtable_cfg cfg; >> + >> + memset(&cfg, 0, sizeof(cfg)); >> + cfg.opheader.opcode = IP_FW_TABLE_XSETCFG; >> + cfg.tbl = xent.tbl; >> + cfg.mask = IPFW_XCFG_NAME; >> + strlcpy(cfg.tablename, *av, sizeof(cfg.tablename)); >> + l = sizeof(cfg); >> + >> + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) { >> + /* >> + * XXX: Provide human-readable error here >> + * by using IP_FW_TABLE_XGETCFG >> + */ >> + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XSETCFG)"); >> + } >> } else >> errx(EX_USAGE, "invalid table command %s", *av); >> } >> @@ -4423,3 +4492,107 @@ table_list(uint16_t num, int need_header) >> >> free(tbl); >> } >> + >> +static uint32_t >> +table_get_num(char *name) >> +{ >> + ipfw_xtable_cfg cfg; >> + socklen_t l; >> + >> + memset(&cfg, 0, sizeof(cfg)); >> + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; >> + l = sizeof(cfg); >> + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); >> + cfg.mask |= IPFW_XCFG_NAMEID; >> + >> + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) >> + err(EX_OSERR, "table name '%s' not found", name); >> + >> + return (cfg.tbl); >> +} >> + >> +static void >> +table_get_info(uint16_t num, char *name, int is_all) >> +{ >> + ipfw_xtable_cfg cfg; >> + socklen_t l; >> + char *t; >> + >> + memset(&cfg, 0, sizeof(cfg)); >> + cfg.opheader.opcode = IP_FW_TABLE_XGETCFG; >> + cfg.mask = IPFW_XCFG_NAME | IPFW_XCFG_TYPE | IPFW_XCFG_REFS | >> + IPFW_XCFG_CNT; >> + l = sizeof(cfg); >> + >> + if (name == NULL) { >> + cfg.tbl = num; >> + cfg.mask |= IPFW_XCFG_NUMID; >> + } else { >> + strlcpy(cfg.tablename, name, sizeof(cfg.tablename)); >> + cfg.mask |= IPFW_XCFG_NAMEID; >> + } >> + >> + if (do_cmd(IP_FW3, &cfg, (uintptr_t)&l) < 0) >> + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETCFG)"); >> + >> + if (is_all != 0) { >> + /* >> + * Skip unreferenced tables with default type >> + */ >> + if (cfg.type == IPFW_TABLE_CIDR && cfg.refcnt == 0) >> + return; >> + printf("---table(%d)---\n", num); >> + } >> + if (cfg.mask & IPFW_XCFG_NAME) >> + printf("Name: %s\n", cfg.tablename); >> + switch (cfg.type) { >> + case IPFW_TABLE_CIDR: >> + t = "cidr"; >> + break; >> + case IPFW_TABLE_INTERFACE: >> + t = "cidr"; >> + break; >> +/* >> + case IPFW_TABLE_U32: >> + t = "u32"; >> + break; >> + case IPFW_TABLE_U16: >> + t = "u16"; >> + break; >> +*/ >> + default: >> + t = "unknown"; >> + break; >> + }; >> + >> + printf("Type: %s\tFormatting: %s\n", t, "default"); >> + printf("References: %u\tEntries: %u\n", cfg.refcnt, cfg.count); >> +} >> + >> +static int >> +table_check_name(char *tablename) >> +{ >> + int c, i, l; >> + >> + /* >> + * Check if tablename is null-terminated and contains >> + * valid symbols only. Valid mask is: >> + * [a-zA-Z\-\.][a-zA-Z0-9\-_\.]{0,62} >> + */ >> + l = strlen(tablename); >> + if (l == 0 || l >= 64) >> + return (EINVAL); >> + /* Restrict first symbol to non-digit */ >> + if (isdigit(tablename[0])) >> + return (EINVAL); >> + for (i = 0; i < l; i++) { >> + c = tablename[i]; >> + if (isalpha(c) || isdigit(c) || c == '_' || >> + c == '-' || c == '.') >> + continue; >> + return (EINVAL); >> + } >> + >> + return (0); >> +} >> + >> Index: sys/netinet/ip_fw.h >> =================================================================== >> --- sys/netinet/ip_fw.h (revision 266310) >> +++ sys/netinet/ip_fw.h (working copy) >> @@ -74,6 +74,8 @@ typedef struct _ip_fw3_opheader { >> #define IP_FW_TABLE_XDEL 87 /* delete entry */ >> #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ >> #define IP_FW_TABLE_XLIST 89 /* list table contents */ >> +#define IP_FW_TABLE_XGETCFG 90 /* configure table */ >> +#define IP_FW_TABLE_XSETCFG 91 /* configure table */ >> >> /* >> * The kernel representation of ipfw rules is made of a list of >> @@ -632,7 +634,7 @@ typedef struct _ipfw_table { >> } ipfw_table; >> >> typedef struct _ipfw_xtable { >> - ip_fw3_opheader opheader; /* eXtended tables are controlled via IP_FW3 */ >> + ip_fw3_opheader opheader; /* IP_FW3 opcode */ >> uint32_t size; /* size of entries in bytes */ >> uint32_t cnt; /* # of entries */ >> uint16_t tbl; /* table number */ >> @@ -640,4 +642,31 @@ typedef struct _ipfw_xtable { >> ipfw_table_xentry xent[0]; /* entries */ >> } ipfw_xtable; >> >> +#define IPFW_XCFG_NAME 0x01 /* get/set name */ >> +#define IPFW_XCFG_TYPE 0x02 /* get/set type */ >> +#define IPFW_XCFG_FTYPE 0x04 /* get/set format type */ >> +#define IPFW_XCFG_REFS 0x08 /* get/set number of references */ >> +#define IPFW_XCFG_CNT 0x10 /* ge number of records */ >> +#define IPFW_XCFG_NUMID 0x20 /* table identified by num */ >> +#define IPFW_XCFG_NAMEID 0x40 /* table identified by name */ >> +typedef struct _ipfw_xtable_cfg { >> + ip_fw3_opheader opheader; /* IP_FW3 opcode */ >> + uint32_t size; /* size of structure in bytes */ >> + uint32_t mask; /* data mask to set/retrieve */ >> + uint32_t tlvmask; /* data mask to set/retrieve */ >> + uint32_t tbl; /* table id */ >> + uint16_t type; /* table type */ >> + uint16_t ftype; /* table format type */ >> + uint32_t count; /* number of records */ >> + uint32_t refcnt; /* number of references */ >> + char tablename[64]; /* table name */ >> +} ipfw_xtable_cfg; >> + >> +typedef struct _ipfw_xtable_tlv { >> + uint16_t type; >> + uint16_t length; >> +} ipfw_xtable_tlv; >> + >> + >> + >> #endif /* _IPFW2_H */ >> Index: sys/netpfil/ipfw/ip_fw2.c >> =================================================================== >> --- sys/netpfil/ipfw/ip_fw2.c (revision 266306) >> +++ sys/netpfil/ipfw/ip_fw2.c (working copy) >> @@ -352,15 +352,16 @@ tcpopts_match(struct tcphdr *tcp, ipfw_insn *cmd) >> } >> >> static int >> -iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, uint32_t *tablearg) >> +iface_match(struct ifnet *ifp, ipfw_insn_if *cmd, struct ip_fw_chain *chain, >> + uint32_t *tablearg) >> { >> if (ifp == NULL) /* no iface with this packet, match fails */ >> return 0; >> /* Check by name or by IP address */ >> if (cmd->name[0] != '\0') { /* match by name */ >> if (cmd->name[0] == '\1') /* use tablearg to match */ >> - return ipfw_lookup_table_extended(chain, cmd->p.glob, >> - ifp->if_xname, tablearg, IPFW_TABLE_INTERFACE); >> + return (ipfw_lookup_table(chain, cmd->p.glob, IFNAMSIZ, >> + ifp, tablearg)); >> /* Check name */ >> if (cmd->p.glob) { >> if (fnmatch(cmd->name, ifp->if_xname, 0) == 0) >> @@ -1492,7 +1493,7 @@ do { \ >> break; >> } >> match = ipfw_lookup_table(chain, >> - cmd->arg1, key, &v); >> + cmd->arg1, sizeof(uint32_t), &key, &v); >> if (!match) >> break; >> if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) >> @@ -1504,9 +1505,9 @@ do { \ >> uint32_t v = 0; >> void *pkey = (cmd->opcode == O_IP_DST_LOOKUP) ? >> &args->f_id.dst_ip6: &args->f_id.src_ip6; >> - match = ipfw_lookup_table_extended(chain, >> - cmd->arg1, pkey, &v, >> - IPFW_TABLE_CIDR); >> + match = ipfw_lookup_table(chain, >> + cmd->arg1, sizeof(uint32_t), >> + pkey, &v); >> if (cmdlen == F_INSN_SIZE(ipfw_insn_u32)) >> match = ((ipfw_insn_u32 *)cmd)->d[0] == v; >> if (match) >> Index: sys/netpfil/ipfw/ip_fw_private.h >> =================================================================== >> --- sys/netpfil/ipfw/ip_fw_private.h (revision 266306) >> +++ sys/netpfil/ipfw/ip_fw_private.h (working copy) >> @@ -217,9 +217,7 @@ struct ip_fw_chain { >> uint32_t id; /* ruleset id */ >> int n_rules; /* number of static rules */ >> LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ >> - struct radix_node_head **tables; /* IPv4 tables */ >> - struct radix_node_head **xtables; /* extended tables */ >> - uint8_t *tabletype; /* Array of table types */ >> + void *tablestate; /* Array of table data */ >> #if defined( __linux__ ) || defined( _WIN32 ) >> spinlock_t rwmtx; >> #else >> @@ -229,6 +227,7 @@ struct ip_fw_chain { >> uint32_t gencnt; /* NAT generation count */ >> struct ip_fw *reap; /* list of rules to reap */ >> struct ip_fw *default_rule; >> + void *tablecfg; /* tables module configuration */ >> #if defined( __linux__ ) || defined( _WIN32 ) >> spinlock_t uh_lock; >> #else >> @@ -303,9 +302,8 @@ int ipfw_chk(struct ip_fw_args *args); >> void ipfw_reap_rules(struct ip_fw *head); >> >> /* In ip_fw_table.c */ >> -struct radix_node; >> -int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, >> - uint32_t *val); >> +int ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t keylen, >> + void *key, uint32_t *val); >> int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, >> uint32_t *val, int type); >> int ipfw_init_tables(struct ip_fw_chain *ch); >> @@ -316,11 +314,18 @@ int ipfw_add_table_entry(struct ip_fw_chain *ch, u >> int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, >> uint8_t plen, uint8_t mlen, uint8_t type); >> int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); >> -int ipfw_dump_table_entry(struct radix_node *rn, void *arg); >> int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); >> -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); >> +int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, >> + uint32_t *cnt); >> int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); >> int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); >> +int ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl); >> +int ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype); >> +int ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl); >> +int ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, >> + size_t *sz); >> +int ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, >> + size_t *sz); >> >> /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ >> >> Index: sys/netpfil/ipfw/ip_fw_sockopt.c >> =================================================================== >> --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266306) >> +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy) >> @@ -146,6 +146,119 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * >> } >> >> /* >> + * In @del==0 mode: >> + * Checks is opcode is referencing table of appropriate type. >> + * Adds reference count for found table if true. >> + * In del==1 mode >> + * Decrements refcount for given table. >> + * >> + * Returns 0 on success and appropriate error code otherwise. >> + */ >> +static int >> +bind_tables(struct ip_fw_chain *chain, struct ip_fw *rule, int del) >> +{ >> + int cmdlen, error, ftype, l, skip, type, v; >> + ipfw_insn *cmd; >> + ipfw_insn_if *cmdif; >> + uint32_t tbl; >> + >> + l = rule->cmd_len; >> + cmd = rule->cmd; >> + cmdlen = 0; >> + tbl = 0; >> + type = 0; >> + ftype = 0; >> + >> + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { >> + cmdlen = F_LEN(cmd); >> + skip = 0; >> + >> + switch (cmd->opcode) { >> + case O_IP_SRC_LOOKUP: >> + case O_IP_DST_LOOKUP: >> + /* Basic IPv4/IPv6 or u32 lookups */ >> + tbl = cmd->arg1; >> + /* Assume CIDR by default */ >> + type = IPFW_TABLE_CIDR; >> + ftype = 0; >> + >> + if (cmdlen > F_INSN_SIZE(ipfw_insn_u32)) { >> + /* generic lookup. The key must be >> + * in 32bit big-endian format. >> + */ >> + v = ((ipfw_insn_u32 *)cmd)->d[1]; >> + switch (v) { >> + case 0: >> + case 1: >> + /* IPv4 src/dst */ >> + break; >> + case 2: >> + case 3: >> + /* src/dst port */ >> + //type = IPFW_TABLE_U16; >> + break; >> + case 4: >> + /* uid/gid */ >> + //type = IPFW_TABLE_U32; >> + case 5: >> + //type = IPFW_TABLE_U32; >> + /* jid */ >> + case 6: >> + //type = IPFW_TABLE_U16; >> + /* dscp */ >> + break; >> + } >> + } >> + break; >> + case O_XMIT: >> + case O_RECV: >> + case O_VIA: >> + /* Interface table, possibly */ >> + cmdif = (ipfw_insn_if *)cmd; >> + if (cmdif->name[0] != '\1') { >> + skip = 1; >> + break; >> + } >> + >> + type = IPFW_TABLE_INTERFACE; >> + ftype = 0; >> + tbl = cmdif->p.glob; >> + break; >> + default: >> + skip = 1; >> + } >> + >> + if (skip != 0) >> + continue; >> + >> + /* ref/unref given table */ >> + if (del != 0) >> + error = ipfw_unref_table(chain, tbl); >> + else >> + error = ipfw_ref_table(chain, tbl, type, ftype); >> + >> + if (error != 0) >> + return (error); >> + } >> + >> + return (0); >> +} >> + >> +/* >> + * Removes table bindings for every rule in rule chain @head. >> + */ >> +static void >> +unbind_tables(struct ip_fw_chain *chain, struct ip_fw *head) >> +{ >> + struct ip_fw *rule; >> + >> + while ((rule = head) != NULL) { >> + head = head->x_next; >> + bind_tables(chain, rule, 1); >> + } >> +} >> + >> +/* >> * Add a new rule to the list. Copy the rule into a malloc'ed area, then >> * possibly create a rule number and add the rule to the list. >> * Update the rule_number in the input struct so the caller knows it as well. >> @@ -157,6 +270,7 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip >> { >> struct ip_fw *rule; >> int i, l, insert_before; >> + int error; >> struct ip_fw **map; /* the new array of pointers */ >> >> if (chain->map == NULL || input_rule->rulenum > IPFW_DEFAULT_RULE - 1) >> @@ -168,9 +282,16 @@ ipfw_add_rule(struct ip_fw_chain *chain, struct ip >> map = get_map(chain, 1, 0 /* not locked */); >> if (map == NULL) { >> free(rule, M_IPFW); >> - return ENOSPC; >> + return (ENOSPC); >> } >> >> + /* Reference tables, if any */ >> + if ((error = bind_tables(chain, input_rule, 0)) != 0) { >> + IPFW_UH_WUNLOCK(chain); >> + free(rule, M_IPFW); >> + return (error); >> + } >> + >> bcopy(input_rule, rule, l); >> /* clear fields not settable from userland */ >> rule->x_next = NULL; >> @@ -421,6 +542,7 @@ del_entry(struct ip_fw_chain *chain, uint32_t arg) >> >> rule = chain->reap; >> chain->reap = NULL; >> + unbind_tables(chain, rule); >> IPFW_UH_WUNLOCK(chain); >> ipfw_reap_rules(rule); >> if (map) >> @@ -934,6 +1056,7 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf >> >> >> #define IP_FW3_OPLENGTH(x) ((x)->sopt_valsize - sizeof(ip_fw3_opheader)) >> + >> /** >> * {set|get}sockopt parser. >> */ >> @@ -1113,7 +1236,7 @@ ipfw_ctl(struct sockopt *sopt) >> sopt->sopt_name == IP_FW_RESETLOG); >> break; >> >> - /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ >> + /*--- TABLE locking is described in ip_fw_table.c ---*/ >> case IP_FW_TABLE_ADD: >> { >> ipfw_table_entry ent; >> @@ -1237,7 +1360,7 @@ ipfw_ctl(struct sockopt *sopt) >> tbl = (uint32_t *)(op3 + 1); >> >> IPFW_RLOCK(chain); >> - error = ipfw_count_xtable(chain, *tbl, tbl); >> + error = ipfw_count_xtable(chain, *tbl, tbl, NULL); >> IPFW_RUNLOCK(chain); >> if (error) >> break; >> @@ -1281,6 +1404,39 @@ ipfw_ctl(struct sockopt *sopt) >> } >> break; >> >> + case IP_FW_TABLE_XGETCFG: /* IP_FW3 */ >> + case IP_FW_TABLE_XSETCFG: /* IP_FW3 */ >> + { >> + ipfw_xtable_cfg *xcfg; >> + >> + if (sopt->sopt_valsize < sizeof(xcfg)) { >> + error = EINVAL; >> + break; >> + } >> + if ((size = sopt->sopt_valsize) > RULE_MAXSIZE) { >> + error = E2BIG; >> + break; >> + } >> + >> + xcfg = malloc(size, M_TEMP, M_WAITOK); >> + error = sooptcopyin(sopt, xcfg, size, sizeof(*xcfg)); >> + if (error != 0) { >> + free(xcfg, M_TEMP); >> + break; >> + } >> + >> + if (opt == IP_FW_TABLE_XGETCFG) { >> + error = ipfw_getconfig_table(chain, xcfg, &size); >> + if (error == 0) >> + error = sooptcopyout(sopt, xcfg, size); >> + } else >> + error = ipfw_setconfig_table(chain, xcfg, &size); >> + >> + free(xcfg, M_TEMP); >> + } >> + break; >> + >> + >> /*--- NAT operations are protected by the IPFW_LOCK ---*/ >> case IP_FW_NAT_CFG: >> if (IPFW_NAT_LOADED) >> Index: sys/netpfil/ipfw/ip_fw_table.c >> =================================================================== >> --- sys/netpfil/ipfw/ip_fw_table.c (revision 266310) >> +++ sys/netpfil/ipfw/ip_fw_table.c (working copy) >> @@ -35,8 +35,13 @@ __FBSDID("$FreeBSD$"); >> * As a degenerate case we can interpret keys as 32-bit integers >> * (with a /32 mask). >> * >> - * The table is protected by the IPFW lock even for manipulation coming >> - * from userland, because operations are typically fast. >> + * Locking resembles ipfw rules model: >> + * changes in: table entries, table count and table_info are protected by holding >> + * both UH and main write locks. >> + * changes in table_config struture are protected by UH lock. >> + * >> + * Userland readers should use UH lock if possible. >> + * >> */ >> >> #include "opt_ipfw.h" >> @@ -48,12 +53,14 @@ __FBSDID("$FreeBSD$"); >> >> #include >> #include >> +#include >> #include >> #include >> #include >> #include >> #include >> #include >> +#include >> #include /* ip_fw.h requires IFNAMSIZ */ >> #include >> #include >> @@ -101,6 +108,65 @@ struct table_xentry { >> }; >> >> /* >> + * filled for non-CIDR or named tables >> + * >> + * Table has the following `type` concepts: >> + * >> + * `tabletype` represents lookup key type (cidr, ifp, uid, etc..) >> + * `ftype` is pure userland field helping to properly format table data >> + * `atype` represents exact lookup algorithm for given tabletype. >> + * For example, we can use more efficient search schemes if we plan >> + * to use some specific table for storing host-routes only. >> + * >> + */ >> +struct table_config { >> + uint8_t tabletype; /* lookup table types */ >> + uint8_t ftype; /* format table type */ >> + uint8_t atype; /* algorith type */ >> + uint8_t spare0; >> + uint32_t refcnt; /* Number of references */ >> + uint32_t count; /* Number of records */ >> + struct table_info *ti; >> + TAILQ_ENTRY(table_config) next; /* namehash */ >> + char tablename[64]; /* table name */ >> +}; >> + >> +typedef int (table_lookup_t)(void *state, void *xstate, void *key, >> + uint32_t keylen, uint32_t *val); >> + >> +struct table_info { >> + void *state; /* IPv4 tables */ >> + void *xstate;/* extended tables */ >> + table_lookup_t *lookup;/* lookup function */ >> + struct table_config *cfg; /* Additional data, can be NULL */ >> +}; >> + >> +static int lookup_cidr(void *, void *, void *, uint32_t, uint32_t *); >> +static int lookup_iface(void *, void *, void *, uint32_t, uint32_t *); >> + >> +static table_lookup_t (*tablehandlers[]) = { >> + NULL, /* type 0 unused */ >> + lookup_cidr, /* IPFW_TABLE_CIDR */ >> + lookup_iface, /* IPFW_TABLE_INTERFACE */ >> +}; >> + >> +#define TABLETYPE(t) ((t)->cfg != NULL ? (t)->cfg->tabletype:IPFW_TABLE_CIDR) >> +#define TABLEFTYPE(ti) ((ti)->cfg != NULL ? (ti)->cfg->ftype : 0) >> +#define TABLEREFS(ti) ((ti)->cfg != NULL ? (ti)->cfg->refcnt : 0) >> + >> +#define TABLENAME_HASH_SIZE 32 >> +struct tables_config { >> + TAILQ_HEAD(, table_config) thash[TABLENAME_HASH_SIZE]; >> +}; >> + >> +#define TABLENAME_HASH(n) (fnv_32_str(n,FNV1_32_INIT)%TABLENAME_HASH_SIZE) >> +static struct table_info *find_table(struct ip_fw_chain *ch, char *tablename); >> + >> +static void flush_table(struct ip_fw_chain *ch, struct table_info *ti); >> +static int change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, >> + struct table_info *ti2, table_lookup_t lookup); >> + >> +/* >> * The radix code expects addr and mask to be array of bytes, >> * with the first byte being the length of the array. rn_inithead >> * is called with the offset in bits of the lookup key within the >> @@ -145,13 +211,19 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 >> struct radix_node *rn; >> in_addr_t addr; >> int offset; >> - void *ent_ptr; >> + uintptr_t state_off; >> + void *ent_ptr, *tablestate; >> struct sockaddr *addr_ptr, *mask_ptr; >> + struct table_info *ti; >> + struct table_config *cfg; >> char c; >> >> if (tbl >= V_fw_tables_max) >> return (EINVAL); >> >> + tablestate = ch->tablestate; >> + ti = &((struct table_info *)tablestate)[tbl]; >> + >> switch (type) { >> case IPFW_TABLE_CIDR: >> if (plen == sizeof(in_addr_t)) { >> @@ -170,7 +242,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 >> addr = *((in_addr_t *)paddr); >> ent->addr.sin_addr.s_addr = addr & ent->mask.sin_addr.s_addr; >> /* Set pointers */ >> - rnh_ptr = &ch->tables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->state; >> ent_ptr = ent; >> addr_ptr = (struct sockaddr *)&ent->addr; >> mask_ptr = (struct sockaddr *)&ent->mask; >> @@ -191,7 +263,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 >> memcpy(&xent->a.addr6.sin6_addr, paddr, sizeof(struct in6_addr)); >> APPLY_MASK(&xent->a.addr6.sin6_addr, &xent->m.mask6.sin6_addr); >> /* Set pointers */ >> - rnh_ptr = &ch->xtables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->xstate; >> ent_ptr = xent; >> addr_ptr = (struct sockaddr *)&xent->a.addr6; >> mask_ptr = (struct sockaddr *)&xent->m.mask6; >> @@ -227,7 +299,7 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 >> mask_ptr = (struct sockaddr *)&xent->m.ifmask; >> #endif >> /* Set pointers */ >> - rnh_ptr = &ch->xtables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->xstate; >> ent_ptr = xent; >> addr_ptr = (struct sockaddr *)&xent->a.iface; >> mask_ptr = NULL; >> @@ -237,48 +309,67 @@ ipfw_add_table_entry(struct ip_fw_chain *ch, uint1 >> return (EINVAL); >> } >> >> + IPFW_UH_WLOCK(ch); >> IPFW_WLOCK(ch); >> >> + /* >> + * Check if tablestate was reallocated. >> + */ >> + if (ch->tablestate != tablestate) { >> + state_off = (uintptr_t)ti - (uintptr_t)tablestate; >> + ti = (struct table_info *) >> + (state_off + (uintptr_t)ch->tablestate); >> + >> + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; >> + rnh_ptr = (struct radix_node_head **) >> + (state_off + (uintptr_t)ch->tablestate); >> + } >> + >> /* Check if tabletype is valid */ >> - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { >> + if (TABLETYPE(ti) != type) { >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> free(ent_ptr, M_IPFW_TBL); >> - return (EINVAL); >> + return (EFTYPE); >> } >> >> /* Check if radix tree exists */ >> if ((rnh = *rnh_ptr) == NULL) { >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> /* Create radix for a new table */ >> if (!rn_inithead((void **)&rnh, offset)) { >> free(ent_ptr, M_IPFW_TBL); >> return (ENOMEM); >> } >> >> + IPFW_UH_WLOCK(ch); >> IPFW_WLOCK(ch); >> if (*rnh_ptr != NULL) { >> /* Tree is already attached by other thread */ >> rn_detachhead((void **)&rnh); >> rnh = *rnh_ptr; >> /* Check table type another time */ >> - if (ch->tabletype[tbl] != type) { >> + if (TABLETYPE(ti) != type) { >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> free(ent_ptr, M_IPFW_TBL); >> - return (EINVAL); >> + return (EFTYPE); >> } >> } else { >> + /* new trie */ >> *rnh_ptr = rnh; >> - /* >> - * Set table type. It can be set already >> - * (if we have IPv6-only table) but setting >> - * it another time does not hurt >> - */ >> - ch->tabletype[tbl] = type; >> } >> } >> >> rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); >> + >> + /* Maintain number of records */ >> + if (rn != NULL && (cfg = ti->cfg) != NULL) >> + cfg->count++; >> + >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> >> if (rn == NULL) { >> free(ent_ptr, M_IPFW_TBL); >> @@ -296,11 +387,18 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> in_addr_t addr; >> struct sockaddr_in sa, mask; >> struct sockaddr *sa_ptr, *mask_ptr; >> + struct table_info *ti; >> + struct table_config *cfg; >> + void *tablestate; >> + uintptr_t state_off; >> char c; >> >> if (tbl >= V_fw_tables_max) >> return (EINVAL); >> >> + tablestate = ch->tablestate; >> + ti = &((struct table_info *)tablestate)[tbl]; >> + >> switch (type) { >> case IPFW_TABLE_CIDR: >> if (plen == sizeof(in_addr_t)) { >> @@ -310,7 +408,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); >> addr = *((in_addr_t *)paddr); >> sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; >> - rnh_ptr = &ch->tables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->state; >> sa_ptr = (struct sockaddr *)&sa; >> mask_ptr = (struct sockaddr *)&mask; >> #ifdef INET6 >> @@ -327,7 +425,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> ipv6_writemask(&mask6.sin6_addr, mlen); >> memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); >> APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); >> - rnh_ptr = &ch->xtables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->xstate; >> sa_ptr = (struct sockaddr *)&sa6; >> mask_ptr = (struct sockaddr *)&mask6; >> #endif >> @@ -362,7 +460,7 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> mask_ptr = NULL; >> memcpy(ifname.ifname, paddr, mlen); >> /* Set pointers */ >> - rnh_ptr = &ch->xtables[tbl]; >> + rnh_ptr = (struct radix_node_head **)&ti->xstate; >> sa_ptr = (struct sockaddr *)&ifname; >> >> break; >> @@ -371,19 +469,42 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> return (EINVAL); >> } >> >> + IPFW_UH_WLOCK(ch); >> IPFW_WLOCK(ch); >> + >> + /* >> + * Check if tablestate was reallocated. >> + */ >> + if (ch->tablestate != tablestate) { >> + state_off = (uintptr_t)ti - (uintptr_t)tablestate; >> + ti = (struct table_info *) >> + (state_off + (uintptr_t)ch->tablestate); >> + >> + state_off = (uintptr_t)rnh_ptr - (uintptr_t)tablestate; >> + rnh_ptr = (struct radix_node_head **) >> + (state_off + (uintptr_t)ch->tablestate); >> + } >> + >> if ((rnh = *rnh_ptr) == NULL) { >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> return (ESRCH); >> } >> >> - if (ch->tabletype[tbl] != type) { >> + if (TABLETYPE(ti) != type) { >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> return (EINVAL); >> } >> >> ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, rnh); >> + >> + /* Maintain number of records */ >> + if (ent != NULL && (cfg = ti->cfg) != NULL) >> + cfg->count--; >> + >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> >> if (ent == NULL) >> return (ESRCH); >> @@ -392,7 +513,381 @@ ipfw_del_table_entry(struct ip_fw_chain *ch, uint1 >> return (0); >> } >> >> +/* >> + * binds newly-created config to given table >> + */ >> static int >> +bind_config(struct ip_fw_chain *ch, uint32_t tbl, struct table_info *ti, >> + struct table_config *cfg) >> +{ >> + int error; >> + uint32_t sz, cnt; >> + >> + /* Set defaults */ >> + cfg->tabletype = IPFW_TABLE_CIDR; >> + >> + /* count current number of entries */ >> + if ((error = ipfw_count_xtable(ch, tbl, &sz, &cnt)) != 0) >> + return (error); >> + cfg->count = cnt; >> + >> + ti->cfg = cfg; >> + cfg->ti = ti; >> + >> + return (0); >> +} >> + >> +static struct table_info * >> +find_table(struct ip_fw_chain *ch, char *tablename) >> +{ >> + struct tables_config *tc; >> + struct table_config *cfg; >> + int hash; >> + >> + hash = TABLENAME_HASH(tablename); >> + tc = (struct tables_config *)ch->tablecfg; >> + >> + TAILQ_FOREACH(cfg, &tc->thash[hash], next) { >> + if (strcmp(cfg->tablename, tablename) == 0) >> + return (cfg->ti); >> + } >> + >> + return (NULL); >> +} >> + >> +/* >> + * Fills in supplied buffer with table configuration info. >> + */ >> +int >> +ipfw_getconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) >> +{ >> + struct table_info *ti; >> + struct table_config *cfg; >> + uint32_t tbl, tsz, cnt; >> + >> + tbl = xcfg->tbl; >> + >> + IPFW_UH_RLOCK(ch); >> + >> + if (xcfg->mask & IPFW_XCFG_NUMID) { >> + /* Table is identified by number */ >> + tbl = xcfg->tbl; >> + >> + if (tbl >= V_fw_tables_max) { >> + IPFW_UH_RUNLOCK(ch); >> + return (EINVAL); >> + } >> + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { >> + /* Let's try to find table by name */ >> + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); >> + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { >> + IPFW_UH_RUNLOCK(ch); >> + return (EINVAL); >> + } >> + >> + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { >> + IPFW_UH_RUNLOCK(ch); >> + return (ESRCH); >> + } >> + >> + /* Guess table number based on offset */ >> + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); >> + } else { >> + IPFW_UH_RUNLOCK(ch); >> + return (ESRCH); >> + } >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + cfg = ti->cfg; >> + >> + /* Save table id anywat */ >> + xcfg->tbl = tbl; >> + >> + if ((xcfg->mask & IPFW_XCFG_NAME) != 0 ) { >> + if (cfg == NULL || cfg->tablename == NULL) { >> + xcfg->tablename[0] = '\0'; >> + xcfg->mask &= ~IPFW_XCFG_NAME; >> + } else >> + strlcpy(xcfg->tablename, cfg->tablename, >> + sizeof(cfg->tablename)); >> + } >> + >> + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) >> + xcfg->type = TABLETYPE(ti); >> + >> + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) >> + xcfg->ftype = TABLEFTYPE(ti); >> + >> + if ((xcfg->mask & IPFW_XCFG_REFS) != 0) >> + xcfg->refcnt = TABLEREFS(ti); >> + >> + if ((xcfg->mask & IPFW_XCFG_CNT) != 0) { >> + /* >> + * Use items count from cfg, if it exists. >> + * Otherwise, calculate manually. >> + */ >> + if (cfg != NULL) >> + xcfg->count = cfg->count; >> + else { >> + IPFW_RLOCK(ch); >> + ipfw_count_xtable(ch, tbl, &tsz, &cnt); >> + IPFW_RUNLOCK(ch); >> + xcfg->count = cnt; >> + } >> + } >> + >> + IPFW_UH_RUNLOCK(ch); >> + >> + return (0); >> +} >> + >> +/* >> + * Fills in supplied buffer with table configuration info. >> + */ >> +int >> +ipfw_setconfig_table(struct ip_fw_chain *ch, ipfw_xtable_cfg *xcfg, size_t *sz) >> +{ >> + struct table_info *ti, ti_storage, *ti2; >> + struct table_config *cfg; >> + uint32_t tbl; >> + int l; >> + int error; >> + >> + tbl = xcfg->tbl; >> + >> + /* Do some preliminary checking */ >> + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { >> + if (xcfg->type > IPFW_TABLE_MAXTYPE || xcfg->type == 0) >> + return (EINVAL); >> + } >> + >> + /* >> + * Check if tablename is null-terminated. >> + * More fine-grained checks should be done by userland. >> + */ >> + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { >> + l = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); >> + if (l == sizeof(xcfg->tablename) || l == 0) >> + return (EINVAL); >> + } >> + >> + /* Checl if we need to allocate config structure */ >> + IPFW_UH_RLOCK(ch); >> + >> + if (xcfg->mask & IPFW_XCFG_NUMID) { >> + /* Table is identified by number */ >> + tbl = xcfg->tbl; >> + >> + if (tbl >= V_fw_tables_max) { >> + IPFW_UH_RUNLOCK(ch); >> + return (EINVAL); >> + } >> + } else if (xcfg->mask & IPFW_XCFG_NAMEID) { >> + /* Let's try to find table by name */ >> + tbl = strnlen(xcfg->tablename, sizeof(xcfg->tablename)); >> + if (tbl == 0 || tbl == sizeof(xcfg->tablename)) { >> + IPFW_UH_RUNLOCK(ch); >> + return (EINVAL); >> + } >> + >> + if ((ti = find_table(ch, xcfg->tablename)) == NULL) { >> + IPFW_UH_RUNLOCK(ch); >> + return (ESRCH); >> + } >> + >> + /* Guess table number based on offset */ >> + tbl = ((uintptr_t)ti - (uintptr_t)ch->tablestate) / sizeof(*ti); >> + } >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + cfg = ti->cfg; >> + IPFW_UH_RUNLOCK(ch); >> + >> + /* >> + * Allocate new config structure if needed. >> + * cfg represents pointer to new structure or NULL >> + */ >> + if (cfg == NULL) >> + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_WAITOK | M_ZERO); >> + else >> + cfg = NULL; >> + >> + IPFW_UH_WLOCK(ch); >> + /* >> + * We need to check another time since there is probability that >> + * V_fw_tables_max was changed or ti->cfg was destroyed >> + */ >> + >> + if (tbl >= V_fw_tables_max) { >> + IPFW_UH_WUNLOCK(ch); >> + return (EINVAL); >> + } >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + if (ti->cfg == NULL) { >> + if (cfg == NULL) { >> + /* destroy_table() had happened before IPFW_UH_WLOCK */ >> + cfg = malloc(sizeof(*cfg), M_IPFW_TBL, M_NOWAIT|M_ZERO); >> + if (cfg == NULL) { >> + IPFW_UH_WUNLOCK(ch); >> + return (ENOMEM); >> + } >> + } >> + >> + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { >> + IPFW_UH_WUNLOCK(ch); >> + free(cfg, M_IPFW_TBL); >> + return (error); >> + } >> + } else if (cfg != NULL) { >> + /* >> + * ti->cfg has been allocated by other thread. >> + * Free our allocation. >> + */ >> + free(cfg, M_IPFW_TBL); >> + } >> + >> + cfg = ti->cfg; >> + >> + /* >> + * Pretend to be atomic: check everything before doing actual job >> + */ >> + error = 0; >> + >> + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { >> + /* >> + * We can set type if >> + * 1) no one hold any references to our table >> + * 2) we have no records in given table >> + */ >> + >> + if ((cfg->tabletype != xcfg->type) && (cfg->refcnt > 0) && >> + (cfg->count > 0)) >> + error = EBUSY; >> + } >> + >> + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { >> + /* Check if new table name exists */ >> + if (find_table(ch, xcfg->tablename) != NULL) >> + error = EEXIST; >> + } >> + >> + if (error != 0) { >> + IPFW_UH_WUNLOCK(ch); >> + return (error); >> + } >> + >> + /* Everything checked, let's get the job done */ >> + ti2 = NULL; >> + >> + if ((xcfg->mask & IPFW_XCFG_TYPE) != 0) { >> + /* we need to change table_info here */ >> + IPFW_WLOCK(ch); >> + cfg->tabletype = xcfg->type; >> + if (change_table_handler(ch, ti, &ti_storage, >> + tablehandlers[cfg->tabletype]) != 0) >> + ti2 = &ti_storage; >> + IPFW_WUNLOCK(ch); >> + } >> + >> + if ((xcfg->mask & IPFW_XCFG_NAME) != 0) { >> + int hash; >> + struct tables_config *tc; >> + tc = (struct tables_config *)ch->tablecfg; >> + >> + if (strlen(cfg->tablename) != 0) { >> + /* unlink from old one */ >> + hash = TABLENAME_HASH(cfg->tablename); >> + TAILQ_REMOVE(&tc->thash[hash], cfg, next); >> + } >> + strlcpy(cfg->tablename, xcfg->tablename, >> + sizeof(cfg->tablename)); >> + /* link new one */ >> + hash = TABLENAME_HASH(cfg->tablename); >> + TAILQ_INSERT_HEAD(&tc->thash[hash], cfg, next); >> + } >> + >> + if ((xcfg->mask & IPFW_XCFG_FTYPE) != 0) >> + cfg->ftype = xcfg->ftype; >> + >> + IPFW_UH_WUNLOCK(ch); >> + >> + /* Free old table state if set */ >> + if (ti2 != NULL) >> + flush_table(ch, ti2); >> + >> + return (0); >> +} >> + >> + >> +/* >> + * Checks if given table's type is the same as @type. >> + * If true, increment @tbl refcount >> + */ >> +int >> +ipfw_ref_table(struct ip_fw_chain *ch, uint32_t tbl, int type, int ftype) >> +{ >> + struct table_info *ti; >> + struct table_config *cfg; >> + int error; >> + >> + IPFW_UH_WLOCK_ASSERT(ch); >> + >> + if (tbl >= V_fw_tables_max) >> + return (EINVAL); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + >> + if (TABLETYPE(ti) != type) >> + return (EFTYPE); >> + >> + /* Ignore ftype for now */ >> + >> + if (ti->cfg == NULL) { >> + /* >> + * Let's create confdata. >> + * XXX: we can fail here >> + */ >> + cfg = malloc(sizeof(struct table_config), M_IPFW_TBL, >> + M_NOWAIT | M_ZERO); >> + >> + if (cfg == NULL) >> + return (ENOMEM); >> + >> + if ((error = bind_config(ch, tbl, ti, cfg)) != 0) { >> + free(cfg, M_IPFW_TBL); >> + return (error); >> + } >> + } >> + >> + ti->cfg->refcnt++; >> + >> + return (0); >> +} >> + >> +int >> +ipfw_unref_table(struct ip_fw_chain *ch, uint32_t tbl) >> +{ >> + struct table_info *ti; >> + >> + IPFW_UH_WLOCK_ASSERT(ch); >> + >> + if (tbl >= V_fw_tables_max) >> + return (EINVAL); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + >> + KASSERT(ti->cfg != 0, ("ipfw: no config for table %d", tbl)); >> + >> + KASSERT(ti->cfg->refcnt > 0, ("ipfw: refcnt for table %d is %d", >> + tbl, ti->cfg->refcnt)); >> + >> + ti->cfg->refcnt--; >> + >> + return (0); >> +} >> + >> +static int >> flush_table_entry(struct radix_node *rn, void *arg) >> { >> struct radix_node_head * const rnh = arg; >> @@ -405,40 +900,138 @@ flush_table_entry(struct radix_node *rn, void *arg >> return (0); >> } >> >> +/* >> + * Flushes data from table state pointers. >> + * Frees pointers itself. >> + */ >> +static void >> +flush_table(struct ip_fw_chain *ch, struct table_info *ti) >> +{ >> + struct radix_node_head *rnh; >> + >> + if ((rnh = ti->state) != NULL) { >> + rnh->rnh_walktree(rnh, flush_table_entry, rnh); >> + rn_detachhead((void **)&rnh); >> + } >> + >> + if ((rnh = ti->xstate) != NULL) { >> + rnh->rnh_walktree(rnh, flush_table_entry, rnh); >> + rn_detachhead((void **)&rnh); >> + } >> +} >> + >> +/* >> + * change table handler saving previous state in @ti2. >> + * Returns 1 if hable had some state, 0 otherwise. >> + */ >> +static int >> +change_table_handler(struct ip_fw_chain *ch, struct table_info *ti, >> + struct table_info *ti2, table_lookup_t lookup) >> +{ >> + >> + if (ti->state == NULL && ti->state == NULL) { >> + /* No state. Set handler and return. */ >> + ti->lookup = lookup; >> + return (0); >> + } >> + >> + *ti2 = *ti; >> + >> + ti->state = NULL; >> + ti->xstate = NULL; >> + ti->lookup = lookup; >> + >> + return (1); >> +} >> + >> +/* >> + * flushes all data in given table leaving table type/naming >> + * intact. >> + */ >> int >> ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) >> { >> - struct radix_node_head *rnh, *xrnh; >> + struct table_info *ti, tti; >> + struct table_config *cfg; >> >> - if (tbl >= V_fw_tables_max) >> + IPFW_UH_WLOCK(ch); >> + IPFW_WLOCK(ch); >> + >> + if (tbl >= V_fw_tables_max) { >> + IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> return (EINVAL); >> + } >> >> - /* >> - * We free both (IPv4 and extended) radix trees and >> - * clear table type here to permit table to be reused >> - * for different type without module reload >> - */ >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + tti = *ti; >> + /* Remove state tables from main structure */ >> + ti->state = NULL; >> + ti->xstate = NULL; >> + if ((cfg = ti->cfg) != NULL) >> + cfg->count = 0; >> + IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> >> + flush_table(ch, &tti); >> + >> + return (0); >> +} >> + >> +static void >> +destroy_table(struct ip_fw_chain *ch, struct table_info *ti) >> +{ >> + struct table_config *cfg; >> + >> + flush_table(ch, ti); >> + >> + if ((cfg = ti->cfg) != NULL) { >> + /* Free configuration state */ >> + free(cfg, M_IPFW_TBL); >> + ti->cfg = NULL; >> + } >> + >> + /* Set lookup pointer back to CIDR */ >> + ti->lookup = lookup_cidr; >> +} >> + >> +/* >> + * Destroys given table iff no rules are referencing it. >> + * Flushes all data in tablestate, destroys any >> + * special configuration/naming associated with the table >> + * and sets its type (ti->cfg == NULL) back to CIDR. >> + */ >> +int >> +ipfw_destroy_table(struct ip_fw_chain *ch, uint32_t tbl) >> +{ >> + struct table_info *ti, tti; >> + struct table_config *cfg; >> + >> + IPFW_UH_WLOCK(ch); >> IPFW_WLOCK(ch); >> - /* Set IPv4 table pointer to zero */ >> - if ((rnh = ch->tables[tbl]) != NULL) >> - ch->tables[tbl] = NULL; >> - /* Set extended table pointer to zero */ >> - if ((xrnh = ch->xtables[tbl]) != NULL) >> - ch->xtables[tbl] = NULL; >> - /* Zero table type */ >> - ch->tabletype[tbl] = 0; >> - IPFW_WUNLOCK(ch); >> >> - if (rnh != NULL) { >> - rnh->rnh_walktree(rnh, flush_table_entry, rnh); >> - rn_detachhead((void **)&rnh); >> + if (tbl >= V_fw_tables_max) { >> + IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> + return (EINVAL); >> } >> >> - if (xrnh != NULL) { >> - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); >> - rn_detachhead((void **)&xrnh); >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + if (((cfg = ti->cfg) != NULL) && (cfg->refcnt != 0)) { >> + /* Table is referenced by some rules */ >> + IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> + return (EBUSY); >> } >> + tti = *ti; >> + /* Remove state tables from main structure */ >> + ti->state = NULL; >> + ti->xstate = NULL; >> + ti->cfg = NULL; >> + IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> + >> + destroy_table(ch, &tti); >> >> return (0); >> } >> @@ -446,152 +1039,179 @@ ipfw_flush_table(struct ip_fw_chain *ch, uint16_t >> void >> ipfw_destroy_tables(struct ip_fw_chain *ch) >> { >> - uint16_t tbl; >> + uint32_t tbl; >> + struct table_info *ti; >> >> /* Flush all tables */ >> - for (tbl = 0; tbl < V_fw_tables_max; tbl++) >> - ipfw_flush_table(ch, tbl); >> + ti = (struct table_info *)ch->tablestate; >> + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) >> + destroy_table(ch, ti); >> >> /* Free pointers itself */ >> - free(ch->tables, M_IPFW); >> - free(ch->xtables, M_IPFW); >> - free(ch->tabletype, M_IPFW); >> + free(ch->tablestate, M_IPFW_TBL); >> + free(ch->tablecfg, M_IPFW_TBL); >> } >> >> int >> ipfw_init_tables(struct ip_fw_chain *ch) >> { >> + struct table_info *ti; >> + struct tables_config *tc; >> + uint32_t tbl; >> + >> /* Allocate pointers */ >> - ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); >> - ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); >> - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); >> + ch->tablestate = malloc(V_fw_tables_max * sizeof(struct table_info), >> + M_IPFW_TBL, M_WAITOK | M_ZERO); >> + >> + ch->tablecfg = malloc(sizeof(struct tables_config), M_IPFW_TBL, >> + M_WAITOK | M_ZERO); >> + >> + /* Set initial handlers for all tables */ >> + ti = (struct table_info *)ch->tablestate; >> + for (tbl = 0; tbl < V_fw_tables_max; tbl++, ti++) >> + ti->lookup = lookup_cidr; >> + >> + /* Set up tablenames hash */ >> + tc = ch->tablecfg; >> + for (tbl = 0; tbl < TABLENAME_HASH_SIZE; tbl++) >> + TAILQ_INIT(&tc->thash[tbl]); >> + >> return (0); >> } >> >> int >> ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables) >> { >> - struct radix_node_head **tables, **xtables, *rnh; >> - struct radix_node_head **tables_old, **xtables_old; >> - uint8_t *tabletype, *tabletype_old; >> unsigned int ntables_old, tbl; >> + struct table_info *ti, *ti_old, *ti_new; >> >> /* Check new value for validity */ >> if (ntables > IPFW_TABLES_MAX) >> ntables = IPFW_TABLES_MAX; >> >> /* Allocate new pointers */ >> - tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); >> - xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); >> - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); >> + ti_new = malloc(ntables * sizeof(struct table_info), >> + M_IPFW_TBL, M_WAITOK | M_ZERO); >> >> + IPFW_UH_WLOCK(ch); >> IPFW_WLOCK(ch); >> >> tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; >> >> /* Copy old table pointers */ >> - memcpy(tables, ch->tables, sizeof(void *) * tbl); >> - memcpy(xtables, ch->xtables, sizeof(void *) * tbl); >> - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); >> + memcpy(ti_new, ch->tablestate, sizeof(struct table_info) * tbl); >> >> /* Change pointers and number of tables */ >> - tables_old = ch->tables; >> - xtables_old = ch->xtables; >> - tabletype_old = ch->tabletype; >> - ch->tables = tables; >> - ch->xtables = xtables; >> - ch->tabletype = tabletype; >> + ti_old = (struct table_info *)ch->tablestate; >> + ch->tablestate = ti_new; >> >> ntables_old = V_fw_tables_max; >> V_fw_tables_max = ntables; >> >> IPFW_WUNLOCK(ch); >> + IPFW_UH_WUNLOCK(ch); >> >> /* Check if we need to destroy radix trees */ >> if (ntables < ntables_old) { >> - for (tbl = ntables; tbl < ntables_old; tbl++) { >> - if ((rnh = tables_old[tbl]) != NULL) { >> - rnh->rnh_walktree(rnh, flush_table_entry, rnh); >> - rn_detachhead((void **)&rnh); >> - } >> + ti = &ti_old[ntables]; >> + for (tbl = ntables; tbl < ntables_old; tbl++, ti++) >> + destroy_table(ch, ti); >> + } >> >> - if ((rnh = xtables_old[tbl]) != NULL) { >> - rnh->rnh_walktree(rnh, flush_table_entry, rnh); >> - rn_detachhead((void **)&rnh); >> - } >> - } >> + /* Check if we need to setup new ones */ >> + if (ntables > ntables_old) { >> + ti = &ti_new[ntables_old]; >> + for (tbl = ntables_old; tbl < ntables; tbl++, ti++) >> + ti->lookup = lookup_cidr; >> } >> >> /* Free old pointers */ >> - free(tables_old, M_IPFW); >> - free(xtables_old, M_IPFW); >> - free(tabletype_old, M_IPFW); >> + free(ti_old, M_IPFW_TBL); >> >> return (0); >> } >> >> -int >> -ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, >> - uint32_t *val) >> +static int >> +lookup_cidr(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) >> { >> struct radix_node_head *rnh; >> struct table_entry *ent; >> - struct sockaddr_in sa; >> + struct table_xentry *xent; >> >> - if (tbl >= V_fw_tables_max) >> + >> + if (keylen == sizeof(struct in_addr)) { >> + /* IPv4 lookup */ >> + struct sockaddr_in sa; >> + >> + if ((rnh = (struct radix_node_head *)st) == NULL) >> + return (0); >> + >> + KEY_LEN(sa) = KEY_LEN_INET; >> + sa.sin_addr.s_addr = ((struct in_addr *)key)->s_addr; >> + ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); >> + if (ent != NULL) { >> + *val = ent->value; >> + return (1); >> + } >> + >> return (0); >> - if ((rnh = ch->tables[tbl]) == NULL) >> + } >> + >> + /* IPv6 lookup */ >> + struct sockaddr_in6 sa6; >> + >> + if ((rnh = (struct radix_node_head *)xst) == NULL) >> return (0); >> - KEY_LEN(sa) = KEY_LEN_INET; >> - sa.sin_addr.s_addr = addr; >> - ent = (struct table_entry *)(rnh->rnh_matchaddr(&sa, rnh)); >> - if (ent != NULL) { >> - *val = ent->value; >> + >> + KEY_LEN(sa6) = KEY_LEN_INET6; >> + memcpy(&sa6.sin6_addr, key, sizeof(struct in6_addr)); >> + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); >> + >> + if (xent != NULL) { >> + *val = xent->value; >> return (1); >> } >> + >> return (0); >> } >> >> -int >> -ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, >> - uint32_t *val, int type) >> +static int >> +lookup_iface(void *st, void *xst, void *key, uint32_t keylen, uint32_t *val) >> { >> struct radix_node_head *rnh; >> + struct xaddr_iface iface; >> struct table_xentry *xent; >> - struct sockaddr_in6 sa6; >> - struct xaddr_iface iface; >> >> - if (tbl >= V_fw_tables_max) >> + if ((rnh = (struct radix_node_head *)xst) == NULL) >> return (0); >> - if ((rnh = ch->xtables[tbl]) == NULL) >> - return (0); >> >> - switch (type) { >> - case IPFW_TABLE_CIDR: >> - KEY_LEN(sa6) = KEY_LEN_INET6; >> - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); >> - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&sa6, rnh)); >> - break; >> - >> - case IPFW_TABLE_INTERFACE: >> - KEY_LEN(iface) = KEY_LEN_IFACE + >> - strlcpy(iface.ifname, (char *)paddr, IF_NAMESIZE) + 1; >> - /* Assume direct match */ >> - /* FIXME: Add interface pattern matching */ >> - xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); >> - break; >> - >> - default: >> - return (0); >> - } >> - >> + /* IPv6 lookup */ >> + KEY_LEN(iface) = KEY_LEN_IFACE + >> + strlcpy(iface.ifname, (char *)key, IF_NAMESIZE) + 1; >> + /* Assume direct match */ >> + xent = (struct table_xentry *)(rnh->rnh_matchaddr(&iface, rnh)); >> if (xent != NULL) { >> *val = xent->value; >> return (1); >> } >> + >> return (0); >> } >> >> +int >> +ipfw_lookup_table(struct ip_fw_chain *ch, uint32_t tbl, >> + uint32_t keylen, void *key, uint32_t *val) >> +{ >> + struct table_info *ti; >> + >> + if (tbl >= V_fw_tables_max) >> + return (0); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + >> + return (ti->lookup(ti->state, ti->xstate, key, keylen, val)); >> +} >> + >> static int >> count_table_entry(struct radix_node *rn, void *arg) >> { >> @@ -605,11 +1225,15 @@ int >> ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) >> { >> struct radix_node_head *rnh; >> + struct table_info *ti; >> >> if (tbl >= V_fw_tables_max) >> return (EINVAL); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + >> *cnt = 0; >> - if ((rnh = ch->tables[tbl]) == NULL) >> + if ((rnh = ti->state) == NULL) >> return (0); >> rnh->rnh_walktree(rnh, count_table_entry, cnt); >> return (0); >> @@ -640,11 +1264,15 @@ int >> ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) >> { >> struct radix_node_head *rnh; >> + struct table_info *ti; >> >> if (tbl->tbl >= V_fw_tables_max) >> return (EINVAL); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; >> + >> tbl->cnt = 0; >> - if ((rnh = ch->tables[tbl->tbl]) == NULL) >> + if ((rnh = ti->state) == NULL) >> return (0); >> rnh->rnh_walktree(rnh, dump_table_entry, tbl); >> return (0); >> @@ -660,20 +1288,32 @@ count_table_xentry(struct radix_node *rn, void *ar >> } >> >> int >> -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) >> +ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *sz, >> + uint32_t *cnt) >> { >> struct radix_node_head *rnh; >> + struct table_info *ti; >> + uint32_t count; >> >> if (tbl >= V_fw_tables_max) >> return (EINVAL); >> - *cnt = 0; >> - if ((rnh = ch->tables[tbl]) != NULL) >> - rnh->rnh_walktree(rnh, count_table_xentry, cnt); >> - if ((rnh = ch->xtables[tbl]) != NULL) >> - rnh->rnh_walktree(rnh, count_table_xentry, cnt); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl]; >> + >> + *sz = 0; >> + count = 0; >> + if ((rnh = ti->state) != NULL) >> + rnh->rnh_walktree(rnh, count_table_xentry, sz); >> + if ((rnh = ti->xstate) != NULL) >> + rnh->rnh_walktree(rnh, count_table_xentry, sz); >> + count = *sz / sizeof(ipfw_table_xentry); >> + >> + if (cnt != NULL) >> + *cnt = count; >> + >> /* Return zero if table is empty */ >> - if (*cnt > 0) >> - (*cnt) += sizeof(ipfw_xtable); >> + if (*sz > 0) >> + (*sz) += sizeof(ipfw_xtable); >> return (0); >> } >> >> @@ -750,14 +1390,18 @@ int >> ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) >> { >> struct radix_node_head *rnh; >> + struct table_info *ti; >> >> if (tbl->tbl >= V_fw_tables_max) >> return (EINVAL); >> + >> + ti = &((struct table_info *)ch->tablestate)[tbl->tbl]; >> + >> tbl->cnt = 0; >> - tbl->type = ch->tabletype[tbl->tbl]; >> - if ((rnh = ch->tables[tbl->tbl]) != NULL) >> + tbl->type = TABLETYPE(ti); >> + if ((rnh = ti->state) != NULL) >> rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); >> - if ((rnh = ch->xtables[tbl->tbl]) != NULL) >> + if ((rnh = ti->xstate) != NULL) >> rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); >> return (0); >> } > From owner-freebsd-net@FreeBSD.ORG Wed May 21 19:18:59 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C7CE68A; Wed, 21 May 2014 19:18:59 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 015072683; Wed, 21 May 2014 19:18:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4LJIwMC072413; Wed, 21 May 2014 19:18:58 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4LJIwKD072412; Wed, 21 May 2014 19:18:58 GMT (envelope-from linimon) Date: Wed, 21 May 2014 19:18:58 GMT Message-Id: <201405211918.s4LJIwKD072412@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/190038: [ipf] ipf -Fa -6 clears v6 and v6 lists X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 19:18:59 -0000 Old Synopsis: ipf -Fa -6 clears v6 and v6 lists New Synopsis: [ipf] ipf -Fa -6 clears v6 and v6 lists Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Wed May 21 19:18:10 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=190038 From owner-freebsd-net@FreeBSD.ORG Wed May 21 19:41:47 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13309DE4; Wed, 21 May 2014 19:41:47 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 578AF2906; Wed, 21 May 2014 19:41:46 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id n12so2475940wgh.5 for ; Wed, 21 May 2014 12:41:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mFeMimeKdxd6Pj6J7GMMZvHFWsVb8Z1c636yfIGELMc=; b=kMa5KPfAkudvGsTAyrAdqUDYb1QRZFH65uX1h20KoNEttRlEcXJyrKpbo+N8SDTKBF ZeAkAVtmIMSsE2FXqPlK+h5KXJfGK9yPF43WaPo6F49gJUUaL8NyHsPWl+tkoCZJxaTH SAKfYloATdz2RkI4GcXCy3ftmwd4OfbIg7SyrqE4QF9oMjS7TwHHCsj7Bfz6FTLwIhrm 4HfXlaK/dmj9sAydBw/CPM68m04oNOJOdg5AIjVboClbH671ksGfXr+qwdeMeOyPylpS orhc+qsWz9EZ0Y0vTdm1MTDSOe/RurPkFYpBSjCLgPu/G5IV1HUmv1mRoC/zmh/YmHtn CtAQ== MIME-Version: 1.0 X-Received: by 10.194.57.38 with SMTP id f6mr19013802wjq.59.1400701304495; Wed, 21 May 2014 12:41:44 -0700 (PDT) Received: by 10.216.116.136 with HTTP; Wed, 21 May 2014 12:41:44 -0700 (PDT) In-Reply-To: References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> Date: Thu, 22 May 2014 03:41:44 +0800 Message-ID: Subject: Re: [CFT]: ipfw named tables / different tabletypes From: Bill Yuan To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Luigi Rizzo , "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 19:41:47 -0000 On Thu, May 22, 2014 at 12:49 AM, Luigi Rizzo wrote: > > > > On Wed, May 21, 2014 at 11:17 AM, Bill Yuan wrote: > >> 1. "each table can have it's own name", I like this idea. I am also >> working on this kind of functions, But I am in different way. In my >> opinion. the "name" or "type" of the table, all this are utility functio= n >> for the user. in the kernel space. I want to keep all the things the sam= e >> as before. and add a translate function in the user land commands, so th= at >> means we need to find a space to keep the mapping of the name<->id or >> type<->id, >> > > =E2=80=8Bthis does not work, you'd have another database to keep in sync > and on which you have races with modifications to the ipfw config. > > cheers > luigi > =E2=80=8B > Hi Luigi, Thanks for your reply. That is my original idea of the implementation. I though it is good because it can minimize the changes in existing code. Yes, It requires to have another database to keep in sync, But it is not necessary to mix a lot with the existing code in the userland. that also means we dont need to make any changes in the existing code of the kernel space. e.g. if we support the table name in the command , it will be: >ipfw table 1 name sales this type of commands to create/update the mapping databases >ipfw table sales add 1.2.3.4 >ipfw table sales list this type of commands need to call function "get_id_from_name()" and it is in the userland command. And if the mapping is "type-name-id" then that means it can be generic purpose. e.g. tablename-sale-10 --> sale table is id 10 pipename-vip-20 --> vip pipe's id is 20 From owner-freebsd-net@FreeBSD.ORG Wed May 21 19:43:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23351E96 for ; Wed, 21 May 2014 19:43:18 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 055BC2916 for ; Wed, 21 May 2014 19:43:17 +0000 (UTC) Received: from [10.12.72.220] (unknown [69.164.56.1]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 3A8B6194138 for ; Wed, 21 May 2014 19:43:16 +0000 (UTC) Subject: re(4) stalls, crashes(supposed patch exists?) From: Sean Bruno Reply-To: sbruno@freebsd.org To: FreeBSD Net Content-Type: text/plain; charset="us-ascii" Date: Wed, 21 May 2014 12:43:15 -0700 Message-ID: <1400701395.1848.16.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 19:43:18 -0000 The Bytemark Site of freebsd.org is experiencing periodic stalls and crashes on the machines being used as routers. I have heard of a rumored patch that exists "somewhere" to resolve this, but when I asked at BSDCan, I got no takers. Any thoughts? FreeBSD igw0.bme.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r265337: Sun May 4 22:05:35 UTC 2014 peter@igw0.bme.freebsd.org:/usr/obj/usr/src/sys/IGW0 amd64 re0@pci0:3:0:0: class=0x020000 card=0x85051043 chip=0x816810ec rev=0x09 hdr=0x00 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' class = network subclass = ethernet re0: port 0xe800-0xe8ff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff irq 18 at device 0.0 on pci3 re0: Using 1 MSI-X message re0: turning off MSI enable bit. re0: Chip rev. 0x48000000 re0: MAC rev. 0x00000000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Ethernet address: 08:60:6e:d7:31:d2 Panic: panic: _mtx_lock_sleep: recursed on non-recursive mutex pf_idhash @ /usr/src/sys/netpfil/pf/pf.c:922 cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00003c5d70 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00003c5e20 vpanic() at vpanic+0x126/frame 0xfffffe00003c5e60 kassert_panic() at kassert_panic+0x136/frame 0xfffffe00003c5ed0 __mtx_lock_sleep() at __mtx_lock_sleep+0x369/frame 0xfffffe00003c5f50 __mtx_lock_flags() at __mtx_lock_flags+0xf5/frame 0xfffffe00003c5fa0 pf_state_insert() at pf_state_insert+0x37d/frame 0xfffffe00003c6040 pf_test_rule() at pf_test_rule+0x26ce/frame 0xfffffe00003c6520 pf_test6() at pf_test6+0x11ef/frame 0xfffffe00003c66c0 pf_check6_in() at pf_check6_in+0x36/frame 0xfffffe00003c66e0 pfil_run_hooks() at pfil_run_hooks+0x93/frame 0xfffffe00003c6770 ip6_input() at ip6_input+0x619/frame 0xfffffe00003c6950 netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe00003c69c0 ether_demux() at ether_demux+0x13c/frame 0xfffffe00003c69f0 ether_nh_input() at ether_nh_input+0x32a/frame 0xfffffe00003c6a20 netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe00003c6a90 re_rxeof() at re_rxeof+0x539/frame 0xfffffe00003c6af0 re_intr_msi() at re_intr_msi+0xcc/frame 0xfffffe00003c6b30 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe00003c6b70 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe00003c6bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe00003c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00003c6bf0 --- trap 0, rip = 0, rsp = 0xfffffe00003c6cb0, rbp = 0 --- KDB: enter: panic Panic: Kernel page fault with the following non-sleepable locks held: shared rw ipsec request (ipsec request) r = 0 (0xfffff80005dea160) locked @ /usr/src/sys/netipsec/ipsec_output.c:802 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame 0xfffffe011ad60f40 kdb_backtrace() at kdb_backtrace+0x37/frame 0xfffffe011ad61000 _witness_debugger() at _witness_debugger+0x2c/frame 0xfffffe011ad61020 witness_warn() at witness_warn+0x2d4/frame 0xfffffe011ad61160 trap_pfault() at trap_pfault+0x6a/frame 0xfffffe011ad611f0 trap() at trap+0x41a/frame 0xfffffe011ad613f0 calltrap() at calltrap+0x8/frame 0xfffffe011ad613f0 --- trap 0xc, rip = 0xffffffff806d5ada, rsp = 0xfffffe011ad614b0, rbp = 0xfffffe011ad61580 --- ipsec6_output_tunnel() at ipsec6_output_tunnel+0xda/frame 0xfffffe011ad61580 ip6_forward() at ip6_forward+0x69a/frame 0xfffffe011ad61710 ip6_input() at ip6_input+0xd06/frame 0xfffffe011ad618a0 netisr_dispatch_src() at netisr_dispatch_src+0x15d/frame 0xfffffe011ad61910 ether_demux() at ether_demux+0x1a9/frame 0xfffffe011ad61940 ether_nh_input() at ether_nh_input+0x209/frame 0xfffffe011ad61980 netisr_dispatch_src() at netisr_dispatch_src+0x15d/frame 0xfffffe011ad619f0 re_rxeof() at re_rxeof+0x4a9/frame 0xfffffe011ad61a60 re_int_task() at re_int_task+0x1ea/frame 0xfffffe011ad61aa0 taskqueue_run_locked() at taskqueue_run_locked+0x93/frame 0xfffffe011ad61b00 taskqueue_run() at taskqueue_run+0x3d/frame 0xfffffe011ad61b20 intr_event_execute_handlers() at intr_event_execute_handlers+0x6a/frame 0xfffffe011ad61b50 ithread_loop() at ithread_loop+0x9b/frame 0xfffffe011ad61ba0 fork_exit() at fork_exit+0x139/frame 0xfffffe011ad61bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011ad61bf0 --- trap 0, rip = 0, rsp = 0xfffffe011ad61cb0, rbp = 0 --- From owner-freebsd-net@FreeBSD.ORG Wed May 21 20:44:03 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD5A757A; Wed, 21 May 2014 20:44:03 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 200D32E36; Wed, 21 May 2014 20:44:02 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id D0B2E7300A; Wed, 21 May 2014 22:48:26 +0200 (CEST) Date: Wed, 21 May 2014 22:48:26 +0200 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140521204826.GA67124@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <537CEC12.8050404@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 20:44:03 -0000 On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: > On 21.05.2014 15:10, Luigi Rizzo wrote: > > On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: > >> Hello list! > >> > >> This patch adds ability to name tables / reference them by name. > >> Additionally, it simplifies adding new table types. > > Hi Alex, > > at a high level, i think your changes here are in the right direction. > Hello! > > > > However i think part of the design, and also of the patch below, > > is not sound/correct so i would like to wait to commit at least > > until some of the problems below are resolved. > > > > 1. The patch as is has several issues (variable declarations in the > > middle of a block, assignment in conditionals, incorrect > > comments in cut&pasted code, missing checks on strings) > > that should be corrected. > ..missing documentation and so on :) > Of course, I'll fix all these (or most of these :)) good thanks > > 3. there is no explanation, here or in the code, on how the > > names are managed. I could try to figure out from the code > > but it would be the wrong way to understand things so let me > > ask, and please explain what you have in mind. > Currently it is very simple non-resizable hash table with fnv hash based > on table name. that is not an issue. The question is whether one needs to lookup the hash table every time you have a 'table' argument (of course i think one should not, and the implementation i propose below gives direct access to the table without name lookups, as the internal identifier is still poiniter or small integer, just one that is not exposed to userspace). > > > > Let me address first the name <-> table-id thing. > > > > Introducing a symbolic name for tables is a great and useful feature. > > However the implementation has some tricky issues: > > > > - do you want to retain the old numeric identifiers or not ? > > I think it is a bad idea to have two namespaces for the same thing, > > so if we are switching to alphanumeric strings for tables we should > > as well get rid of the numbers (i.e. consider them as strings as well). > > > > I am strongly in favour of not using names as aliases for numbers. > > It would require no changes for clients issuing ipfw commands > > from a script, and would not require users to to manually handle > > the name-id translations. > Well. I'd prefer not to. However, code we're discussing assumes that > numeric ids > are primary ones (e.g. you can't have named, but unnumbered table, > you have to choose number by yourself). Switching to named-only tables > can be tricky since we don't want to match them by inside rules and we > don't want > to loose atomicity when allocating table ids via separate cmds before > adding rule. i think this is solved by the implementation i proposed below. > It looks like we can do the following: > 1) Add another IP_FW3_ADD opcode with the following layout: > > { > rule len; > unmodified rule itself; > tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} > tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} > .. > } > Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be > real values > but values described in attached TLVs. > > After validating/parsing/checking under UH lock we replace fake values > with real ones (probably, newly allocated) > and return or rollback atomically. yes this should work, probably not even requiring a new IP_FW3_ADD opcode because the extra tlv entries can appear in a block by themselves. > The same thing can be done for displaying ruleset, however I'd prefer > client to ask for table names and caching them while displaying. > > Old clients will retain the ability to operate on tables but will see > table ids, so nothing will change for them > (except the case when new binary adds table "5" and new binary sees id > which is not 5, but that shouldn't be a problem). we can solve this by using 'low' numbers for the numeric tables (these were limited anyways) and allocate the fake entries in another range. > > > > - The rename command worries me a lot: > > > > > ipfw table name XXX > > > Names (or renames) table. Not the name has to be unique. > > > > because it is neither clear nor intuitive whether you want to > > 1. just rename a table (possibly breaking or creating references > > for rules in the firewall), or > > 2. modify the name-id translation preserving existing pointers. > > > > Consider the sequence > > ipfw table 13 name bar > > ipfw add 100 count dst-ip table bar > > ipfw table 13 name foo > > ipfw table 14 name bar > > ipfw add 200 count src-port 22 dst-ip table bar > > > > Approach #1 would detach rule 100 from table 13 and then connect to 14 > > (in a non-atomic way), whereas approach #2 would make rule 100 and 200 > > point to different tables (which is confusing). > > > > Now part of this problem goes away if we get rid of number altogether. > > > > You may legitimately want to swap tables in an atomic way (we have something > > similar for rulesets): > > ipfw set swap number number > There is some problem here: > Atomic ruleset changes is a great thing, but tables have no atomic support. > We, for example, are solving this by using different table range on > every new configuration. > It won't be possible with named-only tables (since names usually care > some semantic in contrast to numbers). > The only thing I can think of is separate namespace per each set. > What do you think? maybe i am missing some detail but it seems reasonably easy to implement the atomic swap -- and the use case is when you want to move from one configuration to a new one: ipfw table foo-new flush // clear initial content ipfw table foo-new add ... ipfw table swap foo-current foo-new // swap the content of the table objects so you preserve the semantic of the name very easily. > > > > So here is what i would propose: > > - [gradually] introduce new commands that accept strings for every place where > > a number was previously accepted. Internally in the firewall, the old > > 16-bit number is interpreted as a string. > Yup. > > > > - within the kernel create a small set of functions (invoked on insert, list, delete) > > that do proper checks on the string and translate it to a pointer (or short integer, > > i.e. an index in an array) to the proper object. Done properly, we can reuse > > this code also for pipes, sets, nat groups and whatnot. > Yup. > > > > When a rule references an object that does not exist, you create an empty > > object that a subsequent table/pipe/XXX 'create' would initialize. > > On 'destroy', no need to touch the ruleset, just clear the object. > Yes. This looks better than requiring user to create table of given type > _before_ > referencing it. > > > > - for renames, try to follow the approach used for sets i.e. > > ipfw table rename old-name new-name > > changes the name, preserves the object. > > Does not touch the ruleset, only the translation table > Well, I'm not sure about renaming. I'd prefer permitting renaming iff > table is not referenced. > (and why do we need to rename tables?) you are right, let's forget it. And 'ipfw set move' actually does a different thing which has no use here. > > ipfw table swap first-table second-table > > atomically swap the content of the two table objects > > (once again not touching the rulesets) cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 22 03:58:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EACB94C4 for ; Thu, 22 May 2014 03:58:47 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88290200D for ; Thu, 22 May 2014 03:58:47 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id m15so2782256wgh.4 for ; Wed, 21 May 2014 20:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=PHIKTKAGt7t3oJ9pdLaEZE75pmg86cje0b0UWUXlBfA=; b=TNgxL7vz6HUORQMJ8+4rIyqfnazgzeKqzqbX7koS38UhJD9ZeNT4gw/Z9v8Sv18Iqe INYnuJGmcGFwJsAyjUZP/LjWy0t+NcCMZCFq+9SIDpUo4DKCi7S5zZ8FtvQwjsDBnJ8J wR366u3Zm3jh0flY8+ohTj+mv9oppi2TfkRWuRrHgWErTUjRWoqsfh+BlHYWgQ8ZDvfH XaoVv8N/4tyRN/VoWyMyBGJxatuYs5OM3qV0se+a+IBiAQcrZ237lwpC2Adt20E/GXer f2XidNaNCSoh2PqYizQlXM/aj5IBm5VHL5UIupyMxUXFagWybtFK/fuzFmPqtOWjvocG ASVw== MIME-Version: 1.0 X-Received: by 10.194.60.211 with SMTP id j19mr35808081wjr.51.1400731125795; Wed, 21 May 2014 20:58:45 -0700 (PDT) Received: by 10.194.92.144 with HTTP; Wed, 21 May 2014 20:58:45 -0700 (PDT) Date: Thu, 22 May 2014 09:28:45 +0530 Message-ID: Subject: Regarding Netmap in VM From: Prashant Upadhyaya To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 03:58:48 -0000 Hi, Suppose I am on x86 with Intel 82599 NIC. Now I spawn a VM (running FreeBSD as guest OS with Netmap support) For better I/O performance, I have two main choices -- 1. Pass the 82599 NIC as a PCI passthrough device into the VM 2. Use SRIOV VF of 82599 into the VM Question is, will Netmap be able to utilize both the above environments when I run the userspace application in the guest OS in the VM, or will there be any issues. Regards -Prashant From owner-freebsd-net@FreeBSD.ORG Thu May 22 04:02:35 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 549335A2; Thu, 22 May 2014 04:02:35 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5ABC620B0; Thu, 22 May 2014 04:02:34 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id q8so2186172lbi.26 for ; Wed, 21 May 2014 21:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FulnAJHMQuhxV7tpHfhjC4p4KY3IduwrzR3t05KN570=; b=os0pAqFhlbwMAMOxg3NMeSptYyppUcdUKyesWlgSk5cY7m2q/BDEKSS89WYZf9sz5L 5yT3p/tMGkkAozOKxA8vmdwiqtGfD+SCvAGp19HI1Ys8RMtOLFmci6xgGl+R1DUTRGZt AzJktaMVJLHSNtFZ1ajhtLPx+CNUeU/SCLmvR09qWPO+NrYcuZVYY8yAUO0NKs9wdazO 04cDVBxQ8G9bnRBcuzGLzBemUlOsMtnqNNl9ljIUze50TrJWEYQUWrl7Rf/lvQVPxp2h sxrFvZqMDyOwFQaR9nOII79ifqJ3V3IxYOyeg7MQ/ujIWOL7nIEWJfE9vJJGHulqWGWd PWbw== MIME-Version: 1.0 X-Received: by 10.152.23.6 with SMTP id i6mr41156348laf.24.1400731352334; Wed, 21 May 2014 21:02:32 -0700 (PDT) Received: by 10.112.5.10 with HTTP; Wed, 21 May 2014 21:02:32 -0700 (PDT) In-Reply-To: <20140521204826.GA67124@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> Date: Thu, 22 May 2014 12:02:32 +0800 Message-ID: Subject: Re: [CFT]: ipfw named tables / different tabletypes From: Bill Yuan To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Luigi Rizzo , "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 04:02:35 -0000 Sorry I am little bit blur now. And I am going to wait for your code, I think it will be a good opportunity to learn as a newbie here. 1. So use alphanumeric strings for table's id is not a good way. (because will loose atomicity). I agree that, all this feature/function, I would like to name them as utilities. It can be more user friendly with these utilities. 2. And instead, you are going to introduce a IP_FW3_ADD and ... > It looks like we can do the following: > 1) Add another IP_FW3_ADD opcode with the following layout: > > { > rule len; > unmodified rule itself; > tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} > tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} > .. > } > Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be > real values So that is the "database" or "mapping table" which I mentioned. So are you going to translate the name to id before calling kernel-space methods? On Thu, May 22, 2014 at 4:48 AM, Luigi Rizzo wrote: > On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: > > On 21.05.2014 15:10, Luigi Rizzo wrote: > > > On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: > > >> Hello list! > > >> > > >> This patch adds ability to name tables / reference them by name. > > >> Additionally, it simplifies adding new table types. > > > Hi Alex, > > > at a high level, i think your changes here are in the right direction. > > Hello! > > > > > > However i think part of the design, and also of the patch below, > > > is not sound/correct so i would like to wait to commit at least > > > until some of the problems below are resolved. > > > > > > 1. The patch as is has several issues (variable declarations in the > > > middle of a block, assignment in conditionals, incorrect > > > comments in cut&pasted code, missing checks on strings) > > > that should be corrected. > > ..missing documentation and so on :) > > Of course, I'll fix all these (or most of these :)) > > good thanks > > > > 3. there is no explanation, here or in the code, on how the > > > names are managed. I could try to figure out from the code > > > but it would be the wrong way to understand things so let me > > > ask, and please explain what you have in mind. > > Currently it is very simple non-resizable hash table with fnv hash based > > on table name. > > that is not an issue. The question is whether one needs to lookup > the hash table every time you have a 'table' argument (of course i > think one should not, and the implementation i propose below gives > direct access to the table without name lookups, as the internal > identifier is still poiniter or small integer, just one that is not exposed > to userspace). > > > > > > > Let me address first the name <-> table-id thing. > > > > > > Introducing a symbolic name for tables is a great and useful feature. > > > However the implementation has some tricky issues: > > > > > > - do you want to retain the old numeric identifiers or not ? > > > I think it is a bad idea to have two namespaces for the same thing, > > > so if we are switching to alphanumeric strings for tables we should > > > as well get rid of the numbers (i.e. consider them as strings as > well). > > > > > > I am strongly in favour of not using names as aliases for numbers. > > > It would require no changes for clients issuing ipfw commands > > > from a script, and would not require users to to manually handle > > > the name-id translations. > > Well. I'd prefer not to. However, code we're discussing assumes that > > numeric ids > > are primary ones (e.g. you can't have named, but unnumbered table, > > you have to choose number by yourself). Switching to named-only tables > > can be tricky since we don't want to match them by inside rules and we > > don't want > > to loose atomicity when allocating table ids via separate cmds before > > adding rule. > > i think this is solved by the implementation i proposed below. > > > It looks like we can do the following: > > 1) Add another IP_FW3_ADD opcode with the following layout: > > > > { > > rule len; > > unmodified rule itself; > > tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} > > tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} > > .. > > } > > Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be > > real values > > but values described in attached TLVs. > > > > After validating/parsing/checking under UH lock we replace fake values > > with real ones (probably, newly allocated) > > and return or rollback atomically. > > yes this should work, probably not even requiring a new IP_FW3_ADD opcode > because the extra tlv entries can appear in a block by themselves. > > > The same thing can be done for displaying ruleset, however I'd prefer > > client to ask for table names and caching them while displaying. > > > > Old clients will retain the ability to operate on tables but will see > > table ids, so nothing will change for them > > (except the case when new binary adds table "5" and new binary sees id > > which is not 5, but that shouldn't be a problem). > > we can solve this by using 'low' numbers for the numeric tables > (these were limited anyways) and allocate the fake entries in > another range. > > > > > > > - The rename command worries me a lot: > > > > > > > ipfw table name XXX > > > > Names (or renames) table. Not the name has to be unique. > > > > > > because it is neither clear nor intuitive whether you want to > > > 1. just rename a table (possibly breaking or creating references > > > for rules in the firewall), or > > > 2. modify the name-id translation preserving existing pointers. > > > > > > Consider the sequence > > > ipfw table 13 name bar > > > ipfw add 100 count dst-ip table bar > > > ipfw table 13 name foo > > > ipfw table 14 name bar > > > ipfw add 200 count src-port 22 dst-ip table bar > > > > > > Approach #1 would detach rule 100 from table 13 and then connect > to 14 > > > (in a non-atomic way), whereas approach #2 would make rule 100 and > 200 > > > point to different tables (which is confusing). > > > > > > Now part of this problem goes away if we get rid of number > altogether. > > > > > > You may legitimately want to swap tables in an atomic way (we have > something > > > similar for rulesets): > > > ipfw set swap number number > > There is some problem here: > > Atomic ruleset changes is a great thing, but tables have no atomic > support. > > We, for example, are solving this by using different table range on > > every new configuration. > > It won't be possible with named-only tables (since names usually care > > some semantic in contrast to numbers). > > The only thing I can think of is separate namespace per each set. > > What do you think? > > maybe i am missing some detail but it seems reasonably easy to implement > the atomic swap -- and the use case is when you want to move from > one configuration to a new one: > ipfw table foo-new flush // clear initial content > ipfw table foo-new add ... > ipfw table swap foo-current foo-new // swap the content of the > table objects > > so you preserve the semantic of the name very easily. > > > > > > > So here is what i would propose: > > > - [gradually] introduce new commands that accept strings for every > place where > > > a number was previously accepted. Internally in the firewall, the > old > > > 16-bit number is interpreted as a string. > > Yup. > > > > > > - within the kernel create a small set of functions (invoked on > insert, list, delete) > > > that do proper checks on the string and translate it to a pointer > (or short integer, > > > i.e. an index in an array) to the proper object. Done properly, we > can reuse > > > this code also for pipes, sets, nat groups and whatnot. > > Yup. > > > > > > When a rule references an object that does not exist, you create an > empty > > > object that a subsequent table/pipe/XXX 'create' would initialize. > > > On 'destroy', no need to touch the ruleset, just clear the object. > > Yes. This looks better than requiring user to create table of given type > > _before_ > > referencing it. > > > > > > - for renames, try to follow the approach used for sets i.e. > > > ipfw table rename old-name new-name > > > changes the name, preserves the object. > > > Does not touch the ruleset, only the translation table > > Well, I'm not sure about renaming. I'd prefer permitting renaming iff > > table is not referenced. > > (and why do we need to rename tables?) > > you are right, let's forget it. And 'ipfw set move' actually does a > different thing > which has no use here. > > > > ipfw table swap first-table second-table > > > atomically swap the content of the two table objects > > > (once again not touching the rulesets) > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu May 22 07:27:46 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52B78561; Thu, 22 May 2014 07:27:46 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28D0D22BE; Thu, 22 May 2014 07:27:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4M7RjkH054871; Thu, 22 May 2014 07:27:45 GMT (envelope-from mav@freefall.freebsd.org) Received: (from mav@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4M7Rjpb054870; Thu, 22 May 2014 07:27:45 GMT (envelope-from mav) Date: Thu, 22 May 2014 07:27:45 GMT Message-Id: <201405220727.s4M7Rjpb054870@freefall.freebsd.org> To: egrosbein@rdtc.ru, mav@FreeBSD.org, freebsd-net@FreeBSD.org From: mav@FreeBSD.org Subject: Re: kern/182212: [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 07:27:46 -0000 Synopsis: [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly State-Changed-From-To: open->patched State-Changed-By: mav State-Changed-When: Thu May 22 07:27:24 UTC 2014 State-Changed-Why: Patch committed to HEAD. http://www.freebsd.org/cgi/query-pr.cgi?pr=182212 From owner-freebsd-net@FreeBSD.ORG Thu May 22 07:30:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38A88617 for ; Thu, 22 May 2014 07:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A29222D6 for ; Thu, 22 May 2014 07:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4M7U0rJ057353 for ; Thu, 22 May 2014 07:30:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4M7U0uw057349; Thu, 22 May 2014 07:30:00 GMT (envelope-from gnats) Date: Thu, 22 May 2014 07:30:00 GMT Message-Id: <201405220730.s4M7U0uw057349@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: dfilter@FreeBSD.ORG (dfilter service) Subject: Re: kern/182212: commit references a PR Reply-To: dfilter@FreeBSD.ORG (dfilter service) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 07:30:01 -0000 The following reply was made to PR kern/182212; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/182212: commit references a PR Date: Thu, 22 May 2014 07:27:08 +0000 (UTC) Author: mav Date: Thu May 22 07:27:04 2014 New Revision: 266538 URL: http://svnweb.freebsd.org/changeset/base/266538 Log: Make ng_mppc to not disable the node in case of multiple packet loss. Quite often it can be just packet reorder, and killing link in such case is inconvenient. Add few sysctl's to control that behavior. PR: kern/182212 Submitted by: Eugene Grosbein MFC after: 2 weeks Modified: head/sys/netgraph/ng_mppc.c Modified: head/sys/netgraph/ng_mppc.c ============================================================================== --- head/sys/netgraph/ng_mppc.c Thu May 22 07:25:36 2014 (r266537) +++ head/sys/netgraph/ng_mppc.c Thu May 22 07:27:04 2014 (r266538) @@ -55,6 +55,7 @@ #include #include #include +#include #include #include @@ -107,6 +108,23 @@ static MALLOC_DEFINE(M_NETGRAPH_MPPC, "n */ #define MPPE_MAX_REKEY 1000 +SYSCTL_NODE(_net_graph, OID_AUTO, mppe, CTLFLAG_RW, 0, "MPPE"); + +static int mppe_block_on_max_rekey = 0; +TUNABLE_INT("net.graph.mppe.block_on_max_rekey", &mppe_block_on_max_rekey); +SYSCTL_INT(_net_graph_mppe, OID_AUTO, block_on_max_rekey, CTLFLAG_RW, + &mppe_block_on_max_rekey, 0, "Block node on max MPPE key re-calculations"); + +static int mppe_log_max_rekey = 1; +TUNABLE_INT("net.graph.mppe.log_max_rekey", &mppe_log_max_rekey); +SYSCTL_INT(_net_graph_mppe, OID_AUTO, log_max_rekey, CTLFLAG_RW, + &mppe_log_max_rekey, 0, "Log max MPPE key re-calculations event"); + +static int mppe_max_rekey = MPPE_MAX_REKEY; +TUNABLE_INT("net.graph.mppe.max_rekey", &mppe_max_rekey); +SYSCTL_INT(_net_graph_mppe, OID_AUTO, max_rekey, CTLFLAG_RW, + &mppe_max_rekey, 0, "Maximum number of MPPE key re-calculations"); + /* MPPC packet header bits */ #define MPPC_FLAG_FLUSHED 0x8000 /* xmitter reset state */ #define MPPC_FLAG_RESTART 0x4000 /* compress history restart */ @@ -646,12 +664,23 @@ ng_mppc_decompress(node_p node, struct m /* How many times are we going to have to re-key? */ rekey = ((d->cfg.bits & MPPE_STATELESS) != 0) ? numLost : (numLost / (MPPE_UPDATE_MASK + 1)); - if (rekey > MPPE_MAX_REKEY) { - log(LOG_ERR, "%s: too many (%d) packets" - " dropped, disabling node %p!", - __func__, numLost, node); + if (rekey > mppe_max_rekey) { + if (mppe_block_on_max_rekey) { + if (mppe_log_max_rekey) { + log(LOG_ERR, "%s: too many (%d) packets" + " dropped, disabling node %p!\n", + __func__, numLost, node); + } priv->recv.cfg.enable = 0; goto failed; + } else { + if (mppe_log_max_rekey) { + log(LOG_ERR, "%s: %d packets" + " dropped, node %p\n", + __func__, numLost, node); + } + goto failed; + } } /* Re-key as necessary to catch up to peer */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu May 22 14:57:56 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40404C49; Thu, 22 May 2014 14:57:56 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDC662A12; Thu, 22 May 2014 14:57:55 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WnQX0-000K7z-RC; Thu, 22 May 2014 14:47:26 +0400 Message-ID: <537E1029.70007@FreeBSD.org> Date: Thu, 22 May 2014 18:56:41 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> In-Reply-To: <20140521204826.GA67124@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 14:57:56 -0000 On 22.05.2014 00:48, Luigi Rizzo wrote: > On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: >> On 21.05.2014 15:10, Luigi Rizzo wrote: >>> On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: >>>> Hello list! >>>> >>>> This patch adds ability to name tables / reference them by name. >>>> Additionally, it simplifies adding new table types. >>> Hi Alex, >>> at a high level, i think your changes here are in the right direction. >> Hello! >>> However i think part of the design, and also of the patch below, >>> is not sound/correct so i would like to wait to commit at least >>> until some of the problems below are resolved. >>> >>> 1. The patch as is has several issues (variable declarations in the >>> middle of a block, assignment in conditionals, incorrect >>> comments in cut&pasted code, missing checks on strings) >>> that should be corrected. >> ..missing documentation and so on :) >> Of course, I'll fix all these (or most of these :)) > good thanks > >>> 3. there is no explanation, here or in the code, on how the >>> names are managed. I could try to figure out from the code >>> but it would be the wrong way to understand things so let me >>> ask, and please explain what you have in mind. >> Currently it is very simple non-resizable hash table with fnv hash based >> on table name. > that is not an issue. The question is whether one needs to lookup > the hash table every time you have a 'table' argument (of course i > think one should not, and the implementation i propose below gives > direct access to the table without name lookups, as the internal > identifier is still poiniter or small integer, just one that is not exposed > to userspace). > >>> Let me address first the name <-> table-id thing. >>> >>> Introducing a symbolic name for tables is a great and useful feature. >>> However the implementation has some tricky issues: >>> >>> - do you want to retain the old numeric identifiers or not ? >>> I think it is a bad idea to have two namespaces for the same thing, >>> so if we are switching to alphanumeric strings for tables we should >>> as well get rid of the numbers (i.e. consider them as strings as well). >>> >>> I am strongly in favour of not using names as aliases for numbers. >>> It would require no changes for clients issuing ipfw commands >>> from a script, and would not require users to to manually handle >>> the name-id translations. >> Well. I'd prefer not to. However, code we're discussing assumes that >> numeric ids >> are primary ones (e.g. you can't have named, but unnumbered table, >> you have to choose number by yourself). Switching to named-only tables >> can be tricky since we don't want to match them by inside rules and we >> don't want >> to loose atomicity when allocating table ids via separate cmds before >> adding rule. > i think this is solved by the implementation i proposed below. > >> It looks like we can do the following: >> 1) Add another IP_FW3_ADD opcode with the following layout: >> >> { >> rule len; >> unmodified rule itself; >> tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} >> tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} >> .. >> } >> Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be >> real values >> but values described in attached TLVs. >> >> After validating/parsing/checking under UH lock we replace fake values >> with real ones (probably, newly allocated) >> and return or rollback atomically. > yes this should work, probably not even requiring a new IP_FW3_ADD opcode > because the extra tlv entries can appear in a block by themselves. > >> The same thing can be done for displaying ruleset, however I'd prefer >> client to ask for table names and caching them while displaying. >> >> Old clients will retain the ability to operate on tables but will see >> table ids, so nothing will change for them >> (except the case when new binary adds table "5" and new binary sees id >> which is not 5, but that shouldn't be a problem). > we can solve this by using 'low' numbers for the numeric tables > (these were limited anyways) and allocate the fake entries in > another range. Currently we have u16 space available in base opcode. Introducing another range will require additional opcode, additional array for new tables and so on. I'd prefer not to do this. Since table ids are allocated by ourselves we can (and should) pack them efficiently and 65k _real tables_ currently available is a quite good value. We preserve compability for old binaries so people with old userland and scripts should not not notice anything. We have no public userland API so the only base binary is /sbin/ipfw which is either old or new. > >>> - The rename command worries me a lot: >>> >>> > ipfw table name XXX >>> > Names (or renames) table. Not the name has to be unique. >>> >>> because it is neither clear nor intuitive whether you want to >>> 1. just rename a table (possibly breaking or creating references >>> for rules in the firewall), or >>> 2. modify the name-id translation preserving existing pointers. >>> >>> Consider the sequence >>> ipfw table 13 name bar >>> ipfw add 100 count dst-ip table bar >>> ipfw table 13 name foo >>> ipfw table 14 name bar >>> ipfw add 200 count src-port 22 dst-ip table bar >>> >>> Approach #1 would detach rule 100 from table 13 and then connect to 14 >>> (in a non-atomic way), whereas approach #2 would make rule 100 and 200 >>> point to different tables (which is confusing). >>> >>> Now part of this problem goes away if we get rid of number altogether. >>> >>> You may legitimately want to swap tables in an atomic way (we have something >>> similar for rulesets): >>> ipfw set swap number number >> There is some problem here: >> Atomic ruleset changes is a great thing, but tables have no atomic support. >> We, for example, are solving this by using different table range on >> every new configuration. >> It won't be possible with named-only tables (since names usually care >> some semantic in contrast to numbers). >> The only thing I can think of is separate namespace per each set. >> What do you think? > maybe i am missing some detail but it seems reasonably easy to implement > the atomic swap -- and the use case is when you want to move from > one configuration to a new one: > ipfw table foo-new flush // clear initial content > ipfw table foo-new add ... > ipfw table swap foo-current foo-new // swap the content of the table objects > > so you preserve the semantic of the name very easily. Yes. We can easily add atomic table swap that way. However, I'm talking about different use scenario: Atomically swap entire ruleset which has some tables depency: e.g. we have: " 100 allow ip from table(TABLE1) to me 200 allow ip from table(TABLE2) to (TABLE3) 80 table TABLE1 1.1.1.1/32 table TABLE1 1.0.0.0/16 table TABLE2 2.2.2.2/32 table TABLE3 3.3.3.3/32 " and we want to _atomically_ change this to " 100 allow ip from table(TABLE1) to me +200 allow ip from table(TABLE4) to any 300 allow ip from table(TABLE2) to (TABLE3) 80 table TABLE1 1.1.1.1/32 -table TABLE1 1.0.0.0/16 -table TABLE2 2.2.2.2/32 +table TABLE2 77.77.77.0/24 table TABLE3 3.3.3.3/32 +table TABLE4 4.4.4.4/32 " > >>> So here is what i would propose: >>> - [gradually] introduce new commands that accept strings for every place where >>> a number was previously accepted. Internally in the firewall, the old >>> 16-bit number is interpreted as a string. >> Yup. >>> - within the kernel create a small set of functions (invoked on insert, list, delete) >>> that do proper checks on the string and translate it to a pointer (or short integer, >>> i.e. an index in an array) to the proper object. Done properly, we can reuse >>> this code also for pipes, sets, nat groups and whatnot. >> Yup. >>> When a rule references an object that does not exist, you create an empty >>> object that a subsequent table/pipe/XXX 'create' would initialize. >>> On 'destroy', no need to touch the ruleset, just clear the object. >> Yes. This looks better than requiring user to create table of given type >> _before_ >> referencing it. >>> - for renames, try to follow the approach used for sets i.e. >>> ipfw table rename old-name new-name >>> changes the name, preserves the object. >>> Does not touch the ruleset, only the translation table >> Well, I'm not sure about renaming. I'd prefer permitting renaming iff >> table is not referenced. >> (and why do we need to rename tables?) > you are right, let's forget it. And 'ipfw set move' actually does a different thing > which has no use here. > >>> ipfw table swap first-table second-table >>> atomically swap the content of the two table objects >>> (once again not touching the rulesets) > cheers > luigi > From owner-freebsd-net@FreeBSD.ORG Thu May 22 15:34:55 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36BE94EB; Thu, 22 May 2014 15:34:55 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDB672E10; Thu, 22 May 2014 15:34:54 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WnR6m-000KZo-P0; Thu, 22 May 2014 15:24:24 +0400 Message-ID: <537E18D3.2010201@FreeBSD.org> Date: Thu, 22 May 2014 19:33:39 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> In-Reply-To: <537E1029.70007@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 15:34:55 -0000 On 22.05.2014 18:56, Alexander V. Chernikov wrote: It looks like we have reached some kind of consensus on table naming, so I'm going to implement the following as the first part: * named-only tables, no "user-visible" indexes * Keep the same opcodes, use additional TLVs to pass names in rules * Use explicit userland object names retrieval while listing * Make the previous ones easily extendable for other ipfw objects * Introduce table references and explicit typecasting (while permitting user to refernce non-existing tables) * leave table atomics for one the next stages Are you OK with this? > On 22.05.2014 00:48, Luigi Rizzo wrote: >> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: >>> On 21.05.2014 15:10, Luigi Rizzo wrote: >>>> On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov >>>> wrote: >>>>> Hello list! >>>>> >>>>> This patch adds ability to name tables / reference them by name. >>>>> Additionally, it simplifies adding new table types. >>>> Hi Alex, >>>> at a high level, i think your changes here are in the right direction. >>> Hello! >>>> However i think part of the design, and also of the patch below, >>>> is not sound/correct so i would like to wait to commit at least >>>> until some of the problems below are resolved. >>>> >>>> 1. The patch as is has several issues (variable declarations in the >>>> middle of a block, assignment in conditionals, incorrect >>>> comments in cut&pasted code, missing checks on strings) >>>> that should be corrected. >>> ..missing documentation and so on :) >>> Of course, I'll fix all these (or most of these :)) >> good thanks >> >>>> 3. there is no explanation, here or in the code, on how the >>>> names are managed. I could try to figure out from the code >>>> but it would be the wrong way to understand things so let me >>>> ask, and please explain what you have in mind. >>> Currently it is very simple non-resizable hash table with fnv hash >>> based >>> on table name. >> that is not an issue. The question is whether one needs to lookup >> the hash table every time you have a 'table' argument (of course i >> think one should not, and the implementation i propose below gives >> direct access to the table without name lookups, as the internal >> identifier is still poiniter or small integer, just one that is not >> exposed >> to userspace). >> >>>> Let me address first the name <-> table-id thing. >>>> >>>> Introducing a symbolic name for tables is a great and useful feature. >>>> However the implementation has some tricky issues: >>>> >>>> - do you want to retain the old numeric identifiers or not ? >>>> I think it is a bad idea to have two namespaces for the same >>>> thing, >>>> so if we are switching to alphanumeric strings for tables we >>>> should >>>> as well get rid of the numbers (i.e. consider them as strings >>>> as well). >>>> >>>> I am strongly in favour of not using names as aliases for numbers. >>>> It would require no changes for clients issuing ipfw commands >>>> from a script, and would not require users to to manually handle >>>> the name-id translations. >>> Well. I'd prefer not to. However, code we're discussing assumes that >>> numeric ids >>> are primary ones (e.g. you can't have named, but unnumbered table, >>> you have to choose number by yourself). Switching to named-only tables >>> can be tricky since we don't want to match them by inside rules and we >>> don't want >>> to loose atomicity when allocating table ids via separate cmds before >>> adding rule. >> i think this is solved by the implementation i proposed below. >> >>> It looks like we can do the following: >>> 1) Add another IP_FW3_ADD opcode with the following layout: >>> >>> { >>> rule len; >>> unmodified rule itself; >>> tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} >>> tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} >>> .. >>> } >>> Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } >>> won't be >>> real values >>> but values described in attached TLVs. >>> >>> After validating/parsing/checking under UH lock we replace fake values >>> with real ones (probably, newly allocated) >>> and return or rollback atomically. >> yes this should work, probably not even requiring a new IP_FW3_ADD >> opcode >> because the extra tlv entries can appear in a block by themselves. >> >>> The same thing can be done for displaying ruleset, however I'd prefer >>> client to ask for table names and caching them while displaying. >>> >>> Old clients will retain the ability to operate on tables but will see >>> table ids, so nothing will change for them >>> (except the case when new binary adds table "5" and new binary sees id >>> which is not 5, but that shouldn't be a problem). >> we can solve this by using 'low' numbers for the numeric tables >> (these were limited anyways) and allocate the fake entries in >> another range. > Currently we have u16 space available in base opcode. > Introducing another range will require additional opcode, additional > array for new tables and so on. > I'd prefer not to do this. Since table ids are allocated by ourselves > we can (and should) pack them > efficiently and 65k _real tables_ currently available is a quite good > value. > > We preserve compability for old binaries so people with old userland > and scripts should > not not notice anything. We have no public userland API so the only > base binary is /sbin/ipfw which is either old or new. > >> >>>> - The rename command worries me a lot: >>>> >>>> > ipfw table name XXX >>>> > Names (or renames) table. Not the name has to be unique. >>>> >>>> because it is neither clear nor intuitive whether you want to >>>> 1. just rename a table (possibly breaking or creating references >>>> for rules in the firewall), or >>>> 2. modify the name-id translation preserving existing pointers. >>>> >>>> Consider the sequence >>>> ipfw table 13 name bar >>>> ipfw add 100 count dst-ip table bar >>>> ipfw table 13 name foo >>>> ipfw table 14 name bar >>>> ipfw add 200 count src-port 22 dst-ip table bar >>>> >>>> Approach #1 would detach rule 100 from table 13 and then >>>> connect to 14 >>>> (in a non-atomic way), whereas approach #2 would make rule 100 >>>> and 200 >>>> point to different tables (which is confusing). >>>> >>>> Now part of this problem goes away if we get rid of number >>>> altogether. >>>> >>>> You may legitimately want to swap tables in an atomic way (we >>>> have something >>>> similar for rulesets): >>>> ipfw set swap number number >>> There is some problem here: >>> Atomic ruleset changes is a great thing, but tables have no atomic >>> support. >>> We, for example, are solving this by using different table range on >>> every new configuration. >>> It won't be possible with named-only tables (since names usually care >>> some semantic in contrast to numbers). >>> The only thing I can think of is separate namespace per each set. >>> What do you think? >> maybe i am missing some detail but it seems reasonably easy to implement >> the atomic swap -- and the use case is when you want to move from >> one configuration to a new one: >> ipfw table foo-new flush // clear initial content >> ipfw table foo-new add ... >> ipfw table swap foo-current foo-new // swap the content of the >> table objects >> >> so you preserve the semantic of the name very easily. > Yes. We can easily add atomic table swap that way. However, I'm > talking about different use scenario: > Atomically swap entire ruleset which has some tables depency: > > > e.g. we have: > > " > 100 allow ip from table(TABLE1) to me > 200 allow ip from table(TABLE2) to (TABLE3) 80 > > table TABLE1 1.1.1.1/32 > table TABLE1 1.0.0.0/16 > > table TABLE2 2.2.2.2/32 > > table TABLE3 3.3.3.3/32 > " > and we want to _atomically_ change this to > > " > 100 allow ip from table(TABLE1) to me > +200 allow ip from table(TABLE4) to any > 300 allow ip from table(TABLE2) to (TABLE3) 80 > > table TABLE1 1.1.1.1/32 > -table TABLE1 1.0.0.0/16 > > -table TABLE2 2.2.2.2/32 > +table TABLE2 77.77.77.0/24 > > table TABLE3 3.3.3.3/32 > > +table TABLE4 4.4.4.4/32 > " > >> >>>> So here is what i would propose: >>>> - [gradually] introduce new commands that accept strings for every >>>> place where >>>> a number was previously accepted. Internally in the firewall, >>>> the old >>>> 16-bit number is interpreted as a string. >>> Yup. >>>> - within the kernel create a small set of functions (invoked on >>>> insert, list, delete) >>>> that do proper checks on the string and translate it to a >>>> pointer (or short integer, >>>> i.e. an index in an array) to the proper object. Done properly, >>>> we can reuse >>>> this code also for pipes, sets, nat groups and whatnot. >>> Yup. >>>> When a rule references an object that does not exist, you >>>> create an empty >>>> object that a subsequent table/pipe/XXX 'create' would initialize. >>>> On 'destroy', no need to touch the ruleset, just clear the object. >>> Yes. This looks better than requiring user to create table of given >>> type >>> _before_ >>> referencing it. >>>> - for renames, try to follow the approach used for sets i.e. >>>> ipfw table rename old-name new-name >>>> changes the name, preserves the object. >>>> Does not touch the ruleset, only the translation table >>> Well, I'm not sure about renaming. I'd prefer permitting renaming iff >>> table is not referenced. >>> (and why do we need to rename tables?) >> you are right, let's forget it. And 'ipfw set move' actually does a >> different thing >> which has no use here. >> >>>> ipfw table swap first-table second-table >>>> atomically swap the content of the two table objects >>>> (once again not touching the rulesets) >> cheers >> luigi >> > From owner-freebsd-net@FreeBSD.ORG Thu May 22 15:43:22 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C933D8A3; Thu, 22 May 2014 15:43:22 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 8A6902F06; Thu, 22 May 2014 15:43:21 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id B73967300A; Thu, 22 May 2014 17:47:40 +0200 (CEST) Date: Thu, 22 May 2014 17:47:40 +0200 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140522154740.GA76448@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <537E1029.70007@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 15:43:22 -0000 On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov wrote: > On 22.05.2014 00:48, Luigi Rizzo wrote: > > On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: ... > > we can solve this by using 'low' numbers for the numeric tables > > (these were limited anyways) and allocate the fake entries in > > another range. > Currently we have u16 space available in base opcode. yes but the standard range for tables is much more limited: net.inet.ip.fw.tables_max: 128 so one can just (say) use 32k for "old" tables and the rest for tables with non numeric names. Does not seem to be a problem in practice. > > maybe i am missing some detail but it seems reasonably easy to implement > > the atomic swap -- and the use case is when you want to move from > > one configuration to a new one: > > ipfw table foo-new flush // clear initial content > > ipfw table foo-new add ... > > ipfw table swap foo-current foo-new // swap the content of the table objects > > > > so you preserve the semantic of the name very easily. > Yes. We can easily add atomic table swap that way. However, I'm talking > about different use scenario: > Atomically swap entire ruleset which has some tables depency: > > > e.g. we have: > > " > 100 allow ip from table(TABLE1) to me > 200 allow ip from table(TABLE2) to (TABLE3) 80 > > table TABLE1 1.1.1.1/32 > table TABLE1 1.0.0.0/16 > > table TABLE2 2.2.2.2/32 > > table TABLE3 3.3.3.3/32 > " > and we want to _atomically_ change this to > > " > 100 allow ip from table(TABLE1) to me > +200 allow ip from table(TABLE4) to any > 300 allow ip from table(TABLE2) to (TABLE3) 80 > > table TABLE1 1.1.1.1/32 > -table TABLE1 1.0.0.0/16 > > -table TABLE2 2.2.2.2/32 > +table TABLE2 77.77.77.0/24 > > table TABLE3 3.3.3.3/32 > > +table TABLE4 4.4.4.4/32 > " aargh, that's too much -- because between changing one table and all tables there are infinite intermediate points that all make sense. For those cases i think the way to go could be to insert a 'disabled' new ruleset (however complex it is, so it covers all possible cases), and then do the set swap, or disable/enable. cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 22 15:46:18 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2A2EA50; Thu, 22 May 2014 15:46:18 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 832682F36; Thu, 22 May 2014 15:46:18 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 8A6B67300B; Thu, 22 May 2014 17:50:44 +0200 (CEST) Date: Thu, 22 May 2014 17:50:44 +0200 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140522155044.GB76448@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <537E18D3.2010201@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <537E18D3.2010201@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 15:46:18 -0000 On Thu, May 22, 2014 at 07:33:39PM +0400, Alexander V. Chernikov wrote: > On 22.05.2014 18:56, Alexander V. Chernikov wrote: > > It looks like we have reached some kind of consensus on table naming, > so I'm going to implement the following as the first part: > > * named-only tables, no "user-visible" indexes > * Keep the same opcodes, use additional TLVs to pass names in rules > * Use explicit userland object names retrieval while listing > * Make the previous ones easily extendable for other ipfw objects > * Introduce table references and explicit typecasting (while permitting > user to refernce non-existing tables) > > * leave table atomics for one the next stages > > > Are you OK with this? yes i think so, this seems a good plan. thanks for following up cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 22 16:11:11 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D38EF3D8; Thu, 22 May 2014 16:11:11 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95BFC21D7; Thu, 22 May 2014 16:11:11 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WnRfs-000KoN-Qe; Thu, 22 May 2014 16:00:40 +0400 Message-ID: <537E2153.1040005@FreeBSD.org> Date: Thu, 22 May 2014 20:09:55 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> In-Reply-To: <20140522154740.GA76448@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 16:11:12 -0000 On 22.05.2014 19:47, Luigi Rizzo wrote: > On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov wrote: >> On 22.05.2014 00:48, Luigi Rizzo wrote: >>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: > ... >>> we can solve this by using 'low' numbers for the numeric tables >>> (these were limited anyways) and allocate the fake entries in >>> another range. >> Currently we have u16 space available in base opcode. > yes but the standard range for tables is much more limited: > > net.inet.ip.fw.tables_max: 128 > > so one can just (say) use 32k for "old" tables and the rest > for tables with non numeric names. > Does not seem to be a problem in practice. Well, using upper 32k means that you set this default to 65k which consumes 256k of memory on 32-bit arch. Embedded people won't be very happy about this (and changing table numbers on resize would be a nightmare). > >>> maybe i am missing some detail but it seems reasonably easy to implement >>> the atomic swap -- and the use case is when you want to move from >>> one configuration to a new one: >>> ipfw table foo-new flush // clear initial content >>> ipfw table foo-new add ... >>> ipfw table swap foo-current foo-new // swap the content of the table objects >>> >>> so you preserve the semantic of the name very easily. >> Yes. We can easily add atomic table swap that way. However, I'm talking >> about different use scenario: >> Atomically swap entire ruleset which has some tables depency: >> >> >> e.g. we have: >> >> " >> 100 allow ip from table(TABLE1) to me >> 200 allow ip from table(TABLE2) to (TABLE3) 80 >> >> table TABLE1 1.1.1.1/32 >> table TABLE1 1.0.0.0/16 >> >> table TABLE2 2.2.2.2/32 >> >> table TABLE3 3.3.3.3/32 >> " >> and we want to _atomically_ change this to >> >> " >> 100 allow ip from table(TABLE1) to me >> +200 allow ip from table(TABLE4) to any >> 300 allow ip from table(TABLE2) to (TABLE3) 80 >> >> table TABLE1 1.1.1.1/32 >> -table TABLE1 1.0.0.0/16 >> >> -table TABLE2 2.2.2.2/32 >> +table TABLE2 77.77.77.0/24 >> >> table TABLE3 3.3.3.3/32 >> >> +table TABLE4 4.4.4.4/32 >> " > aargh, that's too much -- because between changing > one table and all tables there are infinite intermediate > points that all make sense. It depends. As I said before, we're currently solving this problem by adding new rules (to set X) referencing tables from different range (2048 tables per ruleset) and than doing swap. (And not being able to use named tables to store real names after implementing them is a bit discouraging). > > For those cases i think the way to go could be to > insert a 'disabled' new ruleset (however complex it is, > so it covers all possible cases), and then do the set swap, > or disable/enable. We can think of per-set arrays/namespaces of tables: so "ipfw add 100 set X allow ipfw from table(Y) to ..." will reference table Y in set X and "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". This behavior can break some users setups so we can provide sysctl/tunable to turn this off or on. > > cheers > luigi > From owner-freebsd-net@FreeBSD.ORG Thu May 22 16:33:47 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36600D89; Thu, 22 May 2014 16:33:47 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 8375423CC; Thu, 22 May 2014 16:33:46 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 3EAAA7300A; Thu, 22 May 2014 18:38:12 +0200 (CEST) Date: Thu, 22 May 2014 18:38:12 +0200 From: Luigi Rizzo To: "Alexander V. Chernikov" Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140522163812.GA77634@onelab2.iet.unipi.it> References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <537E2153.1040005@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 16:33:47 -0000 On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. Chernikov wrote: > On 22.05.2014 19:47, Luigi Rizzo wrote: > > On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov wrote: > >> On 22.05.2014 00:48, Luigi Rizzo wrote: > >>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: > > ... > >>> we can solve this by using 'low' numbers for the numeric tables > >>> (these were limited anyways) and allocate the fake entries in > >>> another range. > >> Currently we have u16 space available in base opcode. > > yes but the standard range for tables is much more limited: > > > > net.inet.ip.fw.tables_max: 128 > > > > so one can just (say) use 32k for "old" tables and the rest > > for tables with non numeric names. > > Does not seem to be a problem in practice. > Well, using upper 32k means that you set this default to 65k which > consumes 256k of memory on 32-bit arch. > Embedded people won't be very happy about this (and changing table > numbers on resize would be a nightmare). no no, this is an implementation detail but within the kernel you can just remap the 'old' and 'new' table identifiers to a single contiguous range. The only thing you need to do is that when you push identifiers up to userland, those with 'new' names will be mapped to the 32-64k range. Example: user first specifies tables "18, goodguys, 530, badguys" in the same rule /sbin/ipfw will generate these numbers: 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} The kernel will then do a lookup of those identifiers and 18: internal index 1, name "18" 32768: internal index 2, name "goodguys" 530: internal index 3, name "530" 32769: internal index 4, name "badguys" Then the next rule contains tables 1, badguys, 18 /sbin/ipfw generates 1, 32768, 18 ; tlv {32768:badguys} // note different from before Kernel looks up the names and remaps 1: internal index 5, name "1" 32768: internal index 4, name "badguys" 18: internal index 1, name "18" Finally when you do an 'ipfw show' the kernel will remap names between 1 and 32768 to themselves, and other names to 32768+ (or some other large number, say 40k and above) so as they are found. So the rules will be pushed up with 18, 40000, 530, 40001 1, 40001, 18 we can discusso the other details privately cheers luigi 1. first, the > > > >>> maybe i am missing some detail but it seems reasonably easy to implement > >>> the atomic swap -- and the use case is when you want to move from > >>> one configuration to a new one: > >>> ipfw table foo-new flush // clear initial content > >>> ipfw table foo-new add ... > >>> ipfw table swap foo-current foo-new // swap the content of the table objects > >>> > >>> so you preserve the semantic of the name very easily. > >> Yes. We can easily add atomic table swap that way. However, I'm talking > >> about different use scenario: > >> Atomically swap entire ruleset which has some tables depency: > >> > >> > >> e.g. we have: > >> > >> " > >> 100 allow ip from table(TABLE1) to me > >> 200 allow ip from table(TABLE2) to (TABLE3) 80 > >> > >> table TABLE1 1.1.1.1/32 > >> table TABLE1 1.0.0.0/16 > >> > >> table TABLE2 2.2.2.2/32 > >> > >> table TABLE3 3.3.3.3/32 > >> " > >> and we want to _atomically_ change this to > >> > >> " > >> 100 allow ip from table(TABLE1) to me > >> +200 allow ip from table(TABLE4) to any > >> 300 allow ip from table(TABLE2) to (TABLE3) 80 > >> > >> table TABLE1 1.1.1.1/32 > >> -table TABLE1 1.0.0.0/16 > >> > >> -table TABLE2 2.2.2.2/32 > >> +table TABLE2 77.77.77.0/24 > >> > >> table TABLE3 3.3.3.3/32 > >> > >> +table TABLE4 4.4.4.4/32 > >> " > > aargh, that's too much -- because between changing > > one table and all tables there are infinite intermediate > > points that all make sense. > It depends. As I said before, we're currently solving this problem by > adding new rules (to set X) referencing tables from different range > (2048 tables per ruleset) and than doing swap. > (And not being able to use named tables to store real names after > implementing them is a bit discouraging). > > > > > For those cases i think the way to go could be to > > insert a 'disabled' new ruleset (however complex it is, > > so it covers all possible cases), and then do the set swap, > > or disable/enable. > We can think of per-set arrays/namespaces of tables: > > so "ipfw add 100 set X allow ipfw from table(Y) to ..." will reference > table Y in set X and > "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". > > This behavior can break some users setups so we can provide > sysctl/tunable to turn this off or on. > > > > > cheers > > luigi > > > From owner-freebsd-net@FreeBSD.ORG Thu May 22 14:46:14 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 298B57FF for ; Thu, 22 May 2014 14:46:14 +0000 (UTC) Received: from m50-110.126.com (m50-110.126.com [123.125.50.110]) by mx1.freebsd.org (Postfix) with ESMTP id 375722925 for ; Thu, 22 May 2014 14:46:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=From:Subject:Date:Message-ID:MIME-Version; bh=KZVMn UYbzspfyeRpejGxLWwv/Z2ktlkuUhWfD/pD8MI=; b=KqLqfHuQlEnFgxtiVawPX 9pMmqfrKhzk2q1xNSKe0r0X32JIx+xcMNWN0vjISZjccbBGKQAXoW8g0UsGcfH+S WcbZzBkRxsSd7yTwB+a4qaGVQgP2duZXwQrORhhApB0dQ8AfkJEd2tDm8wuCzZi6 jt/2DJEdKQKHnSZ5zP3GqA= Received: from dshPC (unknown [124.202.191.39]) by smtp4 (Coremail) with SMTP id jdKowEAZP1uMBn5TOpG1AQ--.252S2; Thu, 22 May 2014 22:15:41 +0800 (CST) From: "dongshan" To: Subject: netmap pkt_gen can't compile successfully Date: Thu, 22 May 2014 22:15:48 +0800 Message-ID: <001a01cf75c8$55586880$00093980$@126.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: Ac91yEdv9UA84reMTvil7bafpQAOTA== Content-Language: zh-cn X-CM-TRANSID: jdKowEAZP1uMBn5TOpG1AQ--.252S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7WF1kXw48Zr1UuFy5WF4xXrb_yoW8Cry7pF y3Jwn3Jr47ZrW7G348Kry0v3yFqr1kC3yavr4fZ395Ka4FgasakrWxKF4Y9rW2kr4DW3WD KFyftw1UK34DJaUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UEXdnUUUUU= X-Originating-IP: [124.202.191.39] X-CM-SenderInfo: 5wkrztxv1d0warsqlqqrswhudrp/1tbi1g6TxU0vMqzfDAAAsh X-Mailman-Approved-At: Thu, 22 May 2014 19:00:56 +0000 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 14:46:14 -0000 Hi, I have some problems in running netmap pkt_gen, the receiving mode seems okay, but the sending mode will cause segmentation fault. when i compile pkt_gen.c, it warns three times, in Makefile with CFLAGS -Werror, so the warning is treated as error, so i deleted -Werror, then it can make pkt_gen binary file, but int can't run successfully. I did it on x86 64 bit Linux In file included from pkt-gen.c:43:0: pkt-gen.c: In function 'main': ../sys/net/netmap_user.h:85:58: error: format '%lx' expects argument of type 'long unsigned int', but argument 8 has type 'int' [-Werror=format=] #define NETMAP_TXRING(nifp, index) _NETMAP_OFFSET(struct netmap_ring *, \ ^ ../sys/net/netmap_user.h:81:4: note: in definition of macro '_NETMAP_OFFSET' ((type)(void *)((char *)(ptr) + (offset))) ^ ../sys/net/netmap_user.h:155:33: note: in expansion of macro 'NETMAP_TXRING' __FUNCTION__, __LINE__, ##__VA_ARGS__); \ ^ pkt-gen.c:1825:4: note: in expansion of macro 'D' D(" TX%d at 0x%lx", i, ^ ../sys/net/netmap_user.h:88:58: error: format '%lx' expects argument of type 'long unsigned int', but argument 8 has type 'int' [-Werror=format=] #define NETMAP_RXRING(nifp, index) _NETMAP_OFFSET(struct netmap_ring *, \ ^ ../sys/net/netmap_user.h:81:4: note: in definition of macro '_NETMAP_OFFSET' ((type)(void *)((char *)(ptr) + (offset))) ^ ../sys/net/netmap_user.h:155:33: note: in expansion of macro 'NETMAP_RXRING' __FUNCTION__, __LINE__, ##__VA_ARGS__); \ ^ pkt-gen.c:1829:4: note: in expansion of macro 'D' D(" RX%d at 0x%lx", i, ^ cc1: all warnings being treated as errors make[1]: *** [pkt-gen.o] Error 1 make[1]: Leaving directory `/home/dsh/project/netmap/examples' make: *** [apps] Error 2 Did it happen to you? Best regards, Dongshan From owner-freebsd-net@FreeBSD.ORG Thu May 22 20:04:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97FDB5F4 for ; Thu, 22 May 2014 20:04:26 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 7A0342616 for ; Thu, 22 May 2014 20:04:25 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 477CF3ADD6 for ; Thu, 22 May 2014 12:57:31 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-net@freebsd.org Subject: Trivia: Puzzling eBay response (IP 10.2.98.245) Date: Thu, 22 May 2014 12:57:31 -0700 Message-ID: <26921.1400788651@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 20:04:26 -0000 For those of you who didn't already hear, eBay got hacked the other day... well... not actually the other day, but in February. But they elected to do the decent thing and actually tell all of their affected customers about it as soon as they found out about it. Well, actually, they found out about it two weeks ago, and only just made a press announcement about it yesterday, once they found out that the State of North Dakota has a law in place that would compel them not to hide this information, whwich is apparently what they would have preferred to do... sweep it all under the rug, and decency be damned. And, actually, as far as I know they still haven't even directly told any of their affecetd customers... welll... not me anyway. They did have the courtesy to tell the press about it, knowing, as they surely did, that they (the press) would find out about it eventually anyway. eBay is now encouraging all of their customers to change their passwords, which I dutifully did yesterday. After that, eBay sent me the following message, the last bit of which is rather entirely puzzling: =========================================================================== ----------------------------------------------------------------- eBay Change Password Confirmation ----------------------------------------------------------------- Dear Ron, This is a courtesy message to let you know that your eBay password has been successfully changed. No response is needed. If you did not make this change, please contact us at http://ocs.ebay.com/ws/eBayISAPI.dll?[REDACTED] and sign in as a guest. The password change request was made from: - IP address: 69.62.255.118 - ISP host: 10.2.98.245 =========================================================================== So, I mean, WTF? 69.62.255.118 is indeed my correct static IP address, and is indeed the place from whence I changed my password yesterday. I really do wonder where the bleep they got 10.2.98.245 from. Obviously, that's an RFC1918 address. I do suspect that that IP address has a lot more to do with them, and with the geography of their own internal network than it has to do with _my_ ISP. Regards, rfg P.S. Before I was allowed to change my password, I first had to confirm, just via the web site, that my e-mail address was correct. The text on the eBay was site said that this was needed so that they could confirm my password change via e-mail. I changed the password via the web site and _never_ received any sort of e-mail message asking me to confirm that change. I offer the observation that in the online world, there seems to be a lot that separates good intentions from actual competence. From owner-freebsd-net@FreeBSD.ORG Thu May 22 20:19:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02C89B19 for ; Thu, 22 May 2014 20:19:17 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B91A72735 for ; Thu, 22 May 2014 20:19:16 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id lf12so5138161vcb.1 for ; Thu, 22 May 2014 13:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=XJTfg0jkqrEeE1iwzpIaDA3U6ntpZiH18EGMRbtunOo=; b=CmhfSxZqc6/hwiiJjV0I96qgIny7eDzeKZevTcSqD7rJBqHFXJTD6KU9NgKav+s1gj Mjv1RbSgCFrnNnSWrUwfXKgmbtBMfXcBt+o6aktwe1/8CdDfOOp4nvbxBi6VHetCsg9G fR8LV3J2q+g9LxPAnky168JNL8pZdz/OSsrCsYSTjBf6q4P9qmgJmLVsBpORz+p0xYVO F61a4A9UPP2pI2wF6BijwN0jft/vgO1tZS7TTW/kJ7R48RfH/ezNNiMkqtfB3SwgATsZ PPpnGPYcN+L9SV/pny7gmQuVwoPL/qtajbqjIOo24PY8xw+SLWPw2RqNacVHrC+LHwMl iYLw== MIME-Version: 1.0 X-Received: by 10.221.59.194 with SMTP id wp2mr237567vcb.59.1400789955253; Thu, 22 May 2014 13:19:15 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 22 May 2014 13:19:15 -0700 (PDT) In-Reply-To: <26921.1400788651@server1.tristatelogic.com> References: <26921.1400788651@server1.tristatelogic.com> Date: Thu, 22 May 2014 13:19:15 -0700 Message-ID: Subject: Re: Trivia: Puzzling eBay response (IP 10.2.98.245) From: Kurt Buff To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 20:19:17 -0000 On Thu, May 22, 2014 at 12:57 PM, Ronald F. Guilmette wrote: > After that, eBay sent me the following message, the last bit of which > is rather entirely puzzling: > > =========================================================================== > ----------------------------------------------------------------- > eBay Change Password Confirmation > ----------------------------------------------------------------- > > Dear Ron, > > This is a courtesy message to let you know that your eBay password has been > successfully changed. No response is needed. > > If you did not make this change, please contact us at > http://ocs.ebay.com/ws/eBayISAPI.dll?[REDACTED] and sign in as a guest. > > The password change request was made from: > - IP address: 69.62.255.118 > - ISP host: 10.2.98.245 > =========================================================================== > > So, I mean, WTF? > > 69.62.255.118 is indeed my correct static IP address, and is indeed the > place from whence I changed my password yesterday. > > I really do wonder where the bleep they got 10.2.98.245 from. > > Obviously, that's an RFC1918 address. > > I do suspect that that IP address has a lot more to do with them, and with > the geography of their own internal network than it has to do with _my_ ISP. Rather than incompetence, I'd first suspect Carrier Grade NAT. Kurt From owner-freebsd-net@FreeBSD.ORG Thu May 22 20:49:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E662C8D for ; Thu, 22 May 2014 20:49:11 +0000 (UTC) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 8479F29D4 for ; Thu, 22 May 2014 20:49:10 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id C4C813ADD6 for ; Thu, 22 May 2014 13:49:10 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-net@freebsd.org Subject: Re: Trivia: Puzzling eBay response (IP 10.2.98.245) In-Reply-To: Date: Thu, 22 May 2014 13:49:10 -0700 Message-ID: <28274.1400791750@server1.tristatelogic.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 20:49:11 -0000 In message Kurt Buff wrote: >On Thu, May 22, 2014 at 12:57 PM, Ronald F. Guilmette >> The password change request was made from: >> - IP address: 69.62.255.118 >> - ISP host: 10.2.98.245 >> =========================================================================== >> >> So, I mean, WTF? >> >> 69.62.255.118 is indeed my correct static IP address, and is indeed the >> place from whence I changed my password yesterday. >> >> I really do wonder where the bleep they got 10.2.98.245 from. >> >> Obviously, that's an RFC1918 address. >> >> I do suspect that that IP address has a lot more to do with them, and with >> the geography of their own internal network than it has to do with _my_ ISP. > >Rather than incompetence, I'd first suspect Carrier Grade NAT. Please elaborate. What is it, exactly, that you are suggesting actually has been assigned the RFC1918 address in question? Some box on eBay's network? Some box on my ISPs network? If the latter, then how exactly does this RFC1918 address end up getting routed to eBay? From owner-freebsd-net@FreeBSD.ORG Thu May 22 21:01:02 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70AAA4B9; Thu, 22 May 2014 21:01:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 455B62AE9; Thu, 22 May 2014 21:01:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4ML12ND061490; Thu, 22 May 2014 21:01:02 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4ML122N061489; Thu, 22 May 2014 21:01:02 GMT (envelope-from linimon) Date: Thu, 22 May 2014 21:01:02 GMT Message-Id: <201405222101.s4ML122N061489@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 21:01:02 -0000 Old Synopsis: net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ New Synopsis: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu May 22 21:00:27 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=190102 From owner-freebsd-net@FreeBSD.ORG Thu May 22 21:55:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D5E86C5 for ; Thu, 22 May 2014 21:55:47 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DD9B2FA2 for ; Thu, 22 May 2014 21:55:47 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id ik5so1859866vcb.31 for ; Thu, 22 May 2014 14:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=przZPiNSg05W0TjpspOIZ7DSm0hqxzJOU1e9yt+zxv0=; b=KiIQxEi45WRTAi5lrZVLkOk1p+kBqTGJj1Igh8g2TuSnf82lL0pLIDVZxHcCJjoPti ynIKcJFCBpNQqhBZ3llnqUYoTT7HfFOZt+mUEWtFsB8saYOqDvVlLuWnAe6vPzkPgJoP /oz82duylo2QRMGxFIE44dR6o8DOVuyIkKY57N0rf+94B8OvVzypazOivfuB8clsCaT5 dmyRWxr5lIZf47HDnPkDpH3TMa3jI0EJg35jWAxCv996Ew014hoMA6miHPk0iaejRR+z 8ZvFM1Ng5b2hY+QouGNLcVdOswVnAaUUVLFsNmx4hS/cXnk+KfS5ABCkyS9la0pyl7Eu Sy6w== MIME-Version: 1.0 X-Received: by 10.52.72.4 with SMTP id z4mr241928vdu.71.1400795746507; Thu, 22 May 2014 14:55:46 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 22 May 2014 14:55:46 -0700 (PDT) In-Reply-To: <28274.1400791750@server1.tristatelogic.com> References: <28274.1400791750@server1.tristatelogic.com> Date: Thu, 22 May 2014 14:55:46 -0700 Message-ID: Subject: Re: Trivia: Puzzling eBay response (IP 10.2.98.245) From: Kurt Buff To: "Ronald F. Guilmette" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 21:55:47 -0000 On Thu, May 22, 2014 at 1:49 PM, Ronald F. Guilmette wrote: > > In message > Kurt Buff wrote: > >>On Thu, May 22, 2014 at 12:57 PM, Ronald F. Guilmette >>> The password change request was made from: >>> - IP address: 69.62.255.118 >>> - ISP host: 10.2.98.245 >>> =========================================================================== >>> >>> So, I mean, WTF? >>> >>> 69.62.255.118 is indeed my correct static IP address, and is indeed the >>> place from whence I changed my password yesterday. >>> >>> I really do wonder where the bleep they got 10.2.98.245 from. >>> >>> Obviously, that's an RFC1918 address. >>> >>> I do suspect that that IP address has a lot more to do with them, and with >>> the geography of their own internal network than it has to do with _my_ ISP. >> >>Rather than incompetence, I'd first suspect Carrier Grade NAT. > > Please elaborate. What is it, exactly, that you are suggesting actually > has been assigned the RFC1918 address in question? > > Some box on eBay's network? > > Some box on my ISPs network? > > If the latter, then how exactly does this RFC1918 address end up getting > routed to eBay? I expect eBay performed a traceroute to your public address, and that a box on your ISP's network has the 10.2.98.245 address and emitted it as the last hop before your public address. That's far from certain, but it's the first thing I'd think. Kurt From owner-freebsd-net@FreeBSD.ORG Fri May 23 12:07:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23CB6356 for ; Fri, 23 May 2014 12:07:17 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A24A924E9 for ; Fri, 23 May 2014 12:07:16 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WnoFg-0003Hf-4p for freebsd-net@freebsd.org; Fri, 23 May 2014 14:07:08 +0200 Received: from h87.s239.verisign.com ([216.168.239.87]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 May 2014 14:07:08 +0200 Received: from jcharbon by h87.s239.verisign.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 May 2014 14:07:08 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Fri, 23 May 2014 14:06:55 +0200 Lines: 232 Message-ID: <537F39DF.1090900@verisign.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------030706050106030100090504" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: h87.s239.verisign.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: Cc: George Neville-Neil X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 12:07:17 -0000 This is a multi-part message in MIME format. --------------030706050106030100090504 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, On 27/02/14 11:32, Julien Charbon wrote: > On 07/11/13 14:55, Julien Charbon wrote: >> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >> wrote: >>> I have put technical and how-to-repeat details in below PR: >>> >>> kern/183659: TCP stack lock contention with short-lived connections >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>> >>> We are currently working on this performance improvement effort; it >>> will impact only the TCP locking strategy not the TCP stack logic >>> itself. We will share on freebsd-net the patches we made for >>> reviewing and improvement propositions; anyway this change might also >>> require enough eyeballs to avoid tricky race conditions introduction >>> in TCP stack. > > [...] > > 2. We studied an another lock contention related to INP_INFO when TCP > connections in TIME_WAIT state are cleaned-up. [...] First a follow-up on TCP performance improvements and BSDCan 2014 discussions: - First, changes related to TCP lock connection and TIME_WAIT list have been reviewed, fixed, improved and accepted in HEAD: http://svnweb.freebsd.org/base?view=revision&revision=264321 http://svnweb.freebsd.org/base?view=revision&revision=264342 http://svnweb.freebsd.org/base?view=revision&revision=264351 http://svnweb.freebsd.org/base?view=revision&revision=264356 Thanks to testers, reviewers and committers (jhb, rwatson, rrs, adrian, glebius, gnn, bde, jmg, rrs, etc.). - Second, it has been discussed in BSDCan that it would nice to also share Verisign patches in public github. This is done here: https://github.com/verisign/freebsd We currently maintain two branches: - One that follows 10.0-RELENG branch: https://github.com/verisign/freebsd/commits/share/10.0-next - One that follows HEAD: https://github.com/verisign/freebsd/commits/share/head-next These branches are cloned from on the handy github FreeBSD repository mirror: https://github.com/freebsd/freebsd We pushed our stable enough TCP related patches in this repo. - Third, joined a patch (tcp-scale-syn-v1.patch) we would propose for review. Obviously, it is also available here: https://github.com/verisign/freebsd/commit/38679f5f66b470d069d635bfc8d9ec6c39eb787b This patch is based (like most of our TCP changes) on Robert Watson major change to decompose inpcbinfo lock (aka ipi_lock or INP_INFO:) Decompose the current single inpcbinfo lock into two locks: http://svnweb.freebsd.org/base?view=revision&revision=222488 https://github.com/freebsd/freebsd/commit/fdfdadb612a4b077d62094c7d4aa65de3524cf62 In particular on this comment: " - The TCP syncache still relies on the pcbinfo lock, something that we may want to revisit. " Basically this change prevents to take the TCP ipi_lock lock when a SYN targeting a listening socket is received. We expected a need to add another syncache mutex to take over ipi_lock role, but surprisingly we don't find any requirements for a new mutex (and it doesn't mean that this requirement doesn't exist). This patch is quite stable under our workload, improves only a bit the short-lived TCP connection rate (~+2%), however it can become useful under SYN flood attack pressure. We still have to test more thoroughly this patch in two contexts: - VNET and, - syncache.hashsize sets very low relative to TCP workload As usual, tests, reviews and comments more than welcome. Thanks. -- Julien --------------030706050106030100090504 Content-Type: text/plain; charset=UTF-8; name="tcp-scale-syn-v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tcp-scale-syn-v1.patch" diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c index f9a751c..4b6f41f 100644 --- a/sys/netinet/tcp_input.c +++ b/sys/netinet/tcp_input.c @@ -588,6 +588,7 @@ tcp_input(struct mbuf *m, int off0) #endif /* INET6 */ struct tcpopt to; /* options in this segment */ char *s = NULL; /* address and port logging */ + int needlock; int ti_locked; #define TI_UNLOCKED 1 #define TI_WLOCKED 2 @@ -770,12 +771,12 @@ tcp_input(struct mbuf *m, int off0) /* * Locate pcb for segment; if we're likely to add or remove a - * connection then first acquire pcbinfo lock. There are two cases + * connection then first acquire pcbinfo lock. There are three cases * where we might discover later we need a write lock despite the * flags: ACKs moving a connection out of the syncache, and ACKs for - * a connection in TIMEWAIT. + * a connection in TIMEWAIT, SYNs for a non-listening socket. */ - if ((thflags & (TH_SYN | TH_FIN | TH_RST)) != 0) { + if ((thflags & (TH_FIN | TH_RST)) != 0) { INP_INFO_WLOCK(&V_tcbinfo); ti_locked = TI_WLOCKED; } else @@ -1004,10 +1005,18 @@ tcp_input(struct mbuf *m, int off0) * now be in TIMEWAIT. */ #ifdef INVARIANTS - if ((thflags & (TH_SYN | TH_FIN | TH_RST)) != 0) + if ((thflags & (TH_FIN | TH_RST)) != 0) INP_INFO_WLOCK_ASSERT(&V_tcbinfo); #endif - if (tp->t_state != TCPS_ESTABLISHED) { + needlock = 1; + if (tp->t_state == TCPS_ESTABLISHED) { + if ((thflags & TH_SYN) == 0) + needlock = 0; + } else { + if (tp->t_state == TCPS_LISTEN && (thflags & TH_SYN)) + needlock = 0; + } + if (needlock) { if (ti_locked == TI_UNLOCKED) { if (INP_INFO_TRY_WLOCK(&V_tcbinfo) == 0) { in_pcbref(inp); @@ -1048,17 +1057,13 @@ tcp_input(struct mbuf *m, int off0) /* * When the socket is accepting connections (the INPCB is in LISTEN * state) we look into the SYN cache if this is a new connection - * attempt or the completion of a previous one. Because listen - * sockets are never in TCPS_ESTABLISHED, the V_tcbinfo lock will be - * held in this case. + * attempt or the completion of a previous one. */ if (so->so_options & SO_ACCEPTCONN) { struct in_conninfo inc; KASSERT(tp->t_state == TCPS_LISTEN, ("%s: so accepting but " "tp not listening", __func__)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); - bzero(&inc, sizeof(inc)); #ifdef INET6 if (isipv6) { @@ -1081,6 +1086,8 @@ tcp_input(struct mbuf *m, int off0) * socket appended to the listen queue in SYN_RECEIVED state. */ if ((thflags & (TH_RST|TH_ACK|TH_SYN)) == TH_ACK) { + + INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* * Parse the TCP options here because * syncookies need access to the reflected @@ -1361,8 +1368,12 @@ tcp_input(struct mbuf *m, int off0) syncache_add(&inc, &to, th, inp, &so, m, NULL, NULL); /* * Entry added to syncache and mbuf consumed. - * Everything already unlocked by syncache_add(). + * Nothing is unlocked by syncache_add(). */ + if (ti_locked == TI_WLOCKED) { + INP_INFO_WUNLOCK(&V_tcbinfo); + ti_locked = TI_UNLOCKED; + } INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); return; } else if (tp->t_state == TCPS_LISTEN) { diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c index beb6749..9b981d3 100644 --- a/sys/netinet/tcp_syncache.c +++ b/sys/netinet/tcp_syncache.c @@ -1119,7 +1119,6 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, struct syncache scs; struct ucred *cred; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); /* listen socket */ KASSERT((th->th_flags & (TH_RST|TH_ACK|TH_SYN)) == TH_SYN, ("%s: unexpected tcp flags", __func__)); @@ -1150,13 +1149,11 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, #ifdef MAC if (mac_syncache_init(&maclabel) != 0) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); goto done; } else mac_syncache_create(maclabel, inp); #endif INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); /* * Remember the IP options, if any. --------------030706050106030100090504-- From owner-freebsd-net@FreeBSD.ORG Fri May 23 13:18:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BE0E9C3 for ; Fri, 23 May 2014 13:18:22 +0000 (UTC) Received: from mx7.valuehost.ru (mx7.valuehost.ru [217.112.42.214]) by mx1.freebsd.org (Postfix) with ESMTP id 3EBD02BC1 for ; Fri, 23 May 2014 13:18:21 +0000 (UTC) Received: from mx7.valuehost.ru (localhost.valuehost.ru [127.0.0.1]) by mx7.valuehost.ru (Postfix) with ESMTP id 408862E98B for ; Fri, 23 May 2014 17:18:31 +0400 (MSK) Date: Fri, 23 May 2014 17:18:20 +0400 From: "Peter B. Pokryshev" To: freebsd-net@freebsd.org Subject: mbuf_cluster (FAIL SLEEP) Message-Id: <20140523171820.e631082d8015136ac052fdbd@valuehost.ru> Organization: ValueHost X-Mailer: Sylpheed 3.4.0beta5 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 13:18:22 -0000 Hi. Is it normal after 16 days of uptime: # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP ... 16 Bucket: 152, 0, 24, 101, 193, 0, 0 32 Bucket: 280, 0, 38, 102, 329, 2, 0 64 Bucket: 536, 0, 30, 33, 487, 142, 0 128 Bucket: 1048, 0, 997, 11, 6717030,17345735, 0 ... mbuf_packet: 256, 12896820, 1449, 1646,9062649837,118865, 0 mbuf: 256, 12896820, 2193, 1762,17686258507, 0, 0 mbuf_cluster: 2048, 2015128, 3095, 1793,26759484,241537,1100807 mbuf_jumbo_page: 4096, 1007563, 2160, 864,2326876443, 0, 0 mbuf_jumbo_9k: 9216, 298537, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 167927, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 I mean 128 Bucket (FAIL) and mbuf_cluster (FAIL SLEEP) # uname -a 9.2-STABLE FreeBSD 9.2-STABLE #0 r265433 # netstat -m 3099/3951/7050 mbufs in use (current/cache/total) 1392/3496/4888/2015128 mbuf clusters in use (current/cache/total/max) 1392/1703 mbuf+clusters out of packet secondary zone in use (current/cache) 1679/1345/3024/1007563 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/298537 9k jumbo clusters in use (current/cache/total/max) 0/0/0/167927 16k jumbo clusters in use (current/cache/total/max) 10274K/13359K/23634K bytes allocated to network (current/cache/total) 0/241537/118865 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/1100807/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines # netstat -an |wc -l 29616 # netstat -an | grep -c TIME 27800 + kernel stops showing realtime LA: show normal top, but no changes at first line output: load averages: 2.36, 2.32, 2.28 ^ ^ ^ | | | May be some problem with counters or timers? -- Peter B. Pokryshev From owner-freebsd-net@FreeBSD.ORG Fri May 23 13:35:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05301F32 for ; Fri, 23 May 2014 13:35:03 +0000 (UTC) Received: from dss.incore.de (dss.incore.de [195.145.1.138]) by mx1.freebsd.org (Postfix) with ESMTP id 85C962D87 for ; Fri, 23 May 2014 13:35:02 +0000 (UTC) Received: from inetmail.dmz (inetmail_new [10.3.0.4]) by dss.incore.de (Postfix) with ESMTP id 80EFE5C02D for ; Fri, 23 May 2014 15:27:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at incore.de Received: from dss.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.4]) (amavisd-new, port 10024) with LMTP id k-bhUhuyrAqK for ; Fri, 23 May 2014 15:27:50 +0200 (CEST) Received: from mail.incore (fwintern.dmz [10.0.0.253]) by dss.incore.de (Postfix) with ESMTP id D5D945C024 for ; Fri, 23 May 2014 15:27:50 +0200 (CEST) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.incore (Postfix) with ESMTP id C23F950871 for ; Fri, 23 May 2014 15:27:50 +0200 (CEST) Message-ID: <537F4CD6.8010305@incore.de> Date: Fri, 23 May 2014 15:27:50 +0200 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Support for IPSec VPN's with XAuth (tunnel mode) and L2TP (transport mode) Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 13:35:03 -0000 Hi, as continuation of http://lists.freebsd.org/pipermail/freebsd-net/2012-September/033170.html I like to give patches for two open problems: The first kernel patch of the link above is not needed anymore, vanhu pointed out: > It may be cleaner to have racoon generate the good SP entry, rather > than kernel trying to guess what is right in a SPDADD command. I agree , therefore I now use the following patch for racoon in the ipsec-tools-0.8.0_3 port: --- src/racoon/isakmp_quick.c.orig 2011-03-14 18:18:13.000000000 +0100 +++ src/racoon/isakmp_quick.c 2014-05-12 00:52:25.000000000 +0200 @@ -2012,6 +2012,13 @@ /* make inbound policy */ iph2->src = dst; iph2->dst = src; +#ifdef ENABLE_NATT + /* Without NAT the policies must include all ports. */ + if( !(iph2->ph1->natt_flags & NAT_DETECTED)) { + set_port(iph2->src, 0); + set_port(iph2->dst, 0); + } +#endif if (pk_sendspdupdate2(iph2) < 0) { plog(LLV_ERROR, LOCATION, NULL, "pfkey spdupdate2(inbound) failed.\n"); The other open task are IPSEC/L2TP (transport mode) connections from some clients behind the same NAT router. I do not longer try to solve this in the kernel and/or racoon, because it is a special "port 1701" problem. My propose is to handle this with the help of a connect-script introduced in mpd-5.6: --- src/l2tp.c.orig 2011-12-21 15:58:49.000000000 +0100 +++ src/l2tp.c 2014-05-14 11:39:24.000000000 +0200 @@ -67,6 +67,7 @@ struct u_range peer_addr; /* Peer IP addresses allowed */ in_port_t self_port; /* self port */ in_port_t peer_port; /* Peer port required (or zero) */ + char connect_script[IFACE_MAX_SCRIPT]; struct optinfo options; char callingnum[64]; /* L2TP phone number to use */ char callednum[64]; /* L2TP phone number to use */ @@ -90,6 +91,7 @@ enum { SET_SELFADDR, SET_PEERADDR, + SET_CONNECT_SCRIPT, SET_CALLINGNUM, SET_CALLEDNUM, SET_HOSTNAME, @@ -202,6 +204,8 @@ L2tpSetCommand, NULL, 2, (void *) SET_SELFADDR }, { "peer {ip} [{port}]", "Set remote IP address", L2tpSetCommand, NULL, 2, (void *) SET_PEERADDR }, + { "connect-script [{progname}]", "Set connect script", + L2tpSetCommand, NULL, 2, (void *) SET_CONNECT_SCRIPT }, { "callingnum {number}", "Set calling L2TP telephone number", L2tpSetCommand, NULL, 2, (void *) SET_CALLINGNUM }, { "callednum {number}", "Set called L2TP telephone number", @@ -913,6 +917,8 @@ Printf(", port %u", l2tp->conf.peer_port); Printf("\r\n"); Printf("\tHostname : %s\r\n", l2tp->conf.hostname); + Printf("\t connect-script: \"%s\"\r\n", + *l2tp->conf.connect_script ? l2tp->conf.connect_script : ""); Printf("\tSecret : %s\r\n", (l2tp->conf.callingnum[0])?"******":""); Printf("\tCalling number: %s\r\n", l2tp->conf.callingnum); Printf("\tCalled number: %s\r\n", l2tp->conf.callednum); @@ -1438,6 +1444,22 @@ goto fail; } + /* Call "connect" script */ + if (*pi->conf.connect_script) { + char selfbuf[40],peerbuf[40]; + int res; + + res = ExecCmd(LG_IFACE2, "label", "%s %s %d %s %d", + pi->conf.connect_script, + u_addrtoa(&tun->self_addr, selfbuf, sizeof(selfbuf)),tun->self_port, + u_addrtoa(&tun->peer_addr, peerbuf, sizeof(peerbuf)), + tun->peer_port); + if (res != 0) { + Log(LG_PHYS, ("L2TP: Error running connect-script")); + goto fail; + } + } + /* Create vendor name AVP */ avps = ppp_l2tp_avp_list_create(); @@ -1747,6 +1769,18 @@ l2tp->conf.peer_port = port; } break; + case SET_CONNECT_SCRIPT: + switch (ac) { + case 0: + *l2tp->conf.connect_script = 0; + break; + case 1: + strlcpy(l2tp->conf.connect_script, av[0], sizeof(l2tp->conf.connect_script)); + break; + default: + return(-1); + } + break; case SET_CALLINGNUM: if (ac != 1) return(-1); Now in mpd.conf a connect-script can be defined which will be called for every new incoming L2TP packet: set l2tp connect-script /usr/local/etc/mpd5/l2tpConnect In /etc/ipsec.conf I only the policy for incoming packets is defined: spdadd 0.0.0.0/0[0] xx.xx.xx.xx[1701] udp -P in ipsec esp/transport//require; The policy for outgoing packets is created on the fly by the l2tpConnect script, I give an example thats working for me: set -- $* local_ip=$1 local_port=$2 # always 1701 remote_ip=$3 remote_port=$4 checkSAs() { status=0; while read line; do set -- ${line} param1="$1"; param2="$2"; param3="$3" case ${status} in 0) var1=${param1%[*}; var2=${param2%[*} if [ "${var1}" = "${local_ip}" -a "${var2}" = "${remote_ip}" ]; then dir=outbound; status=1 if [ "${param1}" = "${var1}" ]; then nat=0; port="500" else nat=1; var2=${param2%]}; port=${var2#*[} fi fi if [ "${var2}" = "${local_ip}" -a "${var1}" = "${remote_ip}" ]; then dir=inbound; status=1 fi ;; 1) if [ "${param2}" != "mode=transport" ]; then status=0 else var="${param3}"; var=${var%(*}; var=${var#*=} status=2 fi ;; 2) if [ "${param1}" = "allocated:" ]; then status=0 if [ "${param2}" = "0" -a "${dir}" = "outbound" ]; then SA_remote_port=${port}; SA_nat=${nat} fi fi ;; esac done echo "${SA_nat} ${SA_remote_port}" } SA=$(/usr/local/sbin/setkey -D | checkSAs) set -- ${SA} SA_nat=$1; SA_remote_port=$2 if [ ${SA_nat} = "1" ]; then src_dst="${local_ip}[4500]-${remote_ip}[${SA_remote_port}]" else src_dst="${local_ip}-${remote_ip}" fi echo "spdadd ${local_ip}[${local_port}] ${remote_ip}[${remote_port}] udp -P out ipsec esp/transport/${src_dst}/require;" | /usr/local/sbin/setkey -c -- Andreas Longwitz From owner-freebsd-net@FreeBSD.ORG Fri May 23 16:47:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8C85AE7 for ; Fri, 23 May 2014 16:47:02 +0000 (UTC) Received: from smtp2.wemm.org (smtp2.wemm.org [192.203.228.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp2.wemm.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F0712FF2 for ; Fri, 23 May 2014 16:47:02 +0000 (UTC) Received: from [172.16.21.76] (50-204-120-225-static.hfc.comcastbusiness.net [50.204.120.225]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: peter) by smtp2.wemm.org (Postfix) with ESMTPSA id 86F19E6 for ; Fri, 23 May 2014 09:47:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=m20140428; t=1400863620; bh=zvQEsijk/iDKr+9O6g/AUgB2nKva/koIIAsMy3DqA4I=; h=Date:From:To:Subject:References:In-Reply-To; b=KTVH2eXKwrZlr8JAKZBFOEa2wIJOanuAdqS6voDzkV97U4ngRd2UaJxfsizlZE5ZP 7W2KJoYNdMgqTuebooeYKTz0HMkAYYuwr9hDQ15ggDofykRtyShNZ07ghI4ElOXSpi LDMXYJ7JRAstT95Fl0qKKV9M79Ra0DIwmlwe/b08= Message-ID: <537F7B83.501@wemm.org> Date: Fri, 23 May 2014 09:46:59 -0700 From: Peter Wemm User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: mbuf_cluster (FAIL SLEEP) References: <20140523171820.e631082d8015136ac052fdbd@valuehost.ru> In-Reply-To: <20140523171820.e631082d8015136ac052fdbd@valuehost.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 16:47:02 -0000 On 5/23/14, 6:18 AM, Peter B. Pokryshev wrote: > Hi. > Is it normal after 16 days of uptime: > > # vmstat -z > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > ... > 16 Bucket: 152, 0, 24, 101, 193, 0, 0 > 32 Bucket: 280, 0, 38, 102, 329, 2, 0 > 64 Bucket: 536, 0, 30, 33, 487, 142, 0 > 128 Bucket: 1048, 0, 997, 11, 6717030,17345735, 0 > ... > mbuf_packet: 256, 12896820, 1449, 1646,9062649837,118865, 0 > mbuf: 256, 12896820, 2193, 1762,17686258507, 0, 0 > mbuf_cluster: 2048, 2015128, 3095, 1793,26759484,241537,1100807 > mbuf_jumbo_page: 4096, 1007563, 2160, 864,2326876443, 0, 0 > mbuf_jumbo_9k: 9216, 298537, 0, 0, 0, 0, 0 > mbuf_jumbo_16k: 16384, 167927, 0, 0, 0, 0, 0 > mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 > > I mean 128 Bucket (FAIL) and mbuf_cluster (FAIL SLEEP) Yes, this is normal and it doesn't mean what you might expect. It's a generic failure counter, not an allocation failure counter. eg: if an object that was just freed fails to fit in a per-cpu free items cache it counts as a "FAIL". -Peter From owner-freebsd-net@FreeBSD.ORG Fri May 23 18:06:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93147F52 for ; Fri, 23 May 2014 18:06:36 +0000 (UTC) Received: from mail-wi0-x243.google.com (mail-wi0-x243.google.com [IPv6:2a00:1450:400c:c05::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 313402771 for ; Fri, 23 May 2014 18:06:36 +0000 (UTC) Received: by mail-wi0-f195.google.com with SMTP id ho1so363750wib.2 for ; Fri, 23 May 2014 11:06:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/UqJYejDnTsESzZwDoDkkJd16leILJgkdU5V2SIUA5E=; b=ZMjnIPdKUJxPPnw+JrqiyOWYseY2n2xVowAxzgZp4NaqJQgAPJVFO82/QFMiWgTTo2 PVtdQM9/5CrYNViejMawwAXpousAowIRnRntbIjXtB9TqG6TjEfyLv6rAp0Mtp/aUamQ TJdJqzlrjQ0WhTNca5gEBAWrEuTAKwN/I3vY2VUc01CuSF7lkNPEuPZWMYDooC8gG6qC ryXqpvFvfKJKadmanBoHEuKmJ0ASYGSgC+CkczPH7R66tXQtSWSEdpT422g3mVx6VKHU W+f9B0xN1GFRf67c3YVQTIc3O+w0E36fZ7+cnnWzrwiAeSHF9MHeIzn9OMESgGvr83LJ jTEg== MIME-Version: 1.0 X-Received: by 10.180.13.209 with SMTP id j17mr4823200wic.18.1400868394141; Fri, 23 May 2014 11:06:34 -0700 (PDT) Received: by 10.216.232.68 with HTTP; Fri, 23 May 2014 11:06:34 -0700 (PDT) Date: Sat, 24 May 2014 02:06:34 +0800 Message-ID: Subject: ipfw table add problem From: Sato Kentney To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 18:06:36 -0000 Hello. I'm using free bsd 10 stable. and I meet same issue as http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/189471 currently using thee fix from "bycm82"',it work so thank you but. recently I meet some issue in using ipfw with table. and found some PR about the table in ipfw. can I know what is happening? sorry about my poor english. Thanks, Sato K(=E4=BD=90=E8=97=A4=E6=9F=AF=E5=BE=B7) From owner-freebsd-net@FreeBSD.ORG Fri May 23 20:53:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D619CC75 for ; Fri, 23 May 2014 20:53:08 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB9322595 for ; Fri, 23 May 2014 20:53:07 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WnwSa-0002cd-Jc for freebsd-net@freebsd.org; Fri, 23 May 2014 22:53:00 +0200 Received: from h87.s239.verisign.com ([216.168.239.87]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 May 2014 22:53:00 +0200 Received: from jcharbon by h87.s239.verisign.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 May 2014 22:53:00 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Fri, 23 May 2014 22:52:45 +0200 Lines: 1916 Message-ID: <537FB51D.2060401@verisign.com> References: <537F39DF.1090900@verisign.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------010203040600060802030507" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: h87.s239.verisign.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: <537F39DF.1090900@verisign.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 20:53:08 -0000 This is a multi-part message in MIME format. --------------010203040600060802030507 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, On 23/05/14 14:06, Julien Charbon wrote: > On 27/02/14 11:32, Julien Charbon wrote: >> On 07/11/13 14:55, Julien Charbon wrote: >>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>> wrote: >>>> I have put technical and how-to-repeat details in below PR: >>>> >>>> kern/183659: TCP stack lock contention with short-lived connections >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>> >>>> We are currently working on this performance improvement effort; it >>>> will impact only the TCP locking strategy not the TCP stack logic >>>> itself. We will share on freebsd-net the patches we made for >>>> reviewing and improvement propositions; anyway this change might also >>>> require enough eyeballs to avoid tricky race conditions introduction >>>> in TCP stack. Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan 2014. First one is (tcp-scale-inp-list-v1.patch): [tcp-scaling] Introduce the INP_LIST global mutex for protecting pcbinfo global structures https://github.com/verisign/freebsd/commit/12c62273f052911aabe6ed283cea76cdd72c9493 This change improves nothing in performance (neither degrades), it simply implements what we are trying to achieve: Decompose further pcbinfo lock (aka ipi_lock or INP_INFO). Ideally, pcbinfo globally shared structures are protected by leaf mutexes (mutexes that are taken last), not by a root mutex (mutex taken first). The current lock ordering is: ipi_lock > inpcb lock > ipi_hash_lock, pcbgroup locks ipi_lock being a root mutex is explained by its original task: Protect the pcbinfo as a whole. Then, with this change, we added a new ipi_list_lock leaf mutex dedicated to protect structures previously under ipi_lock umbrella, i.e.: - inpcb global list: ipi_listhead - inpcb global list counter: ipi_count - inpcb global list generated index: ipi_gencnt and it permits to implement the second (meatier) change (tcp-scale-pcbinfo-rlock-v1.patch): [alpha][tcp-scaling] Use INP_INFO_RLOCK in critical path, and use INP_INFO_WLOCK in full INP loops. https://github.com/verisign/freebsd/commit/4633ac8c0b8d379fbda5fb9ffc921c2e4786db46 Now that ipi_lost has lost is duty to protect pcbinfo globally shared structures, its last (clear) duty is to hold inp creation/destruction when a full traversal of global inp list is performed, as this traversals expect inp list to be stable, e.g.: tcp_ccalgounload() https://github.com/verisign/freebsd/blob/388f0a87958fde5e644e01798f44b58588eb1dc2/sys/netinet/tcp_subr.c#L848 Thus (performance-wise) critical paths can now take ipi_lock _read_ lock, e.g.: tcp_input() tcp_usr_shutdown() tcp_usr_close() tcp_twstart() and, on the other side, functions performing full inp list traversal will take the INP_INFO _write_ lock: tcp_ccalgounload() tcp_pcblist() in_pcbpurgeif0() etc... This patch doubles the performance improvement with our short-live TCP workload. _However_ it would be a miracle that this change does not introduce new race condition(s) (hence the 'alpha' tag in commit message). Most of TCP stack locking strategy being now on inpcb lock shoulders. That said, from our tests point of view, this change is completely stable: No kernel/lock assertion, no unexpected TCP behavior, stable performance results. Moreover, before tagging this change as 'beta' we need to test more thoroughly these features: - VNET, - PCBGROUP/RSS/TCP timer per cpu, - TCP Offloading (we need a NIC with a good TCP offloading support) Early testers, test ideas, reviewers and memories about previous (and not documented or unclear) ipi_lock duties are more than welcome. Thanks. -- Julien --------------010203040600060802030507 Content-Type: text/plain; charset=UTF-8; name="tcp-scale-pcbinfo-rlock-v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tcp-scale-pcbinfo-rlock-v1.patch" diff --git a/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c b/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c index a86bf72..f28c83d 100644 --- a/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c +++ b/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c @@ -639,7 +639,7 @@ t3_send_fin(struct toedev *tod, struct tcpcb *tp) unsigned int tid = toep->tp_tid; #endif - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); CTR4(KTR_CXGB, "%s: tid %d, toep %p, flags %x", __func__, tid, toep, @@ -925,12 +925,12 @@ do_act_open_rpl(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) rc = act_open_rpl_status_to_errno(s); if (rc != EAGAIN) - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); toe_connect_failed(tod, inp, rc); toepcb_release(toep); /* unlocks inp */ if (rc != EAGAIN) - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); @@ -1061,7 +1061,7 @@ send_reset(struct toepcb *toep) struct adapter *sc = tod->tod_softc; struct mbuf *m; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); CTR4(KTR_CXGB, "%s: tid %d, toep %p (%x)", __func__, tid, toep, @@ -1172,12 +1172,12 @@ do_rx_data(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) SOCKBUF_UNLOCK(so_rcv); INP_WUNLOCK(inp); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = tcp_drop(tp, ECONNRESET); if (tp) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); @@ -1222,7 +1222,7 @@ do_peer_close(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) struct tcpcb *tp; struct socket *so; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = intotcpcb(inp); @@ -1250,7 +1250,7 @@ do_peer_close(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) case TCPS_FIN_WAIT_2: tcp_twstart(tp); INP_UNLOCK_ASSERT(inp); /* safe, we have a ref on the inp */ - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); INP_WLOCK(inp); toepcb_release(toep); /* no more CPLs expected */ @@ -1264,7 +1264,7 @@ do_peer_close(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) done: INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); @@ -1285,7 +1285,7 @@ do_close_con_rpl(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) struct tcpcb *tp; struct socket *so; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = intotcpcb(inp); @@ -1303,7 +1303,7 @@ do_close_con_rpl(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) tcp_twstart(tp); release: INP_UNLOCK_ASSERT(inp); /* safe, we have a ref on the inp */ - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); INP_WLOCK(inp); toepcb_release(toep); /* no more CPLs expected */ @@ -1328,7 +1328,7 @@ do_close_con_rpl(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) done: INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); @@ -1489,7 +1489,7 @@ do_abort_req(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) return (do_abort_req_synqe(qs, r, m)); inp = toep->tp_inp; - INP_INFO_WLOCK(&V_tcbinfo); /* for tcp_close */ + INP_INFO_RLOCK(&V_tcbinfo); /* for tcp_close */ INP_WLOCK(inp); tp = intotcpcb(inp); @@ -1523,7 +1523,7 @@ do_abort_req(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) INP_WLOCK(inp); /* re-acquire */ toepcb_release(toep); /* no more CPLs expected */ } - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); send_abort_rpl(tod, tid, qset); m_freem(m); diff --git a/sys/dev/cxgb/ulp/tom/cxgb_listen.c b/sys/dev/cxgb/ulp/tom/cxgb_listen.c index 94a219b..631899d 100644 --- a/sys/dev/cxgb/ulp/tom/cxgb_listen.c +++ b/sys/dev/cxgb/ulp/tom/cxgb_listen.c @@ -554,11 +554,11 @@ do_pass_accept_req(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) REJECT_PASS_ACCEPT(); /* no l2te, or ifp mismatch */ } - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); /* Don't offload if the 4-tuple is already in use */ if (toe_4tuple_check(&inc, &th, ifp) != 0) { - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); REJECT_PASS_ACCEPT(); } @@ -571,7 +571,7 @@ do_pass_accept_req(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) * resources tied to this listen context. */ INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); REJECT_PASS_ACCEPT(); } so = inp->inp_socket; @@ -713,7 +713,7 @@ do_pass_establish(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) KASSERT(qs->idx == synqe->qset, ("%s qset mismatch %d %d", __func__, qs->idx, synqe->qset)); - INP_INFO_WLOCK(&V_tcbinfo); /* for syncache_expand */ + INP_INFO_RLOCK(&V_tcbinfo); /* for syncache_expand */ INP_WLOCK(inp); if (__predict_false(inp->inp_flags & INP_DROPPED)) { @@ -727,7 +727,7 @@ do_pass_establish(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) ("%s: listen socket dropped but tid %u not aborted.", __func__, tid)); INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); } @@ -743,7 +743,7 @@ do_pass_establish(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) reset: t3_send_reset_synqe(tod, synqe); INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return (0); } @@ -775,7 +775,7 @@ do_pass_establish(struct sge_qset *qs, struct rsp_desc *r, struct mbuf *m) inp = release_lctx(td, lctx); if (inp) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); release_synqe(synqe); m_freem(m); diff --git a/sys/dev/cxgbe/tom/t4_connect.c b/sys/dev/cxgbe/tom/t4_connect.c index 9973fa5..718f62a 100644 --- a/sys/dev/cxgbe/tom/t4_connect.c +++ b/sys/dev/cxgbe/tom/t4_connect.c @@ -208,12 +208,12 @@ do_act_open_rpl(struct sge_iq *iq, const struct rss_header *rss, rc = act_open_rpl_status_to_errno(status); if (rc != EAGAIN) - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); toe_connect_failed(tod, inp, rc); final_cpl_received(toep); /* unlocks inp */ if (rc != EAGAIN) - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (0); } diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index e2f5c79..12290a8 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -843,7 +843,7 @@ do_peer_close(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) KASSERT(toep->tid == tid, ("%s: toep tid mismatch", __func__)); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = intotcpcb(inp); @@ -897,7 +897,7 @@ do_peer_close(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) case TCPS_FIN_WAIT_2: tcp_twstart(tp); INP_UNLOCK_ASSERT(inp); /* safe, we have a ref on the inp */ - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); INP_WLOCK(inp); final_cpl_received(toep); @@ -909,7 +909,7 @@ do_peer_close(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) } done: INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (0); } @@ -936,7 +936,7 @@ do_close_con_rpl(struct sge_iq *iq, const struct rss_header *rss, KASSERT(m == NULL, ("%s: wasn't expecting payload", __func__)); KASSERT(toep->tid == tid, ("%s: toep tid mismatch", __func__)); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = intotcpcb(inp); @@ -954,7 +954,7 @@ do_close_con_rpl(struct sge_iq *iq, const struct rss_header *rss, tcp_twstart(tp); release: INP_UNLOCK_ASSERT(inp); /* safe, we have a ref on the inp */ - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); INP_WLOCK(inp); final_cpl_received(toep); /* no more CPLs expected */ @@ -978,7 +978,7 @@ do_close_con_rpl(struct sge_iq *iq, const struct rss_header *rss, } done: INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (0); } @@ -1052,7 +1052,7 @@ do_abort_req(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) } inp = toep->inp; - INP_INFO_WLOCK(&V_tcbinfo); /* for tcp_close */ + INP_INFO_RLOCK(&V_tcbinfo); /* for tcp_close */ INP_WLOCK(inp); tp = intotcpcb(inp); @@ -1086,7 +1086,7 @@ do_abort_req(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) final_cpl_received(toep); done: - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); send_abort_rpl(sc, ofld_txq, tid, CPL_ABORT_NO_RST); return (0); } @@ -1200,12 +1200,12 @@ do_rx_data(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) SOCKBUF_UNLOCK(sb); INP_WUNLOCK(inp); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = tcp_drop(tp, ECONNRESET); if (tp) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (0); } diff --git a/sys/dev/cxgbe/tom/t4_listen.c b/sys/dev/cxgbe/tom/t4_listen.c index 0dc02e3..7571d33 100644 --- a/sys/dev/cxgbe/tom/t4_listen.c +++ b/sys/dev/cxgbe/tom/t4_listen.c @@ -1322,11 +1322,11 @@ do_pass_accept_req(struct sge_iq *iq, const struct rss_header *rss, REJECT_PASS_ACCEPT(); rpl = wrtod(wr); - INP_INFO_WLOCK(&V_tcbinfo); /* for 4-tuple check, syncache_add */ + INP_INFO_RLOCK(&V_tcbinfo); /* for 4-tuple check, syncache_add */ /* Don't offload if the 4-tuple is already in use */ if (toe_4tuple_check(&inc, &th, ifp) != 0) { - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); free(wr, M_CXGBE); REJECT_PASS_ACCEPT(); } @@ -1342,7 +1342,7 @@ do_pass_accept_req(struct sge_iq *iq, const struct rss_header *rss, * resources tied to this listen context. */ INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); free(wr, M_CXGBE); REJECT_PASS_ACCEPT(); } @@ -1524,7 +1524,7 @@ do_pass_establish(struct sge_iq *iq, const struct rss_header *rss, KASSERT(synqe->flags & TPF_SYNQE, ("%s: tid %u (ctx %p) not a synqe", __func__, tid, synqe)); - INP_INFO_WLOCK(&V_tcbinfo); /* for syncache_expand */ + INP_INFO_RLOCK(&V_tcbinfo); /* for syncache_expand */ INP_WLOCK(inp); CTR6(KTR_CXGBE, @@ -1622,7 +1622,7 @@ do_pass_establish(struct sge_iq *iq, const struct rss_header *rss, inp = release_lctx(sc, lctx); if (inp != NULL) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); release_synqe(synqe); return (0); diff --git a/sys/netinet/in_pcb.c b/sys/netinet/in_pcb.c index d27086b..f891f30 100644 --- a/sys/netinet/in_pcb.c +++ b/sys/netinet/in_pcb.c @@ -272,7 +272,14 @@ in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) struct inpcb *inp; int error; - INP_INFO_WLOCK_ASSERT(pcbinfo); +#ifdef INVARIANTS + if (pcbinfo == &V_tcbinfo) { + INP_INFO_RLOCK_ASSERT(pcbinfo); + } else { + INP_INFO_WLOCK_ASSERT(pcbinfo); + } +#endif + error = 0; inp = uma_zalloc(pcbinfo->ipi_zone, M_NOWAIT); if (inp == NULL) @@ -1195,7 +1202,13 @@ in_pcbfree(struct inpcb *inp) KASSERT(inp->inp_socket == NULL, ("%s: inp_socket != NULL", __func__)); - INP_INFO_WLOCK_ASSERT(pcbinfo); +#ifdef INVARIANTS + if (pcbinfo == &V_tcbinfo) { + INP_INFO_RLOCK_ASSERT(pcbinfo); + } else { + INP_INFO_WLOCK_ASSERT(pcbinfo); + } +#endif INP_WLOCK_ASSERT(inp); /* XXXRW: Do as much as possible here. */ @@ -1363,7 +1376,7 @@ in_pcbpurgeif0(struct inpcbinfo *pcbinfo, struct ifnet *ifp) struct ip_moptions *imo; int i, gap; - INP_INFO_RLOCK(pcbinfo); + INP_INFO_WLOCK(pcbinfo); LIST_FOREACH(inp, pcbinfo->ipi_listhead, inp_list) { INP_WLOCK(inp); imo = inp->inp_moptions; @@ -1393,7 +1406,7 @@ in_pcbpurgeif0(struct inpcbinfo *pcbinfo, struct ifnet *ifp) } INP_WUNLOCK(inp); } - INP_INFO_RUNLOCK(pcbinfo); + INP_INFO_WUNLOCK(pcbinfo); } /* @@ -2047,7 +2060,14 @@ in_pcbremlists(struct inpcb *inp) { struct inpcbinfo *pcbinfo = inp->inp_pcbinfo; - INP_INFO_WLOCK_ASSERT(pcbinfo); +#ifdef INVARIANTS + if (pcbinfo == &V_tcbinfo) { + INP_INFO_RLOCK_ASSERT(pcbinfo); + } else { + INP_INFO_WLOCK_ASSERT(pcbinfo); + } +#endif + INP_WLOCK_ASSERT(inp); INP_LIST_WLOCK_ASSERT(pcbinfo); @@ -2194,13 +2214,13 @@ inp_apply_all(void (*func)(struct inpcb *, void *), void *arg) { struct inpcb *inp; - INP_INFO_RLOCK(&V_tcbinfo); + INP_INFO_WLOCK(&V_tcbinfo); LIST_FOREACH(inp, V_tcbinfo.ipi_listhead, inp_list) { INP_WLOCK(inp); func(inp, arg); INP_WUNLOCK(inp); } - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_INFO_WUNLOCK(&V_tcbinfo); } struct socket * diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c index 4b6f41f..234932b 100644 --- a/sys/netinet/tcp_input.c +++ b/sys/netinet/tcp_input.c @@ -591,7 +591,7 @@ tcp_input(struct mbuf *m, int off0) int needlock; int ti_locked; #define TI_UNLOCKED 1 -#define TI_WLOCKED 2 +#define TI_RLOCKED 2 #ifdef TCPDEBUG /* @@ -777,8 +777,8 @@ tcp_input(struct mbuf *m, int off0) * a connection in TIMEWAIT, SYNs for a non-listening socket. */ if ((thflags & (TH_FIN | TH_RST)) != 0) { - INP_INFO_WLOCK(&V_tcbinfo); - ti_locked = TI_WLOCKED; + INP_INFO_RLOCK(&V_tcbinfo); + ti_locked = TI_RLOCKED; } else ti_locked = TI_UNLOCKED; @@ -800,8 +800,8 @@ tcp_input(struct mbuf *m, int off0) findpcb: #ifdef INVARIANTS - if (ti_locked == TI_WLOCKED) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) { + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); } else { INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); } @@ -953,20 +953,20 @@ tcp_input(struct mbuf *m, int off0) relocked: if (inp->inp_flags & INP_TIMEWAIT) { if (ti_locked == TI_UNLOCKED) { - if (INP_INFO_TRY_WLOCK(&V_tcbinfo) == 0) { + if (INP_INFO_TRY_RLOCK(&V_tcbinfo) == 0) { in_pcbref(inp); INP_WUNLOCK(inp); - INP_INFO_WLOCK(&V_tcbinfo); - ti_locked = TI_WLOCKED; + INP_INFO_RLOCK(&V_tcbinfo); + ti_locked = TI_RLOCKED; INP_WLOCK(inp); if (in_pcbrele_wlocked(inp)) { inp = NULL; goto findpcb; } } else - ti_locked = TI_WLOCKED; + ti_locked = TI_RLOCKED; } - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); if (thflags & TH_SYN) tcp_dooptions(&to, optp, optlen, TO_SYN); @@ -975,7 +975,7 @@ tcp_input(struct mbuf *m, int off0) */ if (tcp_twcheck(inp, &to, th, m, tlen)) goto findpcb; - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return; } /* @@ -1006,7 +1006,7 @@ tcp_input(struct mbuf *m, int off0) */ #ifdef INVARIANTS if ((thflags & (TH_FIN | TH_RST)) != 0) - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); #endif needlock = 1; if (tp->t_state == TCPS_ESTABLISHED) { @@ -1018,11 +1018,11 @@ tcp_input(struct mbuf *m, int off0) } if (needlock) { if (ti_locked == TI_UNLOCKED) { - if (INP_INFO_TRY_WLOCK(&V_tcbinfo) == 0) { + if (INP_INFO_TRY_RLOCK(&V_tcbinfo) == 0) { in_pcbref(inp); INP_WUNLOCK(inp); - INP_INFO_WLOCK(&V_tcbinfo); - ti_locked = TI_WLOCKED; + INP_INFO_RLOCK(&V_tcbinfo); + ti_locked = TI_RLOCKED; INP_WLOCK(inp); if (in_pcbrele_wlocked(inp)) { inp = NULL; @@ -1030,9 +1030,9 @@ tcp_input(struct mbuf *m, int off0) } goto relocked; } else - ti_locked = TI_WLOCKED; + ti_locked = TI_RLOCKED; } - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); } #ifdef MAC @@ -1087,7 +1087,7 @@ tcp_input(struct mbuf *m, int off0) */ if ((thflags & (TH_RST|TH_ACK|TH_SYN)) == TH_ACK) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); /* * Parse the TCP options here because * syncookies need access to the reflected @@ -1370,8 +1370,8 @@ tcp_input(struct mbuf *m, int off0) * Entry added to syncache and mbuf consumed. * Nothing is unlocked by syncache_add(). */ - if (ti_locked == TI_WLOCKED) { - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) { + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; } INP_INFO_UNLOCK_ASSERT(&V_tcbinfo); @@ -1420,8 +1420,8 @@ tcp_input(struct mbuf *m, int off0) dropwithreset: TCP_PROBE5(receive, NULL, tp, mtod(m, const char *), tp, th); - if (ti_locked == TI_WLOCKED) { - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) { + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; } #ifdef INVARIANTS @@ -1444,8 +1444,8 @@ tcp_input(struct mbuf *m, int off0) if (m != NULL) TCP_PROBE5(receive, NULL, tp, mtod(m, const char *), tp, th); - if (ti_locked == TI_WLOCKED) { - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) { + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; } #ifdef INVARIANTS @@ -1501,13 +1501,13 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, */ if ((thflags & (TH_SYN | TH_FIN | TH_RST)) != 0 || tp->t_state != TCPS_ESTABLISHED) { - KASSERT(ti_locked == TI_WLOCKED, ("%s ti_locked %d for " + KASSERT(ti_locked == TI_RLOCKED, ("%s ti_locked %d for " "SYN/FIN/RST/!EST", __func__, ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); } else { #ifdef INVARIANTS - if (ti_locked == TI_WLOCKED) - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); else { KASSERT(ti_locked == TI_UNLOCKED, ("%s: EST " "ti_locked: %d", __func__, ti_locked)); @@ -1675,8 +1675,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, /* * This is a pure ack for outstanding data. */ - if (ti_locked == TI_WLOCKED) - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; TCPSTAT_INC(tcps_predack); @@ -1779,8 +1779,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, * nothing on the reassembly queue and we have enough * buffer space to take it. */ - if (ti_locked == TI_WLOCKED) - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; /* Clean receiver SACK report if present */ @@ -2013,9 +2013,9 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, tcp_state_change(tp, TCPS_SYN_RECEIVED); } - KASSERT(ti_locked == TI_WLOCKED, ("%s: trimthenstep6: " + KASSERT(ti_locked == TI_RLOCKED, ("%s: trimthenstep6: " "ti_locked %d", __func__, ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tp->t_inpcb); /* @@ -2143,10 +2143,10 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, case TCPS_CLOSE_WAIT: so->so_error = ECONNRESET; close: - KASSERT(ti_locked == TI_WLOCKED, + KASSERT(ti_locked == TI_RLOCKED, ("tcp_do_segment: TH_RST 1 ti_locked %d", ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tcp_state_change(tp, TCPS_CLOSED); TCPSTAT_INC(tcps_drops); @@ -2155,10 +2155,10 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, case TCPS_CLOSING: case TCPS_LAST_ACK: - KASSERT(ti_locked == TI_WLOCKED, + KASSERT(ti_locked == TI_RLOCKED, ("tcp_do_segment: TH_RST 2 ti_locked %d", ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tp = tcp_close(tp); break; @@ -2266,9 +2266,9 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, */ if ((so->so_state & SS_NOFDREF) && tp->t_state > TCPS_CLOSE_WAIT && tlen) { - KASSERT(ti_locked == TI_WLOCKED, ("%s: SS_NOFDEREF && " + KASSERT(ti_locked == TI_RLOCKED, ("%s: SS_NOFDEREF && " "CLOSE_WAIT && tlen ti_locked %d", __func__, ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); if ((s = tcp_log_addrs(inc, th, NULL, NULL))) { log(LOG_DEBUG, "%s; %s: %s: Received %d bytes of data " @@ -2342,9 +2342,9 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, * error and we send an RST and drop the connection. */ if (thflags & TH_SYN) { - KASSERT(ti_locked == TI_WLOCKED, + KASSERT(ti_locked == TI_RLOCKED, ("tcp_do_segment: TH_SYN ti_locked %d", ti_locked)); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tp = tcp_drop(tp, ECONNRESET); rstreason = BANDLIM_UNLIMITED; @@ -2783,9 +2783,9 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, */ case TCPS_CLOSING: if (ourfinisacked) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tcp_twstart(tp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); m_freem(m); return; } @@ -2799,7 +2799,7 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, */ case TCPS_LAST_ACK: if (ourfinisacked) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tp = tcp_close(tp); goto drop; } @@ -3013,18 +3013,18 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, * standard timers. */ case TCPS_FIN_WAIT_2: - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); - KASSERT(ti_locked == TI_WLOCKED, ("%s: dodata " + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); + KASSERT(ti_locked == TI_RLOCKED, ("%s: dodata " "TCP_FIN_WAIT_2 ti_locked: %d", __func__, ti_locked)); tcp_twstart(tp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return; } } - if (ti_locked == TI_WLOCKED) - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; #ifdef TCPDEBUG @@ -3079,8 +3079,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, tcp_trace(TA_DROP, ostate, tp, (void *)tcp_saveipgen, &tcp_savetcp, 0); #endif - if (ti_locked == TI_WLOCKED) - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; tp->t_flags |= TF_ACKNOW; @@ -3090,8 +3090,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, return; dropwithreset: - if (ti_locked == TI_WLOCKED) - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; if (tp != NULL) { @@ -3102,8 +3102,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, return; drop: - if (ti_locked == TI_WLOCKED) { - INP_INFO_WUNLOCK(&V_tcbinfo); + if (ti_locked == TI_RLOCKED) { + INP_INFO_RUNLOCK(&V_tcbinfo); ti_locked = TI_UNLOCKED; } #ifdef INVARIANTS diff --git a/sys/netinet/tcp_subr.c b/sys/netinet/tcp_subr.c index cf7c02f..e4f8994 100644 --- a/sys/netinet/tcp_subr.c +++ b/sys/netinet/tcp_subr.c @@ -850,11 +850,11 @@ tcp_ccalgounload(struct cc_algo *unload_algo) VNET_LIST_RLOCK(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); - INP_INFO_RLOCK(&V_tcbinfo); + INP_INFO_WLOCK(&V_tcbinfo); /* * New connections already part way through being initialised * with the CC algo we're removing will not race with this code - * because the INP_INFO_WLOCK is held during initialisation. We + * because the INP_INFO_RLOCK is held during initialisation. We * therefore don't enter the loop below until the connection * list has stabilised. */ @@ -880,7 +880,7 @@ tcp_ccalgounload(struct cc_algo *unload_algo) } INP_WUNLOCK(inp); } - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_INFO_WUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK(); @@ -898,7 +898,7 @@ tcp_drop(struct tcpcb *tp, int errno) { struct socket *so = tp->t_inpcb->inp_socket; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tp->t_inpcb); if (TCPS_HAVERCVDSYN(tp->t_state)) { @@ -1034,7 +1034,7 @@ tcp_close(struct tcpcb *tp) struct inpcb *inp = tp->t_inpcb; struct socket *so; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); #ifdef TCP_OFFLOAD @@ -1082,7 +1082,7 @@ tcp_drain(void) * where we're really low on mbufs, this is potentially * useful. */ - INP_INFO_RLOCK(&V_tcbinfo); + INP_INFO_WLOCK(&V_tcbinfo); LIST_FOREACH(inpb, V_tcbinfo.ipi_listhead, inp_list) { if (inpb->inp_flags & INP_TIMEWAIT) continue; @@ -1093,7 +1093,7 @@ tcp_drain(void) } INP_WUNLOCK(inpb); } - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_INFO_WUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK_NOSLEEP(); @@ -1206,7 +1206,7 @@ tcp_pcblist(SYSCTL_HANDLER_ARGS) if (inp_list == NULL) return (ENOMEM); - INP_INFO_RLOCK(&V_tcbinfo); + INP_INFO_WLOCK(&V_tcbinfo); for (inp = LIST_FIRST(V_tcbinfo.ipi_listhead), i = 0; inp != NULL && i < n; inp = LIST_NEXT(inp, inp_list)) { INP_WLOCK(inp); @@ -1231,7 +1231,7 @@ tcp_pcblist(SYSCTL_HANDLER_ARGS) } INP_WUNLOCK(inp); } - INP_INFO_RUNLOCK(&V_tcbinfo); + INP_INFO_WUNLOCK(&V_tcbinfo); n = i; error = 0; @@ -1269,14 +1269,14 @@ tcp_pcblist(SYSCTL_HANDLER_ARGS) } else INP_RUNLOCK(inp); } - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); for (i = 0; i < n; i++) { inp = inp_list[i]; INP_RLOCK(inp); if (!in_pcbrele_rlocked(inp)) INP_RUNLOCK(inp); } - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); if (!error) { /* @@ -1453,7 +1453,7 @@ tcp_ctlinput(int cmd, struct sockaddr *sa, void *vip) - offsetof(struct icmp, icmp_ip)); th = (struct tcphdr *)((caddr_t)ip + (ip->ip_hl << 2)); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = in_pcblookup(&V_tcbinfo, faddr, th->th_dport, ip->ip_src, th->th_sport, INPLOOKUP_WLOCKPCB, NULL); if (inp != NULL) { @@ -1513,7 +1513,7 @@ tcp_ctlinput(int cmd, struct sockaddr *sa, void *vip) inc.inc_laddr = ip->ip_src; syncache_unreach(&inc, th); } - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); } else in_pcbnotifyall(&V_tcbinfo, faddr, inetctlerrmap[cmd], notify); } @@ -1586,9 +1586,9 @@ tcp6_ctlinput(int cmd, struct sockaddr *sa, void *d) inc.inc6_faddr = ((struct sockaddr_in6 *)sa)->sin6_addr; inc.inc6_laddr = ip6cp->ip6c_src->sin6_addr; inc.inc_flags |= INC_ISIPV6; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); syncache_unreach(&inc, &th); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); } else in6_pcbnotify(&V_tcbinfo, sa, 0, (const struct sockaddr *)sa6_src, 0, cmd, NULL, notify); @@ -1721,7 +1721,7 @@ tcp_drop_syn_sent(struct inpcb *inp, int errno) { struct tcpcb *tp; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); if ((inp->inp_flags & INP_TIMEWAIT) || @@ -2240,7 +2240,7 @@ sysctl_drop(SYSCTL_HANDLER_ARGS) default: return (EINVAL); } - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); switch (addrs[0].ss_family) { #ifdef INET6 case AF_INET6: @@ -2279,7 +2279,7 @@ sysctl_drop(SYSCTL_HANDLER_ARGS) INP_WUNLOCK(inp); } else error = ESRCH; - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (error); } diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c index 9b981d3..e479f8a 100644 --- a/sys/netinet/tcp_syncache.c +++ b/sys/netinet/tcp_syncache.c @@ -663,7 +663,7 @@ syncache_socket(struct syncache *sc, struct socket *lso, struct mbuf *m) int error; char *s; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); /* * Ok, create the full blown connection, and set things up @@ -945,7 +945,7 @@ syncache_expand(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, * Global TCP locks are held because we manipulate the PCB lists * and create a new socket. */ - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); KASSERT((th->th_flags & (TH_RST|TH_ACK|TH_SYN)) == TH_ACK, ("%s: can handle only ACK", __func__)); diff --git a/sys/netinet/tcp_timer.c b/sys/netinet/tcp_timer.c index 3874f13..c60e48e 100644 --- a/sys/netinet/tcp_timer.c +++ b/sys/netinet/tcp_timer.c @@ -265,7 +265,7 @@ tcp_timer_2msl(void *xtp) /* * XXXRW: Does this actually happen? */ - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; /* * XXXRW: While this assert is in fact correct, bugs in the tcpcb @@ -276,7 +276,7 @@ tcp_timer_2msl(void *xtp) */ if (inp == NULL) { tcp_timer_race++; - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -285,14 +285,14 @@ tcp_timer_2msl(void *xtp) if (callout_pending(&tp->t_timers->tt_2msl) || !callout_active(&tp->t_timers->tt_2msl)) { INP_WUNLOCK(tp->t_inpcb); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } callout_deactivate(&tp->t_timers->tt_2msl); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -328,7 +328,7 @@ tcp_timer_2msl(void *xtp) #endif if (tp != NULL) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } @@ -344,7 +344,7 @@ tcp_timer_keep(void *xtp) ostate = tp->t_state; #endif - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; /* * XXXRW: While this assert is in fact correct, bugs in the tcpcb @@ -355,7 +355,7 @@ tcp_timer_keep(void *xtp) */ if (inp == NULL) { tcp_timer_race++; - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -363,14 +363,14 @@ tcp_timer_keep(void *xtp) if (callout_pending(&tp->t_timers->tt_keep) || !callout_active(&tp->t_timers->tt_keep)) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } callout_deactivate(&tp->t_timers->tt_keep); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -417,7 +417,7 @@ tcp_timer_keep(void *xtp) PRU_SLOWTIMO); #endif INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; @@ -432,7 +432,7 @@ tcp_timer_keep(void *xtp) #endif if (tp != NULL) INP_WUNLOCK(tp->t_inpcb); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } @@ -447,7 +447,7 @@ tcp_timer_persist(void *xtp) ostate = tp->t_state; #endif - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = tp->t_inpcb; /* * XXXRW: While this assert is in fact correct, bugs in the tcpcb @@ -458,7 +458,7 @@ tcp_timer_persist(void *xtp) */ if (inp == NULL) { tcp_timer_race++; - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -466,14 +466,14 @@ tcp_timer_persist(void *xtp) if (callout_pending(&tp->t_timers->tt_persist) || !callout_active(&tp->t_timers->tt_persist)) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } callout_deactivate(&tp->t_timers->tt_persist); if ((inp->inp_flags & INP_DROPPED) != 0) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -518,7 +518,7 @@ tcp_timer_persist(void *xtp) #endif if (tp != NULL) INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } @@ -577,16 +577,16 @@ tcp_timer_rexmt(void * xtp) in_pcbref(inp); INP_INFO_RUNLOCK(&V_tcbinfo); INP_WUNLOCK(inp); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); if (in_pcbrele_wlocked(inp)) { - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } if (inp->inp_flags & INP_DROPPED) { INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); return; } @@ -684,7 +684,7 @@ tcp_timer_rexmt(void * xtp) if (tp != NULL) INP_WUNLOCK(inp); if (headlocked) - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); CURVNET_RESTORE(); } diff --git a/sys/netinet/tcp_timewait.c b/sys/netinet/tcp_timewait.c index 92cf179..c89d511 100644 --- a/sys/netinet/tcp_timewait.c +++ b/sys/netinet/tcp_timewait.c @@ -120,34 +120,8 @@ static VNET_DEFINE(struct rwlock, tw_lock); #define TW_WLOCK_ASSERT(tw) rw_assert(&(tw), RA_WLOCKED) #define TW_UNLOCK_ASSERT(tw) rw_assert(&(tw), RA_UNLOCKED) -static void tcp_tw_2msl_reset(struct tcptw *, int); -static void tcp_tw_2msl_stop(struct tcptw *, int); - -/* - * tw_pcbref() bumps the reference count on an tw in order to maintain - * stability of an tw pointer despite the tw lock being released. - */ -static void -tw_pcbref(struct tcptw *tw) -{ - - KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); - refcount_acquire(&tw->tw_refcount); -} - -/* - * Drop a refcount on an tw elevated using tw_pcbref(). - */ -static int -tw_pcbrele(struct tcptw *tw) -{ - - KASSERT(tw->tw_refcount > 0, ("%s: refcount 0", __func__)); - if (!refcount_release(&tw->tw_refcount)) - return (0); - uma_zfree(V_tcptw_zone, tw); - return (1); -} +static void tcp_tw_2msl_reset(struct tcptw *, int rearm); +static void tcp_tw_2msl_stop(struct tcptw *, int reuse); static int tcptw_auto_size(void) @@ -223,10 +197,10 @@ tcp_tw_destroy(void) { struct tcptw *tw; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); while ((tw = TAILQ_FIRST(&V_twq_2msl)) != NULL) tcp_twclose(tw, 0); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); TW_LOCK_DESTROY(V_tw_lock); uma_zdestroy(V_tcptw_zone); @@ -249,7 +223,7 @@ tcp_twstart(struct tcpcb *tp) int isipv6 = inp->inp_inc.inc_flags & INC_ISIPV6; #endif - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); if (V_nolocaltimewait) { @@ -369,7 +343,7 @@ tcp_twcheck(struct inpcb *inp, struct tcpopt *to, struct tcphdr *th, int thflags; tcp_seq seq; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); /* @@ -470,10 +444,9 @@ tcp_twclose(struct tcptw *tw, int reuse) inp = tw->tw_inpcb; KASSERT((inp->inp_flags & INP_TIMEWAIT), ("tcp_twclose: !timewait")); KASSERT(intotw(inp) == tw, ("tcp_twclose: inp_ppcb != tw")); - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* in_pcbfree() */ + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); /* in_pcbfree() */ INP_WLOCK_ASSERT(inp); - tw->tw_inpcb = NULL; tcp_tw_2msl_stop(tw, reuse); inp->inp_ppcb = NULL; in_pcbdrop(inp); @@ -621,7 +594,7 @@ static void tcp_tw_2msl_reset(struct tcptw *tw, int rearm) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tw->tw_inpcb); TW_WLOCK(V_tw_lock); @@ -636,24 +609,28 @@ static void tcp_tw_2msl_stop(struct tcptw *tw, int reuse) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); TW_WLOCK(V_tw_lock); - TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); - crfree(tw->tw_cred); + tw->tw_inpcb = NULL; + if (!reuse) + TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); + if (tw->tw_cred != NULL) + crfree(tw->tw_cred); tw->tw_cred = NULL; TW_WUNLOCK(V_tw_lock); if (!reuse) - tw_pcbrele(tw); + uma_zfree(V_tcptw_zone, tw); } struct tcptw * tcp_tw_2msl_reuse(void) { struct tcptw *tw; + struct inpcb *inp; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); TW_WLOCK(V_tw_lock); tw = TAILQ_FIRST(&V_twq_2msl); @@ -661,10 +638,24 @@ tcp_tw_2msl_reuse(void) TW_WUNLOCK(V_tw_lock); return NULL; } + if (tw->tw_inpcb != NULL) { + TAILQ_REMOVE(&V_twq_2msl, tw, tw_2msl); + inp = tw->tw_inpcb; + in_pcbref(inp); + } else { + TW_WUNLOCK(V_tw_lock); + return NULL; /* XXXJCH loop? */ + } TW_WUNLOCK(V_tw_lock); - INP_WLOCK(tw->tw_inpcb); - tcp_twclose(tw, 1); + INP_WLOCK(inp); + if (in_pcbrele_wlocked(inp)) + return (NULL); /* XXXJCH loop? */ + tw = intotw(inp); + if (tw != NULL) + tcp_twclose(tw, 1); + else + INP_WUNLOCK(inp); /* XXXJCH loop? */ return (tw); } @@ -673,6 +664,7 @@ void tcp_tw_2msl_scan(void) { struct tcptw *tw; + struct inpcb *inp; for (;;) { TW_RLOCK(V_tw_lock); @@ -681,24 +673,33 @@ tcp_tw_2msl_scan(void) TW_RUNLOCK(V_tw_lock); break; } - tw_pcbref(tw); + if (tw->tw_inpcb != NULL) { + inp = tw->tw_inpcb; + in_pcbref(inp); + } else { + TW_RUNLOCK(V_tw_lock); + continue; + } TW_RUNLOCK(V_tw_lock); - /* Close timewait state */ - if (INP_INFO_TRY_WLOCK(&V_tcbinfo)) { - if (tw_pcbrele(tw)) { - INP_INFO_WUNLOCK(&V_tcbinfo); + if (INP_INFO_TRY_RLOCK(&V_tcbinfo)) { + + INP_WLOCK(inp); + if (in_pcbrele_wlocked(inp)) { + INP_INFO_RUNLOCK(&V_tcbinfo); continue; } - - KASSERT(tw->tw_inpcb != NULL, - ("%s: tw->tw_inpcb == NULL", __func__)); - INP_WLOCK(tw->tw_inpcb); - tcp_twclose(tw, 0); - INP_INFO_WUNLOCK(&V_tcbinfo); + tw = intotw(inp); + if (tw != NULL) + tcp_twclose(tw, 0); + else + INP_WUNLOCK(inp); + INP_INFO_RUNLOCK(&V_tcbinfo); } else { - /* INP_INFO lock is busy; continue later. */ - tw_pcbrele(tw); + /* INP_INFO lock is busy, continue later. */ + INP_WLOCK(inp); + if (!in_pcbrele_wlocked(inp)) + INP_WUNLOCK(inp); break; } } diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c index 42c1e1d..c8c7b4e 100644 --- a/sys/netinet/tcp_usrreq.c +++ b/sys/netinet/tcp_usrreq.c @@ -163,7 +163,7 @@ tcp_detach(struct socket *so, struct inpcb *inp) { struct tcpcb *tp; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); KASSERT(so->so_pcb == inp, ("tcp_detach: so_pcb != inp")); @@ -229,12 +229,12 @@ tcp_usr_detach(struct socket *so) inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_detach: inp == NULL")); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); KASSERT(inp->inp_socket != NULL, ("tcp_usr_detach: inp_socket == NULL")); tcp_detach(so, inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); } #ifdef INET @@ -588,7 +588,7 @@ tcp_usr_disconnect(struct socket *so) int error = 0; TCPDEBUG0; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_disconnect: inp == NULL")); INP_WLOCK(inp); @@ -602,7 +602,7 @@ tcp_usr_disconnect(struct socket *so) out: TCPDEBUG2(PRU_DISCONNECT); INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (error); } @@ -717,7 +717,7 @@ tcp_usr_shutdown(struct socket *so) struct tcpcb *tp = NULL; TCPDEBUG0; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = sotoinpcb(so); KASSERT(inp != NULL, ("inp == NULL")); INP_WLOCK(inp); @@ -735,7 +735,7 @@ tcp_usr_shutdown(struct socket *so) out: TCPDEBUG2(PRU_SHUTDOWN); INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (error); } @@ -797,7 +797,7 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf *m, * this call. */ if (flags & PRUS_EOF) - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_send: inp == NULL")); INP_WLOCK(inp); @@ -854,7 +854,7 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf *m, * Close the send side of the connection after * the data is sent. */ - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); socantsendmore(so); tcp_usrclosed(tp); } @@ -918,7 +918,7 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf *m, ((flags & PRUS_EOF) ? PRU_SEND_EOF : PRU_SEND)); INP_WUNLOCK(inp); if (flags & PRUS_EOF) - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (error); } @@ -935,7 +935,7 @@ tcp_usr_abort(struct socket *so) inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_abort: inp == NULL")); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); KASSERT(inp->inp_socket != NULL, ("tcp_usr_abort: inp_socket == NULL")); @@ -957,7 +957,7 @@ tcp_usr_abort(struct socket *so) inp->inp_flags |= INP_SOCKREF; } INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); } /* @@ -973,7 +973,7 @@ tcp_usr_close(struct socket *so) inp = sotoinpcb(so); KASSERT(inp != NULL, ("tcp_usr_close: inp == NULL")); - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); KASSERT(inp->inp_socket != NULL, ("tcp_usr_close: inp_socket == NULL")); @@ -996,7 +996,7 @@ tcp_usr_close(struct socket *so) inp->inp_flags |= INP_SOCKREF; } INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); } /* @@ -1627,10 +1627,10 @@ tcp_attach(struct socket *so) } so->so_rcv.sb_flags |= SB_AUTOSIZE; so->so_snd.sb_flags |= SB_AUTOSIZE; - INP_INFO_WLOCK(&V_tcbinfo); + INP_INFO_RLOCK(&V_tcbinfo); error = in_pcballoc(so, &V_tcbinfo); if (error) { - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (error); } inp = sotoinpcb(so); @@ -1646,12 +1646,12 @@ tcp_attach(struct socket *so) if (tp == NULL) { in_pcbdetach(inp); in_pcbfree(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (ENOBUFS); } tp->t_state = TCPS_CLOSED; INP_WUNLOCK(inp); - INP_INFO_WUNLOCK(&V_tcbinfo); + INP_INFO_RUNLOCK(&V_tcbinfo); return (0); } @@ -1669,7 +1669,7 @@ tcp_disconnect(struct tcpcb *tp) struct inpcb *inp = tp->t_inpcb; struct socket *so = inp->inp_socket; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); /* @@ -1707,7 +1707,7 @@ static void tcp_usrclosed(struct tcpcb *tp) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(tp->t_inpcb); switch (tp->t_state) { diff --git a/sys/netinet/toecore.c b/sys/netinet/toecore.c index 12f2c38..7e3119c5 100644 --- a/sys/netinet/toecore.c +++ b/sys/netinet/toecore.c @@ -329,7 +329,7 @@ toe_syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, { struct socket *lso = inp->inp_socket; - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); INP_WLOCK_ASSERT(inp); syncache_add(inc, to, th, inp, &lso, NULL, tod, todctx); @@ -340,7 +340,7 @@ toe_syncache_expand(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, struct socket **lsop) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); return (syncache_expand(inc, to, th, lsop, NULL)); } @@ -371,7 +371,7 @@ toe_4tuple_check(struct in_conninfo *inc, struct tcphdr *th, struct ifnet *ifp) if ((inp->inp_flags & INP_TIMEWAIT) && th != NULL) { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); /* for twcheck */ + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); /* for twcheck */ if (!tcp_twcheck(inp, NULL, th, NULL, 0)) return (EADDRINUSE); } else { @@ -575,7 +575,7 @@ toe_connect_failed(struct toedev *tod, struct inpcb *inp, int err) (void) tcp_output(tp); } else { - INP_INFO_WLOCK_ASSERT(&V_tcbinfo); + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); tp = tcp_drop(tp, err); if (tp == NULL) INP_WLOCK(inp); /* re-acquire */ diff --git a/sys/netinet6/in6_pcb.c b/sys/netinet6/in6_pcb.c index 068ac72..578d504 100644 --- a/sys/netinet6/in6_pcb.c +++ b/sys/netinet6/in6_pcb.c @@ -776,7 +776,7 @@ in6_pcbpurgeif0(struct inpcbinfo *pcbinfo, struct ifnet *ifp) struct ip6_moptions *im6o; int i, gap; - INP_INFO_RLOCK(pcbinfo); + INP_INFO_WLOCK(pcbinfo); LIST_FOREACH(in6p, pcbinfo->ipi_listhead, inp_list) { INP_WLOCK(in6p); im6o = in6p->in6p_moptions; @@ -807,7 +807,7 @@ in6_pcbpurgeif0(struct inpcbinfo *pcbinfo, struct ifnet *ifp) } INP_WUNLOCK(in6p); } - INP_INFO_RUNLOCK(pcbinfo); + INP_INFO_WUNLOCK(pcbinfo); } /* --------------010203040600060802030507 Content-Type: text/plain; charset=UTF-8; name="tcp-scale-inp-list-v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tcp-scale-inp-list-v1.patch" diff --git a/sys/netinet/in_pcb.c b/sys/netinet/in_pcb.c index 970cc78..d27086b 100644 --- a/sys/netinet/in_pcb.c +++ b/sys/netinet/in_pcb.c @@ -218,6 +218,7 @@ in_pcbinfo_init(struct inpcbinfo *pcbinfo, const char *name, INP_INFO_LOCK_INIT(pcbinfo, name); INP_HASH_LOCK_INIT(pcbinfo, "pcbinfohash"); /* XXXRW: argument? */ + INP_LIST_LOCK_INIT(pcbinfo, "pcbinfolist"); #ifdef VIMAGE pcbinfo->ipi_vnet = curvnet; #endif @@ -256,6 +257,7 @@ in_pcbinfo_destroy(struct inpcbinfo *pcbinfo) in_pcbgroup_destroy(pcbinfo); #endif uma_zdestroy(pcbinfo->ipi_zone); + INP_LIST_LOCK_DESTROY(pcbinfo); INP_HASH_LOCK_DESTROY(pcbinfo); INP_INFO_LOCK_DESTROY(pcbinfo); } @@ -302,6 +304,8 @@ in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) inp->inp_flags |= IN6P_IPV6_V6ONLY; } #endif + INP_WLOCK(inp); + INP_LIST_WLOCK(pcbinfo); LIST_INSERT_HEAD(pcbinfo->ipi_listhead, inp, inp_list); pcbinfo->ipi_count++; so->so_pcb = (caddr_t)inp; @@ -309,9 +313,9 @@ in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) if (V_ip6_auto_flowlabel) inp->inp_flags |= IN6P_AUTOFLOWLABEL; #endif - INP_WLOCK(inp); inp->inp_gencnt = ++pcbinfo->ipi_gencnt; refcount_init(&inp->inp_refcount, 1); /* Reference from inpcbinfo */ + INP_LIST_WUNLOCK(pcbinfo); #if defined(IPSEC) || defined(MAC) out: if (error != 0) { @@ -1199,8 +1203,10 @@ in_pcbfree(struct inpcb *inp) if (inp->inp_sp != NULL) ipsec_delete_pcbpolicy(inp); #endif + INP_LIST_WLOCK(pcbinfo); inp->inp_gencnt = ++pcbinfo->ipi_gencnt; in_pcbremlists(inp); + INP_LIST_WUNLOCK(pcbinfo); #ifdef INET6 if (inp->inp_vflag & INP_IPV6PROTO) { ip6_freepcbopts(inp->in6p_outputopts); @@ -2043,6 +2049,7 @@ in_pcbremlists(struct inpcb *inp) INP_INFO_WLOCK_ASSERT(pcbinfo); INP_WLOCK_ASSERT(inp); + INP_LIST_WLOCK_ASSERT(pcbinfo); inp->inp_gencnt = ++pcbinfo->ipi_gencnt; if (inp->inp_flags & INP_INHASHLIST) { diff --git a/sys/netinet/in_pcb.h b/sys/netinet/in_pcb.h index 7cfc72a..a58eea4 100644 --- a/sys/netinet/in_pcb.h +++ b/sys/netinet/in_pcb.h @@ -132,19 +132,20 @@ struct icmp6_filter; * and IPv6 sockets. In the case of TCP, further per-connection state is * hung off of inp_ppcb most of the time. Almost all fields of struct inpcb * are static after creation or protected by a per-inpcb rwlock, inp_lock. A - * few fields also require the global pcbinfo lock for the inpcb to be held, - * when modified, such as the global connection lists and hashes, as well as - * binding information (which affects which hash a connection is on). This - * model means that connections can be looked up without holding the - * per-connection lock, which is important for performance when attempting to - * find the connection for a packet given its IP and port tuple. Writing to - * these fields that write locks be held on both the inpcb and global locks. + * few fields also require the global pcblist lock for the inpcb to be held, + * when modified, such as the global connection lists. This model means that + * connections can be looked up without holding the per-connection lock, which + * is important for performance when attempting to find the connection for a + * packet given its IP and port tuple. Writing to these fields that write + * locks be held on both the inpcb and global locks. * * Key: * (c) - Constant after initialization * (g) - Protected by the pcbgroup lock * (i) - Protected by the inpcb lock * (p) - Protected by the pcbinfo lock for the inpcb + * (l) - Protected by the pcblist lock for the inpcb + * (h) - Protected by the pcbhash lock for the inpcb * (s) - Protected by another subsystem's locks * (x) - Undefined locking * @@ -161,13 +162,13 @@ struct icmp6_filter; * The inp_vflag field is overloaded, and would otherwise ideally be (c). */ struct inpcb { - LIST_ENTRY(inpcb) inp_hash; /* (i/p) hash list */ + LIST_ENTRY(inpcb) inp_hash; /* (i/h) hash list */ LIST_ENTRY(inpcb) inp_pcbgrouphash; /* (g/i) hash list */ - LIST_ENTRY(inpcb) inp_list; /* (i/p) list for all PCBs for proto */ + LIST_ENTRY(inpcb) inp_list; /* (i/l) list for all PCBs for proto */ void *inp_ppcb; /* (i) pointer to per-protocol pcb */ struct inpcbinfo *inp_pcbinfo; /* (c) PCB list info */ struct inpcbgroup *inp_pcbgroup; /* (g/i) PCB group list */ - LIST_ENTRY(inpcb) inp_pcbgroup_wild; /* (g/i/p) group wildcard entry */ + LIST_ENTRY(inpcb) inp_pcbgroup_wild; /* (g/i/h) group wildcard entry */ struct socket *inp_socket; /* (i) back pointer to socket */ struct ucred *inp_cred; /* (c) cache of socket cred */ u_int32_t inp_flow; /* (i) IPv6 flow information */ @@ -185,7 +186,7 @@ struct inpcb { * general use */ /* Local and foreign ports, local and foreign addr. */ - struct in_conninfo inp_inc; /* (i/p) list for PCB's local port */ + struct in_conninfo inp_inc; /* (i) list for PCB's local port */ /* MAC and IPSEC policy information. */ struct label *inp_label; /* (i) MAC label */ @@ -210,8 +211,8 @@ struct inpcb { int inp6_cksum; short inp6_hops; } inp_depend6; - LIST_ENTRY(inpcb) inp_portlist; /* (i/p) */ - struct inpcbport *inp_phd; /* (i/p) head of this list */ + LIST_ENTRY(inpcb) inp_portlist; /* (i/h) */ + struct inpcbport *inp_phd; /* (i/h) head of this list */ #define inp_zero_size offsetof(struct inpcb, inp_gencnt) inp_gen_t inp_gencnt; /* (c) generation count */ struct llentry *inp_lle; /* cached L2 information */ @@ -275,16 +276,24 @@ struct inpcbport { * Global data structure for each high-level protocol (UDP, TCP, ...) in both * IPv4 and IPv6. Holds inpcb lists and information for managing them. * - * Each pcbinfo is protected by two locks: ipi_lock and ipi_hash_lock, - * the former covering mutable global fields (such as the global pcb list), - * and the latter covering the hashed lookup tables. The lock order is: + * Each pcbinfo is protected by three locks: ipi_lock, ipi_hash_lock and + * ipi_list_lock: + * - ipi_lock covering the global pcb list stability during loop iteration, + * - ipi_hash_lock covering the hashed lookup tables, + * - ipi_list_lock covering mutable global fields (such as the global + * pcb list) * - * ipi_lock (before) inpcb locks (before) {ipi_hash_lock, pcbgroup locks} + * The lock order is: + * + * ipi_lock (before) + * inpcb locks (before) + * {ipi_hash_lock, ipi_list_lock, pcbgroup locks} * * Locking key: * * (c) Constant or nearly constant after initialisation * (g) Locked by ipi_lock + * (l) Locked by ipi_list_lock * (h) Read using either ipi_hash_lock or inpcb lock; write requires both * (p) Protected by one or more pcbgroup locks * (x) Synchronisation properties poorly defined @@ -298,14 +307,14 @@ struct inpcbinfo { /* * Global list of inpcbs on the protocol. */ - struct inpcbhead *ipi_listhead; /* (g) */ - u_int ipi_count; /* (g) */ + struct inpcbhead *ipi_listhead; /* (g/l) */ + u_int ipi_count; /* (g/l) */ /* * Generation count -- incremented each time a connection is allocated * or freed. */ - u_quad_t ipi_gencnt; /* (g) */ + u_quad_t ipi_gencnt; /* (g/l) */ /* * Fields associated with port lookup and allocation. @@ -363,6 +372,11 @@ struct inpcbinfo { * general use 2 */ void *ipi_pspare[2]; + + /* + * Global lock protecting global inpcb list, inpcb count, etc. + */ + struct rwlock ipi_list_lock; }; #ifdef _KERNEL @@ -462,6 +476,25 @@ short inp_so_options(const struct inpcb *inp); #define INP_INFO_WLOCK_ASSERT(ipi) rw_assert(&(ipi)->ipi_lock, RA_WLOCKED) #define INP_INFO_UNLOCK_ASSERT(ipi) rw_assert(&(ipi)->ipi_lock, RA_UNLOCKED) +#define INP_LIST_LOCK_INIT(ipi, d) \ + rw_init_flags(&(ipi)->ipi_list_lock, (d), 0) +#define INP_LIST_LOCK_DESTROY(ipi) rw_destroy(&(ipi)->ipi_list_lock) +#define INP_LIST_RLOCK(ipi) rw_rlock(&(ipi)->ipi_list_lock) +#define INP_LIST_WLOCK(ipi) rw_wlock(&(ipi)->ipi_list_lock) +#define INP_LIST_TRY_RLOCK(ipi) rw_try_rlock(&(ipi)->ipi_list_lock) +#define INP_LIST_TRY_WLOCK(ipi) rw_try_wlock(&(ipi)->ipi_list_lock) +#define INP_LIST_TRY_UPGRADE(ipi) rw_try_upgrade(&(ipi)->ipi_list_lock) +#define INP_LIST_RUNLOCK(ipi) rw_runlock(&(ipi)->ipi_list_lock) +#define INP_LIST_WUNLOCK(ipi) rw_wunlock(&(ipi)->ipi_list_lock) +#define INP_LIST_LOCK_ASSERT(ipi) \ + rw_assert(&(ipi)->ipi_list_lock, RA_LOCKED) +#define INP_LIST_RLOCK_ASSERT(ipi) \ + rw_assert(&(ipi)->ipi_list_lock, RA_RLOCKED) +#define INP_LIST_WLOCK_ASSERT(ipi) \ + rw_assert(&(ipi)->ipi_list_lock, RA_WLOCKED) +#define INP_LIST_UNLOCK_ASSERT(ipi) \ + rw_assert(&(ipi)->ipi_list_lock, RA_UNLOCKED) + #define INP_HASH_LOCK_INIT(ipi, d) \ rw_init_flags(&(ipi)->ipi_hash_lock, (d), 0) #define INP_HASH_LOCK_DESTROY(ipi) rw_destroy(&(ipi)->ipi_hash_lock) diff --git a/sys/netinet/tcp_subr.c b/sys/netinet/tcp_subr.c index fb2c415..cf7c02f 100644 --- a/sys/netinet/tcp_subr.c +++ b/sys/netinet/tcp_subr.c @@ -1177,8 +1177,10 @@ tcp_pcblist(SYSCTL_HANDLER_ARGS) * OK, now we're committed to doing something. */ INP_INFO_RLOCK(&V_tcbinfo); + INP_LIST_RLOCK(&V_tcbinfo); gencnt = V_tcbinfo.ipi_gencnt; n = V_tcbinfo.ipi_count; + INP_LIST_RUNLOCK(&V_tcbinfo); INP_INFO_RUNLOCK(&V_tcbinfo); m = syncache_pcbcount(); @@ -1285,9 +1287,11 @@ tcp_pcblist(SYSCTL_HANDLER_ARGS) * might be necessary to retry. */ INP_INFO_RLOCK(&V_tcbinfo); + INP_LIST_RLOCK(&V_tcbinfo); xig.xig_gen = V_tcbinfo.ipi_gencnt; xig.xig_sogen = so_gencnt; xig.xig_count = V_tcbinfo.ipi_count + pcb_count; + INP_LIST_RUNLOCK(&V_tcbinfo); INP_INFO_RUNLOCK(&V_tcbinfo); error = SYSCTL_OUT(req, &xig, sizeof xig); } --------------010203040600060802030507-- From owner-freebsd-net@FreeBSD.ORG Fri May 23 21:37:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1B66CEF for ; Fri, 23 May 2014 21:37:42 +0000 (UTC) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9448D290E for ; Fri, 23 May 2014 21:37:42 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id rq2so4663977pbb.5 for ; Fri, 23 May 2014 14:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=KgztaJO+UltdvUKl/DKEkoHnYJOHkZZL9Nt3cKJx8TI=; b=UoQznAG5cysKuJ2fCzVGfQ9AullOUjPcAnTL2w6oaU6v/asGX4FD5OKEjHmFDRI5/3 SodAJHM3aAoOKanr2jyBkSQ+wW3RklF6eG2R5/52n8vRYTwvW/np1CZ0G4ZaVjiGHiB+ IejdOYs1r9MDedVNfI+KORwEbeY1Q/s4xSaF9+492D3EzD52NgePZpnsJoN8z8YTH2QF xQOQHqqnrsi7m2TL5PQPVWzMfFLcS4ZN/3ZOUszthtLlzxAMJTGhXj7Q319EIytHrdQq 0miO4TOi1/QXZHIUpySvc8k7zl24mz2fOVR6JMihb55f7fIVod+qKVxIvaUarjW/UAIU NUVg== X-Received: by 10.67.23.135 with SMTP id ia7mr9194757pad.5.1400881062173; Fri, 23 May 2014 14:37:42 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id de5sm6148239pbc.66.2014.05.23.14.37.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 23 May 2014 14:37:41 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <537FBFA4.1010902@FreeBSD.org> Date: Fri, 23 May 2014 14:37:40 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Julien Charbon , freebsd-net@freebsd.org Subject: Re: TCP stack lock contention with short-lived connections References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> In-Reply-To: <537FB51D.2060401@verisign.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 21:37:42 -0000 On 05/23/14 13:52, Julien Charbon wrote: > > Hi, > > On 23/05/14 14:06, Julien Charbon wrote: >> On 27/02/14 11:32, Julien Charbon wrote: >>> On 07/11/13 14:55, Julien Charbon wrote: >>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>> wrote: >>>>> I have put technical and how-to-repeat details in below PR: >>>>> >>>>> kern/183659: TCP stack lock contention with short-lived connections >>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>> >>>>> We are currently working on this performance improvement effort; it >>>>> will impact only the TCP locking strategy not the TCP stack logic >>>>> itself. We will share on freebsd-net the patches we made for >>>>> reviewing and improvement propositions; anyway this change might also >>>>> require enough eyeballs to avoid tricky race conditions introduction >>>>> in TCP stack. > > Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and > tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan 2014. > > First one is (tcp-scale-inp-list-v1.patch): > > [tcp-scaling] Introduce the INP_LIST global mutex for protecting pcbinfo > global structures > https://github.com/verisign/freebsd/commit/12c62273f052911aabe6ed283cea76cdd72c9493 > > > This change improves nothing in performance (neither degrades), it > simply implements what we are trying to achieve: Decompose further > pcbinfo lock (aka ipi_lock or INP_INFO). > > Ideally, pcbinfo globally shared structures are protected by leaf > mutexes (mutexes that are taken last), not by a root mutex (mutex taken > first). The current lock ordering is: > > ipi_lock > inpcb lock > ipi_hash_lock, pcbgroup locks > > ipi_lock being a root mutex is explained by its original task: Protect > the pcbinfo as a whole. > > Then, with this change, we added a new ipi_list_lock leaf mutex > dedicated to protect structures previously under ipi_lock umbrella, i.e.: > > - inpcb global list: ipi_listhead > - inpcb global list counter: ipi_count > - inpcb global list generated index: ipi_gencnt > > and it permits to implement the second (meatier) change > (tcp-scale-pcbinfo-rlock-v1.patch): > > [alpha][tcp-scaling] Use INP_INFO_RLOCK in critical path, and use > INP_INFO_WLOCK in full INP loops. > https://github.com/verisign/freebsd/commit/4633ac8c0b8d379fbda5fb9ffc921c2e4786db46 > > > Now that ipi_lost has lost is duty to protect pcbinfo globally shared > structures, its last (clear) duty is to hold inp creation/destruction > when a full traversal of global inp list is performed, as this > traversals expect inp list to be stable, e.g.: > > tcp_ccalgounload() > https://github.com/verisign/freebsd/blob/388f0a87958fde5e644e01798f44b58588eb1dc2/sys/netinet/tcp_subr.c#L848 > > > Thus (performance-wise) critical paths can now take ipi_lock _read_ > lock, e.g.: > > tcp_input() > tcp_usr_shutdown() > tcp_usr_close() > tcp_twstart() > > and, on the other side, functions performing full inp list traversal > will take the INP_INFO _write_ lock: > > tcp_ccalgounload() > tcp_pcblist() > in_pcbpurgeif0() > etc... > > This patch doubles the performance improvement with our short-live TCP > workload. > > _However_ it would be a miracle that this change does not introduce new > race condition(s) (hence the 'alpha' tag in commit message). Most of > TCP stack locking strategy being now on inpcb lock shoulders. That > said, from our tests point of view, this change is completely stable: No > kernel/lock assertion, no unexpected TCP behavior, stable performance > results. Moreover, before tagging this change as 'beta' we need to test > more thoroughly these features: > > - VNET, > - PCBGROUP/RSS/TCP timer per cpu, > - TCP Offloading (we need a NIC with a good TCP offloading support) I can assess the impact (and fix any fallout) on the parts of the kernel that deal with TCP_OFFLOAD, and the TOE driver in dev/cxgbe/tom. But I was hoping to do that only after there was general agreement on net@ that these locking changes are sound and should be taken into HEAD. Lack of reviews seems to be holding this back, correct? Regards, Navdeep > > Early testers, test ideas, reviewers and memories about previous (and > not documented or unclear) ipi_lock duties are more than welcome. > > Thanks. > > -- > Julien > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat May 24 02:31:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C960F30 for ; Sat, 24 May 2014 02:31:46 +0000 (UTC) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 410352E17 for ; Sat, 24 May 2014 02:31:46 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id kq14so4889286pab.33 for ; Fri, 23 May 2014 19:31:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-type:content-transfer-encoding:thread-index :content-language; bh=S5yNgy3SMOYELRUaD8vb7E7pGUdwiZ4Pzov4PgiHNbM=; b=I7JTpLUx9DTAu3Y4ufrTIllPvkkf2QofCo6XYbJu6yg5I8f6FFF2U56LRXePwpZCl6 LRT2wZQRX5VySdtTP5S0sGkoglBHeq2yYmyWTCgwmdqm0dqU4CWxsUXWN9I35AP6eZEL QXDDWr9qoh/49zGog/OXF7M1PPEH+r6YQG5VanCtBUxbjL0laVUnKgCkWnGLAvIMrRYJ wEq0NnOd37JXwjdaCrhXDAqeRGwct7Ou/TKmg52xhkCUdpNr3nMBbHlR4KIcNQ2iyJuf o5QfI8mUNlAriqktwWTxAFXvIIbvxU8pIvhSmyllaOPcOSHXzHFYK3oVqN2ft6SgWDxp X6Zw== X-Received: by 10.66.139.73 with SMTP id qw9mr10596042pab.123.1400898705891; Fri, 23 May 2014 19:31:45 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id jq6sm6766090pbb.76.2014.05.23.19.31.43 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 23 May 2014 19:31:45 -0700 (PDT) From: "bycn82" To: "'Sato Kentney'" , References: In-Reply-To: Subject: RE: ipfw table add problem Date: Sat, 24 May 2014 10:31:42 +0800 Message-ID: <002c01cf76f8$4ecc1bc0$ec645340$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJj8+XdRrhatgfYjSz9AQe3UoEI2ZomAkBQ Content-Language: en-us X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 02:31:46 -0000 Hi Sato, My fix is a temp solution, and actually you can just update the source = code to the latest version or use others. According to what I know, = developer Alex is currently working/enhancing the "ipfw table" feature. = I think you will like the new features. Best Regards, bycn82 -----Original Message----- From: owner-freebsd-net@freebsd.org = [mailto:owner-freebsd-net@freebsd.org] On Behalf Of Sato Kentney Sent: 24 May, 2014 2:07 To: freebsd-net@freebsd.org Subject: ipfw table add problem Hello. I'm using free bsd 10 stable. and I meet same issue as http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/189471 currently using thee fix from "bycm82"',it work so thank you but. recently I meet some issue in using ipfw with table. and found some PR = about the table in ipfw. can I know what is happening? sorry about my poor english. Thanks, Sato K(=E4=BD=90=E8=97=A4=E6=9F=AF=E5=BE=B7) _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat May 24 07:40:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE4DC99B for ; Sat, 24 May 2014 07:40:58 +0000 (UTC) Received: from mx7.valuehost.ru (mx7.valuehost.ru [217.112.42.214]) by mx1.freebsd.org (Postfix) with ESMTP id 823A623B3 for ; Sat, 24 May 2014 07:40:58 +0000 (UTC) Received: (from mail@localhost) by mx7.valuehost.ru (8.12.9/8.13.1) id s4O7exfL030220 for freebsd-net@freebsd.org; Sat, 24 May 2014 11:40:59 +0400 (MSD) X-Authentication-Warning: mx7.valuehost.ru: mail set sender to @ using -f Received: from localhost (localhost [127.0.0.1]) by mail.valuehost.ru (Horde) with HTTP for ; Sat, 24 May 2014 11:40:59 +0400 Message-ID: <1400917259.c526ccc8386cc@mail.valuehost.ru> Date: Sat, 24 May 2014 11:40:59 +0400 From: ppb@.SYNTAX-ERROR To: freebsd-net@freebsd.org Subject: Re: mbuf_cluster (FAIL SLEEP) MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1251" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 07:40:59 -0000 > Yes, this is normal and it doesn't mean what you might expect. It's a > generic failure counter, not an allocation failure counter. eg: if an > object that was just freed fails to fit in a per-cpu free items cache it > counts as a "FAIL". Hello, thank you for answer Peter. But what do think about this: -------------------------------------- # netstat -an |wc -l 29616 # netstat -an | grep -c TIME_WAIT 27800 + kernel stops showing realtime LA: show normal top, but no changes at first line output: load averages: 2.36, 2.32, 2.28 ^ ^ ^ | | | -------------------------------------- sysctl net.inet.tcp.msl and tcpdrop do not help, after killing TIME_WAIT this connection state start grow again till 27800. I have to reboot servers couse some apache process goes to state and stop responding. Normal traffic and no ddos at this moment. Again, this problem (netstat & LA) happen at the same time..... From owner-freebsd-net@FreeBSD.ORG Sat May 24 16:19:19 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7F822A5; Sat, 24 May 2014 16:19:19 +0000 (UTC) Received: from mail-pb0-x233.google.com (mail-pb0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B208727FA; Sat, 24 May 2014 16:19:19 +0000 (UTC) Received: by mail-pb0-f51.google.com with SMTP id ma3so5626946pbc.10 for ; Sat, 24 May 2014 09:19:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:mime-version:content-type :thread-index:content-language; bh=kufZk8ihhzFTR05hSHTM6nfNdMjkkw0C6oxWA/fU4FA=; b=U7y1rxPi//x/WrJghgGNDpIh7ZT7ub3cXkm4tDG59XqVuA0oR/0NVETSM9DIzG0idk Nj6tz7etSitDVrsdzCyvLryvu8H2eTIuUzFOlqQ3w1LWjhJxasnBapLwKsBAkntSoYtj mzmP3YyFI5hCy21iqt3XkDNlTWYOe6gaHs6caVYCo3jQbXSCUZyg+xPXyI3MsgXxgRxv TbYlooyRihUZduhdMZ/yD1hMHFJDFeedxvn0n9FGRmgjLoAazsc//EITxFD123AYIjp6 +DUrn9m2OQSCBKyGhooqd351BCJBW6cSU4+LGJbjWGnyTlPGtFwM/8TRKzovMK9djkf8 sg5w== X-Received: by 10.69.19.139 with SMTP id gu11mr15025473pbd.36.1400948359292; Sat, 24 May 2014 09:19:19 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id pq3sm10031019pbb.57.2014.05.24.09.19.15 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 24 May 2014 09:19:18 -0700 (PDT) From: "bycn82" To: "Alexander V. Chernikov" , "Luigi Rizzo" Subject: a defect in ipfw dummynet Date: Sun, 25 May 2014 00:19:15 +0800 Message-ID: <000f01cf776b$ea602810$bf207830$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac93a95BPtMHR/tZQky39x87iu6lXA== Content-Language: en-us Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 16:19:20 -0000 Hi Alexander, =20 Since you guys are working on the =E2=80=9Cnamed table=E2=80=9D feature. = So I have stopped implementing it using my way. Hence I got some time to = read more about the existing codes. This afternoon I just started to = read the dummynet part, then I have another question to ask. Maybe it is = not a small defect, Or just because there are some more story which I = don=E2=80=99t know. anyway. =20 For example, when we run command as below,=20 >ipfw pipe config 1 100kbps the userspace program need to read the bandwidth value which is = =E2=80=9C100kbps=E2=80=9D, And I found the code as below,=20 if ((*end =3D=3D 'B' && _substrcmp2(end, "Bi", "Bit/s") !=3D 0) = || _substrcmp2(end, "by", "bytes") =3D=3D = 0) bw *=3D 8; =20 Sure it works. But I want to ask whether it can be more readable If we = list down all the possibilities and directly =E2=80=9Chard code=E2=80=9D = in the source, At least it can be more accurate.=20 =20 With current logic, we have change to have below situation. =20 root@FB10Head:~ # ipfw pipe config 1 bw 1ByeBye <- = the command will be considered as =E2=80=9C1 Byte per second=E2=80=9D root@FB10Head:~ # ipfw pipe 1 show 00001: 8.000 bit/s 0 ms burst 0=20 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 = droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active root@FB10Head:~ # =20 =20 =20 =20 =20 Best Regards, Bycn82 =20 From owner-freebsd-net@FreeBSD.ORG Sat May 24 17:08:36 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70CF32EE; Sat, 24 May 2014 17:08:36 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 352B02B98; Sat, 24 May 2014 17:08:36 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id fb1so5574729pad.9 for ; Sat, 24 May 2014 10:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=viJfYpG4Fil/8uRT13XMwYf1QVjWglYO8uMwiWjBzAw=; b=gF/YK3lVpAstPdvxXNpaaI8ZimWU82OJzQ19A5zhlONySKl+9Za8wYoemPGAA0HCt2 R0kKW+I8tXY8YgdHdVH50jLV7xX99Q+sOOv9MIp6K5xiOkS2+Hkq4Yq1lzygTM9FbJIi 8Yz6T6axAq9l1GsjGxf94Ybb8b7c1eRFmBylYmUrFwdDB+UyqDxzt9CcuVCP5ibA4+e4 oRny54l0sGbWrFIlhGS7NmbiZcRUStV6/Oae/WFa6ZGzRH99LFgBn14NLYKIhA3S8+Ix UFGfHGGnl1Jt3zNfrXZ0j5nNbAI7GDdSU5Vz0pqHjvUFgt23/bUPUOQGktK67bbpv1pJ W1Qw== X-Received: by 10.68.139.137 with SMTP id qy9mr15297860pbb.11.1400951315751; Sat, 24 May 2014 10:08:35 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id wq10sm31439693pac.24.2014.05.24.10.08.31 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 24 May 2014 10:08:35 -0700 (PDT) From: "bycn82" To: "Alexander V. Chernikov" , "Luigi Rizzo" References: In-Reply-To: Subject: RE: a defect in ipfw dummynet Date: Sun, 25 May 2014 01:08:31 +0800 Message-ID: <003701cf7772$cc3bec50$64b3c4f0$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac93a95BPtMHR/tZQky39x87iu6lXAAAn8aA Content-Language: en-us Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 17:08:36 -0000 Hi , After I think it twice, I think the code and the document are OK, But = the problem is from the examples/document on the internet, Because the = existing logic is not accurate. So users can use different measurements = instead of these four which mentioned in the document. For example most = of time I would like to use `kbps` and `KBps`, (It is my fault, I did = not follow the document, ) =20 I would like to recommend to make some changes in the document only,=20 =20 Current document bw bandwidth | device Bandwidth, measured in [K|M]{bit/s|Byte/s}. =20 Change to=20 bw bandwidth | device Bandwidth, support 4 measurements,(Kbit/s, Mbit/s, KByte/s, = MByte/s) all others are officially not recommended. =20 With this document, Some mistakes can be prevented . for example this = one: >ipfw pipe 1 config bw 1BIT/s =20 =20 =20 =20 Best Regards, Bycn82 =20 =20 From: bycn82 [mailto:bycn82@gmail.com]=20 Sent: 25 May, 2014 0:19 To: Alexander V. Chernikov; Luigi Rizzo Cc: FreeBSD Net Subject: a defect in ipfw dummynet =20 Hi Alexander, =20 Since you guys are working on the =E2=80=9Cnamed table=E2=80=9D feature. = So I have stopped implementing it using my way. Hence I got some time to = read more about the existing codes. This afternoon I just started to = read the dummynet part, then I have another question to ask. Maybe it is = not a small defect, Or just because there are some more story which I = don=E2=80=99t know. anyway. =20 For example, when we run command as below,=20 >ipfw pipe config 1 100kbps the userspace program need to read the bandwidth value which is = =E2=80=9C100kbps=E2=80=9D, And I found the code as below,=20 if ((*end =3D=3D 'B' && _substrcmp2(end, "Bi", "Bit/s") !=3D 0) = || _substrcmp2(end, "by", "bytes") =3D=3D = 0) bw *=3D 8; =20 Sure it works. But I want to ask whether it can be more readable If we = list down all the possibilities and directly =E2=80=9Chard code=E2=80=9D = in the source, At least it can be more accurate.=20 =20 With current logic, we have change to have below situation. =20 root@FB10Head:~ # ipfw pipe config 1 bw 1ByeBye <- = the command will be considered as =E2=80=9C1 Byte per second=E2=80=9D root@FB10Head:~ # ipfw pipe 1 show 00001: 8.000 bit/s 0 ms burst 0=20 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 = droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active root@FB10Head:~ # =20 =20 =20 =20 =20 Best Regards, Bycn82 =20 From owner-freebsd-net@FreeBSD.ORG Sun May 25 04:11:24 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B2DD12B; Sun, 25 May 2014 04:11:24 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EF612A2B; Sun, 25 May 2014 04:11:23 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id lf10so5947225pab.34 for ; Sat, 24 May 2014 21:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=qNJ2SPr//FCRoDwJHKigYeNGs2lThGfL1krvOlISKDE=; b=wJDZljaLLRjnkyl+3H4BF2KXjhc9l8bXDf/WLU4Z/9QhH78lAdwhrPVfeKhRanvoAA /Oba4RZ10gWHO5ioUtxQhjSQXFE8vplMP9EfaxbYBS5GQOppx03HGqqD3ZL/3VHb9hMn Ie7R70bjLEzkTEWzgIYj2WQrXmBvzHKqn1N0kI4o+LQ96vr3ZmZhGck0q8yakTTHHAVU z5qjJq3fgMaTFypiWr9ixEXmjj9gUmyYYF9wi148uIULYZ12H0E7TljwGtWsqi0ZBtah ImPs9m3hYrUUjTHRxpOI9tGd0pfG4AbX+4b8vzPwDX18AcZuQcSc3qSFZK0ZW72mxaZI 4fMg== X-Received: by 10.66.180.141 with SMTP id do13mr17709439pac.93.1400991082521; Sat, 24 May 2014 21:11:22 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id g6sm37732421pat.2.2014.05.24.21.11.18 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 24 May 2014 21:11:22 -0700 (PDT) From: "bycn82" To: "Alexander V. Chernikov" , "Luigi Rizzo" References: In-Reply-To: Subject: RE: a defect in ipfw dummynet Date: Sun, 25 May 2014 12:11:15 +0800 Message-ID: <000901cf77cf$616c0b00$24442100$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac93a95BPtMHR/tZQky39x87iu6lXAAAn8aAABgk6oA= Content-Language: en-us Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 04:11:24 -0000 Ok, anyway, ignore it. :) =20 From: bycn82 [mailto:bycn82@gmail.com]=20 Sent: 25 May, 2014 1:09 To: Alexander V. Chernikov; Luigi Rizzo Cc: FreeBSD Net Subject: RE: a defect in ipfw dummynet =20 Hi , After I think it twice, I think the code and the document are OK, But = the problem is from the examples/document on the internet, Because the = existing logic is not accurate. So users can use different measurements = instead of these four which mentioned in the document. For example most = of time I would like to use `kbps` and `KBps`, (It is my fault, I did = not follow the document, ) =20 I would like to recommend to make some changes in the document only,=20 =20 Current document bw bandwidth | device Bandwidth, measured in [K|M]{bit/s|Byte/s}. =20 Change to=20 bw bandwidth | device Bandwidth, support 4 measurements,(Kbit/s, Mbit/s, KByte/s, = MByte/s) all others are officially not recommended. =20 With this document, Some mistakes can be prevented . for example this = one: >ipfw pipe 1 config bw 1BIT/s =20 =20 =20 =20 Best Regards, Bycn82 =20 =20 From: bycn82 [mailto:bycn82@gmail.com]=20 Sent: 25 May, 2014 0:19 To: Alexander V. Chernikov; Luigi Rizzo Cc: FreeBSD Net Subject: a defect in ipfw dummynet =20 Hi Alexander, =20 Since you guys are working on the =E2=80=9Cnamed table=E2=80=9D feature. = So I have stopped implementing it using my way. Hence I got some time to = read more about the existing codes. This afternoon I just started to = read the dummynet part, then I have another question to ask. Maybe it is = not a small defect, Or just because there are some more story which I = don=E2=80=99t know. anyway. =20 For example, when we run command as below,=20 >ipfw pipe config 1 100kbps the userspace program need to read the bandwidth value which is = =E2=80=9C100kbps=E2=80=9D, And I found the code as below,=20 if ((*end =3D=3D 'B' && _substrcmp2(end, "Bi", "Bit/s") !=3D 0) = || _substrcmp2(end, "by", "bytes") =3D=3D = 0) bw *=3D 8; =20 Sure it works. But I want to ask whether it can be more readable If we = list down all the possibilities and directly =E2=80=9Chard code=E2=80=9D = in the source, At least it can be more accurate.=20 =20 With current logic, we have change to have below situation. =20 root@FB10Head:~ # ipfw pipe config 1 bw 1ByeBye <- = the command will be considered as =E2=80=9C1 Byte per second=E2=80=9D root@FB10Head:~ # ipfw pipe 1 show 00001: 8.000 bit/s 0 ms burst 0=20 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 = droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active root@FB10Head:~ # =20 =20 =20 =20 =20 Best Regards, Bycn82 =20 From owner-freebsd-net@FreeBSD.ORG Mon May 26 09:19:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B07B13C5 for ; Mon, 26 May 2014 09:19:36 +0000 (UTC) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 756102AEB for ; Mon, 26 May 2014 09:19:35 +0000 (UTC) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id BAA0D2700B7B for ; Mon, 26 May 2014 11:11:31 +0200 (CEST) Received: from stever.netasq.com (unknown [10.2.0.1]) by work.netasq.com (Postfix) with ESMTPA id 5DA262700B77 for ; Mon, 26 May 2014 11:11:31 +0200 (CEST) Message-ID: <53830546.1080309@netasq.com> Date: Mon, 26 May 2014 11:11:34 +0200 From: Steve Read User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Problem: no locking around IPv6 prefix structures in prelist_remove Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 09:19:36 -0000 I have recently encountered an interesting double-free crash in prelist_remove() (management of IPv6 prefixes used by interface addresses) using a modified version of 9.2. We've seen this once. It appears that two userland threads tried simultaneously to remove the last interface address that referenced a particular prefix, and both, therefore, tried to remove it from the global list of prefixes. (Feel free to correct my interpretation of the purpose of prelist_remove and how it is invoked.) One of them succeeded, and the other was left holding a chunk of free()ed memory, and crashed when trying to delete it. I looked at the code surrounding this function, and I can find no sign of locking around the prefix list or, indeed, anywhere in the call-stack (sys_ioctl=>kern_ioctl=>soo_ioctl==>ifi_ioctl=>in6_control=>prelist_remove). I looked in HEAD, and this part of the code appears to be more or less the same, in particular the question of locking. Should I submit a PR (no, we can't retry with a generic kernel)? -- Steve Read From owner-freebsd-net@FreeBSD.ORG Mon May 26 09:25:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 489EC59F for ; Mon, 26 May 2014 09:25:08 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 003802BA8 for ; Mon, 26 May 2014 09:25:07 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 7FAFB25D3810; Mon, 26 May 2014 09:25:05 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id AA4EAC22BFD; Mon, 26 May 2014 09:25:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id TCBAFir-Mxmz; Mon, 26 May 2014 09:25:03 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:d98d:32f5:c3e6:8937] (unknown [IPv6:fde9:577b:c1a9:4410:d98d:32f5:c3e6:8937]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id B4331C22BF5; Mon, 26 May 2014 09:25:01 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Problem: no locking around IPv6 prefix structures in prelist_remove From: "Bjoern A. Zeeb" In-Reply-To: <53830546.1080309@netasq.com> Date: Mon, 26 May 2014 09:24:42 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <232CE242-ECDD-4627-AE44-8265B9CC4690@FreeBSD.org> References: <53830546.1080309@netasq.com> To: Steve Read X-Mailer: Apple Mail (2.1878.2) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 09:25:08 -0000 On 26 May 2014, at 09:11 , Steve Read wrote: > I have recently encountered an interesting double-free crash in = prelist_remove() (management of IPv6 prefixes used by interface = addresses) using a modified version of 9.2. We've seen this once. >=20 > It appears that two userland threads tried simultaneously to remove = the last interface address that referenced a particular prefix, and = both, therefore, tried to remove it from the global list of prefixes. = (Feel free to correct my interpretation of the purpose of prelist_remove = and how it is invoked.) One of them succeeded, and the other was left = holding a chunk of free()ed memory, and crashed when trying to delete = it. >=20 > I looked at the code surrounding this function, and I can find no sign = of locking around the prefix list or, indeed, anywhere in the call-stack = (sys_ioctl=3D>kern_ioctl=3D>soo_ioctl=3D=3D>ifi_ioctl=3D>in6_control=3D>pr= elist_remove). I looked in HEAD, and this part of the code appears to be = more or less the same, in particular the question of locking. >=20 > Should I submit a PR (no, we can't retry with a generic kernel)? No need to for either. markj@ has a patch to fix a good deal of racy prefix list locking which = needs review and testing. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Mon May 26 09:38:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29B80A92; Mon, 26 May 2014 09:38:39 +0000 (UTC) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id E25692CDE; Mon, 26 May 2014 09:38:38 +0000 (UTC) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 3BE372700B7B; Mon, 26 May 2014 11:38:37 +0200 (CEST) Received: from stever.netasq.com (unknown [10.2.0.1]) by work.netasq.com (Postfix) with ESMTPA id 0ED202700AFD; Mon, 26 May 2014 11:38:37 +0200 (CEST) Message-ID: <53830B9D.40509@netasq.com> Date: Mon, 26 May 2014 11:38:37 +0200 From: Steve Read User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: "Bjoern A. Zeeb" Subject: Re: Problem: no locking around IPv6 prefix structures in prelist_remove References: <53830546.1080309@netasq.com> <232CE242-ECDD-4627-AE44-8265B9CC4690@FreeBSD.org> In-Reply-To: <232CE242-ECDD-4627-AE44-8265B9CC4690@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 09:38:39 -0000 OK, thanks. -- Steve Read On 26.05.2014 11:24, Bjoern A. Zeeb wrote: > On 26 May 2014, at 09:11 , Steve Read wrote: > >> I have recently encountered an interesting double-free crash in prelist_remove() (management of IPv6 prefixes used by interface addresses) using a modified version of 9.2. We've seen this once. >> >> It appears that two userland threads tried simultaneously to remove the last interface address that referenced a particular prefix, and both, therefore, tried to remove it from the global list of prefixes. (Feel free to correct my interpretation of the purpose of prelist_remove and how it is invoked.) One of them succeeded, and the other was left holding a chunk of free()ed memory, and crashed when trying to delete it. >> >> I looked at the code surrounding this function, and I can find no sign of locking around the prefix list or, indeed, anywhere in the call-stack (sys_ioctl=>kern_ioctl=>soo_ioctl==>ifi_ioctl=>in6_control=>prelist_remove). I looked in HEAD, and this part of the code appears to be more or less the same, in particular the question of locking. >> >> Should I submit a PR (no, we can't retry with a generic kernel)? > No need to for either. > > markj@ has a patch to fix a good deal of racy prefix list locking which needs review and testing. > > — > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 > From owner-freebsd-net@FreeBSD.ORG Mon May 26 11:06:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3EFAE47 for ; Mon, 26 May 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFD7C24E5 for ; Mon, 26 May 2014 11:06:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4QB6nNZ032095 for ; Mon, 26 May 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4QB6n3x032093 for freebsd-net@FreeBSD.org; Mon, 26 May 2014 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 26 May 2014 11:06:49 GMT Message-Id: <201405261106.s4QB6n3x032093@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/190102 net [tcp] net.inet.tcp.drop_synfin=1 no longer works on Fr o kern/190038 net [ipf] ipf -Fa -6 clears v6 and v6 lists o kern/189531 net [fxp] Wake on Lan (WOL) enabled for Intel 82562EZ ethe o kern/189234 net [em] Big lag with Ethernet Connection I217-V o kern/189219 net [dummynet] [patch] using dummynet on sparc64 and confi o kern/188899 net [cas] cas ethernet driver seems to have issues with so o kern/188897 net [dc] dc ethernet driver seems to prevent the detection o kern/188421 net [libnetgraph] [patch] ng_callout() timeouts trigger pa o kern/188245 net [vlan] Critical vlan problem with OpenBGP o kern/188032 net [lo] IPv6 on lo never leaves 'tentative' state if conf o kern/187980 net [igmp] IGMP query not responded to between 2 FreeBSD h o bin/187835 net ngctl(8) strange behavior when adding more than 530 vl o kern/187718 net [igb] UDP bad performance and out-of-order packet with o kern/187451 net [vlan] [carp] Some vlans in bridge + carp result in hu o kern/187341 net [netinet] [patch] CARP addresses in backup state shoul o kern/187258 net [bxe] BCM57810 bxe(4) unstable/fails to initialize o kern/187194 net [hang] Server hangs if -arp is present on an IP addres o kern/187149 net [patch] [netmap] fix netmap's pkt-gen behavior with IP o kern/187068 net [em] network data slow/stops with em driver o kern/186949 net [re] re0 driver crashes under load every 10-20 minutes o kern/186872 net [msk] Upgrade from 9.2 to 10, msk0 watchdog timeout [r o kern/186449 net ifconfig(8) fails to autoload if_tap.ko when creating o kern/186304 net [bmc] watchdog service causes BMC controller reset eve o kern/185996 net [ip6] For IPv6, ipsec_address returns pointer to corru o kern/185967 net [lagg] [patch] Link Aggregation LAGG: LACP not working o kern/185496 net [re] RTL8169 doesn't receive unicast ethernet packets o kern/185427 net [igb] [panic] freebsd 8.4, 9.1 and 9.2 panic Double-Fa o kern/185387 net [axe] if_axe usb ethernet interface no ssh, no http wi o kern/185023 net [tun] Closing tun interface deconfigures IP address o kern/185022 net [tun] ls /dev/tun creates tun interface o kern/184389 net [libalias] libalias fails to adjust MTU from jails o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o kern/183970 net [ofed] [vlan] [panic] mellanox drivers and vlan usage o kern/183920 net [ixgbe] [patch] Incorrect ifconfig media on INTEL X520 o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host a kern/183666 net Compiled-in bxe(4) breaks kgzip(1) kernel p kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183381 net [em] [patch] Use of 9k buffers in if_em.c hangs with r o kern/183271 net [em] statistic not updated on em in netmap mode o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - p kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) f kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179473 net [new driver] if_vether.c: Source code contribution of o kern/179429 net [tap] STP enabled tap bridge a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [net] [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171550 net [ndis] [patch] "no match for InterlockedCompareExchang o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/168913 net tcpdump(8) causing interface to drop o kern/168440 net [ixgbe] [patch] ixgbe flow control tunable regression o kern/168414 net [ixgbe] [patch] ixgbe commit r234137 introduces code/c p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] [panic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129036 net [ipfw] 'ipfw fwd' does not change outgoing interface n o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message s kern/60293 net [netinet] [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/25986 net Socket would hang at LAST_ACK forever. f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 485 problems total. From owner-freebsd-net@FreeBSD.ORG Mon May 26 13:37:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEBEECB4 for ; Mon, 26 May 2014 13:37:01 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B11323EE for ; Mon, 26 May 2014 13:37:01 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Wov53-0007TY-37 for freebsd-net@freebsd.org; Mon, 26 May 2014 15:36:45 +0200 Received: from h87.s239.verisign.com ([216.168.239.87]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 26 May 2014 15:36:45 +0200 Received: from jcharbon by h87.s239.verisign.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 26 May 2014 15:36:45 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Mon, 26 May 2014 15:36:40 +0200 Lines: 100 Message-ID: <53834368.6070103@verisign.com> References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <537FBFA4.1010902@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: h87.s239.verisign.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: <537FBFA4.1010902@FreeBSD.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 13:37:01 -0000 Hi Navdeep On 23/05/14 23:37, Navdeep Parhar wrote: > On 05/23/14 13:52, Julien Charbon wrote: >> On 23/05/14 14:06, Julien Charbon wrote: >>> On 27/02/14 11:32, Julien Charbon wrote: >>>> On 07/11/13 14:55, Julien Charbon wrote: >>>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>>> wrote: >>>>>> I have put technical and how-to-repeat details in below >>>>>> PR: >>>>>> >>>>>> kern/183659: TCP stack lock contention with short-lived >>>>>> connections >>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>>> >>>>>> We are currently working on this performance improvement >>>>>> effort; it will impact only the TCP locking strategy not >>>>>> the TCP stack logic itself. We will share on freebsd-net >>>>>> the patches we made for reviewing and improvement >>>>>> propositions; anyway this change might also require enough >>>>>> eyeballs to avoid tricky race conditions introduction in >>>>>> TCP stack. >> >> Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and >> tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan >> 2014. >> >> [...] >> >> _However_ it would be a miracle that this change does not introduce >> new race condition(s) (hence the 'alpha' tag in commit message). >> Most of TCP stack locking strategy being now on inpcb lock >> shoulders. That said, from our tests point of view, this change is >> completely stable: No kernel/lock assertion, no unexpected TCP >> behavior, stable performance results. Moreover, before tagging >> this change as 'beta' we need to test more thoroughly these >> features: >> >> - VNET, - PCBGROUP/RSS/TCP timer per cpu, - TCP Offloading (we need >> a NIC with a good TCP offloading support) > > I can assess the impact (and fix any fallout) on the parts of the > kernel that deal with TCP_OFFLOAD, and the TOE driver in > dev/cxgbe/tom. But I was hoping to do that only after there was > general agreement on net@ that these locking changes are sound and > should be taken into HEAD. Lack of reviews seems to be holding this > back, correct? Correct, these changes have been reviewed and tested only internally yet. As we aren't finding any issues, we share them for wider testing, comments and reviews. An advice for reviewers: When reading code don't be fooled by remaining ipi_lock read lock (INP_INFO_RLOCK), you should consider ipi_lock as completely deleted in TCP stack. All TCP code that was under ipi_lock umbrella is now only protected by inp_lock. Just keep that in mind. Below, just for your information, more details on context of these changes: o The rough consensus at BSDCan was that there is a shared interest for scalability improvement of TCP workloads with potential high rate of connections establishment and tear-down. o Our requirements for this task were: - Achieve more than 100k TCP connections per second without dropping a single packet in reception - Use a strategy that does not require to change all network stacks in a row (TCP, UDP, RAW, etc.) - Be able to progressively introduce better performance, leveraging already in place mutex strategy - Keep the TCP stack stable (obviously) o Solutions we did try to implement and that failed: #1 Decompose ipi_lock in finer grained locks on ipi_hashbase's bucket (i.e. add a mutex in struct inpcbhead). Did not work as the induced change was quite big, and keeping network stacks shared code (in in_pcb.{h, c}) clean was much more difficult than expected. #2 Revert the lock ordering from: ipi_lock > inpcb lock > ipi_hash_lock, pcbgroup locks to: inpcb lock > ipi_lock, ipi_hash_lock, pcbgroup locks The change was just a all-or-nothing giant patch, it did not handle the full inp list traversal functions and having a different lock ordering between TCP stack and other network stacks was quite a challenge to maintain. My 2 cents. -- Julien From owner-freebsd-net@FreeBSD.ORG Mon May 26 17:02:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33F41F76 for ; Mon, 26 May 2014 17:02:12 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id E9E5727C4 for ; Mon, 26 May 2014 17:02:11 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id C5E5C7300A; Mon, 26 May 2014 19:06:43 +0200 (CEST) Date: Mon, 26 May 2014 19:06:43 +0200 From: Luigi Rizzo To: Prashant Upadhyaya Subject: Re: Regarding Netmap in VM Message-ID: <20140526170643.GE40172@onelab2.iet.unipi.it> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 17:02:12 -0000 On Thu, May 22, 2014 at 09:28:45AM +0530, Prashant Upadhyaya wrote: > Hi, > > Suppose I am on x86 with Intel 82599 NIC. > Now I spawn a VM (running FreeBSD as guest OS with Netmap support) > For better I/O performance, I have two main choices -- > > 1. Pass the 82599 NIC as a PCI passthrough device into the VM > 2. Use SRIOV VF of 82599 into the VM > > Question is, will Netmap be able to utilize both the above environments > when I run the userspace application in the guest OS in the VM, or will > there be any issues. if i remember well we do not have sriov vf support yet in freebsd so you should follow the pci-passthrough approach. It should probably work, i am not 100% sure who is in charge of programming the IOMMU for the guest os when accessing a device through pci passthrough. If this part is missing, the symptoms you should see is that packets will have all bytes set to 0x00 cheers luigi From owner-freebsd-net@FreeBSD.ORG Mon May 26 17:07:44 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F35202CE; Mon, 26 May 2014 17:07:43 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62A492815; Mon, 26 May 2014 17:07:43 +0000 (UTC) Received: from static-70-107-251-21.ny325.east.verizon.net ([70.107.251.21]:50016 helo=[192.168.73.81]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1WoyNA-0001z4-4s; Mon, 26 May 2014 13:07:42 -0400 From: "George Neville-Neil" To: "Eygene Ryabinkin" Subject: Re: Allowing CARP to use arbitrary OUI prefix and allocating block from FreeBSD's OUI space assignment for that Date: Mon, 26 May 2014 13:07:35 -0400 Message-ID: <43D946E6-809D-4F1C-96DD-6C4F6199FB07@neville-neil.com> In-Reply-To: References: <20140508200404.GA50446@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=_MailMate_8591B0AD-C432-4569-8F7A-3A0730619473_="; micalg=pgp-sha1; protocol="application/pgp-signature" X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: Marcelo Araujo , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 17:07:44 -0000 This is an OpenPGP/MIME signed message (RFC 3156 and 4880). --=_MailMate_8591B0AD-C432-4569-8F7A-3A0730619473_= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 21 May 2014, at 10:58, Eygene Ryabinkin wrote: > Mon, May 12, 2014 at 12:39:49AM +0400, Eygene Ryabinkin wrote: >> Sun, May 11, 2014 at 04:30:32PM -0400, George Neville-Neil wrote: >>> On May 8, 2014, at 16:04 , Gleb Smirnoff wrote:= >>>> On Thu, May 08, 2014 at 12:10:48PM +0400, Eygene Ryabinkin wrote: >>>> E> - I'll do a patch for carp(4) that will allow it to use configur= able >>>> E> OUI from a sysctl knob (first 5 bytes of OUI); >>>> >>>> Please no sysctl knobs. This should be configurable via ifconfig(8) >>>> per vhid. >>> >>> Agree, please do this via ifconfig. >> >> http://codelabs.ru/fbsd/carp-ouibase.diff > > Updated the patch, URL remains the same: > http://codelabs.ru/fbsd/carp-ouibase.diff > > Changes: > > - full MAC is settable via ether/lladdr/link keyword, no > ouibase keyword now exists; > > - these keywords accept "carp" and "vrrp" keywords making > them to set new and old bases with the last octet set to > the VHID; > > - network.subr was updated not to mess with any keywords that > go after 'vhid' and just pass it down to ifconfig as is. > > I did two days of testing and hadn't yet found any problems. Hi, One note on CARP and VRRP moving forwards. I realize that this patch is = partly a CARP specific issue but a lot of people on a few mailing lists seem to = have assumed that VRRP is no longer under patent threat. According to the FreeBSD Foundation's lawyers there are two extant patent= s, due to expire in 2017, 3 years hence. = CISCO =E2=80=93 US6108300: Method and apparatus for transparently providi= ng a failover network device = IBM: US6148410: Fault tolerant recoverable TCP/IP connection router, Bas= key, Dillenberger, Goldszmidt, Hunt, Levy-Abegnoli, Nick,Schmidt, Novembe= r 14, 2000. = So let's not go assuming that VRRP is "free" just yet. As to updating our CARP to behave correctly, that's still on the table. Best, George --=_MailMate_8591B0AD-C432-4569-8F7A-3A0730619473_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlODdNcACgkQYdh2wUQKM9IPlACcDenpylH6IMpVHxS0wgqJKEPQ 8/sAn3DfeoJK0XiTQE5NCNI//BxGfdk8 =5zx6 -----END PGP SIGNATURE----- --=_MailMate_8591B0AD-C432-4569-8F7A-3A0730619473_=-- From owner-freebsd-net@FreeBSD.ORG Tue May 27 03:00:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A590D4F8 for ; Tue, 27 May 2014 03:00:01 +0000 (UTC) Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 637BF2467 for ; Tue, 27 May 2014 03:00:01 +0000 (UTC) Received: by mail-ve0-f180.google.com with SMTP id db12so9940550veb.39 for ; Mon, 26 May 2014 20:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cv25YWAsFvESElHMNGub3gzjAayqBZFUay2DU46ZzvA=; b=TO/Cr3UHAmJrm3SH8nieoXmYAbkLtNkx7mBLJ/GzICF4ZN+26+fZLqsxIE/HYGkacO lF9s00fZiV+V9epeEOEP704gOKZlZM+ZFL8TJdSZePcwCpNZqMxIMokOJt6hwJix7ceV RDuI/W5hNpMNiqe7G0TRVTj8HraxPJ4NGts2zEc3QYRchMGhpfiACG1BM3q2usHdoWZ/ 3OllG8KfnyxBlB1q3NJWTbeB7CEMFZepb98CQu2iZ/A4geg5sCl7NeploLCnzM2pWUps 1uNapEL2lsXwAQDrSlHbJWGwgQ9o/A6bTRTSuFFlTRqPMZTamfN/RvcnS+G/NEkZ9wZw ArZg== MIME-Version: 1.0 X-Received: by 10.220.174.137 with SMTP id t9mr5125544vcz.12.1401159600395; Mon, 26 May 2014 20:00:00 -0700 (PDT) Received: by 10.221.5.74 with HTTP; Mon, 26 May 2014 20:00:00 -0700 (PDT) In-Reply-To: <20140526170643.GE40172@onelab2.iet.unipi.it> References: <20140526170643.GE40172@onelab2.iet.unipi.it> Date: Mon, 26 May 2014 20:00:00 -0700 Message-ID: Subject: Re: Regarding Netmap in VM From: Jack Vogel To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 03:00:01 -0000 I don't think we've ever tried this at Intel either. I had meant to check the ixv code to see if it had your code in it Luigi but got distracted, if it doesn't then it obviously can't work for the case of a VM. I think with the new 40G VF driver, when that gets committed it might be desirable to have netmap support. And, although we don't have SRIOV yet... that should be changing before long :) Jack On Mon, May 26, 2014 at 10:06 AM, Luigi Rizzo wrote: > On Thu, May 22, 2014 at 09:28:45AM +0530, Prashant Upadhyaya wrote: > > Hi, > > > > Suppose I am on x86 with Intel 82599 NIC. > > Now I spawn a VM (running FreeBSD as guest OS with Netmap support) > > For better I/O performance, I have two main choices -- > > > > 1. Pass the 82599 NIC as a PCI passthrough device into the VM > > 2. Use SRIOV VF of 82599 into the VM > > > > Question is, will Netmap be able to utilize both the above environments > > when I run the userspace application in the guest OS in the VM, or will > > there be any issues. > > if i remember well we do not have sriov vf support yet in freebsd > so you should follow the pci-passthrough approach. > > It should probably work, i am not 100% sure who is in charge of > programming the IOMMU for the guest os when accessing a device > through pci passthrough. If this part is missing, the symptoms > you should see is that packets will have all bytes set to 0x00 > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue May 27 09:49:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82048A14 for ; Tue, 27 May 2014 09:49:56 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19CB62538 for ; Tue, 27 May 2014 09:49:55 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id cc10so1338957wib.10 for ; Tue, 27 May 2014 02:49:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=7l3Z7vsy2GrSZkd6eNUWOdsLOAD4vK4hqzTpiZfj9Nw=; b=TsYlObz2YzfrLhHF0unuz+nhlcVxjYeHHq90kND0Z7m/glfYQgOryH3SIUQG90HHkQ cf2Hr0zXgQTZ0J90qWZuTBINQ+D9wEQBqXQWlQBCWd22KpU81xK64OabzFPAdXFzOXHk TsQ/P2vvJxB/lQTn2h0ki5nfJYCVc1Vi1VYb2D1EBKY1y8CDpRALiUDmIZoB4GnhBkYw /FlcTIZigge2olNhRN33F8AU48ejeBYQ8auDK5BEHf1iM+EE6hctSLBmPPq8ibWJJFUr UkUqNXU6t5BaD+msGeqsS3t9BB+IejRskGLmelDS7JlvlfLFKiCL9ydHGMYUI6z86Q1b udzQ== MIME-Version: 1.0 X-Received: by 10.194.246.234 with SMTP id xz10mr37003498wjc.77.1401184193759; Tue, 27 May 2014 02:49:53 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Tue, 27 May 2014 02:49:53 -0700 (PDT) In-Reply-To: References: <20140526170643.GE40172@onelab2.iet.unipi.it> Date: Tue, 27 May 2014 11:49:53 +0200 X-Google-Sender-Auth: 1jdyFRiuEMmdxSp8-XSVUvXhC88 Message-ID: Subject: Re: Regarding Netmap in VM From: Luigi Rizzo To: Jack Vogel Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 09:49:56 -0000 On Tue, May 27, 2014 at 5:00 AM, Jack Vogel wrote: > I don't think we've ever tried this at Intel either. I had meant to check > the ixv code to > see if it had your code in it Luigi but got distracted, if it doesn't the= n > it obviously can't > work for the case of a VM. > =E2=80=8Bixv.c does not contain the netmap hooks though it might be relatively straightforward to add them, possibly reusing the same netmap hooks and support functions used in ixgbe.c As a matter of fact, there seems to be a large amount of duplicated code between ixv.c and ixgbe.c, so i wonder whether it makes sense to remove the duplication and use just a single copy of the common parts ? cheers luigi From owner-freebsd-net@FreeBSD.ORG Tue May 27 14:12:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 137A9435 for ; Tue, 27 May 2014 14:12:42 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CADE42E2C for ; Tue, 27 May 2014 14:12:41 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id j5so13997499qga.14 for ; Tue, 27 May 2014 07:12:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=HgQMyhhNWBDeEYbcMmWF2tjNmfvOCW4K3A/0C2JlJk8=; b=buwr/MHxqhR8ckdkDsR06pQMHogakpDCNc6toZBh08xUqxWFzA/EGoIJo3Lu5DSzQa UFlqvHdfWT5WPQ3UPqWIPo440H8HPCORJ7cKnxEhE2tvx2BjodCb0VHPQsGD537Q1jZO xyJP5GOIwJeGyKpYGVJyR9yrp+YGjPg6lk+f602hsiRgYeiX6FiFKFQMcQD/Qj0kjfx4 020uNtnvS7PeAesJ9go2A0jnQBQasDjkxlheTdaL0vUffQ9yqxFIAsvTUYyHfiZ/kUl3 MZ3pnt24K7VIvK+fo0oB8wqQ745ojlL/4XYKMSYBIURcN2VyPqtUpZrUwtTsgX9zVv4l cp3A== MIME-Version: 1.0 X-Received: by 10.140.109.201 with SMTP id l67mr40785423qgf.72.1401199960992; Tue, 27 May 2014 07:12:40 -0700 (PDT) Received: by 10.224.216.138 with HTTP; Tue, 27 May 2014 07:12:40 -0700 (PDT) Date: Tue, 27 May 2014 22:12:40 +0800 Message-ID: Subject: About TFTP From: Niu Zhixiong To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 14:12:42 -0000 Dear all, I want to modify the source code of TFTP. However it is invoked by inetd. I really need a standalone service of TFTP without inetd. But, when a start tftpd without inetd. it says recvfrom: Socket operation on non-socket. Could someone give me some suggestion to start tftpd without inetd? Thanks in advance. Regards, Niu Zhixiong =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF= =BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D kaiaixi@gmail.com From owner-freebsd-net@FreeBSD.ORG Tue May 27 14:48:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBCA8956 for ; Tue, 27 May 2014 14:48:13 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CFDE21BD for ; Tue, 27 May 2014 14:48:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=pe10UphZ0tONdKc5U76+xqElLh9b9B45zrUSr46fHNg=; b=X+QEB3jWj1uZO1kO/3t95qZCjFmgtwG9QF8CoEHykeD/KnsIAg4TI1HauPaKK2voV7TtyxADJd4102hZia9Ghs/6l/OMmQ1OSaJnZ4fldlusWrTS1HVUQQiZOf9uyDSfdC3ZDxTvTZmST+hNP4g8tndpdEJWekcbpN30p0iiiMFZ9M8W45gFPsSBaunt5+BQ3KHctdyYhf5M9oK/ryCpUn8HEzKYj/JCvNPuaGDN9aNBiyUsnm/EBbOkkMZfrXAHXR/XYL3IuUtf+gDODzhHZ97KFZ4b9iLQqYXLjgHrpt6Ba/q3Mnc3UdRruo6k4RHuql/y4xsw4hkmVzby73jYj/kpW/gyHD+yIY8SBP1/ofsdFbh09/Q9CMunjxqEg4CtguandRhqPWwkjaCd4IOHnun72ClWY9+jKWvvbVaR75855ase0IwtJyyCW5Dq08sXSBBaWQt3BgZ+EeyxBKI4hvMafFySyuZNH7trMwkKabcA4uckJTB1RrtQK4pgqxYyUHSBoCJVzZK+H0ItXkT8OWbudlggs8QG3cQs70sIPkyOxnsl+4ZbBVAGaFe416z3it2xoSFEY+jAuNlLFzcLqI4H9HOeXQAy6CGrQ+GDgDsP7kBU6zrlFZYoMBPnGynRq6NZjSXSSLNri3rmYAdct6xq7NUbvyqtPrAPEFUS00k=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WpIfj-000KNN-Ic; Tue, 27 May 2014 18:48:11 +0400 Date: Tue, 27 May 2014 18:48:09 +0400 From: Eygene Ryabinkin To: Niu Zhixiong Subject: Re: About TFTP Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="78WCQgNEg63vc82l" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 14:48:14 -0000 --78WCQgNEg63vc82l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Tue, May 27, 2014 at 10:12:40PM +0800, Niu Zhixiong wrote: > I want to modify the source code of TFTP. However it is invoked by > inetd. I really need a standalone service of TFTP without inetd. Why? Inetd provides network pipe and connection initiation for TFTP daemon and then it acts on the "cooked" socket for the port 69, but does the rest of communication by itself, including opening data connection back to the client. > But, when a start tftpd without inetd. it says recvfrom: Socket > operation on non-socket. Because tftp daemon expects its standard input and output to be network sockets. > Could someone give me some suggestion to start tftpd without inetd? FreeBSD stock TFTP daemon doesn't support this. You may probably look at the ports collection (/usr/ports/net) and software with tftp in its name: there are at least 3 packages that look like TFTP daemon. But first I'd reconsider the need to drop inetd =66rom the game. Or, if you're up to some coding, you can extend our base tftp daemon with stand-alone capabilities. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --78WCQgNEg63vc82l Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOEpalfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PsbGgEAkETDGQkPoV9fQi3pRVDF+gtB xS3IqudMevWEBRnflC8A/A8kh/UFU2q4QT4PBL0rqjKOckJO9aBgY3F5IXGC6UBK =HBx1 -----END PGP SIGNATURE----- --78WCQgNEg63vc82l-- From owner-freebsd-net@FreeBSD.ORG Tue May 27 16:32:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B75E6121 for ; Tue, 27 May 2014 16:32:52 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 668C22C60 for ; Tue, 27 May 2014 16:32:52 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s4RGWoNB094186 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 27 May 2014 10:32:50 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s4RGWoUU094183; Tue, 27 May 2014 10:32:50 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 27 May 2014 10:32:50 -0600 (MDT) From: Warren Block To: Niu Zhixiong Subject: Re: About TFTP In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 27 May 2014 10:32:51 -0600 (MDT) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 16:32:52 -0000 On Tue, 27 May 2014, Niu Zhixiong wrote: > Dear all, > I want to modify the source code of TFTP. However it is invoked by > inetd. I really need a standalone service of TFTP without inetd. But, when > a start tftpd without inetd. it says recvfrom: Socket operation on > non-socket. > > Could someone give me some suggestion to start tftpd without inetd? Thanks > in advance. ftp/tftp-hpa can do this, starting from /etc/rc.conf with tftpd_enable="YES" tftpd_flags="..." From owner-freebsd-net@FreeBSD.ORG Tue May 27 17:19:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 208C3AF7 for ; Tue, 27 May 2014 17:19:06 +0000 (UTC) Received: from mail-ve0-x22c.google.com (mail-ve0-x22c.google.com [IPv6:2607:f8b0:400c:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D0DE8209C for ; Tue, 27 May 2014 17:19:05 +0000 (UTC) Received: by mail-ve0-f172.google.com with SMTP id oz11so11020862veb.31 for ; Tue, 27 May 2014 10:19:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1K+5ByemqK7OR2J8idrmSKkSy1zjL1ruyzNJdZNl7Dw=; b=UYs3wNZjZ3dn++8Lo7uJS03aINrmZMMb+J8PZMdbw+wVflj5ZCBxiuvVUtBkY3LSB4 szNrQwqqI55RGNt3viEvQ6pt3Fdi5j1QvSrKajJAgyL5NZS4/WOPPXuMHnKXtl9M7oPD SRzSv3YWh8iiP5V0iJl/NSq3Nlr//frbxuKJf4N50fMuBm1xPNk2yFf6DdJetgO6He6J qtdG2KbJoLZ8s1O6iRo/85SN+Y4Ky3XmJnVEPGh1QA8jHMldA/HYLE3fUyF+81Lh2GWA 9v8qKgo/xwpKqDHaLVMbAatW5NjbwjzzEEzFHrfRsF7ktoKyACVND39y/3u51HwLJ2NU eQ0w== MIME-Version: 1.0 X-Received: by 10.58.74.164 with SMTP id u4mr2202394vev.81.1401211144962; Tue, 27 May 2014 10:19:04 -0700 (PDT) Received: by 10.221.5.74 with HTTP; Tue, 27 May 2014 10:19:04 -0700 (PDT) In-Reply-To: References: <20140526170643.GE40172@onelab2.iet.unipi.it> Date: Tue, 27 May 2014 10:19:04 -0700 Message-ID: Subject: Re: Regarding Netmap in VM From: Jack Vogel To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 17:19:06 -0000 Ya, it might be nice to do a bunch of cleanup like that, maybe once the i40e release happens I'll have some time to look into that. Jack On Tue, May 27, 2014 at 2:49 AM, Luigi Rizzo wrote: > > > > On Tue, May 27, 2014 at 5:00 AM, Jack Vogel wrote: > >> I don't think we've ever tried this at Intel either. I had meant to check >> the ixv code to >> see if it had your code in it Luigi but got distracted, if it doesn't >> then it obviously can't >> work for the case of a VM. >> > > ixv.c does not contain the netmap hooks though it might be > relatively straightforward to add them, possibly reusing > the same netmap hooks and support functions used in ixgbe.c > > As a matter of fact, there seems to be a large amount > of duplicated code between ixv.c and ixgbe.c, > so i wonder whether it makes sense to remove the duplication and > use just a single copy of the common parts ? > > cheers > luigi > From owner-freebsd-net@FreeBSD.ORG Tue May 27 23:49:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CFB8A4E for ; Tue, 27 May 2014 23:49:51 +0000 (UTC) Received: from mail-pb0-x234.google.com (mail-pb0-x234.google.com [IPv6:2607:f8b0:400e:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62CC7231A for ; Tue, 27 May 2014 23:49:51 +0000 (UTC) Received: by mail-pb0-f52.google.com with SMTP id rr13so10156862pbb.11 for ; Tue, 27 May 2014 16:49:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=q89WN56I5/uZf/fweXFkGCcnOcdPD0L94O4wjROjDCs=; b=nq13mzVV51nAEhlcYjN6mawEwa0nA0shUu+CFgH2ad3YjDU5mjaXaU5791Nm2EsAse wmZjd94ZDq/+1EBZxsIe0W4+5WNZK2eKQt8g+LhQqGujYTqy6jBoYh570aosgA3atHvh JtXrkjdDyu28vCbY+bPE4q08jJymSll9upu4ZoQPL7/MDGbJrdVdbVUpj2mexqH9QS8D QLw6YD5cn+Gv+Vqdrig0rS6MBtE+xR0uqQ7GJBM2QsybpvNKh3B3iHNEdts0MQuMdiSu lKFkvhjXIbzP2RamfLO+pSyAYvs0YTATSdlpKrzgctt9LrbN9RxQufGl1/CnITnrMDKR F6cw== X-Received: by 10.68.197.134 with SMTP id iu6mr17709091pbc.164.1401234591012; Tue, 27 May 2014 16:49:51 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id uk1sm79781328pac.26.2014.05.27.16.49.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 27 May 2014 16:49:50 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <5385249D.9050501@FreeBSD.org> Date: Tue, 27 May 2014 16:49:49 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Luigi Rizzo , "freebsd-net@freebsd.org" Subject: change netmap global lock to sx? Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 23:49:51 -0000 I'd like to change the netmap global lock from a mutex into a sleepable shared/exclusive lock. This will allow a driver's nm_register hook (which is called with the global lock held) to sleep if it has to. I've casually used pkt-gen after this conversion (patch attached) and the witness hasn't complained about it. Thoughts? Regards, Navdeep diff -r 0300d80260f4 sys/dev/netmap/netmap_kern.h --- a/sys/dev/netmap/netmap_kern.h Fri May 23 19:00:56 2014 -0700 +++ b/sys/dev/netmap/netmap_kern.h Sat May 24 12:49:15 2014 -0700 @@ -43,13 +43,13 @@ #define unlikely(x) __builtin_expect((long)!!(x), 0L) #define NM_LOCK_T struct mtx -#define NMG_LOCK_T struct mtx -#define NMG_LOCK_INIT() mtx_init(&netmap_global_lock, \ - "netmap global lock", NULL, MTX_DEF) -#define NMG_LOCK_DESTROY() mtx_destroy(&netmap_global_lock) -#define NMG_LOCK() mtx_lock(&netmap_global_lock) -#define NMG_UNLOCK() mtx_unlock(&netmap_global_lock) -#define NMG_LOCK_ASSERT() mtx_assert(&netmap_global_lock, MA_OWNED) +#define NMG_LOCK_T struct sx +#define NMG_LOCK_INIT() sx_init(&netmap_global_lock, \ + "netmap global lock") +#define NMG_LOCK_DESTROY() sx_destroy(&netmap_global_lock) +#define NMG_LOCK() sx_xlock(&netmap_global_lock) +#define NMG_UNLOCK() sx_xunlock(&netmap_global_lock) +#define NMG_LOCK_ASSERT() sx_assert(&netmap_global_lock, SA_XLOCKED) #define NM_SELINFO_T struct selinfo #define MBUF_LEN(m) ((m)->m_pkthdr.len) From owner-freebsd-net@FreeBSD.ORG Wed May 28 00:32:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B089C4BA; Wed, 28 May 2014 00:32:35 +0000 (UTC) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1809C26D3; Wed, 28 May 2014 00:32:34 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id t60so10221471wes.41 for ; Tue, 27 May 2014 17:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OAL/nD0p6JU8LHqKqy/dwjG8e1xTkupcJs//dsW9ONc=; b=abrAh2UyiipD4ZTbqgj7czytY9qJ/hIWV/g7zGeZOszCXeZhAy2ZKVZzqPzDV6bs3G LWSzcCZbNALSL5qaLNRXyfw0izEA60FijoQ4CMEGrOznYCwVOU0U9k0cKs/YhY0ngOvB +Rtg2pfOxat5Ojl+8keAWGKhbCl3WEF6XFmqVt8+L1XuyUfYMRXdMk8zj/H3/YI95m1T KcEADHqwaNYfl631ndGlb9qhnYtXO5+Bt7Vpxg0kEw+tbLENLGw03g+H3WJpgqQSc9/y i0bMsreDWGRnoaFUbvDI3d9cl7ppNuPke9enqEGoJocSXFdt2BshQiXjdTl3j21oWB59 vG5A== MIME-Version: 1.0 X-Received: by 10.194.92.7 with SMTP id ci7mr46481720wjb.7.1401237152769; Tue, 27 May 2014 17:32:32 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Tue, 27 May 2014 17:32:32 -0700 (PDT) In-Reply-To: <5385249D.9050501@FreeBSD.org> References: <5385249D.9050501@FreeBSD.org> Date: Wed, 28 May 2014 02:32:32 +0200 X-Google-Sender-Auth: JxG0DtEgfwF76IDZi8UzjKrMzNo Message-ID: Subject: Re: change netmap global lock to sx? From: Luigi Rizzo To: Navdeep Parhar Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 00:32:35 -0000 On Wed, May 28, 2014 at 1:49 AM, Navdeep Parhar wrote: > I'd like to change the netmap global lock from a mutex into a sleepable > shared/exclusive lock. This will allow a driver's nm_register hook > (which is called with the global lock held) to sleep if it has to. I've > casually used pkt-gen after this conversion (patch attached) and the > witness hasn't complained about it. > > =E2=80=8Bno objections, let me give this a try on stable/10 stable/9 to make sure we can use the same code there as well cheers luigi =E2=80=8B > Thoughts? > > Regards, > Navdeep > > > diff -r 0300d80260f4 sys/dev/netmap/netmap_kern.h > --- a/sys/dev/netmap/netmap_kern.h Fri May 23 19:00:56 2014 -0700 > +++ b/sys/dev/netmap/netmap_kern.h Sat May 24 12:49:15 2014 -0700 > @@ -43,13 +43,13 @@ > #define unlikely(x) __builtin_expect((long)!!(x), 0L) > > #define NM_LOCK_T struct mtx > -#define NMG_LOCK_T struct mtx > -#define NMG_LOCK_INIT() mtx_init(&netmap_global_lock, \ > - "netmap global lock", NULL, MTX_DEF) > -#define NMG_LOCK_DESTROY() mtx_destroy(&netmap_global_lock) > -#define NMG_LOCK() mtx_lock(&netmap_global_lock) > -#define NMG_UNLOCK() mtx_unlock(&netmap_global_lock) > -#define NMG_LOCK_ASSERT() mtx_assert(&netmap_global_lock, MA_OWNED) > +#define NMG_LOCK_T struct sx > +#define NMG_LOCK_INIT() sx_init(&netmap_global_lock, \ > + "netmap global lock") > +#define NMG_LOCK_DESTROY() sx_destroy(&netmap_global_lock) > +#define NMG_LOCK() sx_xlock(&netmap_global_lock) > +#define NMG_UNLOCK() sx_xunlock(&netmap_global_lock) > +#define NMG_LOCK_ASSERT() sx_assert(&netmap_global_lock, SA_XLOCKED= ) > > #define NM_SELINFO_T struct selinfo > #define MBUF_LEN(m) ((m)->m_pkthdr.len) > From owner-freebsd-net@FreeBSD.ORG Wed May 28 16:44:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03C97DEF for ; Wed, 28 May 2014 16:44:18 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B23902AF5 for ; Wed, 28 May 2014 16:44:17 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Wpgwt-0000rT-6T for freebsd-net@freebsd.org; Wed, 28 May 2014 18:43:31 +0200 Received: from h87.s239.verisign.com ([216.168.239.87]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 May 2014 18:43:31 +0200 Received: from jcharbon by h87.s239.verisign.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 May 2014 18:43:31 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Wed, 28 May 2014 18:42:49 +0200 Lines: 59 Message-ID: <53861209.2000306@verisign.com> References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: h87.s239.verisign.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: <537FB51D.2060401@verisign.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 16:44:18 -0000 Hi, On 23/05/14 22:52, Julien Charbon wrote: > On 23/05/14 14:06, Julien Charbon wrote: >> On 27/02/14 11:32, Julien Charbon wrote: >>> On 07/11/13 14:55, Julien Charbon wrote: >>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>> wrote: >>>>> I have put technical and how-to-repeat details in below PR: >>>>> >>>>> kern/183659: TCP stack lock contention with short-lived connections >>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>> >>>>> We are currently working on this performance improvement effort; it >>>>> will impact only the TCP locking strategy not the TCP stack logic >>>>> itself. We will share on freebsd-net the patches we made for >>>>> reviewing and improvement propositions; anyway this change might also >>>>> require enough eyeballs to avoid tricky race conditions introduction >>>>> in TCP stack. > > Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and > tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan 2014. At BSDCan 2014 we were also asked to provide flame graph [1][2] to highlight impacts of these TCP changes. The Dtrace sampling was done on a NIC receive queue IRQ bound core. o First CPU flame graph on 10.0-RELENG at 40k TCP connection/secs: https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/releng10-40k.html Note: - __rw_wlock_hard on ipi_lock contention is clear as usual. o Second, same test with all our patches applied (thus from 10.0-next branch [3]): https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-40k.html Note: - Almost all __rw_wlock_hard on ipi_lock contention is converted in idle time. o Third, still using 10.0-next branch, the flame graph when doubling the rate to 80k TCP connection/sec: https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-80k.html My 2 cents. [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html [2] https://wiki.freebsd.org/201405DevSummit/NetworkStack [3] https://github.com/verisign/freebsd/commits/share/10.0-next -- Julien From owner-freebsd-net@FreeBSD.ORG Thu May 29 05:46:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 972BA474; Thu, 29 May 2014 05:46:50 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3858F2EF3; Thu, 29 May 2014 05:46:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date; bh=47sqiRQR/hCVPeBhvSfDE+VrEzpHwjjnqhKcSJ0KjTE=; b=qYpZD72mQ3vXNIYVwLtQue43aGmsnutXD+h5/ZkScwxChE93u2HJ5SwYFpWTioEMaXmEIWJIrSBeQIdRC8Gb1bBIwd/EfFVzcqHEc219ANl6mTPK5ZmSJnQ1JiadS78e3JgmwUH1klf2wzWBeCWNQoErkOQdbdfGLB5rXabd011L9eI/G2hEDNtEyx3Mfw1vB5LxdFjBJxFn42dCOhBiHJIL10LyBLOVxaxO3tHO4UBPRhoMRToJc0EbhJQfMX9i58tSb2nIUONxLDLMnQEL+evls8RpzO5HW4O7jroXY+rJKjDnPC0yBoeJCRT4P8FeRo3wrprMM8MtFCvKiE8/Hr4G4QkSZzJE5eDenkCI8MPvotKrijG6ZZavK1nOolitW4x6dqOSa5grvQWBfH3O9MVpbpNZuh2jW4BXYiD/GJ/8R3FuZcEo783gDOXGkho5JiiPFitI0UNdNJk573bOKGJ3lEYR0w60oMJWkLezfej4Cre0HjZnXgBE971wJH6yDA1CUfymxF5Ap651WKxpWZbPnckUy/uFShibmYWejYXuBW2Z9jTuCxJjbYvbn+SxPxi7m3vKKtcoZibEvrwQ57pDigwiidWeXmKlEigfhtTW99xOY/qpwvEyOc9mfcrvCUzkN/wcM+tcHcUfBf6vvJ/m8mVNisUCdEecZajXGHY=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WptAt-000GjP-Sm; Thu, 29 May 2014 09:46:48 +0400 Date: Thu, 29 May 2014 09:46:45 +0400 From: Eygene Ryabinkin To: FreeBSD GNATS followup , freebsd-net@freebsd.org Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Message-ID: <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> References: <201405222101.s4ML122N061489@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="LXx4g46d83wF7unj" Content-Disposition: inline In-Reply-To: <201405222101.s4ML122N061489@freefall.freebsd.org> Sender: rea@codelabs.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 05:46:50 -0000 --LXx4g46d83wF7unj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I assume that your pf(4) is enabled during these tests, you have "scrub" statements in the ruleset and removing "scrub" will restore the expected behaviour on 10.x? I am slightly amused that on 9.x with "scrub" you're getting the expected behaviour, because clearing FIN bit for SYN packets was the standard behaviour of pf since approximately at least 10 years, http://svnweb.freebsd.org/base/vendor-sys/pf/dist/sys/contrib/pf/net/pf_n= orm.c?view=3Dmarkup&pathrev=3D126258#l1242 Can you show relevant parts of the pf.conf from both machines and output from 'pfctl -s rules' if you are sure that both machines are configured identically pf-wise? Thanks! --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --LXx4g46d83wF7unj Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOGycVfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Pv7kQD+JjKVNIOqBBGv12DsILxmIr+U 5A76OhcjmiaO5ricQ2oA/jJy8E/D2nXSdaaAqYsNJaelqQ72Lx927Sxyj50hLDpx =2WMS -----END PGP SIGNATURE----- --LXx4g46d83wF7unj-- From owner-freebsd-net@FreeBSD.ORG Thu May 29 05:50:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37AC0608 for ; Thu, 29 May 2014 05:50:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 230A82F15 for ; Thu, 29 May 2014 05:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4T5o0lG099400 for ; Thu, 29 May 2014 05:50:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4T5o0vn099387; Thu, 29 May 2014 05:50:00 GMT (envelope-from gnats) Date: Thu, 29 May 2014 05:50:00 GMT Message-Id: <201405290550.s4T5o0vn099387@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Eygene Ryabinkin Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Reply-To: Eygene Ryabinkin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 05:50:01 -0000 The following reply was made to PR kern/190102; it has been noted by GNATS. From: Eygene Ryabinkin To: FreeBSD GNATS followup , freebsd-net@freebsd.org Cc: Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Date: Thu, 29 May 2014 09:46:45 +0400 --LXx4g46d83wF7unj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I assume that your pf(4) is enabled during these tests, you have "scrub" statements in the ruleset and removing "scrub" will restore the expected behaviour on 10.x? I am slightly amused that on 9.x with "scrub" you're getting the expected behaviour, because clearing FIN bit for SYN packets was the standard behaviour of pf since approximately at least 10 years, http://svnweb.freebsd.org/base/vendor-sys/pf/dist/sys/contrib/pf/net/pf_n= orm.c?view=3Dmarkup&pathrev=3D126258#l1242 Can you show relevant parts of the pf.conf from both machines and output from 'pfctl -s rules' if you are sure that both machines are configured identically pf-wise? Thanks! --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --LXx4g46d83wF7unj Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOGycVfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Pv7kQD+JjKVNIOqBBGv12DsILxmIr+U 5A76OhcjmiaO5ricQ2oA/jJy8E/D2nXSdaaAqYsNJaelqQ72Lx927Sxyj50hLDpx =2WMS -----END PGP SIGNATURE----- --LXx4g46d83wF7unj-- From owner-freebsd-net@FreeBSD.ORG Thu May 29 06:34:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85180C50 for ; Thu, 29 May 2014 06:34:49 +0000 (UTC) Received: from mail-qg0-x234.google.com (mail-qg0-x234.google.com [IPv6:2607:f8b0:400d:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 398E422A5 for ; Thu, 29 May 2014 06:34:49 +0000 (UTC) Received: by mail-qg0-f52.google.com with SMTP id a108so20538884qge.25 for ; Wed, 28 May 2014 23:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=J1TsuwMHUXVi0Mwi7/DBBu36gQEJEwfEhsPI4vsh7go=; b=fclImDoRFZHT7HJNFNYJK1lxC29nUZTyQNkXPnbW+FhIAVeN/Il3Po6lQ0xXYpEBYh SkqGt4L/OccHPmgRMdOiTu67SVrJXFWVJ3U2yCO20Bz0/jPZ8jCbTGQq0lEVCWabzobP dyvwzlnuGRTq4jRmaT7FCR2VQlUjB6V7gFmKQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to:cc :content-type; bh=J1TsuwMHUXVi0Mwi7/DBBu36gQEJEwfEhsPI4vsh7go=; b=fXe8J+KoOuQK0w9oquHZW9+iZRt7jAs6t3pORgklgIF97yYAOBbva1XVHN6bEMGyX3 /W63RCV2QwQvkpnDl4idc+365b2PnBO304HIEDxgIZKJvxlR25tajj29eJTJwqRnrDsh ZanQMaaghqACuHY20FXCQEAhU2aOQv8Gua63uGWiiMOxoIjTW2KryKx9msTMi2T3dl9B tuABY6rMLOo/x+AYHES14csBM1IulHlCPvwLRBXRaTUWr4/YuL/R+/C5BoRoGOt2Uql4 rZwjz6I0CG5hl72bXuI5IKAQ1RA4kTFJr5W5E4yQuI91CA61NMzA//Vb6cPECBy5Gsmm grTQ== X-Gm-Message-State: ALoCoQkJKzhcZtfLv9PpDRsekt1H3RTcQX4IaKG6+bw5MFEGMAl1eWfmg44wO4xeYdPS8Ij9iGC/ MIME-Version: 1.0 X-Received: by 10.140.23.7 with SMTP id 7mr6726303qgo.0.1401345288130; Wed, 28 May 2014 23:34:48 -0700 (PDT) Received: by 10.140.92.198 with HTTP; Wed, 28 May 2014 23:34:47 -0700 (PDT) X-Originating-IP: [75.128.101.59] Date: Thu, 29 May 2014 02:34:47 -0400 Message-ID: Subject: [VIMAGE][udplite] FreeBSD 10-STABLE/powerpc From: Jason Hellenthal To: "[FreeBSD Stable]" , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: kevlo@freebsd.org, adrian@freebsd.org, jhb@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 06:34:49 -0000 Is anyone aware that VIMAGE on powerpc is currently broken ? In file included from /export/usr/src/sys/netinet/in_proto.c:83: /export/usr/src/sys/netinet/udp_var.h: In function 'get_inpcbinfo': /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer to incomplete type /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer to incomplete type /export/usr/src/sys/netinet/udp_var.h: In function 'get_pcblist': /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer to incomplete type /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer to incomplete type *** Error code 1 The relevant code in that header is: get_inpcbinfo(uint8_t protocol) { return (protocol == IPPROTO_UDP) ? &V_udbinfo : &V_ulitecbinfo; } get_pcblist(uint8_t protocol) { return (protocol == IPPROTO_UDP) ? &V_udb : &V_ulitecb; } Working Copy Root Path: /usr/src URL: svn://svn.freebsd.org/base/stable/10 Relative URL: ^/stable/10 Repository Root: svn://svn.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 266818 Node Kind: directory Schedule: normal Last Changed Author: delphij Last Changed Rev: 266816 Last Changed Date: 2014-05-28 14:51:49 -0400 (Wed, 28 May 2014) Also looking at svn it appears to have come from this commit... I have backed out the change here and it appears to be following through so more attention to VIMAGE and udplite seems to be needed. ------------------------------------------------------------------------ r265946 | kevlo | 2014-05-13 02:05:53 -0400 (Tue, 13 May 2014) | 14 lines Changed paths: M /stable/10 M /stable/10/lib/libc/net/getaddrinfo.c M /stable/10/sys/netinet/in.c M /stable/10/sys/netinet/in.h M /stable/10/sys/netinet/in_pcb.c M /stable/10/sys/netinet/in_proto.c M /stable/10/sys/netinet/udp_usrreq.c M /stable/10/sys/netinet/udp_var.h A /stable/10/sys/netinet/udplite.h (from /head/sys/netinet/udplite.h:264212) M /stable/10/sys/netinet6/in6_ifattach.c M /stable/10/sys/netinet6/in6_proto.c M /stable/10/sys/netinet6/udp6_usrreq.c M /stable/10/sys/netinet6/udp6_var.h M /stable/10/sys/sys/param.h MFC r264212,r264213,r264248,r265776,r265811,r265909: - Add support for UDP-Lite protocol (RFC 3828) to IPv4 and IPv6 stacks. Tested with vlc and a test suite [1]. [1] http://www.erg.abdn.ac.uk/~gerrit/udp-lite/files/udplite_linux.tar.gz Reviewed by: jhb, glebius, adrian - Fix a logic bug which prevented the sending of UDP packet with 0 checksum. - Disable TX checksum offload for UDP-Lite completely. It wasn't used for partial checksum coverage, but even for full checksum coverage it doesn't work. ------------------------------------------------------------------------ From owner-freebsd-net@FreeBSD.ORG Thu May 29 06:52:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CED65FF5; Thu, 29 May 2014 06:52:52 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A747243B; Thu, 29 May 2014 06:52:52 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id j107so20898372qga.20 for ; Wed, 28 May 2014 23:52:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zWuclZRX08CqtGryGc0afjgkRF2gjyKhnr0cgNDx/E0=; b=SX/CcxG1X9kso276YBCKA55L8EFADe9TczQphLsRXwgQ2nuX98qic/Y0QakVN4JVOf 0RgW3OwOtuPLkXomyS8dZtwSonQLHGbar6mj2OZu4ApBq1vq0ck0qOQ7OZXTckOZakTk 4/wG3VKhpSyXTsgx0b+2Vh2TUyMUgAcTvzX8dTw91jNcWlGLChPQBaMbEDpx4R4OmC7A Ui4OUAPD7kpJwoWHjcgKXZHZE2rmYtELdkZHvqbg/PjfiA+HP5tqq56+u46fH9fhjYlM dNsb/JjYU0UY2GY6wJDJPiVByDCc0SVKvmBvY+TK1HhtroohXueAGfsIiuYq6Q0JLUEf i5+A== MIME-Version: 1.0 X-Received: by 10.140.25.166 with SMTP id 35mr6363697qgt.103.1401346371441; Wed, 28 May 2014 23:52:51 -0700 (PDT) Received: by 10.96.122.133 with HTTP; Wed, 28 May 2014 23:52:51 -0700 (PDT) In-Reply-To: <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> References: <201405222101.s4ML122N061489@freefall.freebsd.org> <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> Date: Wed, 28 May 2014 23:52:51 -0700 Message-ID: Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] From: hiren panchasara To: Eygene Ryabinkin Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , FreeBSD GNATS followup X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 06:52:52 -0000 On Wed, May 28, 2014 at 10:46 PM, Eygene Ryabinkin wrote: > I assume that your pf(4) is enabled during these tests, you have > "scrub" statements in the ruleset and removing "scrub" will restore > the expected behaviour on 10.x? I can confirm that I see exactly what you are saying on a stable/10 box. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Thu May 29 07:00:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40ABB1A5 for ; Thu, 29 May 2014 07:00:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E2FB2485 for ; Thu, 29 May 2014 07:00:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4T7003V027289 for ; Thu, 29 May 2014 07:00:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4T700HV027288; Thu, 29 May 2014 07:00:00 GMT (envelope-from gnats) Date: Thu, 29 May 2014 07:00:00 GMT Message-Id: <201405290700.s4T700HV027288@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: hiren panchasara Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Reply-To: hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 07:00:01 -0000 The following reply was made to PR kern/190102; it has been noted by GNATS. From: hiren panchasara To: Eygene Ryabinkin Cc: FreeBSD GNATS followup , "freebsd-net@freebsd.org" Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Date: Wed, 28 May 2014 23:52:51 -0700 On Wed, May 28, 2014 at 10:46 PM, Eygene Ryabinkin wrote: > I assume that your pf(4) is enabled during these tests, you have > "scrub" statements in the ruleset and removing "scrub" will restore > the expected behaviour on 10.x? I can confirm that I see exactly what you are saying on a stable/10 box. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Thu May 29 07:25:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B31CF6B7 for ; Thu, 29 May 2014 07:25:22 +0000 (UTC) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D8602702 for ; Thu, 29 May 2014 07:25:22 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id kq14so12435607pab.38 for ; Thu, 29 May 2014 00:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=BuujjfCPtSdNEQRM9/qw4tzDXlAve6/UAPB+thB9qMU=; b=zCwNsMyX5/JLEmwec2sByUXjHcUW1EM2ShWQLYhL/7yQuN/HKCef8d1TtUYb4nWmjt fXsy5MQ4wv5EHKecohe0Rk8nX8NeWzoTDcCFvw1Svnsp5ojKfCp1UpUix5T+ZVJkHGGR CYthkKytMFqqs9RuhdJhEVEpnbKh8jxla1pwCk3xg9/gSIQSl4hUoYvmXK3cZnArdykI xtJZlf8MDlB416eJCusvSO1PZ0NMKnSPhjMB5Kz3wKMaP+u4UreEVpbFMg8RquKfiO3y VfCFyqtnLsrdWDHiEGOW0sB7nOPTuFbiH6zttgn8HHNHtU8iGjxfAk0lf8CKBaTtqN2r GNMQ== MIME-Version: 1.0 X-Received: by 10.66.192.104 with SMTP id hf8mr6367673pac.13.1401348322180; Thu, 29 May 2014 00:25:22 -0700 (PDT) Received: by 10.70.126.138 with HTTP; Thu, 29 May 2014 00:25:22 -0700 (PDT) Date: Thu, 29 May 2014 15:25:22 +0800 Message-ID: Subject: propose a new generic purpose rule option for ipfw From: Bill Yuan To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 07:25:22 -0000 hi the rule of ipfw is kind of semantic, and it is powerful. so it means good for normal users. but not for developers of it, because simplicity actually is hidden complexity.that is the reason developers fulfilled so many rule options to match the traffic. and the man page of ipfw becomes long long. (maybe the manpage for ipfw should be spitted into multiple pages) Yesterday I was thinking, "a firewall is ... when the traffic comes, it will be filtered based on the rule, and the action will be executed when the rule matched". so actually the job is quite simple. So I was thinking whether there is a generic method to handle the filtering? And the "U32" module of iptables came into my mind immediately.I think the feature is cool. and I am going to introduce this feature into ipfw, if have people like this feature, since i am free recently :). So i am proposing a new rule option `u32` and the usage will be "u32 " e.g. >ipfw add 1 allow all from any to any u32 0 0x112233445566 layer2 It means if part of the traffic(start from position 0) is equal to the 0x112233445566, then it means matched. Or maybe the usage will be more complex that the above. maybe "u32 " e.g >ipfw add 1 allow all from any to any u32 0 0xFFFFFF000000FFFFFF000000 0x111111000000222222000000 layer2 the traffic will be AND the before comparing the . It sounds like "nothing impossible" with this feature!. It is a really powerful thing in my opinion. but it has requirement, to master it requires the knowledge of the structure of the packet/frame/whatever. Anyone like this feature? Like it ? please voice out. Best Regards, bycn82 From owner-freebsd-net@FreeBSD.ORG Thu May 29 07:43:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38F72AAE for ; Thu, 29 May 2014 07:43:00 +0000 (UTC) Received: from quix.smartspb.net (quix.smartspb.net [217.119.16.133]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E995E28BF for ; Thu, 29 May 2014 07:42:59 +0000 (UTC) Received: from dyr.smartspb.net ([217.119.16.26] helo=[127.0.0.1]) by quix.smartspb.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.61 (FreeBSD)) (envelope-from ) id 1Wpv3O-00065Y-GL; Thu, 29 May 2014 11:47:10 +0400 Message-ID: <5386E4F6.1000001@smartspb.net> Date: Thu, 29 May 2014 11:42:46 +0400 From: Dennis Yusupoff User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Bill Yuan , FreeBSD Net Subject: Re: propose a new generic purpose rule option for ipfw References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Antivirus: avast! (VPS 140528-1, 28.05.2014), Outbound message X-Antivirus-Status: Clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 07:43:00 -0000 Looks very similar to ng_bpf + ipfw ngtee, isn't it? 29.05.2014 11:25, Bill Yuan ÐŋÐļŅˆÐĩŅ‚: > It is a really powerful thing in my opinion. but it has requirement, > to master it requires the knowledge of the structure of the > packet/frame/whatever. Anyone like this feature? Like it ? please > voice out. -- Best regards, Dennis Yusupoff, network engineer of Smart-Telecom ISP Russia, Saint-Petersburg From owner-freebsd-net@FreeBSD.ORG Thu May 29 08:00:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01441CF; Thu, 29 May 2014 08:00:10 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9384729F3; Thu, 29 May 2014 08:00:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=A+0Svm+SbdgUThZDPWRpNFd5tUxXdAnY2lAxIOLY4k4=; b=c4UY/kdivaFf5032K1Rof3OPti5mdwcCgiPCahWqQO4p5raNWfIMl36q0h9kXf5PB1hkkncotkgFRBMAImaWzWRFu6Zqade3/O3lyMnAuAJb8u/oDcwQWz1w28lgfz2Qzd79brEXP/UlK3K5myYAdSbFoGILG02KrxHZJ2ntz9XtWexgnegPIbfO6qT5e3HMEsuxRaAhDqzFbi7XWzbPpeDJ/cHPWbz6fWeTwtelDS5qdnjO951DOU+nIF3AhWFfQ4nKz1iRa7S0TNrOWbIYEIW1T1ZxG+BfNBXS6pddrnTqVmP+Mzk7zioAo2MKvGIcFjh8O5g/WgmrKoz6blejFGiH232dgyp9hYj1+w94UElt2sOTvqIgcch0i3U9WOx/SCIOQKYV1sGOqWSE2f0LmVgDtPd5s5M8yFWUmZH2BtBKi0ZCA9pU5j+PDs01Bk/memE82VWVIOwmgO97VnTiA7dqDgr5FS7SAdhKXk7ddp5J/L65aUxG/sFCOuLUIKDmJscNbt/qwwVNMnjKtIzrDMkHZ98i5D3i0GPrkEaveraxE0kfNolAVIq2zpi2Bp40qtBe/CBhCEameA8nnfrVySyMtBFc5cEbbi+G+RXBC3mmxKOjtQo2Yx93hQTPeqRZkTCzpQTh1HFYWkuNWxeB0t5R4JpqqQM+cF+H8ohOVas=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WpvFv-000JZl-6m; Thu, 29 May 2014 12:00:07 +0400 Date: Thu, 29 May 2014 12:00:05 +0400 From: Eygene Ryabinkin To: hiren panchasara Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Message-ID: References: <201405222101.s4ML122N061489@freefall.freebsd.org> <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="keyOwv2R5UpfANsk" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: "freebsd-net@freebsd.org" , FreeBSD GNATS followup X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 08:00:10 -0000 --keyOwv2R5UpfANsk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Wed, May 28, 2014 at 11:52:51PM -0700, hiren panchasara wrote: > On Wed, May 28, 2014 at 10:46 PM, Eygene Ryabinkin wrot= e: > > I assume that your pf(4) is enabled during these tests, you have > > "scrub" statements in the ruleset and removing "scrub" will restore > > the expected behaviour on 10.x? >=20 > I can confirm that I see exactly what you are saying on a stable/10 box. I had found 2 flavors of 9.x boxen: 9.1/9.2 that behave like 10.x and some 9.0 that are dropping SYN|FIN even in the presence of "scrub". The trouble is that the latter boxes are in full production, so I need some time to try to reproduce that on the text box. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --keyOwv2R5UpfANsk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOG6QVfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PtX4gEAlfR1J3rriTRJrZSkZMvZ6wRP jVK+1i9Qvupkk+wiooIA+wTk7wyrdGMlW6j/+7MmLcJN8buTeOAsUG18GJ9ef/AH =xpit -----END PGP SIGNATURE----- --keyOwv2R5UpfANsk-- From owner-freebsd-net@FreeBSD.ORG Thu May 29 08:10:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53E474B6 for ; Thu, 29 May 2014 08:10:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4129B2B13 for ; Thu, 29 May 2014 08:10:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4T8A1g5078488 for ; Thu, 29 May 2014 08:10:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4T8A16q078487; Thu, 29 May 2014 08:10:01 GMT (envelope-from gnats) Date: Thu, 29 May 2014 08:10:01 GMT Message-Id: <201405290810.s4T8A16q078487@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Eygene Ryabinkin Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Reply-To: Eygene Ryabinkin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 08:10:01 -0000 The following reply was made to PR kern/190102; it has been noted by GNATS. From: Eygene Ryabinkin To: hiren panchasara Cc: "freebsd-net@freebsd.org" , FreeBSD GNATS followup Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Date: Thu, 29 May 2014 12:00:05 +0400 --keyOwv2R5UpfANsk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Wed, May 28, 2014 at 11:52:51PM -0700, hiren panchasara wrote: > On Wed, May 28, 2014 at 10:46 PM, Eygene Ryabinkin wrot= e: > > I assume that your pf(4) is enabled during these tests, you have > > "scrub" statements in the ruleset and removing "scrub" will restore > > the expected behaviour on 10.x? >=20 > I can confirm that I see exactly what you are saying on a stable/10 box. I had found 2 flavors of 9.x boxen: 9.1/9.2 that behave like 10.x and some 9.0 that are dropping SYN|FIN even in the presence of "scrub". The trouble is that the latter boxes are in full production, so I need some time to try to reproduce that on the text box. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --keyOwv2R5UpfANsk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOG6QVfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PtX4gEAlfR1J3rriTRJrZSkZMvZ6wRP jVK+1i9Qvupkk+wiooIA+wTk7wyrdGMlW6j/+7MmLcJN8buTeOAsUG18GJ9ef/AH =xpit -----END PGP SIGNATURE----- --keyOwv2R5UpfANsk-- From owner-freebsd-net@FreeBSD.ORG Thu May 29 08:14:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB669790 for ; Thu, 29 May 2014 08:14:57 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1AD2BBC for ; Thu, 29 May 2014 08:14:57 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id m15so12443993wgh.20 for ; Thu, 29 May 2014 01:14:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/p3GR+7KW/n/8493Wr0vTs6AIozgNqgGx6qjR5lmhLk=; b=tQIScgmWQ8XqGLw6AjMDQ3RzB2PVl7Cfh5rgE8UxfqaRWCRn/TY9x1ewyj3URZoexV YHOgbg1Ddni5NtoX9y+kK3BqUAQ1sbrrWGFrv+gLjnmhuINjaKkBmED9pC/rfH0qt8AS 0YdMuFuCRQ7Iky0gNdF5yqbECLeUhJIxr3pCagARJd1/QZZmUB9210P+oonaj8DzHbCd tnPFuBYOgiADpis8t66bsoJrxbQSx/AgyAwaAeVUBwCH0g60UWlA9CRvzCPYmBC5zL9P 8WZaikobI+mHwLtt0C2U5Hm6DnAbGXtG6I8psReJN2oVOHIlVsx5rEPj9B6s0s5MdqU8 Kp5w== MIME-Version: 1.0 X-Received: by 10.180.73.66 with SMTP id j2mr8999527wiv.36.1401351294724; Thu, 29 May 2014 01:14:54 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Thu, 29 May 2014 01:14:54 -0700 (PDT) In-Reply-To: References: Date: Thu, 29 May 2014 10:14:54 +0200 X-Google-Sender-Auth: aG_97w0A5L3X3CAHZPG1j7tpfLo Message-ID: Subject: Re: propose a new generic purpose rule option for ipfw From: Luigi Rizzo To: Bill Yuan Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 08:14:57 -0000 On Thu, May 29, 2014 at 9:25 AM, Bill Yuan wrote: > hi > > the rule of ipfw is kind of semantic, and it is powerful. so it means goo= d > for normal users. but not for developers of it, because simplicity actual= ly > =E2=80=8B...=E2=80=8B > > So i am proposing a new rule option `u32` and the usage will be "u32 > " > > e.g. > > >ipfw add 1 allow all from any to any u32 0 0x112233445566 layer2 > > It means if part of the traffic(start from position 0) is equal to the > 0x112233445566, then it means matched. > > Or maybe the usage will be more complex that the above. maybe "u32 > " > > e.g > > >ipfw add 1 allow all from any to any u32 0 0xFFFFFF000000FFFFFF000000 > 0x111111000000222222000000 > layer2 > the traffic will be AND the before comparing the . > > > It sounds like "nothing impossible" with this feature!. > > It is a really powerful thing in my opinion. but it has requirement, to > master it requires the knowledge of the structure of the > packet/frame/whatever. > > Anyone like this feature? Like it ? please voice out. > > Sure your generic binary match could be a welcome addition to ipfw. But its usefulness is extremely limited in practice, as it only lets you match stuff in fixed position of a packet, and it is not even good to do other relatively simple things such as skip options and the like. If you have spare time you might try how hard would it be to hook the in-kernel bpf to ipfw. That would be a superset of what you propose, and avoids duplication of effort and features. This said, even bpf is not a generic firewall mechanism. Much of the power of a firewall comes from the availability of high level functions and data structures and metadata access functions to reduce the complexity and improve the quality of matches. Many of the ipfw options are just for that: address and port bitmaps, tables, reverse path lookup, metadata lookups and so on. cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 29 08:36:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48546E1A; Thu, 29 May 2014 08:36:54 +0000 (UTC) Received: from ns.kevlo.org (220-135-115-6.HINET-IP.hinet.net [220.135.115.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ns.kevlo.org", Issuer "ns.kevlo.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EEAE12DBA; Thu, 29 May 2014 08:36:53 +0000 (UTC) Received: from ns.kevlo.org (localhost [127.0.0.1]) by ns.kevlo.org (8.14.8/8.14.8) with ESMTP id s4T8aK4B008445 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 29 May 2014 16:36:21 +0800 (CST) (envelope-from kevlo@ns.kevlo.org) Received: (from kevlo@localhost) by ns.kevlo.org (8.14.8/8.14.8/Submit) id s4T8aJiH008444; Thu, 29 May 2014 16:36:19 +0800 (CST) (envelope-from kevlo) Date: Thu, 29 May 2014 16:36:19 +0800 From: Kevin Lo To: Jason Hellenthal Subject: Re: [VIMAGE][udplite] FreeBSD 10-STABLE/powerpc Message-ID: <20140529083619.GA8437@ns.kevlo.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-net@freebsd.org" , adrian@freebsd.org, "\[FreeBSD Stable\]" , jhb@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 08:36:54 -0000 On Thu, May 29, 2014 at 02:34:47AM -0400, Jason Hellenthal wrote: > Is anyone aware that VIMAGE on powerpc is currently broken ? Hi Jason, Did you mean that compile VIMAGE support into your kernel will fail? If so, what compiler do you use? Thanks. > > In file included from /export/usr/src/sys/netinet/in_proto.c:83: > /export/usr/src/sys/netinet/udp_var.h: In function 'get_inpcbinfo': > /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer to > incomplete type > /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer to > incomplete type > /export/usr/src/sys/netinet/udp_var.h: In function 'get_pcblist': > /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer to > incomplete type > /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer to > incomplete type > *** Error code 1 > > The relevant code in that header is: > get_inpcbinfo(uint8_t protocol) > { > return (protocol == IPPROTO_UDP) ? &V_udbinfo : &V_ulitecbinfo; > } > > get_pcblist(uint8_t protocol) > { > return (protocol == IPPROTO_UDP) ? &V_udb : &V_ulitecb; > } > > Working Copy Root Path: /usr/src > URL: svn://svn.freebsd.org/base/stable/10 > Relative URL: ^/stable/10 > Repository Root: svn://svn.freebsd.org/base > Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f > Revision: 266818 > Node Kind: directory > Schedule: normal > Last Changed Author: delphij > Last Changed Rev: 266816 > Last Changed Date: 2014-05-28 14:51:49 -0400 (Wed, 28 May 2014) > > Also looking at svn it appears to have come from this commit... I have > backed out the change here and it appears to be following through so more > attention to VIMAGE and udplite seems to be needed. > > ------------------------------------------------------------------------ > r265946 | kevlo | 2014-05-13 02:05:53 -0400 (Tue, 13 May 2014) | 14 lines > Changed paths: > M /stable/10 > M /stable/10/lib/libc/net/getaddrinfo.c > M /stable/10/sys/netinet/in.c > M /stable/10/sys/netinet/in.h > M /stable/10/sys/netinet/in_pcb.c > M /stable/10/sys/netinet/in_proto.c > M /stable/10/sys/netinet/udp_usrreq.c > M /stable/10/sys/netinet/udp_var.h > A /stable/10/sys/netinet/udplite.h (from > /head/sys/netinet/udplite.h:264212) > M /stable/10/sys/netinet6/in6_ifattach.c > M /stable/10/sys/netinet6/in6_proto.c > M /stable/10/sys/netinet6/udp6_usrreq.c > M /stable/10/sys/netinet6/udp6_var.h > M /stable/10/sys/sys/param.h > > MFC r264212,r264213,r264248,r265776,r265811,r265909: > > - Add support for UDP-Lite protocol (RFC 3828) to IPv4 and IPv6 stacks. > Tested with vlc and a test suite [1]. > [1] http://www.erg.abdn.ac.uk/~gerrit/udp-lite/files/udplite_linux.tar.gz > > Reviewed by: jhb, glebius, adrian > > - Fix a logic bug which prevented the sending of UDP packet with 0 checksum. > > - Disable TX checksum offload for UDP-Lite completely. It wasn't used for > partial checksum coverage, but even for full checksum coverage it doesn't > work. > > ------------------------------------------------------------------------ Kevin From owner-freebsd-net@FreeBSD.ORG Thu May 29 12:30:01 2014 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5027239C for ; Thu, 29 May 2014 12:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25AF62441 for ; Thu, 29 May 2014 12:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4TCU1QD077758 for ; Thu, 29 May 2014 12:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4TCU0f9077757; Thu, 29 May 2014 12:30:00 GMT (envelope-from gnats) Date: Thu, 29 May 2014 12:30:00 GMT Message-Id: <201405291230.s4TCU0f9077757@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Mark Felder Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10 [regression] Reply-To: Mark Felder X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 12:30:01 -0000 The following reply was made to PR kern/190102; it has been noted by GNATS. From: Mark Felder To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10 [regression] Date: Thu, 29 May 2014 07:25:31 -0500 The test box in particular is using pf and does not have any scrub statements in pf.conf. The dropping of SYN+FIN worked for us in 9.1 and older just by setting net.inet.tcp.drop_synfin=1. We skipped 9.2 for the most part, so I don't have any experience with its behavior in production. From owner-freebsd-net@FreeBSD.ORG Thu May 29 12:39:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3D04597 for ; Thu, 29 May 2014 12:39:26 +0000 (UTC) Received: from mailgw12.technion.ac.il (mailgw12.technion.ac.il [132.68.225.12]) by mx1.freebsd.org (Postfix) with ESMTP id 1BC4E2522 for ; Thu, 29 May 2014 12:39:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AncDALgph1OERHMGjGdsb2JhbABZg1mCcqd3BQEBBoM8jR6HPoENFg4BAQEnPIJTEz8OLjQFBxFEiELSK4VDFwSFUYh9HYMVgRUEihyPWAGBPZUnOg X-IPAS-Result: AncDALgph1OERHMGjGdsb2JhbABZg1mCcqd3BQEBBoM8jR6HPoENFg4BAQEnPIJTEz8OLjQFBxFEiELSK4VDFwSFUYh9HYMVgRUEihyPWAGBPZUnOg X-IronPort-AV: E=Sophos;i="4.98,934,1392156000"; d="scan'208";a="109364595" Received: from fermat.math.technion.ac.il ([132.68.115.6]) by mailgw12.technion.ac.il with ESMTP; 29 May 2014 15:38:14 +0300 Received: by fermat.math.technion.ac.il (Postfix, from userid 4298) id 8591983E42; Thu, 29 May 2014 15:33:06 +0300 (IDT) Date: Thu, 29 May 2014 15:33:06 +0300 From: Nadav Har'El To: freebsd-net@freebsd.org Subject: Route caching Message-ID: <20140529123306.GA16644@fermat.math.technion.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hebrew-Date: 29 Iyyar 5774 User-Agent: Mutt/1.5.20 (2009-12-10) Cc: osv-dev@googlegroups.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 12:39:26 -0000 Hi, I'm working on the OSv project (http://osv.io/), a new BSD-licensed operating system for virtual machines. OSv's networking code is based on that of FreeBSD. I recently noticed an inefficiency that I believe exists also in FreeBSD's networking code, and I was wondering why this was done, and whether FreeBSD can also be improved in the same way by fixing this problem. My issue is that, for example, when running a UDP server answering hundreds of thousands of requests per second, I get the same number of calls to the routing table lookup function (rtalloc_ign_fib(), etc.). These calls are relatively slow: Each involves several mutex locks and unlocks (a rwlock for the radix tree, and a mutex for the individual route), which are relatively slow in the uncontended case, but even worse when several CPUs start to access the network heavily, and we start to see context switches hurting the performance of the server even further. Looking at FreeBSD's udp_output(), I see it does the following: error = ip_output(m, inp->inp_options, NULL, ipflags, inp->inp_moptions, inp) Note how NULL is passed as the third parameter. This tells ip_output that it can't cache the previously found route, and needs to look for it again and again on every packet output - even in the common case where a socket will only ever send packets on one interface. It seems that this change was done around FreeBSD 5.4. In the original UCB code (4.4Lite), I see this: error = ip_output(m, inp->inp_options, &inp->inp_route, inp->inp_socket->so_options & (SO_DONTROUTE | SO_BROADCAST), inp->inp_moptions); So the last-found route was cached in inp->inp_route, and possibly reused on the next packet to be sent. Does anyone have any idea why inp->inp_route was removed in FreeBSD? Doesn't this also hurt FreeBSD's network performance? Thanks, Nadav. -- Nadav Har'El | Thursday, May 29 2014, 29 Iyyar 5774 nyh@math.technion.ac.il |----------------------------------------- Phone +972-523-790466, ICQ 13349191 |If glory comes after death, I'm not in a http://nadav.harel.org.il |hurry. (Latin proverb) From owner-freebsd-net@FreeBSD.ORG Thu May 29 12:45:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 136E37F8 for ; Thu, 29 May 2014 12:45:30 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D810125DB for ; Thu, 29 May 2014 12:45:29 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id ld10so312787pab.20 for ; Thu, 29 May 2014 05:45:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=ZbG43PLuPiP7LxXZQHp2MuHp4d2iAvXKNaK5QAyhoco=; b=R5JCcq3hbwoTXfQ4TRqCYULGCCJ87Vr8Tvcq4NnZ5vA5mmev4Bj1W6JTcA1tWOgn26 fKvX4AefLkLdWUYh+h3NBUFHqSmHbnG50q8DaWy8q+89nGbTK884Ur+StaOhfDkkqszU Us2gwXo8FG+lTU3/wSWDdJMudgcSem3SVWenrg8AeGbssnRyIM49phWIW9pkgKfDFM1+ hEepxO2OyDhXJEjFe8ZrOvArYCBSwmwGZdUuK/GKRmIeyf36WveZzLoEecJGq2baY4WX r9aMXltWWMFZDChVgPNSGdGNgsF/nNJo4n7e6IlDan8+4vDdM3CLfmkXtbX5Htc3ngK5 QRpg== X-Received: by 10.66.164.5 with SMTP id ym5mr8728920pab.50.1401367529524; Thu, 29 May 2014 05:45:29 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id qw8sm1140806pbb.27.2014.05.29.05.45.27 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 29 May 2014 05:45:28 -0700 (PDT) From: "bycn82" To: "'Luigi Rizzo'" References: In-Reply-To: Subject: RE: propose a new generic purpose rule option for ipfw Date: Thu, 29 May 2014 20:45:26 +0800 Message-ID: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQGHvkjU79R1H+5sSqlQcstB+XJ0CwD6Vot3m98ZEXA= Content-Language: en-us Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 12:45:30 -0000 =20 Sure your generic binary match could be a welcome addition to ipfw. But its usefulness is extremely limited in practice, as it only lets you match stuff in fixed position of a packet, and it is not even good to do other relatively simple things such as skip options and the like. =20 Sure. When the user need to fulfill a feature and there is a rule option = for it. Then in this scenario . the user should definitely use the = option. Except the user this testing this feature. =20 If you have spare time you might try how hard would it be to hook the in-kernel bpf to ipfw. That would be a superset of what you propose, and avoids duplication of effort and features. =20 By the way, what is in-kernel bpf. I ve read your dummynet already, = still have some place not clear. =20 This said, even bpf is not a generic firewall mechanism. =20 Much of the power of a firewall comes from the availability of high level functions and data structures and metadata access functions to reduce the complexity and improve the quality of matches. Many of the ipfw options are just for that: address and port bitmaps, tables, reverse path lookup, metadata lookups and so on. =20 Sure, that is the reason why developers are providing more and more rule = options. But the my question is do we have enough options to match all = the fixed position values? =20 cheers luigi =20 From owner-freebsd-net@FreeBSD.ORG Thu May 29 13:05:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00CA2B7C for ; Thu, 29 May 2014 13:05:40 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id B67402791 for ; Thu, 29 May 2014 13:05:40 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 7C8937300A; Thu, 29 May 2014 15:10:15 +0200 (CEST) Date: Thu, 29 May 2014 15:10:15 +0200 From: 'Luigi Rizzo' To: bycn82 Subject: Re: propose a new generic purpose rule option for ipfw Message-ID: <20140529131015.GA72798@onelab2.iet.unipi.it> References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 13:05:41 -0000 On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: ... > > Sure, that is the reason why developers are providing more and more rule options. But the my question is do we have enough options to match all the fixed position values? we do not have an option for fixed position matching. As i said, feel free to submit one and i will be happy to import it if the code is clean (btw i am still waiting for fixes to the other 'rate limiting' option you sent), but keep in mind that 'fixed position' is mostly useless. More useful options would be one where you express the position as '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' so at least you can adapt to variant headers, or one where you can look for a pattern in the entire packet or in a portion of it. cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 29 13:32:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 562235F0 for ; Thu, 29 May 2014 13:32:41 +0000 (UTC) Received: from mail-oa0-x22e.google.com (mail-oa0-x22e.google.com [IPv6:2607:f8b0:4003:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C93B2A22 for ; Thu, 29 May 2014 13:32:41 +0000 (UTC) Received: by mail-oa0-f46.google.com with SMTP id g18so316476oah.19 for ; Thu, 29 May 2014 06:32:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8VYFCqPLNFAEP+uen4+ioovRy59A9jssHdpS0PYPkWk=; b=A1Uh10tiiNxRXT4AlJxCOyfQTrx+8ZhxbFlSSvEjQ+fVFu29z6D0X+5ptUtWjM3bGS pEUtV+mdj2atDcCEtWIvK5F5kPUo+7AAIuMDgn+an8eRMCdpnmKBGvGm5iOIlid8XzC8 5/cXNuu++un59GB9sXg1AhriYnN3D7AxORVKAA0VUQgEyB4GVETLRF9WTXHgKeRG9TKb fIjYQ+iAxWIYKV7fGolEAx4sqyY1R01Kk9Es1nleWZ/UM9fM/cFsTj+Dr1UXt2dKgrGs n5BvSLgtyA+LwBt3oJey5zd9oJeDrKLC9EIrekyMrvLFXndcAAUfQHaOrnjaD+h3kyTo shbA== MIME-Version: 1.0 X-Received: by 10.182.109.226 with SMTP id hv2mr8188602obb.79.1401370360420; Thu, 29 May 2014 06:32:40 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Thu, 29 May 2014 06:32:40 -0700 (PDT) In-Reply-To: <20140529131015.GA72798@onelab2.iet.unipi.it> References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> Date: Thu, 29 May 2014 15:32:40 +0200 Message-ID: Subject: Re: propose a new generic purpose rule option for ipfw From: Andreas Nilsson To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , bycn82 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 13:32:41 -0000 On Thu, May 29, 2014 at 3:10 PM, Luigi Rizzo wrote: > On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: > ... > > > > Sure, that is the reason why developers are providing more and more rule > options. But the my question is do we have enough options to match all the > fixed position values? > > we do not have an option for fixed position matching. > > As i said, feel free to submit one and i will be happy to > import it if the code is clean (btw i am still waiting > for fixes to the other 'rate limiting' option you sent), > but keep in mind that 'fixed position' is mostly useless. > > More useful options would be one where you express the position as > > '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' > > so at least you can adapt to variant headers, or one where you can look > for a pattern in the entire packet or in a portion of it. > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Wouldn't PAYLOAD require possibly reassembly of a fragmented packet? It certainly is a good feature, don't get me wrong. But what are the performance hits? Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Thu May 29 13:39:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A033B06 for ; Thu, 29 May 2014 13:39:35 +0000 (UTC) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25CF52AA2 for ; Thu, 29 May 2014 13:39:34 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id k14so403300wgh.31 for ; Thu, 29 May 2014 06:39:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=S1ckVrltOVQCqrrfBc9jk3AM26D14YAlzJqW1NH3lwA=; b=MmLKx96deSxeACji/GZdDi+95O1cXiTDskuALMzjmpkdt8nJmj7TWeYDQyvnk50hQZ VqA5Ryh3weQxPKHdZwnj7e7c8SPLDxX9ipbqAK8Z/O085TNUWrjmR1j1id+oc9u+El2w hCNK2SsFuZidEEp/wRHvEda+KQ5ByEX65M1x81dhbrNOjai7a4YUyT1hJ11TGXtc5QUX ChrnhyzE1FEZ+M3emv+0CTmFgcr0gY06hswfEgxwh/I7P/XVRWPRu/SSwRDSVAfuSvqX aIwTXmmmdKy7t+6b5B3ydyIAHNdQtJmpqBkIjaRBlRx3HvqM52W0NWpYOrws+IV1kU+K 2MsA== MIME-Version: 1.0 X-Received: by 10.180.183.131 with SMTP id em3mr59271054wic.56.1401370773039; Thu, 29 May 2014 06:39:33 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Thu, 29 May 2014 06:39:32 -0700 (PDT) In-Reply-To: References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> Date: Thu, 29 May 2014 15:39:32 +0200 X-Google-Sender-Auth: _8lPprQ6CPgBw9hbSMYLuj4or3A Message-ID: Subject: Re: propose a new generic purpose rule option for ipfw From: Luigi Rizzo To: Andreas Nilsson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , bycn82 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 13:39:35 -0000 On Thu, May 29, 2014 at 3:32 PM, Andreas Nilsson wrote= : > On Thu, May 29, 2014 at 3:10 PM, Luigi Rizzo wrote: > >> On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: >> ... >> > >> > Sure, that is the reason why developers are providing more and more >> rule options. But the my question is do we have enough options to match = all >> the fixed position values? >> >> we do not have an option for fixed position matching. >> >> As i said, feel free to submit one and i will be happy to >> import it if the code is clean (btw i am still waiting >> for fixes to the other 'rate limiting' option you sent), >> but keep in mind that 'fixed position' is mostly useless. >> >> More useful options would be one where you express the position as >> >> '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' >> >> so at least you can adapt to variant headers, or one where you can look >> for a pattern in the entire packet or in a portion of it. >> >> cheers >> luigi >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > Wouldn't PAYLOAD require possibly reassembly of a fragmented packet? > =E2=80=8Bwell, other firewalls do reassemble fragments, ipfw does not (actually there was some code floating around in the past that did implement a reassembly, not sure if it was committed). With this in mind, PAYLOAD would not be that different from TCP if you think that you can have a ton of IPV6 headers and extensions.=E2=80=8B So if/when we implement reassembly, that would be the default for any action that searches past the end of the first fragment. Except from fragmentation, all ipfw instructions already track the beginning of the relevant header for the info at hand (typically skipping ip options or ipv6 headers). It costs something, but not a fortune. cheers luigi > It certainly is a good feature, don't get me wrong. But what are the > performance hits? > > Best regards > Andreas > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Thu May 29 13:44:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8195D5C for ; Thu, 29 May 2014 13:44:47 +0000 (UTC) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB9BF2B3E for ; Thu, 29 May 2014 13:44:47 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id wm4so325787obc.40 for ; Thu, 29 May 2014 06:44:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jT4SpgtE+BCURAYka4tJ9pbCaLdAR+fnPD1t9mNQSoM=; b=kY1XwsbeyLsR3WRv6pVuk5fpHzDA2rrmSZGoyyaU0EpDo9C6jDNr8zIWMqMCZF7M35 Aj4lrFnnyJwMYaSvfSDW28/BweaDUC5kRnAN3zXop9bfrOlhoDmY9uRxlIGX6dj+Pnqk N0C5ByzskrH1kCRU+WbC9s8hjk5DzPWiFOKOzDVsT14F8CdmjH5NL8Si+Bg2AeQSnWfj aCyrwVc6TdSSnX4Zz/QCFrl9e1N+NVcZD6o0mwExekYHC28xekv5P+YqMIv3WnnzjAR2 px4D2prDTqfIo9WWyRJmV1pqhHyS6kT2sX1yBvMKRnL5FVmpxgWJtYh7ywptufbWE0/M 4Dog== MIME-Version: 1.0 X-Received: by 10.60.74.163 with SMTP id u3mr8867695oev.2.1401371086910; Thu, 29 May 2014 06:44:46 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Thu, 29 May 2014 06:44:46 -0700 (PDT) In-Reply-To: References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> Date: Thu, 29 May 2014 15:44:46 +0200 Message-ID: Subject: Re: propose a new generic purpose rule option for ipfw From: Andreas Nilsson To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , bycn82 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 13:44:48 -0000 On Thu, May 29, 2014 at 3:39 PM, Luigi Rizzo wrote: > > > > On Thu, May 29, 2014 at 3:32 PM, Andreas Nilsson wrot= e: > >> On Thu, May 29, 2014 at 3:10 PM, Luigi Rizzo wrote: >> >>> On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: >>> ... >>> > >>> > Sure, that is the reason why developers are providing more and more >>> rule options. But the my question is do we have enough options to match= all >>> the fixed position values? >>> >>> we do not have an option for fixed position matching. >>> >>> As i said, feel free to submit one and i will be happy to >>> import it if the code is clean (btw i am still waiting >>> for fixes to the other 'rate limiting' option you sent), >>> but keep in mind that 'fixed position' is mostly useless. >>> >>> More useful options would be one where you express the position as >>> >>> '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' >>> >>> so at least you can adapt to variant headers, or one where you can look >>> for a pattern in the entire packet or in a portion of it. >>> >>> cheers >>> luigi >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> Wouldn't PAYLOAD require possibly reassembly of a fragmented packet? >> > > Good enough for me. I might be able to get some time on a xena 10g device to check some numbers if there is any interest for coming changesets. Best regards Andreas > =E2=80=8Bwell, other firewalls do reassemble fragments, ipfw does not > (actually there was some code floating around in the past that > did implement a reassembly, not sure if it was committed). > With this in mind, PAYLOAD would not be that different > from TCP if you think that you can have a ton of IPV6 headers and > extensions.=E2=80=8B So if/when we implement reassembly, that would be > the default for any action that searches past the end of > the first fragment. > > Except from fragmentation, all ipfw instructions already track > the beginning of the relevant header for the info at hand > (typically skipping ip options or ipv6 headers). > It costs something, but not a fortune. > > cheers > luigi > > >> It certainly is a good feature, don't get me wrong. But what are the >> performance hits? >> >> Best regards >> Andreas >> > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > From owner-freebsd-net@FreeBSD.ORG Thu May 29 13:49:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1EEBF7E for ; Thu, 29 May 2014 13:49:02 +0000 (UTC) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 818812B88 for ; Thu, 29 May 2014 13:49:02 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id uo5so405629pbc.28 for ; Thu, 29 May 2014 06:49:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=qmSP4djW9CqpcTBxJDSvJB3VJNCcmzU7rfS9A2ebyFE=; b=VBhOUpkFhDojckNbyq0W/K1czIDnQUo0A742Y+zrKjDRE1O/DNv57jMmLM7ey3DeOf 5xyya+t/9Q7pxO3azNj1YfyUzlCVBrvdbfq40rjxoxVCzkqF3d+qqdfUwVb8L6bsB6EW ZM32y/rgqAEUH5L+At5Vwp9JDpod96dqNv6IPM3fqk2yEd+7j+dTvximWS56QO4Ef6bH fihV3PI8Z0l8mgBDSaMM3/ZEjcpOtEGqixfipm/JuGOA9OLG89m7yjO7HVc0UNo7d0gK 0UhrmiVQHb+ddDzKf+j0qxlQMBTxwGV8wZ7tVXH1Q/KTIeLSa7oWsb8a3WqdfNN+AyZB L5uA== X-Received: by 10.66.120.201 with SMTP id le9mr9030702pab.98.1401371341999; Thu, 29 May 2014 06:49:01 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id jt7sm1388966pbc.46.2014.05.29.06.48.58 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 29 May 2014 06:49:01 -0700 (PDT) From: "bycn82" To: "'Luigi Rizzo'" References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> In-Reply-To: <20140529131015.GA72798@onelab2.iet.unipi.it> Subject: RE: propose a new generic purpose rule option for ipfw Date: Thu, 29 May 2014 21:48:58 +0800 Message-ID: <003201cf7b44$bfd6ed40$3f84c7c0$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQGHvkjU79R1H+5sSqlQcstB+XJ0CwD6Vot3AjU0ir0BPakgnpvDnOTQ Content-Language: en-us Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 13:49:02 -0000 =20 -----Original Message----- From: 'Luigi Rizzo' [mailto:rizzo@iet.unipi.it]=20 Sent: 29 May, 2014 21:10 To: bycn82 Cc: 'FreeBSD Net' Subject: Re: propose a new generic purpose rule option for ipfw =20 On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: ... >=20 > Sure, that is the reason why developers are providing more and more = rule options. But the my question is do we have enough options to match = all the fixed position values? =20 we do not have an option for fixed position matching. =20 Can I say that =E2=80=9CIt will be useful when a user come up with a = special requirement which cannot be fulfilled by any existing rule = option.=E2=80=9D Since there are so many rule options already. So I = don=E2=80=99t know when that special requirement will appear. L that is = what you said =E2=80=9Cuseless=E2=80=9D, I accept that . =20 As i said, feel free to submit one and i will be happy to import it if = the code is clean (btw i am still waiting for fixes to the other 'rate = limiting' option you sent), but keep in mind that 'fixed position' is = mostly useless. Which `rate limiting`, the `Packet per second`?=20 http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/189720 =20 =20 More useful options would be one where you express the position as =20 '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' =20 It is possible, =20 match the can be a pattern , then that means it can match multiple = value at the same time. =20 so at least you can adapt to variant headers, or one where you can look = for a pattern in the entire packet or in a portion of it. =20 cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu May 29 14:11:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65B6A8FA for ; Thu, 29 May 2014 14:11:24 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 260B02E6C for ; Thu, 29 May 2014 14:11:23 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id BEA407300A; Thu, 29 May 2014 16:15:59 +0200 (CEST) Date: Thu, 29 May 2014 16:15:59 +0200 From: 'Luigi Rizzo' To: bycn82 Subject: Re: propose a new generic purpose rule option for ipfw Message-ID: <20140529141559.GC74344@onelab2.iet.unipi.it> References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> <003201cf7b44$bfd6ed40$3f84c7c0$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <003201cf7b44$bfd6ed40$3f84c7c0$@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 14:11:24 -0000 On Thu, May 29, 2014 at 09:48:58PM +0800, bycn82 wrote: > > > -----Original Message----- > From: 'Luigi Rizzo' [mailto:rizzo@iet.unipi.it] > Sent: 29 May, 2014 21:10 > To: bycn82 > Cc: 'FreeBSD Net' > Subject: Re: propose a new generic purpose rule option for ipfw > > > > On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: > > ... > > > > > > Sure, that is the reason why developers are providing more and more rule options. But the my question is do we have enough options to match all the fixed position values? > > > > we do not have an option for fixed position matching. > > > > Can I say that ???It will be useful when a user come up with a special requirement which cannot be fulfilled by any existing rule option.??? Since there are so many rule options already. So I don???t know when that special requirement will appear. L that is what you said ???useless???, I accept that . please re-read what i said below. 'mostly useless' != 'useless', and i am ok importing a clean implementation. > As i said, feel free to submit one and i will be happy to import it if the code is clean (btw i am still waiting for fixes to the other 'rate limiting' option you sent), but keep in mind that 'fixed position' is mostly useless. > > Which `rate limiting`, the `Packet per second`? > > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/189720 ok commented the remaining problems with a separate email. > More useful options would be one where you express the position as > > > > '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' > > > > It is possible, > > match > > the can be a pattern , then that means it can match multiple value at the same time. what i wrote is a completely different thing. Never mind. cheers luigi > > > > so at least you can adapt to variant headers, or one where you can look for a pattern in the entire packet or in a portion of it. > > > cheers > > luigi > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu May 29 14:40:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DEE1A8F0 for ; Thu, 29 May 2014 14:40:51 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A57C2121 for ; Thu, 29 May 2014 14:40:51 +0000 (UTC) Received: by mail-ig0-f174.google.com with SMTP id h3so3838903igd.13 for ; Thu, 29 May 2014 07:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=WxbrLTbUlj6KXldYZPrIcUjNGah7fzzfEhphkI/HuJc=; b=djK+FWjL+OiV/YQRh4jmSmBko86n95wwpzoF3EWcFTk7TqD7qXiOCYxLKvhWG3L/ZL EMFLCcaReHmV+3oKsRViTrKHFoqvYiZBeKfaeR+SOl0ZLoD6N8sv+/wD9mHL10ZX3aLL XWL/gm84PLI9vAUmdMYZ2QEsATVDebo/55Jug= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=WxbrLTbUlj6KXldYZPrIcUjNGah7fzzfEhphkI/HuJc=; b=XQkS5WBxuoGn/JrnmclGCXuLufb9pwcp7I+857Menf+mFMaxivtJYzRethNWbIR8OC v95+NKH6cvc6z44MlPMoP+07yt084cfDoRcLdRGG03NOeby81JleIts0mQIiHEBWc5xB jV84nVnKLsvMuSaIc6E7SzU0t0VrKNj9+gfIfMAaBE8yo8ThD3iffIOy7NM8Pvv9e7a1 mh+W1RPFnSNV44jhqLsFehSASrzXDsrvFlFY1wqGyQuX+sZ1Gkb3sqyNRCpshQLWVGQA fv9OOTVqUVWIANNWTBNM97eS3RiJQeltG4a53I4XmpahT29HJWeTYL+ZrPJDNk8P01Ho s7Vg== X-Gm-Message-State: ALoCoQmHFXk2Y23xnBV4jt8fZfnFfqze5SUxt19l2yPuDD0bZdbBCl6LObr6MLSrFrKveDN283Qo X-Received: by 10.50.30.6 with SMTP id o6mr53355692igh.43.1401374450996; Thu, 29 May 2014 07:40:50 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id mk2sm24295178igb.8.2014.05.29.07.40.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 29 May 2014 07:40:49 -0700 (PDT) References: <20140529083619.GA8437@ns.kevlo.org> Mime-Version: 1.0 (1.0) In-Reply-To: <20140529083619.GA8437@ns.kevlo.org> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-CFF60091-233D-4119-AC52-B5A8ECC1508A; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: [VIMAGE][udplite] FreeBSD 10-STABLE/powerpc Date: Thu, 29 May 2014 10:40:48 -0400 To: Kevin Lo X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , "adrian@freebsd.org" , "\[FreeBSD Stable\]" , "jhb@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 14:40:52 -0000 --Apple-Mail-CFF60091-233D-4119-AC52-B5A8ECC1508A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hi Kevin, Default on PowerPC is GCC 4.2.1 Its hard to see that this wouldn't turn up elsewhere on other arch' stop tho= ugh as from what I seen doesn't seem to be dependent on PowerPC alone. But to confirm my previous build, after backing out udplite the build did co= mplete just fine. I'll find out in a little while whether it runs :-) crossi= ng fingers. =20 --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On May 29, 2014, at 4:36, Kevin Lo wrote: >=20 >> On Thu, May 29, 2014 at 02:34:47AM -0400, Jason Hellenthal wrote: >> Is anyone aware that VIMAGE on powerpc is currently broken ? >=20 > Hi Jason, >=20 > Did you mean that compile VIMAGE support into your kernel will fail? > If so, what compiler do you use? Thanks. >=20 >>=20 >> In file included from /export/usr/src/sys/netinet/in_proto.c:83: >> /export/usr/src/sys/netinet/udp_var.h: In function 'get_inpcbinfo': >> /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer t= o >> incomplete type >> /export/usr/src/sys/netinet/udp_var.h:153: error: dereferencing pointer t= o >> incomplete type >> /export/usr/src/sys/netinet/udp_var.h: In function 'get_pcblist': >> /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer t= o >> incomplete type >> /export/usr/src/sys/netinet/udp_var.h:159: error: dereferencing pointer t= o >> incomplete type >> *** Error code 1 >>=20 >> The relevant code in that header is: >> get_inpcbinfo(uint8_t protocol) >> { >> return (protocol =3D=3D IPPROTO_UDP) ? &V_udbinfo : &V_ulitecbinfo= ; >> } >>=20 >> get_pcblist(uint8_t protocol) >> { >> return (protocol =3D=3D IPPROTO_UDP) ? &V_udb : &V_ulitecb; >> } >>=20 >> Working Copy Root Path: /usr/src >> URL: svn://svn.freebsd.org/base/stable/10 >> Relative URL: ^/stable/10 >> Repository Root: svn://svn.freebsd.org/base >> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f >> Revision: 266818 >> Node Kind: directory >> Schedule: normal >> Last Changed Author: delphij >> Last Changed Rev: 266816 >> Last Changed Date: 2014-05-28 14:51:49 -0400 (Wed, 28 May 2014) >>=20 >> Also looking at svn it appears to have come from this commit... I have >> backed out the change here and it appears to be following through so more= >> attention to VIMAGE and udplite seems to be needed. >>=20 >> ------------------------------------------------------------------------ >> r265946 | kevlo | 2014-05-13 02:05:53 -0400 (Tue, 13 May 2014) | 14 lines= >> Changed paths: >> M /stable/10 >> M /stable/10/lib/libc/net/getaddrinfo.c >> M /stable/10/sys/netinet/in.c >> M /stable/10/sys/netinet/in.h >> M /stable/10/sys/netinet/in_pcb.c >> M /stable/10/sys/netinet/in_proto.c >> M /stable/10/sys/netinet/udp_usrreq.c >> M /stable/10/sys/netinet/udp_var.h >> A /stable/10/sys/netinet/udplite.h (from >> /head/sys/netinet/udplite.h:264212) >> M /stable/10/sys/netinet6/in6_ifattach.c >> M /stable/10/sys/netinet6/in6_proto.c >> M /stable/10/sys/netinet6/udp6_usrreq.c >> M /stable/10/sys/netinet6/udp6_var.h >> M /stable/10/sys/sys/param.h >>=20 >> MFC r264212,r264213,r264248,r265776,r265811,r265909: >>=20 >> - Add support for UDP-Lite protocol (RFC 3828) to IPv4 and IPv6 stacks. >> Tested with vlc and a test suite [1]. >> [1] http://www.erg.abdn.ac.uk/~gerrit/udp-lite/files/udplite_linux.tar.g= z >>=20 >> Reviewed by: jhb, glebius, adrian >>=20 >> - Fix a logic bug which prevented the sending of UDP packet with 0 checks= um. >>=20 >> - Disable TX checksum offload for UDP-Lite completely. It wasn't used for= >> partial checksum coverage, but even for full checksum coverage it doesn'= t >> work. >>=20 >> ------------------------------------------------------------------------ > =20 > Kevin --Apple-Mail-CFF60091-233D-4119-AC52-B5A8ECC1508A Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDUyOTE0NDA1MFowIwYJKoZIhvcN AQkEMRYEFFaYD16Azj4++WcaGvwLxpeJonSYMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAVWsrOwvApJmwkqCQVedH HQ7AQANyuRd0UYaafxvFWO/E4N21f6t2MeAgZ3fbX7kmM3BIXKxVOyX6POjyuPSjAFvWOwrrDeGy 6ZMKDZawRWwKz5nUzFc4e7mNlJcqMaW3LGzC9ZcaduiUMuse+/nTrE9948hS+W09LcSJvt1oatTh OAOTW0wS5nMizSZZwamBwjAYQj2U6AFzjaYnIA8JOPBhXxRjPyT4G5zYfpN9ILM9vUOcivF+ruXB MXsDZ89fnQTzeAWY8ot2IVvfdGfkO5fP/LDiW7y67PoORNOxfujyQiK8dHUl7bAXKNMLnKVOkX7W AVp3VW1t6ze+Kh4AxAAAAAAAAA== --Apple-Mail-CFF60091-233D-4119-AC52-B5A8ECC1508A-- From owner-freebsd-net@FreeBSD.ORG Thu May 29 15:02:38 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10CA6404; Thu, 29 May 2014 15:02:38 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A69082378; Thu, 29 May 2014 15:02:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=Ktc1kqCI+CdnEZ4AGOJ3u4RiZM917fl6MYuUe+QsZDo=; b=eYk+y56paK1LcJVTEXJSgAIdLu60E+01mcHgroN4m6jgCafQxOpA97Mnh2RIeYFdT7SASNtTtoFhMqUnRFvwLkYLtFdX9ISTgQucSccFf22YOMkXHE+TJDtAnrcmBmsVZhBthckFScyyNS9OYJdU4L9jgalFK3syGw3ZMBOMx3FPzIfVDiZf2SAVdDLOCZu4ELa2Q7lxRGmrzRstJAgfjxfdaGiOysQRBJ20hx4VaKUkgq+rVXbQVU9t5yLhWpi0OhfGNCfcKSI5CyxzjIHNQoKdrkmUKgpW9k+5W4TDe+RemIh+rPrDUw08MU6F2NIF3Ns5sr1p1u18OQrzqi9BBEDjiAX5kOALzdClcXqLOCFxTTWSzoVbJGYiUbz9CKD9izHdSDA+yC+U9LZlOyCpXjUCJcNY/K/LMP3nG+/8bBC2yfInSfNRLE9Z2RKsoA/jJkaNwEUwXdbIYWeBt8gLkAY4x2+5J4xQBh9Tsk0zWO67WSzGFFuVHJZRbxfx64hXaiypJz+74Y5yCqjQeoTkRFoMgG5kPt8/TZuw7g3qZMrGlONEsmy4dHLCrHMqseTuWcQJ7nEVucZamFluHsZnHf55G/df4be1iExsAKMOEr4oaQoUEmXE693V/qizNiw4ZyDhvu4fa63db/ZQ/isR4p817qXigqo7FVts4CLiXhc=; Received: from light.codelabs.ru (v-light.codelabs.ru [144.206.233.83]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1Wq1ql-0001zm-0q; Thu, 29 May 2014 19:02:35 +0400 Date: Thu, 29 May 2014 19:02:27 +0400 From: Eygene Ryabinkin To: Mark Felder Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10 [regression] Message-ID: References: <201405291230.s4TCU0f9077757@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="5I6of5zJg18YgZEa" Content-Disposition: inline In-Reply-To: <201405291230.s4TCU0f9077757@freefall.freebsd.org> Sender: rea@codelabs.ru Cc: freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 15:02:38 -0000 --5I6of5zJg18YgZEa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Then it will be very good to see your pf.conf and pfctl -s all, because just now I can't reproduce that on 10.x without "scrub". --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --5I6of5zJg18YgZEa Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOHTAJfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7PvRlAD/aghcF+Y4iDc0J7sb4uuUZ1mm 1raCQLpVphHnHaKVtVYA/3zvX8X2k2tWBPdGJsdIzaB3vwfx8UCeH6zr3VESWphb =WfpF -----END PGP SIGNATURE----- --5I6of5zJg18YgZEa-- From owner-freebsd-net@FreeBSD.ORG Fri May 30 04:12:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD8B8663 for ; Fri, 30 May 2014 04:12:28 +0000 (UTC) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 817512A62 for ; Fri, 30 May 2014 04:12:28 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id uo5so1271616pbc.0 for ; Thu, 29 May 2014 21:12:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=CTAiAj3QanUqGgcwuksGu4rvTNqBiBS5/MttvsQuizw=; b=z4DnJEO4QXwXhYv9raQPPmkVikxsEVLp8n3lBoRW0Bc55dxoFOQo9Tk7I6vR2e/Aul x9bQJpKNK/eEB5XPYq6HhRXoctnboInsIGLymTQuiL6cOcHMIAikrWJ7To8BHyat2vyU UFY5QAP7BVr+T0VEJAJzTs38JZuz8yOsY++BjwvWgibAj8WNfUq7bxwvnuA6Te9WmB35 mbVCvT0zU/8XmB3ThQozL/sIQHCRqJcN0L6mxc6cakFKH6fmSyKM85HfHxDDQH10lK0+ q69i2wmZ2xiux+xSgrGGVkxv4zno+fgpHSraUu3rstOgzhNqTA1ts73WENZZtxY8oJBV Bq9g== X-Received: by 10.68.197.195 with SMTP id iw3mr14755570pbc.139.1401423148059; Thu, 29 May 2014 21:12:28 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id i10sm12233536pat.36.2014.05.29.21.12.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 29 May 2014 21:12:27 -0700 (PDT) Message-ID: <53880525.6000203@gmail.com> Date: Fri, 30 May 2014 12:12:21 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: TCP stack lock contention with short-lived connections References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <53861209.2000306@verisign.com> In-Reply-To: <53861209.2000306@verisign.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 04:12:28 -0000 Hi, Does any plan commit and MFC to the 10-stable ? Regards Simon 䚎 14-5-29 0:42, Julien Charbon 写道: > > Hi, > > On 23/05/14 22:52, Julien Charbon wrote: >> On 23/05/14 14:06, Julien Charbon wrote: >>> On 27/02/14 11:32, Julien Charbon wrote: >>>> On 07/11/13 14:55, Julien Charbon wrote: >>>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>>> wrote: >>>>>> I have put technical and how-to-repeat details in below PR: >>>>>> >>>>>> kern/183659: TCP stack lock contention with short-lived connections >>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>>> >>>>>> We are currently working on this performance improvement >>>>>> effort; it >>>>>> will impact only the TCP locking strategy not the TCP stack logic >>>>>> itself. We will share on freebsd-net the patches we made for >>>>>> reviewing and improvement propositions; anyway this change might >>>>>> also >>>>>> require enough eyeballs to avoid tricky race conditions introduction >>>>>> in TCP stack. >> >> Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and >> tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan 2014. > > At BSDCan 2014 we were also asked to provide flame graph [1][2] to > highlight impacts of these TCP changes. The Dtrace sampling was done on > a NIC receive queue IRQ bound core. > > o First CPU flame graph on 10.0-RELENG at 40k TCP connection/secs: > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/releng10-40k.html > > Note: > > - __rw_wlock_hard on ipi_lock contention is clear as usual. > > o Second, same test with all our patches applied (thus from 10.0-next > branch [3]): > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-40k.html > > > Note: > > - Almost all __rw_wlock_hard on ipi_lock contention is converted in > idle time. > > o Third, still using 10.0-next branch, the flame graph when doubling > the rate to 80k TCP connection/sec: > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-80k.html > > > My 2 cents. > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html > [2] https://wiki.freebsd.org/201405DevSummit/NetworkStack > [3] https://github.com/verisign/freebsd/commits/share/10.0-next > > -- > Julien > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri May 30 11:09:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C3757A8 for ; Fri, 30 May 2014 11:09:52 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D526229AD for ; Fri, 30 May 2014 11:09:51 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WqGmM-0001cG-PR; Fri, 30 May 2014 10:59:02 +0400 Message-ID: <538866A5.9050901@FreeBSD.org> Date: Fri, 30 May 2014 15:08:21 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Nadav Har'El Subject: Re: Route caching References: <20140529123306.GA16644@fermat.math.technion.ac.il> In-Reply-To: <20140529123306.GA16644@fermat.math.technion.ac.il> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, osv-dev@googlegroups.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 11:09:52 -0000 On 29.05.2014 16:33, Nadav Har'El wrote: > Hi, Hello! > I'm working on the OSv project (http://osv.io/), a new BSD-licensed > operating system for virtual machines. OSv's networking code is based > on that of FreeBSD. > > I recently noticed an inefficiency that I believe exists also in > FreeBSD's networking code, and I was wondering why this was done, > and whether FreeBSD can also be improved in the same way by fixing > this problem. > > My issue is that, for example, when running a UDP server answering > hundreds of thousands of requests per second, I get the same number of > calls to the routing table lookup function (rtalloc_ign_fib(), etc.). > These calls are relatively slow: Each involves several mutex locks and > unlocks (a rwlock for the radix tree, and a mutex for the individual > route), which are relatively slow in the uncontended case, but even worse > when several CPUs start to access the network heavily, and we start to see > context switches hurting the performance of the server even further. Yes, that's true. > Looking at FreeBSD's udp_output(), I see it does the following: > > error = ip_output(m, inp->inp_options, NULL, ipflags, > inp->inp_moptions, inp) > > Note how NULL is passed as the third parameter. This tells ip_output > that it can't cache the previously found route, and needs to look for > it again and again on every packet output - even in the common case > where a socket will only ever send packets on one interface. > > It seems that this change was done around FreeBSD 5.4. In the original > UCB code (4.4Lite), I see this: > > error = ip_output(m, inp->inp_options, &inp->inp_route, > inp->inp_socket->so_options & (SO_DONTROUTE | SO_BROADCAST), > inp->inp_moptions); > > So the last-found route was cached in inp->inp_route, and possibly > reused on the next packet to be sent. > > Does anyone have any idea why inp->inp_route was removed in FreeBSD? > Doesn't this also hurt FreeBSD's network performance? Well, there are two problems: First, using cached routes makes it more complex to change routing stack to be more efficient. The second one is basically the fact that using cached routes is simply incorrect: no one protects/notifies you on interface removal. That's why we've removed cached route support from various tunneling schemes. There is some ongoing work to change rte_* api and eliminate the need for per-rte mutex (and use different, more efficient lookup mechanisms). There is also another alternative which you can currently use: flowtable (not included in GENERIC). It has been fixed recently and should work better in your case. > Thanks, > Nadav. > > From owner-freebsd-net@FreeBSD.ORG Fri May 30 17:58:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0BC2900; Fri, 30 May 2014 17:58:15 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DC39201C; Fri, 30 May 2014 17:58:15 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so6493409qgd.1 for ; Fri, 30 May 2014 10:58:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qnhkhkj6ILpxRSI9Tsm9qQusW/jBBnQnz/brp58wR6I=; b=ou7HA+sYU1p1fovbAxV6cuOHZw7D+ut8uY5gYzpvtczkFC6RCdGGhH9BpnIAVd3WEr yAui+ZYSeR1ERsxW3UVt9Ow4ekFI3jRm3VHL/HeDQRQAqCLpFWMVION0C4BPVdnEUa7o Be0QPjL709kHjRNCvMUEADXVm0ZfnL7EQYdM7bc/6tZsid8ZF+0MEVMDWU5QvpFeSDq+ 7PURzh5kwm9lZzZHED/fDAAIEHdOpBVHOJN/kG2Oso5F5U72/j4Dz3He3h/PSArjGUCo Cfle/A2+YhOG5JXm2dalWFJ+gWk4uTSf4Axww66KRURtnPiO/POppwuB7b1Pun3bLQhB tC5g== MIME-Version: 1.0 X-Received: by 10.224.166.73 with SMTP id l9mr23722700qay.34.1401472694592; Fri, 30 May 2014 10:58:14 -0700 (PDT) Received: by 10.96.122.133 with HTTP; Fri, 30 May 2014 10:58:14 -0700 (PDT) In-Reply-To: <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> References: <201405222101.s4ML122N061489@freefall.freebsd.org> <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> Date: Fri, 30 May 2014 10:58:14 -0700 Message-ID: Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] From: hiren panchasara To: Eygene Ryabinkin Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 17:58:15 -0000 - bugs (as this is not related to it) On Wed, May 28, 2014 at 10:46 PM, Eygene Ryabinkin wrote: > clearing FIN bit for SYN packets was > the standard behaviour of pf since approximately at least 10 years, > http://svnweb.freebsd.org/base/vendor-sys/pf/dist/sys/contrib/pf/net/pf_norm.c?view=markup&pathrev=126258#l1242 I am curious, what's the rationale for this behavior? Why does PF clear the FIN bit for such a packet being a firewall? Cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Fri May 30 20:33:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33C4A301 for ; Fri, 30 May 2014 20:33:55 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E10712DC7 for ; Fri, 30 May 2014 20:33:54 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WqTUn-00038q-E0 for freebsd-net@freebsd.org; Fri, 30 May 2014 22:33:45 +0200 Received: from h87.s239.verisign.com ([216.168.239.87]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 30 May 2014 22:33:45 +0200 Received: from jcharbon by h87.s239.verisign.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 30 May 2014 22:33:45 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: TCP stack lock contention with short-lived connections Date: Fri, 30 May 2014 22:33:35 +0200 Lines: 12 Message-ID: <5388EB1F.6010307@verisign.com> References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <53861209.2000306@verisign.com> <53880525.6000203@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: h87.s239.verisign.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: <53880525.6000203@gmail.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 20:33:55 -0000 Hi Simon, On 30/05/14 06:12, k simon wrote: > Does any plan commit and MFC to the 10-stable ? These patches are still under active review and testing, no plan to commit soon yet. As usual having more people testing these changes (and reporting found issues - or no issue) might accelerate the commit process. -- Julien From owner-freebsd-net@FreeBSD.ORG Sat May 31 13:57:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4578CBA1 for ; Sat, 31 May 2014 13:57:44 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB92C2B1E for ; Sat, 31 May 2014 13:57:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=zWwEokbVXEFGaG/kWP9BLKNdC9DYeA0rVt+4pGVC/58=; b=QuDJJ4fa5J8P50LUhZSoX17/rV4UMFsDFxqK+kFNYX5zUjmb5tvN8+7cmWyAqXPF1/dClFzPxuR/3RrBCdQgSbjZdLi4h9nucH4+H+8jpADHkWLfl4hKKyXZAVktJOxw3MJ+klzhg40MkQIkIJWQiVIy6GPO6diMjRgNOw4nUr0DtrCWeG06bLrplj9n9jFkPTWxvxTWU9xJLsLk1puTOhlwRmmSQCoB6gcmGIMWBPVmZFXHmR5u7nJyALgJZz0D66sV96vkU39CnUN38LZ6H2QgeRu+4pd+9fHIpsIUXqZfSmBjt/k/CiUMxRED1kcsrZh/XfvJy9W0Q1nzVSHskjabSHuV+kz2QaxogGj+7grBtqpQjEgj6u67MeLfgOYBsPAH0R6IPY5mwaOmInNJNy0S0igWpLKXo8Oe+74j4ur8mAXqoQx7ExaAokvNpj3V0MTBb1ykc3xQnOHLHN84LEokJq/abIp7sQwZP7pw7CSgO3FKWMpWDMPKp2G2jze241L8+wMMVXaDrUPzMu+1v/X6WAdr0RuG07KMUjCqqA5rqlqcYPkfNkc28lzI/ldB1XiI3O7zhLFG5urdPhB1d7B8Q54iJ98CPgZMJAJi+jWldunZsfmUBx/FB2G07CSGNRRju9mhNzatF4Of6q8t9j5epURYmwE4GyPYtrndISY=; Received: from light.codelabs.ru ([83.149.9.41]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1Wqjn2-0009Ix-9A; Sat, 31 May 2014 17:57:41 +0400 Date: Sat, 31 May 2014 17:57:33 +0400 From: Eygene Ryabinkin To: hiren panchasara Subject: Re: kern/190102: [tcp] net.inet.tcp.drop_synfin=1 no longer works on FreeBSD 10+ [regression] Message-ID: References: <201405222101.s4ML122N061489@freefall.freebsd.org> <+Uw/Ss5bElti5gir++ydy1GLu7M@dHhGgwofm7uNfL6/X5+bGIkDUYs> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: Sender: rea@codelabs.ru Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 13:57:44 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Fri, May 30, 2014 at 10:58:14AM -0700, hiren panchasara wrote: > > clearing FIN bit for SYN packets was > > the standard behaviour of pf since approximately at least 10 years, > > http://svnweb.freebsd.org/base/vendor-sys/pf/dist/sys/contrib/pf/net/= pf_norm.c?view=3Dmarkup&pathrev=3D126258#l1242 >=20 > I am curious, what's the rationale for this behavior? Why does PF > clear the FIN bit for such a packet being a firewall? My understanding is that it is done to conceal specific reaction of the host's TCP stack that pf's "scrub" rule protects from the outer world scanning. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --G4iJoqBmSsgzjUCe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOJ381fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Pte6wEAkiGss/VwccxO8UM0ppH7RzX1 4JxYLE8Z6ArUUoq07fUA/1KgTR9KGOYfkNP8uXd4VXAGUuRq49QRiQHiiHH5zu84 =POwG -----END PGP SIGNATURE----- --G4iJoqBmSsgzjUCe-- From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 02:23:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA54DBCE; Sun, 1 Jun 2014 02:23:36 +0000 (UTC) Received: from remote.thehowies.com (50-197-91-217-static.hfc.comcastbusiness.net [50.197.91.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "remote.thehowies.com", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7656D233C; Sun, 1 Jun 2014 02:23:35 +0000 (UTC) Received: from PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820]) by PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820%11]) with mapi id 14.01.0438.000; Sat, 31 May 2014 19:22:27 -0700 From: John Howie To: "freebsd-arm@freebsd.org" , "freebsd-net@freebsd.org" , "freebsd-questions@freebsd.org" Subject: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Topic: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Index: AQHPfUBUW2MQdiTIGUuysOzuLv/2xg== Date: Sun, 1 Jun 2014 02:22:26 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.2.140509 x-originating-ip: [203.147.52.194] Content-Type: multipart/mixed; boundary="_003_CFB0A140426CBjohnthehowiescom_" MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 02:23:36 -0000 --_003_CFB0A140426CBjohnthehowiescom_ Content-Type: text/plain; charset="Windows-1252" Content-ID: <07D53905702BB74B9802B9E3C82B70C7@thehowies.local> Content-Transfer-Encoding: quoted-printable Hi all, I apologize for the cross posting of this email, but I believe it will be of interest to people across all three groups. Please feel free to forward to additional groups if you feel they would benefit. I have seen a few posts on and off over the years about Windows Server DHCP not working with FreeBSD. Specifically, the Windows Server DHCP would not return the boot host name/IP address and the root path. The typical response to =B3why won=B9t it work" has typically been that there is a flaw= in Windows Server DHCP code. I did a little digging and found that the problem actually lies in code in FreeBSD. Section 3.5 of RFC 2131 (the DHCP RFC) states that =B3...Second, in its initial DHCPDISCOVER or DHCPREQUEST message, a client may provide the server with a list of specific parameters the client is interested in=8A=B2 and =B3...The client can inform the server which configuration parameters the client is interested in by including the 'parameter request list' option. The data portion of this option explicitly lists the options requested by tag number=8A=B2. A DHCP Server is not required to return any parameter that a client does not ask for. It appears that the ISC-DHCP server, which is recommended by most, will return configured options regardless of whether or not the client asks for them. There are two places in the FreeBSD codebase that DHCP is used to boot the system over a network. The first is in the boot loader, which uses code in lib/libstand. The second is in the NFS code, in sys/nfs. The code is sys/nfs is not always used if the boot loader sets up the environment for the NFS code, either by passing parameters to the kernel (as PXEBOOT appears to do), or information is configured in the boot loader configuration files, e.g. /boot/loader.rc. I have attached two unified diff files which I ask people to test, before I submit them for inclusion into the codebase as fixes. The first, bootp.c.diff fixes the code in lib/libstand/bootp.c to request the boot host (option 12, aka TAG_HOSTNAME) and the NFS root path (option 17, aka TAG_ROOTPATH). This fix has been tested with PXEBOOT on an amd64 box and ubldr on an ARM/RaspberryPI system. The second, bootp_subr.c.diff, fixes code in sys/nfs/bootp_subr.c to request the same options and also to fix bugs that erroneously reported the IP address of the BOOTP/DHCP server. The code assumed that the BOOTP/DHCP server was also the boot host. Please send me all feedback directly. The diff files work with 10.0-RELEASE through 10.0-RELEASE-p3, but will likely work with 9.0 and also CURRENT and STABLE, including 11.0, as the code is old code that does not appear to have changed in a while. If you want to try it on those systems please, please make sure you have backup copies just in case. If you do not have experience configuring Windows Server DHCP just drop me an email, and I will send you a cheat sheet to get you up and running. I am going to grab the latest ubldr code to see if I can get it to work more like PXEBOOT, that appears to pass parameters to the kernel to avoid the need for the NFS BOOTP/DHCP process. If you test on an ARM system with ubldr in RELEASE you will see a lot of unnecessary network activity going on, that we should be able to fix. Regards, John john@thehowies.com (personal) | jhowie@email.arizona.edu (academic) | j.howie@napier.ac.uk (academic) | jhowie@cloudsecurityalliance.org (work) --_003_CFB0A140426CBjohnthehowiescom_ Content-Type: application/octet-stream; name="bootp_subr.c.diff" Content-Description: bootp_subr.c.diff Content-Disposition: attachment; filename="bootp_subr.c.diff"; size=2626; creation-date="Sun, 01 Jun 2014 02:22:26 GMT"; modification-date="Sun, 01 Jun 2014 02:22:26 GMT" Content-ID: <48E8CD1FA2F5FF40B7454A0D1B2A8C6F@thehowies.local> Content-Transfer-Encoding: base64 LS0tIHN5cy9uZnMvYm9vdHBfc3Vici5jCTIwMTQtMDEtMjYgMTU6NTU6NDIuMDAwMDAwMDAwIC0w ODAwCisrKyAvaG9tZS9qb2huL3RtcC9ib290cF9zdWJyLmMJMjAxNC0wNS0yNCAxOTowMzo1OS4w MDAwMDAwMDAgLTA3MDAKQEAgLTE5OCw2ICsxOTgsNyBAQAogCiAvKiBESENQIHNwZWNpZmljIHRh Z3MgKi8KICNkZWZpbmUgVEFHX09WRVJMT0FECSA1MiAgLyogT3B0aW9uIE92ZXJsb2FkICovCisj ZGVmaW5lIFRBR19QQVJBTV9SRVEgICAgNTUgIC8qIFRhZyBkZW5vdGluZyBvcHRpb25hbCB0YWdz IGluIERIQ1AgY2xpZW50IG1lc3NhZ2UsIHJlcXVlc3RpbmcgZGF0YSBmcm9tIERIQ1Agc2VydmVy ICovCiAjZGVmaW5lIFRBR19NQVhNU0dTSVpFICAgNTcgIC8qIE1heGltdW0gREhDUCBNZXNzYWdl IFNpemUgKi8KIAogI2RlZmluZSBUQUdfRU5ECQkyNTUgIC8qIEVuZCBPcHRpb24gKGkuZS4gbm8g bW9yZSBvcHRpb25zKSAqLwpAQCAtNTczLDcgKzU3NCw3IEBACiBzdGF0aWMgaW50CiBib290cGNf Y2FsbChzdHJ1Y3QgYm9vdHBjX2dsb2JhbGNvbnRleHQgKmdjdHgsIHN0cnVjdCB0aHJlYWQgKnRk KQogewotCXN0cnVjdCBzb2NrYWRkcl9pbiAqc2luLCBkc3Q7CisJc3RydWN0IHNvY2thZGRyX2lu ICpzaW4sIGRzdCwgKmZyb207CiAJc3RydWN0IHVpbyBhdWlvOwogCXN0cnVjdCBzb2Nrb3B0IHNv cHQ7CiAJc3RydWN0IGlvdmVjIGFpbzsKQEAgLTU4Nyw2ICs1ODgsNyBAQAogCWludCByZXRyeTsK IAljb25zdCBjaGFyICpzOwogCisgICAgZnJvbSA9IChzdHJ1Y3Qgc29ja2FkZHJfaW4gKikgMDsK IAl0di50dl9zZWMgPSAxOwogCXR2LnR2X3VzZWMgPSAwOwogCWJ6ZXJvKCZzb3B0LCBzaXplb2Yo c29wdCkpOwpAQCAtNzYwLDkgKzc2MiwxNCBAQAogCQlpZiAodGltbyA8IE1BWF9SRVNFTkRfREVM QVkpCiAJCQl0aW1vKys7CiAJCWVsc2UgewotCQkJcHJpbnRmKCJESENQL0JPT1RQIHRpbWVvdXQg Zm9yIHNlcnZlciAiKTsKLQkJCXByaW50X3Npbl9hZGRyKCZkc3QpOwotCQkJcHJpbnRmKCJcbiIp OworICAgICAgICAgICAgaWYgKGZyb20gIT0gKHN0cnVjdCBzb2NrYWRkcl9pbiAqKSAwKSB7Cisg ICAgICAgICAgICAgICAgcHJpbnRmKCJESENQL0JPT1RQIHRpbWVvdXQgZm9yIHNlcnZlciAiKTsK KyAgICAgICAgICAgICAgICBwcmludF9zaW5fYWRkcihmcm9tKTsKKyAgICAgICAgICAgICAgICBw cmludGYoIlxuIik7CisgICAgICAgICAgICB9CisgICAgICAgICAgICBlbHNlIHsKKyAgICAgICAg ICAgICAgICBwcmludGYgKCJESENQL0JPT1RQIHRpbWVvdXRcbiIpOworICAgICAgICAgICAgfQog CQl9CiAKIAkJLyoKQEAgLTc4Myw3ICs3OTAsMTEgQEAKIAkJCWF1aW8udWlvX3RkID0gdGQ7CiAK IAkJCXJjdmZsZyA9IDA7Ci0JCQllcnJvciA9IHNvcmVjZWl2ZShib290cF9zbywgTlVMTCwgJmF1 aW8sCisgICAgICAgICAgICBpZiAoZnJvbSAhPSAoc3RydWN0IHNvY2thZGRyX2luICopIDApIHsK KyAgICAgICAgICAgICAgICBmcmVlIChmcm9tLCBNX1NPTkFNRSk7CisgICAgICAgICAgICAgICAg ZnJvbSA9IChzdHJ1Y3Qgc29ja2FkZHJfaW4gKikgMDsKKyAgICAgICAgICAgIH0KKwkJCWVycm9y ID0gc29yZWNlaXZlKGJvb3RwX3NvLCAoc3RydWN0IHNvY2thZGRyICoqKSAmZnJvbSwgJmF1aW8s CiAJCQkJCSAgTlVMTCwgTlVMTCwgJnJjdmZsZyk7CiAJCQlnY3R4LT5zZWNzID0gdGltZV9zZWNv bmQgLSBnY3R4LT5zdGFydHRpbWU7CiAJCQlTVEFJTFFfRk9SRUFDSChpZmN0eCwgJmdjdHgtPmlu dGVyZmFjZXMsIG5leHQpIHsKQEAgLTg1MCw3ICs4NjEsNyBAQAogCQkJCSAgICAgICAiIG9uICVz IGZyb20gIiwKIAkJCQkgICAgICAgcywKIAkJCQkgICAgICAgaWZjdHgtPmlyZXEuaWZyX25hbWUp OwotCQkJCXByaW50X2luX2FkZHIoZ2N0eC0+cmVwbHkuc2lhZGRyKTsKKwkJCQlwcmludF9zaW5f YWRkcihmcm9tKTsKIAkJCQlpZiAoZ2N0eC0+cmVwbHkuZ2lhZGRyLnNfYWRkciAhPQogCQkJCSAg ICBodG9ubChJTkFERFJfQU5ZKSkgewogCQkJCQlwcmludGYoIiB2aWEgIik7CkBAIC05MzMsNiAr OTQ0LDEwIEBACiAJZXJyb3IgPSBFVElNRURPVVQ7CiAKIG91dDoKKyAgICBpZiAoZnJvbSAhPSAo c3RydWN0IHNvY2thZGRyX2luICopIDApIHsKKyAgICAgICAgZnJlZSAoZnJvbSwgTV9TT05BTUUp OworICAgICAgICBmcm9tID0gKHN0cnVjdCBzb2NrYWRkcl9pbiAqKSAwOworICAgIH0KIAlyZXR1 cm4gKGVycm9yKTsKIH0KIApAQCAtMTI3Nyw2ICsxMjkyLDE1IEBACiAJCWxlYXNldGltZSA9IGh0 b25sKDMwMCk7CiAJCW1lbWNweSh2ZW5kcCwgJmxlYXNldGltZSwgNCk7CiAJCXZlbmRwICs9IDQ7 CisgICAgICAgICAgICAKKyAgICAgICAgLyoKKyAgICAgICAgICogUmVxdWVzdCBob3N0IGFuZCBw YXRoIG9mIHJvb3QgZGlzayAocmVxdWlyZWQgZm9yIFdpbmRvd3MgU2VydmVyIERIQ1ApCisgICAg ICAgICAqLworICAgICAgICAgICAgCisgICAgICAgICp2ZW5kcCsrID0gVEFHX1BBUkFNX1JFUTsK KyAgICAgICAgKnZlbmRwKysgPSAyOworICAgICAgICAqdmVuZHArKyA9IFRBR19IT1NUTkFNRTsK KyAgICAgICAgKnZlbmRwKysgPSBUQUdfUk9PVDsKIAkJYnJlYWs7CiAJZGVmYXVsdDoKIAkJYnJl YWs7Cg== --_003_CFB0A140426CBjohnthehowiescom_ Content-Type: application/octet-stream; name="bootp.c.diff" Content-Description: bootp.c.diff Content-Disposition: attachment; filename="bootp.c.diff"; size=907; creation-date="Sun, 01 Jun 2014 02:22:26 GMT"; modification-date="Sun, 01 Jun 2014 02:22:26 GMT" Content-ID: <257902279A9442499AC0B670B2BC93E1@thehowies.local> Content-Transfer-Encoding: base64 LS0tIGxpYi9saWJzdGFuZC9ib290cC5jCTIwMTQtMDQtMDQgMTI6MTg6NDAuMDAwMDAwMDAwIC0w NzAwCisrKyAvaG9tZS9qb2huL3RtcC9ib290cC5jCTIwMTQtMDUtMTEgMjA6MDA6NDQuMDAwMDAw MDAwIC0wNzAwCkBAIC0xODYsMTMgKzE4NiwyMiBAQAogCQlicC0+YnBfdmVuZFsyMF0gPSA0Owog CQlsZWFzZXRpbWUgPSBodG9ubCgzMDApOwogCQliY29weSgmbGVhc2V0aW1lLCAmYnAtPmJwX3Zl bmRbMjFdLCA0KTsKKyAgICAgICAgCisgICAgICAgIC8qCisgICAgICAgICAqIFJlcXVlc3QgaG9z dCBhbmQgcGF0aCBvZiByb290IGRpc2sgKHJlcXVpcmVkIGZvciBXaW5kb3dzIFNlcnZlciBESENQ KQorICAgICAgICAgKi8KKyAgICAgICAgCisJCWJwLT5icF92ZW5kWzI1XSA9IFRBR19QQVJBTV9S RVE7CisJCWJwLT5icF92ZW5kWzI2XSA9IDI7CisJCWJwLT5icF92ZW5kWzI3XSA9IFRBR19IT1NU TkFNRTsKKwkJYnAtPmJwX3ZlbmRbMjhdID0gVEFHX1JPT1RQQVRIOwogCQlpZiAoZmxhZyAmIEJP T1RQX1BYRSkgewotCQkJYnAtPmJwX3ZlbmRbMjVdID0gVEFHX0NMQVNTSUQ7Ci0JCQlicC0+YnBf dmVuZFsyNl0gPSA5OwotCQkJYmNvcHkoIlBYRUNsaWVudCIsICZicC0+YnBfdmVuZFsyN10sIDkp OwotCQkJYnAtPmJwX3ZlbmRbMzZdID0gVEFHX0VORDsKKwkJCWJwLT5icF92ZW5kWzI5XSA9IFRB R19DTEFTU0lEOworCQkJYnAtPmJwX3ZlbmRbMzBdID0gOTsKKwkJCWJjb3B5KCJQWEVDbGllbnQi LCAmYnAtPmJwX3ZlbmRbMzFdLCA5KTsKKwkJCWJwLT5icF92ZW5kWzQwXSA9IFRBR19FTkQ7CiAJ CX0gZWxzZQotCQkJYnAtPmJwX3ZlbmRbMjVdID0gVEFHX0VORDsKKwkJCWJwLT5icF92ZW5kWzI5 XSA9IFRBR19FTkQ7CiAKIAkJZXhwZWN0ZWRfZGhjcG1zZ3R5cGUgPSBESENQQUNLOwogCg== --_003_CFB0A140426CBjohnthehowiescom_-- From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 06:31:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E83F6EF for ; Sun, 1 Jun 2014 06:31:32 +0000 (UTC) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id 8A9A72B96 for ; Sun, 1 Jun 2014 06:31:31 +0000 (UTC) Received: (qmail 43464 invoked from network); 1 Jun 2014 06:24:48 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 1 Jun 2014 06:24:48 -0000 Date: Sun, 01 Jun 2014 08:24:48 +0200 (CEST) Message-Id: <20140601.082448.74710838.sthaug@nethelp.no> To: john@thehowies.com Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP From: sthaug@nethelp.no In-Reply-To: References: X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org, freebsd-questions@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 06:31:32 -0000 > Section 3.5 of RFC 2131 (the DHCP RFC) states that "...Second, in its > initial DHCPDISCOVER or DHCPREQUEST message, a client may provide the > server with a list of specific parameters the client is interested in" > and "...The client can inform the server which configuration parameters > the client is interested in by including the 'parameter request list' > option." The data portion of this option explicitly lists the options > requested by tag number. A DHCP Server is not required to return any > parameter that a client does not ask for. It appears that the ISC-DHCP > server, which is recommended by most, will return configured options > regardless of whether or not the client asks for them. As far as I know this is wrong. ISC DHCP does *not* behave this way. Do you have packet sniffer traces to indicate oterwise? In any case - yes, the client should absolutely request all the parameters it wants. Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 07:31:28 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD529C46; Sun, 1 Jun 2014 07:31:28 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EABE2031; Sun, 1 Jun 2014 07:31:28 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id j15so1102562qaq.9 for ; Sun, 01 Jun 2014 00:31:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sFCkWM3JXkevKQilYOeCG3Ba4+fa33fEZcD4cfQRAm0=; b=c1gKZ9bbJcyQD059+m4QpiD85/bXidInFsgEtDpy1chQpS9oKR99u3NaSt/3c1R2ba OEuQmzDk9cgeXN4vO91ZDg3F71DxvFlrvjsW0mkK2/33Cqmr2Yt24zs1o6W3HTv/hL2J lRBq6oxXSxE+sKnF7JKjlj8HmUG5hCt4vhaFaiC5hmCcPPlX4BNhDXVnC0zA4uH1z/Ty B4ulZWuureGFpYIamkPfmWmE6CkTAApD5djZMuUzGyA3j8bkrfzq0BtVpFXvzTdGbGGv hXQ38vgbdZA3b/4JOncdQY5/XjwN+D+4yG7kOOgCgC0bbd944yE51r0OXxi+nUjik+bY pujA== MIME-Version: 1.0 X-Received: by 10.224.7.6 with SMTP id b6mr38236901qab.45.1401607887661; Sun, 01 Jun 2014 00:31:27 -0700 (PDT) Received: by 10.96.122.133 with HTTP; Sun, 1 Jun 2014 00:31:27 -0700 (PDT) In-Reply-To: References: <533BF902.7010506@sfc.wide.ad.jp> Date: Sun, 1 Jun 2014 00:31:27 -0700 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: hiren panchasara To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: Midori Kato , "Eggert, Lars" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 07:31:28 -0000 On Tue, Apr 8, 2014 at 9:14 PM, hiren panchasara wrote: > On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd wrote: >> Hi! Cool! can you file a FreeBSD PR with this? > > I'm testing this patch right now. > > I will make sure it doesn't get lost. :-) Committed as r266941. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 09:08:58 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BAE3DD9; Sun, 1 Jun 2014 09:08:58 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AEF3D27AC; Sun, 1 Jun 2014 09:08:57 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id a1so3805342wgh.27 for ; Sun, 01 Jun 2014 02:08:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Xf6FKPQdQJ3m+LnBMIaIghnmm/BEoY+G6eIhjrYvVNU=; b=QzppuXibGQphb8YK9T7L6xogS6Se0AI1xh6U9qdy96TWXX/GweC7Jz62JAm8+VCcEF oeZOnnjTV1qJcKDbqVtzsEnRvCsMHmDvMUMUej8qENwfzfSkg97uqZftIBLhBdRAQyPC Ek7NYOacebY9CYw4std7HfJ3SCMC2sM6mwrf5cIZ5Xp0abiQAjFOzKI5zi3mNCDb5NcT 1WpdMUiXp40ytozRnxCi9c4NtxWZZOU7f/acRDl7YtogfMuWwrx6cERMMyJRWSsIJdDW lcHpdeTuWuoUHA3dlwj/Kp5cueio2113tTkJq8HoYDArrl/i/Wcod0UTLrsZQAohx8M+ l61Q== MIME-Version: 1.0 X-Received: by 10.180.221.163 with SMTP id qf3mr12956064wic.56.1401613735685; Sun, 01 Jun 2014 02:08:55 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Sun, 1 Jun 2014 02:08:55 -0700 (PDT) In-Reply-To: References: <533BF902.7010506@sfc.wide.ad.jp> Date: Sun, 1 Jun 2014 11:08:55 +0200 X-Google-Sender-Auth: t_P9h4CMSV2952lAGMzOIp3ZzM8 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: Luigi Rizzo To: hiren panchasara Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Adrian Chadd , Midori Kato , "Eggert, Lars" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 09:08:58 -0000 On Sunday, June 1, 2014, hiren panchasara wrote: > On Tue, Apr 8, 2014 at 9:14 PM, hiren panchasara > > wrote: > > On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd > wrote: > >> Hi! Cool! can you file a FreeBSD PR with this? > > > > I'm testing this patch right now. > > > > I will make sure it doesn't get lost. :-) > > Committed as r266941. > > I don't think we need the DNOLD_IS_ECN flag and translation. That stuff is meant for old binaries Cheers Luigi > cheers, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 10:22:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A210663E; Sun, 1 Jun 2014 10:22:18 +0000 (UTC) Received: from remote.thehowies.com (50-197-91-217-static.hfc.comcastbusiness.net [50.197.91.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "remote.thehowies.com", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C0122D08; Sun, 1 Jun 2014 10:22:17 +0000 (UTC) Received: from PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820]) by PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820%11]) with mapi id 14.01.0438.000; Sun, 1 Jun 2014 03:22:15 -0700 From: John Howie To: "sthaug@nethelp.no" Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Topic: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Index: AQHPfUBUW2MQdiTIGUuysOzuLv/2xptcPzcAgAC3qoA= Date: Sun, 1 Jun 2014 10:22:15 +0000 Message-ID: References: <20140601.082448.74710838.sthaug@nethelp.no> In-Reply-To: <20140601.082448.74710838.sthaug@nethelp.no> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.2.140509 x-originating-ip: [203.147.52.194] Content-Type: text/plain; charset="us-ascii" Content-ID: <4894C64CE3DA02458F33AE8BB40F8A9F@thehowies.local> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" , "freebsd-arm@freebsd.org" , "freebsd-questions@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 10:22:18 -0000 Hi Steinar, In short, no, I have no packet traces. Given that the DHCP code in the FreeBSD boot loader and NFS subsystem does not request those options, but that ISC-DHCP does provide them, I will go out on a limb and say that it must be serving them without being asked if they are configured. Regards, John On 6/1/14, 1:24 PM, "sthaug@nethelp.no" wrote: >> Section 3.5 of RFC 2131 (the DHCP RFC) states that "...Second, in its >> initial DHCPDISCOVER or DHCPREQUEST message, a client may provide the >> server with a list of specific parameters the client is interested in" >> and "...The client can inform the server which configuration parameters >> the client is interested in by including the 'parameter request list' >> option." The data portion of this option explicitly lists the options >> requested by tag number. A DHCP Server is not required to return any >> parameter that a client does not ask for. It appears that the ISC-DHCP >> server, which is recommended by most, will return configured options >> regardless of whether or not the client asks for them. > >As far as I know this is wrong. ISC DHCP does *not* behave this way. >Do you have packet sniffer traces to indicate oterwise? > >In any case - yes, the client should absolutely request all the >parameters it wants. > >Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 12:01:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4F9BDB7; Sun, 1 Jun 2014 12:01:45 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 52BD92650; Sun, 1 Jun 2014 12:01:45 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqkEAHUEi1ODaFve/2dsb2JhbABYgyYzWIJsuCyGaFEBgR50giUBAQEDAQEBASArIAsFFhgRGQIpAQkYAQ0GCAcEARwCAogZCA2xLaQxF41nCRACARs0B4I0DzISgTkElxyEIpFvg1QhNIE+ X-IronPort-AV: E=Sophos;i="4.98,951,1392181200"; d="scan'208";a="125510873" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 01 Jun 2014 08:01:42 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E12BEB41A3; Sun, 1 Jun 2014 08:01:42 -0400 (EDT) Date: Sun, 1 Jun 2014 08:01:42 -0400 (EDT) From: Rick Macklem To: John Howie Message-ID: <1468927196.9638531.1401624102908.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP MIME-Version: 1.0 X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org, freebsd-questions@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 12:01:45 -0000 John Howie wrote: > Hi all, >=20 > I apologize for the cross posting of this email, but I believe it > will be > of interest to people across all three groups. Please feel free to > forward > to additional groups if you feel they would benefit. >=20 > I have seen a few posts on and off over the years about Windows > Server > DHCP not working with FreeBSD. Specifically, the Windows Server DHCP > would > not return the boot host name/IP address and the root path. The > typical > response to =C2=B3why won=C2=B9t it work" has typically been that there i= s a > flaw in > Windows Server DHCP code. I did a little digging and found that the > problem actually lies in code in FreeBSD. >=20 > Section 3.5 of RFC 2131 (the DHCP RFC) states that =C2=B3...Second, in it= s > initial DHCPDISCOVER or DHCPREQUEST message, a client may provide the > server with a list of specific parameters the client is interested > in=C5=A0=C2=B2 > and =C2=B3...The client can inform the server which configuration > parameters > the client is interested in by including the 'parameter request list' > option. The data portion of this option explicitly lists the options > requested by tag number=C5=A0=C2=B2. A DHCP Server is not required to ret= urn > any > parameter that a client does not ask for. It appears that the > ISC-DHCP > server, which is recommended by most, will return configured options > regardless of whether or not the client asks for them. >=20 > There are two places in the FreeBSD codebase that DHCP is used to > boot the > system over a network. The first is in the boot loader, which uses > code in > lib/libstand. The second is in the NFS code, in sys/nfs. The code is > sys/nfs is not always used if the boot loader sets up the environment > for > the NFS code, either by passing parameters to the kernel (as PXEBOOT > appears to do), or information is configured in the boot loader > configuration files, e.g. /boot/loader.rc. >=20 > I have attached two unified diff files which I ask people to test, > before > I submit them for inclusion into the codebase as fixes. The first, > bootp.c.diff fixes the code in lib/libstand/bootp.c to request the > boot > host (option 12, aka TAG_HOSTNAME) and the NFS root path (option 17, > aka > TAG_ROOTPATH). This fix has been tested with PXEBOOT on an amd64 box > and > ubldr on an ARM/RaspberryPI system. The second, bootp_subr.c.diff, > fixes > code in sys/nfs/bootp_subr.c to request the same options and also to > fix > bugs that erroneously reported the IP address of the BOOTP/DHCP > server. > The code assumed that the BOOTP/DHCP server was also the boot host. > Please > send me all feedback directly. >=20 > The diff files work with 10.0-RELEASE through 10.0-RELEASE-p3, but > will > likely work with 9.0 and also CURRENT and STABLE, including 11.0, as > the > code is old code that does not appear to have changed in a while. If > you > want to try it on those systems please, please make sure you have > backup > copies just in case. >=20 > If you do not have experience configuring Windows Server DHCP just > drop me > an email, and I will send you a cheat sheet to get you up and > running. >=20 > I am going to grab the latest ubldr code to see if I can get it to > work > more like PXEBOOT, that appears to pass parameters to the kernel to > avoid > the need for the NFS BOOTP/DHCP process. If you test on an ARM system > with > ubldr in RELEASE you will see a lot of unnecessary network activity > going > on, that we should be able to fix. >=20 Actually, I tend to think that using the code in sys/nfs/bootp_subr.c is preferable to using the NFS stuff in stand that pxeboot does. The only reason I know for pxeboot doing the NFS stuff and filling in the nfsv3_diskless structure is historical. (Or that's what most people use for x86, so it would be a POLA violation if it breaks, if you prefer.) (Basically, the code in sys/nfs/bootp_subr.c is easier to maintain than the stand alone NFS client pxeboot uses.) As such, I don't think this work is necessary, rick > Regards, >=20 > John >=20 > john@thehowies.com (personal) | jhowie@email.arizona.edu (academic) | > j.howie@napier.ac.uk (academic) | jhowie@cloudsecurityalliance.org > (work) >=20 >=20 >=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 12:23:17 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22DE545C for ; Sun, 1 Jun 2014 12:23:17 +0000 (UTC) Received: from mail-vc0-x233.google.com (mail-vc0-x233.google.com [IPv6:2607:f8b0:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA247281F for ; Sun, 1 Jun 2014 12:23:16 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id im17so3945077vcb.24 for ; Sun, 01 Jun 2014 05:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5z3NZ89AtLMlY6fOxOw91aKDtNo+c3jOHWgbKbuLdTg=; b=SMZQ8YzeXMrhnntFgIKEJhy0x4/4DCz5OifYk6Un8A5fow/gO78uUwBTYmFExcnwVP xA1ZOGmkt80PFhn7YnoU8ysDX1DtE1tBw54GZB/fMco0G7YenVyk49Wvu2aheRJEKIvH fh0wsMfYyMLEXLfnLjmLexhEHM9PeOWDrs3i0YsDCKYI8aRNAPGHLlKaA0QDfycA8Yva app5zAA8sFL2TOfQFBcKHU8WRD9bfg/Ak96ugAS9v9U4aM5f0qVtqyu/km6OqLwOI9KT v3WBxhK8t2G6ShHC+On0zHho57e8HTu9N/8msZoF93Ethdo3MLd/fDyNOjBebe5/V8RK bAcQ== MIME-Version: 1.0 X-Received: by 10.220.249.198 with SMTP id ml6mr55235vcb.36.1401625395120; Sun, 01 Jun 2014 05:23:15 -0700 (PDT) Received: by 10.58.67.227 with HTTP; Sun, 1 Jun 2014 05:23:15 -0700 (PDT) Date: Sun, 1 Jun 2014 16:53:15 +0430 Message-ID: Subject: Hint about netmap From: Mohammad Badie Zadegan To: net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 12:23:17 -0000 Hi Everybody, I have a question about netmap(the fast packet I/O framework) I have released my own firewall and now I want to test it at the maximum packet/s range that available but still I can not select netmap or pktgen! Does netmap really is faster than pktgen? What is the real difference between netmap vs pktgen? Thanks. -- Mohammad BadieZadegan Penetration Test Manager Payampardaz Security Team =C2=AE Cell# +98 (913) 325 0665 www.payampardaz.net From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 12:30:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D5B05F4 for ; Sun, 1 Jun 2014 12:30:37 +0000 (UTC) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id 3FFBA2863 for ; Sun, 1 Jun 2014 12:30:35 +0000 (UTC) Received: (qmail 47967 invoked from network); 1 Jun 2014 12:30:33 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 1 Jun 2014 12:30:33 -0000 Date: Sun, 01 Jun 2014 14:30:33 +0200 (CEST) Message-Id: <20140601.143033.41674928.sthaug@nethelp.no> To: john@thehowies.com Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP From: sthaug@nethelp.no In-Reply-To: References: <20140601.082448.74710838.sthaug@nethelp.no> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org, freebsd-questions@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 12:30:37 -0000 > In short, no, I have no packet traces. Given that the DHCP code in the > FreeBSD boot loader and NFS subsystem does not request those options, but > that ISC-DHCP does provide them, I will go out on a limb and say that it > must be serving them without being asked if they are configured. In that case I'm afraid I must stand by my claim that you're wrong and ISC DHCP does *not* provide configured options unless the client asks for them. (And I have copious amounts of packet sniffer traces to prove this.) Not that this is particularly relevant to FreeBSD any more... Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 13:01:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8FD42CA; Sun, 1 Jun 2014 13:01:51 +0000 (UTC) Received: from remote.thehowies.com (50-197-91-217-static.hfc.comcastbusiness.net [50.197.91.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "remote.thehowies.com", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82A4C2B0E; Sun, 1 Jun 2014 13:01:50 +0000 (UTC) Received: from PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820]) by PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820%11]) with mapi id 14.01.0438.000; Sun, 1 Jun 2014 06:01:48 -0700 From: John Howie To: "sthaug@nethelp.no" Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Topic: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Index: AQHPfUBUW2MQdiTIGUuysOzuLv/2xptcPzcAgAC3qoD//66HgP//k2Lc Date: Sun, 1 Jun 2014 13:01:48 +0000 Message-ID: <0FF4F173-8D6A-4664-AA31-FB1BF28A7E74@thehowies.com> References: <20140601.082448.74710838.sthaug@nethelp.no> , <20140601.143033.41674928.sthaug@nethelp.no> In-Reply-To: <20140601.143033.41674928.sthaug@nethelp.no> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" , "freebsd-arm@freebsd.org" , "freebsd-questions@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 13:01:51 -0000 Hi Steinar, I could ask you to 'prove it', too, but I can easily check when I get back = from my current travels :-) It important to note that even if it does (as I think it does) it is NOT in= violation of the RFC. The RFC simply says that if a client wants something= it should ask for it, and not that a server cannot send the options unsoli= cited. Best regards, John Sent from my iPhone On Jun 1, 2014, at 19:30, "sthaug@nethelp.no" wrote: >> In short, no, I have no packet traces. Given that the DHCP code in the >> FreeBSD boot loader and NFS subsystem does not request those options, bu= t >> that ISC-DHCP does provide them, I will go out on a limb and say that it >> must be serving them without being asked if they are configured. >=20 > In that case I'm afraid I must stand by my claim that you're wrong > and ISC DHCP does *not* provide configured options unless the client > asks for them. >=20 > (And I have copious amounts of packet sniffer traces to prove this.) >=20 > Not that this is particularly relevant to FreeBSD any more... >=20 > Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 13:15:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB744905; Sun, 1 Jun 2014 13:15:57 +0000 (UTC) Received: from remote.thehowies.com (50-197-91-217-static.hfc.comcastbusiness.net [50.197.91.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "remote.thehowies.com", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C1552BED; Sun, 1 Jun 2014 13:15:56 +0000 (UTC) Received: from PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820]) by PRIMARY.thehowies.local ([fe80::967:7eb6:ee49:3820%11]) with mapi id 14.01.0438.000; Sun, 1 Jun 2014 06:15:56 -0700 From: John Howie To: Rick Macklem Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Topic: Patches for BOOTP/DHCP code to support Windows Server DHCP Thread-Index: AQHPfUBUW2MQdiTIGUuysOzuLv/2xptcnVgA//+fZCc= Date: Sun, 1 Jun 2014 13:15:55 +0000 Message-ID: References: , <1468927196.9638531.1401624102908.JavaMail.root@uoguelph.ca> In-Reply-To: <1468927196.9638531.1401624102908.JavaMail.root@uoguelph.ca> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" , "freebsd-arm@freebsd.org" , "freebsd-questions@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 13:15:58 -0000 Hi Rick, That is an excellent point and a good debate to have. I have not looked in detail at how PXEBOOT does it, but I think a clean up = of the code to somehow pass arguments to the kernel is preferable to having= a diskless client send a slew of needless requests to the DHCP server to r= equest information already requested and obtained in previous stages of the= boot process. The number of DHCP requests made by a client when using U-Bo= ot and ubldr is dizzying. The NFS code will look to see if the boot loader = configuration file has specified the root filesystem through use of vfs.roo= t.mountfrom, so something should be possible. Another area for possible attention is handling the scenario when there are= a number of DHCP servers able to reply to requests. This can happen when y= ou have multiple NICs on segments with DHCP Servers or where you have multi= ple servers on the same segment. The current code just grabs the first serv= er to reply at each stage (going through each NIC in turn) and has affinity= to it for the remainder of that stage but not through the entire boot proc= ess. Regards, John Sent from my iPhone > On Jun 1, 2014, at 19:01, "Rick Macklem" wrote: >=20 > John Howie wrote: >> Hi all, >>=20 >> I apologize for the cross posting of this email, but I believe it >> will be >> of interest to people across all three groups. Please feel free to >> forward >> to additional groups if you feel they would benefit. >>=20 >> I have seen a few posts on and off over the years about Windows >> Server >> DHCP not working with FreeBSD. Specifically, the Windows Server DHCP >> would >> not return the boot host name/IP address and the root path. The >> typical >> response to =B3why won=B9t it work" has typically been that there is a >> flaw in >> Windows Server DHCP code. I did a little digging and found that the >> problem actually lies in code in FreeBSD. >>=20 >> Section 3.5 of RFC 2131 (the DHCP RFC) states that =B3...Second, in its >> initial DHCPDISCOVER or DHCPREQUEST message, a client may provide the >> server with a list of specific parameters the client is interested >> in=8A=B2 >> and =B3...The client can inform the server which configuration >> parameters >> the client is interested in by including the 'parameter request list' >> option. The data portion of this option explicitly lists the options >> requested by tag number=8A=B2. A DHCP Server is not required to return >> any >> parameter that a client does not ask for. It appears that the >> ISC-DHCP >> server, which is recommended by most, will return configured options >> regardless of whether or not the client asks for them. >>=20 >> There are two places in the FreeBSD codebase that DHCP is used to >> boot the >> system over a network. The first is in the boot loader, which uses >> code in >> lib/libstand. The second is in the NFS code, in sys/nfs. The code is >> sys/nfs is not always used if the boot loader sets up the environment >> for >> the NFS code, either by passing parameters to the kernel (as PXEBOOT >> appears to do), or information is configured in the boot loader >> configuration files, e.g. /boot/loader.rc. >>=20 >> I have attached two unified diff files which I ask people to test, >> before >> I submit them for inclusion into the codebase as fixes. The first, >> bootp.c.diff fixes the code in lib/libstand/bootp.c to request the >> boot >> host (option 12, aka TAG_HOSTNAME) and the NFS root path (option 17, >> aka >> TAG_ROOTPATH). This fix has been tested with PXEBOOT on an amd64 box >> and >> ubldr on an ARM/RaspberryPI system. The second, bootp_subr.c.diff, >> fixes >> code in sys/nfs/bootp_subr.c to request the same options and also to >> fix >> bugs that erroneously reported the IP address of the BOOTP/DHCP >> server. >> The code assumed that the BOOTP/DHCP server was also the boot host. >> Please >> send me all feedback directly. >>=20 >> The diff files work with 10.0-RELEASE through 10.0-RELEASE-p3, but >> will >> likely work with 9.0 and also CURRENT and STABLE, including 11.0, as >> the >> code is old code that does not appear to have changed in a while. If >> you >> want to try it on those systems please, please make sure you have >> backup >> copies just in case. >>=20 >> If you do not have experience configuring Windows Server DHCP just >> drop me >> an email, and I will send you a cheat sheet to get you up and >> running. >>=20 >> I am going to grab the latest ubldr code to see if I can get it to >> work >> more like PXEBOOT, that appears to pass parameters to the kernel to >> avoid >> the need for the NFS BOOTP/DHCP process. If you test on an ARM system >> with >> ubldr in RELEASE you will see a lot of unnecessary network activity >> going >> on, that we should be able to fix. > Actually, I tend to think that using the code in sys/nfs/bootp_subr.c > is preferable to using the NFS stuff in stand that pxeboot does. >=20 > The only reason I know for pxeboot doing the NFS stuff and filling in > the nfsv3_diskless structure is historical. (Or that's what most people > use for x86, so it would be a POLA violation if it breaks, if you prefer.= ) > (Basically, the code in sys/nfs/bootp_subr.c is easier to maintain than t= he > stand alone NFS client pxeboot uses.) >=20 > As such, I don't think this work is necessary, rick >=20 >> Regards, >>=20 >> John >>=20 >> john@thehowies.com (personal) | jhowie@email.arizona.edu (academic) | >> j.howie@napier.ac.uk (academic) | jhowie@cloudsecurityalliance.org >> (work) >>=20 >>=20 >>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 13:30:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1D35E02; Sun, 1 Jun 2014 13:30:18 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7416E2CCB; Sun, 1 Jun 2014 13:30:17 +0000 (UTC) Received: from jre-mbp.elischer.org (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s51DU6tW078105 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 1 Jun 2014 06:30:08 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <538B2AD8.5050608@freebsd.org> Date: Sun, 01 Jun 2014 21:30:00 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Rick Macklem , John Howie Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP References: <1468927196.9638531.1401624102908.JavaMail.root@uoguelph.ca> In-Reply-To: <1468927196.9638531.1401624102908.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org, freebsd-questions@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 13:30:18 -0000 On 6/1/14, 8:01 PM, Rick Macklem wrote: > John Howie wrote: >> [...] > Actually, I tend to think that using the code in sys/nfs/bootp_subr.c > is preferable to using the NFS stuff in stand that pxeboot does. > > The only reason I know for pxeboot doing the NFS stuff and filling in > the nfsv3_diskless structure is historical. (Or that's what most people > use for x86, so it would be a POLA violation if it breaks, if you prefer.) > (Basically, the code in sys/nfs/bootp_subr.c is easier to maintain than the > stand alone NFS client pxeboot uses.) > > As such, I don't think this work is necessary, rick it's probably worth having both options John. great work BTW. > >> Regards, >> >> John >> >> john@thehowies.com (personal) | jhowie@email.arizona.edu (academic) | >> j.howie@napier.ac.uk (academic) | jhowie@cloudsecurityalliance.org >> (work) >> >> >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 13:40:50 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C9F0292 for ; Sun, 1 Jun 2014 13:40:50 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF4282E16 for ; Sun, 1 Jun 2014 13:40:49 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id ik5so3768560vcb.5 for ; Sun, 01 Jun 2014 06:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=T5okD8IsPrWXLB25aQPYqy25XgMgvgs643YlLPtUDHQ=; b=Y0v3de6KqxfG7o2xg/vIUNZ9kOtOR3QVhj3tm4CBNr1y4eE65KBFSjt63DbBdSBeb3 Zy+yZDBOZZC/Z/evOlZ2MkkQIvMuG9fOD5W2/TECteCwvdjniIA9qtyFSCHY+J0ReeHH g10GTWju3dIoAoCNymSm0/3n/aKuEHYEG1FlJGpWfclXBwXbHQC/5tXaw2a5NDKmjMlV oUumXPgmJq2ZsE94VX8WVfc96UqdBjxb78OtpC6bzysRkV77dziDc2g4kwXKvWzUcnUh i53Z8o1wVa1our/SccSopBI6b/b6o9LbymPy4IMIgoFuan/xEJffJcCHYJx5fLE3PD3U ODMw== X-Received: by 10.58.219.166 with SMTP id pp6mr25512569vec.1.1401630048697; Sun, 01 Jun 2014 06:40:48 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.4.131 with HTTP; Sun, 1 Jun 2014 06:40:28 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Sun, 1 Jun 2014 15:40:28 +0200 X-Google-Sender-Auth: 441WnaOhkDHQFTqPoW3LDD88A0w Message-ID: Subject: Re: Hint about netmap To: Mohammad Badie Zadegan Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 13:40:50 -0000 On Sun, Jun 1, 2014 at 2:23 PM, Mohammad Badie Zadegan wrote: > Hi Everybody, > > I have a question about netmap(the fast packet I/O framework) > I have released my own firewall and now I want to test it at the maximum > packet/s range that available but still I can not select netmap or pktgen! > > Does netmap really is faster than pktgen? > What is the real difference between netmap vs pktgen? > Hi, netmap is "just" a packet input/output framework (like intel DPDK libraries) : Software that receive (like tcpdump) or generate (like iperf) high rate of packet need to be adapted to use the netmap framework. pktgen is a small example of packet generator/receiver using the netmap framework. For obtaining a fast firewall with netmap: You need to adapt a firewall software to use netmap. The first example of such firewall is netmap-ipfw ( https://code.google.com/p/netmap-ipfw/), but as a userland firewall separated from the host IP stack, it can be used as a "bridge-firewall" only and not a classical "router-firewall". I don't know the status of the current work of adapting a full IP routing stack to netmap, but it should be not easy: Some company like 6wind ( http://www.6wind.com/) seems dedicated to this task (in their case with Intel DPDK). Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 13:53:11 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 193874C4; Sun, 1 Jun 2014 13:53:11 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 41EAC2ED6; Sun, 1 Jun 2014 13:53:10 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wr2HR-000LsZ-AB; Sun, 01 Jun 2014 13:42:17 +0400 Message-ID: <538B2FE5.6070407@FreeBSD.org> Date: Sun, 01 Jun 2014 17:51:33 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> In-Reply-To: <20140522163812.GA77634@onelab2.iet.unipi.it> Content-Type: multipart/mixed; boundary="------------090406000404070502000405" Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 13:53:11 -0000 This is a multi-part message in MIME format. --------------090406000404070502000405 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 22.05.2014 20:38, Luigi Rizzo wrote: Long story short, new version is ready. I've tried to minimize changes in this patch to ease review/commit. Changes: * Add namedobject set-aware api capable of searching/allocation objects by their name/idx. * Switch tables code to use string ids for configuration tasks. * Change locking model: most configuration changes are protected with UH lock, runtime-visible are protected with both locks. * Reduce number of arguments passed to ipfw_table_add/del by using separate structure. * Add internal V_fw_tables_sets tunable (set to 0) to prepare for set-aware tables (requires opcodes/client support) * Implement typed table referencing (and tables are implicitly allocated with all state like radix ptrs on reference) * Add "destroy" ipfw(8) using new IP_FW_DELOBJ opcode Namedobj more detailed: * Blackbox api providing methods to add/del/search/enumerate objects * Statically-sized hashes for names/indexes * Per-set bitmask to indicate free indexes * Separate methods for index alloc/delete/resize Basically, there should not be any user-visible changes except the following: * reducing table_max is not supported * flush & add change table type won't work if table is referenced I haven't removed any numbering restrictions to protect the following case: one (with old client) unintentionally references too many tables (e.g. 1000-1128), tries to allocate table from "valid" range and fails. Old client does not have any ability to destroy any table, so the only way to solve this is either module unload or reboot. I've uploaded the same patch to phabricator since it provides quite handy diffs: https://phabric.freebsd.org/D139 (no login required). > On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. Chernikov wrote: >> On 22.05.2014 19:47, Luigi Rizzo wrote: >>> On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov wrote: >>>> On 22.05.2014 00:48, Luigi Rizzo wrote: >>>>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: >>> ... >>>>> we can solve this by using 'low' numbers for the numeric tables >>>>> (these were limited anyways) and allocate the fake entries in >>>>> another range. >>>> Currently we have u16 space available in base opcode. >>> yes but the standard range for tables is much more limited: >>> >>> net.inet.ip.fw.tables_max: 128 >>> >>> so one can just (say) use 32k for "old" tables and the rest >>> for tables with non numeric names. >>> Does not seem to be a problem in practice. >> Well, using upper 32k means that you set this default to 65k which >> consumes 256k of memory on 32-bit arch. >> Embedded people won't be very happy about this (and changing table >> numbers on resize would be a nightmare). > no no, this is an implementation detail but > within the kernel you can just remap the 'old' and 'new' > table identifiers to a single contiguous range. > The only thing you need to do is that when you push > identifiers up to userland, those with 'new' names will > be mapped to the 32-64k range. > > Example: > user first specifies tables > "18, goodguys, 530, badguys" in the same rule > /sbin/ipfw will generate these numbers: > 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} > The kernel will then do a lookup of those identifiers and > 18: internal index 1, name "18" > 32768: internal index 2, name "goodguys" > 530: internal index 3, name "530" > 32769: internal index 4, name "badguys" > > Then the next rule contains tables > 1, badguys, 18 > /sbin/ipfw generates > 1, 32768, 18 ; tlv {32768:badguys} // note different from before > Kernel looks up the names and remaps > 1: internal index 5, name "1" > 32768: internal index 4, name "badguys" > 18: internal index 1, name "18" > > Finally when you do an 'ipfw show' the kernel will remap names > between 1 and 32768 to themselves, and other names to 32768+ > (or some other large number, say 40k and above) so > as they are found. So the rules will be pushed up with > 18, 40000, 530, 40001 > 1, 40001, 18 > > we can discusso the other details privately > > cheers > luigi > > > 1. first, the >>>>> maybe i am missing some detail but it seems reasonably easy to implement >>>>> the atomic swap -- and the use case is when you want to move from >>>>> one configuration to a new one: >>>>> ipfw table foo-new flush // clear initial content >>>>> ipfw table foo-new add ... >>>>> ipfw table swap foo-current foo-new // swap the content of the table objects >>>>> >>>>> so you preserve the semantic of the name very easily. >>>> Yes. We can easily add atomic table swap that way. However, I'm talking >>>> about different use scenario: >>>> Atomically swap entire ruleset which has some tables depency: >>>> >>>> >>>> e.g. we have: >>>> >>>> " >>>> 100 allow ip from table(TABLE1) to me >>>> 200 allow ip from table(TABLE2) to (TABLE3) 80 >>>> >>>> table TABLE1 1.1.1.1/32 >>>> table TABLE1 1.0.0.0/16 >>>> >>>> table TABLE2 2.2.2.2/32 >>>> >>>> table TABLE3 3.3.3.3/32 >>>> " >>>> and we want to _atomically_ change this to >>>> >>>> " >>>> 100 allow ip from table(TABLE1) to me >>>> +200 allow ip from table(TABLE4) to any >>>> 300 allow ip from table(TABLE2) to (TABLE3) 80 >>>> >>>> table TABLE1 1.1.1.1/32 >>>> -table TABLE1 1.0.0.0/16 >>>> >>>> -table TABLE2 2.2.2.2/32 >>>> +table TABLE2 77.77.77.0/24 >>>> >>>> table TABLE3 3.3.3.3/32 >>>> >>>> +table TABLE4 4.4.4.4/32 >>>> " >>> aargh, that's too much -- because between changing >>> one table and all tables there are infinite intermediate >>> points that all make sense. >> It depends. As I said before, we're currently solving this problem by >> adding new rules (to set X) referencing tables from different range >> (2048 tables per ruleset) and than doing swap. >> (And not being able to use named tables to store real names after >> implementing them is a bit discouraging). >> >>> For those cases i think the way to go could be to >>> insert a 'disabled' new ruleset (however complex it is, >>> so it covers all possible cases), and then do the set swap, >>> or disable/enable. >> We can think of per-set arrays/namespaces of tables: >> >> so "ipfw add 100 set X allow ipfw from table(Y) to ..." will reference >> table Y in set X and >> "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". >> >> This behavior can break some users setups so we can provide >> sysctl/tunable to turn this off or on. >> >>> cheers >>> luigi >>> --------------090406000404070502000405 Content-Type: text/x-patch; name="ipfw_ntables4.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ipfw_ntables4.diff" Index: sys/netinet/ip_fw.h =================================================================== --- sys/netinet/ip_fw.h (revision 266310) +++ sys/netinet/ip_fw.h (working copy) @@ -37,6 +37,11 @@ #define IPFW_DEFAULT_RULE 65535 /* + * Number of sets supported by ipfw + */ +#define IPFW_MAX_SETS 32 + +/* * Default number of ipfw tables. */ #define IPFW_TABLES_MAX 65535 @@ -74,6 +79,7 @@ typedef struct _ip_fw3_opheader { #define IP_FW_TABLE_XDEL 87 /* delete entry */ #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ #define IP_FW_TABLE_XLIST 89 /* list table contents */ +#define IP_FW_OBJ_DEL 90 /* del table/pipe/etc */ /* * The kernel representation of ipfw rules is made of a list of @@ -632,7 +638,7 @@ typedef struct _ipfw_table { } ipfw_table; typedef struct _ipfw_xtable { - ip_fw3_opheader opheader; /* eXtended tables are controlled via IP_FW3 */ + ip_fw3_opheader opheader; /* IP_FW3 opcode */ uint32_t size; /* size of entries in bytes */ uint32_t cnt; /* # of entries */ uint16_t tbl; /* table number */ @@ -640,4 +646,26 @@ typedef struct _ipfw_xtable { ipfw_table_xentry xent[0]; /* entries */ } ipfw_xtable; +typedef struct _ipfw_xtable_tlv { + uint16_t type; /* TLV type */ + uint16_t length; /* Total length, aligned to u32 */ +} ipfw_xtable_tlv; + +#define IPFW_TLV_NAME 1 +/* Object name TLV */ +typedef struct _ipfw_xtable_ntlv { + ipfw_xtable_tlv head; /* TLV header */ + uint16_t idx; /* Name index */ + uint16_t spare; /* unused */ + char name[64]; /* Null-terminated name */ +} ipfw_xtable_ntlv; + +typedef struct _ipfw_obj_header { + ip_fw3_opheader opheader; /* IP_FW3 opcode */ + uint32_t set; /* Set we're operating */ + uint16_t idx; /* object name index */ + uint16_t objtype; /* object type */ +} ipfw_obj_header; +#define IPFW_OBJTYPE_TABLE 1 + #endif /* _IPFW2_H */ Index: sys/netpfil/ipfw/ip_fw2.c =================================================================== --- sys/netpfil/ipfw/ip_fw2.c (revision 266306) +++ sys/netpfil/ipfw/ip_fw2.c (working copy) @@ -121,6 +121,7 @@ VNET_DEFINE(int, autoinc_step); VNET_DEFINE(int, fw_one_pass) = 1; VNET_DEFINE(unsigned int, fw_tables_max); +VNET_DEFINE(unsigned int, fw_tables_sets) = 0; /* Don't use set-aware tables */ /* Use 128 tables by default */ static unsigned int default_fw_tables = IPFW_TABLES_DEFAULT; @@ -2719,7 +2720,6 @@ vnet_ipfw_uninit(const void *unused) ipfw_dyn_uninit(0); /* run the callout_drain */ IPFW_WUNLOCK(chain); - ipfw_destroy_tables(chain); reap = NULL; IPFW_WLOCK(chain); for (i = 0; i < chain->n_rules; i++) { @@ -2731,6 +2731,7 @@ vnet_ipfw_uninit(const void *unused) free(chain->map, M_IPFW); IPFW_WUNLOCK(chain); IPFW_UH_WUNLOCK(chain); + ipfw_destroy_tables(chain); if (reap != NULL) ipfw_reap_rules(reap); IPFW_LOCK_DESTROY(chain); Index: sys/netpfil/ipfw/ip_fw_private.h =================================================================== --- sys/netpfil/ipfw/ip_fw_private.h (revision 266306) +++ sys/netpfil/ipfw/ip_fw_private.h (working copy) @@ -212,6 +212,11 @@ VNET_DECLARE(int, autoinc_step); VNET_DECLARE(unsigned int, fw_tables_max); #define V_fw_tables_max VNET(fw_tables_max) +VNET_DECLARE(unsigned int, fw_tables_sets); +#define V_fw_tables_sets VNET(fw_tables_sets) + +struct tables_config; + struct ip_fw_chain { struct ip_fw **map; /* array of rule ptrs to ease lookup */ uint32_t id; /* ruleset id */ @@ -219,7 +224,6 @@ struct ip_fw_chain { LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ struct radix_node_head **tables; /* IPv4 tables */ struct radix_node_head **xtables; /* extended tables */ - uint8_t *tabletype; /* Array of table types */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t rwmtx; #else @@ -229,6 +233,7 @@ struct ip_fw_chain { uint32_t gencnt; /* NAT generation count */ struct ip_fw *reap; /* list of rules to reap */ struct ip_fw *default_rule; + struct tables_config *tblcfg; /* tables module data */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t uh_lock; #else @@ -295,13 +300,84 @@ struct sockopt; /* used by tcp_var.h */ #define IPFW_UH_WLOCK(p) rw_wlock(&(p)->uh_lock) #define IPFW_UH_WUNLOCK(p) rw_wunlock(&(p)->uh_lock) +struct tid_info { + uint32_t set; /* table set */ + uint16_t uidx; /* table index */ + uint8_t type; /* table type */ + uint8_t spare; + void *tlvs; /* Pointer to first TLV */ + int tlen; /* Total TLV size block */ +}; + +struct obj_idx { + uint16_t uidx; /* internal index supplied by userland */ + uint16_t kidx; /* kernel object index */ + uint16_t off; /* tlv offset from rule end in 4-byte words */ + uint8_t new; /* index is newly-allocated */ + uint8_t type; /* object type within its category */ +}; + +struct rule_check_info { + uint16_t table_opcodes; /* count of opcodes referencing table */ + uint16_t new_tables; /* count of opcodes referencing table */ + uint32_t tableset; /* ipfw set id for table */ + void *tlvs; /* Pointer to first TLV if any */ + int tlen; /* *Total TLV size block */ + uint8_t fw3; /* opcode is new */ + struct ip_fw *krule; /* resulting rule pointer */ + struct obj_idx obuf[8]; /* table references storage */ +}; + +struct tentry_info { + void *paddr; + int plen; /* Total entry length */ + uint8_t masklen; /* mask length */ + uint8_t spare; + uint16_t flags; /* record flags */ + uint32_t value; /* value */ +}; + /* In ip_fw_sockopt.c */ int ipfw_find_rule(struct ip_fw_chain *chain, uint32_t key, uint32_t id); -int ipfw_add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule); int ipfw_ctl(struct sockopt *sopt); int ipfw_chk(struct ip_fw_args *args); void ipfw_reap_rules(struct ip_fw *head); +struct namedobj_instance; + +struct named_object { + TAILQ_ENTRY(named_object) nn_next; /* namehash */ + TAILQ_ENTRY(named_object) nv_next; /* valuehash */ + char *name; /* object name */ + uint8_t type; /* object type */ + uint8_t compat; /* Object name is number */ + uint16_t kidx; /* object kernel index */ + uint16_t uidx; /* userland idx for compat records */ + uint32_t set; /* set object belongs to */ + uint32_t refcnt; /* number of references */ +}; +TAILQ_HEAD(namedobjects_head, named_object); + +typedef void (objhash_cb_t)(struct namedobj_instance *ni, struct named_object *, + void *arg); +struct namedobj_instance *ipfw_objhash_create(uint32_t items); +void ipfw_objhash_destroy(struct namedobj_instance *); +void ipfw_objhash_bitmap_alloc(uint32_t items, void **idx, int *pblocks); +int ipfw_objhash_bitmap_merge(struct namedobj_instance *ni, + void **idx, int *blocks); +void ipfw_objhash_bitmap_free(void *idx, int blocks); +struct named_object *ipfw_objhash_lookup_name(struct namedobj_instance *ni, + uint32_t set, char *name); +struct named_object *ipfw_objhash_lookup_idx(struct namedobj_instance *ni, + uint32_t set, uint16_t idx); +void ipfw_objhash_add(struct namedobj_instance *ni, struct named_object *no); +void ipfw_objhash_del(struct namedobj_instance *ni, struct named_object *no); +void ipfw_objhash_foreach(struct namedobj_instance *ni, objhash_cb_t *f, + void *arg); +int ipfw_objhash_free_idx(struct namedobj_instance *ni, uint32_t set, + uint16_t idx); +int ipfw_objhash_alloc_idx(void *n, uint32_t set, uint16_t *pidx); + /* In ip_fw_table.c */ struct radix_node; int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, @@ -309,18 +385,28 @@ int ipfw_lookup_table(struct ip_fw_chain *ch, uint int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, uint32_t *val, int type); int ipfw_init_tables(struct ip_fw_chain *ch); +int ipfw_destroy_table(struct ip_fw_chain *ch, struct tid_info *ti, int force); void ipfw_destroy_tables(struct ip_fw_chain *ch); -int ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl); -int ipfw_add_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type, uint32_t value); -int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type); -int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); +int ipfw_flush_table(struct ip_fw_chain *ch, struct tid_info *ti); +int ipfw_add_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei); +int ipfw_del_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei); +int ipfw_count_table(struct ip_fw_chain *ch, struct tid_info *ti, + uint32_t *cnt); int ipfw_dump_table_entry(struct radix_node *rn, void *arg); -int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); -int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); +int ipfw_dump_table(struct ip_fw_chain *ch, struct tid_info *ti, + ipfw_table *tbl); +int ipfw_count_xtable(struct ip_fw_chain *ch, struct tid_info *ti, + uint32_t *cnt); +int ipfw_dump_xtable(struct ip_fw_chain *ch, struct tid_info *ti, + ipfw_xtable *tbl); int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); +int ipfw_rewrite_table_uidx(struct ip_fw_chain *chain, + struct rule_check_info *ci); +int ipfw_rewrite_table_kidx(struct ip_fw_chain *chain, struct ip_fw *rule); +void ipfw_unbind_table_rule(struct ip_fw_chain *chain, struct ip_fw *rule); +void ipfw_unbind_table_list(struct ip_fw_chain *chain, struct ip_fw *head); /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ Index: sys/netpfil/ipfw/ip_fw_sockopt.c =================================================================== --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266306) +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy) @@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -67,6 +68,25 @@ __FBSDID("$FreeBSD$"); #include #endif +#define NAMEDOBJ_HASH_SIZE 32 + +struct namedobj_instance { + struct namedobjects_head *names; + struct namedobjects_head *values; + uint32_t nn_size; /* names hash size */ + uint32_t nv_size; /* number hash size */ + u_long *idx_mask; /* used items bitmask */ + uint32_t max_blocks; /* number of "long" blocks in bitmask */ + uint16_t free_off[IPFW_MAX_SETS]; /* first possible free offset */ +}; +#define BLOCK_ITEMS (8 * sizeof(u_long)) /* Number of items for ffsl() */ +//#define IDX_SET(i, s) (((i) & 0xFF) << 16 | ((s) & 0xFFFF)) +#define IDX_SET(i, s) (i) + +static uint32_t objhash_hash_name(struct namedobj_instance *ni, char *name); +static uint32_t objhash_hash_val(struct namedobj_instance *ni, uint32_t val); + + MALLOC_DEFINE(M_IPFW, "IpFw/IpAcct", "IpFw/IpAcct chain's"); /* @@ -152,8 +172,9 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * * XXX DO NOT USE FOR THE DEFAULT RULE. * Must be called without IPFW_UH held */ -int -ipfw_add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule) +static int +add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule, + struct rule_check_info *ci) { struct ip_fw *rule; int i, l, insert_before; @@ -164,19 +185,37 @@ swap_map(struct ip_fw_chain *chain, struct ip_fw * l = RULESIZE(input_rule); rule = malloc(l, M_IPFW, M_WAITOK | M_ZERO); + bcopy(input_rule, rule, l); + /* clear fields not settable from userland */ + rule->x_next = NULL; + rule->next_rule = NULL; + IPFW_ZERO_RULE_COUNTER(rule); + + /* Check if we need to do table remap */ + if (ci->table_opcodes > 0) { + ci->krule = rule; + i = ipfw_rewrite_table_uidx(chain, ci); + if (i != 0) { + /* rewrite failed, return error */ + free(rule, M_IPFW); + return (i); + } + } + /* get_map returns with IPFW_UH_WLOCK if successful */ map = get_map(chain, 1, 0 /* not locked */); if (map == NULL) { + if (ci->table_opcodes > 0) { + /* We need to unbind tables */ + IPFW_UH_WLOCK(chain); + ipfw_unbind_table_rule(chain, rule); + IPFW_UH_WUNLOCK(chain); + } + free(rule, M_IPFW); - return ENOSPC; + return (ENOSPC); } - bcopy(input_rule, rule, l); - /* clear fields not settable from userland */ - rule->x_next = NULL; - rule->next_rule = NULL; - IPFW_ZERO_RULE_COUNTER(rule); - if (V_autoinc_step < 1) V_autoinc_step = 1; else if (V_autoinc_step > 1000) @@ -421,6 +460,7 @@ del_entry(struct ip_fw_chain *chain, uint32_t arg) rule = chain->reap; chain->reap = NULL; + ipfw_unbind_table_list(chain, rule); IPFW_UH_WUNLOCK(chain); ipfw_reap_rules(rule); if (map) @@ -517,7 +557,7 @@ zero_entry(struct ip_fw_chain *chain, u_int32_t ar * Rules are simple, so this mostly need to check rule sizes. */ static int -check_ipfw_struct(struct ip_fw *rule, int size) +check_ipfw_struct(struct ip_fw *rule, int size, struct rule_check_info *ci) { int l, cmdlen = 0; int have_action=0; @@ -529,10 +569,15 @@ static int } /* first, check for valid size */ l = RULESIZE(rule); - if (l != size) { + if (l > size) { printf("ipfw: size mismatch (have %d want %d)\n", size, l); return (EINVAL); } + if (size > l) { + /* Save TLV information */ + ci->tlvs = (caddr_t)rule + l; + ci->tlen = size - l; + } if (rule->act_ofs >= rule->cmd_len) { printf("ipfw: bogus action offset (%u > %u)\n", rule->act_ofs, rule->cmd_len - 1); @@ -662,6 +707,7 @@ static int cmdlen != F_INSN_SIZE(ipfw_insn_u32) + 1 && cmdlen != F_INSN_SIZE(ipfw_insn_u32)) goto bad_size; + ci->table_opcodes++; break; case O_MACADDR2: if (cmdlen != F_INSN_SIZE(ipfw_insn_mac)) @@ -694,6 +740,8 @@ static int case O_RECV: case O_XMIT: case O_VIA: + if (((ipfw_insn_if *)cmd)->name[0] == '\1') + ci->table_opcodes++; if (cmdlen != F_INSN_SIZE(ipfw_insn_if)) goto bad_size; break; @@ -879,7 +927,7 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf char *bp = buf; char *ep = bp + space; struct ip_fw *rule, *dst; - int l, i; + int error, i, l; time_t boot_seconds; boot_seconds = boottime.tv_sec; @@ -890,8 +938,11 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf /* Convert rule to FreeBSd 7.2 format */ l = RULESIZE7(rule); if (bp + l + sizeof(uint32_t) <= ep) { - int error; bcopy(rule, bp, l + sizeof(uint32_t)); + error = ipfw_rewrite_table_kidx(chain, + (struct ip_fw *)bp); + if (error != 0) + return (0); error = convert_rule_to_7((struct ip_fw *) bp); if (error) return 0; /*XXX correct? */ @@ -918,6 +969,13 @@ ipfw_getrules(struct ip_fw_chain *chain, void *buf } dst = (struct ip_fw *)bp; bcopy(rule, dst, l); + error = ipfw_rewrite_table_kidx(chain, dst); + if (error != 0) { + printf("Stop on rule %d. Fail to convert table\n", + rule->rulenum); + break; + } + /* * XXX HACK. Store the disable mask in the "next" * pointer in a wild attempt to keep the ABI the same. @@ -949,6 +1007,7 @@ ipfw_ctl(struct sockopt *sopt) uint32_t opt; char xbuf[128]; ip_fw3_opheader *op3 = NULL; + struct rule_check_info ci; error = priv_check(sopt->sopt_td, PRIV_NETINET_IPFW); if (error) @@ -1027,6 +1086,8 @@ ipfw_ctl(struct sockopt *sopt) error = sooptcopyin(sopt, rule, RULE_MAXSIZE, sizeof(struct ip_fw7) ); + memset(&ci, 0, sizeof(struct rule_check_info)); + /* * If the size of commands equals RULESIZE7 then we assume * a FreeBSD7.2 binary is talking to us (set is7=1). @@ -1044,15 +1105,15 @@ ipfw_ctl(struct sockopt *sopt) return error; } if (error == 0) - error = check_ipfw_struct(rule, RULESIZE(rule)); + error = check_ipfw_struct(rule, RULESIZE(rule), &ci); } else { is7 = 0; if (error == 0) - error = check_ipfw_struct(rule, sopt->sopt_valsize); + error = check_ipfw_struct(rule, sopt->sopt_valsize,&ci); } if (error == 0) { - /* locking is done within ipfw_add_rule() */ - error = ipfw_add_rule(chain, rule); + /* locking is done within add_rule() */ + error = add_rule(chain, rule, &ci); size = RULESIZE(rule); if (!error && sopt->sopt_dir == SOPT_GET) { if (is7) { @@ -1114,30 +1175,59 @@ ipfw_ctl(struct sockopt *sopt) break; /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ - case IP_FW_TABLE_ADD: + case IP_FW_OBJ_DEL: /* IP_FW3 */ { - ipfw_table_entry ent; + struct _ipfw_obj_header *oh; + struct tid_info ti; - error = sooptcopyin(sopt, &ent, - sizeof(ent), sizeof(ent)); - if (error) + if (sopt->sopt_valsize < sizeof(*oh)) { + error = EINVAL; break; - error = ipfw_add_table_entry(chain, ent.tbl, - &ent.addr, sizeof(ent.addr), ent.masklen, - IPFW_TABLE_CIDR, ent.value); + } + + oh = (struct _ipfw_obj_header *)(op3 + 1); + + switch (oh->objtype) { + case IPFW_OBJTYPE_TABLE: + memset(&ti, 0, sizeof(ti)); + ti.set = oh->set; + ti.uidx = oh->idx; + ti.tlvs = (oh + 1); + ti.tlen = sopt->sopt_valsize - sizeof(*oh); + error = ipfw_destroy_table(chain, &ti, 0); + break; + default: + error = ENOTSUP; + break; + } + break; } - break; - + case IP_FW_TABLE_ADD: case IP_FW_TABLE_DEL: { ipfw_table_entry ent; + struct tentry_info tei; + struct tid_info ti; error = sooptcopyin(sopt, &ent, sizeof(ent), sizeof(ent)); if (error) break; - error = ipfw_del_table_entry(chain, ent.tbl, - &ent.addr, sizeof(ent.addr), ent.masklen, IPFW_TABLE_CIDR); + + memset(&tei, 0, sizeof(tei)); + tei.paddr = &ent.addr; + tei.plen = sizeof(ent.addr); + tei.masklen = ent.masklen; + tei.flags = 0; + tei.value = ent.value; + memset(&ti, 0, sizeof(ti)); + ti.set = RESVD_SET; + ti.uidx = ent.tbl; + ti.type = IPFW_TABLE_CIDR; + + error = (opt == IP_FW_TABLE_ADD) ? + ipfw_add_table_entry(chain, &ti, &tei) : + ipfw_del_table_entry(chain, &ti, &tei); } break; @@ -1145,6 +1235,8 @@ ipfw_ctl(struct sockopt *sopt) case IP_FW_TABLE_XDEL: /* IP_FW3 */ { ipfw_table_xentry *xent = (ipfw_table_xentry *)(op3 + 1); + struct tentry_info tei; + struct tid_info ti; /* Check minimum header size */ if (IP_FW3_OPLENGTH(sopt) < offsetof(ipfw_table_xentry, k)) { @@ -1160,35 +1252,52 @@ ipfw_ctl(struct sockopt *sopt) len = xent->len - offsetof(ipfw_table_xentry, k); + memset(&tei, 0, sizeof(tei)); + tei.paddr = &xent->k; + tei.plen = len; + tei.masklen = xent->masklen; + tei.flags = 0; + tei.value = xent->value; + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = xent->tbl; + ti.type = xent->type; + error = (opt == IP_FW_TABLE_XADD) ? - ipfw_add_table_entry(chain, xent->tbl, &xent->k, - len, xent->masklen, xent->type, xent->value) : - ipfw_del_table_entry(chain, xent->tbl, &xent->k, - len, xent->masklen, xent->type); + ipfw_add_table_entry(chain, &ti, &tei) : + ipfw_del_table_entry(chain, &ti, &tei); } break; case IP_FW_TABLE_FLUSH: { u_int16_t tbl; + struct tid_info ti; error = sooptcopyin(sopt, &tbl, sizeof(tbl), sizeof(tbl)); if (error) break; - error = ipfw_flush_table(chain, tbl); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl; + error = ipfw_flush_table(chain, &ti); } break; case IP_FW_TABLE_GETSIZE: { u_int32_t tbl, cnt; + struct tid_info ti; if ((error = sooptcopyin(sopt, &tbl, sizeof(tbl), sizeof(tbl)))) break; + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl; IPFW_RLOCK(chain); - error = ipfw_count_table(chain, tbl, &cnt); + error = ipfw_count_table(chain, &ti, &cnt); IPFW_RUNLOCK(chain); if (error) break; @@ -1199,6 +1308,7 @@ ipfw_ctl(struct sockopt *sopt) case IP_FW_TABLE_LIST: { ipfw_table *tbl; + struct tid_info ti; if (sopt->sopt_valsize < sizeof(*tbl)) { error = EINVAL; @@ -1213,8 +1323,11 @@ ipfw_ctl(struct sockopt *sopt) } tbl->size = (size - sizeof(*tbl)) / sizeof(ipfw_table_entry); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl->tbl; IPFW_RLOCK(chain); - error = ipfw_dump_table(chain, tbl); + error = ipfw_dump_table(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) { free(tbl, M_TEMP); @@ -1228,6 +1341,7 @@ ipfw_ctl(struct sockopt *sopt) case IP_FW_TABLE_XGETSIZE: /* IP_FW3 */ { uint32_t *tbl; + struct tid_info ti; if (IP_FW3_OPLENGTH(sopt) < sizeof(uint32_t)) { error = EINVAL; @@ -1236,8 +1350,11 @@ ipfw_ctl(struct sockopt *sopt) tbl = (uint32_t *)(op3 + 1); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = *tbl; IPFW_RLOCK(chain); - error = ipfw_count_xtable(chain, *tbl, tbl); + error = ipfw_count_xtable(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) break; @@ -1248,6 +1365,7 @@ ipfw_ctl(struct sockopt *sopt) case IP_FW_TABLE_XLIST: /* IP_FW3 */ { ipfw_xtable *tbl; + struct tid_info ti; if ((size = valsize) < sizeof(ipfw_xtable)) { error = EINVAL; @@ -1260,8 +1378,11 @@ ipfw_ctl(struct sockopt *sopt) /* Get maximum number of entries we can store */ tbl->size = (size - sizeof(ipfw_xtable)) / sizeof(ipfw_table_xentry); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl->tbl; IPFW_RLOCK(chain); - error = ipfw_dump_xtable(chain, tbl); + error = ipfw_dump_xtable(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) { free(tbl, M_TEMP); @@ -1444,4 +1565,271 @@ convert_rule_to_8(struct ip_fw *rule) return 0; } +/* + * Named object api + * + */ + +void +ipfw_objhash_bitmap_alloc(uint32_t items, void **idx, int *pblocks) +{ + size_t size; + int max_blocks; + void *idx_mask; + + items = roundup2(items, BLOCK_ITEMS); /* Align to block size */ + max_blocks = items / BLOCK_ITEMS; + size = items / 8; + idx_mask = malloc(size * IPFW_MAX_SETS, M_IPFW, M_WAITOK); + /* Mark all as free */ + memset(idx_mask, 0xFF, size * IPFW_MAX_SETS); + + *idx = idx_mask; + *pblocks = max_blocks; +} + +int +ipfw_objhash_bitmap_merge(struct namedobj_instance *ni, void **idx, int *blocks) +{ + int old_blocks, new_blocks; + u_long *old_idx, *new_idx; + int i; + + old_idx = ni->idx_mask; + old_blocks = ni->max_blocks; + new_idx = *idx; + new_blocks = *blocks; + + /* + * FIXME: Permit reducing total amount of tables + */ + if (old_blocks > new_blocks) + return (1); + + for (i = 0; i < IPFW_MAX_SETS; i++) { + memcpy(&new_idx[new_blocks * i], &old_idx[old_blocks * i], + old_blocks * sizeof(u_long)); + } + + ni->idx_mask = new_idx; + ni->max_blocks = new_blocks; + + /* Save old values */ + *idx = old_idx; + *blocks = old_blocks; + + return (0); +} + +void +ipfw_objhash_bitmap_free(void *idx, int blocks) +{ + + free(idx, M_IPFW); +} + +/* + * Creates named hash instance. + * Must be called without holding any locks. + * Return pointer to new instance. + */ +struct namedobj_instance * +ipfw_objhash_create(uint32_t items) +{ + struct namedobj_instance *ni; + int i; + size_t size; + + size = sizeof(struct namedobj_instance) + + sizeof(struct namedobjects_head) * NAMEDOBJ_HASH_SIZE + + sizeof(struct namedobjects_head) * NAMEDOBJ_HASH_SIZE; + + ni = malloc(size, M_IPFW, M_WAITOK | M_ZERO); + ni->nn_size = NAMEDOBJ_HASH_SIZE; + ni->nv_size = NAMEDOBJ_HASH_SIZE; + + ni->names = (struct namedobjects_head *)(ni +1); + ni->values = &ni->names[ni->nn_size]; + + for (i = 0; i < ni->nn_size; i++) + TAILQ_INIT(&ni->names[i]); + + for (i = 0; i < ni->nv_size; i++) + TAILQ_INIT(&ni->values[i]); + + /* Allocate bitmask separately due to possible resize */ + ipfw_objhash_bitmap_alloc(items, (void*)&ni->idx_mask, &ni->max_blocks); + + return (ni); +} + +void +ipfw_objhash_destroy(struct namedobj_instance *ni) +{ + + free(ni->idx_mask, M_IPFW); + free(ni, M_IPFW); +} + +static uint32_t +objhash_hash_name(struct namedobj_instance *ni, char *name) +{ + uint32_t v; + + v = fnv_32_str(name, FNV1_32_INIT); + + return (v % ni->nn_size); +} + +static uint32_t +objhash_hash_val(struct namedobj_instance *ni, uint32_t val) +{ + uint32_t v; + + v = val % 31; /* Assume hash size to be 32 */ + + return (v % ni->nv_size); +} + +struct named_object * +ipfw_objhash_lookup_name(struct namedobj_instance *ni, uint32_t set, char *name) +{ + struct named_object *no; + uint32_t hash; + + hash = objhash_hash_name(ni, name); + + TAILQ_FOREACH(no, &ni->names[hash], nn_next) { + if ((strcmp(no->name, name) == 0) && (no->set == set)) + return (no); + } + + return (NULL); +} + +struct named_object * +ipfw_objhash_lookup_idx(struct namedobj_instance *ni, uint32_t set, + uint16_t idx) +{ + struct named_object *no; + uint32_t hash; + + hash = objhash_hash_val(ni, IDX_SET(idx, set)); + + TAILQ_FOREACH(no, &ni->values[hash], nv_next) { + if ((no->kidx == idx) && (no->set == set)) + return (no); + } + + return (NULL); +} + +void +ipfw_objhash_add(struct namedobj_instance *ni, struct named_object *no) +{ + uint32_t hash; + + hash = objhash_hash_name(ni, no->name); + TAILQ_INSERT_HEAD(&ni->names[hash], no, nn_next); + + hash = objhash_hash_val(ni, IDX_SET(no->kidx, no->set)); + TAILQ_INSERT_HEAD(&ni->values[hash], no, nv_next); +} + +void +ipfw_objhash_del(struct namedobj_instance *ni, struct named_object *no) +{ + uint32_t hash; + + hash = objhash_hash_name(ni, no->name); + TAILQ_REMOVE(&ni->names[hash], no, nn_next); + + hash = objhash_hash_val(ni, no->kidx); + TAILQ_REMOVE(&ni->values[hash], no, nv_next); +} + +/* + * Runs @func for each found named object. + * It is safe to delete objects from callback + */ +void +ipfw_objhash_foreach(struct namedobj_instance *ni, objhash_cb_t *f, void *arg) +{ + struct named_object *no, *no_tmp; + int i; + + for (i = 0; i < ni->nn_size; i++) { + TAILQ_FOREACH_SAFE(no, &ni->names[i], nn_next, no_tmp) + f(ni, no, arg); + } +} + +/* + * Removes index from given set. + * Returns 0 on success. + */ +int +ipfw_objhash_free_idx(struct namedobj_instance *ni, uint32_t set, uint16_t idx) +{ + u_long *mask; + int i, v; + + i = idx / BLOCK_ITEMS; + v = idx % BLOCK_ITEMS; + + if ((i >= ni->max_blocks) || set >= IPFW_MAX_SETS) + return (1); + + mask = &ni->idx_mask[set * ni->max_blocks + i]; + + if ((*mask & ((u_long)1 << v)) != 0) + return (1); + + /* Mark as free */ + *mask |= (u_long)1 << v; + + /* Update free offset */ + if (ni->free_off[set] > i) + ni->free_off[set] = i; + + return (0); +} + +/* + * Allocate new index in given set and stores in in @pidx. + * Returns 0 on success. + */ +int +ipfw_objhash_alloc_idx(void *n, uint32_t set, uint16_t *pidx) +{ + struct namedobj_instance *ni; + u_long *mask; + int i, off, v; + + if (set >= IPFW_MAX_SETS) + return (-1); + + ni = (struct namedobj_instance *)n; + + off = ni->free_off[set]; + mask = &ni->idx_mask[set * ni->max_blocks + off]; + + for (i = off; i < ni->max_blocks; i++, mask++) { + if ((v = ffsl(*mask)) == 0) + continue; + + /* Mark as busy */ + *mask &= ~ ((u_long)1 << (v - 1)); + + ni->free_off[set] = i; + + v = BLOCK_ITEMS * i + v - 1; + + *pidx = v; + return (0); + } + + return (1); +} + /* end of file */ Index: sys/netpfil/ipfw/ip_fw_table.c =================================================================== --- sys/netpfil/ipfw/ip_fw_table.c (revision 266310) +++ sys/netpfil/ipfw/ip_fw_table.c (working copy) @@ -100,6 +100,49 @@ struct table_xentry { u_int32_t value; }; + /* + * Table has the following `type` concepts: + * + * `type` represents lookup key type (cidr, ifp, uid, etc..) + * `ftype` is pure userland field helping to properly format table data + * `atype` represents exact lookup algorithm for given tabletype. + * For example, we can use more efficient search schemes if we plan + * to use some specific table for storing host-routes only. + * + */ +struct table_config { + struct named_object no; + uint8_t ftype; /* format table type */ + uint8_t atype; /* algorith type */ + uint8_t linked; /* 1 if already linked */ + uint8_t spare0; + uint32_t count; /* Number of records */ + char tablename[64]; /* table name */ + void *state; /* Store some state if needed */ + void *xstate; +}; +#define TABLE_SET(set) ((V_fw_tables_sets != 0) ? set : 0) + +struct tables_config { + struct namedobj_instance *namehash; +}; + +static struct table_config *find_table(struct namedobj_instance *ni, + struct tid_info *ti); +static struct table_config *alloc_table_config(struct namedobj_instance *ni, + struct tid_info *ti); +static void free_table_config(struct namedobj_instance *ni, + struct table_config *tc); +static void link_table(struct ip_fw_chain *chain, struct table_config *tc); +static void unlink_table(struct ip_fw_chain *chain, struct table_config *tc); +static int alloc_table_state(void **state, void **xstate, uint8_t type); +static void free_table_state(void **state, void **xstate, uint8_t type); + + +#define CHAIN_TO_TCFG(chain) ((struct tables_config *)(chain)->tblcfg) +#define CHAIN_TO_NI(chain) (CHAIN_TO_TCFG(chain)->namehash) + + /* * The radix code expects addr and mask to be array of bytes, * with the first byte being the length of the array. rn_inithead @@ -136,10 +179,10 @@ ipv6_writemask(struct in6_addr *addr6, uint8_t mas #endif int -ipfw_add_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type, uint32_t value) +ipfw_add_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei) { - struct radix_node_head *rnh, **rnh_ptr; + struct radix_node_head *rnh; struct table_entry *ent; struct table_xentry *xent; struct radix_node *rn; @@ -147,51 +190,57 @@ int int offset; void *ent_ptr; struct sockaddr *addr_ptr, *mask_ptr; + struct table_config *tc, *tc_new; + struct namedobj_instance *ni; char c; + uint8_t mlen; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); - switch (type) { + mlen = tei->masklen; + + switch (ti->type) { case IPFW_TABLE_CIDR: - if (plen == sizeof(in_addr_t)) { + if (tei->plen == sizeof(in_addr_t)) { #ifdef INET /* IPv4 case */ if (mlen > 32) return (EINVAL); ent = malloc(sizeof(*ent), M_IPFW_TBL, M_WAITOK | M_ZERO); - ent->value = value; + ent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(ent->addr) = KEY_LEN_INET; KEY_LEN(ent->mask) = KEY_LEN_INET; /* Set offset of IPv4 address in bits */ offset = OFF_LEN_INET; - ent->mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); - addr = *((in_addr_t *)paddr); + ent->mask.sin_addr.s_addr = + htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); + addr = *((in_addr_t *)tei->paddr); ent->addr.sin_addr.s_addr = addr & ent->mask.sin_addr.s_addr; /* Set pointers */ - rnh_ptr = &ch->tables[tbl]; ent_ptr = ent; addr_ptr = (struct sockaddr *)&ent->addr; mask_ptr = (struct sockaddr *)&ent->mask; #endif #ifdef INET6 - } else if (plen == sizeof(struct in6_addr)) { + } else if (tei->plen == sizeof(struct in6_addr)) { /* IPv6 case */ if (mlen > 128) return (EINVAL); xent = malloc(sizeof(*xent), M_IPFW_TBL, M_WAITOK | M_ZERO); - xent->value = value; + xent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(xent->a.addr6) = KEY_LEN_INET6; KEY_LEN(xent->m.mask6) = KEY_LEN_INET6; /* Set offset of IPv6 address in bits */ offset = OFF_LEN_INET6; ipv6_writemask(&xent->m.mask6.sin6_addr, mlen); - memcpy(&xent->a.addr6.sin6_addr, paddr, sizeof(struct in6_addr)); + memcpy(&xent->a.addr6.sin6_addr, tei->paddr, + sizeof(struct in6_addr)); APPLY_MASK(&xent->a.addr6.sin6_addr, &xent->m.mask6.sin6_addr); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.addr6; mask_ptr = (struct sockaddr *)&xent->m.mask6; @@ -204,22 +253,23 @@ int case IPFW_TABLE_INTERFACE: /* Check if string is terminated */ - c = ((char *)paddr)[IF_NAMESIZE - 1]; - ((char *)paddr)[IF_NAMESIZE - 1] = '\0'; - if (((mlen = strlen((char *)paddr)) == IF_NAMESIZE - 1) && (c != '\0')) + c = ((char *)tei->paddr)[IF_NAMESIZE - 1]; + ((char *)tei->paddr)[IF_NAMESIZE - 1] = '\0'; + mlen = strlen((char *)tei->paddr); + if ((mlen == IF_NAMESIZE - 1) && (c != '\0')) return (EINVAL); /* Include last \0 into comparison */ mlen++; xent = malloc(sizeof(*xent), M_IPFW_TBL, M_WAITOK | M_ZERO); - xent->value = value; + xent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(xent->a.iface) = KEY_LEN_IFACE + mlen; KEY_LEN(xent->m.ifmask) = KEY_LEN_IFACE + mlen; /* Set offset of interface name in bits */ offset = OFF_LEN_IFACE; - memcpy(xent->a.iface.ifname, paddr, mlen); + memcpy(xent->a.iface.ifname, tei->paddr, mlen); /* Assume direct match */ /* TODO: Add interface pattern matching */ #if 0 @@ -227,7 +277,6 @@ int mask_ptr = (struct sockaddr *)&xent->m.ifmask; #endif /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.iface; mask_ptr = NULL; @@ -237,84 +286,128 @@ int return (EINVAL); } - IPFW_WLOCK(ch); + IPFW_UH_WLOCK(ch); - /* Check if tabletype is valid */ - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { - IPFW_WUNLOCK(ch); - free(ent_ptr, M_IPFW_TBL); - return (EINVAL); - } + ni = CHAIN_TO_NI(ch); - /* Check if radix tree exists */ - if ((rnh = *rnh_ptr) == NULL) { - IPFW_WUNLOCK(ch); - /* Create radix for a new table */ - if (!rn_inithead((void **)&rnh, offset)) { - free(ent_ptr, M_IPFW_TBL); + tc_new = NULL; + if ((tc = find_table(ni, ti)) == NULL) { + /* Not found. We have to create new one */ + IPFW_UH_WUNLOCK(ch); + + tc_new = alloc_table_config(ni, ti); + if (tc_new == NULL) return (ENOMEM); - } - IPFW_WLOCK(ch); - if (*rnh_ptr != NULL) { - /* Tree is already attached by other thread */ - rn_detachhead((void **)&rnh); - rnh = *rnh_ptr; - /* Check table type another time */ - if (ch->tabletype[tbl] != type) { - IPFW_WUNLOCK(ch); - free(ent_ptr, M_IPFW_TBL); + IPFW_UH_WLOCK(ch); + + /* Check if table has already allocated by other thread */ + if ((tc = find_table(ni, ti)) != NULL) { + if (tc->no.type != ti->type) { + IPFW_UH_WUNLOCK(ch); + free_table_config(ni, tc); return (EINVAL); } } else { - *rnh_ptr = rnh; - /* - * Set table type. It can be set already - * (if we have IPv6-only table) but setting - * it another time does not hurt + /* + * New table. + * Set tc_new to zero not to free it afterwards. */ - ch->tabletype[tbl] = type; + tc = tc_new; + tc_new = NULL; + + /* Allocate table index. */ + if (ipfw_objhash_alloc_idx(ni, ti->set, &kidx) != 0) { + /* Index full. */ + IPFW_UH_WUNLOCK(ch); + printf("Unable to allocate index for table %s." + " Consider increasing " + "net.inet.ip.fw.tables_max", + tc->no.name); + free_table_config(ni, tc); + return (EBUSY); + } + /* Save kidx */ + tc->no.kidx = kidx; } + } else { + /* We still have to check table type */ + if (tc->no.type != ti->type) { + IPFW_UH_WUNLOCK(ch); + return (EINVAL); + } } + kidx = tc->no.kidx; + /* We've got valid table in @tc. Let's add data */ + IPFW_WLOCK(ch); + + if (tc->linked == 0) { + link_table(ch, tc); + } + + /* XXX: Temporary until splitting add/del to per-type functions */ + rnh = NULL; + switch (ti->type) { + case IPFW_TABLE_CIDR: + if (tei->plen == sizeof(in_addr_t)) + rnh = ch->tables[kidx]; + else + rnh = ch->xtables[kidx]; + break; + case IPFW_TABLE_INTERFACE: + rnh = ch->xtables[kidx]; + break; + } + rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + if (tc_new != NULL) + free_table_config(ni, tc); + if (rn == NULL) { free(ent_ptr, M_IPFW_TBL); return (EEXIST); } + return (0); } int -ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type) +ipfw_del_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei) { - struct radix_node_head *rnh, **rnh_ptr; + struct radix_node_head *rnh; struct table_entry *ent; in_addr_t addr; struct sockaddr_in sa, mask; struct sockaddr *sa_ptr, *mask_ptr; + struct table_config *tc; + struct namedobj_instance *ni; char c; + uint8_t mlen; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); - switch (type) { + mlen = tei->masklen; + + switch (ti->type) { case IPFW_TABLE_CIDR: - if (plen == sizeof(in_addr_t)) { + if (tei->plen == sizeof(in_addr_t)) { /* Set 'total' structure length */ KEY_LEN(sa) = KEY_LEN_INET; KEY_LEN(mask) = KEY_LEN_INET; mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); - addr = *((in_addr_t *)paddr); + addr = *((in_addr_t *)tei->paddr); sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; - rnh_ptr = &ch->tables[tbl]; sa_ptr = (struct sockaddr *)&sa; mask_ptr = (struct sockaddr *)&mask; #ifdef INET6 - } else if (plen == sizeof(struct in6_addr)) { + } else if (tei->plen == sizeof(struct in6_addr)) { /* IPv6 case */ if (mlen > 128) return (EINVAL); @@ -325,9 +418,9 @@ int KEY_LEN(sa6) = KEY_LEN_INET6; KEY_LEN(mask6) = KEY_LEN_INET6; ipv6_writemask(&mask6.sin6_addr, mlen); - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); + memcpy(&sa6.sin6_addr, tei->paddr, + sizeof(struct in6_addr)); APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); - rnh_ptr = &ch->xtables[tbl]; sa_ptr = (struct sockaddr *)&sa6; mask_ptr = (struct sockaddr *)&mask6; #endif @@ -339,9 +432,10 @@ int case IPFW_TABLE_INTERFACE: /* Check if string is terminated */ - c = ((char *)paddr)[IF_NAMESIZE - 1]; - ((char *)paddr)[IF_NAMESIZE - 1] = '\0'; - if (((mlen = strlen((char *)paddr)) == IF_NAMESIZE - 1) && (c != '\0')) + c = ((char *)tei->paddr)[IF_NAMESIZE - 1]; + ((char *)tei->paddr)[IF_NAMESIZE - 1] = '\0'; + mlen = strlen((char *)tei->paddr); + if ((mlen == IF_NAMESIZE - 1) && (c != '\0')) return (EINVAL); struct xaddr_iface ifname, ifmask; @@ -360,9 +454,8 @@ int mask_ptr = (struct sockaddr *)&ifmask; #endif mask_ptr = NULL; - memcpy(ifname.ifname, paddr, mlen); + memcpy(ifname.ifname, tei->paddr, mlen); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; sa_ptr = (struct sockaddr *)&ifname; break; @@ -371,20 +464,39 @@ int return (EINVAL); } - IPFW_WLOCK(ch); - if ((rnh = *rnh_ptr) == NULL) { - IPFW_WUNLOCK(ch); + IPFW_UH_RLOCK(ch); + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_RUNLOCK(ch); return (ESRCH); } - if (ch->tabletype[tbl] != type) { - IPFW_WUNLOCK(ch); + if (tc->no.type != ti->type) { + IPFW_UH_RUNLOCK(ch); return (EINVAL); } + kidx = tc->no.kidx; + IPFW_WLOCK(ch); + + rnh = NULL; + switch (ti->type) { + case IPFW_TABLE_CIDR: + if (tei->plen == sizeof(in_addr_t)) + rnh = ch->tables[kidx]; + else + rnh = ch->xtables[kidx]; + break; + case IPFW_TABLE_INTERFACE: + rnh = ch->xtables[kidx]; + break; + } + ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, rnh); IPFW_WUNLOCK(ch); + IPFW_UH_RUNLOCK(ch); + if (ent == NULL) return (ESRCH); @@ -405,66 +517,161 @@ flush_table_entry(struct radix_node *rn, void *arg return (0); } +/* + * Flushes all entries in given table minimizing hoding chain WLOCKs. + * + */ int -ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) +ipfw_flush_table(struct ip_fw_chain *ch, struct tid_info *ti) { - struct radix_node_head *rnh, *xrnh; + struct namedobj_instance *ni; + struct table_config *tc; + void *ostate, *oxstate; + void *state, *xstate; + int error; + uint8_t type; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); /* - * We free both (IPv4 and extended) radix trees and - * clear table type here to permit table to be reused - * for different type without module reload + * Stage 1: determine table type. + * Reference found table to ensure it won't disappear. */ + IPFW_UH_WLOCK(ch); + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_WUNLOCK(ch); + return (ESRCH); + } + type = tc->no.type; + tc->no.refcnt++; + IPFW_UH_WUNLOCK(ch); + /* + * Stage 2: allocate new state for given type. + */ + if ((error = alloc_table_state(&state, &xstate, type)) != 0) { + IPFW_UH_WLOCK(ch); + tc->no.refcnt--; + IPFW_UH_WUNLOCK(ch); + return (error); + } + + /* + * Stage 3: swap old state pointers with newly-allocated ones. + * Decrease refcount. + */ + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); - /* Set IPv4 table pointer to zero */ - if ((rnh = ch->tables[tbl]) != NULL) - ch->tables[tbl] = NULL; - /* Set extended table pointer to zero */ - if ((xrnh = ch->xtables[tbl]) != NULL) - ch->xtables[tbl] = NULL; - /* Zero table type */ - ch->tabletype[tbl] = 0; + + ni = CHAIN_TO_NI(ch); + kidx = tc->no.kidx; + + ostate = ch->tables[kidx]; + ch->tables[kidx] = state; + oxstate = ch->xtables[kidx]; + ch->xtables[kidx] = xstate; + + tc->no.refcnt--; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); - if (rnh != NULL) { - rnh->rnh_walktree(rnh, flush_table_entry, rnh); - rn_detachhead((void **)&rnh); + /* + * Stage 4: perform real flush. + */ + free_table_state(&ostate, &xstate, tc->no.type); + + return (0); +} + +/* + * Destroys given table @ti: flushes it, + */ +int +ipfw_destroy_table(struct ip_fw_chain *ch, struct tid_info *ti, int force) +{ + struct namedobj_instance *ni; + struct table_config *tc; + + ti->set = TABLE_SET(ti->set); + + IPFW_UH_WLOCK(ch); + + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_WUNLOCK(ch); + return (ESRCH); } - if (xrnh != NULL) { - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); - rn_detachhead((void **)&xrnh); + /* Do not permit destroying used tables */ + if (tc->no.refcnt > 0 && force == 0) { + IPFW_UH_WUNLOCK(ch); + return (EBUSY); } + IPFW_WLOCK(ch); + unlink_table(ch, tc); + IPFW_WUNLOCK(ch); + + /* Free obj index */ + if (ipfw_objhash_free_idx(ni, tc->no.set, tc->no.kidx) != 0) + printf("Error unlinking kidx %d from table %s\n", + tc->no.kidx, tc->tablename); + + IPFW_UH_WUNLOCK(ch); + + free_table_config(ni, tc); + return (0); } +static void +destroy_table_locked(struct namedobj_instance *ni, struct named_object *no, + void *arg) +{ + + unlink_table((struct ip_fw_chain *)arg, (struct table_config *)no); + if (ipfw_objhash_free_idx(ni, no->set, no->kidx) != 0) + printf("Error unlinking kidx %d from table %s\n", + no->kidx, no->name); + free_table_config(ni, (struct table_config *)no); +} + void ipfw_destroy_tables(struct ip_fw_chain *ch) { - uint16_t tbl; - /* Flush all tables */ - for (tbl = 0; tbl < V_fw_tables_max; tbl++) - ipfw_flush_table(ch, tbl); + /* Remove all tables from working set */ + IPFW_UH_WLOCK(ch); + IPFW_WLOCK(ch); + ipfw_objhash_foreach(CHAIN_TO_NI(ch), destroy_table_locked, ch); + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); /* Free pointers itself */ free(ch->tables, M_IPFW); free(ch->xtables, M_IPFW); - free(ch->tabletype, M_IPFW); + + ipfw_objhash_destroy(CHAIN_TO_NI(ch)); + free(CHAIN_TO_TCFG(ch), M_IPFW); } int ipfw_init_tables(struct ip_fw_chain *ch) { + struct tables_config *tcfg; + /* Allocate pointers */ ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + + tcfg = malloc(sizeof(struct tables_config), M_IPFW, M_WAITOK | M_ZERO); + tcfg->namehash = ipfw_objhash_create(V_fw_tables_max); + ch->tblcfg = tcfg; + return (0); } @@ -473,8 +680,10 @@ ipfw_resize_tables(struct ip_fw_chain *ch, unsigne { struct radix_node_head **tables, **xtables, *rnh; struct radix_node_head **tables_old, **xtables_old; - uint8_t *tabletype, *tabletype_old; unsigned int ntables_old, tbl; + struct namedobj_instance *ni; + void *new_idx; + int new_blocks; /* Check new value for validity */ if (ntables > IPFW_TABLES_MAX) @@ -483,24 +692,31 @@ ipfw_resize_tables(struct ip_fw_chain *ch, unsigne /* Allocate new pointers */ tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + ipfw_objhash_bitmap_alloc(ntables, (void *)&new_idx, &new_blocks); IPFW_WLOCK(ch); tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; + ni = CHAIN_TO_NI(ch); + /* Temportary restrict decreasing max_tables */ + if (ipfw_objhash_bitmap_merge(ni, &new_idx, &new_blocks) != 0) { + IPFW_WUNLOCK(ch); + free(tables, M_IPFW); + free(xtables, M_IPFW); + ipfw_objhash_bitmap_free(new_idx, new_blocks); + return (EINVAL); + } + /* Copy old table pointers */ memcpy(tables, ch->tables, sizeof(void *) * tbl); memcpy(xtables, ch->xtables, sizeof(void *) * tbl); - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); /* Change pointers and number of tables */ tables_old = ch->tables; xtables_old = ch->xtables; - tabletype_old = ch->tabletype; ch->tables = tables; ch->xtables = xtables; - ch->tabletype = tabletype; ntables_old = V_fw_tables_max; V_fw_tables_max = ntables; @@ -525,7 +741,7 @@ ipfw_resize_tables(struct ip_fw_chain *ch, unsigne /* Free old pointers */ free(tables_old, M_IPFW); free(xtables_old, M_IPFW); - free(tabletype_old, M_IPFW); + ipfw_objhash_bitmap_free(new_idx, new_blocks); return (0); } @@ -602,14 +818,17 @@ count_table_entry(struct radix_node *rn, void *arg } int -ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) +ipfw_count_table(struct ip_fw_chain *ch, struct tid_info *ti, uint32_t *cnt) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (ESRCH); *cnt = 0; - if ((rnh = ch->tables[tbl]) == NULL) + if ((rnh = ch->tables[tc->no.kidx]) == NULL) return (0); rnh->rnh_walktree(rnh, count_table_entry, cnt); return (0); @@ -637,14 +856,17 @@ dump_table_entry(struct radix_node *rn, void *arg) } int -ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) +ipfw_dump_table(struct ip_fw_chain *ch, struct tid_info *ti, ipfw_table *tbl) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl->tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (ESRCH); tbl->cnt = 0; - if ((rnh = ch->tables[tbl->tbl]) == NULL) + if ((rnh = ch->tables[tc->no.kidx]) == NULL) return (0); rnh->rnh_walktree(rnh, dump_table_entry, tbl); return (0); @@ -660,16 +882,19 @@ count_table_xentry(struct radix_node *rn, void *ar } int -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) +ipfw_count_xtable(struct ip_fw_chain *ch, struct tid_info *ti, uint32_t *cnt) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); *cnt = 0; - if ((rnh = ch->tables[tbl]) != NULL) + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (0); /* XXX: We should return ESRCH */ + if ((rnh = ch->tables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, count_table_xentry, cnt); - if ((rnh = ch->xtables[tbl]) != NULL) + if ((rnh = ch->xtables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, count_table_xentry, cnt); /* Return zero if table is empty */ if (*cnt > 0) @@ -747,19 +972,700 @@ dump_table_xentry_extended(struct radix_node *rn, } int -ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) +ipfw_dump_xtable(struct ip_fw_chain *ch, struct tid_info *ti, ipfw_xtable *tbl) { struct radix_node_head *rnh; + struct table_config *tc; if (tbl->tbl >= V_fw_tables_max) return (EINVAL); tbl->cnt = 0; - tbl->type = ch->tabletype[tbl->tbl]; - if ((rnh = ch->tables[tbl->tbl]) != NULL) + + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (0); /* XXX: We should return ESRCH */ + tbl->type = tc->no.type; + if ((rnh = ch->tables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); - if ((rnh = ch->xtables[tbl->tbl]) != NULL) + if ((rnh = ch->xtables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); return (0); } +/* + * Tables rewriting code + * + */ + +/* + * Determine table number and lookup type for @cmd. + * Fill @tbl and @type with appropriate values. + * Returns 0 for relevant opcodes, 1 otherwise. + */ +static int +classify_table_opcode(ipfw_insn *cmd, uint16_t *puidx, uint8_t *ptype) +{ + ipfw_insn_if *cmdif; + int skip; + uint16_t v; + + skip = 1; + + switch (cmd->opcode) { + case O_IP_SRC_LOOKUP: + case O_IP_DST_LOOKUP: + /* Basic IPv4/IPv6 or u32 lookups */ + *puidx = cmd->arg1; + /* Assume CIDR by default */ + *ptype = IPFW_TABLE_CIDR; + skip = 0; + + if (F_LEN(cmd) > F_INSN_SIZE(ipfw_insn_u32)) { + /* + * generic lookup. The key must be + * in 32bit big-endian format. + */ + v = ((ipfw_insn_u32 *)cmd)->d[1]; + switch (v) { + case 0: + case 1: + /* IPv4 src/dst */ + break; + case 2: + case 3: + /* src/dst port */ + //type = IPFW_TABLE_U16; + break; + case 4: + /* uid/gid */ + //type = IPFW_TABLE_U32; + case 5: + //type = IPFW_TABLE_U32; + /* jid */ + case 6: + //type = IPFW_TABLE_U16; + /* dscp */ + break; + } + } + break; + case O_XMIT: + case O_RECV: + case O_VIA: + /* Interface table, possibly */ + cmdif = (ipfw_insn_if *)cmd; + if (cmdif->name[0] != '\1') + break; + + *ptype = IPFW_TABLE_INTERFACE; + *puidx = cmdif->p.glob; + skip = 0; + break; + } + + return (skip); +} + +/* + * Sets new table value for given opcode. + * Assume the same opcodes as classify_table_opcode() + */ +static void +update_table_opcode(ipfw_insn *cmd, uint16_t idx) +{ + ipfw_insn_if *cmdif; + + switch (cmd->opcode) { + case O_IP_SRC_LOOKUP: + case O_IP_DST_LOOKUP: + /* Basic IPv4/IPv6 or u32 lookups */ + cmd->arg1 = idx; + break; + case O_XMIT: + case O_RECV: + case O_VIA: + /* Interface table, possibly */ + cmdif = (ipfw_insn_if *)cmd; + cmdif->p.glob = idx; + break; + } +} + +static char * +find_name_tlv(void *tlvs, int len, uint16_t uidx) +{ + ipfw_xtable_ntlv *ntlv; + uintptr_t pa, pe; + int l; + + pa = (uintptr_t)tlvs; + pe = pa + len; + l = 0; + for (; pa < pe; pa += l) { + ntlv = (ipfw_xtable_ntlv *)pa; + l = ntlv->head.length; + if (ntlv->head.type != IPFW_TLV_NAME) + continue; + if (ntlv->idx != uidx) + continue; + + return (ntlv->name); + } + + return (NULL); +} + +static struct table_config * +find_table(struct namedobj_instance *ni, struct tid_info *ti) +{ + char *name, bname[16]; + struct named_object *no; + + if (ti->tlvs != NULL) { + name = find_name_tlv(ti->tlvs, ti->tlen, ti->uidx); + if (name == NULL) + return (NULL); + } else { + snprintf(bname, sizeof(bname), "%d", ti->uidx); + name = bname; + } + + no = ipfw_objhash_lookup_name(ni, ti->set, name); + + return ((struct table_config *)no); +} + +static int +alloc_table_state(void **state, void **xstate, uint8_t type) +{ + + switch (type) { + case IPFW_TABLE_CIDR: + if (!rn_inithead(state, OFF_LEN_INET)) + return (ENOMEM); + if (!rn_inithead(xstate, OFF_LEN_INET6)) { + rn_detachhead(state); + return (ENOMEM); + } + break; + case IPFW_TABLE_INTERFACE: + *state = NULL; + if (!rn_inithead(xstate, OFF_LEN_IFACE)) + return (ENOMEM); + break; + } + + return (0); +} + + +static struct table_config * +alloc_table_config(struct namedobj_instance *ni, struct tid_info *ti) +{ + char *name, bname[16]; + struct table_config *tc; + int error; + + if (ti->tlvs != NULL) { + name = find_name_tlv(ti->tlvs, ti->tlen, ti->uidx); + if (name == NULL) + return (NULL); + } else { + snprintf(bname, sizeof(bname), "%d", ti->uidx); + name = bname; + } + + tc = malloc(sizeof(struct table_config), M_IPFW, M_WAITOK | M_ZERO); + tc->no.name = tc->tablename; + tc->no.type = ti->type; + tc->no.set = ti->set; + strlcpy(tc->tablename, name, sizeof(tc->tablename)); + + if (ti->tlvs == NULL) { + tc->no.compat = 1; + tc->no.uidx = ti->uidx; + } + + /* Preallocate data structures for new tables */ + error = alloc_table_state(&tc->state, &tc->xstate, ti->type); + if (error != 0) { + free(tc, M_IPFW); + return (NULL); + } + + return (tc); +} + +static void +free_table_state(void **state, void **xstate, uint8_t type) +{ + struct radix_node_head *rnh; + + switch (type) { + case IPFW_TABLE_CIDR: + rnh = (struct radix_node_head *)(*state); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(state); + + rnh = (struct radix_node_head *)(*xstate); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(xstate); + break; + case IPFW_TABLE_INTERFACE: + rnh = (struct radix_node_head *)(*xstate); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(xstate); + break; + } +} + +static void +free_table_config(struct namedobj_instance *ni, struct table_config *tc) +{ + + if (tc->linked == 0) + free_table_state(&tc->state, &tc->xstate, tc->no.type); + + free(tc, M_IPFW); +} + +/* + * Links @tc to @chain table named instance. + * Sets appropriate type/states in @chain table info. + */ +static void +link_table(struct ip_fw_chain *chain, struct table_config *tc) +{ + struct namedobj_instance *ni; + uint16_t kidx; + + IPFW_UH_WLOCK_ASSERT(chain); + IPFW_WLOCK_ASSERT(chain); + + ni = CHAIN_TO_NI(chain); + kidx = tc->no.kidx; + + ipfw_objhash_add(ni, &tc->no); + chain->tables[kidx] = tc->state; + chain->xtables[kidx] = tc->xstate; + + tc->linked = 1; +} + +/* + * Unlinks @tc from @chain table named instance. + * Zeroes states in @chain and stores them in @tc. + */ +static void +unlink_table(struct ip_fw_chain *chain, struct table_config *tc) +{ + struct namedobj_instance *ni; + uint16_t kidx; + + IPFW_UH_WLOCK_ASSERT(chain); + IPFW_WLOCK_ASSERT(chain); + + ni = CHAIN_TO_NI(chain); + kidx = tc->no.kidx; + + /* Clear state and save pointers for flush */ + ipfw_objhash_del(ni, &tc->no); + tc->state = chain->tables[kidx]; + chain->tables[kidx] = NULL; + tc->xstate = chain->xtables[kidx]; + chain->xtables[kidx] = NULL; + + tc->linked = 0; +} + +/* + * Finds named object by @uidx number. + * Refs found object, allocate new index for non-existing object. + * Fills in @pidx with userland/kernel indexes. + * + * Returns 0 on success. + */ +static int +bind_table(struct namedobj_instance *ni, struct rule_check_info *ci, + struct obj_idx *pidx, struct tid_info *ti) +{ + struct table_config *tc; + + tc = find_table(ni, ti); + + pidx->uidx = ti->uidx; + pidx->type = ti->type; + + if (tc == NULL) { + /* Try to acquire refcount */ + if (ipfw_objhash_alloc_idx(ni, ti->set, &pidx->kidx) != 0) { + printf("Unable to allocate table index in set %u." + " Consider increasing net.inet.ip.fw.tables_max", + ti->set); + return (EBUSY); + } + + pidx->new = 1; + ci->new_tables++; + + return (0); + } + + /* Check if table type if valid first */ + if (tc->no.type != ti->type) + return (EINVAL); + + tc->no.refcnt++; + + pidx->kidx = tc->no.kidx; + + return (0); +} + +/* + * Compatibility function for old ipfw(8) binaries. + * Rewrites table kernel indices with userland ones. + * Works for \d+ talbes only (e.g. for tables, converted + * from old numbered system calls). + * + * Returns 0 on success. + * Raises error on any other tables. + */ +int +ipfw_rewrite_table_kidx(struct ip_fw_chain *chain, struct ip_fw *rule) +{ + int cmdlen, l; + ipfw_insn *cmd; + uint32_t set; + uint16_t kidx; + uint8_t type; + struct named_object *no; + struct namedobj_instance *ni; + + ni = CHAIN_TO_NI(chain); + + set = TABLE_SET(rule->set); + + l = rule->cmd_len; + cmd = rule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &kidx, &type) != 0) + continue; + + if ((no = ipfw_objhash_lookup_idx(ni, set, kidx)) == NULL) + return (1); + + if (no->compat == 0) + return (2); + + update_table_opcode(cmd, no->uidx); + } + + return (0); +} + + +/* + * Checks is opcode is referencing table of appropriate type. + * Adds reference count for found table if true. + * Rewrites user-supplied opcode values with kernel ones. + * + * Returns 0 on success and appropriate error code otherwise. + */ +int +ipfw_rewrite_table_uidx(struct ip_fw_chain *chain, + struct rule_check_info *ci) +{ + int cmdlen, error, ftype, l; + ipfw_insn *cmd; + uint16_t uidx; + uint8_t type; + struct table_config *tc; + struct namedobj_instance *ni; + struct named_object *no, *no_n, *no_tmp; + struct obj_idx *pidx, *p, *oib; + struct namedobjects_head nh; + struct tid_info ti; + + ni = CHAIN_TO_NI(chain); + + /* + * Prepare an array for storing opcode indices. + * Use stack allocation by default. + */ + if (ci->table_opcodes <= (sizeof(ci->obuf)/sizeof(ci->obuf[0]))) { + /* Stack */ + pidx = ci->obuf; + } else + pidx = malloc(ci->table_opcodes * sizeof(struct obj_idx), + M_IPFW, M_WAITOK | M_ZERO); + + oib = pidx; + error = 0; + + type = 0; + ftype = 0; + + ci->tableset = TABLE_SET(ci->krule->set); + + memset(&ti, 0, sizeof(ti)); + ti.set = ci->tableset; + ti.tlvs = ci->tlvs; + ti.tlen = ci->tlen; + + /* + * Stage 1: reference existing tables and determine number + * of tables we need to allocate + */ + IPFW_UH_WLOCK(chain); + + l = ci->krule->cmd_len; + cmd = ci->krule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &ti.uidx, &ti.type) != 0) + continue; + + /* + * Got table opcode with necessary info. + * Try to reference existing tables and allocate + * indices for non-existing one while holding write lock. + */ + if ((error = bind_table(ni, ci, pidx, &ti)) != 0) + break; + + /* + * @pidx stores either existing ref'd table id or new one. + * Move to next index + */ + + pidx++; + } + + if (error != 0) { + /* Unref everything we have already done */ + for (p = oib; p < pidx; p++) { + if (p->new != 0) { + ipfw_objhash_free_idx(ni, ci->tableset,p->kidx); + continue; + } + + /* Find & unref by existing idx */ + no = ipfw_objhash_lookup_idx(ni, ci->tableset, p->kidx); + KASSERT(no!=NULL, ("Ref'd table %d disappeared", + p->kidx)); + + no->refcnt--; + } + + IPFW_UH_WUNLOCK(chain); + + if (oib != ci->obuf) + free(oib, M_IPFW); + + return (error); + } + + IPFW_UH_WUNLOCK(chain); + + /* + * Stage 2: allocate table configs for every non-existent table + */ + + if (ci->new_tables > 0) { + /* Prepare queue to store configs */ + TAILQ_INIT(&nh); + + for (p = oib; p < pidx; p++) { + if (p->new == 0) + continue; + + /* TODO: get name from TLV */ + ti.uidx = p->uidx; + ti.type = p->type; + + tc = alloc_table_config(ni, &ti); + + if (tc == NULL) { + error = ENOMEM; + goto free; + } + + tc->no.kidx = p->kidx; + tc->no.refcnt = 1; + + /* Add to list */ + TAILQ_INSERT_TAIL(&nh, &tc->no, nn_next); + } + + /* + * Stage 2.1: Check if we're going to create 2 tables + * with the same name, but different table types. + */ + TAILQ_FOREACH(no, &nh, nn_next) { + TAILQ_FOREACH(no_tmp, &nh, nn_next) { + if (strcmp(no->name, no_tmp->name) != 0) + continue; + if (no->type != no_tmp->type) { + error = EINVAL; + goto free; + } + } + } + + /* + * Stage 3: link & reference new table configs + */ + + IPFW_UH_WLOCK(chain); + + /* + * Step 3.1: Check if some tables we need to create have been + * already created with different table type. + */ + + error = 0; + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) { + no_n = ipfw_objhash_lookup_name(ni, no->set, no->name); + if (no_n == NULL) + continue; + + if (no_n->type != no->type) { + error = EINVAL; + break; + } + + } + + if (error != 0) { + /* + * Someone has allocated table with different table type. + * We have to rollback everything. + */ + IPFW_UH_WUNLOCK(chain); + + goto free; + } + + + /* + * Finally, attach tables and rewrite rule. + * We need to set table type for each new table, + * so we have to acquire main WLOCK. + */ + IPFW_WLOCK(chain); + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) { + no_n = ipfw_objhash_lookup_name(ni, no->set, no->name); + if (no_n != NULL) { + /* Increase refcount for existing table */ + no_n->refcnt++; + /* Keep oib array in sync: update kindx */ + for (p = oib; p < pidx; p++) { + if (p->kidx == no->kidx) { + p->kidx = no_n->kidx; + break; + } + } + + continue; + } + + /* New table. Attach to runtime hash */ + TAILQ_REMOVE(&nh, no, nn_next); + + link_table(chain, (struct table_config *)no); + } + IPFW_WUNLOCK(chain); + + /* Perform rule rewrite */ + l = ci->krule->cmd_len; + cmd = ci->krule->cmd; + cmdlen = 0; + pidx = oib; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &uidx, &type) != 0) + continue; + update_table_opcode(cmd, pidx->kidx); + pidx++; + } + + IPFW_UH_WUNLOCK(chain); + } + + error = 0; + + /* + * Stage 4: free resources + */ +free: + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) + free_table_config(ni, tc); + + if (oib != ci->obuf) + free(oib, M_IPFW); + + return (error); +} + +/* + * Remove references from every table used in @rule. + */ +void +ipfw_unbind_table_rule(struct ip_fw_chain *chain, struct ip_fw *rule) +{ + int cmdlen, l; + ipfw_insn *cmd; + struct namedobj_instance *ni; + struct named_object *no; + uint32_t set; + uint16_t kidx; + uint8_t type; + + ni = CHAIN_TO_NI(chain); + + set = TABLE_SET(rule->set); + + l = rule->cmd_len; + cmd = rule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &kidx, &type) != 0) + continue; + + no = ipfw_objhash_lookup_idx(ni, set, kidx); + + KASSERT(no != NULL, ("table id %d not found", kidx)); + KASSERT(no->type == type, ("wrong type %d (%d) for table id %d", + no->type, type, kidx)); + KASSERT(no->refcnt > 0, ("refcount for table %d is %d", + kidx, no->refcnt)); + + no->refcnt--; + } +} + + +/* + * Removes table bindings for every rule in rule chain @head. + */ +void +ipfw_unbind_table_list(struct ip_fw_chain *chain, struct ip_fw *head) +{ + struct ip_fw *rule; + + while ((rule = head) != NULL) { + head = head->x_next; + ipfw_unbind_table_rule(chain, rule); + } +} + + /* end of file */ --------------090406000404070502000405-- From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 14:51:22 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDA47408 for ; Sun, 1 Jun 2014 14:51:22 +0000 (UTC) Received: from m50-111.126.com (m50-111.126.com [123.125.50.111]) by mx1.freebsd.org (Postfix) with ESMTP id 57BAA22E3 for ; Sun, 1 Jun 2014 14:51:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=From:Subject:Date:Message-ID:MIME-Version; bh=OpZ6U OX0hfsiSQHfzCLPc0ZIhXe8x84sTuouLf15Uco=; b=apxLpyfDUiffaDVcxcQjF iLvVFxdL0s7noiAbrjSqg5R9iCSrXNYu9nWPJwRScNnwNg2Q/t4JxI3MI9qUF9sE E9o5k+QgFaTAx3lDek5bxxtG5NQNffyeEHb48F0m2WH4ndJHH8s8bnE2F27jHsiP 3L0GSKlA+gNrhMh50qQ0RU= Received: from dshPC (unknown [111.161.77.200]) by smtp5 (Coremail) with SMTP id jtKowEB5XFTlPYtTUTIKAA--.1493S2; Sun, 01 Jun 2014 22:51:17 +0800 (CST) From: "dongshan" To: References: In-Reply-To: Subject: problem of netmap running on PowerPC platform board Date: Sun, 1 Jun 2014 22:51:19 +0800 Message-ID: <006501cf7da8$f2f23110$d8d69330$@126.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQNTdI7Zy6vugcn83UKu5UbBrvA+6ZhUaGbQ Content-Language: zh-cn X-CM-TRANSID: jtKowEB5XFTlPYtTUTIKAA--.1493S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7WrWfuw43Zw1Dtw13uF4xtFb_yoW8ZF1Dpr ZrKr9akF4kXrWxKr97Kr48AF1S9r93AFW5Xw1IvayFy3W5WF1xXrs2kw15W3Z5Cwn3KF1j yFnFg3yxtas8XaUanT9S1TB71UUUUUJqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UEkskUUUUU= X-Originating-IP: [111.161.77.200] X-CM-SenderInfo: 5wkrztxv1d0warsqlqqrswhudrp/1tbitBedxUX9rCezqwAAsQ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 14:51:22 -0000 Hi, very good work you have done, thank you for sharing the source code of netmap. These days I am trying to test netmap performance on PowerPC 32 bit board, followed the guide, I rebuilt the kernel and netmap modules, then I tested pkt-gen. when configured it as RX mode, it seems work fine. But when I test the TX mode, segmentation fault error occurs. Am sure the procedures I did was right, as I followed the procedures as what I did on X86 platform. Here is the info: root@p4080ds:~/netmap_modules# ./pkt-gen -i eth0 -f tx -n 500111222 -l 60 -w 10 886.299155 main [1624] interface is eth0 886.299712 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 886.299731 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 886.517024 main [1807] mapped 334980KB at 0x48003000 Sending on netmap:eth0: 1 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff) 886.517087 main [1885] Sending 512 packets every 0.000000000 s 886.517099 main [1887] Wait 10 secs for phy reset 896.517259 main [1889] Ready... 896.517293 nm_open [457] overriding ifname eth0 ringid 0x0 flags 0x1 896.517428 sender_body [996] start Segmentation fault After carefully debugging, I found where the error occurs. In pkt-gen.c file, line 691: . } else if (options & OPT_MEMCPY) { memcpyu(frame, p, size); if ( fcnt == nfrags) . The 'p' address in the code is out of range of the mmap(). Then I traced it and found the root error but I don't know how to correct it. I used GDB to debug the code, in netmap_user.h file: 525, I added info of r->ofs as: . for(i = 0; i <+ d->req.nr_tx_rings, i++) { struct netmap_ring *r = NETMAP_TXRING(d->nifp, i); D("TX%d %p h %d c %d t %d ofs 0x%lx", i, r, r->head, r->cur, r->tail, r->buf_ofs); } . The red is what I added. It outcomes the 'ofs' is 0x8000000, however 0x800 is expected. The board I tested is PowerPC big endian 32 bit board, kernel: linux 3.8. I am not whether it effects the outcome. Best regards, Dongshan From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 15:17:50 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E16FDD7E for ; Sun, 1 Jun 2014 15:17:49 +0000 (UTC) Received: from m50-111.126.com (m50-111.126.com [123.125.50.111]) by mx1.freebsd.org (Postfix) with ESMTP id EBFA324AF for ; Sun, 1 Jun 2014 15:17:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=From:Subject:Date:Message-ID:MIME-Version; bh=945N0 A10sDDfTjQoTI3p+YNxFLQpDvbmlEGr5xu16tw=; b=qS5rAlymgyICW2QPfD6MS jzuMiPTEUSyNdIbajbEg7uk9cRuiFZg2iXOShQy4eBR6Ww7aoJLcLyWMAF8c4C8L u86M1a1owhI1q5RIwCQno3ZbOF8rhbkFeZS7amkA817Q8e6lfP4eCgaMxH2rSev7 /8nVTXLfAYyyYQOg8IXymA= Received: from dshPC (unknown [111.161.77.200]) by smtp5 (Coremail) with SMTP id jtKowECppUcHPYtT2xgKAA--.319S2; Sun, 01 Jun 2014 22:47:35 +0800 (CST) From: "dongshan" To: Subject: problem of netmap running on PowerPC platform board Date: Sun, 1 Jun 2014 22:47:37 +0800 Message-ID: <005601cf7da8$6ea46900$4bed3b00$@126.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: Ac99oTYeh8bRSeeiRiSjFBMCDQe6kw== Content-Language: zh-cn X-CM-TRANSID: jtKowECppUcHPYtT2xgKAA--.319S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7WrWfuw43Zw1Dtw13uF4xtFb_yoW8ZrW5pr ZrKr9akF4vqrWxKr97Kr48AF1S9r93AFW5Xw1IvayFy3Z8WF1xXrs2kw15W3Z5uwn3KF1j yFnFg3yxtas8XaUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UieOAUUUUU= X-Originating-IP: [111.161.77.200] X-CM-SenderInfo: 5wkrztxv1d0warsqlqqrswhudrp/1tbitQqdxUX9qlld+gAAsL Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 15:17:50 -0000 Hi, very good work you have done, thank you for sharing the source code of netmap. These days I am trying to test netmap performance on PowerPC 32 bit board, followed the guide, I rebuilt the kernel and netmap modules, then I tested pkt-gen. when configured it as RX mode, it seems work fine. But when I test the TX mode, segmentation fault error occurs. Am sure the procedures I did was right, as I followed the procedures as what I did on X86 platform. Here is the info: root@p4080ds:~/netmap_modules# ./pkt-gen -i eth0 -f tx -n 500111222 -l 60 -w 10 886.299155 main [1624] interface is eth0 886.299712 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 886.299731 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 886.517024 main [1807] mapped 334980KB at 0x48003000 Sending on netmap:eth0: 1 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff) 886.517087 main [1885] Sending 512 packets every 0.000000000 s 886.517099 main [1887] Wait 10 secs for phy reset 896.517259 main [1889] Ready... 896.517293 nm_open [457] overriding ifname eth0 ringid 0x0 flags 0x1 896.517428 sender_body [996] start Segmentation fault After carefully debugging, I found where the error occurs. In pkt-gen.c file, line 691: . } else if (options & OPT_MEMCPY) { memcpyu(frame, p, size); if ( fcnt == nfrags) . The 'p' address in the code is out of range of the mmap(). Then I traced it and found the root error but I don't know how to correct it. I used GDB to debug the code, in netmap_user.h file: 525, I added info of r->ofs as: . for(i = 0; i <+ d->req.nr_tx_rings, i++) { struct netmap_ring *r = NETMAP_TXRING(d->nifp, i); D("TX%d %p h %d c %d t %d ofs 0x%lx", i, r, r->head, r->cur, r->tail, r->buf_ofs); } . The red is what I added. It outcomes the 'ofs' is 0x8000000, however 0x800 is expected. The board I tested is PowerPC big endian 32 bit board, I am not whether it effects the outcome. Best regards, Dongshan From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 15:21:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50D45E3A; Sun, 1 Jun 2014 15:21:57 +0000 (UTC) Received: from mail-qg0-x236.google.com (mail-qg0-x236.google.com [IPv6:2607:f8b0:400d:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0422E2531; Sun, 1 Jun 2014 15:21:56 +0000 (UTC) Received: by mail-qg0-f54.google.com with SMTP id q108so9045256qgd.41 for ; Sun, 01 Jun 2014 08:21:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=mm0FbfX7Iq8bd9dkoPFO8NwZ+MHsXpC5vrNfjC2R/QA=; b=ePoxpmcG8Y5YokOrUBcgUPmDOFrYwd5eGtImVUxBW82eDVcIFqTOtqK0vzOyHtFpJi 1qBpY2iV1x1/Zi0HcKUa/k2k2Yln7HLVYMqVqkc32l4RFtK1eUC6a+A3duG0aom7pXNS pTB7meL2dstePdL8uLH6mMFC8K52VUSoWV99Mh+DCftETfkgjYBmYkKqMYbJMrOsyNrY 6Y6wBJ4YqhaKTDzoOIgCYdkchyR8gKw1O0jx9M2sXkChsv/3shHdYla2rwQLf9PUGooC b8kHamUGwgfw2fz+8DpIFMUqJBCUMSIcgUprfQ2lVX6YsnlcEhMv+wRa01vFcqDqEEHK 2NUg== MIME-Version: 1.0 X-Received: by 10.140.35.212 with SMTP id n78mr38284529qgn.87.1401636115836; Sun, 01 Jun 2014 08:21:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 1 Jun 2014 08:21:55 -0700 (PDT) Date: Sun, 1 Jun 2014 08:21:55 -0700 X-Google-Sender-Auth: 339m2wbqWM-d-pP9UDSMZlHRUuI Message-ID: Subject: [rfc] TCP timewait and credential handling - why would we get a TW with no credential? From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 15:21:57 -0000 Hi, I've been seeing this panic under high very short-term connection TCP throughput tests (30,000 connections a second) with SO_REUSEPORT: Current language: auto; currently minimal (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) print tw $1 = (struct tcptw *) 0xfffff801348b5780 (kgdb) print tw->tw_cred $2 = (struct ucred *) 0x0 (kgdb) and: #10 0xffffffff808d017e in crfree (cr=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_prot.c:1837 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 #12 0xffffffff80a6bc53 in tcp_twcheck (inp=, to=, th=, m=, tlen=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:425 #13 0xffffffff80a5cf3f in tcp_input (m=, off0=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_input.c:949 #14 0xffffffff809fa4e7 in ip_input (m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/ip_input.c:729 #15 0xffffffff80998441 in netisr_dispatch_src (proto=, source=, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #16 0xffffffff80990173 in ether_demux (ifp=, m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:775 #17 0xffffffff80990dce in ether_nh_input (m=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:582 #18 0xffffffff80998441 in netisr_dispatch_src (proto=, source=, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #19 0xffffffff809903c6 in ether_input (ifp=, m=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:683 #20 0xffffffff804e9d67 in igb_rxeof (count=99) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:4882 #21 0xffffffff804ea47f in igb_msix_que (arg=0xfffff8000d47e670) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:1626 #22 0xffffffff808aef03 in intr_event_execute_handlers (p=, ie=0xfffff8000f76cb00) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1263 #23 0xffffffff808af866 in ithread_loop (arg=0xfffff8000f908d20) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1276 #24 0xffffffff808acc41 in fork_exit (callout=0xffffffff808af7d0 , arg=0xfffff8000f908d20, frame=0xfffffe1046c2cac0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_fork.c:977 #25 0xffffffff80c939ae in fork_trampoline () at /usr/home/adrian/git/github/erikarn/freebsd/sys/amd64/amd64/exception.S:605 #26 0x0000000000000000 in ?? () ... there's only one path to deleting the credentials and that's via tcp_twclose() -> tcp_tw_2msl_stop(). Has anyone seen this kind of problem before? -a From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 15:23:16 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A72FF73; Sun, 1 Jun 2014 15:23:16 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C08F42549; Sun, 1 Jun 2014 15:23:15 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1Wr3gc-000NJ2-1I; Sun, 01 Jun 2014 15:12:22 +0400 Message-ID: <538B4502.8080700@FreeBSD.org> Date: Sun, 01 Jun 2014 19:21:38 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Bill Yuan , Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Luigi Rizzo , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 15:23:16 -0000 On 22.05.2014 08:02, Bill Yuan wrote: > Sorry I am little bit blur now. And I am going to wait for your code, I Hello Bill. It looks like table changes are not going very fast. However, if you are still going to add mac address tables to ipfw, it is fine to use current syntax and current opcodes. I'd be happy to convert/merge the changes as soon as new api becomes available. > think it will be a good opportunity to learn as a newbie here. > > 1. So use alphanumeric strings for table's id is not a good way. (because > will loose atomicity). I agree that, all this feature/function, I would > like to name them as utilities. It can be more user friendly with these > utilities. > > 2. And instead, you are going to introduce a IP_FW3_ADD and ... > >> It looks like we can do the following: >> 1) Add another IP_FW3_ADD opcode with the following layout: >> >> { >> rule len; >> unmodified rule itself; >> tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} >> tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} >> .. >> } >> Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be >> real values > So that is the "database" or "mapping table" which I mentioned. So are you > going to translate the name to id before calling kernel-space methods? > > > > > > > > On Thu, May 22, 2014 at 4:48 AM, Luigi Rizzo wrote: > >> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov wrote: >>> On 21.05.2014 15:10, Luigi Rizzo wrote: >>>> On Mon, May 19, 2014 at 04:51:08PM +0400, Alexander V. Chernikov wrote: >>>>> Hello list! >>>>> >>>>> This patch adds ability to name tables / reference them by name. >>>>> Additionally, it simplifies adding new table types. >>>> Hi Alex, >>>> at a high level, i think your changes here are in the right direction. >>> Hello! >>>> However i think part of the design, and also of the patch below, >>>> is not sound/correct so i would like to wait to commit at least >>>> until some of the problems below are resolved. >>>> >>>> 1. The patch as is has several issues (variable declarations in the >>>> middle of a block, assignment in conditionals, incorrect >>>> comments in cut&pasted code, missing checks on strings) >>>> that should be corrected. >>> ..missing documentation and so on :) >>> Of course, I'll fix all these (or most of these :)) >> good thanks >> >>>> 3. there is no explanation, here or in the code, on how the >>>> names are managed. I could try to figure out from the code >>>> but it would be the wrong way to understand things so let me >>>> ask, and please explain what you have in mind. >>> Currently it is very simple non-resizable hash table with fnv hash based >>> on table name. >> that is not an issue. The question is whether one needs to lookup >> the hash table every time you have a 'table' argument (of course i >> think one should not, and the implementation i propose below gives >> direct access to the table without name lookups, as the internal >> identifier is still poiniter or small integer, just one that is not exposed >> to userspace). >> >>>> Let me address first the name <-> table-id thing. >>>> >>>> Introducing a symbolic name for tables is a great and useful feature. >>>> However the implementation has some tricky issues: >>>> >>>> - do you want to retain the old numeric identifiers or not ? >>>> I think it is a bad idea to have two namespaces for the same thing, >>>> so if we are switching to alphanumeric strings for tables we should >>>> as well get rid of the numbers (i.e. consider them as strings as >> well). >>>> I am strongly in favour of not using names as aliases for numbers. >>>> It would require no changes for clients issuing ipfw commands >>>> from a script, and would not require users to to manually handle >>>> the name-id translations. >>> Well. I'd prefer not to. However, code we're discussing assumes that >>> numeric ids >>> are primary ones (e.g. you can't have named, but unnumbered table, >>> you have to choose number by yourself). Switching to named-only tables >>> can be tricky since we don't want to match them by inside rules and we >>> don't want >>> to loose atomicity when allocating table ids via separate cmds before >>> adding rule. >> i think this is solved by the implementation i proposed below. >> >>> It looks like we can do the following: >>> 1) Add another IP_FW3_ADD opcode with the following layout: >>> >>> { >>> rule len; >>> unmodified rule itself; >>> tlv1 {type=table name;len=..;id=1;"_TABLENAME11_"} >>> tlv2 {type=table name;len=..;id=2;"_TABLENAME4_"} >>> .. >>> } >>> Values inside appropriate opcodes { O_IP_XXX_LOOKUP and so on } won't be >>> real values >>> but values described in attached TLVs. >>> >>> After validating/parsing/checking under UH lock we replace fake values >>> with real ones (probably, newly allocated) >>> and return or rollback atomically. >> yes this should work, probably not even requiring a new IP_FW3_ADD opcode >> because the extra tlv entries can appear in a block by themselves. >> >>> The same thing can be done for displaying ruleset, however I'd prefer >>> client to ask for table names and caching them while displaying. >>> >>> Old clients will retain the ability to operate on tables but will see >>> table ids, so nothing will change for them >>> (except the case when new binary adds table "5" and new binary sees id >>> which is not 5, but that shouldn't be a problem). >> we can solve this by using 'low' numbers for the numeric tables >> (these were limited anyways) and allocate the fake entries in >> another range. >> >>>> - The rename command worries me a lot: >>>> >>>> > ipfw table name XXX >>>> > Names (or renames) table. Not the name has to be unique. >>>> >>>> because it is neither clear nor intuitive whether you want to >>>> 1. just rename a table (possibly breaking or creating references >>>> for rules in the firewall), or >>>> 2. modify the name-id translation preserving existing pointers. >>>> >>>> Consider the sequence >>>> ipfw table 13 name bar >>>> ipfw add 100 count dst-ip table bar >>>> ipfw table 13 name foo >>>> ipfw table 14 name bar >>>> ipfw add 200 count src-port 22 dst-ip table bar >>>> >>>> Approach #1 would detach rule 100 from table 13 and then connect >> to 14 >>>> (in a non-atomic way), whereas approach #2 would make rule 100 and >> 200 >>>> point to different tables (which is confusing). >>>> >>>> Now part of this problem goes away if we get rid of number >> altogether. >>>> You may legitimately want to swap tables in an atomic way (we have >> something >>>> similar for rulesets): >>>> ipfw set swap number number >>> There is some problem here: >>> Atomic ruleset changes is a great thing, but tables have no atomic >> support. >>> We, for example, are solving this by using different table range on >>> every new configuration. >>> It won't be possible with named-only tables (since names usually care >>> some semantic in contrast to numbers). >>> The only thing I can think of is separate namespace per each set. >>> What do you think? >> maybe i am missing some detail but it seems reasonably easy to implement >> the atomic swap -- and the use case is when you want to move from >> one configuration to a new one: >> ipfw table foo-new flush // clear initial content >> ipfw table foo-new add ... >> ipfw table swap foo-current foo-new // swap the content of the >> table objects >> >> so you preserve the semantic of the name very easily. >> >>>> So here is what i would propose: >>>> - [gradually] introduce new commands that accept strings for every >> place where >>>> a number was previously accepted. Internally in the firewall, the >> old >>>> 16-bit number is interpreted as a string. >>> Yup. >>>> - within the kernel create a small set of functions (invoked on >> insert, list, delete) >>>> that do proper checks on the string and translate it to a pointer >> (or short integer, >>>> i.e. an index in an array) to the proper object. Done properly, we >> can reuse >>>> this code also for pipes, sets, nat groups and whatnot. >>> Yup. >>>> When a rule references an object that does not exist, you create an >> empty >>>> object that a subsequent table/pipe/XXX 'create' would initialize. >>>> On 'destroy', no need to touch the ruleset, just clear the object. >>> Yes. This looks better than requiring user to create table of given type >>> _before_ >>> referencing it. >>>> - for renames, try to follow the approach used for sets i.e. >>>> ipfw table rename old-name new-name >>>> changes the name, preserves the object. >>>> Does not touch the ruleset, only the translation table >>> Well, I'm not sure about renaming. I'd prefer permitting renaming iff >>> table is not referenced. >>> (and why do we need to rename tables?) >> you are right, let's forget it. And 'ipfw set move' actually does a >> different thing >> which has no use here. >> >>>> ipfw table swap first-table second-table >>>> atomically swap the content of the two table objects >>>> (once again not touching the rulesets) >> cheers >> luigi >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 18:59:31 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76D6D18D for ; Sun, 1 Jun 2014 18:59:31 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06C6226B8 for ; Sun, 1 Jun 2014 18:59:30 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id x48so4050917wes.8 for ; Sun, 01 Jun 2014 11:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=737o6sXUAm0wkvCHjH5ddh/lAhxk5gv2aBe5ZhLObao=; b=tN5lyimps62RkuJu+XWE1sriHwwByM1U4mstQa7CsiJ1flQWiE0W7GXOX1S/6Z2ZTW L3sCAVbJ/nr9ALGHedi9EHleLXD6Kd7X95uhPZjOJg/g08nrHLhuyAqQBxZ+DupVH/qI w4ahjr7kcT0xWTClfCCHYPQogootJC4ul3cgpNvdmfxDzpqiPsZvzsGAjUDp9pvkybxs QxtDNYqWxn7uN0fEkGHP4iCFRyBz2+4VVbRAeD7AfWHxTBoSf0lZA5jp2liy0rKXohOL ZDRZqn8AzRZBAl3m7pBaiOKw4B+ys5dHA4yWI9oI8IZ5yiRhqOeHnKmUmzf0oYL8Wz1k fQ+g== MIME-Version: 1.0 X-Received: by 10.194.246.234 with SMTP id xz10mr42782051wjc.77.1401649169260; Sun, 01 Jun 2014 11:59:29 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Sun, 1 Jun 2014 11:59:29 -0700 (PDT) In-Reply-To: <006501cf7da8$f2f23110$d8d69330$@126.com> References: <006501cf7da8$f2f23110$d8d69330$@126.com> Date: Sun, 1 Jun 2014 20:59:29 +0200 X-Google-Sender-Auth: ZcKW9ividbqFiLJy4yvvD0RvhUc Message-ID: Subject: Re: problem of netmap running on PowerPC platform board From: Luigi Rizzo To: dongshan Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 18:59:31 -0000 On Sunday, June 1, 2014, dongshan wrote: > Hi, > > > > very good work you have done, thank you for sharing the source code of > netmap. > > > > These days I am trying to test netmap performance on PowerPC 32 bit board, > followed the guide, I rebuilt the kernel and netmap modules, then I tested > pkt-gen. when configured it as RX mode, it seems work fine. But when I test > the TX mode, segmentation fault error > As you probably figured out, it is an endianness issue (I have not had a > chance to test netmap on a bigendian machine). To further track down the problem could you try and see if using vale:xx as interface name still gives a segfault ? This would help me identify what part of the code to look at. Also what network interface are you using (nic model/driver) ? Thanks Luigi > > occurs. Am sure the procedures I did > was right, as I followed the procedures as what I did on X86 platform. > > > > Here is the info: > > root@p4080ds:~/netmap_modules# ./pkt-gen -i eth0 -f tx -n 500111222 -l 60 > -w > 10 > 886.299155 main [1624] interface is eth0 > 886.299712 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 > 886.299731 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 > 886.517024 main [1807] mapped 334980KB at 0x48003000 > Sending on netmap:eth0: 1 queues, 1 threads and 1 cpus. > 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff) > 886.517087 main [1885] Sending 512 packets every 0.000000000 s > 886.517099 main [1887] Wait 10 secs for phy reset > 896.517259 main [1889] Ready... > 896.517293 nm_open [457] overriding ifname eth0 ringid 0x0 flags 0x1 > 896.517428 sender_body [996] start > Segmentation fault > > > > After carefully debugging, I found where the error occurs. In pkt-gen.c > file, line 691: > > . > > } else if (options & OPT_MEMCPY) { > > memcpyu(frame, p, size); > > if ( fcnt == nfrags) > > . > > The 'p' address in the code is out of range of the mmap(). Then I traced it > and found the root error but I don't know how to correct it. I used GDB to > debug the code, in netmap_user.h file: 525, I added info of r->ofs as: > > . > > for(i = 0; i <+ d->req.nr_tx_rings, i++) { > > struct netmap_ring *r = NETMAP_TXRING(d->nifp, i); > > D("TX%d %p h %d c %d t %d ofs 0x%lx", i, r, r->head, r->cur, > r->tail, > r->buf_ofs); > > } > > . > > The red is what I added. It outcomes the 'ofs' is 0x8000000, however 0x800 > is expected. > > The board I tested is PowerPC big endian 32 bit board, kernel: linux 3.8. I > am not whether it effects the outcome. > > > > Best regards, > > Dongshan > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 20:24:18 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 803E4C94; Sun, 1 Jun 2014 20:24:18 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F5062DAA; Sun, 1 Jun 2014 20:24:18 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id j5so1627614qaq.15 for ; Sun, 01 Jun 2014 13:24:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6XFQUeYcqI1JQRHIKH/xsgRgdfwKT5nARMxAaRJjEYA=; b=OsjWr5Hp2kgxQTarIWcblneofHtDk08tPxe9jJbYFyvlHu7M4/ukMYb4KAHF6v/m8h UK8+l3CkTdqACTp/601qChjOIrNWfGrWC0hWjX7NLwxuTKKqXrAHxni65wN1OOvbRR7v 56nRQABQ9+eUEBgTpm7HAI+4vQEK7C+ZE6gjQxYzd0E6msr2euptBcyeENddNas1NIHP d/N1CsBz5ogfV7uuecRSHr7M9r05/LNni4z6qDIQpeld64vCtLb6P+hZnsymmz6UqTOs cvupeT/dknFxtb7jVUVj/2I79h+M0dTBMutLiq7zpnphyHywBcfLJuuwmeOATxxfTq46 aiCA== MIME-Version: 1.0 X-Received: by 10.140.98.234 with SMTP id o97mr40978878qge.35.1401654257150; Sun, 01 Jun 2014 13:24:17 -0700 (PDT) Received: by 10.96.122.133 with HTTP; Sun, 1 Jun 2014 13:24:16 -0700 (PDT) In-Reply-To: References: <533BF902.7010506@sfc.wide.ad.jp> Date: Sun, 1 Jun 2014 13:24:16 -0700 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: hiren panchasara To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Cc: Adrian Chadd , Midori Kato , "Eggert, Lars" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 20:24:18 -0000 On Sun, Jun 1, 2014 at 2:08 AM, Luigi Rizzo wrote: > > > On Sunday, June 1, 2014, hiren panchasara > wrote: >> >> On Tue, Apr 8, 2014 at 9:14 PM, hiren panchasara >> wrote: >> > On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd wrote: >> >> Hi! Cool! can you file a FreeBSD PR with this? >> > >> > I'm testing this patch right now. >> > >> > I will make sure it doesn't get lost. :-) >> >> Committed as r266941. >> > > I don't think we need the DNOLD_IS_ECN flag and translation. That stuff is > meant for old binaries Thanks for pointing it out. Committed as r266955. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sun Jun 1 21:19:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0EE7E9C7; Sun, 1 Jun 2014 21:19:06 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 40CF62164; Sun, 1 Jun 2014 21:19:04 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAGKYi1ODaFve/2dsb2JhbABZg1lYgmy4LYZoUQGBIHSCJQEBAQMBAQEBICsgBgUFFhgCAg0ZAikBCRgBDQYIBwQBHAICiBkIDbExpC0XgSqMPQkQAgEbATMHgnWBSwSXHIQikW+DVCE0gT4 X-IronPort-AV: E=Sophos;i="4.98,952,1392181200"; d="scan'208";a="125575012" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 01 Jun 2014 17:18:30 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 0F161B4045; Sun, 1 Jun 2014 17:18:30 -0400 (EDT) Date: Sun, 1 Jun 2014 17:18:30 -0400 (EDT) From: Rick Macklem To: John Howie Message-ID: <867385953.9728745.1401657510047.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Patches for BOOTP/DHCP code to support Windows Server DHCP MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org, freebsd-questions@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 21:19:06 -0000 John Howie wrote: > Hi Rick, >=20 > That is an excellent point and a good debate to have. >=20 > I have not looked in detail at how PXEBOOT does it, but I think a > clean up of the code to somehow pass arguments to the kernel is > preferable to having a diskless client send a slew of needless > requests to the DHCP server to request information already requested > and obtained in previous stages of the boot process. The number of > DHCP requests made by a client when using U-Boot and ubldr is > dizzying. The NFS code will look to see if the boot loader > configuration file has specified the root filesystem through use of > vfs.root.mountfrom, so something should be possible. >=20 Well, pxeboot uses a standalone nfs client to do all the things that sys/nfs/bootp_subr.c does. I bothered to add NFSv3 to this stand alone nfs client, but there is no way I will be doing NFSv4 for it. (Although I don't see much use for an NFSv4 root fs, a couple of people have already asked about it. To do it, modifying the NFSv4 kernel client code to use the stuff in bootp_subr.c is definitely the way to go.) I have no intention of removing the pxeboot stuff, but I do suspect many would be surprised to see they are still using NFSv2 (with a performance penalty) for the diskless root fs over pxeboot. (If you have a pxeboot built from sources post-r212125 Sep. 2010, it can use NFSv3, but I think you have to specify the "nfsv3" option in the line for "/" in the /etc/fsta= b in the rootfs being exported to the client.) Btw, if you are running a fairly recent system with a diskless NFS root fs, you might want to type "nfsstat -m" to see what it is actually using for th= e root fs. It is mainly the difficulties w.r.t. maintaining lib/libstand/nfs.c which m= akes sys/nfs/bootp_subr.c preferable from my point of view. Anyhow, have fun with it, rick > Another area for possible attention is handling the scenario when > there are a number of DHCP servers able to reply to requests. This > can happen when you have multiple NICs on segments with DHCP Servers > or where you have multiple servers on the same segment. The current > code just grabs the first server to reply at each stage (going > through each NIC in turn) and has affinity to it for the remainder > of that stage but not through the entire boot process. >=20 > Regards, >=20 > John >=20 > Sent from my iPhone >=20 > > On Jun 1, 2014, at 19:01, "Rick Macklem" > > wrote: > >=20 > > John Howie wrote: > >> Hi all, > >>=20 > >> I apologize for the cross posting of this email, but I believe it > >> will be > >> of interest to people across all three groups. Please feel free to > >> forward > >> to additional groups if you feel they would benefit. > >>=20 > >> I have seen a few posts on and off over the years about Windows > >> Server > >> DHCP not working with FreeBSD. Specifically, the Windows Server > >> DHCP > >> would > >> not return the boot host name/IP address and the root path. The > >> typical > >> response to =C2=B3why won=C2=B9t it work" has typically been that ther= e is a > >> flaw in > >> Windows Server DHCP code. I did a little digging and found that > >> the > >> problem actually lies in code in FreeBSD. > >>=20 > >> Section 3.5 of RFC 2131 (the DHCP RFC) states that =C2=B3...Second, in > >> its > >> initial DHCPDISCOVER or DHCPREQUEST message, a client may provide > >> the > >> server with a list of specific parameters the client is interested > >> in=C5=A0=C2=B2 > >> and =C2=B3...The client can inform the server which configuration > >> parameters > >> the client is interested in by including the 'parameter request > >> list' > >> option. The data portion of this option explicitly lists the > >> options > >> requested by tag number=C5=A0=C2=B2. A DHCP Server is not required to = return > >> any > >> parameter that a client does not ask for. It appears that the > >> ISC-DHCP > >> server, which is recommended by most, will return configured > >> options > >> regardless of whether or not the client asks for them. > >>=20 > >> There are two places in the FreeBSD codebase that DHCP is used to > >> boot the > >> system over a network. The first is in the boot loader, which uses > >> code in > >> lib/libstand. The second is in the NFS code, in sys/nfs. The code > >> is > >> sys/nfs is not always used if the boot loader sets up the > >> environment > >> for > >> the NFS code, either by passing parameters to the kernel (as > >> PXEBOOT > >> appears to do), or information is configured in the boot loader > >> configuration files, e.g. /boot/loader.rc. > >>=20 > >> I have attached two unified diff files which I ask people to test, > >> before > >> I submit them for inclusion into the codebase as fixes. The first, > >> bootp.c.diff fixes the code in lib/libstand/bootp.c to request the > >> boot > >> host (option 12, aka TAG_HOSTNAME) and the NFS root path (option > >> 17, > >> aka > >> TAG_ROOTPATH). This fix has been tested with PXEBOOT on an amd64 > >> box > >> and > >> ubldr on an ARM/RaspberryPI system. The second, bootp_subr.c.diff, > >> fixes > >> code in sys/nfs/bootp_subr.c to request the same options and also > >> to > >> fix > >> bugs that erroneously reported the IP address of the BOOTP/DHCP > >> server. > >> The code assumed that the BOOTP/DHCP server was also the boot > >> host. > >> Please > >> send me all feedback directly. > >>=20 > >> The diff files work with 10.0-RELEASE through 10.0-RELEASE-p3, but > >> will > >> likely work with 9.0 and also CURRENT and STABLE, including 11.0, > >> as > >> the > >> code is old code that does not appear to have changed in a while. > >> If > >> you > >> want to try it on those systems please, please make sure you have > >> backup > >> copies just in case. > >>=20 > >> If you do not have experience configuring Windows Server DHCP just > >> drop me > >> an email, and I will send you a cheat sheet to get you up and > >> running. > >>=20 > >> I am going to grab the latest ubldr code to see if I can get it to > >> work > >> more like PXEBOOT, that appears to pass parameters to the kernel > >> to > >> avoid > >> the need for the NFS BOOTP/DHCP process. If you test on an ARM > >> system > >> with > >> ubldr in RELEASE you will see a lot of unnecessary network > >> activity > >> going > >> on, that we should be able to fix. > > Actually, I tend to think that using the code in > > sys/nfs/bootp_subr.c > > is preferable to using the NFS stuff in stand that pxeboot does. > >=20 > > The only reason I know for pxeboot doing the NFS stuff and filling > > in > > the nfsv3_diskless structure is historical. (Or that's what most > > people > > use for x86, so it would be a POLA violation if it breaks, if you > > prefer.) > > (Basically, the code in sys/nfs/bootp_subr.c is easier to maintain > > than the > > stand alone NFS client pxeboot uses.) > >=20 > > As such, I don't think this work is necessary, rick > >=20 > >> Regards, > >>=20 > >> John > >>=20 > >> john@thehowies.com (personal) | jhowie@email.arizona.edu > >> (academic) | > >> j.howie@napier.ac.uk (academic) | jhowie@cloudsecurityalliance.org > >> (work) > >>=20 > >>=20 > >>=20 > >>=20 > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to > >> "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Mon Jun 2 02:45:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1122D9C9 for ; Mon, 2 Jun 2014 02:45:45 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE1182C0D for ; Mon, 2 Jun 2014 02:45:44 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 674CF139CA for ; Sun, 1 Jun 2014 23:40:11 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1401676809; x=1402540810; bh=7bR1mp7mqCa4 ZrCm+dxYWU1UnNCnfacjTWXPXpzlyac=; b=crVHbTZumtZxrLY4U1NccbkN3/ec IyVJagxYSXRzSJ4lkizy3N/UmqfuMGekKj6tEL7vaJF8MTpXvZ6DSpzo37idA4ba nMtW54eWp0wlvbcPiQ1qr/26Xr66KgcbkhpwOTlyM/3EkVFcLqAXuywL84pGPk2b K536024YoBGSbNw= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UJEZmm43KExn for ; Sun, 1 Jun 2014 23:40:09 -0300 (BRT) Received: from MacBook-de-Gondim-2.local (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 56A0F139C9 for ; Sun, 1 Jun 2014 23:40:09 -0300 (BRT) Message-ID: <538BE3F9.5040500@bsdinfo.com.br> Date: Sun, 01 Jun 2014 23:39:53 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <53777F09.5030000@FreeBSD.org> In-Reply-To: <53777F09.5030000@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 02:45:45 -0000 Em 17/05/14 12:23, Alexander V. Chernikov escreveu: > On 17.05.2014 19:14, Andreas Nilsson wrote: >> >> >> >> On Sat, May 17, 2014 at 3:44 PM, Alexander V. Chernikov >> > wrote: >> >> On 13.05.2014 16:05, Dennis Yusupoff wrote: >> >> I think that universal table for all kind of data (ipv4, >> ipv6, ports, >> etc) is a bad idea by design. At least unless you haven't any >> ability to >> >> It is not always "universal" in kernel. >> Actually, different radix tables are used to store both IPv4 and >> IPv6 in single table. >> >> specify address family on add, to avoid attempts to guess >> what user >> meant. Something like "ipfw table X add DEEF.DE >> ipv6". >> >> I'm going to add explicit table type/naming setup soon. >> Idea is the following: >> >> 1) Existing table can be named and addressed by either number or >> name. >> However, you still need to assign table number manually. >> >> 2) Table type/name can be specified explicitly via one of the >> following commands: >> * ipfw table 1 create [type ] [name >> "table_name"] >> * ipfw table name "table_name" >> * ipfw table "table_name" type >> >> 3) ipfw(8) stops trying to guess appropriate type based on used >> value. Instead, >> it requests table type from kernel and interprets value according >> to returned type. >> Default type for all tables is cidr >> >> 4) Table(s) can be returned to default values using ipfw table >> destroy. >> Destroy means: >> * flush >> * table tries (or other structures) freed >> * type set to cidr >> >> >> >> >> >> 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >> >> On 13.05.2014 13:46, Dennis Yusupoff wrote: >> >> May be this will help? See answer on >> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >> >> I'll try to fix it within a few days. >> >> Fixed in r266310. >> >> With all of these changes, would it be possible to get tablearg to >> store ipv6 as well? I seem to remember it is 32bit only today. > Well, I'd prefer not to increase tablearg directly. > It is probably possible to implement some kind of "nexthop" table adds > additional auto-filled nexthop array. >> >> Best regards >> Andreas > Could tell if this patch is to be incorporated into the 10-stable? http://svnweb.freebsd.org/base?view=revision&revision=266310 Thanks and good job for all. []'s Gondim From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 01:35:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 090DE11E; Tue, 3 Jun 2014 01:35:02 +0000 (UTC) Received: from ns.kevlo.org (220-135-115-6.HINET-IP.hinet.net [220.135.115.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ns.kevlo.org", Issuer "ns.kevlo.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 95FBF2A4A; Tue, 3 Jun 2014 01:35:00 +0000 (UTC) Received: from ns.kevlo.org (localhost [127.0.0.1]) by ns.kevlo.org (8.14.8/8.14.8) with ESMTP id s531YJl3052881 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 3 Jun 2014 09:34:20 +0800 (CST) (envelope-from kevlo@ns.kevlo.org) Received: (from kevlo@localhost) by ns.kevlo.org (8.14.8/8.14.8/Submit) id s531YI7j052880; Tue, 3 Jun 2014 09:34:18 +0800 (CST) (envelope-from kevlo) Date: Tue, 3 Jun 2014 09:34:18 +0800 From: Kevin Lo To: Jason Hellenthal Subject: Re: [VIMAGE][udplite] FreeBSD 10-STABLE/powerpc Message-ID: <20140603013418.GA52860@ns.kevlo.org> References: <20140529083619.GA8437@ns.kevlo.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-net@freebsd.org" , "adrian@freebsd.org" , "\[FreeBSD Stable\]" , "jhb@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 01:35:02 -0000 On Thu, May 29, 2014 at 10:40:48AM -0400, Jason Hellenthal wrote: > Hi Kevin, Hi Jason, > Default on PowerPC is GCC 4.2.1 > > Its hard to see that this wouldn't turn up elsewhere on other arch' stop though as from what I seen doesn't seem to be dependent on PowerPC alone. > > But to confirm my previous build, after backing out udplite the build did complete just fine. I'll find out in a little while whether it runs :-) crossing fingers. Committed a fix as r266990, thanks. > > -- > Jason Hellenthal > Voice: 95.30.17.6/616 > JJH48-ARIN Kevin From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 02:10:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8B42E1DD for ; Tue, 3 Jun 2014 02:10:32 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B5152D26 for ; Tue, 3 Jun 2014 02:10:31 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id rl12so5355271iec.37 for ; Mon, 02 Jun 2014 19:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=RLh9fu2mIbth8d69G9ppuUNq83SkfFZNC5jUmotBEqA=; b=DYSqq19doFK/O6s6JgrmgpG7lFfkLvfiBKVCbCB5UNoGrA2syKqx3nUzEBHrioabx0 Qe1WcOcU5eyNJS6i8zKl8SpDmoVpwy5gKMfMcso2iI/z/o37JQfoSxR3KXxlyccowx8w FFw+SBOQRljdiZpmK18jOQQj9YWvkisVIMAJg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=RLh9fu2mIbth8d69G9ppuUNq83SkfFZNC5jUmotBEqA=; b=C0e89hDAUlEEsxkIcirFQVwNSI0z1Ym9iShRtQQ8Q6AaMrox8rxLRfBDKEA2XnTKid zaHvH28W3qeguhbglRU3yxi+TSYbucdWISbD8RltkqcSQI5OedqD8U0ITPi+eVWBstbx 654xh3tm3i4/L4EsNKg/TLIFyqtBY4o5MQtUxG3Pc9fB684RZMzbVLxqOUYRHszu/5uF kculN9T51y+3Bss9GMSmefageVNDoZQ7YP/YaKmXQpoV4XFs0oJUaD0dQH7E4s1zvQkt bJQyNpR830ws1D0ee+fP7Uke14XefNWj8Ckt3qWeo/NaA7FRv8iVDY00oVPs7TTUaquN qn5w== X-Gm-Message-State: ALoCoQnsz9hiZBV7jmajuq96+Lst8BtVF3xpwTN9uj5Ftki8mQbl+QeKDhT0wSU+lpzVBIHvR5eO X-Received: by 10.43.65.145 with SMTP id xm17mr38078277icb.44.1401761431333; Mon, 02 Jun 2014 19:10:31 -0700 (PDT) Received: from [172.31.35.2] (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id y7sm33237453igl.13.2014.06.02.19.10.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 02 Jun 2014 19:10:30 -0700 (PDT) References: <20140529083619.GA8437@ns.kevlo.org> <20140603013418.GA52860@ns.kevlo.org> Mime-Version: 1.0 (1.0) In-Reply-To: <20140603013418.GA52860@ns.kevlo.org> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-AB77EF8A-DF2A-42C1-9933-6099EA8CB47C; protocol="application/pkcs7-signature" Content-Transfer-Encoding: 7bit Message-Id: <17AA4EF1-B96B-4E67-AE9D-988C8B2AE08C@dataix.net> X-Mailer: iPhone Mail (11B554a) From: Jason Hellenthal Subject: Re: [VIMAGE][udplite] FreeBSD 10-STABLE/powerpc Date: Mon, 2 Jun 2014 22:10:27 -0400 To: Kevin Lo X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , "adrian@freebsd.org" , "\[FreeBSD Stable\]" , "jhb@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 02:10:32 -0000 --Apple-Mail-AB77EF8A-DF2A-42C1-9933-6099EA8CB47C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Thank you Kevin. Much appreciated. --=20 Jason Hellenthal Voice: 95.30.17.6/616 JJH48-ARIN > On Jun 2, 2014, at 21:34, Kevin Lo wrote: >=20 >> On Thu, May 29, 2014 at 10:40:48AM -0400, Jason Hellenthal wrote: >> Hi Kevin, >=20 > Hi Jason, >=20 >> Default on PowerPC is GCC 4.2.1 >>=20 >> Its hard to see that this wouldn't turn up elsewhere on other arch' stop t= hough as from what I seen doesn't seem to be dependent on PowerPC alone. >>=20 >> But to confirm my previous build, after backing out udplite the build did= complete just fine. I'll find out in a little while whether it runs :-) cro= ssing fingers. =20 >=20 > Committed a fix as r266990, thanks. >=20 >>=20 >> --=20 >> Jason Hellenthal >> Voice: 95.30.17.6/616 >> JJH48-ARIN >=20 > Kevin --Apple-Mail-AB77EF8A-DF2A-42C1-9933-6099EA8CB47C Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIUOTCCBjAw ggUYoAMCAQICAwaijjANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0 YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcx ODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB MB4XDTEzMDUxODA4NTA0OFoXDTE0MDUxOTIyMDk0N1owSDEfMB0GA1UEAwwWamhlbGxlbnRoYWxA ZGF0YWl4Lm5ldDElMCMGCSqGSIb3DQEJARYWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALgnYFS1bWZr3KhKBzWAdRwrY+En+RRV8nCaYubqrMG+ YJbuenaIKSbIuFiDWipW4RHYTpE28pKaSnaVTG9WtAZvsWj0gYN9g2fYCnCOUceES2Yvi3RavxpB hsuzKIfsHb8iNNSEuczLu6gn4mQyaHwE4x6xSUKmbK8njR+YoF522F60wjsnq5dlOJdTrhDfObE5 5P23279WbRp8azgZX1VRB66wdKRDuSI1vBts4Nsha2paXd6HUUduHrPACBQREJTGXN8XtEKVwo63 aKUhRgtUwHNEuSWck/xwVl7PBUWH2dORAWTCqHjNuCKNOQ1/0LMiyMj7FdsBjN4dgL4YZpsCAwEA AaOCAtwwggLYMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDAdBgNVHQ4EFgQU29qUrmZtgQ7ZVoDKogfpJOSfk+YwHwYDVR0jBBgwFoAUU3Ltkpzg 2ssBXHx+ljVO8tS4UYIwIQYDVR0RBBowGIEWamhlbGxlbnRoYWxAZGF0YWl4Lm5ldDCCAUwGA1Ud IASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZp Y2F0aW9uIEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29y ZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRD b20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCug KaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEFBQcBAQSB gTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMS9jbGll bnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8wDQYJ KoZIhvcNAQELBQADggEBAHsw8/Hw07gsNTKYnld74NBFtHnQOPkXYuccWx3j0PGQe9nqNxeingBf 2yvx+xBQzBoi4J1u84Jbrbe8Ii3+LLD/QMW9cN0SBIgRStPQLVee4STdjeabGmpXQa7omC02wYYO 83qh6CgJEIbmrsBSZH8ZSVrjkC4UmZS8wAQMS3qTWAPF0ZQGWx2+Gks2fXuacyt2LpNR+p9ogjAZ 1/rmUKjNhQZLswytaLRUdwAwSfQ3+TNs68h6Kv1LC3bNGBT3NEtr2q/nzzb5MzuFcDE6f9exroAC 4BHmokAprhna/vZdb6BrPjpXgRAlWAh3wEMxw75M9S/Nbzj/jNp+I+lvUJYwggY0MIIEHKADAgEC AgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMSkwJwYDVQQDEyBT dGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQy MTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xh c3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB DwAwggEKAoIBAQDHCYPMzi3YGrEppC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6E RKKnu8zPf1Jwuk0tsvVCk6U9b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1 PKHG/FaR/wpbfuIqu54qzHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvur yGaC/o2/ceD2uYDX9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSM TGKkDiXm6/3/4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNV HRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4 UYIwHwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6Ly93 d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8vd3d3LnN0 YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3Nmc2Nh LmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3 LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3Ns LmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4ICAQAKgwh9eKssBly4Y4xerhy5 I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8pyTUnFsukDFUI22zF5bVHzuJ+GxhnS qN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMmCeUTyMyikfbUFvIBivtvkR8ZFAk22BZy +pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIzuQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4Yj Cl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVmz/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586 YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3 a0LwZrp8MQ+Z77U1uL7TelWO5lApsbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0q ZW2Niy/QvVNKbb43A43ny076khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjx kJh8BYtv9ePsXklAxtm8J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV 27ioRKbj/cIq7JRXun0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCB8kwggWxoAMCAQICAQEw DQYJKoZIhvcNAQEFBQAwfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzAp BgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0 Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzNloXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRpZmlj YXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4 n5V7tTOQ8nJi1sE2ICIkXs7pd/JDCqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZ Z7bEBn0KnnSF1nlMgDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N 3gs7SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5+kbWEyqo uaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2ROFzYh87xL6gU1YlbFEJ 96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWpW+pDDAL/mevobE1wWyllnN2qXcyvATHs DOvSjejqnHvmbvcnZgwaSNduQuM/3iE+e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B 7hIQDcYyfxj//PT6zW6R6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/ kLfjkdLd78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg8nov gdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQTUksCAwEAAaOCAlIw ggJOMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgGuMB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD 0EGu8jBkBgNVHR8EXTBbMCygKqAohiZodHRwOi8vY2VydC5zdGFydGNvbS5vcmcvc2ZzY2EtY3Js LmNybDAroCmgJ4YlaHR0cDovL2NybC5zdGFydGNvbS5vcmcvc2ZzY2EtY3JsLmNybDCCAV0GA1Ud IASCAVQwggFQMIIBTAYLKwYBBAGBtTcBAQEwggE7MC8GCCsGAQUFBwIBFiNodHRwOi8vY2VydC5z dGFydGNvbS5vcmcvcG9saWN5LnBkZjA1BggrBgEFBQcCARYpaHR0cDovL2NlcnQuc3RhcnRjb20u b3JnL2ludGVybWVkaWF0ZS5wZGYwgdAGCCsGAQUFBwICMIHDMCcWIFN0YXJ0IENvbW1lcmNpYWwg KFN0YXJ0Q29tKSBMdGQuMAMCAQEagZdMaW1pdGVkIExpYWJpbGl0eSwgcmVhZCB0aGUgc2VjdGlv biAqTGVnYWwgTGltaXRhdGlvbnMqIG9mIHRoZSBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eSBQb2xpY3kgYXZhaWxhYmxlIGF0IGh0dHA6Ly9jZXJ0LnN0YXJ0Y29tLm9yZy9wb2xpY3ku cGRmMBEGCWCGSAGG+EIBAQQEAwIABzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wg Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkwDQYJKoZIhvcNAQEFBQADggIBABZsmfRmDDT10IVefQrs 2hBOOBxe36YlBUuRMsHoO/E93UQJWwdJiinLZgK3sZr3JZgJPI4b4d02hytLu2jTOWY9oCbH8jmR HVGrgnt+1c5a5OIDV3Bplwj5XlimCt+MBppFFhY4Cl5X9mLHegIF5rwetfKe9Kkpg/iyFONuKIdE w5Aa3jipPKxDTWRFzt0oqVzyc3sE+Bfoq7HzLlxkbnMxOhK4vLMR5H2PgVGaO42J9E2TZns8A+3T mh2a82VQ9aDQdZ8vr/DqgkOY+GmciXnEQ45GcuNkNhKv9yUeOImQd37Da2q5w8tES6x4kIvnxywe SxFEyDRSJ80KXZ+FwYnVGnjylRBTMt2AhGZ12bVoKPthLr6EqDjAmRKGpR5nZK0GLi+pcIXHlg98 iWX1jkNUDqvdpYA5lGDANMmWcCyjEvUfSHu9HH5rt52Q9CI7rvj8Ksr6glKg769LVZPrwbXwIous NE4mIgShhyx1SrflfRPXuAxkwDbSyS+GEowjCcEbgjtzSaNqV4eU5dZ4xZlDY+NN4Hct4WWZcmkE GkcJ5g8BViT7H78OealYLrnECQF+lbptAAY+supKEDnY0Cv1v+x1v5cCxQkbCNxVN+KB+zeEQ2Ig yudWS2Xq/mzBJJMkoTTrBf+aIq6bfT/xZVEKpjBqs/SIHIAN/HKK6INeMYIDbzCCA2sCAQEwgZQw gYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUg RGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFBy aW1hcnkgSW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMAkGBSsOAwIaBQCgggGvMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDYwMzAyMTAyOFowIwYJKoZIhvcN AQkEMRYEFIjDAZ+FZaUsfL1HYLEdSouwhrUbMIGlBgkrBgEEAYI3EAQxgZcwgZQwgYwxCzAJBgNV BAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBD ZXJ0aWZpY2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50 ZXJtZWRpYXRlIENsaWVudCBDQQIDBqKOMIGnBgsqhkiG9w0BCRACCzGBl6CBlDCBjDELMAkGA1UE BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENl cnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRl cm1lZGlhdGUgQ2xpZW50IENBAgMGoo4wDQYJKoZIhvcNAQEBBQAEggEAlDgfWIKoou+V1TPfE5FL k4SGOiyZ05EwYmnkqlDoN0C6PfmWb4MZkluxRmcycv4z3kFuoP74B2JrdmmQOiD42+jfot5rr2UR L/uzZrao7boopQ6JbhkKjonPlh28qEDmwXgwvmYQ7vsgffyyxtJNRPsq6HG8TRgjPi6i2Pp+h6+w d8Ih9RH52V7BAvw3nQT5dYQtt9BE9/dV6qmvzlm6OSq+ESN+ZNdM6ZvR31sVG99LSFK3JLG4s47H OMGV7bBKn942hmxYKN2BwUnNsMqjcw9Li7RpP6+bT+xsAr5chCHtOEihI9PhXKbv/QAVlK3Q5RR+ BTNZTVgrm7R4S8jgIAAAAAAAAA== --Apple-Mail-AB77EF8A-DF2A-42C1-9933-6099EA8CB47C-- From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 10:42:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06ED5DD2 for ; Tue, 3 Jun 2014 10:42:55 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B5E7F298A for ; Tue, 3 Jun 2014 10:42:54 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WrmB2-0006DF-QQ for freebsd-net@freebsd.org; Tue, 03 Jun 2014 12:42:44 +0200 Received: from 217.30.88.7 ([217.30.88.7]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Jun 2014 12:42:44 +0200 Received: from jcharbon by 217.30.88.7 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Jun 2014 12:42:44 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Julien Charbon Subject: Re: [rfc] TCP timewait and credential handling - why would we get a TW with no credential? Date: Tue, 03 Jun 2014 12:42:46 +0200 Lines: 64 Message-ID: <538DA6A6.6040201@verisign.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 217.30.88.7 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 10:42:55 -0000 Hi Adrian, On 01/06/14 17:21, Adrian Chadd wrote: > I've been seeing this panic under high very short-term connection TCP > throughput tests (30,000 connections a second) with SO_REUSEPORT: > > Current language: auto; currently minimal > > (kgdb) frame 11 > > #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) > at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 > > 644 crfree(tw->tw_cred); > > (kgdb) frame 11 > > #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) > at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 > > 644 crfree(tw->tw_cred); > > (kgdb) print tw > > $1 = (struct tcptw *) 0xfffff801348b5780 > > (kgdb) print tw->tw_cred > > $2 = (struct ucred *) 0x0 > [...] > > ... there's only one path to deleting the credentials and that's via > tcp_twclose() -> tcp_tw_2msl_stop(). > > Has anyone seen this kind of problem before? Interesting question: I already got this issue also under short-lived high connection rate (~120k connections per second) and using SO_REUSEPORT _however_ it was with our TCP scaling patches applied (thus it might be a red herring for you) and I fixed it with a: tcp_tw_2msl_stop(): ... if (tw->tw_cred != NULL) crfree(tw->tw_cred); https://github.com/verisign/freebsd/commit/918fe94ad3f3ccd1f6bf577e7e79a138378262a0#diff-4f0c0d9222bfb69f2f358113d424cf89R610 At that time I thought you could create a tw with its tw->tw_cred field set to NULL and then tcp_tw_2msl_stop() simply missed a sanity check before calling crfree(). Now I re-checked how tw->tw_cred is set, its seems impossible to create a tw with tw->tw_cred = NULL (aha!). As this tw_cred field seems well protected by both INP_INFO_WLOCK and INP_WLOCK from race conditions, I am currently checking if something could inadvertently overwrite tw_cred in struct tcptw. My 2 cents. -- Julien From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 12:10:36 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6915FB91 for ; Tue, 3 Jun 2014 12:10:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51CED2214 for ; Tue, 3 Jun 2014 12:10:36 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s53CAauQ070724 for ; Tue, 3 Jun 2014 13:10:36 +0100 (BST) (envelope-from no-reply-bugzilla-daemon@freebsd.org) From: no-reply-bugzilla-daemon@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183659] [tcp] TCP stack lock contention with short-lived connections Date: Tue, 03 Jun 2014 12:10:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jcharbon@verisign.com X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: http://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 12:10:36 -0000 http://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 jcharbon@verisign.com changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|[tcp] ]TCP stack lock |[tcp] TCP stack lock |contention with short-lived |contention with short-lived |connections |connections -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 17:18:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2765C8A4 for ; Tue, 3 Jun 2014 17:18:44 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC17E2197 for ; Tue, 3 Jun 2014 17:18:43 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id i13so5401586qae.35 for ; Tue, 03 Jun 2014 10:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=AsrdcF6LUdXjfmDm2pG9cSdg4Ah6/zJVi5lc6Y1sF4E=; b=oKmhEG1+YrltRTqxb29AET9K64547jVAsW6+/XkKZIAyCYCtyLD/OrryWIEreb4Y4J 1IirF9uCnBXSipP4wX9FutTAVgQ+Zxm2oZI5bFQpJBmQRU7GjoE9cq5OYQUKLJzn4F72 G773BgWdYe9IzBCM2qRtwY1+/V4ZZWOTPouPLAVJGE8QRrgrF5ZI8DOn4YIG1IU2uS0n Zb4oAmsZEyS3JW5hxr0B/Bfd+NYSVwh3Fs/zRUVWOn4ls6zEah8lyJ136ipQJ4XBBW3x Yn0Nnlx+qFetMD5zIsJOnXbVF/tKJ7zIBe8tB9Xh6efjmLi2ObYHUCjc14ZwTLFSzTJg bl/A== MIME-Version: 1.0 X-Received: by 10.224.43.148 with SMTP id w20mr63778066qae.26.1401815922903; Tue, 03 Jun 2014 10:18:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Tue, 3 Jun 2014 10:18:42 -0700 (PDT) In-Reply-To: <538DA6A6.6040201@verisign.com> References: <538DA6A6.6040201@verisign.com> Date: Tue, 3 Jun 2014 10:18:42 -0700 X-Google-Sender-Auth: iYEZRbv9eKHKhJsYg5gOJrwt9Mk Message-ID: Subject: Re: [rfc] TCP timewait and credential handling - why would we get a TW with no credential? From: Adrian Chadd To: Julien Charbon Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 17:18:44 -0000 [snip] Hm, lemme try adding some debugging to the tw_cred = crhold() bit to see if it assigns a NULL pointer. I've noticed that I'm getting NULL capability sets when doing some kqueue stuff and it causes it to return with ENOCAPABLE on newly created sockets that I'm trying to set write readiness on. Maybe this is related. -a From owner-freebsd-net@FreeBSD.ORG Tue Jun 3 21:53:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D672B6D5 for ; Tue, 3 Jun 2014 21:53:51 +0000 (UTC) Received: from mail-ve0-x230.google.com (mail-ve0-x230.google.com [IPv6:2607:f8b0:400c:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 943E02F02 for ; Tue, 3 Jun 2014 21:53:51 +0000 (UTC) Received: by mail-ve0-f176.google.com with SMTP id jz11so7685986veb.21 for ; Tue, 03 Jun 2014 14:53:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ecr/RYdCDUo4x2t3AqgHy4gsZ1PN8+0ezLYnEjf3RR8=; b=KivPHt6+XcvciFXLyv2+YO34JtyH05MEvYKvxzGxbafStFCYvmDospgzvzbiP2gDHL TS0KPpy5fhAFH+OGIkdl0o+zq4Rq+Hhu+cEtHt1hYUkRfZQH/cbIjc2o6b9GUdWks153 lmVyA2xZnJgL8llnHGflSEFtlVN+90ukJjshtnPpUsLQI8lYaxbZukxYhSXTbsPYBe0h WDbSXo5XaSqHRhKDpWLa3fpSmWB/L57XzF76/38mI4EKLNc8PbY7JsbD0ntyTg4h1G92 302HSmqIP9IwhtaOxYx/KWZSVIOaMyxeeTR0LJtj3sqrvTbr5Maky5qkq6myX47JPZ5h D0Qw== MIME-Version: 1.0 X-Received: by 10.52.135.148 with SMTP id ps20mr3822742vdb.75.1401832430625; Tue, 03 Jun 2014 14:53:50 -0700 (PDT) Received: by 10.58.73.102 with HTTP; Tue, 3 Jun 2014 14:53:50 -0700 (PDT) Date: Tue, 3 Jun 2014 17:53:50 -0400 Message-ID: Subject: Query regarding net.inet.tcp.nolocaltimewait From: suraj sandhu To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 21:53:52 -0000 Hi, I am using FreeBsd 8.2 and the behavior I am trying to understand is why there is no final ACK being sent by the client if net.inet.tcp.nolocaltimewait is set. Due to this server side communication need to rely upon receiving the RST for clean-up. Especially in the cases where client port is reused soon after the the connection being closed, additional resets are triggered. 198 void 199 tcp_twstart (struct tcpcb *tp ) 200 { 201 struct tcptw *tw; 202 struct inpcb *inp = tp ->t_inpcb; 203 int acknow; 204 struct socket *so ; 205 206 INP_INFO_WLOCK_ASSERT (&V_tcbinfo ); */* tcp_tw_2msl_reset(). */* 207 INP_WLOCK_ASSERT (inp ); 208 209 if (V_nolocaltimewait && in_localip (inp ->inp_faddr )) { 210 tp = tcp_close (tp );<==== Closing the connection without sending Final ACK. 211 if (tp != NULL ) 212 INP_WUNLOCK (inp ); 213 return; 214 } ... ... I will really appreciate any feedback on it. -Suraj From owner-freebsd-net@FreeBSD.ORG Wed Jun 4 08:00:14 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 612261E3 for ; Wed, 4 Jun 2014 08:00:14 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E97F224A for ; Wed, 4 Jun 2014 08:00:14 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5480EBH049559 for ; Wed, 4 Jun 2014 09:00:14 +0100 (BST) (envelope-from bz-noreply@freebsd.org) Message-Id: <201406040800.s5480EBH049559@kenobi.freebsd.org> From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Wed, 04 Jun 2014 08:00:14 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 08:00:14 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is an experimental feature. Please file a bug or mail bugmeister@ with concerns. This search was scheduled by eadler@FreeBSD.org. (9 bugs) Bug 127360: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=127360 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [socket] TOE socket options missing from sosetopt() Bug 137776: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137776 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [rum] panic in rum(4) driver on 8.0-BETA2 Bug 137841: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137841 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [patch] wpa_supplicant(8) cannot verify SHA256 signed certificates Bug 139204: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139204 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [arp] DHCP server replies rejected, ARP entry lost before max_age Bug 145600: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=145600 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: TCP/ECN behaves different to CE/CWR than ns2 reference Bug 154169: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154169 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [multicast] [ip6] Node Information Query multicast address wrong (FF02::2:x:y) Bug 168294: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168294 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [ixgbe] [patch] ixgbe driver compiled in kernel has no Flow Director support although loaded as a module has one Bug 172113: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=172113 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type Bug 182212: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182212 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 05:37:25 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1126ACC6 for ; Thu, 5 Jun 2014 05:37:25 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED2112F68 for ; Thu, 5 Jun 2014 05:37:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s555bO7D053496 for ; Thu, 5 Jun 2014 06:37:24 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 182212] [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly Date: Thu, 05 Jun 2014 05:37:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.4-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 05:37:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182212 --- Comment #4 from commit-hook@freebsd.org --- A commit references this bug: Author: mav Date: Thu Jun 5 05:36:56 UTC 2014 New revision: 267093 URL: http://svnweb.freebsd.org/changeset/base/267093 Log: MFC r266538: Make ng_mppc to not disable the node in case of multiple packet loss. Quite often it can be just packet reorder, and killing link in such case is inconvenient. Add few sysctl's to control that behavior. PR: kern/182212 Submitted by: Eugene Grosbein Changes: _U stable/10/ stable/10/sys/netgraph/ng_mppc.c -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 05:59:27 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 428E8285 for ; Thu, 5 Jun 2014 05:59:27 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A7732105 for ; Thu, 5 Jun 2014 05:59:27 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s555xRav000786 for ; Thu, 5 Jun 2014 06:59:27 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 182212] [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly Date: Thu, 05 Jun 2014 05:59:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.4-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 05:59:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182212 --- Comment #5 from commit-hook@freebsd.org --- A commit references this bug: Author: mav Date: Thu Jun 5 05:58:55 UTC 2014 New revision: 267094 URL: http://svnweb.freebsd.org/changeset/base/267094 Log: MFC r266538: Make ng_mppc to not disable the node in case of multiple packet loss. Quite often it can be just packet reorder, and killing link in such case is inconvenient. Add few sysctl's to control that behavior. PR: kern/182212 Submitted by: Eugene Grosbein Approved by: re (glebius) Changes: _U stable/9/ _U stable/9/sys/ stable/9/sys/netgraph/ng_mppc.c -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 06:00:28 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A82DB451 for ; Thu, 5 Jun 2014 06:00:28 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9003F211D for ; Thu, 5 Jun 2014 06:00:28 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5560SPP008169 for ; Thu, 5 Jun 2014 07:00:28 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 182212] [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly Date: Thu, 05 Jun 2014 06:00:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.4-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 06:00:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182212 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: mav Date: Thu Jun 5 06:00:08 UTC 2014 New revision: 267095 URL: http://svnweb.freebsd.org/changeset/base/267095 Log: MFC r266538: Make ng_mppc to not disable the node in case of multiple packet loss. Quite often it can be just packet reorder, and killing link in such case is inconvenient. Add few sysctl's to control that behavior. PR: kern/182212 Submitted by: Eugene Grosbein Changes: _U stable/8/ _U stable/8/sys/ _U stable/8/sys/netgraph/ stable/8/sys/netgraph/ng_mppc.c -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 06:02:52 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B23A4528 for ; Thu, 5 Jun 2014 06:02:52 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A38121AA for ; Thu, 5 Jun 2014 06:02:52 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5562qg2051466 for ; Thu, 5 Jun 2014 07:02:52 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 182212] [patch] [ng_mppc] ng_mppc(4) blocks on network errors unconditionaly Date: Thu, 05 Jun 2014 06:02:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.4-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mav@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 06:02:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182212 Alexander Motin changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |mav@FreeBSD.org Resolution|--- |FIXED --- Comment #7 from Alexander Motin --- Patch merged sown to stable/8. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 06:46:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 482423EB for ; Thu, 5 Jun 2014 06:46:54 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B954E2522 for ; Thu, 5 Jun 2014 06:46:53 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id f8so9692133wiw.8 for ; Wed, 04 Jun 2014 23:46:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=cis3Kcv+mBoX2g8959yPHYQnuGi/jgYC9EvodBGNIJ4=; b=SDN08np8iB6ORSRa3fUqDVpAwjkMudDElShRJQckWsAYi1GJSsnGWHXlLjDh2b16D1 ady5UyfvUtC4GXXu/MIPfnXFHUKJyL/kdvNAlBVxF7Gvecwbz8rjE6mJYzHFigX1doNO 5q9dVBpuzsgMCbTsfE1PRcXKfEWqHDqXVQdWSm44bH0tELlOiuu1rY/6sR7nIL/FM8iW jwZrfdaSvtTTe3FPRLyDASsFZtPWIXnMNV34SVbiyksKca3b5+X1V51Rk1+QLpw8yqlI jKsrvc1GS2OoebH2+AUivriToQtVaV9TFNr65N7CJq54dqMs0RkbrmsZg8FPHPj07udW DVkg== MIME-Version: 1.0 X-Received: by 10.180.210.237 with SMTP id mx13mr12688010wic.49.1401950811881; Wed, 04 Jun 2014 23:46:51 -0700 (PDT) Received: by 10.216.153.194 with HTTP; Wed, 4 Jun 2014 23:46:51 -0700 (PDT) Date: Thu, 5 Jun 2014 09:46:51 +0300 Message-ID: Subject: FreeBSD 10 - ixgbe packet drop From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= To: "freebsd-net@freebsd.org" , Jack Vogel Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 06:46:54 -0000 Hi I'm using FreeBSD 10. My ix0 is connected to my backbone switch. Traffic is about 90Mbit/s. But after 3 minutes it stops working. ifconfig ix0 down ; ifconfig ix0 up solves problem temprorarily. It's strange that, dev.ix.0.dropped is 0 but, netstat's Idrop counter is growing. How can i debug this situation? Regards Outputs : # sysctl dev.ix.0 dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.15 dev.ix.0.%driver: ix dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI0.PT1A.SLT0 dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x0003 class=0x020000 dev.ix.0.%parent: pci7 dev.ix.0.fc: 3 dev.ix.0.enable_aim: 1 dev.ix.0.advertise_speed: 0 dev.ix.0.dropped: 0 dev.ix.0.mbuf_defrag_failed: 0 dev.ix.0.watchdog_events: 0 dev.ix.0.link_irq: 1972 dev.ix.0.queue0.interrupt_rate: 62500 dev.ix.0.queue0.irqs: 2548180 dev.ix.0.queue0.txd_head: 1880 dev.ix.0.queue0.txd_tail: 1880 dev.ix.0.queue0.tso_tx: 0 dev.ix.0.queue0.no_tx_dma_setup: 0 dev.ix.0.queue0.no_desc_avail: 0 dev.ix.0.queue0.tx_packets: 1702224 dev.ix.0.queue0.rxd_head: 801 dev.ix.0.queue0.rxd_tail: 800 dev.ix.0.queue0.rx_packets: 1388687 dev.ix.0.queue0.rx_bytes: 763445 dev.ix.0.queue0.rx_copies: 4245 dev.ix.0.queue0.lro_queued: 0 dev.ix.0.queue0.lro_flushed: 0 dev.ix.0.queue1.interrupt_rate: 83333 dev.ix.0.queue1.irqs: 2189570 dev.ix.0.queue1.txd_head: 587 dev.ix.0.queue1.txd_tail: 587 dev.ix.0.queue1.tso_tx: 0 dev.ix.0.queue1.no_tx_dma_setup: 0 dev.ix.0.queue1.no_desc_avail: 0 dev.ix.0.queue1.tx_packets: 1765976 dev.ix.0.queue1.rxd_head: 1349 dev.ix.0.queue1.rxd_tail: 1348 dev.ix.0.queue1.rx_packets: 1003258 dev.ix.0.queue1.rx_bytes: 907093 dev.ix.0.queue1.rx_copies: 4741 dev.ix.0.queue1.lro_queued: 0 dev.ix.0.queue1.lro_flushed: 0 dev.ix.0.queue2.interrupt_rate: 83333 dev.ix.0.queue2.irqs: 6122648 dev.ix.0.queue2.txd_head: 238 dev.ix.0.queue2.txd_tail: 238 dev.ix.0.queue2.tso_tx: 0 dev.ix.0.queue2.no_tx_dma_setup: 0 dev.ix.0.queue2.no_desc_avail: 0 dev.ix.0.queue2.tx_packets: 4626026 dev.ix.0.queue2.rxd_head: 219 dev.ix.0.queue2.rxd_tail: 218 dev.ix.0.queue2.rx_packets: 3109132 dev.ix.0.queue2.rx_bytes: 701344 dev.ix.0.queue2.rx_copies: 3641 dev.ix.0.queue2.lro_queued: 0 dev.ix.0.queue2.lro_flushed: 0 dev.ix.0.queue3.interrupt_rate: 22727 dev.ix.0.queue3.irqs: 1809645 dev.ix.0.queue3.txd_head: 807 dev.ix.0.queue3.txd_tail: 807 dev.ix.0.queue3.tso_tx: 0 dev.ix.0.queue3.no_tx_dma_setup: 0 dev.ix.0.queue3.no_desc_avail: 0 dev.ix.0.queue3.tx_packets: 1684602 dev.ix.0.queue3.rxd_head: 1870 dev.ix.0.queue3.rxd_tail: 1869 dev.ix.0.queue3.rx_packets: 805931 dev.ix.0.queue3.rx_bytes: 768700 dev.ix.0.queue3.rx_copies: 3153 dev.ix.0.queue3.lro_queued: 0 dev.ix.0.queue3.lro_flushed: 0 dev.ix.0.queue4.interrupt_rate: 83333 dev.ix.0.queue4.irqs: 2332359 dev.ix.0.queue4.txd_head: 10 dev.ix.0.queue4.txd_tail: 10 dev.ix.0.queue4.tso_tx: 0 dev.ix.0.queue4.no_tx_dma_setup: 0 dev.ix.0.queue4.no_desc_avail: 0 dev.ix.0.queue4.tx_packets: 1964932 dev.ix.0.queue4.rxd_head: 593 dev.ix.0.queue4.rxd_tail: 592 dev.ix.0.queue4.rx_packets: 982226 dev.ix.0.queue4.rx_bytes: 651351 dev.ix.0.queue4.rx_copies: 4139 dev.ix.0.queue4.lro_queued: 0 dev.ix.0.queue4.lro_flushed: 0 dev.ix.0.queue5.interrupt_rate: 62500 dev.ix.0.queue5.irqs: 2387496 dev.ix.0.queue5.txd_head: 1023 dev.ix.0.queue5.txd_tail: 1023 dev.ix.0.queue5.tso_tx: 0 dev.ix.0.queue5.no_tx_dma_setup: 0 dev.ix.0.queue5.no_desc_avail: 0 dev.ix.0.queue5.tx_packets: 2392009 dev.ix.0.queue5.rxd_head: 66 dev.ix.0.queue5.rxd_tail: 65 dev.ix.0.queue5.rx_packets: 1411126 dev.ix.0.queue5.rx_bytes: 1031757 dev.ix.0.queue5.rx_copies: 5320 dev.ix.0.queue5.lro_queued: 0 dev.ix.0.queue5.lro_flushed: 0 dev.ix.0.queue6.interrupt_rate: 100000 dev.ix.0.queue6.irqs: 2137407 dev.ix.0.queue6.txd_head: 1559 dev.ix.0.queue6.txd_tail: 1559 dev.ix.0.queue6.tso_tx: 0 dev.ix.0.queue6.no_tx_dma_setup: 0 dev.ix.0.queue6.no_desc_avail: 0 dev.ix.0.queue6.tx_packets: 2110680 dev.ix.0.queue6.rxd_head: 1310 dev.ix.0.queue6.rxd_tail: 1309 dev.ix.0.queue6.rx_packets: 1152477 dev.ix.0.queue6.rx_bytes: 638730 dev.ix.0.queue6.rx_copies: 2744 dev.ix.0.queue6.lro_queued: 0 dev.ix.0.queue6.lro_flushed: 0 dev.ix.0.queue7.interrupt_rate: 27777 dev.ix.0.queue7.irqs: 2313122 dev.ix.0.queue7.txd_head: 1956 dev.ix.0.queue7.txd_tail: 1956 dev.ix.0.queue7.tso_tx: 0 dev.ix.0.queue7.no_tx_dma_setup: 0 dev.ix.0.queue7.no_desc_avail: 0 dev.ix.0.queue7.tx_packets: 1901071 dev.ix.0.queue7.rxd_head: 669 dev.ix.0.queue7.rxd_tail: 668 dev.ix.0.queue7.rx_packets: 1080747 dev.ix.0.queue7.rx_bytes: 649663 dev.ix.0.queue7.rx_copies: 4163 dev.ix.0.queue7.lro_queued: 0 dev.ix.0.queue7.lro_flushed: 0 dev.ix.0.queue8.interrupt_rate: 5208 dev.ix.0.queue8.irqs: 1297927 dev.ix.0.queue8.txd_head: 800 dev.ix.0.queue8.txd_tail: 800 dev.ix.0.queue8.tso_tx: 0 dev.ix.0.queue8.no_tx_dma_setup: 0 dev.ix.0.queue8.no_desc_avail: 0 dev.ix.0.queue8.tx_packets: 531043 dev.ix.0.queue8.rxd_head: 762 dev.ix.0.queue8.rxd_tail: 761 dev.ix.0.queue8.rx_packets: 956856 dev.ix.0.queue8.rx_bytes: 4104502 dev.ix.0.queue8.rx_copies: 3760 dev.ix.0.queue8.lro_queued: 0 dev.ix.0.queue8.lro_flushed: 0 dev.ix.0.queue9.interrupt_rate: 62500 dev.ix.0.queue9.irqs: 1223043 dev.ix.0.queue9.txd_head: 263 dev.ix.0.queue9.txd_tail: 263 dev.ix.0.queue9.tso_tx: 0 dev.ix.0.queue9.no_tx_dma_setup: 0 dev.ix.0.queue9.no_desc_avail: 0 dev.ix.0.queue9.tx_packets: 1905309 dev.ix.0.queue9.rxd_head: 1672 dev.ix.0.queue9.rxd_tail: 1671 dev.ix.0.queue9.rx_packets: 1535904 dev.ix.0.queue9.rx_bytes: 833210 dev.ix.0.queue9.rx_copies: 5066 dev.ix.0.queue9.lro_queued: 0 dev.ix.0.queue9.lro_flushed: 0 dev.ix.0.queue10.interrupt_rate: 5208 dev.ix.0.queue10.irqs: 1333160 dev.ix.0.queue10.txd_head: 569 dev.ix.0.queue10.txd_tail: 569 dev.ix.0.queue10.tso_tx: 0 dev.ix.0.queue10.no_tx_dma_setup: 0 dev.ix.0.queue10.no_desc_avail: 0 dev.ix.0.queue10.tx_packets: 541056 dev.ix.0.queue10.rxd_head: 1099 dev.ix.0.queue10.rxd_tail: 1098 dev.ix.0.queue10.rx_packets: 992037 dev.ix.0.queue10.rx_bytes: 683236 dev.ix.0.queue10.rx_copies: 2498 dev.ix.0.queue10.lro_queued: 0 dev.ix.0.queue10.lro_flushed: 0 dev.ix.0.queue11.interrupt_rate: 25000 dev.ix.0.queue11.irqs: 1167328 dev.ix.0.queue11.txd_head: 897 dev.ix.0.queue11.txd_tail: 897 dev.ix.0.queue11.tso_tx: 0 dev.ix.0.queue11.no_tx_dma_setup: 0 dev.ix.0.queue11.no_desc_avail: 0 dev.ix.0.queue11.tx_packets: 628343 dev.ix.0.queue11.rxd_head: 440 dev.ix.0.queue11.rxd_tail: 439 dev.ix.0.queue11.rx_packets: 807500 dev.ix.0.queue11.rx_bytes: 887029 dev.ix.0.queue11.rx_copies: 5733 dev.ix.0.queue11.lro_queued: 0 dev.ix.0.queue11.lro_flushed: 0 dev.ix.0.queue12.interrupt_rate: 31250 dev.ix.0.queue12.irqs: 1428643 dev.ix.0.queue12.txd_head: 356 dev.ix.0.queue12.txd_tail: 356 dev.ix.0.queue12.tso_tx: 0 dev.ix.0.queue12.no_tx_dma_setup: 0 dev.ix.0.queue12.no_desc_avail: 0 dev.ix.0.queue12.tx_packets: 2217914 dev.ix.0.queue12.rxd_head: 2047 dev.ix.0.queue12.rxd_tail: 2047 dev.ix.0.queue12.rx_packets: 1020147 dev.ix.0.queue12.rx_bytes: 0 dev.ix.0.queue12.rx_copies: 0 dev.ix.0.queue12.lro_queued: 0 dev.ix.0.queue12.lro_flushed: 0 dev.ix.0.queue13.interrupt_rate: 5434 dev.ix.0.queue13.irqs: 1079537 dev.ix.0.queue13.txd_head: 1037 dev.ix.0.queue13.txd_tail: 1037 dev.ix.0.queue13.tso_tx: 0 dev.ix.0.queue13.no_tx_dma_setup: 0 dev.ix.0.queue13.no_desc_avail: 0 dev.ix.0.queue13.tx_packets: 587585 dev.ix.0.queue13.rxd_head: 1551 dev.ix.0.queue13.rxd_tail: 1550 dev.ix.0.queue13.rx_packets: 899834 dev.ix.0.queue13.rx_bytes: 801311 dev.ix.0.queue13.rx_copies: 4854 dev.ix.0.queue13.lro_queued: 0 dev.ix.0.queue13.lro_flushed: 0 dev.ix.0.queue14.interrupt_rate: 25000 dev.ix.0.queue14.irqs: 1484788 dev.ix.0.queue14.txd_head: 173 dev.ix.0.queue14.txd_tail: 173 dev.ix.0.queue14.tso_tx: 0 dev.ix.0.queue14.no_tx_dma_setup: 0 dev.ix.0.queue14.no_desc_avail: 0 dev.ix.0.queue14.tx_packets: 1009215 dev.ix.0.queue14.rxd_head: 334 dev.ix.0.queue14.rxd_tail: 333 dev.ix.0.queue14.rx_packets: 1736273 dev.ix.0.queue14.rx_bytes: 842627 dev.ix.0.queue14.rx_copies: 3571 dev.ix.0.queue14.lro_queued: 0 dev.ix.0.queue14.lro_flushed: 0 dev.ix.0.queue15.interrupt_rate: 31250 dev.ix.0.queue15.irqs: 1230363 dev.ix.0.queue15.txd_head: 726 dev.ix.0.queue15.txd_tail: 726 dev.ix.0.queue15.tso_tx: 0 dev.ix.0.queue15.no_tx_dma_setup: 0 dev.ix.0.queue15.no_desc_avail: 0 dev.ix.0.queue15.tx_packets: 594232 dev.ix.0.queue15.rxd_head: 697 dev.ix.0.queue15.rxd_tail: 696 dev.ix.0.queue15.rx_packets: 933871 dev.ix.0.queue15.rx_bytes: 4919560 dev.ix.0.queue15.rx_copies: 5556 dev.ix.0.queue15.lro_queued: 0 dev.ix.0.queue15.lro_flushed: 0 dev.ix.0.mac_stats.crc_errs: 1 dev.ix.0.mac_stats.ill_errs: 0 dev.ix.0.mac_stats.byte_errs: 0 dev.ix.0.mac_stats.short_discards: 0 dev.ix.0.mac_stats.local_faults: 33864 dev.ix.0.mac_stats.remote_faults: 18168 dev.ix.0.mac_stats.rec_len_errs: 0 dev.ix.0.mac_stats.xon_txd: 8 dev.ix.0.mac_stats.xon_recvd: 0 dev.ix.0.mac_stats.xoff_txd: 839470 dev.ix.0.mac_stats.xoff_recvd: 0 dev.ix.0.mac_stats.total_octets_rcvd: 8015310767 dev.ix.0.mac_stats.good_octets_rcvd: 7951220250 dev.ix.0.mac_stats.total_pkts_rcvd: 21287239 dev.ix.0.mac_stats.good_pkts_rcvd: 19916695 dev.ix.0.mac_stats.mcast_pkts_rcvd: 3035980 dev.ix.0.mac_stats.bcast_pkts_rcvd: 681566 dev.ix.0.mac_stats.rx_frames_64: 10512769 dev.ix.0.mac_stats.rx_frames_65_127: 3981208 dev.ix.0.mac_stats.rx_frames_128_255: 585704 dev.ix.0.mac_stats.rx_frames_256_511: 1549673 dev.ix.0.mac_stats.rx_frames_512_1023: 518172 dev.ix.0.mac_stats.rx_frames_1024_1522: 4019518 dev.ix.0.mac_stats.recv_undersized: 0 dev.ix.0.mac_stats.recv_fragmented: 0 dev.ix.0.mac_stats.recv_oversized: 0 dev.ix.0.mac_stats.recv_jabberd: 0 dev.ix.0.mac_stats.management_pkts_rcvd: 0 dev.ix.0.mac_stats.management_pkts_drpd: 0 dev.ix.0.mac_stats.checksum_errs: 20381 dev.ix.0.mac_stats.good_octets_txd: 28143082408 dev.ix.0.mac_stats.total_pkts_txd: 26968378 dev.ix.0.mac_stats.good_pkts_txd: 26128899 dev.ix.0.mac_stats.bcast_pkts_txd: 16314 dev.ix.0.mac_stats.mcast_pkts_txd: 2654 dev.ix.0.mac_stats.management_pkts_txd: 0 dev.ix.0.mac_stats.tx_frames_64: 4904275 dev.ix.0.mac_stats.tx_frames_65_127: 1427985 dev.ix.0.mac_stats.tx_frames_128_255: 523084 dev.ix.0.mac_stats.tx_frames_256_511: 725365 dev.ix.0.mac_stats.tx_frames_512_1023: 657348 dev.ix.0.mac_stats.tx_frames_1024_1522: 17890843 # ifconfig ix0 ix0: flags=8943 metric 0 mtu 1500 description: Ethernet: LAN1 options=8407bb ether 90:e2:ba:37:e5:94 inet6 fe80::92e2:baff:fe37:e594%ix0 prefixlen 64 scopeid 0x1 inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 inet 1.1.106.254 netmask 0xffffff00 broadcast 1.1.106.255 inet6 2001:a98:890:1::ffff prefixlen 64 nd6 options=21 media: Ethernet autoselect (10Gbase-SR ) status: active # netstat -nI ix0 Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll ix0 1500 90:e2:ba:37:e5:94 20693324 1 1337299 27184636 0 0 ix0 1500 fe80::92e2:ba fe80::92e2:baff:f 0 - - 45 - - ix0 1500 10.0.0.0/24 10.0.0.1 17534 - - 1784 - - ix0 1500 1.1.106.0/24 1.1.106.254 0 - - 0 - - ix0 1500 2001:a98:890: 2001:a98:890:1::f 0 - - 0 - - # netstat -m 166263/17757/184020 mbufs in use (current/cache/total) 165169/8835/174004/1048576 mbuf clusters in use (current/cache/total/max) 165169/8796 mbuf+clusters out of packet secondary zone in use (current/cache) 309/626/935/65536 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/19418 9k jumbo clusters in use (current/cache/total/max) 0/0/0/10922 16k jumbo clusters in use (current/cache/total/max) 373140K/24613K/397753K bytes allocated to network (current/cache/total) 3293/44053/173965 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 125/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 307 requests for I/O initiated by sendfile From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 08:00:12 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A07378E for ; Thu, 5 Jun 2014 08:00:12 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D53C2B4A for ; Thu, 5 Jun 2014 08:00:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5580C7L038969 for ; Thu, 5 Jun 2014 09:00:12 +0100 (BST) (envelope-from bz-noreply@freebsd.org) Message-Id: <201406050800.s5580C7L038969@kenobi.freebsd.org> From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Thu, 05 Jun 2014 08:00:12 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 08:00:12 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is an experimental feature. Please file a bug or mail bugmeister@ with concerns. This search was scheduled by eadler@FreeBSD.org. (8 bugs) Bug 127360: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=127360 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [socket] TOE socket options missing from sosetopt() Bug 137776: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137776 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [rum] panic in rum(4) driver on 8.0-BETA2 Bug 137841: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137841 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [patch] wpa_supplicant(8) cannot verify SHA256 signed certificates Bug 139204: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139204 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [arp] DHCP server replies rejected, ARP entry lost before max_age Bug 145600: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=145600 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: TCP/ECN behaves different to CE/CWR than ns2 reference Bug 154169: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154169 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [multicast] [ip6] Node Information Query multicast address wrong (FF02::2:x:y) Bug 168294: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168294 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [ixgbe] [patch] ixgbe driver compiled in kernel has no Flow Director support although loaded as a module has one Bug 172113: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=172113 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 10:24:11 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D075368F; Thu, 5 Jun 2014 10:24:11 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05EDA2895; Thu, 5 Jun 2014 10:24:10 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1WsQvF-00083h-Ag; Thu, 05 Jun 2014 10:13:09 +0400 Message-ID: <539044E4.1020904@ipfw.ru> Date: Thu, 05 Jun 2014 14:22:28 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> In-Reply-To: <538B2FE5.6070407@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------060502050508080706040508" Cc: Luigi Rizzo , Bill Yuan , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 10:24:11 -0000 This is a multi-part message in MIME format. --------------060502050508080706040508 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 01.06.2014 17:51, Alexander V. Chernikov wrote: > On 22.05.2014 20:38, Luigi Rizzo wrote: > > Long story short, new version is ready. > I've tried to minimize changes in this patch to ease review/commit. > > Changes: > * Add namedobject set-aware api capable of searching/allocation > objects by their name/idx. > * Switch tables code to use string ids for configuration tasks. > * Change locking model: most configuration changes are protected with > UH lock, runtime-visible are protected with both locks. > * Reduce number of arguments passed to ipfw_table_add/del by using > separate structure. > * Add internal V_fw_tables_sets tunable (set to 0) to prepare for > set-aware tables (requires opcodes/client support) > * Implement typed table referencing (and tables are implicitly > allocated with all state like radix ptrs on reference) > * Add "destroy" ipfw(8) using new IP_FW_DELOBJ opcode > > Namedobj more detailed: > * Blackbox api providing methods to add/del/search/enumerate objects > * Statically-sized hashes for names/indexes > * Per-set bitmask to indicate free indexes > * Separate methods for index alloc/delete/resize > > > Basically, there should not be any user-visible changes except the > following: > * reducing table_max is not supported > * flush & add change table type won't work if table is referenced > > > I haven't removed any numbering restrictions to protect the following > case: > one (with old client) unintentionally references too many tables (e.g. > 1000-1128), > tries to allocate table from "valid" range and fails. Old client does > not have any ability to > destroy any table, so the only way to solve this is either module > unload or reboot. > > I've uploaded the same patch to phabricator since it provides quite > handy diffs: > https://phabric.freebsd.org/D139 (no login required). A bit cleaner version attached. > >> On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. Chernikov wrote: >>> On 22.05.2014 19:47, Luigi Rizzo wrote: >>>> On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov >>>> wrote: >>>>> On 22.05.2014 00:48, Luigi Rizzo wrote: >>>>>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov >>>>>> wrote: >>>> ... >>>>>> we can solve this by using 'low' numbers for the numeric tables >>>>>> (these were limited anyways) and allocate the fake entries in >>>>>> another range. >>>>> Currently we have u16 space available in base opcode. >>>> yes but the standard range for tables is much more limited: >>>> >>>> net.inet.ip.fw.tables_max: 128 >>>> >>>> so one can just (say) use 32k for "old" tables and the rest >>>> for tables with non numeric names. >>>> Does not seem to be a problem in practice. >>> Well, using upper 32k means that you set this default to 65k which >>> consumes 256k of memory on 32-bit arch. >>> Embedded people won't be very happy about this (and changing table >>> numbers on resize would be a nightmare). >> no no, this is an implementation detail but >> within the kernel you can just remap the 'old' and 'new' >> table identifiers to a single contiguous range. >> The only thing you need to do is that when you push >> identifiers up to userland, those with 'new' names will >> be mapped to the 32-64k range. >> >> Example: >> user first specifies tables >> "18, goodguys, 530, badguys" in the same rule >> /sbin/ipfw will generate these numbers: >> 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} >> The kernel will then do a lookup of those identifiers and >> 18: internal index 1, name "18" >> 32768: internal index 2, name "goodguys" >> 530: internal index 3, name "530" >> 32769: internal index 4, name "badguys" >> >> Then the next rule contains tables >> 1, badguys, 18 >> /sbin/ipfw generates >> 1, 32768, 18 ; tlv {32768:badguys} // note different from before >> Kernel looks up the names and remaps >> 1: internal index 5, name "1" >> 32768: internal index 4, name "badguys" >> 18: internal index 1, name "18" >> >> Finally when you do an 'ipfw show' the kernel will remap names >> between 1 and 32768 to themselves, and other names to 32768+ >> (or some other large number, say 40k and above) so >> as they are found. So the rules will be pushed up with >> 18, 40000, 530, 40001 >> 1, 40001, 18 >> >> we can discusso the other details privately >> >> cheers >> luigi >> >> >> 1. first, the >>>>>> maybe i am missing some detail but it seems reasonably easy to >>>>>> implement >>>>>> the atomic swap -- and the use case is when you want to move from >>>>>> one configuration to a new one: >>>>>> ipfw table foo-new flush // clear initial content >>>>>> ipfw table foo-new add ... >>>>>> ipfw table swap foo-current foo-new // swap the content of >>>>>> the table objects >>>>>> >>>>>> so you preserve the semantic of the name very easily. >>>>> Yes. We can easily add atomic table swap that way. However, I'm >>>>> talking >>>>> about different use scenario: >>>>> Atomically swap entire ruleset which has some tables depency: >>>>> >>>>> >>>>> e.g. we have: >>>>> >>>>> " >>>>> 100 allow ip from table(TABLE1) to me >>>>> 200 allow ip from table(TABLE2) to (TABLE3) 80 >>>>> >>>>> table TABLE1 1.1.1.1/32 >>>>> table TABLE1 1.0.0.0/16 >>>>> >>>>> table TABLE2 2.2.2.2/32 >>>>> >>>>> table TABLE3 3.3.3.3/32 >>>>> " >>>>> and we want to _atomically_ change this to >>>>> >>>>> " >>>>> 100 allow ip from table(TABLE1) to me >>>>> +200 allow ip from table(TABLE4) to any >>>>> 300 allow ip from table(TABLE2) to (TABLE3) 80 >>>>> >>>>> table TABLE1 1.1.1.1/32 >>>>> -table TABLE1 1.0.0.0/16 >>>>> >>>>> -table TABLE2 2.2.2.2/32 >>>>> +table TABLE2 77.77.77.0/24 >>>>> >>>>> table TABLE3 3.3.3.3/32 >>>>> >>>>> +table TABLE4 4.4.4.4/32 >>>>> " >>>> aargh, that's too much -- because between changing >>>> one table and all tables there are infinite intermediate >>>> points that all make sense. >>> It depends. As I said before, we're currently solving this problem by >>> adding new rules (to set X) referencing tables from different range >>> (2048 tables per ruleset) and than doing swap. >>> (And not being able to use named tables to store real names after >>> implementing them is a bit discouraging). >>> >>>> For those cases i think the way to go could be to >>>> insert a 'disabled' new ruleset (however complex it is, >>>> so it covers all possible cases), and then do the set swap, >>>> or disable/enable. >>> We can think of per-set arrays/namespaces of tables: >>> >>> so "ipfw add 100 set X allow ipfw from table(Y) to ..." will reference >>> table Y in set X and >>> "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". >>> >>> This behavior can break some users setups so we can provide >>> sysctl/tunable to turn this off or on. >>> >>>> cheers >>>> luigi >>>> > --------------060502050508080706040508 Content-Type: text/x-patch; name="D139_4.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="D139_4.diff" Index: sbin/ipfw/ipfw2.c =================================================================== --- sbin/ipfw/ipfw2.c +++ sbin/ipfw/ipfw2.c @@ -4243,6 +4243,23 @@ do { table_list(xent.tbl, is_all); } while (++xent.tbl < a); + } else if (_substrcmp(*av, "destroy") == 0) { + char xbuf[sizeof(ipfw_obj_header) + sizeof(ipfw_xtable_ntlv)]; + ipfw_obj_header *oh; + ipfw_xtable_ntlv *ntlv; + + memset(xbuf, 0, sizeof(xbuf)); + oh = (ipfw_obj_header *)xbuf; + ntlv = (ipfw_xtable_ntlv *)(oh + 1); + + ntlv->head.type = IPFW_TLV_NAME; + ntlv->head.length = sizeof(*ntlv); + ntlv->idx = 1; + snprintf(ntlv->name, sizeof(ntlv->name), "%d", xent.tbl); + oh->idx = 1; + oh->objtype = IPFW_OBJTYPE_TABLE; + if (do_setcmd3(IP_FW_OBJ_DEL, xbuf, sizeof(xbuf)) != 0) + err(EX_OSERR, "setsockopt(IP_FW_OBJ_DEL)"); } else errx(EX_USAGE, "invalid table command %s", *av); } Index: sys/netinet/ip_fw.h =================================================================== --- sys/netinet/ip_fw.h +++ sys/netinet/ip_fw.h @@ -37,6 +37,11 @@ #define IPFW_DEFAULT_RULE 65535 /* + * Number of sets supported by ipfw + */ +#define IPFW_MAX_SETS 32 + +/* * Default number of ipfw tables. */ #define IPFW_TABLES_MAX 65535 @@ -74,6 +79,7 @@ #define IP_FW_TABLE_XDEL 87 /* delete entry */ #define IP_FW_TABLE_XGETSIZE 88 /* get table size */ #define IP_FW_TABLE_XLIST 89 /* list table contents */ +#define IP_FW_OBJ_DEL 90 /* del table/pipe/etc */ /* * The kernel representation of ipfw rules is made of a list of @@ -632,12 +638,34 @@ } ipfw_table; typedef struct _ipfw_xtable { - ip_fw3_opheader opheader; /* eXtended tables are controlled via IP_FW3 */ + ip_fw3_opheader opheader; /* IP_FW3 opcode */ uint32_t size; /* size of entries in bytes */ uint32_t cnt; /* # of entries */ uint16_t tbl; /* table number */ uint8_t type; /* table type */ ipfw_table_xentry xent[0]; /* entries */ } ipfw_xtable; +typedef struct _ipfw_xtable_tlv { + uint16_t type; /* TLV type */ + uint16_t length; /* Total length, aligned to u32 */ +} ipfw_xtable_tlv; + +#define IPFW_TLV_NAME 1 +/* Object name TLV */ +typedef struct _ipfw_xtable_ntlv { + ipfw_xtable_tlv head; /* TLV header */ + uint16_t idx; /* Name index */ + uint16_t spare; /* unused */ + char name[64]; /* Null-terminated name */ +} ipfw_xtable_ntlv; + +typedef struct _ipfw_obj_header { + ip_fw3_opheader opheader; /* IP_FW3 opcode */ + uint32_t set; /* Set we're operating */ + uint16_t idx; /* object name index */ + uint16_t objtype; /* object type */ +} ipfw_obj_header; +#define IPFW_OBJTYPE_TABLE 1 + #endif /* _IPFW2_H */ Index: sys/netpfil/ipfw/ip_fw2.c =================================================================== --- sys/netpfil/ipfw/ip_fw2.c +++ sys/netpfil/ipfw/ip_fw2.c @@ -121,6 +121,7 @@ VNET_DEFINE(int, fw_one_pass) = 1; VNET_DEFINE(unsigned int, fw_tables_max); +VNET_DEFINE(unsigned int, fw_tables_sets) = 0; /* Don't use set-aware tables */ /* Use 128 tables by default */ static unsigned int default_fw_tables = IPFW_TABLES_DEFAULT; @@ -2719,7 +2720,6 @@ ipfw_dyn_uninit(0); /* run the callout_drain */ IPFW_WUNLOCK(chain); - ipfw_destroy_tables(chain); reap = NULL; IPFW_WLOCK(chain); for (i = 0; i < chain->n_rules; i++) { @@ -2731,6 +2731,7 @@ free(chain->map, M_IPFW); IPFW_WUNLOCK(chain); IPFW_UH_WUNLOCK(chain); + ipfw_destroy_tables(chain); if (reap != NULL) ipfw_reap_rules(reap); IPFW_LOCK_DESTROY(chain); Index: sys/netpfil/ipfw/ip_fw_private.h =================================================================== --- sys/netpfil/ipfw/ip_fw_private.h +++ sys/netpfil/ipfw/ip_fw_private.h @@ -212,14 +212,18 @@ VNET_DECLARE(unsigned int, fw_tables_max); #define V_fw_tables_max VNET(fw_tables_max) +VNET_DECLARE(unsigned int, fw_tables_sets); +#define V_fw_tables_sets VNET(fw_tables_sets) + +struct tables_config; + struct ip_fw_chain { struct ip_fw **map; /* array of rule ptrs to ease lookup */ uint32_t id; /* ruleset id */ int n_rules; /* number of static rules */ LIST_HEAD(nat_list, cfg_nat) nat; /* list of nat entries */ struct radix_node_head **tables; /* IPv4 tables */ struct radix_node_head **xtables; /* extended tables */ - uint8_t *tabletype; /* Array of table types */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t rwmtx; #else @@ -229,6 +233,7 @@ uint32_t gencnt; /* NAT generation count */ struct ip_fw *reap; /* list of rules to reap */ struct ip_fw *default_rule; + struct tables_config *tblcfg; /* tables module data */ #if defined( __linux__ ) || defined( _WIN32 ) spinlock_t uh_lock; #else @@ -295,32 +300,113 @@ #define IPFW_UH_WLOCK(p) rw_wlock(&(p)->uh_lock) #define IPFW_UH_WUNLOCK(p) rw_wunlock(&(p)->uh_lock) +struct tid_info { + uint32_t set; /* table set */ + uint16_t uidx; /* table index */ + uint8_t type; /* table type */ + uint8_t spare; + void *tlvs; /* Pointer to first TLV */ + int tlen; /* Total TLV size block */ +}; + +struct obj_idx { + uint16_t uidx; /* internal index supplied by userland */ + uint16_t kidx; /* kernel object index */ + uint16_t off; /* tlv offset from rule end in 4-byte words */ + uint8_t new; /* index is newly-allocated */ + uint8_t type; /* object type within its category */ +}; + +struct rule_check_info { + uint16_t table_opcodes; /* count of opcodes referencing table */ + uint16_t new_tables; /* count of opcodes referencing table */ + uint32_t tableset; /* ipfw set id for table */ + void *tlvs; /* Pointer to first TLV if any */ + int tlen; /* *Total TLV size block */ + uint8_t fw3; /* opcode is new */ + struct ip_fw *krule; /* resulting rule pointer */ + struct obj_idx obuf[8]; /* table references storage */ +}; + +struct tentry_info { + void *paddr; + int plen; /* Total entry length */ + uint8_t masklen; /* mask length */ + uint8_t spare; + uint16_t flags; /* record flags */ + uint32_t value; /* value */ +}; + /* In ip_fw_sockopt.c */ int ipfw_find_rule(struct ip_fw_chain *chain, uint32_t key, uint32_t id); -int ipfw_add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule); int ipfw_ctl(struct sockopt *sopt); int ipfw_chk(struct ip_fw_args *args); void ipfw_reap_rules(struct ip_fw *head); +struct namedobj_instance; + +struct named_object { + TAILQ_ENTRY(named_object) nn_next; /* namehash */ + TAILQ_ENTRY(named_object) nv_next; /* valuehash */ + char *name; /* object name */ + uint8_t type; /* object type */ + uint8_t compat; /* Object name is number */ + uint16_t kidx; /* object kernel index */ + uint16_t uidx; /* userland idx for compat records */ + uint32_t set; /* set object belongs to */ + uint32_t refcnt; /* number of references */ +}; +TAILQ_HEAD(namedobjects_head, named_object); + +typedef void (objhash_cb_t)(struct namedobj_instance *ni, struct named_object *, + void *arg); +struct namedobj_instance *ipfw_objhash_create(uint32_t items); +void ipfw_objhash_destroy(struct namedobj_instance *); +void ipfw_objhash_bitmap_alloc(uint32_t items, void **idx, int *pblocks); +int ipfw_objhash_bitmap_merge(struct namedobj_instance *ni, + void **idx, int *blocks); +void ipfw_objhash_bitmap_free(void *idx, int blocks); +struct named_object *ipfw_objhash_lookup_name(struct namedobj_instance *ni, + uint32_t set, char *name); +struct named_object *ipfw_objhash_lookup_idx(struct namedobj_instance *ni, + uint32_t set, uint16_t idx); +void ipfw_objhash_add(struct namedobj_instance *ni, struct named_object *no); +void ipfw_objhash_del(struct namedobj_instance *ni, struct named_object *no); +void ipfw_objhash_foreach(struct namedobj_instance *ni, objhash_cb_t *f, + void *arg); +int ipfw_objhash_free_idx(struct namedobj_instance *ni, uint32_t set, + uint16_t idx); +int ipfw_objhash_alloc_idx(void *n, uint32_t set, uint16_t *pidx); + /* In ip_fw_table.c */ struct radix_node; int ipfw_lookup_table(struct ip_fw_chain *ch, uint16_t tbl, in_addr_t addr, uint32_t *val); int ipfw_lookup_table_extended(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, uint32_t *val, int type); int ipfw_init_tables(struct ip_fw_chain *ch); +int ipfw_destroy_table(struct ip_fw_chain *ch, struct tid_info *ti, int force); void ipfw_destroy_tables(struct ip_fw_chain *ch); -int ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl); -int ipfw_add_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type, uint32_t value); -int ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type); -int ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); +int ipfw_flush_table(struct ip_fw_chain *ch, struct tid_info *ti); +int ipfw_add_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei); +int ipfw_del_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei); +int ipfw_count_table(struct ip_fw_chain *ch, struct tid_info *ti, + uint32_t *cnt); int ipfw_dump_table_entry(struct radix_node *rn, void *arg); -int ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl); -int ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt); -int ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl); +int ipfw_dump_table(struct ip_fw_chain *ch, struct tid_info *ti, + ipfw_table *tbl); +int ipfw_count_xtable(struct ip_fw_chain *ch, struct tid_info *ti, + uint32_t *cnt); +int ipfw_dump_xtable(struct ip_fw_chain *ch, struct tid_info *ti, + ipfw_xtable *tbl); int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables); +int ipfw_rewrite_table_uidx(struct ip_fw_chain *chain, + struct rule_check_info *ci); +int ipfw_rewrite_table_kidx(struct ip_fw_chain *chain, struct ip_fw *rule); +void ipfw_unbind_table_rule(struct ip_fw_chain *chain, struct ip_fw *rule); +void ipfw_unbind_table_list(struct ip_fw_chain *chain, struct ip_fw *head); /* In ip_fw_nat.c -- XXX to be moved to ip_var.h */ Index: sys/netpfil/ipfw/ip_fw_sockopt.c =================================================================== --- sys/netpfil/ipfw/ip_fw_sockopt.c +++ sys/netpfil/ipfw/ip_fw_sockopt.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include #include @@ -67,6 +68,25 @@ #include #endif +#define NAMEDOBJ_HASH_SIZE 32 + +struct namedobj_instance { + struct namedobjects_head *names; + struct namedobjects_head *values; + uint32_t nn_size; /* names hash size */ + uint32_t nv_size; /* number hash size */ + u_long *idx_mask; /* used items bitmask */ + uint32_t max_blocks; /* number of "long" blocks in bitmask */ + uint16_t free_off[IPFW_MAX_SETS]; /* first possible free offset */ +}; +#define BLOCK_ITEMS (8 * sizeof(u_long)) /* Number of items for ffsl() */ + +static uint32_t objhash_hash_name(struct namedobj_instance *ni, uint32_t set, + char *name); +static uint32_t objhash_hash_val(struct namedobj_instance *ni, uint32_t set, + uint32_t val); + + MALLOC_DEFINE(M_IPFW, "IpFw/IpAcct", "IpFw/IpAcct chain's"); /* @@ -152,8 +172,9 @@ * XXX DO NOT USE FOR THE DEFAULT RULE. * Must be called without IPFW_UH held */ -int -ipfw_add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule) +static int +add_rule(struct ip_fw_chain *chain, struct ip_fw *input_rule, + struct rule_check_info *ci) { struct ip_fw *rule; int i, l, insert_before; @@ -164,19 +185,37 @@ l = RULESIZE(input_rule); rule = malloc(l, M_IPFW, M_WAITOK | M_ZERO); - /* get_map returns with IPFW_UH_WLOCK if successful */ - map = get_map(chain, 1, 0 /* not locked */); - if (map == NULL) { - free(rule, M_IPFW); - return ENOSPC; - } - bcopy(input_rule, rule, l); /* clear fields not settable from userland */ rule->x_next = NULL; rule->next_rule = NULL; IPFW_ZERO_RULE_COUNTER(rule); + /* Check if we need to do table remap */ + if (ci->table_opcodes > 0) { + ci->krule = rule; + i = ipfw_rewrite_table_uidx(chain, ci); + if (i != 0) { + /* rewrite failed, return error */ + free(rule, M_IPFW); + return (i); + } + } + + /* get_map returns with IPFW_UH_WLOCK if successful */ + map = get_map(chain, 1, 0 /* not locked */); + if (map == NULL) { + if (ci->table_opcodes > 0) { + /* We need to unbind tables */ + IPFW_UH_WLOCK(chain); + ipfw_unbind_table_rule(chain, rule); + IPFW_UH_WUNLOCK(chain); + } + + free(rule, M_IPFW); + return (ENOSPC); + } + if (V_autoinc_step < 1) V_autoinc_step = 1; else if (V_autoinc_step > 1000) @@ -421,6 +460,7 @@ rule = chain->reap; chain->reap = NULL; + ipfw_unbind_table_list(chain, rule); IPFW_UH_WUNLOCK(chain); ipfw_reap_rules(rule); if (map) @@ -517,7 +557,7 @@ * Rules are simple, so this mostly need to check rule sizes. */ static int -check_ipfw_struct(struct ip_fw *rule, int size) +check_ipfw_struct(struct ip_fw *rule, int size, struct rule_check_info *ci) { int l, cmdlen = 0; int have_action=0; @@ -662,6 +702,7 @@ cmdlen != F_INSN_SIZE(ipfw_insn_u32) + 1 && cmdlen != F_INSN_SIZE(ipfw_insn_u32)) goto bad_size; + ci->table_opcodes++; break; case O_MACADDR2: if (cmdlen != F_INSN_SIZE(ipfw_insn_mac)) @@ -694,6 +735,8 @@ case O_RECV: case O_XMIT: case O_VIA: + if (((ipfw_insn_if *)cmd)->name[0] == '\1') + ci->table_opcodes++; if (cmdlen != F_INSN_SIZE(ipfw_insn_if)) goto bad_size; break; @@ -879,7 +922,7 @@ char *bp = buf; char *ep = bp + space; struct ip_fw *rule, *dst; - int l, i; + int error, i, l; time_t boot_seconds; boot_seconds = boottime.tv_sec; @@ -890,8 +933,11 @@ /* Convert rule to FreeBSd 7.2 format */ l = RULESIZE7(rule); if (bp + l + sizeof(uint32_t) <= ep) { - int error; bcopy(rule, bp, l + sizeof(uint32_t)); + error = ipfw_rewrite_table_kidx(chain, + (struct ip_fw *)bp); + if (error != 0) + return (0); error = convert_rule_to_7((struct ip_fw *) bp); if (error) return 0; /*XXX correct? */ @@ -918,6 +964,13 @@ } dst = (struct ip_fw *)bp; bcopy(rule, dst, l); + error = ipfw_rewrite_table_kidx(chain, dst); + if (error != 0) { + printf("Stop on rule %d. Fail to convert table\n", + rule->rulenum); + break; + } + /* * XXX HACK. Store the disable mask in the "next" * pointer in a wild attempt to keep the ABI the same. @@ -949,6 +1002,7 @@ uint32_t opt; char xbuf[128]; ip_fw3_opheader *op3 = NULL; + struct rule_check_info ci; error = priv_check(sopt->sopt_td, PRIV_NETINET_IPFW); if (error) @@ -1027,6 +1081,8 @@ error = sooptcopyin(sopt, rule, RULE_MAXSIZE, sizeof(struct ip_fw7) ); + memset(&ci, 0, sizeof(struct rule_check_info)); + /* * If the size of commands equals RULESIZE7 then we assume * a FreeBSD7.2 binary is talking to us (set is7=1). @@ -1044,15 +1100,15 @@ return error; } if (error == 0) - error = check_ipfw_struct(rule, RULESIZE(rule)); + error = check_ipfw_struct(rule, RULESIZE(rule), &ci); } else { is7 = 0; if (error == 0) - error = check_ipfw_struct(rule, sopt->sopt_valsize); + error = check_ipfw_struct(rule, sopt->sopt_valsize,&ci); } if (error == 0) { - /* locking is done within ipfw_add_rule() */ - error = ipfw_add_rule(chain, rule); + /* locking is done within add_rule() */ + error = add_rule(chain, rule, &ci); size = RULESIZE(rule); if (!error && sopt->sopt_dir == SOPT_GET) { if (is7) { @@ -1114,37 +1170,67 @@ break; /*--- TABLE manipulations are protected by the IPFW_LOCK ---*/ - case IP_FW_TABLE_ADD: + case IP_FW_OBJ_DEL: /* IP_FW3 */ { - ipfw_table_entry ent; + struct _ipfw_obj_header *oh; + struct tid_info ti; - error = sooptcopyin(sopt, &ent, - sizeof(ent), sizeof(ent)); - if (error) + if (sopt->sopt_valsize < sizeof(*oh)) { + error = EINVAL; break; - error = ipfw_add_table_entry(chain, ent.tbl, - &ent.addr, sizeof(ent.addr), ent.masklen, - IPFW_TABLE_CIDR, ent.value); - } - break; + } + + oh = (struct _ipfw_obj_header *)(op3 + 1); + switch (oh->objtype) { + case IPFW_OBJTYPE_TABLE: + memset(&ti, 0, sizeof(ti)); + ti.set = oh->set; + ti.uidx = oh->idx; + ti.tlvs = (oh + 1); + ti.tlen = sopt->sopt_valsize - sizeof(*oh); + error = ipfw_destroy_table(chain, &ti, 0); + break; + default: + error = ENOTSUP; + break; + } + break; + } + case IP_FW_TABLE_ADD: case IP_FW_TABLE_DEL: { ipfw_table_entry ent; + struct tentry_info tei; + struct tid_info ti; error = sooptcopyin(sopt, &ent, sizeof(ent), sizeof(ent)); if (error) break; - error = ipfw_del_table_entry(chain, ent.tbl, - &ent.addr, sizeof(ent.addr), ent.masklen, IPFW_TABLE_CIDR); + + memset(&tei, 0, sizeof(tei)); + tei.paddr = &ent.addr; + tei.plen = sizeof(ent.addr); + tei.masklen = ent.masklen; + tei.value = ent.value; + memset(&ti, 0, sizeof(ti)); + ti.set = RESVD_SET; + ti.uidx = ent.tbl; + ti.type = IPFW_TABLE_CIDR; + + error = (opt == IP_FW_TABLE_ADD) ? + ipfw_add_table_entry(chain, &ti, &tei) : + ipfw_del_table_entry(chain, &ti, &tei); } break; case IP_FW_TABLE_XADD: /* IP_FW3 */ case IP_FW_TABLE_XDEL: /* IP_FW3 */ { ipfw_table_xentry *xent = (ipfw_table_xentry *)(op3 + 1); + struct tentry_info tei; + struct tid_info ti; /* Check minimum header size */ if (IP_FW3_OPLENGTH(sopt) < offsetof(ipfw_table_xentry, k)) { @@ -1160,35 +1246,51 @@ len = xent->len - offsetof(ipfw_table_xentry, k); + memset(&tei, 0, sizeof(tei)); + tei.paddr = &xent->k; + tei.plen = len; + tei.masklen = xent->masklen; + tei.value = xent->value; + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = xent->tbl; + ti.type = xent->type; + error = (opt == IP_FW_TABLE_XADD) ? - ipfw_add_table_entry(chain, xent->tbl, &xent->k, - len, xent->masklen, xent->type, xent->value) : - ipfw_del_table_entry(chain, xent->tbl, &xent->k, - len, xent->masklen, xent->type); + ipfw_add_table_entry(chain, &ti, &tei) : + ipfw_del_table_entry(chain, &ti, &tei); } break; case IP_FW_TABLE_FLUSH: { u_int16_t tbl; + struct tid_info ti; error = sooptcopyin(sopt, &tbl, sizeof(tbl), sizeof(tbl)); if (error) break; - error = ipfw_flush_table(chain, tbl); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl; + error = ipfw_flush_table(chain, &ti); } break; case IP_FW_TABLE_GETSIZE: { u_int32_t tbl, cnt; + struct tid_info ti; if ((error = sooptcopyin(sopt, &tbl, sizeof(tbl), sizeof(tbl)))) break; + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl; IPFW_RLOCK(chain); - error = ipfw_count_table(chain, tbl, &cnt); + error = ipfw_count_table(chain, &ti, &cnt); IPFW_RUNLOCK(chain); if (error) break; @@ -1199,6 +1301,7 @@ case IP_FW_TABLE_LIST: { ipfw_table *tbl; + struct tid_info ti; if (sopt->sopt_valsize < sizeof(*tbl)) { error = EINVAL; @@ -1213,8 +1316,11 @@ } tbl->size = (size - sizeof(*tbl)) / sizeof(ipfw_table_entry); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl->tbl; IPFW_RLOCK(chain); - error = ipfw_dump_table(chain, tbl); + error = ipfw_dump_table(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) { free(tbl, M_TEMP); @@ -1228,16 +1334,20 @@ case IP_FW_TABLE_XGETSIZE: /* IP_FW3 */ { uint32_t *tbl; + struct tid_info ti; if (IP_FW3_OPLENGTH(sopt) < sizeof(uint32_t)) { error = EINVAL; break; } tbl = (uint32_t *)(op3 + 1); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = *tbl; IPFW_RLOCK(chain); - error = ipfw_count_xtable(chain, *tbl, tbl); + error = ipfw_count_xtable(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) break; @@ -1248,6 +1358,7 @@ case IP_FW_TABLE_XLIST: /* IP_FW3 */ { ipfw_xtable *tbl; + struct tid_info ti; if ((size = valsize) < sizeof(ipfw_xtable)) { error = EINVAL; @@ -1260,8 +1371,11 @@ /* Get maximum number of entries we can store */ tbl->size = (size - sizeof(ipfw_xtable)) / sizeof(ipfw_table_xentry); + memset(&ti, 0, sizeof(ti)); + ti.set = 0; /* XXX: No way to specify set */ + ti.uidx = tbl->tbl; IPFW_RLOCK(chain); - error = ipfw_dump_xtable(chain, tbl); + error = ipfw_dump_xtable(chain, &ti, tbl); IPFW_RUNLOCK(chain); if (error) { free(tbl, M_TEMP); @@ -1444,4 +1558,271 @@ return 0; } +/* + * Named object api + * + */ + +void +ipfw_objhash_bitmap_alloc(uint32_t items, void **idx, int *pblocks) +{ + size_t size; + int max_blocks; + void *idx_mask; + + items = roundup2(items, BLOCK_ITEMS); /* Align to block size */ + max_blocks = items / BLOCK_ITEMS; + size = items / 8; + idx_mask = malloc(size * IPFW_MAX_SETS, M_IPFW, M_WAITOK); + /* Mark all as free */ + memset(idx_mask, 0xFF, size * IPFW_MAX_SETS); + + *idx = idx_mask; + *pblocks = max_blocks; +} + +int +ipfw_objhash_bitmap_merge(struct namedobj_instance *ni, void **idx, int *blocks) +{ + int old_blocks, new_blocks; + u_long *old_idx, *new_idx; + int i; + + old_idx = ni->idx_mask; + old_blocks = ni->max_blocks; + new_idx = *idx; + new_blocks = *blocks; + + /* + * FIXME: Permit reducing total amount of tables + */ + if (old_blocks > new_blocks) + return (1); + + for (i = 0; i < IPFW_MAX_SETS; i++) { + memcpy(&new_idx[new_blocks * i], &old_idx[old_blocks * i], + old_blocks * sizeof(u_long)); + } + + ni->idx_mask = new_idx; + ni->max_blocks = new_blocks; + + /* Save old values */ + *idx = old_idx; + *blocks = old_blocks; + + return (0); +} + +void +ipfw_objhash_bitmap_free(void *idx, int blocks) +{ + + free(idx, M_IPFW); +} + +/* + * Creates named hash instance. + * Must be called without holding any locks. + * Return pointer to new instance. + */ +struct namedobj_instance * +ipfw_objhash_create(uint32_t items) +{ + struct namedobj_instance *ni; + int i; + size_t size; + + size = sizeof(struct namedobj_instance) + + sizeof(struct namedobjects_head) * NAMEDOBJ_HASH_SIZE + + sizeof(struct namedobjects_head) * NAMEDOBJ_HASH_SIZE; + + ni = malloc(size, M_IPFW, M_WAITOK | M_ZERO); + ni->nn_size = NAMEDOBJ_HASH_SIZE; + ni->nv_size = NAMEDOBJ_HASH_SIZE; + + ni->names = (struct namedobjects_head *)(ni +1); + ni->values = &ni->names[ni->nn_size]; + + for (i = 0; i < ni->nn_size; i++) + TAILQ_INIT(&ni->names[i]); + + for (i = 0; i < ni->nv_size; i++) + TAILQ_INIT(&ni->values[i]); + + /* Allocate bitmask separately due to possible resize */ + ipfw_objhash_bitmap_alloc(items, (void*)&ni->idx_mask, &ni->max_blocks); + + return (ni); +} + +void +ipfw_objhash_destroy(struct namedobj_instance *ni) +{ + + free(ni->idx_mask, M_IPFW); + free(ni, M_IPFW); +} + +static uint32_t +objhash_hash_name(struct namedobj_instance *ni, uint32_t set, char *name) +{ + uint32_t v; + + v = fnv_32_str(name, FNV1_32_INIT); + + return (v % ni->nn_size); +} + +static uint32_t +objhash_hash_val(struct namedobj_instance *ni, uint32_t set, uint32_t val) +{ + uint32_t v; + + v = val % (ni->nv_size - 1); + + return (v); +} + +struct named_object * +ipfw_objhash_lookup_name(struct namedobj_instance *ni, uint32_t set, char *name) +{ + struct named_object *no; + uint32_t hash; + + hash = objhash_hash_name(ni, set, name); + + TAILQ_FOREACH(no, &ni->names[hash], nn_next) { + if ((strcmp(no->name, name) == 0) && (no->set == set)) + return (no); + } + + return (NULL); +} + +struct named_object * +ipfw_objhash_lookup_idx(struct namedobj_instance *ni, uint32_t set, + uint16_t idx) +{ + struct named_object *no; + uint32_t hash; + + hash = objhash_hash_val(ni, set, idx); + + TAILQ_FOREACH(no, &ni->values[hash], nv_next) { + if ((no->kidx == idx) && (no->set == set)) + return (no); + } + + return (NULL); +} + +void +ipfw_objhash_add(struct namedobj_instance *ni, struct named_object *no) +{ + uint32_t hash; + + hash = objhash_hash_name(ni, no->set, no->name); + TAILQ_INSERT_HEAD(&ni->names[hash], no, nn_next); + + hash = objhash_hash_val(ni, no->set, no->kidx); + TAILQ_INSERT_HEAD(&ni->values[hash], no, nv_next); +} + +void +ipfw_objhash_del(struct namedobj_instance *ni, struct named_object *no) +{ + uint32_t hash; + + hash = objhash_hash_name(ni, no->set, no->name); + TAILQ_REMOVE(&ni->names[hash], no, nn_next); + + hash = objhash_hash_val(ni, no->set, no->kidx); + TAILQ_REMOVE(&ni->values[hash], no, nv_next); +} + +/* + * Runs @func for each found named object. + * It is safe to delete objects from callback + */ +void +ipfw_objhash_foreach(struct namedobj_instance *ni, objhash_cb_t *f, void *arg) +{ + struct named_object *no, *no_tmp; + int i; + + for (i = 0; i < ni->nn_size; i++) { + TAILQ_FOREACH_SAFE(no, &ni->names[i], nn_next, no_tmp) + f(ni, no, arg); + } +} + +/* + * Removes index from given set. + * Returns 0 on success. + */ +int +ipfw_objhash_free_idx(struct namedobj_instance *ni, uint32_t set, uint16_t idx) +{ + u_long *mask; + int i, v; + + i = idx / BLOCK_ITEMS; + v = idx % BLOCK_ITEMS; + + if ((i >= ni->max_blocks) || set >= IPFW_MAX_SETS) + return (1); + + mask = &ni->idx_mask[set * ni->max_blocks + i]; + + if ((*mask & ((u_long)1 << v)) != 0) + return (1); + + /* Mark as free */ + *mask |= (u_long)1 << v; + + /* Update free offset */ + if (ni->free_off[set] > i) + ni->free_off[set] = i; + + return (0); +} + +/* + * Allocate new index in given set and stores in in @pidx. + * Returns 0 on success. + */ +int +ipfw_objhash_alloc_idx(void *n, uint32_t set, uint16_t *pidx) +{ + struct namedobj_instance *ni; + u_long *mask; + int i, off, v; + + if (set >= IPFW_MAX_SETS) + return (-1); + + ni = (struct namedobj_instance *)n; + + off = ni->free_off[set]; + mask = &ni->idx_mask[set * ni->max_blocks + off]; + + for (i = off; i < ni->max_blocks; i++, mask++) { + if ((v = ffsl(*mask)) == 0) + continue; + + /* Mark as busy */ + *mask &= ~ ((u_long)1 << (v - 1)); + + ni->free_off[set] = i; + + v = BLOCK_ITEMS * i + v - 1; + + *pidx = v; + return (0); + } + + return (1); +} + /* end of file */ Index: sys/netpfil/ipfw/ip_fw_table.c =================================================================== --- sys/netpfil/ipfw/ip_fw_table.c +++ sys/netpfil/ipfw/ip_fw_table.c @@ -100,6 +100,49 @@ u_int32_t value; }; + /* + * Table has the following `type` concepts: + * + * `type` represents lookup key type (cidr, ifp, uid, etc..) + * `ftype` is pure userland field helping to properly format table data + * `atype` represents exact lookup algorithm for given tabletype. + * For example, we can use more efficient search schemes if we plan + * to use some specific table for storing host-routes only. + * + */ +struct table_config { + struct named_object no; + uint8_t ftype; /* format table type */ + uint8_t atype; /* algorith type */ + uint8_t linked; /* 1 if already linked */ + uint8_t spare0; + uint32_t count; /* Number of records */ + char tablename[64]; /* table name */ + void *state; /* Store some state if needed */ + void *xstate; +}; +#define TABLE_SET(set) ((V_fw_tables_sets != 0) ? set : 0) + +struct tables_config { + struct namedobj_instance *namehash; +}; + +static struct table_config *find_table(struct namedobj_instance *ni, + struct tid_info *ti); +static struct table_config *alloc_table_config(struct namedobj_instance *ni, + struct tid_info *ti); +static void free_table_config(struct namedobj_instance *ni, + struct table_config *tc); +static void link_table(struct ip_fw_chain *chain, struct table_config *tc); +static void unlink_table(struct ip_fw_chain *chain, struct table_config *tc); +static int alloc_table_state(void **state, void **xstate, uint8_t type); +static void free_table_state(void **state, void **xstate, uint8_t type); + + +#define CHAIN_TO_TCFG(chain) ((struct tables_config *)(chain)->tblcfg) +#define CHAIN_TO_NI(chain) (CHAIN_TO_TCFG(chain)->namehash) + + /* * The radix code expects addr and mask to be array of bytes, * with the first byte being the length of the array. rn_inithead @@ -136,62 +179,68 @@ #endif int -ipfw_add_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type, uint32_t value) +ipfw_add_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei) { - struct radix_node_head *rnh, **rnh_ptr; + struct radix_node_head *rnh; struct table_entry *ent; struct table_xentry *xent; struct radix_node *rn; in_addr_t addr; int offset; void *ent_ptr; struct sockaddr *addr_ptr, *mask_ptr; + struct table_config *tc, *tc_new; + struct namedobj_instance *ni; char c; + uint8_t mlen; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); - switch (type) { + mlen = tei->masklen; + + switch (ti->type) { case IPFW_TABLE_CIDR: - if (plen == sizeof(in_addr_t)) { + if (tei->plen == sizeof(in_addr_t)) { #ifdef INET /* IPv4 case */ if (mlen > 32) return (EINVAL); ent = malloc(sizeof(*ent), M_IPFW_TBL, M_WAITOK | M_ZERO); - ent->value = value; + ent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(ent->addr) = KEY_LEN_INET; KEY_LEN(ent->mask) = KEY_LEN_INET; /* Set offset of IPv4 address in bits */ offset = OFF_LEN_INET; - ent->mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); - addr = *((in_addr_t *)paddr); + ent->mask.sin_addr.s_addr = + htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); + addr = *((in_addr_t *)tei->paddr); ent->addr.sin_addr.s_addr = addr & ent->mask.sin_addr.s_addr; /* Set pointers */ - rnh_ptr = &ch->tables[tbl]; ent_ptr = ent; addr_ptr = (struct sockaddr *)&ent->addr; mask_ptr = (struct sockaddr *)&ent->mask; #endif #ifdef INET6 - } else if (plen == sizeof(struct in6_addr)) { + } else if (tei->plen == sizeof(struct in6_addr)) { /* IPv6 case */ if (mlen > 128) return (EINVAL); xent = malloc(sizeof(*xent), M_IPFW_TBL, M_WAITOK | M_ZERO); - xent->value = value; + xent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(xent->a.addr6) = KEY_LEN_INET6; KEY_LEN(xent->m.mask6) = KEY_LEN_INET6; /* Set offset of IPv6 address in bits */ offset = OFF_LEN_INET6; ipv6_writemask(&xent->m.mask6.sin6_addr, mlen); - memcpy(&xent->a.addr6.sin6_addr, paddr, sizeof(struct in6_addr)); + memcpy(&xent->a.addr6.sin6_addr, tei->paddr, + sizeof(struct in6_addr)); APPLY_MASK(&xent->a.addr6.sin6_addr, &xent->m.mask6.sin6_addr); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.addr6; mask_ptr = (struct sockaddr *)&xent->m.mask6; @@ -204,30 +253,30 @@ case IPFW_TABLE_INTERFACE: /* Check if string is terminated */ - c = ((char *)paddr)[IF_NAMESIZE - 1]; - ((char *)paddr)[IF_NAMESIZE - 1] = '\0'; - if (((mlen = strlen((char *)paddr)) == IF_NAMESIZE - 1) && (c != '\0')) + c = ((char *)tei->paddr)[IF_NAMESIZE - 1]; + ((char *)tei->paddr)[IF_NAMESIZE - 1] = '\0'; + mlen = strlen((char *)tei->paddr); + if ((mlen == IF_NAMESIZE - 1) && (c != '\0')) return (EINVAL); /* Include last \0 into comparison */ mlen++; xent = malloc(sizeof(*xent), M_IPFW_TBL, M_WAITOK | M_ZERO); - xent->value = value; + xent->value = tei->value; /* Set 'total' structure length */ KEY_LEN(xent->a.iface) = KEY_LEN_IFACE + mlen; KEY_LEN(xent->m.ifmask) = KEY_LEN_IFACE + mlen; /* Set offset of interface name in bits */ offset = OFF_LEN_IFACE; - memcpy(xent->a.iface.ifname, paddr, mlen); + memcpy(xent->a.iface.ifname, tei->paddr, mlen); /* Assume direct match */ /* TODO: Add interface pattern matching */ #if 0 memset(xent->m.ifmask.ifname, 0xFF, IF_NAMESIZE); mask_ptr = (struct sockaddr *)&xent->m.ifmask; #endif /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; ent_ptr = xent; addr_ptr = (struct sockaddr *)&xent->a.iface; mask_ptr = NULL; @@ -237,84 +286,128 @@ return (EINVAL); } - IPFW_WLOCK(ch); + IPFW_UH_WLOCK(ch); - /* Check if tabletype is valid */ - if ((ch->tabletype[tbl] != 0) && (ch->tabletype[tbl] != type)) { - IPFW_WUNLOCK(ch); - free(ent_ptr, M_IPFW_TBL); - return (EINVAL); - } + ni = CHAIN_TO_NI(ch); - /* Check if radix tree exists */ - if ((rnh = *rnh_ptr) == NULL) { - IPFW_WUNLOCK(ch); - /* Create radix for a new table */ - if (!rn_inithead((void **)&rnh, offset)) { - free(ent_ptr, M_IPFW_TBL); + tc_new = NULL; + if ((tc = find_table(ni, ti)) == NULL) { + /* Not found. We have to create new one */ + IPFW_UH_WUNLOCK(ch); + + tc_new = alloc_table_config(ni, ti); + if (tc_new == NULL) return (ENOMEM); - } - IPFW_WLOCK(ch); - if (*rnh_ptr != NULL) { - /* Tree is already attached by other thread */ - rn_detachhead((void **)&rnh); - rnh = *rnh_ptr; - /* Check table type another time */ - if (ch->tabletype[tbl] != type) { - IPFW_WUNLOCK(ch); - free(ent_ptr, M_IPFW_TBL); + IPFW_UH_WLOCK(ch); + + /* Check if table has already allocated by other thread */ + if ((tc = find_table(ni, ti)) != NULL) { + if (tc->no.type != ti->type) { + IPFW_UH_WUNLOCK(ch); + free_table_config(ni, tc); return (EINVAL); } } else { - *rnh_ptr = rnh; - /* - * Set table type. It can be set already - * (if we have IPv6-only table) but setting - * it another time does not hurt + /* + * New table. + * Set tc_new to zero not to free it afterwards. */ - ch->tabletype[tbl] = type; + tc = tc_new; + tc_new = NULL; + + /* Allocate table index. */ + if (ipfw_objhash_alloc_idx(ni, ti->set, &kidx) != 0) { + /* Index full. */ + IPFW_UH_WUNLOCK(ch); + printf("Unable to allocate index for table %s." + " Consider increasing " + "net.inet.ip.fw.tables_max", + tc->no.name); + free_table_config(ni, tc); + return (EBUSY); + } + /* Save kidx */ + tc->no.kidx = kidx; } + } else { + /* We still have to check table type */ + if (tc->no.type != ti->type) { + IPFW_UH_WUNLOCK(ch); + return (EINVAL); + } + } + kidx = tc->no.kidx; + + /* We've got valid table in @tc. Let's add data */ + IPFW_WLOCK(ch); + + if (tc->linked == 0) { + link_table(ch, tc); + } + + /* XXX: Temporary until splitting add/del to per-type functions */ + rnh = NULL; + switch (ti->type) { + case IPFW_TABLE_CIDR: + if (tei->plen == sizeof(in_addr_t)) + rnh = ch->tables[kidx]; + else + rnh = ch->xtables[kidx]; + break; + case IPFW_TABLE_INTERFACE: + rnh = ch->xtables[kidx]; + break; } rn = rnh->rnh_addaddr(addr_ptr, mask_ptr, rnh, ent_ptr); IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); + + if (tc_new != NULL) + free_table_config(ni, tc); if (rn == NULL) { free(ent_ptr, M_IPFW_TBL); return (EEXIST); } + return (0); } int -ipfw_del_table_entry(struct ip_fw_chain *ch, uint16_t tbl, void *paddr, - uint8_t plen, uint8_t mlen, uint8_t type) +ipfw_del_table_entry(struct ip_fw_chain *ch, struct tid_info *ti, + struct tentry_info *tei) { - struct radix_node_head *rnh, **rnh_ptr; + struct radix_node_head *rnh; struct table_entry *ent; in_addr_t addr; struct sockaddr_in sa, mask; struct sockaddr *sa_ptr, *mask_ptr; + struct table_config *tc; + struct namedobj_instance *ni; char c; + uint8_t mlen; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); - switch (type) { + mlen = tei->masklen; + + switch (ti->type) { case IPFW_TABLE_CIDR: - if (plen == sizeof(in_addr_t)) { + if (tei->plen == sizeof(in_addr_t)) { /* Set 'total' structure length */ KEY_LEN(sa) = KEY_LEN_INET; KEY_LEN(mask) = KEY_LEN_INET; mask.sin_addr.s_addr = htonl(mlen ? ~((1 << (32 - mlen)) - 1) : 0); - addr = *((in_addr_t *)paddr); + addr = *((in_addr_t *)tei->paddr); sa.sin_addr.s_addr = addr & mask.sin_addr.s_addr; - rnh_ptr = &ch->tables[tbl]; sa_ptr = (struct sockaddr *)&sa; mask_ptr = (struct sockaddr *)&mask; #ifdef INET6 - } else if (plen == sizeof(struct in6_addr)) { + } else if (tei->plen == sizeof(struct in6_addr)) { /* IPv6 case */ if (mlen > 128) return (EINVAL); @@ -325,9 +418,9 @@ KEY_LEN(sa6) = KEY_LEN_INET6; KEY_LEN(mask6) = KEY_LEN_INET6; ipv6_writemask(&mask6.sin6_addr, mlen); - memcpy(&sa6.sin6_addr, paddr, sizeof(struct in6_addr)); + memcpy(&sa6.sin6_addr, tei->paddr, + sizeof(struct in6_addr)); APPLY_MASK(&sa6.sin6_addr, &mask6.sin6_addr); - rnh_ptr = &ch->xtables[tbl]; sa_ptr = (struct sockaddr *)&sa6; mask_ptr = (struct sockaddr *)&mask6; #endif @@ -339,9 +432,10 @@ case IPFW_TABLE_INTERFACE: /* Check if string is terminated */ - c = ((char *)paddr)[IF_NAMESIZE - 1]; - ((char *)paddr)[IF_NAMESIZE - 1] = '\0'; - if (((mlen = strlen((char *)paddr)) == IF_NAMESIZE - 1) && (c != '\0')) + c = ((char *)tei->paddr)[IF_NAMESIZE - 1]; + ((char *)tei->paddr)[IF_NAMESIZE - 1] = '\0'; + mlen = strlen((char *)tei->paddr); + if ((mlen == IF_NAMESIZE - 1) && (c != '\0')) return (EINVAL); struct xaddr_iface ifname, ifmask; @@ -360,31 +454,49 @@ mask_ptr = (struct sockaddr *)&ifmask; #endif mask_ptr = NULL; - memcpy(ifname.ifname, paddr, mlen); + memcpy(ifname.ifname, tei->paddr, mlen); /* Set pointers */ - rnh_ptr = &ch->xtables[tbl]; sa_ptr = (struct sockaddr *)&ifname; break; default: return (EINVAL); } - IPFW_WLOCK(ch); - if ((rnh = *rnh_ptr) == NULL) { - IPFW_WUNLOCK(ch); + IPFW_UH_RLOCK(ch); + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_RUNLOCK(ch); return (ESRCH); } - if (ch->tabletype[tbl] != type) { - IPFW_WUNLOCK(ch); + if (tc->no.type != ti->type) { + IPFW_UH_RUNLOCK(ch); return (EINVAL); } + kidx = tc->no.kidx; + + IPFW_WLOCK(ch); + + rnh = NULL; + switch (ti->type) { + case IPFW_TABLE_CIDR: + if (tei->plen == sizeof(in_addr_t)) + rnh = ch->tables[kidx]; + else + rnh = ch->xtables[kidx]; + break; + case IPFW_TABLE_INTERFACE: + rnh = ch->xtables[kidx]; + break; + } ent = (struct table_entry *)rnh->rnh_deladdr(sa_ptr, mask_ptr, rnh); IPFW_WUNLOCK(ch); + IPFW_UH_RUNLOCK(ch); + if (ent == NULL) return (ESRCH); @@ -405,102 +517,206 @@ return (0); } +/* + * Flushes all entries in given table minimizing hoding chain WLOCKs. + * + */ int -ipfw_flush_table(struct ip_fw_chain *ch, uint16_t tbl) +ipfw_flush_table(struct ip_fw_chain *ch, struct tid_info *ti) { - struct radix_node_head *rnh, *xrnh; + struct namedobj_instance *ni; + struct table_config *tc; + void *ostate, *oxstate; + void *state, *xstate; + int error; + uint8_t type; + uint16_t kidx; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); /* - * We free both (IPv4 and extended) radix trees and - * clear table type here to permit table to be reused - * for different type without module reload + * Stage 1: determine table type. + * Reference found table to ensure it won't disappear. + */ + IPFW_UH_WLOCK(ch); + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_WUNLOCK(ch); + return (ESRCH); + } + type = tc->no.type; + tc->no.refcnt++; + IPFW_UH_WUNLOCK(ch); + + /* + * Stage 2: allocate new state for given type. */ + if ((error = alloc_table_state(&state, &xstate, type)) != 0) { + IPFW_UH_WLOCK(ch); + tc->no.refcnt--; + IPFW_UH_WUNLOCK(ch); + return (error); + } + /* + * Stage 3: swap old state pointers with newly-allocated ones. + * Decrease refcount. + */ + IPFW_UH_WLOCK(ch); IPFW_WLOCK(ch); - /* Set IPv4 table pointer to zero */ - if ((rnh = ch->tables[tbl]) != NULL) - ch->tables[tbl] = NULL; - /* Set extended table pointer to zero */ - if ((xrnh = ch->xtables[tbl]) != NULL) - ch->xtables[tbl] = NULL; - /* Zero table type */ - ch->tabletype[tbl] = 0; + + ni = CHAIN_TO_NI(ch); + kidx = tc->no.kidx; + + ostate = ch->tables[kidx]; + ch->tables[kidx] = state; + oxstate = ch->xtables[kidx]; + ch->xtables[kidx] = xstate; + + tc->no.refcnt--; + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); - if (rnh != NULL) { - rnh->rnh_walktree(rnh, flush_table_entry, rnh); - rn_detachhead((void **)&rnh); + /* + * Stage 4: perform real flush. + */ + free_table_state(&ostate, &xstate, tc->no.type); + + return (0); +} + +/* + * Destroys given table @ti: flushes it, + */ +int +ipfw_destroy_table(struct ip_fw_chain *ch, struct tid_info *ti, int force) +{ + struct namedobj_instance *ni; + struct table_config *tc; + + ti->set = TABLE_SET(ti->set); + + IPFW_UH_WLOCK(ch); + + ni = CHAIN_TO_NI(ch); + if ((tc = find_table(ni, ti)) == NULL) { + IPFW_UH_WUNLOCK(ch); + return (ESRCH); } - if (xrnh != NULL) { - xrnh->rnh_walktree(xrnh, flush_table_entry, xrnh); - rn_detachhead((void **)&xrnh); + /* Do not permit destroying used tables */ + if (tc->no.refcnt > 0 && force == 0) { + IPFW_UH_WUNLOCK(ch); + return (EBUSY); } + IPFW_WLOCK(ch); + unlink_table(ch, tc); + IPFW_WUNLOCK(ch); + + /* Free obj index */ + if (ipfw_objhash_free_idx(ni, tc->no.set, tc->no.kidx) != 0) + printf("Error unlinking kidx %d from table %s\n", + tc->no.kidx, tc->tablename); + + IPFW_UH_WUNLOCK(ch); + + free_table_config(ni, tc); + return (0); } +static void +destroy_table_locked(struct namedobj_instance *ni, struct named_object *no, + void *arg) +{ + + unlink_table((struct ip_fw_chain *)arg, (struct table_config *)no); + if (ipfw_objhash_free_idx(ni, no->set, no->kidx) != 0) + printf("Error unlinking kidx %d from table %s\n", + no->kidx, no->name); + free_table_config(ni, (struct table_config *)no); +} + void ipfw_destroy_tables(struct ip_fw_chain *ch) { - uint16_t tbl; - /* Flush all tables */ - for (tbl = 0; tbl < V_fw_tables_max; tbl++) - ipfw_flush_table(ch, tbl); + /* Remove all tables from working set */ + IPFW_UH_WLOCK(ch); + IPFW_WLOCK(ch); + ipfw_objhash_foreach(CHAIN_TO_NI(ch), destroy_table_locked, ch); + IPFW_WUNLOCK(ch); + IPFW_UH_WUNLOCK(ch); /* Free pointers itself */ free(ch->tables, M_IPFW); free(ch->xtables, M_IPFW); - free(ch->tabletype, M_IPFW); + + ipfw_objhash_destroy(CHAIN_TO_NI(ch)); + free(CHAIN_TO_TCFG(ch), M_IPFW); } int ipfw_init_tables(struct ip_fw_chain *ch) { + struct tables_config *tcfg; + /* Allocate pointers */ ch->tables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); ch->xtables = malloc(V_fw_tables_max * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - ch->tabletype = malloc(V_fw_tables_max * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + + tcfg = malloc(sizeof(struct tables_config), M_IPFW, M_WAITOK | M_ZERO); + tcfg->namehash = ipfw_objhash_create(V_fw_tables_max); + ch->tblcfg = tcfg; + return (0); } int ipfw_resize_tables(struct ip_fw_chain *ch, unsigned int ntables) { struct radix_node_head **tables, **xtables, *rnh; struct radix_node_head **tables_old, **xtables_old; - uint8_t *tabletype, *tabletype_old; unsigned int ntables_old, tbl; + struct namedobj_instance *ni; + void *new_idx; + int new_blocks; /* Check new value for validity */ if (ntables > IPFW_TABLES_MAX) ntables = IPFW_TABLES_MAX; /* Allocate new pointers */ tables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); xtables = malloc(ntables * sizeof(void *), M_IPFW, M_WAITOK | M_ZERO); - tabletype = malloc(ntables * sizeof(uint8_t), M_IPFW, M_WAITOK | M_ZERO); + ipfw_objhash_bitmap_alloc(ntables, (void *)&new_idx, &new_blocks); IPFW_WLOCK(ch); tbl = (ntables >= V_fw_tables_max) ? V_fw_tables_max : ntables; + ni = CHAIN_TO_NI(ch); + + /* Temportary restrict decreasing max_tables */ + if (ipfw_objhash_bitmap_merge(ni, &new_idx, &new_blocks) != 0) { + IPFW_WUNLOCK(ch); + free(tables, M_IPFW); + free(xtables, M_IPFW); + ipfw_objhash_bitmap_free(new_idx, new_blocks); + return (EINVAL); + } /* Copy old table pointers */ memcpy(tables, ch->tables, sizeof(void *) * tbl); memcpy(xtables, ch->xtables, sizeof(void *) * tbl); - memcpy(tabletype, ch->tabletype, sizeof(uint8_t) * tbl); /* Change pointers and number of tables */ tables_old = ch->tables; xtables_old = ch->xtables; - tabletype_old = ch->tabletype; ch->tables = tables; ch->xtables = xtables; - ch->tabletype = tabletype; ntables_old = V_fw_tables_max; V_fw_tables_max = ntables; @@ -525,7 +741,7 @@ /* Free old pointers */ free(tables_old, M_IPFW); free(xtables_old, M_IPFW); - free(tabletype_old, M_IPFW); + ipfw_objhash_bitmap_free(new_idx, new_blocks); return (0); } @@ -602,14 +818,17 @@ } int -ipfw_count_table(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) +ipfw_count_table(struct ip_fw_chain *ch, struct tid_info *ti, uint32_t *cnt) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (ESRCH); *cnt = 0; - if ((rnh = ch->tables[tbl]) == NULL) + if ((rnh = ch->tables[tc->no.kidx]) == NULL) return (0); rnh->rnh_walktree(rnh, count_table_entry, cnt); return (0); @@ -637,14 +856,17 @@ } int -ipfw_dump_table(struct ip_fw_chain *ch, ipfw_table *tbl) +ipfw_dump_table(struct ip_fw_chain *ch, struct tid_info *ti, ipfw_table *tbl) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl->tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (ESRCH); tbl->cnt = 0; - if ((rnh = ch->tables[tbl->tbl]) == NULL) + if ((rnh = ch->tables[tc->no.kidx]) == NULL) return (0); rnh->rnh_walktree(rnh, dump_table_entry, tbl); return (0); @@ -660,16 +882,19 @@ } int -ipfw_count_xtable(struct ip_fw_chain *ch, uint32_t tbl, uint32_t *cnt) +ipfw_count_xtable(struct ip_fw_chain *ch, struct tid_info *ti, uint32_t *cnt) { struct radix_node_head *rnh; + struct table_config *tc; - if (tbl >= V_fw_tables_max) + if (ti->uidx >= V_fw_tables_max) return (EINVAL); *cnt = 0; - if ((rnh = ch->tables[tbl]) != NULL) + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (0); /* XXX: We should return ESRCH */ + if ((rnh = ch->tables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, count_table_xentry, cnt); - if ((rnh = ch->xtables[tbl]) != NULL) + if ((rnh = ch->xtables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, count_table_xentry, cnt); /* Return zero if table is empty */ if (*cnt > 0) @@ -747,19 +972,700 @@ } int -ipfw_dump_xtable(struct ip_fw_chain *ch, ipfw_xtable *tbl) +ipfw_dump_xtable(struct ip_fw_chain *ch, struct tid_info *ti, ipfw_xtable *tbl) { struct radix_node_head *rnh; + struct table_config *tc; if (tbl->tbl >= V_fw_tables_max) return (EINVAL); tbl->cnt = 0; - tbl->type = ch->tabletype[tbl->tbl]; - if ((rnh = ch->tables[tbl->tbl]) != NULL) + + if ((tc = find_table(CHAIN_TO_NI(ch), ti)) == NULL) + return (0); /* XXX: We should return ESRCH */ + tbl->type = tc->no.type; + if ((rnh = ch->tables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_base, tbl); - if ((rnh = ch->xtables[tbl->tbl]) != NULL) + if ((rnh = ch->xtables[tc->no.kidx]) != NULL) rnh->rnh_walktree(rnh, dump_table_xentry_extended, tbl); return (0); } +/* + * Tables rewriting code + * + */ + +/* + * Determine table number and lookup type for @cmd. + * Fill @tbl and @type with appropriate values. + * Returns 0 for relevant opcodes, 1 otherwise. + */ +static int +classify_table_opcode(ipfw_insn *cmd, uint16_t *puidx, uint8_t *ptype) +{ + ipfw_insn_if *cmdif; + int skip; + uint16_t v; + + skip = 1; + + switch (cmd->opcode) { + case O_IP_SRC_LOOKUP: + case O_IP_DST_LOOKUP: + /* Basic IPv4/IPv6 or u32 lookups */ + *puidx = cmd->arg1; + /* Assume CIDR by default */ + *ptype = IPFW_TABLE_CIDR; + skip = 0; + + if (F_LEN(cmd) > F_INSN_SIZE(ipfw_insn_u32)) { + /* + * generic lookup. The key must be + * in 32bit big-endian format. + */ + v = ((ipfw_insn_u32 *)cmd)->d[1]; + switch (v) { + case 0: + case 1: + /* IPv4 src/dst */ + break; + case 2: + case 3: + /* src/dst port */ + //type = IPFW_TABLE_U16; + break; + case 4: + /* uid/gid */ + //type = IPFW_TABLE_U32; + case 5: + //type = IPFW_TABLE_U32; + /* jid */ + case 6: + //type = IPFW_TABLE_U16; + /* dscp */ + break; + } + } + break; + case O_XMIT: + case O_RECV: + case O_VIA: + /* Interface table, possibly */ + cmdif = (ipfw_insn_if *)cmd; + if (cmdif->name[0] != '\1') + break; + + *ptype = IPFW_TABLE_INTERFACE; + *puidx = cmdif->p.glob; + skip = 0; + break; + } + + return (skip); +} + +/* + * Sets new table value for given opcode. + * Assume the same opcodes as classify_table_opcode() + */ +static void +update_table_opcode(ipfw_insn *cmd, uint16_t idx) +{ + ipfw_insn_if *cmdif; + + switch (cmd->opcode) { + case O_IP_SRC_LOOKUP: + case O_IP_DST_LOOKUP: + /* Basic IPv4/IPv6 or u32 lookups */ + cmd->arg1 = idx; + break; + case O_XMIT: + case O_RECV: + case O_VIA: + /* Interface table, possibly */ + cmdif = (ipfw_insn_if *)cmd; + cmdif->p.glob = idx; + break; + } +} + +static char * +find_name_tlv(void *tlvs, int len, uint16_t uidx) +{ + ipfw_xtable_ntlv *ntlv; + uintptr_t pa, pe; + int l; + + pa = (uintptr_t)tlvs; + pe = pa + len; + l = 0; + for (; pa < pe; pa += l) { + ntlv = (ipfw_xtable_ntlv *)pa; + l = ntlv->head.length; + if (ntlv->head.type != IPFW_TLV_NAME) + continue; + if (ntlv->idx != uidx) + continue; + + return (ntlv->name); + } + + return (NULL); +} + +static struct table_config * +find_table(struct namedobj_instance *ni, struct tid_info *ti) +{ + char *name, bname[16]; + struct named_object *no; + + if (ti->tlvs != NULL) { + name = find_name_tlv(ti->tlvs, ti->tlen, ti->uidx); + if (name == NULL) + return (NULL); + } else { + snprintf(bname, sizeof(bname), "%d", ti->uidx); + name = bname; + } + + no = ipfw_objhash_lookup_name(ni, ti->set, name); + + return ((struct table_config *)no); +} + +static int +alloc_table_state(void **state, void **xstate, uint8_t type) +{ + + switch (type) { + case IPFW_TABLE_CIDR: + if (!rn_inithead(state, OFF_LEN_INET)) + return (ENOMEM); + if (!rn_inithead(xstate, OFF_LEN_INET6)) { + rn_detachhead(state); + return (ENOMEM); + } + break; + case IPFW_TABLE_INTERFACE: + *state = NULL; + if (!rn_inithead(xstate, OFF_LEN_IFACE)) + return (ENOMEM); + break; + } + + return (0); +} + + +static struct table_config * +alloc_table_config(struct namedobj_instance *ni, struct tid_info *ti) +{ + char *name, bname[16]; + struct table_config *tc; + int error; + + if (ti->tlvs != NULL) { + name = find_name_tlv(ti->tlvs, ti->tlen, ti->uidx); + if (name == NULL) + return (NULL); + } else { + snprintf(bname, sizeof(bname), "%d", ti->uidx); + name = bname; + } + + tc = malloc(sizeof(struct table_config), M_IPFW, M_WAITOK | M_ZERO); + tc->no.name = tc->tablename; + tc->no.type = ti->type; + tc->no.set = ti->set; + strlcpy(tc->tablename, name, sizeof(tc->tablename)); + + if (ti->tlvs == NULL) { + tc->no.compat = 1; + tc->no.uidx = ti->uidx; + } + + /* Preallocate data structures for new tables */ + error = alloc_table_state(&tc->state, &tc->xstate, ti->type); + if (error != 0) { + free(tc, M_IPFW); + return (NULL); + } + + return (tc); +} + +static void +free_table_state(void **state, void **xstate, uint8_t type) +{ + struct radix_node_head *rnh; + + switch (type) { + case IPFW_TABLE_CIDR: + rnh = (struct radix_node_head *)(*state); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(state); + + rnh = (struct radix_node_head *)(*xstate); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(xstate); + break; + case IPFW_TABLE_INTERFACE: + rnh = (struct radix_node_head *)(*xstate); + rnh->rnh_walktree(rnh, flush_table_entry, rnh); + rn_detachhead(xstate); + break; + } +} + +static void +free_table_config(struct namedobj_instance *ni, struct table_config *tc) +{ + + if (tc->linked == 0) + free_table_state(&tc->state, &tc->xstate, tc->no.type); + + free(tc, M_IPFW); +} + +/* + * Links @tc to @chain table named instance. + * Sets appropriate type/states in @chain table info. + */ +static void +link_table(struct ip_fw_chain *chain, struct table_config *tc) +{ + struct namedobj_instance *ni; + uint16_t kidx; + + IPFW_UH_WLOCK_ASSERT(chain); + IPFW_WLOCK_ASSERT(chain); + + ni = CHAIN_TO_NI(chain); + kidx = tc->no.kidx; + + ipfw_objhash_add(ni, &tc->no); + chain->tables[kidx] = tc->state; + chain->xtables[kidx] = tc->xstate; + + tc->linked = 1; +} + +/* + * Unlinks @tc from @chain table named instance. + * Zeroes states in @chain and stores them in @tc. + */ +static void +unlink_table(struct ip_fw_chain *chain, struct table_config *tc) +{ + struct namedobj_instance *ni; + uint16_t kidx; + + IPFW_UH_WLOCK_ASSERT(chain); + IPFW_WLOCK_ASSERT(chain); + + ni = CHAIN_TO_NI(chain); + kidx = tc->no.kidx; + + /* Clear state and save pointers for flush */ + ipfw_objhash_del(ni, &tc->no); + tc->state = chain->tables[kidx]; + chain->tables[kidx] = NULL; + tc->xstate = chain->xtables[kidx]; + chain->xtables[kidx] = NULL; + + tc->linked = 0; +} + +/* + * Finds named object by @uidx number. + * Refs found object, allocate new index for non-existing object. + * Fills in @pidx with userland/kernel indexes. + * + * Returns 0 on success. + */ +static int +bind_table(struct namedobj_instance *ni, struct rule_check_info *ci, + struct obj_idx *pidx, struct tid_info *ti) +{ + struct table_config *tc; + + tc = find_table(ni, ti); + + pidx->uidx = ti->uidx; + pidx->type = ti->type; + + if (tc == NULL) { + /* Try to acquire refcount */ + if (ipfw_objhash_alloc_idx(ni, ti->set, &pidx->kidx) != 0) { + printf("Unable to allocate table index in set %u." + " Consider increasing net.inet.ip.fw.tables_max", + ti->set); + return (EBUSY); + } + + pidx->new = 1; + ci->new_tables++; + + return (0); + } + + /* Check if table type if valid first */ + if (tc->no.type != ti->type) + return (EINVAL); + + tc->no.refcnt++; + + pidx->kidx = tc->no.kidx; + + return (0); +} + +/* + * Compatibility function for old ipfw(8) binaries. + * Rewrites table kernel indices with userland ones. + * Works for \d+ talbes only (e.g. for tables, converted + * from old numbered system calls). + * + * Returns 0 on success. + * Raises error on any other tables. + */ +int +ipfw_rewrite_table_kidx(struct ip_fw_chain *chain, struct ip_fw *rule) +{ + int cmdlen, l; + ipfw_insn *cmd; + uint32_t set; + uint16_t kidx; + uint8_t type; + struct named_object *no; + struct namedobj_instance *ni; + + ni = CHAIN_TO_NI(chain); + + set = TABLE_SET(rule->set); + + l = rule->cmd_len; + cmd = rule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &kidx, &type) != 0) + continue; + + if ((no = ipfw_objhash_lookup_idx(ni, set, kidx)) == NULL) + return (1); + + if (no->compat == 0) + return (2); + + update_table_opcode(cmd, no->uidx); + } + + return (0); +} + + +/* + * Checks is opcode is referencing table of appropriate type. + * Adds reference count for found table if true. + * Rewrites user-supplied opcode values with kernel ones. + * + * Returns 0 on success and appropriate error code otherwise. + */ +int +ipfw_rewrite_table_uidx(struct ip_fw_chain *chain, + struct rule_check_info *ci) +{ + int cmdlen, error, ftype, l; + ipfw_insn *cmd; + uint16_t uidx; + uint8_t type; + struct table_config *tc; + struct namedobj_instance *ni; + struct named_object *no, *no_n, *no_tmp; + struct obj_idx *pidx, *p, *oib; + struct namedobjects_head nh; + struct tid_info ti; + + ni = CHAIN_TO_NI(chain); + + /* + * Prepare an array for storing opcode indices. + * Use stack allocation by default. + */ + if (ci->table_opcodes <= (sizeof(ci->obuf)/sizeof(ci->obuf[0]))) { + /* Stack */ + pidx = ci->obuf; + } else + pidx = malloc(ci->table_opcodes * sizeof(struct obj_idx), + M_IPFW, M_WAITOK | M_ZERO); + + oib = pidx; + error = 0; + + type = 0; + ftype = 0; + + ci->tableset = TABLE_SET(ci->krule->set); + + memset(&ti, 0, sizeof(ti)); + ti.set = ci->tableset; + ti.tlvs = ci->tlvs; + ti.tlen = ci->tlen; + + /* + * Stage 1: reference existing tables and determine number + * of tables we need to allocate + */ + IPFW_UH_WLOCK(chain); + + l = ci->krule->cmd_len; + cmd = ci->krule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &ti.uidx, &ti.type) != 0) + continue; + + /* + * Got table opcode with necessary info. + * Try to reference existing tables and allocate + * indices for non-existing one while holding write lock. + */ + if ((error = bind_table(ni, ci, pidx, &ti)) != 0) + break; + + /* + * @pidx stores either existing ref'd table id or new one. + * Move to next index + */ + + pidx++; + } + + if (error != 0) { + /* Unref everything we have already done */ + for (p = oib; p < pidx; p++) { + if (p->new != 0) { + ipfw_objhash_free_idx(ni, ci->tableset,p->kidx); + continue; + } + + /* Find & unref by existing idx */ + no = ipfw_objhash_lookup_idx(ni, ci->tableset, p->kidx); + KASSERT(no!=NULL, ("Ref'd table %d disappeared", + p->kidx)); + + no->refcnt--; + } + + IPFW_UH_WUNLOCK(chain); + + if (oib != ci->obuf) + free(oib, M_IPFW); + + return (error); + } + + IPFW_UH_WUNLOCK(chain); + + /* + * Stage 2: allocate table configs for every non-existent table + */ + + if (ci->new_tables > 0) { + /* Prepare queue to store configs */ + TAILQ_INIT(&nh); + + for (p = oib; p < pidx; p++) { + if (p->new == 0) + continue; + + /* TODO: get name from TLV */ + ti.uidx = p->uidx; + ti.type = p->type; + + tc = alloc_table_config(ni, &ti); + + if (tc == NULL) { + error = ENOMEM; + goto free; + } + + tc->no.kidx = p->kidx; + tc->no.refcnt = 1; + + /* Add to list */ + TAILQ_INSERT_TAIL(&nh, &tc->no, nn_next); + } + + /* + * Stage 2.1: Check if we're going to create 2 tables + * with the same name, but different table types. + */ + TAILQ_FOREACH(no, &nh, nn_next) { + TAILQ_FOREACH(no_tmp, &nh, nn_next) { + if (strcmp(no->name, no_tmp->name) != 0) + continue; + if (no->type != no_tmp->type) { + error = EINVAL; + goto free; + } + } + } + + /* + * Stage 3: link & reference new table configs + */ + + IPFW_UH_WLOCK(chain); + + /* + * Step 3.1: Check if some tables we need to create have been + * already created with different table type. + */ + + error = 0; + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) { + no_n = ipfw_objhash_lookup_name(ni, no->set, no->name); + if (no_n == NULL) + continue; + + if (no_n->type != no->type) { + error = EINVAL; + break; + } + + } + + if (error != 0) { + /* + * Someone has allocated table with different table type. + * We have to rollback everything. + */ + IPFW_UH_WUNLOCK(chain); + + goto free; + } + + + /* + * Finally, attach tables and rewrite rule. + * We need to set table type for each new table, + * so we have to acquire main WLOCK. + */ + IPFW_WLOCK(chain); + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) { + no_n = ipfw_objhash_lookup_name(ni, no->set, no->name); + if (no_n != NULL) { + /* Increase refcount for existing table */ + no_n->refcnt++; + /* Keep oib array in sync: update kindx */ + for (p = oib; p < pidx; p++) { + if (p->kidx == no->kidx) { + p->kidx = no_n->kidx; + break; + } + } + + continue; + } + + /* New table. Attach to runtime hash */ + TAILQ_REMOVE(&nh, no, nn_next); + + link_table(chain, (struct table_config *)no); + } + IPFW_WUNLOCK(chain); + + /* Perform rule rewrite */ + l = ci->krule->cmd_len; + cmd = ci->krule->cmd; + cmdlen = 0; + pidx = oib; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &uidx, &type) != 0) + continue; + update_table_opcode(cmd, pidx->kidx); + pidx++; + } + + IPFW_UH_WUNLOCK(chain); + } + + error = 0; + + /* + * Stage 4: free resources + */ +free: + TAILQ_FOREACH_SAFE(no, &nh, nn_next, no_tmp) + free_table_config(ni, tc); + + if (oib != ci->obuf) + free(oib, M_IPFW); + + return (error); +} + +/* + * Remove references from every table used in @rule. + */ +void +ipfw_unbind_table_rule(struct ip_fw_chain *chain, struct ip_fw *rule) +{ + int cmdlen, l; + ipfw_insn *cmd; + struct namedobj_instance *ni; + struct named_object *no; + uint32_t set; + uint16_t kidx; + uint8_t type; + + ni = CHAIN_TO_NI(chain); + + set = TABLE_SET(rule->set); + + l = rule->cmd_len; + cmd = rule->cmd; + cmdlen = 0; + for ( ; l > 0 ; l -= cmdlen, cmd += cmdlen) { + cmdlen = F_LEN(cmd); + + if (classify_table_opcode(cmd, &kidx, &type) != 0) + continue; + + no = ipfw_objhash_lookup_idx(ni, set, kidx); + + KASSERT(no != NULL, ("table id %d not found", kidx)); + KASSERT(no->type == type, ("wrong type %d (%d) for table id %d", + no->type, type, kidx)); + KASSERT(no->refcnt > 0, ("refcount for table %d is %d", + kidx, no->refcnt)); + + no->refcnt--; + } +} + + +/* + * Removes table bindings for every rule in rule chain @head. + */ +void +ipfw_unbind_table_list(struct ip_fw_chain *chain, struct ip_fw *head) +{ + struct ip_fw *rule; + + while ((rule = head) != NULL) { + head = head->x_next; + ipfw_unbind_table_rule(chain, rule); + } +} + + /* end of file */ --------------060502050508080706040508-- From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 13:01:27 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3681B8BD; Thu, 5 Jun 2014 13:01:27 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F12542983; Thu, 5 Jun 2014 13:01:26 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id z10so1039749pdj.26 for ; Thu, 05 Jun 2014 06:01:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=+S6/F6pGb4YucabfL/DKazC3jvqnIOGixBU+b2uMu9I=; b=qdLP0TGo52wNBFQGQQZ2SLs+n9Kn1CLyxKE1Gd9peJGZmQ7vKG6aETy9H2RkxeDVj9 fpfwy05LAk0Lc0sS5lBkl4q/okjwvzTGo4vdt2ZAgi7l5Glg+gxAvb7wU/d7cPbvKIhQ nW/xQG9w9NMZ9jmhXKiFb9bUz3a5B3nU7o7Hhbcednl1xqdtSOMRdcSkX9uYOLJs0ssO 2rscae0o/73JLG3WQLFvBg0aMRyE2tWvvc5VumyRW6V3T0LQ061AypZsagaLqXuethAv TGU1KBX488ZpdfAonWCoV1vmH4k74xEDtqQPmYixknUlBJRcH8Brze1aJYpV5gEq9ocd SN6g== X-Received: by 10.68.135.195 with SMTP id pu3mr27300507pbb.10.1401973286482; Thu, 05 Jun 2014 06:01:26 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id gg3sm22741670pbc.34.2014.06.05.06.01.21 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 05 Jun 2014 06:01:24 -0700 (PDT) From: "bycn82" To: "'Alexander V. Chernikov'" References: <5379FE3C.6060501@FreeBSD.org> <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> In-Reply-To: <539044E4.1020904@ipfw.ru> Subject: RE: [CFT]: ipfw named tables / different tabletypes Date: Thu, 5 Jun 2014 21:01:19 +0800 Message-ID: <000c01cf80be$41194370$c34bca50$@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_000D_01CF8101.4F3D6DD0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQHVthFPJT+R8Q+z/PfZZcF60gm+OADP7m1eASCyZGECMm0rcwKNzWvvAlvxzMYB26ii/AFrQU/SAWbxMPACXuZNZJrVQjzA Content-Language: en-us Cc: 'Luigi Rizzo' , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 13:01:27 -0000 This is a multipart message in MIME format. ------=_NextPart_000_000D_01CF8101.4F3D6DD0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Alex, Here is my patch, with this patch, the ipfw can support below commands,=20 root@FB10Head:~ # ipfw table 1 name saleteam root@FB10Head:~ # ipfw table 1 name show saleteam root@FB10Head:~ # ipfw table saleteam add 1.2.3.4 root@FB10Head:~ # ipfw table saleteam list 1.2.3.4/32 0 root@FB10Head:~ # ipfw table 1 list 1.2.3.4/32 0 root@FB10Head:~ # Currently still cleaning the table handling function, and did not add = the lock in the kernel functions when changing the `mapping chain`. Regards, bycn82 > -----Original Message----- > From: Alexander V. Chernikov [mailto:melifaro@ipfw.ru] > Sent: 05 June, 2014 18:22 > To: Luigi Rizzo > Cc: Luigi Rizzo; FreeBSD Net; Bill Yuan > Subject: Re: [CFT]: ipfw named tables / different tabletypes >=20 > On 01.06.2014 17:51, Alexander V. Chernikov wrote: > > On 22.05.2014 20:38, Luigi Rizzo wrote: > > > > Long story short, new version is ready. > > I've tried to minimize changes in this patch to ease review/commit. > > > > Changes: > > * Add namedobject set-aware api capable of searching/allocation > > objects by their name/idx. > > * Switch tables code to use string ids for configuration tasks. > > * Change locking model: most configuration changes are protected = with > > UH lock, runtime-visible are protected with both locks. > > * Reduce number of arguments passed to ipfw_table_add/del by using > > separate structure. > > * Add internal V_fw_tables_sets tunable (set to 0) to prepare for > > set-aware tables (requires opcodes/client support) > > * Implement typed table referencing (and tables are implicitly > > allocated with all state like radix ptrs on reference) > > * Add "destroy" ipfw(8) using new IP_FW_DELOBJ opcode > > > > Namedobj more detailed: > > * Blackbox api providing methods to add/del/search/enumerate objects > > * Statically-sized hashes for names/indexes > > * Per-set bitmask to indicate free indexes > > * Separate methods for index alloc/delete/resize > > > > > > Basically, there should not be any user-visible changes except the > > following: > > * reducing table_max is not supported > > * flush & add change table type won't work if table is referenced > > > > > > I haven't removed any numbering restrictions to protect the = following > > case: > > one (with old client) unintentionally references too many tables = (e.g. > > 1000-1128), > > tries to allocate table from "valid" range and fails. Old client = does > > not have any ability to destroy any table, so the only way to solve > > this is either module unload or reboot. > > > > I've uploaded the same patch to phabricator since it provides quite > > handy diffs: > > https://phabric.freebsd.org/D139 (no login required). > A bit cleaner version attached. > > > >> On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. Chernikov > wrote: > >>> On 22.05.2014 19:47, Luigi Rizzo wrote: > >>>> On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov > >>>> wrote: > >>>>> On 22.05.2014 00:48, Luigi Rizzo wrote: > >>>>>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. = Chernikov > >>>>>> wrote: > >>>> ... > >>>>>> we can solve this by using 'low' numbers for the numeric tables > >>>>>> (these were limited anyways) and allocate the fake entries in > >>>>>> another range. > >>>>> Currently we have u16 space available in base opcode. > >>>> yes but the standard range for tables is much more limited: > >>>> > >>>> net.inet.ip.fw.tables_max: 128 > >>>> > >>>> so one can just (say) use 32k for "old" tables and the rest for > >>>> tables with non numeric names. > >>>> Does not seem to be a problem in practice. > >>> Well, using upper 32k means that you set this default to 65k which > >>> consumes 256k of memory on 32-bit arch. > >>> Embedded people won't be very happy about this (and changing table > >>> numbers on resize would be a nightmare). > >> no no, this is an implementation detail but within the kernel you = can > >> just remap the 'old' and 'new' > >> table identifiers to a single contiguous range. > >> The only thing you need to do is that when you push identifiers up = to > >> userland, those with 'new' names will be mapped to the 32-64k = range. > >> > >> Example: > >> user first specifies tables > >> "18, goodguys, 530, badguys" in the same rule > >> /sbin/ipfw will generate these numbers: > >> 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} > >> The kernel will then do a lookup of those identifiers and > >> 18: internal index 1, name "18" > >> 32768: internal index 2, name "goodguys" > >> 530: internal index 3, name "530" > >> 32769: internal index 4, name "badguys" > >> > >> Then the next rule contains tables > >> 1, badguys, 18 > >> /sbin/ipfw generates > >> 1, 32768, 18 ; tlv {32768:badguys} // note different from = before > >> Kernel looks up the names and remaps > >> 1: internal index 5, name "1" > >> 32768: internal index 4, name "badguys" > >> 18: internal index 1, name "18" > >> > >> Finally when you do an 'ipfw show' the kernel will remap names > >> between 1 and 32768 to themselves, and other names to 32768+ (or = some > >> other large number, say 40k and above) so as they are found. So the > >> rules will be pushed up with > >> 18, 40000, 530, 40001 > >> 1, 40001, 18 > >> > >> we can discusso the other details privately > >> > >> cheers > >> luigi > >> > >> > >> 1. first, the > >>>>>> maybe i am missing some detail but it seems reasonably easy to > >>>>>> implement the atomic swap -- and the use case is when you want = to > >>>>>> move from one configuration to a new one: > >>>>>> ipfw table foo-new flush // clear initial content > >>>>>> ipfw table foo-new add ... > >>>>>> ipfw table swap foo-current foo-new // swap the content of > >>>>>> the table objects > >>>>>> > >>>>>> so you preserve the semantic of the name very easily. > >>>>> Yes. We can easily add atomic table swap that way. However, I'm > >>>>> talking about different use scenario: > >>>>> Atomically swap entire ruleset which has some tables depency: > >>>>> > >>>>> > >>>>> e.g. we have: > >>>>> > >>>>> " > >>>>> 100 allow ip from table(TABLE1) to me > >>>>> 200 allow ip from table(TABLE2) to (TABLE3) 80 > >>>>> > >>>>> table TABLE1 1.1.1.1/32 > >>>>> table TABLE1 1.0.0.0/16 > >>>>> > >>>>> table TABLE2 2.2.2.2/32 > >>>>> > >>>>> table TABLE3 3.3.3.3/32 > >>>>> " > >>>>> and we want to _atomically_ change this to > >>>>> > >>>>> " > >>>>> 100 allow ip from table(TABLE1) to me > >>>>> +200 allow ip from table(TABLE4) to any > >>>>> 300 allow ip from table(TABLE2) to (TABLE3) 80 > >>>>> > >>>>> table TABLE1 1.1.1.1/32 > >>>>> -table TABLE1 1.0.0.0/16 > >>>>> > >>>>> -table TABLE2 2.2.2.2/32 > >>>>> +table TABLE2 77.77.77.0/24 > >>>>> > >>>>> table TABLE3 3.3.3.3/32 > >>>>> > >>>>> +table TABLE4 4.4.4.4/32 > >>>>> " > >>>> aargh, that's too much -- because between changing > >>>> one table and all tables there are infinite intermediate > >>>> points that all make sense. > >>> It depends. As I said before, we're currently solving this problem > by > >>> adding new rules (to set X) referencing tables from different = range > >>> (2048 tables per ruleset) and than doing swap. > >>> (And not being able to use named tables to store real names after > >>> implementing them is a bit discouraging). > >>> > >>>> For those cases i think the way to go could be to > >>>> insert a 'disabled' new ruleset (however complex it is, > >>>> so it covers all possible cases), and then do the set swap, > >>>> or disable/enable. > >>> We can think of per-set arrays/namespaces of tables: > >>> > >>> so "ipfw add 100 set X allow ipfw from table(Y) to ..." will > reference > >>> table Y in set X and > >>> "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". > >>> > >>> This behavior can break some users setups so we can provide > >>> sysctl/tunable to turn this off or on. > >>> > >>>> cheers > >>>> luigi > >>>> > > ------=_NextPart_000_000D_01CF8101.4F3D6DD0 Content-Type: application/octet-stream; name="mapping.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mapping.patch" Index: sbin/ipfw/ipfw2.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sbin/ipfw/ipfw2.c (revision 267049)=0A= +++ sbin/ipfw/ipfw2.c (working copy)=0A= @@ -370,6 +370,17 @@=0A= { NULL, 0 } /* terminator */=0A= };=0A= =0A= +static struct _s_x table_actions[] =3D {=0A= + { "add", TOK_TBL_ADD },=0A= + { "delete", TOK_TBL_DEL },=0A= + { "del", TOK_TBL_DEL },=0A= + { "list", TOK_TBL_LIST },=0A= + { "flush", TOK_TBL_FLUSH },=0A= + { "name", TOK_TBL_NAME },=0A= + { "type", TOK_TBL_TYPE },=0A= + { NULL, 0 } /* terminator */=0A= +};=0A= +=0A= /*=0A= * Helper routine to print a possibly unaligned uint64_t on=0A= * various platform. If width > 0, print the value with=0A= @@ -4163,11 +4174,11 @@=0A= ipfw_table_handler(int ac, char *av[])=0A= {=0A= ipfw_table_xentry xent;=0A= - int do_add;=0A= int is_all;=0A= uint32_t a;=0A= uint32_t tables_max;=0A= =0A= +=0A= tables_max =3D ipfw_get_tables_max();=0A= =0A= memset(&xent, 0, sizeof(xent));=0A= @@ -4176,75 +4187,136 @@=0A= if (ac && isdigit(**av)) {=0A= xent.tbl =3D atoi(*av);=0A= is_all =3D 0;=0A= - ac--; av++;=0A= } else if (ac && _substrcmp(*av, "all") =3D=3D 0) {=0A= xent.tbl =3D 0;=0A= is_all =3D 1;=0A= - ac--; av++;=0A= - } else=0A= - errx(EX_USAGE, "table number or 'all' keyword required");=0A= - if (xent.tbl >=3D tables_max)=0A= - errx(EX_USAGE, "The table number exceeds the maximum allowed "=0A= - "value (%d)", tables_max - 1);=0A= + } else {=0A= + /* get table id via name and type*/=0A= + xent.tbl=3Dmapping_getid(IPFW_TABLE_NAME,*av);=0A= + is_all =3D 0;=0A= + }=0A= + ac--; av++;=0A= + if (xent.tbl >=3D tables_max){=0A= + errx(EX_USAGE, "The table number (%d) exceeds the maximum allowed "=0A= + "value (%d)",xent.tbl, tables_max - 1);=0A= + }=0A= NEED1("table needs command");=0A= - if (is_all && _substrcmp(*av, "list") !=3D 0=0A= - && _substrcmp(*av, "flush") !=3D 0)=0A= - errx(EX_USAGE, "table number required");=0A= -=0A= - if (_substrcmp(*av, "add") =3D=3D 0 ||=0A= - _substrcmp(*av, "delete") =3D=3D 0) {=0A= - do_add =3D **av =3D=3D 'a';=0A= - ac--; av++;=0A= - if (!ac)=0A= - errx(EX_USAGE, "address required");=0A= -=0A= - table_fill_xentry(*av, &xent);=0A= -=0A= - ac--; av++;=0A= - if (do_add && ac) {=0A= - unsigned int tval;=0A= - /* isdigit is a bit of a hack here.. */=0A= - if (strchr(*av, (int)'.') =3D=3D NULL && isdigit(**av)) {=0A= - xent.value =3D strtoul(*av, NULL, 0);=0A= - } else {=0A= - if (lookup_host(*av, (struct in_addr *)&tval) =3D=3D 0) {=0A= - /* The value must be stored in host order *=0A= - * so that the values < 65k can be distinguished */=0A= - xent.value =3D ntohl(tval);=0A= - } else {=0A= - errx(EX_NOHOST, "hostname ``%s'' unknown", *av);=0A= + int matched_index =3D match_token(table_actions, *av);=0A= + switch(matched_index) {=0A= + case TOK_TBL_ADD:=0A= + case TOK_TBL_DEL:=0A= + {=0A= + ac--; av++;=0A= + if (!ac){=0A= + errx(EX_USAGE, "address required");=0A= }=0A= + table_fill_xentry(*av, &xent);=0A= + ac--; av++;=0A= + if (matched_index=3D=3DTOK_TBL_ADD && ac) {=0A= + unsigned int tval;=0A= + /* isdigit is a bit of a hack here.. */=0A= + if (strchr(*av, (int)'.') =3D=3D NULL && isdigit(**av)) {=0A= + xent.value =3D strtoul(*av, NULL, 0);=0A= + } else {=0A= + if (lookup_host(*av, (struct in_addr *)&tval) =3D=3D 0) {=0A= + /* The value must be stored in host order *=0A= + * so that the values < 65k can be distinguished */=0A= + xent.value =3D ntohl(tval);=0A= + } else {=0A= + errx(EX_NOHOST, "hostname ``%s'' unknown", *av);=0A= + }=0A= + }=0A= + } else{=0A= + xent.value =3D 0;=0A= + }=0A= + if (do_setcmd3(matched_index=3D=3DTOK_TBL_ADD ? IP_FW_TABLE_XADD : = IP_FW_TABLE_XDEL,=0A= + &xent, xent.len) < 0) {=0A= + /* If running silent, don't bomb out on these errors. */=0A= + if (!(co.do_quiet && (errno =3D=3D (matched_index=3D=3DTOK_TBL_ADD = ? EEXIST : ESRCH))))=0A= + err(EX_OSERR, "setsockopt(IP_FW_TABLE_%s)",=0A= + matched_index=3D=3DTOK_TBL_ADD ? "XADD" : "XDEL");=0A= + /* In silent mode, react to a failed add by deleting */=0A= + if (matched_index=3D=3DTOK_TBL_ADD) {=0A= + do_setcmd3(IP_FW_TABLE_XDEL, &xent, xent.len);=0A= + if (do_setcmd3(IP_FW_TABLE_XADD, &xent, xent.len) < 0)=0A= + err(EX_OSERR,=0A= + "setsockopt(IP_FW_TABLE_XADD)");=0A= + }=0A= + }=0A= }=0A= - } else=0A= - xent.value =3D 0;=0A= - if (do_setcmd3(do_add ? IP_FW_TABLE_XADD : IP_FW_TABLE_XDEL,=0A= - &xent, xent.len) < 0) {=0A= - /* If running silent, don't bomb out on these errors. */=0A= - if (!(co.do_quiet && (errno =3D=3D (do_add ? EEXIST : ESRCH))))=0A= - err(EX_OSERR, "setsockopt(IP_FW_TABLE_%s)",=0A= - do_add ? "XADD" : "XDEL");=0A= - /* In silent mode, react to a failed add by deleting */=0A= - if (do_add) {=0A= - do_setcmd3(IP_FW_TABLE_XDEL, &xent, xent.len);=0A= - if (do_setcmd3(IP_FW_TABLE_XADD, &xent, xent.len) < 0)=0A= - err(EX_OSERR,=0A= - "setsockopt(IP_FW_TABLE_XADD)");=0A= + break;=0A= + case TOK_TBL_FLUSH:=0A= + {=0A= + a =3D is_all ? tables_max : (uint32_t)(xent.tbl + 1);=0A= + do {=0A= + if (do_cmd(IP_FW_TABLE_FLUSH, &xent.tbl,=0A= + sizeof(xent.tbl)) < 0)=0A= + err(EX_OSERR, "setsockopt(IP_FW_TABLE_FLUSH)");=0A= + } while (++xent.tbl < a);=0A= }=0A= - }=0A= - } else if (_substrcmp(*av, "flush") =3D=3D 0) {=0A= - a =3D is_all ? tables_max : (uint32_t)(xent.tbl + 1);=0A= - do {=0A= - if (do_cmd(IP_FW_TABLE_FLUSH, &xent.tbl,=0A= - sizeof(xent.tbl)) < 0)=0A= - err(EX_OSERR, "setsockopt(IP_FW_TABLE_FLUSH)");=0A= - } while (++xent.tbl < a);=0A= - } else if (_substrcmp(*av, "list") =3D=3D 0) {=0A= - a =3D is_all ? tables_max : (uint32_t)(xent.tbl + 1);=0A= - do {=0A= - table_list(xent.tbl, is_all);=0A= - } while (++xent.tbl < a);=0A= - } else=0A= - errx(EX_USAGE, "invalid table command %s", *av);=0A= + break;=0A= + case TOK_TBL_LIST:=0A= + {=0A= + a =3D is_all ? tables_max : (uint32_t)(xent.tbl + 1);=0A= + do {=0A= + table_list(xent.tbl, is_all);=0A= + } while (++xent.tbl < a);=0A= + }=0A= + break;=0A= + case TOK_TBL_NAME:=0A= + case TOK_TBL_TYPE:=0A= + {=0A= + if (ac=3D=3D1){=0A= + errx(EX_USAGE, "more option required");=0A= + }=0A= + =0A= + int type;=0A= + if(matched_index=3D=3DTOK_TBL_NAME){=0A= + type=3DIPFW_TABLE_NAME;=0A= + }else if(matched_index=3D=3DTOK_TBL_TYPE){=0A= + type=3DIPFW_TABLE_TYPE;=0A= + }=0A= + av++;=0A= + if (_substrcmp(*av, "delete") =3D=3D 0){=0A= + ipfw_mapping mapping_element;=0A= + memset(&mapping_element, 0, sizeof(mapping_element));=0A= + mapping_element.id=3Dxent.tbl;=0A= + mapping_element.type=3Dtype;=0A= + if (do_setcmd3(IP_FW_MAPPING_DEL,&mapping_element,=0A= + sizeof(mapping_element)) < 0){=0A= + err(EX_OSERR,"setsockopt(IP_FW_MAPPING_SET)");=0A= + }=0A= + }else if (_substrcmp(*av, "show") =3D=3D 0){=0A= + ip_fw3_opheader *op3;=0A= + socklen_t len;=0A= + len =3D sizeof(ip_fw3_opheader) + sizeof(ipfw_mapping);=0A= + op3 =3D alloca(len);=0A= + memset(op3, 0, sizeof(ip_fw3_opheader));=0A= + ipfw_mapping *mapping=3D(ipfw_mapping *)(op3+1);=0A= + mapping->id=3Dxent.tbl;=0A= + mapping->type=3Dtype;=0A= + op3->opcode =3D IP_FW_MAPPING_GET;=0A= + if (do_cmd(IP_FW3, op3, (uintptr_t)&len) < 0){=0A= + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETSIZE)");=0A= + }=0A= + printf("%s\n",mapping->label);=0A= + }else{=0A= + ipfw_mapping mapping_element;=0A= + memset(&mapping_element, 0, sizeof(mapping_element));=0A= + mapping_element.id=3Dxent.tbl;=0A= + mapping_element.type=3Dtype;=0A= + strcpy(mapping_element.label,*av);=0A= + if (do_setcmd3(IP_FW_MAPPING_SET,&mapping_element,=0A= + sizeof(mapping_element)) < 0){=0A= + err(EX_OSERR,"setsockopt(IP_FW_MAPPING_SET)");=0A= + }=0A= + }=0A= +=0A= + }=0A= + break;=0A= + default:=0A= + errx(EX_USAGE, "invalid table command %s", *av);=0A= + }=0A= }=0A= =0A= static void=0A= @@ -4423,3 +4495,20 @@=0A= =0A= free(tbl);=0A= }=0A= +int mapping_getid(int type,char *label){=0A= + ip_fw3_opheader *op3;=0A= + socklen_t len;=0A= + len =3D sizeof(ip_fw3_opheader) + sizeof(ipfw_mapping);=0A= + op3 =3D alloca(len);=0A= + memset(op3, 0, sizeof(ip_fw3_opheader));=0A= + ipfw_mapping *mapping=3D(ipfw_mapping *)(op3+1);=0A= +=0A= + mapping->type=3Dtype;=0A= + mapping->id=3D0;=0A= + strcpy(mapping->label,label);=0A= + op3->opcode =3D IP_FW_MAPPING_GET;=0A= + if (do_cmd(IP_FW3, op3, (uintptr_t)&len) < 0){=0A= + err(EX_OSERR, "getsockopt(IP_FW_TABLE_XGETSIZE)");=0A= + }=0A= + return mapping->id;=0A= +}=0A= Index: sbin/ipfw/ipfw2.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sbin/ipfw/ipfw2.h (revision 267049)=0A= +++ sbin/ipfw/ipfw2.h (working copy)=0A= @@ -205,6 +205,13 @@=0A= TOK_LOOKUP,=0A= TOK_SOCKARG,=0A= TOK_SETDSCP,=0A= + /* table tokens */=0A= + TOK_TBL_ADD,=0A= + TOK_TBL_DEL,=0A= + TOK_TBL_LIST,=0A= + TOK_TBL_FLUSH,=0A= + TOK_TBL_NAME,=0A= + TOK_TBL_TYPE,=0A= };=0A= /*=0A= * the following macro returns an error message if we run out of=0A= @@ -297,3 +304,9 @@=0A= void fill_unreach6_code(u_short *codep, char *str);=0A= void fill_icmp6types(struct _ipfw_insn_icmp6 *cmd, char *av, int cblen);=0A= int fill_ext6hdr(struct _ipfw_insn *cmd, char *av);=0A= +=0A= +/* table mapping */=0A= +int mapping_getid(int type,char *label);=0A= +void mapping_getlabel(int id,int type,char *label);=0A= +void mapping_add(int id,int type,char *label);=0A= +void mapping_delete(int id,int type);=0A= Index: sys/modules/ipfw/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/ipfw/Makefile (revision 267049)=0A= +++ sys/modules/ipfw/Makefile (working copy)=0A= @@ -17,7 +17,7 @@=0A= #CFLAGS+=3D -DIPFIREWALL_VERBOSE_LIMIT=3D100=0A= #=0A= #If you want it to pass all packets by default=0A= -#CFLAGS+=3D -DIPFIREWALL_DEFAULT_TO_ACCEPT=0A= +CFLAGS+=3D -DIPFIREWALL_DEFAULT_TO_ACCEPT=0A= #=0A= =0A= .if !defined(KERNBUILDDIR)=0A= Index: sys/netinet/ip_fw.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netinet/ip_fw.h (revision 267049)=0A= +++ sys/netinet/ip_fw.h (working copy)=0A= @@ -75,6 +75,10 @@=0A= #define IP_FW_TABLE_XGETSIZE 88 /* get table size */=0A= #define IP_FW_TABLE_XLIST 89 /* list table contents */=0A= =0A= +#define IP_FW_MAPPING_GET 90=0A= +#define IP_FW_MAPPING_SET 91=0A= +#define IP_FW_MAPPING_DEL 92=0A= +=0A= /*=0A= * The kernel representation of ipfw rules is made of a list of=0A= * 'instructions' (for all practical purposes equivalent to BPF=0A= @@ -602,6 +606,11 @@=0A= #define IPFW_TABLE_INTERFACE 2 /* Table for holding interface names */=0A= #define IPFW_TABLE_MAXTYPE 2 /* Maximum valid number */=0A= =0A= +#define IPFW_TABLE_NAME 1 /* type in mapping */=0A= +#define IPFW_TABLE_TYPE 2 /* type in mapping */=0A= +#define IPFW_PIPE_NAME 3 /* type in mapping */=0A= +#define IPFW_QUEUE_NAME 4 /* type in mapping */=0A= +=0A= typedef struct _ipfw_table_entry {=0A= in_addr_t addr; /* network address */=0A= u_int32_t value; /* value */=0A= @@ -624,6 +633,18 @@=0A= } ipfw_table_xentry;=0A= #define IPFW_TCF_INET 0x01 /* CIDR flags: IPv4 record */=0A= =0A= +typedef struct _ipfw_mapping{=0A= + uint16_t id;=0A= + uint8_t type;=0A= + char label[20];=0A= + struct _ipfw_mapping *next;=0A= +} ipfw_mapping;=0A= +=0A= +typedef struct _ipfw_mapping_query{=0A= + ip_fw3_opheader opheader;=0A= + ipfw_mapping mapping;=0A= +} ipfw_mapping_query;=0A= +=0A= typedef struct _ipfw_table {=0A= u_int32_t size; /* size of entries in bytes */=0A= u_int32_t cnt; /* # of entries */=0A= Index: sys/netpfil/ipfw/ip_fw_private.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netpfil/ipfw/ip_fw_private.h (revision 267049)=0A= +++ sys/netpfil/ipfw/ip_fw_private.h (working copy)=0A= @@ -233,6 +233,7 @@=0A= spinlock_t uh_lock;=0A= #else=0A= struct rwlock uh_lock; /* lock for upper half */=0A= + struct _ipfw_mapping *mapping;=0A= #endif=0A= };=0A= =0A= Index: sys/netpfil/ipfw/ip_fw_sockopt.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 267049)=0A= +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy)=0A= @@ -1280,7 +1280,116 @@=0A= free(tbl, M_TEMP);=0A= }=0A= break;=0A= -=0A= + /* MAPPING */=0A= + case IP_FW_MAPPING_SET:=0A= + {=0A= + ipfw_mapping *ele=3D (ipfw_mapping *)(op3 + 1);=0A= + ipfw_mapping *current_ele,*new_ele;=0A= + int found=3D0,current_need_checked=3D1;=0A= + if ((size =3D valsize) < sizeof(ipfw_mapping)) {=0A= + error =3D EINVAL;=0A= + break;=0A= + }=0A= + new_ele=3D malloc(size, M_TEMP, M_ZERO | M_WAITOK);=0A= + memcpy(new_ele, ele, sizeof(ipfw_mapping));=0A= + current_ele=3Dchain->mapping;=0A= + if(current_ele!=3DNULL){=0A= + while(current_need_checked=3D=3D1)=0A= + {=0A= + if(ele->id=3D=3Dcurrent_ele->id && ele->type=3D=3D = current_ele->type){=0A= + found=3D1;=0A= + break;=0A= + }=0A= + current_need_checked=3D0; =0A= + /* if current is the last element, then just quit the loop.*/=0A= + if(current_ele->next!=3DNULL){=0A= + current_ele=3Dcurrent_ele->next;=0A= + current_need_checked=3D1;=0A= + }=0A= + }=0A= + /* if found, then update the label, otherwise, append to the last = element, */=0A= + if(found){=0A= + strcpy(current_ele->label,ele->label);=0A= + }else{=0A= + current_ele->next=3Dnew_ele;=0A= + }=0A= + }else{=0A= + /* append the new_element into the chain */=0A= + chain->mapping=3Dnew_ele;=0A= + }=0A= + }=0A= + break;=0A= + case IP_FW_MAPPING_GET:=0A= + {=0A= + ipfw_mapping *ele=3D (ipfw_mapping *)(op3 + 1);=0A= + ipfw_mapping *current_ele;=0A= + int found=3D0,current_need_checked=3D1;=0A= + current_ele=3Dchain->mapping;=0A= + if(current_ele!=3DNULL){=0A= + while(current_need_checked=3D=3D1)=0A= + {=0A= + if(ele->id=3D=3D0){=0A= + if(ele->type=3D=3D current_ele->type && = strcmp(ele->label,current_ele->label)=3D=3D0){=0A= + found=3D1;=0A= + break;=0A= + }=0A= + }else{=0A= + if(ele->id=3D=3Dcurrent_ele->id && ele->type=3D=3D = current_ele->type){=0A= + found=3D1;=0A= + break;=0A= + }=0A= + }=0A= + current_need_checked=3D0; =0A= + if(current_ele->next!=3DNULL){=0A= + current_ele=3Dcurrent_ele->next;=0A= + current_need_checked=3D1; =0A= + }=0A= + }=0A= + /* if found, copy out*/=0A= + if(found){=0A= + if(ele->id=3D=3D0){=0A= + ele->id=3Dcurrent_ele->id;=0A= + }else{=0A= + strcpy(ele->label,current_ele->label);=0A= + }=0A= + error =3D sooptcopyout(sopt, op3, sizeof(ipfw_mapping_query));=0A= + }=0A= + }else{=0A= + error =3D EINVAL;=0A= + }=0A= + }=0A= + break;=0A= + case IP_FW_MAPPING_DEL:=0A= + {=0A= + ipfw_mapping *ele=3D (ipfw_mapping *)(op3 + 1);=0A= + ipfw_mapping *previous_ele,*current_ele,*next_ele;=0A= + current_ele=3Dchain->mapping;=0A= + if(current_ele!=3DNULL){=0A= + /* if delete the first element */=0A= + if(ele->id=3D=3Dcurrent_ele->id && ele->type=3D=3D = current_ele->type){=0A= + next_ele=3Dcurrent_ele->next;=0A= + if(next_ele!=3DNULL){=0A= + chain->mapping=3Dnext_ele;=0A= + }=0A= + free(current_ele,M_TEMP);=0A= + }else{=0A= + next_ele=3Dcurrent_ele->next;=0A= + while(next_ele!=3DNULL){=0A= + previous_ele=3Dcurrent_ele;=0A= + current_ele=3Dcurrent_ele->next;=0A= + next_ele=3Dcurrent_ele->next;=0A= + if(ele->id=3D=3Dcurrent_ele->id && ele->type=3D=3D = current_ele->type){=0A= + free(current_ele,M_TEMP);=0A= + break;=0A= + }=0A= + previous_ele->next=3Dnext_ele;=0A= + }=0A= + }=0A= + }else{=0A= + error =3D EINVAL;=0A= + }=0A= + }=0A= + break;=0A= /*--- NAT operations are protected by the IPFW_LOCK ---*/=0A= case IP_FW_NAT_CFG:=0A= if (IPFW_NAT_LOADED)=0A= ------=_NextPart_000_000D_01CF8101.4F3D6DD0-- From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 13:38:18 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20F2BAE8; Thu, 5 Jun 2014 13:38:18 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6D9802E88; Thu, 5 Jun 2014 13:38:16 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 1FBB97300A; Thu, 5 Jun 2014 15:42:56 +0200 (CEST) Date: Thu, 5 Jun 2014 15:42:56 +0200 From: Luigi Rizzo To: bycn82 Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140605134256.GA81234@onelab2.iet.unipi.it> References: <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000c01cf80be$41194370$c34bca50$@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: 'Luigi Rizzo' , "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 13:38:18 -0000 On Thu, Jun 05, 2014 at 09:01:19PM +0800, bycn82 wrote: > Hi Alex, > > Here is my patch, with this patch, the ipfw can support below commands, > > root@FB10Head:~ # ipfw table 1 name saleteam > root@FB10Head:~ # ipfw table 1 name show > saleteam Bill, as it was discussed previously, exposing the numeric identifier for the table is useless and potentially dangerous so the two commands above must not exist. In the new firewall the only names exposed externally are strings, and a table name "1" will be interpreted as a string, not a number. Only for transitional purposes (new kernel with old firewall), the kernel will accept commands with the old numeric identifiers and map them to strings, in and out. cheers luigi > root@FB10Head:~ # ipfw table saleteam add 1.2.3.4 > root@FB10Head:~ # ipfw table saleteam list > 1.2.3.4/32 0 > root@FB10Head:~ # ipfw table 1 list > 1.2.3.4/32 0 > root@FB10Head:~ # > > > Currently still cleaning the table handling function, and did not add the lock in the kernel functions when changing the `mapping chain`. > > Regards, > bycn82 > > > > > -----Original Message----- > > From: Alexander V. Chernikov [mailto:melifaro@ipfw.ru] > > Sent: 05 June, 2014 18:22 > > To: Luigi Rizzo > > Cc: Luigi Rizzo; FreeBSD Net; Bill Yuan > > Subject: Re: [CFT]: ipfw named tables / different tabletypes > > > > On 01.06.2014 17:51, Alexander V. Chernikov wrote: > > > On 22.05.2014 20:38, Luigi Rizzo wrote: > > > > > > Long story short, new version is ready. > > > I've tried to minimize changes in this patch to ease review/commit. > > > > > > Changes: > > > * Add namedobject set-aware api capable of searching/allocation > > > objects by their name/idx. > > > * Switch tables code to use string ids for configuration tasks. > > > * Change locking model: most configuration changes are protected with > > > UH lock, runtime-visible are protected with both locks. > > > * Reduce number of arguments passed to ipfw_table_add/del by using > > > separate structure. > > > * Add internal V_fw_tables_sets tunable (set to 0) to prepare for > > > set-aware tables (requires opcodes/client support) > > > * Implement typed table referencing (and tables are implicitly > > > allocated with all state like radix ptrs on reference) > > > * Add "destroy" ipfw(8) using new IP_FW_DELOBJ opcode > > > > > > Namedobj more detailed: > > > * Blackbox api providing methods to add/del/search/enumerate objects > > > * Statically-sized hashes for names/indexes > > > * Per-set bitmask to indicate free indexes > > > * Separate methods for index alloc/delete/resize > > > > > > > > > Basically, there should not be any user-visible changes except the > > > following: > > > * reducing table_max is not supported > > > * flush & add change table type won't work if table is referenced > > > > > > > > > I haven't removed any numbering restrictions to protect the following > > > case: > > > one (with old client) unintentionally references too many tables (e.g. > > > 1000-1128), > > > tries to allocate table from "valid" range and fails. Old client does > > > not have any ability to destroy any table, so the only way to solve > > > this is either module unload or reboot. > > > > > > I've uploaded the same patch to phabricator since it provides quite > > > handy diffs: > > > https://phabric.freebsd.org/D139 (no login required). > > A bit cleaner version attached. > > > > > >> On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. Chernikov > > wrote: > > >>> On 22.05.2014 19:47, Luigi Rizzo wrote: > > >>>> On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. Chernikov > > >>>> wrote: > > >>>>> On 22.05.2014 00:48, Luigi Rizzo wrote: > > >>>>>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. Chernikov > > >>>>>> wrote: > > >>>> ... > > >>>>>> we can solve this by using 'low' numbers for the numeric tables > > >>>>>> (these were limited anyways) and allocate the fake entries in > > >>>>>> another range. > > >>>>> Currently we have u16 space available in base opcode. > > >>>> yes but the standard range for tables is much more limited: > > >>>> > > >>>> net.inet.ip.fw.tables_max: 128 > > >>>> > > >>>> so one can just (say) use 32k for "old" tables and the rest for > > >>>> tables with non numeric names. > > >>>> Does not seem to be a problem in practice. > > >>> Well, using upper 32k means that you set this default to 65k which > > >>> consumes 256k of memory on 32-bit arch. > > >>> Embedded people won't be very happy about this (and changing table > > >>> numbers on resize would be a nightmare). > > >> no no, this is an implementation detail but within the kernel you can > > >> just remap the 'old' and 'new' > > >> table identifiers to a single contiguous range. > > >> The only thing you need to do is that when you push identifiers up to > > >> userland, those with 'new' names will be mapped to the 32-64k range. > > >> > > >> Example: > > >> user first specifies tables > > >> "18, goodguys, 530, badguys" in the same rule > > >> /sbin/ipfw will generate these numbers: > > >> 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} > > >> The kernel will then do a lookup of those identifiers and > > >> 18: internal index 1, name "18" > > >> 32768: internal index 2, name "goodguys" > > >> 530: internal index 3, name "530" > > >> 32769: internal index 4, name "badguys" > > >> > > >> Then the next rule contains tables > > >> 1, badguys, 18 > > >> /sbin/ipfw generates > > >> 1, 32768, 18 ; tlv {32768:badguys} // note different from before > > >> Kernel looks up the names and remaps > > >> 1: internal index 5, name "1" > > >> 32768: internal index 4, name "badguys" > > >> 18: internal index 1, name "18" > > >> > > >> Finally when you do an 'ipfw show' the kernel will remap names > > >> between 1 and 32768 to themselves, and other names to 32768+ (or some > > >> other large number, say 40k and above) so as they are found. So the > > >> rules will be pushed up with > > >> 18, 40000, 530, 40001 > > >> 1, 40001, 18 > > >> > > >> we can discusso the other details privately > > >> > > >> cheers > > >> luigi > > >> > > >> > > >> 1. first, the > > >>>>>> maybe i am missing some detail but it seems reasonably easy to > > >>>>>> implement the atomic swap -- and the use case is when you want to > > >>>>>> move from one configuration to a new one: > > >>>>>> ipfw table foo-new flush // clear initial content > > >>>>>> ipfw table foo-new add ... > > >>>>>> ipfw table swap foo-current foo-new // swap the content of > > >>>>>> the table objects > > >>>>>> > > >>>>>> so you preserve the semantic of the name very easily. > > >>>>> Yes. We can easily add atomic table swap that way. However, I'm > > >>>>> talking about different use scenario: > > >>>>> Atomically swap entire ruleset which has some tables depency: > > >>>>> > > >>>>> > > >>>>> e.g. we have: > > >>>>> > > >>>>> " > > >>>>> 100 allow ip from table(TABLE1) to me > > >>>>> 200 allow ip from table(TABLE2) to (TABLE3) 80 > > >>>>> > > >>>>> table TABLE1 1.1.1.1/32 > > >>>>> table TABLE1 1.0.0.0/16 > > >>>>> > > >>>>> table TABLE2 2.2.2.2/32 > > >>>>> > > >>>>> table TABLE3 3.3.3.3/32 > > >>>>> " > > >>>>> and we want to _atomically_ change this to > > >>>>> > > >>>>> " > > >>>>> 100 allow ip from table(TABLE1) to me > > >>>>> +200 allow ip from table(TABLE4) to any > > >>>>> 300 allow ip from table(TABLE2) to (TABLE3) 80 > > >>>>> > > >>>>> table TABLE1 1.1.1.1/32 > > >>>>> -table TABLE1 1.0.0.0/16 > > >>>>> > > >>>>> -table TABLE2 2.2.2.2/32 > > >>>>> +table TABLE2 77.77.77.0/24 > > >>>>> > > >>>>> table TABLE3 3.3.3.3/32 > > >>>>> > > >>>>> +table TABLE4 4.4.4.4/32 > > >>>>> " > > >>>> aargh, that's too much -- because between changing > > >>>> one table and all tables there are infinite intermediate > > >>>> points that all make sense. > > >>> It depends. As I said before, we're currently solving this problem > > by > > >>> adding new rules (to set X) referencing tables from different range > > >>> (2048 tables per ruleset) and than doing swap. > > >>> (And not being able to use named tables to store real names after > > >>> implementing them is a bit discouraging). > > >>> > > >>>> For those cases i think the way to go could be to > > >>>> insert a 'disabled' new ruleset (however complex it is, > > >>>> so it covers all possible cases), and then do the set swap, > > >>>> or disable/enable. > > >>> We can think of per-set arrays/namespaces of tables: > > >>> > > >>> so "ipfw add 100 set X allow ipfw from table(Y) to ..." will > > reference > > >>> table Y in set X and > > >>> "ipfw table ABC list" can differ from "ipfw table ABC set 5 list". > > >>> > > >>> This behavior can break some users setups so we can provide > > >>> sysctl/tunable to turn this off or on. > > >>> > > >>>> cheers > > >>>> luigi > > >>>> > > > > From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 14:49:38 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16C5E44C for ; Thu, 5 Jun 2014 14:49:38 +0000 (UTC) Received: from mail-pb0-x232.google.com (mail-pb0-x232.google.com [IPv6:2607:f8b0:400e:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D812E25D6 for ; Thu, 5 Jun 2014 14:49:37 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id ma3so1210841pbc.23 for ; Thu, 05 Jun 2014 07:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=eOC2uWrvrcCRD/scZ5k7EIbk1xCzTfxXoM5wIuePriU=; b=P9NWuk/Sud362jGMQb2X58FUBKUo0dtpd6eNwPETdmhRdvwCojavnznIoeNe8tsfEU aNFcYAk8bvXQd+I3W534arxpjTwqra57xC/dVvM6LNo9A6U4NKaQPIsfy45VMbZmvqKU f8KoJq953CNvVwwz9ica8WNNhSKzAggHPKH8GEA91tmH5TCrTO6++KjGXNikevTbTQ7o K3mPrpY4LF80n79MxcvaVvSM/zn+R6DvUmbzqr6iSYNkrK+xMO202p5WCGOtTom2dBaJ 53J0E2+lESFg0D/mfoEGgF8WmOu9mkLXWC0nvcIiCM8fDEZsRNzi+R8azGfx2e23ZiQ6 fVSw== X-Received: by 10.68.200.10 with SMTP id jo10mr77682553pbc.143.1401979775958; Thu, 05 Jun 2014 07:49:35 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id ib5sm23755075pbb.55.2014.06.05.07.49.31 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 05 Jun 2014 07:49:34 -0700 (PDT) From: "bycn82" To: "'Luigi Rizzo'" , References: <20140521111002.GB62462@onelab2.iet.unipi.it> <537CEC12.8050404@FreeBSD.org> <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> In-Reply-To: <20140605134256.GA81234@onelab2.iet.unipi.it> Subject: RE: [CFT]: ipfw named tables / different tabletypes Date: Thu, 5 Jun 2014 22:49:27 +0800 Message-ID: <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQDP7m1eF4Hlla5jUUEWXMQiUH1mRAEgsmRhAjJtK3MCjc1r7wJb8czGAduoovwBa0FP0gFm8TDwAl7mTWQC+qrnHAHOZXswnMEZ9kA= Content-Language: en-us Cc: "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 14:49:38 -0000 Hi Luigi, Yes, use string instead of integer for the ID of table, but the same = method cannot apply to the feature `set type of table`. And in the = kernel, compare string will cause more than compare an integer. In my = opinion, actually they are just alias name for the object. Users already = accept that every object has an integer ID. Hi Alex, Why not clean the ipfw_table_handler() function using the switch/case? = Like in my patch, It can be easier to understand the code. Best Regards, bycn82 > -----Original Message----- > From: Luigi Rizzo [mailto:rizzo@iet.unipi.it] > Sent: 05 June, 2014 21:43 > To: bycn82 > Cc: 'Alexander V. Chernikov'; 'Luigi Rizzo'; 'FreeBSD Net' > Subject: Re: [CFT]: ipfw named tables / different tabletypes >=20 > On Thu, Jun 05, 2014 at 09:01:19PM +0800, bycn82 wrote: > > Hi Alex, > > > > Here is my patch, with this patch, the ipfw can support below > > commands, > > > > root@FB10Head:~ # ipfw table 1 name saleteam root@FB10Head:~ # ipfw > > table 1 name show saleteam >=20 > Bill, > as it was discussed previously, exposing the numeric identifier for = the > table is useless and potentially dangerous so the two commands above > must not exist. >=20 > In the new firewall the only names exposed externally are strings, and = a > table name "1" will be interpreted as a string, not a number. >=20 > Only for transitional purposes (new kernel with old firewall), the > kernel will accept commands with the old numeric identifiers and map > them to strings, in and out. >=20 > cheers > luigi >=20 > > root@FB10Head:~ # ipfw table saleteam add 1.2.3.4 root@FB10Head:~ # > > ipfw table saleteam list > > 1.2.3.4/32 0 > > root@FB10Head:~ # ipfw table 1 list > > 1.2.3.4/32 0 > > root@FB10Head:~ # > > > > > > Currently still cleaning the table handling function, and did not = add > the lock in the kernel functions when changing the `mapping chain`. > > > > Regards, > > bycn82 > > > > > > > > > -----Original Message----- > > > From: Alexander V. Chernikov [mailto:melifaro@ipfw.ru] > > > Sent: 05 June, 2014 18:22 > > > To: Luigi Rizzo > > > Cc: Luigi Rizzo; FreeBSD Net; Bill Yuan > > > Subject: Re: [CFT]: ipfw named tables / different tabletypes > > > > > > On 01.06.2014 17:51, Alexander V. Chernikov wrote: > > > > On 22.05.2014 20:38, Luigi Rizzo wrote: > > > > > > > > Long story short, new version is ready. > > > > I've tried to minimize changes in this patch to ease = review/commit. > > > > > > > > Changes: > > > > * Add namedobject set-aware api capable of searching/allocation > > > > objects by their name/idx. > > > > * Switch tables code to use string ids for configuration tasks. > > > > * Change locking model: most configuration changes are protected > > > > with UH lock, runtime-visible are protected with both locks. > > > > * Reduce number of arguments passed to ipfw_table_add/del by = using > > > > separate structure. > > > > * Add internal V_fw_tables_sets tunable (set to 0) to prepare = for > > > > set-aware tables (requires opcodes/client support) > > > > * Implement typed table referencing (and tables are implicitly > > > > allocated with all state like radix ptrs on reference) > > > > * Add "destroy" ipfw(8) using new IP_FW_DELOBJ opcode > > > > > > > > Namedobj more detailed: > > > > * Blackbox api providing methods to add/del/search/enumerate > > > > objects > > > > * Statically-sized hashes for names/indexes > > > > * Per-set bitmask to indicate free indexes > > > > * Separate methods for index alloc/delete/resize > > > > > > > > > > > > Basically, there should not be any user-visible changes except = the > > > > following: > > > > * reducing table_max is not supported > > > > * flush & add change table type won't work if table is = referenced > > > > > > > > > > > > I haven't removed any numbering restrictions to protect the > > > > following > > > > case: > > > > one (with old client) unintentionally references too many tables > (e.g. > > > > 1000-1128), > > > > tries to allocate table from "valid" range and fails. Old client > > > > does not have any ability to destroy any table, so the only way = to > > > > solve this is either module unload or reboot. > > > > > > > > I've uploaded the same patch to phabricator since it provides > > > > quite handy diffs: > > > > https://phabric.freebsd.org/D139 (no login required). > > > A bit cleaner version attached. > > > > > > > >> On Thu, May 22, 2014 at 08:09:55PM +0400, Alexander V. = Chernikov > > > wrote: > > > >>> On 22.05.2014 19:47, Luigi Rizzo wrote: > > > >>>> On Thu, May 22, 2014 at 06:56:41PM +0400, Alexander V. > > > >>>> Chernikov > > > >>>> wrote: > > > >>>>> On 22.05.2014 00:48, Luigi Rizzo wrote: > > > >>>>>> On Wed, May 21, 2014 at 10:10:26PM +0400, Alexander V. > > > >>>>>> Chernikov > > > >>>>>> wrote: > > > >>>> ... > > > >>>>>> we can solve this by using 'low' numbers for the numeric > > > >>>>>> tables (these were limited anyways) and allocate the fake > > > >>>>>> entries in another range. > > > >>>>> Currently we have u16 space available in base opcode. > > > >>>> yes but the standard range for tables is much more limited: > > > >>>> > > > >>>> net.inet.ip.fw.tables_max: 128 > > > >>>> > > > >>>> so one can just (say) use 32k for "old" tables and the rest = for > > > >>>> tables with non numeric names. > > > >>>> Does not seem to be a problem in practice. > > > >>> Well, using upper 32k means that you set this default to 65k > > > >>> which consumes 256k of memory on 32-bit arch. > > > >>> Embedded people won't be very happy about this (and changing > > > >>> table numbers on resize would be a nightmare). > > > >> no no, this is an implementation detail but within the kernel = you > > > >> can just remap the 'old' and 'new' > > > >> table identifiers to a single contiguous range. > > > >> The only thing you need to do is that when you push identifiers > > > >> up to userland, those with 'new' names will be mapped to the = 32- > 64k range. > > > >> > > > >> Example: > > > >> user first specifies tables > > > >> "18, goodguys, 530, badguys" in the same rule > > > >> /sbin/ipfw will generate these numbers: > > > >> 18, 32768, 530, 32769 ; tlv {32768:goodguys, 32769:badguys} > > > >> The kernel will then do a lookup of those identifiers and > > > >> 18: internal index 1, name "18" > > > >> 32768: internal index 2, name "goodguys" > > > >> 530: internal index 3, name "530" > > > >> 32769: internal index 4, name "badguys" > > > >> > > > >> Then the next rule contains tables > > > >> 1, badguys, 18 > > > >> /sbin/ipfw generates > > > >> 1, 32768, 18 ; tlv {32768:badguys} // note different from > before > > > >> Kernel looks up the names and remaps > > > >> 1: internal index 5, name "1" > > > >> 32768: internal index 4, name "badguys" > > > >> 18: internal index 1, name "18" > > > >> > > > >> Finally when you do an 'ipfw show' the kernel will remap names > > > >> between 1 and 32768 to themselves, and other names to 32768+ = (or > > > >> some other large number, say 40k and above) so as they are = found. > > > >> So the rules will be pushed up with > > > >> 18, 40000, 530, 40001 > > > >> 1, 40001, 18 > > > >> > > > >> we can discusso the other details privately > > > >> > > > >> cheers > > > >> luigi > > > >> > > > >> > > > >> 1. first, the > > > >>>>>> maybe i am missing some detail but it seems reasonably easy > > > >>>>>> to implement the atomic swap -- and the use case is when = you > > > >>>>>> want to move from one configuration to a new one: > > > >>>>>> ipfw table foo-new flush // clear initial content > > > >>>>>> ipfw table foo-new add ... > > > >>>>>> ipfw table swap foo-current foo-new // swap the content > > > >>>>>> of the table objects > > > >>>>>> > > > >>>>>> so you preserve the semantic of the name very easily. > > > >>>>> Yes. We can easily add atomic table swap that way. However, > > > >>>>> I'm talking about different use scenario: > > > >>>>> Atomically swap entire ruleset which has some tables = depency: > > > >>>>> > > > >>>>> > > > >>>>> e.g. we have: > > > >>>>> > > > >>>>> " > > > >>>>> 100 allow ip from table(TABLE1) to me > > > >>>>> 200 allow ip from table(TABLE2) to (TABLE3) 80 > > > >>>>> > > > >>>>> table TABLE1 1.1.1.1/32 > > > >>>>> table TABLE1 1.0.0.0/16 > > > >>>>> > > > >>>>> table TABLE2 2.2.2.2/32 > > > >>>>> > > > >>>>> table TABLE3 3.3.3.3/32 > > > >>>>> " > > > >>>>> and we want to _atomically_ change this to > > > >>>>> > > > >>>>> " > > > >>>>> 100 allow ip from table(TABLE1) to me > > > >>>>> +200 allow ip from table(TABLE4) to any > > > >>>>> 300 allow ip from table(TABLE2) to (TABLE3) 80 > > > >>>>> > > > >>>>> table TABLE1 1.1.1.1/32 > > > >>>>> -table TABLE1 1.0.0.0/16 > > > >>>>> > > > >>>>> -table TABLE2 2.2.2.2/32 > > > >>>>> +table TABLE2 77.77.77.0/24 > > > >>>>> > > > >>>>> table TABLE3 3.3.3.3/32 > > > >>>>> > > > >>>>> +table TABLE4 4.4.4.4/32 > > > >>>>> " > > > >>>> aargh, that's too much -- because between changing one table > > > >>>> and all tables there are infinite intermediate points that = all > > > >>>> make sense. > > > >>> It depends. As I said before, we're currently solving this > > > >>> problem > > > by > > > >>> adding new rules (to set X) referencing tables from different > > > >>> range > > > >>> (2048 tables per ruleset) and than doing swap. > > > >>> (And not being able to use named tables to store real names > > > >>> after implementing them is a bit discouraging). > > > >>> > > > >>>> For those cases i think the way to go could be to insert a > > > >>>> 'disabled' new ruleset (however complex it is, so it covers = all > > > >>>> possible cases), and then do the set swap, or disable/enable. > > > >>> We can think of per-set arrays/namespaces of tables: > > > >>> > > > >>> so "ipfw add 100 set X allow ipfw from table(Y) to ..." will > > > reference > > > >>> table Y in set X and > > > >>> "ipfw table ABC list" can differ from "ipfw table ABC set 5 > list". > > > >>> > > > >>> This behavior can break some users setups so we can provide > > > >>> sysctl/tunable to turn this off or on. > > > >>> > > > >>>> cheers > > > >>>> luigi > > > >>>> > > > > > > From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 15:49:18 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5361A325 for ; Thu, 5 Jun 2014 15:49:18 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 108862C48 for ; Thu, 5 Jun 2014 15:49:17 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 5E8F57300A; Thu, 5 Jun 2014 17:54:02 +0200 (CEST) Date: Thu, 5 Jun 2014 17:54:02 +0200 From: 'Luigi Rizzo' To: bycn82 Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140605155402.GA81905@onelab2.iet.unipi.it> References: <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 15:49:18 -0000 On Thu, Jun 05, 2014 at 10:49:27PM +0800, bycn82 wrote: > Hi Luigi, > Yes, use string instead of integer for the ID of table, but the same method cannot apply to the feature `set type of table`. And in the kernel, compare string will cause more than compare an integer. In my opinion, actually they are just alias name for the object. Users already accept that every object has an integer ID. Bill, I appreciate your enthusiasm on contributing to the project, but before starting to code, you (and everyone else, including myself) should spend some time at the drawing board, question your assumptions, and consider possible implementations. Also the fact that previous implementations had bugs or restrictions does not imply that we should repeat the same mistakes now. Specifically: - comparing strings in the kernel is perfectly fine, we do it all the time (sysctl names, filenames, interfaces and whatnot). What would be terribly wrong is having to do, on every packet, an expensive string or number lookup (note- it's the entire lookup process, not a string comparison) to locate the table. However, this is not the case.   The way to implement named objects (tables etc.) is to accept strings   as object names, and when the rule is added the string is checked and converted to a pointers or an integer which permits direct access to the table. This is as fast as it can be at runtime, which is all what matters. - at the moment tables and pipes have a single ID, which happens to be a sequence of digits for mostly historical reasons (you can read it as laziness of the original authors; i can take the blame for pipes). If we extend the namespace to include strings we improve the user's experience and remain compatible with the existing user interface. Instead, if we require users to create a mapping before using alphanumeric names, not only we add a step that was not there before, but also create two names for the same object which makes it harder to reason about the rulesets, and there is also the issue of how to handle references to non-existing tables (which are trivially supported now or with the approach i suggested, and would instead require rescanning the ruleset whenever a name-number association is added or deleted. Another troubles with that approach is that you opening a race window between the creation of the name-number mapping and its use, and this is also something you don't want in a security-related tool. We have had a similar discussion about your 'pps' extension for ipfw: useful feature, but your various implementations have issues (even the final one), and we have only so much time for reviewing patches; please do not burn it. cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 16:10:34 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9324BB12 for ; Thu, 5 Jun 2014 16:10:34 +0000 (UTC) Received: from mail-pb0-x22b.google.com (mail-pb0-x22b.google.com [IPv6:2607:f8b0:400e:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DCAC2E95 for ; Thu, 5 Jun 2014 16:10:34 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id up15so1325505pbc.2 for ; Thu, 05 Jun 2014 09:10:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:thread-index:content-language; bh=7ITMaFxCFSHOW+DtI7rfJdv+cxcn463NKo5NyYfvgyg=; b=LVwHL2CKIJ1Lxd2TZINR34jg1E6kF9KHttd4mF8Gim12A64VTGnctxnSZ8VW+2SE9l 52ZPdy033RKo8bNjE2T7qeeNxIwrMw6vj8Q5SmZMJscpdkarIY2alEuQeQWlmDrgF6tV mmr4GoyEwPiUjDm+DUAuqUwiKiwuFS9FO2r6F6bLcN4Y09cKacbyhAdeHYt6WZD2HY9G SoSj1qdJ3DIL7tMwuwP/aC05csn0T8teZtoMt9H056c608KoAOeriiF/iB2uXdIrZN/6 udYeuPWzcZT/n4TGN/wEqSWjvITARyweJbWmaJaKFN/sEApRpxxXIJA/pGFBLYYExAp6 nQPA== X-Received: by 10.68.216.101 with SMTP id op5mr79067389pbc.148.1401984633893; Thu, 05 Jun 2014 09:10:33 -0700 (PDT) Received: from billwin7 (amx-tls2.starhub.net.sg. [203.116.164.12]) by mx.google.com with ESMTPSA id ak1sm24493269pbc.58.2014.06.05.09.10.29 for (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 05 Jun 2014 09:10:32 -0700 (PDT) From: "bycn82" To: "'Luigi Rizzo'" References: <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> In-Reply-To: <20140605155402.GA81905@onelab2.iet.unipi.it> Subject: RE: [CFT]: ipfw named tables / different tabletypes Date: Fri, 6 Jun 2014 00:10:26 +0800 Message-ID: <000401cf80d8$ad1bb840$075328c0$@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0005_01CF811B.BB40A5F0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQIybStz9zMiwtQqal1c8BJmK0TJagKNzWvvAlvxzMYB26ii/AFrQU/SAWbxMPACXuZNZAL6quccAc5lezAC2ZyC3wK96E6hmeob/RA= Content-Language: en-us Cc: "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 16:10:34 -0000 This is a multipart message in MIME format. ------=_NextPart_000_0005_01CF811B.BB40A5F0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sorry for waste you time to explain it again, I will read the code = first. And the latest patch of `PPS` should be OK, I checked the logic = carefully this time. I sent it out last weekend. logic as below, PPS actually will be fulfilled using `PPT`,(N packets = per M ticks). + case O_PPT: + case O_PPS:{ + ipfw_insn_pps *pps =3D (ipfw_insn_pps *)cmd; + if(pps->limit=3D=3D0){ + int limit,duration_in_ticks; + if(1000/hz > pps->duration){ + duration_in_ticks=3D1; + }else{ + duration_in_ticks=3D(pps->duration * hz +500)/1000; + }=09 + limit=3D(cmd->arg1 *1000 * duration_in_ticks + hz * = pps->duration/2)/(hz * pps->duration); + pps->limit=3Dlimit; + pps->duration_in_ticks=3Dduration_in_ticks; + } + if(pps->start_time+pps->duration_in_ticks>=3D ticks){ + if(pps->count < pps->limit){ + retval =3D IP_FW_PASS; + }else{ + retval =3D IP_FW_DENY; + } + pps->count++; + }else{ + pps->start_time=3Dticks; + pps->count=3D1; + retval =3D IP_FW_PASS; + } + l =3D 0; =09 + done =3D 1; + break;=09 + } =20 > -----Original Message----- > From: 'Luigi Rizzo' [mailto:rizzo@iet.unipi.it] > Sent: 05 June, 2014 23:54 > To: bycn82 > Cc: 'Alexander V. Chernikov'; 'FreeBSD Net' > Subject: Re: [CFT]: ipfw named tables / different tabletypes >=20 > On Thu, Jun 05, 2014 at 10:49:27PM +0800, bycn82 wrote: > > Hi Luigi, > > Yes, use string instead of integer for the ID of table, but the same > method cannot apply to the feature `set type of table`. And in the > kernel, compare string will cause more than compare an integer. In my > opinion, actually they are just alias name for the object. Users = already > accept that every object has an integer ID. >=20 > Bill, > I appreciate your enthusiasm on contributing to the project, but = before > starting to code, you (and everyone else, including myself) should = spend > some time at the drawing board, question your assumptions, and = consider > possible implementations. >=20 > Also the fact that previous implementations had bugs or restrictions > does not imply that we should repeat the same mistakes now. >=20 > Specifically: >=20 > - comparing strings in the kernel is perfectly fine, we do it all the > time (sysctl names, filenames, interfaces and whatnot). > What would be terribly wrong is having to do, on every packet, > an expensive string or number lookup (note- it's the entire lookup > process, not a string comparison) to locate the table. > However, this is not the case. > The way to implement named objects (tables etc.) is to accept = strings > as object names, and when the rule is added the string is checked > and converted to a pointers or an integer which permits direct > access to the table. > This is as fast as it can be at runtime, which is all what matters. >=20 > - at the moment tables and pipes have a single ID, which happens to be = a > sequence of digits for mostly historical reasons (you can read it > as laziness of the original authors; i can take the blame for = pipes). >=20 > If we extend the namespace to include strings we improve the user's > experience and remain compatible with the existing user interface. >=20 > Instead, if we require users to create a mapping before using > alphanumeric names, not only we add a step that was not there = before, > but also create two names for the same object which makes it harder > to reason about the rulesets, and there is also the issue of how > to handle references to non-existing tables (which are trivially > supported now or with the approach i suggested, and would instead > require rescanning the ruleset whenever a name-number association > is added or deleted. > Another troubles with that approach is that you opening a race > window between the creation of the name-number mapping and its use, > and this is also something you don't want in a security-related = tool. >=20 > We have had a similar discussion about your 'pps' extension for ipfw: > useful feature, but your various implementations have issues (even the > final one), and we have only so much time for reviewing patches; = please > do not burn it. >=20 > cheers > luigi ------=_NextPart_000_0005_01CF811B.BB40A5F0 Content-Type: application/octet-stream; name="pps_and_ppt.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="pps_and_ppt.patch" Index: sbin/ipfw/ipfw.8=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sbin/ipfw/ipfw.8 (revision 266886)=0A= +++ sbin/ipfw/ipfw.8 (working copy)=0A= @@ -602,6 +602,22 @@=0A= Note: logging is done after all other packet matching conditions=0A= have been successfully verified, and before performing the final=0A= action (accept, deny, etc.) on the packet.=0A= +.It Cm pps Ar limit duration=0A= +Rule with the =0A= +.Cm pps=0A= +keyword will allow the first=0A= +.Ar limit=0A= +packets in recent =0A= +.Ar duration =0A= +milliseconds.=0A= +.It Cm ppt Ar limit duration=0A= +Rule with the =0A= +.Cm ppt=0A= +keyword will allow the first=0A= +.Ar limit=0A= +packets in recent =0A= +.Ar duration =0A= +ticks.=0A= .It Cm tag Ar number=0A= When a packet matches a rule with the=0A= .Cm tag=0A= Index: sbin/ipfw/ipfw2.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sbin/ipfw/ipfw2.c (revision 266886)=0A= +++ sbin/ipfw/ipfw2.c (working copy)=0A= @@ -244,6 +244,8 @@=0A= { "allow", TOK_ACCEPT },=0A= { "permit", TOK_ACCEPT },=0A= { "count", TOK_COUNT },=0A= + { "pps", TOK_PPS },=0A= + { "ppt", TOK_PPT },=0A= { "pipe", TOK_PIPE },=0A= { "queue", TOK_QUEUE },=0A= { "divert", TOK_DIVERT },=0A= @@ -1232,6 +1234,20 @@=0A= PRINT_UINT_ARG("skipto ", cmd->arg1);=0A= break;=0A= =0A= + case O_PPS:=0A= + {=0A= + ipfw_insn_pps *pps=3D(ipfw_insn_pps *)cmd;=0A= + printf("pps %d %d",cmd->arg1,pps->duration);=0A= + break; =0A= + }=0A= + =0A= + case O_PPT:=0A= + {=0A= + ipfw_insn_pps *pps=3D(ipfw_insn_pps *)cmd;=0A= + printf("ppt %d %d",pps->limit,pps->duration_in_ticks);=0A= + break; =0A= + }=0A= +=0A= case O_PIPE:=0A= PRINT_UINT_ARG("pipe ", cmd->arg1);=0A= break;=0A= @@ -2985,7 +3001,43 @@=0A= case TOK_COUNT:=0A= action->opcode =3D O_COUNT;=0A= break;=0A= + =0A= + case TOK_PPS:=0A= + action->opcode =3D O_PPS;=0A= + ipfw_insn_pps *pps =3D (ipfw_insn_pps *)action;=0A= + action->len =3D F_INSN_SIZE(ipfw_insn_pps);=0A= + if (isdigit(**av)) {=0A= + action->arg1 =3D strtoul(*av, NULL, 10);=0A= + av++;=0A= + }else{=0A= + errx(EX_USAGE, "illegal argument pps `limit` %s", *av);=0A= + }=0A= + if (isdigit(**av)) {=0A= + pps->duration=3D strtoul(*av, NULL, 10);=0A= + av++; =0A= + }else{=0A= + errx(EX_USAGE,"illegal arugment pps `duration` %s", *av);=0A= + }=0A= + break; =0A= =0A= + case TOK_PPT:=0A= + action->opcode =3D O_PPT;=0A= + ipfw_insn_pps *ppt =3D (ipfw_insn_pps *)action;=0A= + action->len =3D F_INSN_SIZE(ipfw_insn_pps);=0A= + if (isdigit(**av)) {=0A= + ppt->limit =3D strtoul(*av, NULL, 10);=0A= + av++;=0A= + }else{=0A= + errx(EX_USAGE, "illegal argument ppt `limit` %s", *av);=0A= + }=0A= + if (isdigit(**av)) {=0A= + ppt->duration_in_ticks=3D strtoul(*av, NULL, 10);=0A= + av++; =0A= + }else{=0A= + errx(EX_USAGE,"illegal arugment ppt `ticks` %s", *av);=0A= + }=0A= + break; =0A= +=0A= case TOK_NAT:=0A= action->opcode =3D O_NAT;=0A= action->len =3D F_INSN_SIZE(ipfw_insn_nat);=0A= Index: sbin/ipfw/ipfw2.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sbin/ipfw/ipfw2.h (revision 266886)=0A= +++ sbin/ipfw/ipfw2.h (working copy)=0A= @@ -92,6 +92,8 @@=0A= TOK_NGTEE,=0A= TOK_FORWARD,=0A= TOK_SKIPTO,=0A= + TOK_PPS,=0A= + TOK_PPT,=0A= TOK_DENY,=0A= TOK_REJECT,=0A= TOK_RESET,=0A= Index: sys/netinet/ip_fw.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netinet/ip_fw.h (revision 266886)=0A= +++ sys/netinet/ip_fw.h (working copy)=0A= @@ -165,6 +165,8 @@=0A= O_REJECT, /* arg1=3Dicmp arg (same as deny) */=0A= O_COUNT, /* none */=0A= O_SKIPTO, /* arg1=3Dnext rule number */=0A= + O_PPS, /* arg1=3Dlimit, pps->duration */=0A= + O_PPT, /* pps->limit, pps->duration_in_ticks */=0A= O_PIPE, /* arg1=3Dpipe number */=0A= O_QUEUE, /* arg1=3Dqueue number */=0A= O_DIVERT, /* arg1=3Dport number */=0A= @@ -378,6 +380,18 @@=0A= } ipfw_insn_log;=0A= =0A= /*=0A= + * This is used for PPS=0A= + */=0A= +typedef struct _ipfw_insn_pps{=0A= + ipfw_insn o;=0A= + uint32_t start_time;=0A= + uint32_t count;=0A= + uint32_t duration; =0A= + uint32_t limit;=0A= + uint32_t duration_in_ticks;=0A= +} ipfw_insn_pps;=0A= +=0A= +/*=0A= * Data structures required by both ipfw(8) and ipfw(4) but not part of = the=0A= * management API are protected by IPFW_INTERNAL.=0A= */=0A= Index: sys/netpfil/ipfw/ip_fw2.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netpfil/ipfw/ip_fw2.c (revision 266886)=0A= +++ sys/netpfil/ipfw/ip_fw2.c (working copy)=0A= @@ -2188,6 +2188,37 @@=0A= skip_or =3D 0;=0A= continue;=0A= break; /* not reached */=0A= + =0A= + case O_PPT:=0A= + case O_PPS:{=0A= + ipfw_insn_pps *pps =3D (ipfw_insn_pps *)cmd;=0A= + if(pps->limit=3D=3D0){=0A= + int limit,duration_in_ticks;=0A= + if(1000/hz > pps->duration){=0A= + duration_in_ticks=3D1;=0A= + }else{=0A= + duration_in_ticks=3D(pps->duration * hz +500)/1000;=0A= + } =0A= + limit=3D(cmd->arg1 *1000 * duration_in_ticks + hz * = pps->duration/2)/(hz * pps->duration);=0A= + pps->limit=3Dlimit;=0A= + pps->duration_in_ticks=3Dduration_in_ticks;=0A= + }=0A= + if(pps->start_time+pps->duration_in_ticks>=3D ticks){=0A= + if(pps->count < pps->limit){=0A= + retval =3D IP_FW_PASS;=0A= + }else{=0A= + retval =3D IP_FW_DENY;=0A= + }=0A= + pps->count++;=0A= + }else{=0A= + pps->start_time=3Dticks;=0A= + pps->count=3D1;=0A= + retval =3D IP_FW_PASS;=0A= + }=0A= + l =3D 0; =0A= + done =3D 1;=0A= + break; =0A= + }=0A= =0A= case O_CALLRETURN: {=0A= /*=0A= Index: sys/netpfil/ipfw/ip_fw_sockopt.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/netpfil/ipfw/ip_fw_sockopt.c (revision 266886)=0A= +++ sys/netpfil/ipfw/ip_fw_sockopt.c (working copy)=0A= @@ -703,6 +703,13 @@=0A= goto bad_size;=0A= break;=0A= =0A= + case O_PPS:=0A= + case O_PPT:=0A= + have_action=3D1;=0A= + if (cmdlen !=3D F_INSN_SIZE(ipfw_insn_pps))=0A= + goto bad_size;=0A= + break;=0A= +=0A= case O_PIPE:=0A= case O_QUEUE:=0A= if (cmdlen !=3D F_INSN_SIZE(ipfw_insn))=0A= ------=_NextPart_000_0005_01CF811B.BB40A5F0-- From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 16:53:22 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C7107FE for ; Thu, 5 Jun 2014 16:53:22 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 1A9D722CF for ; Thu, 5 Jun 2014 16:53:21 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 3E09A7300A; Thu, 5 Jun 2014 18:58:06 +0200 (CEST) Date: Thu, 5 Jun 2014 18:58:06 +0200 From: 'Luigi Rizzo' To: bycn82 Subject: Re: [CFT]: ipfw named tables / different tabletypes Message-ID: <20140605165806.GC81905@onelab2.iet.unipi.it> References: <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> <000401cf80d8$ad1bb840$075328c0$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000401cf80d8$ad1bb840$075328c0$@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 16:53:22 -0000 On Fri, Jun 06, 2014 at 12:10:26AM +0800, bycn82 wrote: > Sorry for waste you time to explain it again, I will read the code first. > > And the latest patch of `PPS` should be OK, I checked the logic carefully this time. I sent it out last weekend. > > logic as below, PPS actually will be fulfilled using `PPT`,(N packets per M ticks). Bill, sorry but even this version has multiple issues -- performance, correctness, style. performance: duration_in_ticks and limit do not need to be recomputed on every packet, just once when you install the rule. correctness limit should be not lower than the user-specified value, so if the equation has the form \lceil{a}{b} \rceil, the correct computation using integer arithmetic is (a + b - 1)/b and not (a + b/2) / b You should also check for corner cases, overflows etc. I haven't actually checked that the equation you use is correct, so a comment explaining why it is would be appropriate. BTW you have no checks on the input arguments, so you can have duration =0 and then you have a division by 0 in the kernel. style boring as it might be, we use spaces around keywords and { } I am also unhappy on the ppt option, because the duration of a tick can change across reboots, so this option would give variable behaviour. With all sources of uncertainty on packet arrivals and when the timer fires to advance 'ticks', there is really no point in having PPT to avoid the rounding error with some strange values of HZ. Now before you think i am too picky: some of the above (style) are trivial issues, but they require manual editing of the patch before applying and this is the reason many project and people usually bounce patches just because of style (the process does not scale otherwise). If it were just for that, i would not mind. But many of the other problems are more serious, and the fact they keep coming out after so many iterations suggest me that _you_ should spend more time reviewing your code before submitting it and asking others to find and fix problems. cheers luigi > > + case O_PPT: > + case O_PPS:{ > + ipfw_insn_pps *pps = (ipfw_insn_pps *)cmd; > + if(pps->limit==0){ > + int limit,duration_in_ticks; > + if(1000/hz > pps->duration){ > + duration_in_ticks=1; > + }else{ > + duration_in_ticks=(pps->duration * hz +500)/1000; > + } > + limit=(cmd->arg1 *1000 * duration_in_ticks + hz * pps->duration/2)/(hz * pps->duration); > + pps->limit=limit; > + pps->duration_in_ticks=duration_in_ticks; > + } > + if(pps->start_time+pps->duration_in_ticks>= ticks){ > + if(pps->count < pps->limit){ > + retval = IP_FW_PASS; > + }else{ > + retval = IP_FW_DENY; > + } > + pps->count++; > + }else{ > + pps->start_time=ticks; > + pps->count=1; > + retval = IP_FW_PASS; > + } > + l = 0; > + done = 1; > + break; > + } From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:27:53 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E08D45E0 for ; Thu, 5 Jun 2014 22:27:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C7E35222B for ; Thu, 5 Jun 2014 22:27:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MRrBM086080 for ; Thu, 5 Jun 2014 23:27:53 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 127360] [socket] TOE socket options missing from sosetopt() Date: Thu, 05 Jun 2014 22:27:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 1.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:27:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=127360 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:28:46 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95C1A67A for ; Thu, 5 Jun 2014 22:28:46 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D4F8223B for ; Thu, 5 Jun 2014 22:28:46 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MSkBd095802 for ; Thu, 5 Jun 2014 23:28:46 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 137776] [rum] panic in rum(4) driver on 8.0-BETA2 Date: Thu, 05 Jun 2014 22:28:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 7.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:28:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137776 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:30:15 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 776EE768 for ; Thu, 5 Jun 2014 22:30:15 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5EC022274 for ; Thu, 5 Jun 2014 22:30:15 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MUF2n013070 for ; Thu, 5 Jun 2014 23:30:15 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 137841] [patch] wpa_supplicant(8) cannot verify SHA256 signed certificates Date: Thu, 05 Jun 2014 22:30:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 8.0-BETA2 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:30:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137841 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:31:18 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45BC4816 for ; Thu, 5 Jun 2014 22:31:18 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D0E92302 for ; Thu, 5 Jun 2014 22:31:18 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MVIWK030258 for ; Thu, 5 Jun 2014 23:31:18 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 139204] [arp] DHCP server replies rejected, ARP entry lost before max_age Date: Thu, 05 Jun 2014 22:31:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:31:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139204 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:33:46 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACC4A924 for ; Thu, 5 Jun 2014 22:33:46 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 93E65232E for ; Thu, 5 Jun 2014 22:33:46 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MXkNq076854 for ; Thu, 5 Jun 2014 23:33:46 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 145600] TCP/ECN behaves different to CE/CWR than ns2 reference Date: Thu, 05 Jun 2014 22:33:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:33:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=145600 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:36:14 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AF93AD1 for ; Thu, 5 Jun 2014 22:36:14 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51BA82355 for ; Thu, 5 Jun 2014 22:36:14 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MaE9c004319 for ; Thu, 5 Jun 2014 23:36:14 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 168294] [ixgbe] [patch] ixgbe driver compiled in kernel has no Flow Director support although loaded as a module has one Date: Thu, 05 Jun 2014 22:36:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:36:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168294 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:36:58 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC3F9B61 for ; Thu, 5 Jun 2014 22:36:58 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2E3E2364 for ; Thu, 5 Jun 2014 22:36:58 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55MawUd012632 for ; Thu, 5 Jun 2014 23:36:58 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 172113] [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type Date: Thu, 05 Jun 2014 22:36:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:36:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=172113 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 5 22:41:55 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D5FBC9C for ; Thu, 5 Jun 2014 22:41:55 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64902240B for ; Thu, 5 Jun 2014 22:41:55 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s55Mftfv021736 for ; Thu, 5 Jun 2014 23:41:55 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 154169] [multicast] [ip6] Node Information Query multicast address wrong (FF02::2:x:y) Date: Thu, 05 Jun 2014 22:41:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-PRERELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 22:41:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154169 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |hiren@FreeBSD.org Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Jun 6 03:23:31 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7549955 for ; Fri, 6 Jun 2014 03:23:31 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E4A82BDE for ; Fri, 6 Jun 2014 03:23:31 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id z12so2166116wgg.24 for ; Thu, 05 Jun 2014 20:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JLjKmfYnL2kM6sFpk2JpuPxlEqfpc7tH7qppgyiKwkk=; b=KiQ71sw9ToN4hpbAOJ5ACeC+vHLPDBR8ynX1VgdB6xK4iWpy1xlD3gkxgAhcjDqZcW cVuysuKfFVqxo2Gz6NU+rrp9tce74w3x7KhnFh3/LU4/HBcAXBSdOz4Lua9scs0AfxEu p8JfuM3U7YKXGu6/aSnQ77plWgPSilhKANN1eKtyT9Bs34bp+T0Jwn0JS1UdY7ICIwl/ pOBm+H0CEnHrXn7/CBwSX4ccbAskKQPKBxtx9+/4oT7+MnvK2UVc5H49dSfUsvlygFf+ 4JoVN8BI/H3A4NRFzLsEcgf0TIdepBIJRcQLihJL5a25hp9HHDmgItR26Nh4EFI8y+Ux QaWg== MIME-Version: 1.0 X-Received: by 10.195.11.132 with SMTP id ei4mr45622wjd.95.1402025009515; Thu, 05 Jun 2014 20:23:29 -0700 (PDT) Received: by 10.216.232.68 with HTTP; Thu, 5 Jun 2014 20:23:29 -0700 (PDT) In-Reply-To: <20140605165806.GC81905@onelab2.iet.unipi.it> References: <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> <000401cf80d8$ad1bb840$075328c0$@gmail.com> <20140605165806.GC81905@onelab2.iet.unipi.it> Date: Fri, 6 Jun 2014 11:23:29 +0800 Message-ID: Subject: Re: [CFT]: ipfw named tables / different tabletypes From: Sato Kentney To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: bycn82 , "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 03:23:31 -0000 Your original porpose of pps is good enough for me, it controls packets number for each second. it is simple and make sence, As a network administrator, I think if you provide "packets per second" or "packets per minute" ,that is good enough for 99% percent of users. Thanks, Sato K(=E4=BD=90=E8=97=A4=E6=9F=AF=E5=BE=B7) On Fri, Jun 6, 2014 at 12:58 AM, Luigi Rizzo wrote: > On Fri, Jun 06, 2014 at 12:10:26AM +0800, bycn82 wrote: > > Sorry for waste you time to explain it again, I will read the code firs= t. > > > > And the latest patch of `PPS` should be OK, I checked the logic > carefully this time. I sent it out last weekend. > > > > logic as below, PPS actually will be fulfilled using `PPT`,(N packets > per M ticks). > > Bill, > sorry but even this version has multiple issues -- performance, > correctness, style. > > performance: > duration_in_ticks and limit do not need to be recomputed > on every packet, just once when you install the rule. > > correctness > limit should be not lower than the user-specified value, > so if the equation has the form \lceil{a}{b} \rceil, > the correct computation using integer arithmetic is > (a + b - 1)/b and not (a + b/2) / b > > You should also check for corner cases, overflows etc. > I haven't actually checked that the equation you use is > correct, so a comment explaining why it is would be appropriate. > > BTW you have no checks on the input arguments, so you can > have duration =3D0 and then you have a division by 0 in the kernel. > > style > boring as it might be, we use spaces around keywords and { } > > I am also unhappy on the ppt option, because the duration of a tick > can change across reboots, so this option would give variable > behaviour. With all sources of uncertainty on packet arrivals > and when the timer fires to advance 'ticks', there is really > no point in having PPT to avoid the rounding error with some > strange values of HZ. > > > Now before you think i am too picky: > some of the above (style) are trivial issues, but they require > manual editing of the patch before applying and this is the reason > many project and people usually bounce patches just because of style > (the process does not scale otherwise). > If it were just for that, i would not mind. > > But many of the other problems are more serious, and the fact they > keep coming out after so many iterations suggest me that _you_ > should spend more time reviewing your code before submitting it > and asking others to find and fix problems. > > cheers > luigi > > > > > + case O_PPT: > > + case O_PPS:{ > > + ipfw_insn_pps *pps =3D (ipfw_insn_pps *)c= md; > > + if(pps->limit=3D=3D0){ > > + int limit,duration_in_ticks; > > + if(1000/hz > pps->duration){ > > + duration_in_ticks=3D1; > > + }else{ > > + > duration_in_ticks=3D(pps->duration * hz +500)/1000; > > + } > > + limit=3D(cmd->arg1 *1000 * > duration_in_ticks + hz * pps->duration/2)/(hz * pps->duration); > > + pps->limit=3Dlimit; > > + > pps->duration_in_ticks=3Dduration_in_ticks; > > + } > > + > if(pps->start_time+pps->duration_in_ticks>=3D ticks){ > > + if(pps->count < pps->limit){ > > + retval =3D IP_FW_PASS; > > + }else{ > > + retval =3D IP_FW_DENY; > > + } > > + pps->count++; > > + }else{ > > + pps->start_time=3Dticks; > > + pps->count=3D1; > > + retval =3D IP_FW_PASS; > > + } > > + l =3D 0; > > + done =3D 1; > > + break; > > + } > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Jun 6 07:18:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D2C4ADC for ; Fri, 6 Jun 2014 07:18:18 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD3AC2E25 for ; Fri, 6 Jun 2014 07:18:16 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1WsoPk-0002Rp-Dm for freebsd-net@freebsd.org; Fri, 06 Jun 2014 09:18:12 +0200 Received: from tempe0.bbox.io ([24.249.180.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 06 Jun 2014 09:18:12 +0200 Received: from kevin.bowling by tempe0.bbox.io with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 06 Jun 2014 09:18:12 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Kevin Bowling Subject: Re: FreeBSD 10 - ixgbe packet drop Date: Fri, 06 Jun 2014 00:17:56 -0700 Lines: 21 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: tempe0.bbox.io User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Thunderbird/30.0 In-Reply-To: X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 07:18:18 -0000 On 6/4/2014 11:46 PM, Özkan KIRIK wrote: > Hi > > I'm using FreeBSD 10. > My ix0 is connected to my backbone switch. > Traffic is about 90Mbit/s. > But after 3 minutes it stops working. > ifconfig ix0 down ; ifconfig ix0 up solves problem temprorarily. > > It's strange that, dev.ix.0.dropped is 0 but, netstat's Idrop counter is > growing. > How can i debug this situation? > dev.ix.0.mac_stats.local_faults: 33864 > dev.ix.0.mac_stats.remote_faults: 18168 These faults look similar to a problem I have with the X520 using copper twinax cables. What is your physical topology including cabling type? From owner-freebsd-net@FreeBSD.ORG Fri Jun 6 13:42:10 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED577385 for ; Fri, 6 Jun 2014 13:42:10 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAC3A2254 for ; Fri, 6 Jun 2014 13:42:10 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WsqU9-0001of-5p; Fri, 06 Jun 2014 13:30:53 +0400 Message-ID: <5391C4BB.50106@FreeBSD.org> Date: Fri, 06 Jun 2014 17:40:11 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Ian Smith , bycn82 Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> <000401cf80d8$ad1bb840$075328c0$@gmail.com> <20140606222753.W15833@sola.nimnet.asn.au> In-Reply-To: <20140606222753.W15833@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: 'Luigi Rizzo' , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 13:42:11 -0000 On 06.06.2014 17:31, Ian Smith wrote: > On Fri, 6 Jun 2014 00:10:26 +0800, bycn82 wrote: Guys, I do understand that this is an important discussion about useful ipfw feature, but can you please stop invading this (totally unrelated) topic and return to original one? Thank you. > > Hi Bill, > > > Sorry for waste you time to explain it again, I will read the code first. > > Especially the code provided in free tutorials by your busy professor .. > > > And the latest patch of `PPS` should be OK, I checked the logic carefully this time. I sent it out last weekend. > > > > logic as below, PPS actually will be fulfilled using `PPT`,(N packets per M ticks). > > I think a few people have pointed out likely problems with 'packets per > tick(s)', and that people tend to prefer packets per second as a more > natural and familiar concept. I can see use cases for that, especially > when applied by easily updateable (and soon, saveable) tables. > > Remember that HZ may be set at boot time, and will at times by people > experimenting with, as one example, dummynet latency versus cpu use, so > rulesets specifying packets per tick would need also to be modified to > match, which won't happen. Packets per second is independent of HZ and > far easier to comprehend. See inetd(8) for a typical PPM example, while > PPS makes more sense for a firewall. > > I wonder if something like Bresenham's Linedrawing Algorithm might help? > > cheers, Ian > From owner-freebsd-net@FreeBSD.ORG Fri Jun 6 13:42:31 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8277740F for ; Fri, 6 Jun 2014 13:42:31 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3B54E2265 for ; Fri, 6 Jun 2014 13:42:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s56DVvMo018411; Fri, 6 Jun 2014 23:31:57 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Fri, 6 Jun 2014 23:31:57 +1000 (EST) From: Ian Smith To: bycn82 Subject: RE: [CFT]: ipfw named tables / different tabletypes In-Reply-To: <000401cf80d8$ad1bb840$075328c0$@gmail.com> Message-ID: <20140606222753.W15833@sola.nimnet.asn.au> References: <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> <000401cf80d8$ad1bb840$075328c0$@gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: 'Luigi Rizzo' , "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 13:42:31 -0000 On Fri, 6 Jun 2014 00:10:26 +0800, bycn82 wrote: Hi Bill, > Sorry for waste you time to explain it again, I will read the code first. Especially the code provided in free tutorials by your busy professor .. > And the latest patch of `PPS` should be OK, I checked the logic carefully this time. I sent it out last weekend. > > logic as below, PPS actually will be fulfilled using `PPT`,(N packets per M ticks). I think a few people have pointed out likely problems with 'packets per tick(s)', and that people tend to prefer packets per second as a more natural and familiar concept. I can see use cases for that, especially when applied by easily updateable (and soon, saveable) tables. Remember that HZ may be set at boot time, and will at times by people experimenting with, as one example, dummynet latency versus cpu use, so rulesets specifying packets per tick would need also to be modified to match, which won't happen. Packets per second is independent of HZ and far easier to comprehend. See inetd(8) for a typical PPM example, while PPS makes more sense for a firewall. I wonder if something like Bresenham's Linedrawing Algorithm might help? cheers, Ian From owner-freebsd-net@FreeBSD.ORG Fri Jun 6 14:09:16 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 964F0B61 for ; Fri, 6 Jun 2014 14:09:16 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4894C24C9 for ; Fri, 6 Jun 2014 14:09:16 +0000 (UTC) Received: from Julian-MBP3.local (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s56E93wR024716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 6 Jun 2014 07:09:07 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5391CB79.5040908@freebsd.org> Date: Fri, 06 Jun 2014 22:08:57 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Ian Smith , bycn82 Subject: Re: [CFT]: ipfw named tables / different tabletypes References: <20140521204826.GA67124@onelab2.iet.unipi.it> <537E1029.70007@FreeBSD.org> <20140522154740.GA76448@onelab2.iet.unipi.it> <537E2153.1040005@FreeBSD.org> <20140522163812.GA77634@onelab2.iet.unipi.it> <538B2FE5.6070407@FreeBSD.org> <539044E4.1020904@ipfw.ru> <000c01cf80be$41194370$c34bca50$@gmail.com> <20140605134256.GA81234@onelab2.iet.unipi.it> <000001cf80cd$5dc1d9b0$19458d10$@gmail.com> <20140605155402.GA81905@onelab2.iet.unipi.it> <000401cf80d8$ad1bb840$075328c0$@gmail.com> <20140606222753.W15833@sola.nimnet.asn.au> In-Reply-To: <20140606222753.W15833@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: 'Luigi Rizzo' , "'Alexander V. Chernikov'" , 'FreeBSD Net' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 14:09:16 -0000 On 6/6/14, 9:31 PM, Ian Smith wrote: > On Fri, 6 Jun 2014 00:10:26 +0800, bycn82 wrote: > > Hi Bill, > > > Sorry for waste you time to explain it again, I will read the code first. > > Especially the code provided in free tutorials by your busy professor .. > > > And the latest patch of `PPS` should be OK, I checked the logic carefully this time. I sent it out last weekend. > > > > logic as below, PPS actually will be fulfilled using `PPT`,(N packets per M ticks). > > I think a few people have pointed out likely problems with 'packets per > tick(s)', and that people tend to prefer packets per second as a more > natural and familiar concept. I can see use cases for that, especially > when applied by easily updateable (and soon, saveable) tables. > > Remember that HZ may be set at boot time, and will at times by people > experimenting with, as one example, dummynet latency versus cpu use, so > rulesets specifying packets per tick would need also to be modified to > match, which won't happen. Packets per second is independent of HZ and > far easier to comprehend. See inetd(8) for a typical PPM example, while > PPS makes more sense for a firewall. > > I wonder if something like Bresenham's Linedrawing Algorithm might help? I already gave him a sketch of such an algorithm. > > cheers, Ian > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 06:25:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9DA3B4E9 for ; Sat, 7 Jun 2014 06:25:39 +0000 (UTC) Received: from nm34.bullet.mail.ne1.yahoo.com (nm34.bullet.mail.ne1.yahoo.com [98.138.229.27]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5EB6E209B for ; Sat, 7 Jun 2014 06:25:38 +0000 (UTC) Received: from [127.0.0.1] by nm34.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:25:32 -0000 Received: from [98.138.100.112] by nm34.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:22:47 -0000 Received: from [66.196.81.173] by tm103.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:22:47 -0000 Received: from [98.139.212.246] by tm19.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 06:22:47 -0000 Received: from [127.0.0.1] by omp1055.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 06:22:47 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 204927.34637.bm@omp1055.mail.bf1.yahoo.com Received: (qmail 81892 invoked by uid 60001); 7 Jun 2014 06:22:47 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1402122167; bh=Wu8u18gM9w3gWWUPnglObGilxwCuunMpyvsGDVkeUXY=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=cxLZrsUlCHoFYQ8Ua9Q+aEOOBkxQGb2q+s7vA0rTnt5bgyPtIneqr5NUyr83s8OQKGTmOsH2SsZkseATUfWgBXELiLhtbI6cbyZWIW9YNSBs8lSJmnL+rLgONvYjN9W0vIlvEin6EPh1sk8Mkt/j/hGICAe5eEZaWFrEJ2dQqCo= X-YMail-OSG: tKqsHOkVM1kL4NJuG1DEp8TGGiqp0CO0icB.mNitX32cZ10 fk3F2Sf4PqCUzMSBPVy0awmwZASG3kKTzUBaRZzQ3PGQlFUTDln3FzwWm26R p864tFw9jcYgxi8SfNtR5h6EnB67LX7yd8aJyWfw908erc_zmHPqyPLo1VZo SC5YJpdHAwHMnJtUX.xXQ7TbLgRXBktGRzCAhhRESyUlmdDDaXvVu1aVoJ76 zONRSs_ltG20T_xRkJh.jpqqoCXvP1FlZq30y8JEmkxr_ur6uowf.5Rbcw1X Y3oT94N7osE92kLYOMj0TpEH8wUf7fEuwUgvh1gWYJIEGuDw8Dn7d.iCiIPa fTSG9I6B_JddGYe_ag35uP1Dopd8d1EToLM_KADuplhTnQ31dI8xDsVQcZVi 2lKm8CLHUbIm7agoAyA_ooRRy.I9I97WEEVSoY8g.kluS.MV_yVfIiSFlZp6 DmXLcRiwUHwmvPHOTLpKFAQXuYNvF4HctT68ZjxudKFfrM.EfZCcBhYveK7p li5TDTPcEEAQ- Received: from [12.202.173.2] by web162101.mail.bf1.yahoo.com via HTTP; Fri, 06 Jun 2014 23:22:46 PDT X-Rocket-MIMEInfo: 002.001, SSd2ZSBidWlsdCBhIGxvdCBvZiBnYXRld2F5cy9yb3V0ZXJzIHdpdGggRnJlZUJTRCAtIGJ1dCB0aGV5IGhhdmUgYWx3YXlzIGJlZW4gd2l0aCByZWFsLCByb3V0YWJsZSBJUHMuCgpFeHRlcm5hbCBJUCBpcyByZWFsLCBpbnRlcm5hbCBJUCBpcyByZWFsLCBhbmQgYWxsIEkgbmVlZCBpcyBnYXRld2F5X2VuYWJsZT0ieWVzIiBhbmQgYSBuZXh0LWhvcCByb3V0ZSBmcm9tIG15IElTUC4KCk5vIE5BVCwgbm8gZGl2ZXJ0LCBubyBpcGZ3IHJ1bGVzLCBub3RoaW5nLgoKQlVULCB3aGF0IGlmIG15IElTUCBpcyBnaXYBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 Message-ID: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> Date: Fri, 6 Jun 2014 23:22:46 -0700 (PDT) From: None Secure Reply-To: None Secure Subject: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? To: "freebsd-net@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 06:25:39 -0000 I've built a lot of gateways/routers with FreeBSD - but they have always be= en with real, routable IPs.=0A=0AExternal IP is real, internal IP is real, = and all I need is gateway_enable=3D"yes" and a next-hop route from my ISP.= =0A=0ANo NAT, no divert, no ipfw rules, nothing.=0A=0ABUT, what if my ISP i= s giving me a private IP, and my internal network is also private IPs ? =A0= External gateway address is 192.168.1.2 and internal gateway address is 10.= 10.10.1 ... the ONLY way I could make this work is with natd and ipfw diver= t rules.=0A=0AMy question is: =A0is it possible to have a network of non-ro= utable IPs, and a gateway with non-routable Ips on internal and external in= terfaces, and NOT use natd/divert ? =A0Can it be done with no ipfw rules at= all, just like I used to ?=0A=0AThanks. From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 06:28:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 465D25CF for ; Sat, 7 Jun 2014 06:28:31 +0000 (UTC) Received: from nm46.bullet.mail.ne1.yahoo.com (nm46.bullet.mail.ne1.yahoo.com [98.138.120.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F339F20BC for ; Sat, 7 Jun 2014 06:28:30 +0000 (UTC) Received: from [127.0.0.1] by nm46.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:28:24 -0000 Received: from [98.138.101.128] by nm46.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:25:34 -0000 Received: from [66.196.81.173] by tm16.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 06:25:33 -0000 Received: from [98.139.212.207] by tm19.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 06:25:33 -0000 Received: from [127.0.0.1] by omp1016.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 06:25:33 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 781178.47850.bm@omp1016.mail.bf1.yahoo.com Received: (qmail 85055 invoked by uid 60001); 7 Jun 2014 06:25:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1402122333; bh=MMUawwih6R1vKqlCaXTvePi6evc33TO04GYEQW5dZjk=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=jqlW3adUsNJ9Fkj6lYLQlX0KW8cOQ0N7AAVhHA3wGhHKjNVADSqhCC3+HX04/PTUFHHgFtlBmzJ99Z1q41IqQ6JVmTN+48iwDATo2/GhmAjw/Sdg+F0H56uJVSPPOSmgffQoiAf0JdwYIEoIe51xx/sqtuCuanLu5IsWbTDnsnk= X-YMail-OSG: QiEi.FgVM1lLqHXF4.ZwCQpwhZqBYbAMVenKYa_njujmsIO OBRZdKkHXvgLQBKHpTdXhZjrqZkZ2FtM1xuftBywDGDVBuxylaBdb1lVX_dP vA45cCojEHJc3Ei1zJY86MlHp_FtAXssuxiR5S3ivQlexpTNwGHX_iGXP2As QT5nXF.0Qqd5eqfy_g_JxItjCmVJKkVT3M1A708ByURjXOcKNtgEnaH.kNfE gcHyhxjUPDqzfUd_GBLaAqolLFEDO4Of3GCaBAP35OxDg_eV8APmTwFcIdsb qEVt.F4eaedpxriOa7y9as_IlMzbDlf0qLO4PyvaVONe1Ye4ic7Kk2_zxNbm 2vrDM4Bpsyivey4fDSc4Gcl1AY5IwDi2FJiYEqqyvKfOHJWkZZklPu9ivGo5 .ztXQOEtOGOq0vJprrRtkvLcnQ59qHRoQisS13lrbdwM6bTe2fmGkyOsCb5. ezfbp23QLN8KlisSA1wM1oRwlQ.72VDwMCtht03YoSSjGWqLpEr_SQzGjSTo UdybIKBJg Received: from [12.202.173.2] by web162101.mail.bf1.yahoo.com via HTTP; Fri, 06 Jun 2014 23:25:33 PDT X-Rocket-MIMEInfo: 002.001, SSB3b3VsZCBsaWtlIHZlcnkgbXVjaCB0byB1c2Ugc3NodXR0bGUgZm9yIGFuIGluZm9ybWFsIFZQTi4KCkhvd2V2ZXIsIHNzaHV0dGxlIHNldHMgdXAgYSBsb3Qgb2YgY29tcGxleGl0eSBpbiBvcmRlciB0byByb3V0ZSBETlMgcmVxdWVzdHMgb3ZlciB0aGUgc3NoIHR1bm5lbCAuLi4gaXQgdXNlcyBkaXZlcnQgcnVsZXMgZm9yIGRucyB0cmFmZmljLCBhbmQgSSBkb24ndCB0aGluayB0aGV5IGV2ZW4gdGVzdGVkIGl0IGJlY2F1c2UgaXQgZmFpbHMgdG8gc3RhcnQgb3IgdXRpbGl6ZSBuYXRkLgoKVGhlIHN0YXQBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 Message-ID: <1402122333.57974.YahooMailNeo@web162101.mail.bf1.yahoo.com> Date: Fri, 6 Jun 2014 23:25:33 -0700 (PDT) From: None Secure Reply-To: None Secure Subject: Does FreeBSD have the ability to properly forward UDP traffic ? To: "freebsd-net@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 06:28:31 -0000 I would like very much to use sshuttle for an informal VPN. However, sshuttle sets up a lot of complexity in order to route DNS requests over the ssh tunnel ... it uses divert rules for dns traffic, and I don't think they even tested it because it fails to start or utilize natd. The stated reason by sshuttle project is that you can't just forward UDP traffic properly with BSD, like you can with linux - they say it doesn't keep track of port numbers or connections properly. Is this true ? Or is it possible to properly forward UDP traffic with ipfw rules, and not use natd/divert ? Thanks. From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 06:33:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 108FC750 for ; Sat, 7 Jun 2014 06:33:19 +0000 (UTC) Received: from mail-pd0-x22b.google.com (mail-pd0-x22b.google.com [IPv6:2607:f8b0:400e:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA0B02150 for ; Sat, 7 Jun 2014 06:33:18 +0000 (UTC) Received: by mail-pd0-f171.google.com with SMTP id y13so3283194pdi.16 for ; Fri, 06 Jun 2014 23:33:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=y4vZUE36FY7c5T+X1rBQxvRWuzVgIiirZlWyVd8M+ck=; b=uCsnaiT9suvAgND7TNPRaEnuFrvWI/Hs82ASp41PyjLYikEsUBPp/gXQbRJvwSH46Q /izuom30AjmFVY4mVHoovCPbmKObdnE6WlEwjYn+/fCzZDuelI47cjWUoXJlbJBWWtCL t0ZLdru47sudNEieBdUrBaX3cJMhKMN73KbZBqTMB5fvkOxQsXjhOcWsYdo33DbdIxQ2 kgDXrM4Ek2Hs1+PqgKCl0p8FP+b9irOUgarsD0EjA00Zj6Ls3oAJR8IsPj7uqLbscSA2 9+YUBgyBWSFIapezu//ZMeIUL2wi3NvWdI2hJBxWd9dq1GCYNNBJ+/MdkJ0DeFzlLYq8 PSPw== MIME-Version: 1.0 X-Received: by 10.68.97.129 with SMTP id ea1mr8746533pbb.73.1402122798176; Fri, 06 Jun 2014 23:33:18 -0700 (PDT) Received: by 10.70.75.195 with HTTP; Fri, 6 Jun 2014 23:33:18 -0700 (PDT) Received: by 10.70.75.195 with HTTP; Fri, 6 Jun 2014 23:33:18 -0700 (PDT) In-Reply-To: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> Date: Sat, 7 Jun 2014 09:33:18 +0300 Message-ID: Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? From: Sami Halabi To: None Secure Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 06:33:19 -0000 Apparently from your mail you dont need anything since your isp is making the nat. Sami =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A 7 =D7=91=D7=99=D7=95=D7=A0 2014 09:25,= "None Secure via freebsd-net" < freebsd-net@freebsd.org> =D7=9B=D7=AA=D7=91: > I've built a lot of gateways/routers with FreeBSD - but they have always > been with real, routable IPs. > > External IP is real, internal IP is real, and all I need is > gateway_enable=3D"yes" and a next-hop route from my ISP. > > No NAT, no divert, no ipfw rules, nothing. > > BUT, what if my ISP is giving me a private IP, and my internal network is > also private IPs ? External gateway address is 192.168.1.2 and internal > gateway address is 10.10.10.1 ... the ONLY way I could make this work is > with natd and ipfw divert rules. > > My question is: is it possible to have a network of non-routable IPs, an= d > a gateway with non-routable Ips on internal and external interfaces, and > NOT use natd/divert ? Can it be done with no ipfw rules at all, just lik= e > I used to ? > > Thanks. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 06:40:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51A8989E for ; Sat, 7 Jun 2014 06:40:56 +0000 (UTC) Received: from alogt.com (alogt.com [69.36.191.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2ACB021F7 for ; Sat, 7 Jun 2014 06:40:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=aT61qvFh2h+/vNpIKM5+plsqU/CzF0z9u4IRp+x5ggI=; b=wx9hkNuvM3ZV+k0EROYO/wSDq8ONxPQrXy6UZPdjgWFlr25gHWUofG6W4AO9oblFfjpkVxVPeeY7NL5/7aoIn3vvHSRccYGVeK7erAGz+VTWVeiYDayzmOWNOilm3hOgbQ7s8YYZOPRZHl6FS0lHExLvvS/Evje21T/P7ld92y0=; Received: from [182.14.146.137] (port=12631 helo=X220.alogt.com) by sl-508-2.slc.westdc.net with esmtpsa (SSLv3:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WtAJ6-003HTn-5j; Sat, 07 Jun 2014 00:40:48 -0600 Date: Sat, 7 Jun 2014 14:40:43 +0800 From: Erich Dollansky To: None Secure Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? Message-ID: <20140607144043.3d4be435@X220.alogt.com> In-Reply-To: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> Organization: ALO Green Technologies X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erich@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 06:40:56 -0000 Hi, On Fri, 6 Jun 2014 23:22:46 -0700 (PDT) None Secure via freebsd-net wrote: > BUT, what if my ISP is giving me a private IP, and my internal > network is also private IPs ? =A0External gateway address is > 192.168.1.2 and internal gateway address is 10.10.10.1 ... the ONLY > way I could make this work is with natd and ipfw divert rules. >=20 > My question is: =A0is it possible to have a network of non-routable > IPs, and a gateway with non-routable Ips on internal and external > interfaces, and NOT use natd/divert ? =A0Can it be done with no ipfw > rules at all, just like I used to ? >=20 what should be the problem? I did some time ago when the ISP gave us only a single IP address. The local machines connected to the gateway, the gateway connected via a second interface to the ISP. Of course, only the gateway was visible from outside. If you want to access the internal machines from outisde, you will need NAT.=20 Erich From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 09:35:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B621F385 for ; Sat, 7 Jun 2014 09:35:04 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "ca.infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5FCA72E00 for ; Sat, 7 Jun 2014 09:35:04 +0000 (UTC) Received: from seedling.local (seedling.black-earth.co.uk [81.2.117.99]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.8/8.14.8) with ESMTP id s579Ynr6055675 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sat, 7 Jun 2014 10:34:56 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: lucid-nonsense.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.8.3 smtp.infracaninophile.co.uk s579Ynr6055675 Authentication-Results: smtp.infracaninophile.co.uk/s579Ynr6055675; dkim=none reason="no signature"; dkim-adsp=none X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host seedling.black-earth.co.uk [81.2.117.99] claimed to be seedling.local Message-ID: <5392DCAF.8090302@FreeBSD.org> Date: Sat, 07 Jun 2014 10:34:39 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> In-Reply-To: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ShQTA79DVmQTirs9tTCF4u9HJPFGUB5Cu" X-Virus-Scanned: clamav-milter 0.98.3 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-3.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 09:35:04 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ShQTA79DVmQTirs9tTCF4u9HJPFGUB5Cu Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 07/06/2014 07:22, None Secure via freebsd-net wrote: > BUT, what if my ISP is giving me a private IP, and my internal > network is also private IPs ? External gateway address is > 192.168.1.2 and internal gateway address is 10.10.10.1 ... the ONLY > way I could make this work is with natd and ipfw divert rules. >=20 > My question is: is it possible to have a network of non-routable > IPs, and a gateway with non-routable Ips on internal and external > interfaces, and NOT use natd/divert ? Can it be done with no ipfw > rules at all, just like I used to ? Sure, it's possible, in theory. It just depends on whether your ISP's kit will NAT for your 10.10.10.1 range as well as the 192.168.1.2 address they've assigned to you. Which I doubt -- the ISP kit is probably only going to do the minimum necessary to provide service so that it can support the maximum possible number of customers. However, running your own NAT gateway between 192.168.1.2 and 10.10.10.1 shouldn't be a problem. You can NAT multiple times between where you are and the Internet usually with no worse consequence than a bit of extra latency on your traffic. Cheers, Matthew PS. Roll on IPv6. None of this Heath-Robinsonesq NAT on top of NAT is necessary in an IPv6 world. --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --ShQTA79DVmQTirs9tTCF4u9HJPFGUB5Cu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJTkty4XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATvPYP/1DbnGNbodSyDzg3DJxv+Qtp Ldry3WvH4n9GEn7hiRqibvXxd8ro6JMc5Bo2Y4tH1CZqmMWrGlZkVRb+/7KaX+OY ZpmfbDmAnnDLcnWGp6ulaPRv8Hat5zDzUy1uGr27A/qg2obRIWGUs1COzxkd+cMl e3h24FR4muy80QqxoViVAufUIjzbDoWOplAPMlV1LBvPl1X9l3B+mgiQrDlwTjWI 6CHpdKRekMRP9Tzs9N6kgWEkvmiaWWrF+Us/jfNaykji4Lm68318vsSp5RQ3fcuT hRYvnLVrXT3U/ozgFZa1xixs5oFC3Ng4YaYLnmpIgfcg7zEAfkj+atoIrvKtKSvz hJQK4pBVr7b+tO8NT6W6zPWnKEfe7zo1No/gSIEoZ71wf8UWiWXHXNhG1c3J8vT7 WGFvSpk5dGoFv3dS+KvPJyJNtvjaNquPM221fSuF5VB/OaZYi2AQzznGG7EuQCyW jIBznbIRNgJmC/sFW+3feyrnN3r5AQ/AEDGWnHszhRolo9BQ8mWkqY27K6BjP0rr l/cawuXoqcE2520xQBVEkuQ8x+5oU+fKNMMDqzHMTEhW0BTUuSBE15MdOxIFrtBx M6JP3uPO9kmOUk76gf9fC3LUBT+oMNL2ZD4cy+AAOpzb9/syN+MQEFdcFiMKSw1b 1mPig8BKmbFMKRZ8Eea+ =Zk6j -----END PGP SIGNATURE----- --ShQTA79DVmQTirs9tTCF4u9HJPFGUB5Cu-- From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 16:51:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C360A1E for ; Sat, 7 Jun 2014 16:51:38 +0000 (UTC) Received: from nm32.bullet.mail.ne1.yahoo.com (nm32.bullet.mail.ne1.yahoo.com [98.138.229.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 097CF2E00 for ; Sat, 7 Jun 2014 16:51:37 +0000 (UTC) Received: from [127.0.0.1] by nm32.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 16:51:31 -0000 Received: from [98.138.226.176] by nm32.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 16:48:40 -0000 Received: from [98.139.212.150] by tm11.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 16:48:39 -0000 Received: from [98.139.212.213] by tm7.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 16:48:39 -0000 Received: from [127.0.0.1] by omp1022.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 16:48:39 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 692139.15803.bm@omp1022.mail.bf1.yahoo.com Received: (qmail 55487 invoked by uid 60001); 7 Jun 2014 16:48:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1402159719; bh=BdSXr+3gCLZdMrY27cSgSNxM19tdGb8DnAUvHMGJRY8=; h=References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=lOMdIgplD2R3lsR4Dd9pCCvmiukF6I9mlXSDBlbjcfu8D2h2Vrh+W3CXHYBF6jy6ZRzsVTK7r02pLW37lLLYKWr18ZtfzheHsnu4jEUlrHOKLGqhraPxpFOkC6i8FpiJ1nvZNbc+rKu3rMNI44EhReAVjiy2XaxJjIpC3QMGZBk= X-YMail-OSG: BgMF.uQVM1lpDGBdh7TMFFa2SH3.CHtEE_zh3GaAQaUpG.M NzOh_D2DvR6bu5seu1J6atIIIRIg1f82EvN7DxgFaMd69EJbfK_9IrkuhvOZ c9XNGO4RN9475qe8rJ8AKS6zZtp6ELkIPuKnrIg3gy07HR7wSLX3TpsZWxOe _zoJvoCrXM4NtzjJwzBl_4Gjc3tYedgGsZimi5QQwWeel96ltuP9yCIwe9eJ rSAuwWs_cUcTcSvwgcaQVKN8GhEfUnc9jDTLzpmt1TvLnGibH055PWYqEyAe S0Qv3GssdNYz4gWo5c9gLBL4x2VR03C1O6bHDHtsTcZ5Uwjw8kvry74aPsag iEHwMMc0u1k.hKsc5vvydZ7SKkAlrGolhMow62kFEaM05YUA5UG542ZWAtZ3 rkYtLD2lMI6d4LffbIewuoqUb6BA3x_VwXhFpDmBYwsiH2UDxqCD1S8dEjz8 CtH77jEvsRJr389.O0GNYT6PAu42nOpeYoDXEiyTZzl5q3L54BQpSnm9IPJT mEyUf4F54fg-- Received: from [12.202.173.2] by web162105.mail.bf1.yahoo.com via HTTP; Sat, 07 Jun 2014 09:48:39 PDT X-Rocket-MIMEInfo: 002.001, WWVzLCBidXQgaW4gdGhpcyBjYXNlIEJPVEggSVBzIG9mIHRoZSBnYXRld2F5IC0gYm90aCB0aGUgZXh0ZXJuYWwgYW5kIHRoZSBpbnRlcm5hbCBpbnRlcmZhY2VzIC0gYXJlIG5vbi1yb3V0YWJsZSBJUHMsIGFuZCBzbyBpcyBteSBJU1AgY2FibGUgbW9kZW0uCgoxOTIuMTY4LjEuMSBpcyB0aGUgY2FibGUgbW9kZW0KMTkyLjE2OC4xLjIgaXMgZXh0ZXJuYWwgaW50ZXJmYWNlIG9mIG15IEZyZWVCU0QKMTAuMTAuMTAuMSBpcyBpbnRlcm5hbCBpbnRlcmZhY2Ugb2YgbXkgRnJlZUJTRAoKLi4uIGFuZCBteSBjbGkBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> <20140607144043.3d4be435@X220.alogt.com> Message-ID: <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> Date: Sat, 7 Jun 2014 09:48:39 -0700 (PDT) From: None Secure Reply-To: None Secure Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? To: Erich Dollansky In-Reply-To: <20140607144043.3d4be435@X220.alogt.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 16:51:38 -0000 Yes, but in this case BOTH IPs of the gateway - both the external and the i= nternal interfaces - are non-routable IPs, and so is my ISP cable modem.=0A= =0A192.168.1.1 is the cable modem=0A192.168.1.2 is external interface of my= FreeBSD=0A10.10.10.1 is internal interface of my FreeBSD=0A=0A... and my c= lient (10.10.10.2) could not get through to the outside world using just pl= ain old gateway_enable=3Dyes. =A0The configuration that always works with r= eal IPs did not work with this.=0A=0ASo, I followed the FreeBSD handbook wh= ich uses divert and natd, and it worked perfectly.=0A=0ANo, I am not trying= to access the internal systems from the outside world - I don't have a nee= d for that.=0A=0ABUT, I am wondering if it is any way possible to run a gat= eway like this *without* divert and natd ?=0A=0AThanks.=0A=0A=0AOn Friday, = June 6, 2014 11:40 PM, Erich Dollansky wrote:=0A =0A=0A= =0AHi,=0A=0A=0AOn Fri, 6 Jun 2014 23:22:46 -0700 (PDT)=0ANone Secure via fr= eebsd-net wrote:=0A=0A> BUT, what if my ISP is gi= ving me a private IP, and my internal=0A> network is also private IPs ? =A0= External gateway address is=0A> 192.168.1.2 and internal gateway address is= 10.10.10.1 ... the ONLY=0A> way I could make this work is with natd and ip= fw divert rules.=0A> =0A> My question is: =A0is it possible to have a netwo= rk of non-routable=0A> IPs, and a gateway with non-routable Ips on internal= and external=0A> interfaces, and NOT use natd/divert ? =A0Can it be done w= ith no ipfw=0A> rules at all, just like I used to ?=0A> =0Awhat should be t= he problem? I did some time ago when the ISP gave us=0Aonly a single IP add= ress. The local machines connected to the gateway,=0Athe gateway connected = via a second interface to the ISP.=0A=0AOf course, only the gateway was vis= ible from outside. If you want to=0Aaccess the internal machines from outis= de, you will need NAT. =0A=0AErich From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 16:56:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2245D2A for ; Sat, 7 Jun 2014 16:56:25 +0000 (UTC) Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CE322E49 for ; Sat, 7 Jun 2014 16:56:25 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id oy12so4841867veb.38 for ; Sat, 07 Jun 2014 09:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UfyZY5O4Q0Xn7C/B9wBPK4GFkaGQOVr9hYmf/+X2IJc=; b=gph9ebqg2FJd/nvwOxDg7VFN0QxlVBR8hEV8tpQufhhwUIvpKxozuEdjhKT4BMwmU3 sbcPQ+aKaoAPTc/36zby+FGAWJYKVW7HSDKl05IhwPOneH5RMaKz/cMDjn+mayTej268 KyJnFV+GRLEnPsMaAwR7RGeC2D9U3kZyTp2us4aAdILpQkHopSWbRmNavB1f2/HS61FD JhLjhXNtjOclRMSvviF7xX1w1W/oBr6mraby4CztdEw1EY0qCboCjxPGKDRQ9kltn8CI rjvE9IRMVh2AyZDBrxQ1gm6KXEb96WtlzAhCsOLkrTY+E50yRRsHo3EX+eOFv/GViEo/ zP5A== MIME-Version: 1.0 X-Received: by 10.58.160.134 with SMTP id xk6mr12108202veb.64.1402160184630; Sat, 07 Jun 2014 09:56:24 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.220.186.193 with HTTP; Sat, 7 Jun 2014 09:56:24 -0700 (PDT) In-Reply-To: <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> <20140607144043.3d4be435@X220.alogt.com> <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> Date: Sat, 7 Jun 2014 12:56:24 -0400 X-Google-Sender-Auth: rG7tNweuDxnYkuKVTEbFdy2rJHI Message-ID: Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? From: Adrian Chadd To: None Secure Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Erich Dollansky X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 16:56:25 -0000 On 7 June 2014 12:48, None Secure via freebsd-net wrote: > Yes, but in this case BOTH IPs of the gateway - both the external and the internal interfaces - are non-routable IPs, and so is my ISP cable modem. > > 192.168.1.1 is the cable modem > 192.168.1.2 is external interface of my FreeBSD > 10.10.10.1 is internal interface of my FreeBSD > > ... and my client (10.10.10.2) could not get through to the outside world using just plain old gateway_enable=yes. The configuration that always works with real IPs did not work with this. > > So, I followed the FreeBSD handbook which uses divert and natd, and it worked perfectly. > > No, I am not trying to access the internal systems from the outside world - I don't have a need for that. > > BUT, I am wondering if it is any way possible to run a gateway like this *without* divert and natd ? There's inkernel natd these days. There's also pf and ipfilter. -a From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 17:15:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49BFD8C0 for ; Sat, 7 Jun 2014 17:15:10 +0000 (UTC) Received: from nm46.bullet.mail.ne1.yahoo.com (nm46.bullet.mail.ne1.yahoo.com [98.138.120.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 097B72FEA for ; Sat, 7 Jun 2014 17:15:09 +0000 (UTC) Received: from [127.0.0.1] by nm46.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 17:15:08 -0000 Received: from [98.138.100.117] by nm46.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 17:12:14 -0000 Received: from [66.196.81.174] by tm108.bullet.mail.ne1.yahoo.com with NNFMP; 07 Jun 2014 17:12:14 -0000 Received: from [98.139.212.243] by tm20.bullet.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 17:12:14 -0000 Received: from [127.0.0.1] by omp1052.mail.bf1.yahoo.com with NNFMP; 07 Jun 2014 17:12:14 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 386414.70195.bm@omp1052.mail.bf1.yahoo.com Received: (qmail 5445 invoked by uid 60001); 7 Jun 2014 17:12:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1402161134; bh=vL0I5DOtsBkWGBDwVN/NzHvBlZ5nC96aC3mBPQ7x6Fs=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=tNazCxIMRvppnJSXacsiYQAkZ3T/hReSy+HcR8QS2a1U2AOAXr76ySSaCVzyTAZhMs2uRInUj4QyyM2X4k+tR+9/pAwR1BJge3Y92EBikQhfQXqwbJ+Z6/tP7oaVWhbZm2T//6x0AgMvuDD+d7fuFghbd3iFAstE/6alEwD/Csk= X-YMail-OSG: 4uQ0BJ4VM1nwmI4n6QNwNzi.aoihMk4quNkE9TA3UXg0GZW H1V2.GTzadzx7OMgBpi2s7C8AaheLvUj4xLJ1ERmgkgghoyRTzRaGIRlHblf ybfZRHFJzJRgqLRhlt1_CGeLRcgWvPWvO0IghQsJXvVbi9xFMVZqSqV0xeIA 9K0RctS7.uM5w9lRLD9JJXbbvstLAZgEQEZpiM4q_UQ_pYVnFCAuYrCs8zIW YUjxANrLKMaAC6knr_jvQ9ScRuM4Rne4Y0WK6n4R5..As17XYEfnC6es5CfU uY4l5PQbpzALEW9Ok6451x.8YH34kLIaHKdz9u9dDMjB43KNa1aI1mnQc.uC KLaQbqINhvyDeMzk8OeM30Xaec8UrPRcLY9xqF65pThItw93Ftp2kcPg_FKC xX1e1zV8PFEH6W3ARGbVwa_.8g..jbrbNdMuUqvdDfyMJcil3O1S0bSCSzFU FFYfrmi0sy_775SYqvPv6n4Fs9xMfZsr3bRhNj50FG32qYa07PpZUPySrNum K0uwMIlM- Received: from [12.202.173.2] by web162104.mail.bf1.yahoo.com via HTTP; Sat, 07 Jun 2014 10:12:14 PDT X-Rocket-MIMEInfo: 002.001, TWF0dGhldywKClRoYW5rcyBmb3IgeW91ciByZXNwb25zZSAtIEkgc3VzcGVjdCB0aGF0IHdhcyB0aGUgcHJvYmxlbSBJIHdhcyBlbmNvdW50ZXJpbmcgKHRoYXQgdGhlIElTUCB3aWxsIE5BVCBmb3IgbXkgZXh0ZXJuYWwgYWRkcmVzcykgYW5kIHRoYXQgaXMgd2h5IEkgc3dpdGNoZWQgdG8gbmF0ZC9kaXZlcnQsIGFuZCBpdCBpcyBpbmRlZWQgd29ya2luZyBwcm9wZXJseS4KClNvIHdoYXQgaXMgdGhlIHByb2JsZW0gPyDCoFdlbGwsIHRoZSBwcm9ibGVtIGlzIEkgYW0gdHJ5aW5nIHRvIHVzZSBzc2h1dHRsZSwBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 Message-ID: <1402161134.5132.YahooMailNeo@web162104.mail.bf1.yahoo.com> Date: Sat, 7 Jun 2014 10:12:14 -0700 (PDT) From: None Secure Reply-To: None Secure Subject: RE: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? To: "freebsd-net@freebsd.org" , "matthew@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 17:15:10 -0000 Matthew,=0A=0AThanks for your response - I suspect that was the problem I w= as encountering (that the ISP will NAT for my external address) and that is= why I switched to natd/divert, and it is indeed working properly.=0A=0ASo = what is the problem ? =A0Well, the problem is I am trying to use sshuttle, = which inserts it's own set of divert rules into the ipfw table ... so I hav= e one natd_enable, and a set of divert rules ... and then we add another se= t of divert rules from sshuttle (which does not, btw, start it's own natd).= =0A=0ASo when you say that I can NAT multiple times ... can I NAT multiple = times on the same system ? =A0If I start a second natd (which sounds ridicu= lous to me) how does it know which set of diverts it is supposed to work on= ?=0A=0ABasically my system is working fine with natd/divert, but now I eit= her need to make it work without natd/divert (so that sshuttle can do its o= wn) or I need to find a way to use two sets of natd/divert ...=0A=0AComment= s ? From owner-freebsd-net@FreeBSD.ORG Sat Jun 7 17:31:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 901D2D35; Sat, 7 Jun 2014 17:31:11 +0000 (UTC) Received: from oj.bangj.com (amt0.gin.ntt.net [129.250.11.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B19A2168; Sat, 7 Jun 2014 17:31:10 +0000 (UTC) Received: from [10.51.51.109] (unknown [166.170.43.133]) by oj.bangj.com (Postfix) with ESMTPA id 5BB735DD; Sat, 7 Jun 2014 13:21:25 -0400 (EDT) References: <1402161134.5132.YahooMailNeo@web162104.mail.bf1.yahoo.com> Mime-Version: 1.0 (1.0) In-Reply-To: <1402161134.5132.YahooMailNeo@web162104.mail.bf1.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <917ED0A1-774C-4A46-B5E0-A750E2DDF6C2@bangj.com> X-Mailer: iPhone Mail (12A4265u) From: Tom Pusateri Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? Date: Sat, 7 Jun 2014 10:21:24 -0700 To: None Secure Cc: "freebsd-net@freebsd.org" , "matthew@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 17:31:11 -0000 I've seen this setup with IPv4 before when the ISP does native IPv6. Maybe y= ou can get global IPv6 addresses and can SSH directly over that. If not, at l= east go on record requesting IPv6 with your provider to push them along. Tom > On Jun 7, 2014, at 10:12 AM, None Secure via freebsd-net wrote: >=20 > Matthew, >=20 > Thanks for your response - I suspect that was the problem I was encounteri= ng (that the ISP will NAT for my external address) and that is why I switche= d to natd/divert, and it is indeed working properly. >=20 > So what is the problem ? Well, the problem is I am trying to use sshuttle= , which inserts it's own set of divert rules into the ipfw table ... so I ha= ve one natd_enable, and a set of divert rules ... and then we add another se= t of divert rules from sshuttle (which does not, btw, start it's own natd). >=20 > So when you say that I can NAT multiple times ... can I NAT multiple times= on the same system ? If I start a second natd (which sounds ridiculous to m= e) how does it know which set of diverts it is supposed to work on ? >=20 > Basically my system is working fine with natd/divert, but now I either nee= d to make it work without natd/divert (so that sshuttle can do its own) or I= need to find a way to use two sets of natd/divert ... >=20 > Comments ? > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 00:24:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A4BBE02 for ; Sun, 8 Jun 2014 00:24:04 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F0A6520CF for ; Sun, 8 Jun 2014 00:24:03 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s580O0WA028314 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 7 Jun 2014 17:24:00 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s580NxRk028313; Sat, 7 Jun 2014 17:23:59 -0700 (PDT) (envelope-from jmg) Date: Sat, 7 Jun 2014 17:23:59 -0700 From: John-Mark Gurney To: None Secure Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? Message-ID: <20140608002359.GJ31367@funkthat.com> Mail-Followup-To: None Secure , Erich Dollansky , "freebsd-net@freebsd.org" References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> <20140607144043.3d4be435@X220.alogt.com> <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 07 Jun 2014 17:24:00 -0700 (PDT) Cc: "freebsd-net@freebsd.org" , Erich Dollansky X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 00:24:04 -0000 None Secure via freebsd-net wrote this message on Sat, Jun 07, 2014 at 09:48 -0700: > Yes, but in this case BOTH IPs of the gateway - both the external and the internal interfaces - are non-routable IPs, and so is my ISP cable modem. You keep saying non-routable IPs, but really, RFC1918 addresses are only not routed on the public internet... You can route these addresses all you want in your own network, and even between networks if you and the other network agree to route them... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 01:12:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61372494 for ; Sun, 8 Jun 2014 01:12:34 +0000 (UTC) Received: from alogt.com (alogt.com [69.36.191.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A4B423F6 for ; Sun, 8 Jun 2014 01:12:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=DKy0HKbrs+h7YnIOfGWCmK/8Ex+KmloQqf3yxvZ0qfQ=; b=PkW4B7Q/yfQc/niytMauGEz7EftT5k/eXmSct5OOlTu7SWRZzFuFnj+Z8E1lfcyiVBTOtfekyHbYHOrlm+064NM2a3AjfUQuKdLDy4znEnR2QFE2gnEgHuoB3Ew9G3fZLcKpHBM+D+jwFWPnmmF3qtye10Db8nJDIYDIVu6zIQw=; Received: from [39.193.232.236] (port=55197 helo=X220.alogt.com) by sl-508-2.slc.westdc.net with esmtpsa (SSLv3:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WtRey-001GhC-57; Sat, 07 Jun 2014 19:12:32 -0600 Date: Sun, 8 Jun 2014 09:12:26 +0800 From: Erich Dollansky To: None Secure Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? Message-ID: <20140608091226.3bb9fe60@X220.alogt.com> In-Reply-To: <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> <20140607144043.3d4be435@X220.alogt.com> <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 01:12:34 -0000 Hi, On Sat, 7 Jun 2014 09:48:39 -0700 (PDT) None Secure via freebsd-net wrote: > Yes, but in this case BOTH IPs of the gateway - both the external and > the internal interfaces - are non-routable IPs, and so is my ISP > cable modem. > > 192.168.1.1 is the cable modem > 192.168.1.2 is external interface of my FreeBSD > 10.10.10.1 is internal interface of my FreeBSD > I have had before the reverse situation. 10.x.x.x was the external address and 192.168.x.x the internal. There have been two interfaces in the machine. I cannot remember that I have had any problems with this. I have now an external router so I do not need to use FreeBSD for this. Erich From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 10:38:25 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4E57EFB for ; Sun, 8 Jun 2014 10:38:25 +0000 (UTC) Received: from smtpbg299.qq.com (smtpbg299.qq.com [184.105.67.99]) by mx1.freebsd.org (Postfix) with SMTP id BD83B2826 for ; Sun, 8 Jun 2014 10:38:25 +0000 (UTC) X-QQ-FEAT: 4vsdfhmGkDglPcvGQ47x3sS5jX4U6cKYI+Awq3m22MJAfToj/BFwykctTEZbD nenHmkN3Um7iPP8vVj19cnQscwKVUDRla21Zg9b78SQ1qT6FvvsUh6/oz1o5C1EO3HKXo0A M1W9s/LaUwmdH45+7LbkRYBHnHvFXFgJNFqWBp0= X-QQ-SSF: 000000000000000000000000000000P X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 113.206.240.32 X-QQ-STYLE: X-QQ-mid: webmail807t1402223894t5116745 From: "=?ISO-8859-1?B?bHllcnd0eXI=?=" To: "=?ISO-8859-1?B?bmV0?=" Sender: mmxcq@qq.com Subject: Hello, about whether netmap under e1000e cannot receive the package problem help look? Thank you Mime-Version: 1.0 Date: Sun, 8 Jun 2014 18:38:14 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-SENDSIZE: 520 X-QQ-FName: A8C589D971144EC4A3CB1C97E00B0E90 X-QQ-LocalIP: 58.250.134.100 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 10:38:25 -0000 THVpZ2kgUml6em8gSGVsbG8sIHRoYW5rIHlvdSBmb3IgeW91ciBzZWxmbGVzcyBjb250cmli dXRpb25zIG5ldG1hcCB0aGUgZmFzdCBwYWNrZXQgSS9PIGZyYW1ld29yaywNCg0KSSBoYXZl IGEgbGl0dGxlIHByb2JsZW0gbm93LCB0aGUgdGhpbmcgaXMgc3VjaCwgZG8gbm90IGtub3cg aWYgeW91IGNhbiBoZWxwIG1lIHRvIGhhdmUgYSBsb29rPyBUaGFuayB5b3UNCg0KSSAgZG8g dGhlIHRlc3QgaW4gdGhlIExpbnV4IGtlcm5lbCB2ZXJzaW9uIDMuMC44MCBiZWxvdywgSSBk aXNjb3ZlcmVkIHRoYXQgIG5ldG1hcCBjYW5ub3QgcmVjZWl2ZSBwYWNrZXRzLCBtYWNoaW5l IEkgaXMgdGhlIGZvbGxvd2luZw0KDQpJIHVzZSB0aGUgbHNtb2QgY29tbWFuZCBvdXRwdXQs IHRoYXQgSSBzaG91bGQgaGF2ZSBiZWVuIGxvYWRlZCB3aXRoIG5ldG1hcF9saW4ua28gYW5k IGUxMDAwZS5rbw0KDQoNCltyb290QGxvY2FsaG9zdCB+XSMgbHNtb2QNCk1vZHVsZSAgICAg ICAgICAgICAgICAgIFNpemUgIFVzZWQgYnkNCmUxMDAwZSAgICAgICAgICAgICAgICAgMjA5 OTYgIDAgDQpuZXRtYXBfbGluICAgICAgICAgICAgMjUzOTg0ICAxIGUxMDAwZQ0KDQoNCg0K DQoNClRoZW4gSSB1c2UgdGNwZHVtcCAtaSBldGgxIHRvIGNhcHR1cmUgY2FuIG5vcm1hbGx5 IHJlY2VpdmUgYWxsIHBhY2tldHMNCg0KQnV0IEkgdGNwZHVtcCAtaSBuZXRtYXA6ZXRoMQ0K DQovLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vDQoNCltyb290QGNl bnRvcyBleGFtcGxlc10jIHRjcGR1bXAgLWkgbmV0bWFwOmV0aDENCg0KUGNhcF9uZXRtYXBf YWN0aXZhdGUgZGV2aWNlIG5ldG1hcDpldGgxIHByaXYgMHgxYmFjNjAwIEZEIDMgcG9ydHMg MC4uMA0KDQpQY2FwX25ldG1hcF9pb2N0bCBldGgxIElPQ1RMIDB4ODkxMyByZXR1cm5zIDAN Cg0KUGNhcF9uZXRtYXBfaW9jdGwgZXRoMSBJT0NUTCAweDg5MTQgcmV0dXJucyAwDQoNClRj cGR1bXA6IHZlcmJvc2Ugb3V0cHV0IHN1cHByZXNzZWQsIHVzZSAtdiBvciAtdnYgZm9yIGZ1 bGwgcHJvdG9jb2wgZGVjb2RlDQoNCkxpc3RlbmluZyBvbiBuZXRtYXA6ZXRoMSwgbGluay10 eXBlIEVOMTBNQiAoRXRoZXJuZXQpLCBjYXB0dXJlIHNpemUgNjU1MzUgYnl0ZXMNCg0KMDg6 MjU6MjYuMTIyMzYxIElQIDE5Mi4xNjguMTg4LjIubmV0Ymlvcy1ucyA+IDE5Mi4xNjguMTg4 LjI1NS5uZXRiaW9zLW5zOiBOQlQgVURQIFBBQ0tFVCAoMTM3KTogUVVFUlk7IFJFUVVFU1Q7 IEJST0FEQ0FTVA0KDQowODoyNToyNi44NzE5MjMgSVAgMTkyLjE2OC4xODguMi5uZXRiaW9z LW5zID4gMTkyLjE2OC4xODguMjU1Lm5ldGJpb3MtbnM6IE5CVCBVRFAgUEFDS0VUICgxMzcp OiBRVUVSWTsgUkVRVUVTVDsgQlJPQURDQVNUDQoNCjA4OjI1OjI3LjYyMjQ4NCBJUCAxOTIu MTY4LjE4OC4yLm5ldGJpb3MtbnMgPiAxOTIuMTY4LjE4OC4yNTUubmV0Ymlvcy1uczogTkJU IFVEUCBQQUNLRVQgKDEzNyk6IFFVRVJZOyBSRVFVRVNUOyBCUk9BRENBU1QNCg0KTm90IHJl Y2VpdmluZyBhbnkgdXNlZnVsIHBhY2thZ2UsIEkgYW0gdmVyeSBjb25mdXNlZCBkbyBub3Qg a25vdyBpcyB3aGF0IHJlYXNvbg0KDQovLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vDQoNCkkgdXNlIHRoZSBmb2xsb3dpbmcgbmV0bWFwL2V4YW1wbGUg dG9vbHMgcGt0LWdlbiB0ZXN0IGRvZXMgbm90IHJlY2VpdmUgYWxsIHRoZSBwYWNrYWdlcyBi ZWxvdyBpcyBteSBvcGVyYXRpb24gcmVzdWx0cw0KDQpbcm9vdEBjZW50b3MgZXhhbXBsZXNd Iy4vcGt0LWdlbiAtaSBldGgxIC1mIFINCg0KMzAuNTI0MDI5IG1haW4gWzE2MjRdIGludGVy ZmFjZSBpcyBldGgxDQoNCjMwLjUyNDUyNyBleHRyYWN0X2lwX3JhbmdlIFsyNzVdIHJhbmdl IGlzIDEwLjAuMC4xOjAgdG8gMTAuMC4wLjE6MA0KDQozMC41MjQ1NTAgZXh0cmFjdF9pcF9y YW5nZSBbMjc1XSByYW5nZSBpcyAxMC4xLjAuMTowIHRvIDEwLjEuMC4xOjANCg0KMzAuNjc2 NzgyIG1haW4gWzE4MDddIG1hcHBlZCAzMzQ5ODBLQiBhdCAweDdmZWRlOWM5YjAwMA0KDQpS ZWNlaXZpbmcgZnJvbSBuZXRtYXA6ZXRoMTogMSBxdWV1ZXMsIDEgdGhyZWFkcyBhbmQgMSBj cHVzLg0KDQozMC42NzY4OTkgbWFpbiBbMTg4N10gV2FpdCAyIHNlY3MgZm9yIFBIWSByZXNl dA0KDQozMi42NzY5OTIgbWFpbiBbMTg4OV0gUmVhZHkuLi4NCg0KMzIuNjc3MDQ0IG5tX29w ZW4gWzQ1N10gb3ZlcnJpZGluZyBpZm5hbWUgZXRoMSByaW5naWQgMHgwIGZsYWdzIDB4MQ0K DQozMy42NzgxOTAgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBw YWNrZXRzLCBwb2xsIHJldHVybnMgMDANCg0KMzMuNjc4MjY1IG1haW5fdGhyZWFkIFsxNDIx XSAwIFBQUyAoMCBQS1RTIGluIDEwMDExMjEgVVNFQykNCg0KMzQuNjc5MjkyIHJlY2VpdmVy X2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5z IDAwDQoNCjM0LjY3OTQzNyBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAx MDAxMTczIFVTRUMpDQoNCjM1LjY4MDM0NyByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5n IGZvciBpbml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQozNS42ODA1OTUgbWFp bl90aHJlYWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTE1OCBVU0VDKQ0KDQozNi42 ODEzODYgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRz LCBwb2xsIHJldHVybnMgMDANCg0KMzYuNjgxNjM0IG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQ UyAoMCBQS1RTIGluIDEwMDEwMzggVVNFQykNCg0KMzcuNjgyNDY3IHJlY2VpdmVyX2JvZHkg WzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoN CjM3LjY4MjcwNyBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMDc0 IFVTRUMpDQoNCjM4LjY4MzUwMiByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBp bml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQozOC42ODM4NzAgbWFpbl90aHJl YWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTE2MyBVU0VDKQ0KDQozOS42ODQ1NTYg cmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xs IHJldHVybnMgMDANCg0KMzkuNjg1MDIyIG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQ S1RTIGluIDEwMDExNTIgVVNFQykNCg0KNDAuNjg1NTkxIHJlY2VpdmVyX2JvZHkgWzExNzBd IHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQwLjY4 NjA1NiBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMDM0IFVTRUMp DQoNCjQxLjY4NjYzMiByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFs IHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0MS42ODcxOTMgbWFpbl90aHJlYWQgWzE0 MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTEzNyBVU0VDKQ0KDQo0Mi42ODc2NjQgcmVjZWl2 ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVy bnMgMDANCg0KNDIuNjg4MzUzIG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGlu IDEwMDExNjAgVVNFQykNCg0KNDMuNjg4NzA3IHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRp bmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQzLjY4OTUwMiBt YWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMTQ5IFVTRUMpDQoNCjQ0 LjY4OTc1MSByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFsIHBhY2tl dHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0NC42OTA1NDkgbWFpbl90aHJlYWQgWzE0MjFdIDAg UFBTICgwIFBLVFMgaW4gMTAwMTA0NyBVU0VDKQ0KDQo0NS42OTA3ODQgcmVjZWl2ZXJfYm9k eSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVybnMgMDAN Cg0KNDUuNjkxNjQxIG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGluIDEwMDEw OTIgVVNFQykNCg0KNDYuNjkxODI1IHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9y IGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQ2LjY5MjgwNyBtYWluX3Ro cmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMTY2IFVTRUMpDQoNCjQ3LjY5Mjg1 NyByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFsIHBhY2tldHMsIHBv bGwgcmV0dXJucyAwMA0KDQo0Ny42OTM5NjIgbWFpbl90aHJlYWQgWzE0MjFdIDAgUFBTICgw IFBLVFMgaW4gMTAwMTE1NCBVU0VDKQ0KDQo0OC42OTM4OTIgcmVjZWl2ZXJfYm9keSBbMTE3 MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVybnMgMDANCg0KNDgu Njk0OTk2IG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGluIDEwMDEwMzUgVVNF QykNCg0KNDkuNjk0OTI2IHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRp YWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQ5LjY5NjEzMiBtYWluX3RocmVhZCBb MTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMTM1IFVTRUMpDQoNCjUwLjY5NTk1OCByZWNl aXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFsIHBhY2tldHMsIHBvbGwgcmV0 dXJucyAwMA0KDQo1MC42OTcyOTUgbWFpbl90aHJlYWQgWzE0MjFdIDAgUFBTICgwIFBLVFMg aW4gMTAwMTE2NCBVU0VDKQ0KDQo1MS42OTY5OTcgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2Fp dGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVybnMgMDANCg0KNTEuNjk4NDUx IG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGluIDEwMDExNTUgVVNFQykNCg0K NTIuNjk4MDQ2IHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRpDQoNCi8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLw0KDQpJIGRvbid0 IGtub3cgSXMgaXQgcmlnaHQ/IEUxMDAwZSB0aGUgZHJpdmVyIHdoZXJlIEkgZGlkbid0IHVz ZSB0byBiYWcgaXMgbm90IG5vcm1hbA0KDQpCdXQgSSdtIGFsc28gYSBtYWNoaW5lIHRoaXMg bWFjaGluZSBpcyBpbnN0YWxsZWQgYWJvdmUgdGhlIGRyaXZlciBpcyBFMTAwMCBjZW50b3M2 LjIgTGludXgga2VybmVsIHZlcnNpb24gaXMgMy4wLjgwDQoNCkluIHRoaXMgbWFjaGluZSBh Ym92ZSBJIGNhbiB1c2UgdGhlIG5vcm1hbCBuZXRtYXAgYmFnLg0KDQpUaGFuayB5b3UsIHNp bmNlcmVseSB3YWl0aW5nIGZvciB5b3VyIHJlcGx5LCB0aGFuayB5b3U= From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 11:17:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A52F7650 for ; Sun, 8 Jun 2014 11:17:08 +0000 (UTC) Received: from 1und1.siccegge.de (1und1.siccegge.de [IPv6:2001:8d8:898:b600::9d:1585]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 602182B03 for ; Sun, 8 Jun 2014 11:17:08 +0000 (UTC) Received: from [2001:470:74e1:2000:4a5b:39ff:fec9:ec73] (helo=hepworth) by 1und1.siccegge.de with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1Wtb5j-00019v-LL; Sun, 08 Jun 2014 13:17:05 +0200 Date: Sun, 8 Jun 2014 13:19:49 +0200 From: Christoph Egger To: freebsd-net@freebsd.org Message-ID: <20140608111949.GA21787@anonymous.siccegge.de> References: <5B8E8711-24A5-46AD-893A-12C087117672@hellmuth-michaelis.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/9DWx/yDrRhgMJTb" Content-Disposition: inline In-Reply-To: <5B8E8711-24A5-46AD-893A-12C087117672@hellmuth-michaelis.de> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: 2001:470:74e1:2000:4a5b:39ff:fec9:ec73 X-SA-Exim-Mail-From: christoph@christoph-egger.org X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on 1und1.siccegge.de X-Spam-Level: X-Spam-Status: No, score=-2.9 required=10.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 Subject: Re: pf and ipv6 pmtu discovery not working X-SA-Exim-Version: 4.2.1 (built Mon, 26 Dec 2011 16:24:06 +0000) X-SA-Exim-Scanned: Yes (on 1und1.siccegge.de) Cc: Hellmuth Michaelis X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 11:17:08 -0000 --/9DWx/yDrRhgMJTb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi! On Thu, May 01, 2014 at 02:52:47PM +0200, Hellmuth Michaelis wrote: > for some time now i'm trying to nail down a phenomenon of hanging ipv6 > transfers, it looks to me like there is a problem of getting ipv6 path=20 > mtu discovery right when using FreeBSD and pf. > > When the "set skip on fxp0" line in pf.conf is removed, the above > described pmtu scenario does not work anymore: I'm suffering from the same (probably) problem. Has there been any progress towards fixing this? Christoph --=20 9FED 5C6C E206 B70A 5857 70CA 9655 22B9 D49A E731 Debian Developer | Lisp Hacker | CaCert Assurer --/9DWx/yDrRhgMJTb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJTlEbVAAoJEKv/7bJACMb5B5QP/jgleiDfp+5SnnP7HkStbaM3 zGEyq4Fpqhk0ZQD2nPw0L1UgoObij/4QUmOqDYT/dAQQ8cLXosMo51+xlgnlmHwz IpfHBayT0A9+sZ2Ysfo1ZmwIeUOCfnJHtE1vkwDwh1RI/eyjfAeRwQ+7qI109rV5 z3Cn9jo3lxLD/stoVGihCsbG3IcUPDMhGh8fFv+ABV86skdhVuCRoE91Lm1AOtyr Eauckok9PvRqBphd1iM6n8rd24PLQUg60mx5ulbZg7zJxEBkMegzwrhPiH0ObcJ5 kyUPZOI7PHFuzYvrmDpW5Z8OqmHOSTNdRR9wH8Up/XFKGjaDILqiIez9zX/0oSuL ERT3JSDZVd/LCq0eSUxP4EoTCbaDHVS/Png4ryIIyYZWCJB4x7gO9XjH15Bxofr6 ync2+h3rm8+rJHzr7nxKeAUvraBVvKAmUZvjxYjyDmgdficqN8P0Zh/NB+VeTb8u GNFD8fBtCHiHvCmqGRYM6vWYPdYtgThNI9u598hvlLMaqkKS3VLKah3eIhzpwfqW HFzr9xEpOe0KSaQdb1rCiIWBol1liKVjr7OBA72ZtlQOR/YD6cIzBBC0aRMffZWi tiy3fyv0RcUkQo6ZhffaWBzZMaGNmMa/UlsXQP976LW+9KZcqn94DEltnmBxBEIl prItfvyWNFqdDRjBqpL8 =Zc5L -----END PGP SIGNATURE----- --/9DWx/yDrRhgMJTb-- From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 11:40:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97297C96 for ; Sun, 8 Jun 2014 11:40:38 +0000 (UTC) Received: from mail-oa0-x235.google.com (mail-oa0-x235.google.com [IPv6:2607:f8b0:4003:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E4062D07 for ; Sun, 8 Jun 2014 11:40:38 +0000 (UTC) Received: by mail-oa0-f53.google.com with SMTP id m1so4767786oag.40 for ; Sun, 08 Jun 2014 04:40:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GuV0JqsdvxVWvmZ2QyB9vimP2Owt5drKe2Ae8Z066YQ=; b=gBLoSCmGkmS17rsDNA1EN+1TvUWor3U3V6f0xwIc8erXANjnofxjODDXfd2DSYTDUe d52wAoUfkgjAxFwVbJnvbaqb1KDiV5k2mgCzFzoSUiiDfckC4aTtb6eJiZWD6vgMavaU 1QLEKaKnRghon7FC2FG8SQSujqo9UJqwraFtQQ2lZ7O6Ta27etf8xz2Gpe7qiF7gR5pw iUHP+kkmjYMYFJTF/MjS6PnPUhwaDATfLWkiFRN+h8kkbQXcuZovIk/1RWIcxxuZt6ze PWodyyGiZEu0qHh/M4HAweGnIq0zgh/oonYgevmo6Nn5U3h9qSgPOSXI7Jy4tBDS0gJI UjNg== MIME-Version: 1.0 X-Received: by 10.60.73.129 with SMTP id l1mr1566865oev.2.1402227637663; Sun, 08 Jun 2014 04:40:37 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Sun, 8 Jun 2014 04:40:37 -0700 (PDT) In-Reply-To: <20140608091226.3bb9fe60@X220.alogt.com> References: <1402122166.37214.YahooMailNeo@web162101.mail.bf1.yahoo.com> <20140607144043.3d4be435@X220.alogt.com> <1402159719.88183.YahooMailNeo@web162105.mail.bf1.yahoo.com> <20140608091226.3bb9fe60@X220.alogt.com> Date: Sun, 8 Jun 2014 13:40:37 +0200 Message-ID: Subject: Re: Can you create a FreeBSD gateway, with private IPs, without NAT/divert ? From: Andreas Nilsson To: Erich Dollansky Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , None Secure X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 11:40:38 -0000 On Sun, Jun 8, 2014 at 3:12 AM, Erich Dollansky wrote: > Hi, > > On Sat, 7 Jun 2014 09:48:39 -0700 (PDT) > None Secure via freebsd-net wrote: > > > Yes, but in this case BOTH IPs of the gateway - both the external and > > the internal interfaces - are non-routable IPs, and so is my ISP > > cable modem. > > > > 192.168.1.1 is the cable modem > > 192.168.1.2 is external interface of my FreeBSD > > 10.10.10.1 is internal interface of my FreeBSD > > > I have had before the reverse situation. 10.x.x.x was the external > address and 192.168.x.x the internal. There have been two interfaces in > the machine. I cannot remember that I have had any problems with this. > I have now an external router so I do not need to use FreeBSD for this. > > Erich > It's sad that they choose CGN in the first place :( But now that they have, couldn't you ask them to give you a /24 subnet of that rfc1918 block? Then you wouldn't really need a gateway at all, just a firewall (which should block arp and so on. And as stated by others: routing/forwarding table really has no notion about human "constraints" on IP addresses, kernel will happily forward traffic from one rfc1918 block to another. That being said, your ISP is running nat, from 192.168.1.1 to some public ip. If the ISP sees traffic from 10.10.10.0/24 ( just guessing about the /24), they will be confused, which is why you need to run nat to translate those internal addresses to your external IP. As for ipfw, could you show us the rules that get added? If it uses divert you definitely should be able to send to different natd processes, as divert sends the traffic to a specified port. If you want to use in-kernel nat it might be a bit trickier, but what if you the process in question in a jail? Then you could easily distinguish it in ipfw. Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Sun Jun 8 14:07:44 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 588D4B81 for ; Sun, 8 Jun 2014 14:07:44 +0000 (UTC) Received: from smtpbg299.qq.com (smtpbg299.qq.com [184.105.67.99]) by mx1.freebsd.org (Postfix) with SMTP id 3DC95285B for ; Sun, 8 Jun 2014 14:07:43 +0000 (UTC) X-QQ-FEAT: 9iO8kzCY5wyMYNeRB11NE352Eqk3f/R2tuhc4UuD//3AUFGOzkev9xmaY+D/S PjOC/mVApbyGYP9v+Q1v5Ajb/0Nyi7YQub5SzUsnhV2aoRolCYVczoT7qWxVo5yRIV75rlW XBGguADqz/pvbJstBwAZPwbOJmHBOd8KQR+un/w= X-QQ-SSF: 000000000000000000000000000000P X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 113.206.206.195 In-Reply-To: References: X-QQ-STYLE: X-QQ-mid: webmail807t1402236459t9645618 From: "=?ISO-8859-1?B?bHllcnd0eXI=?=" To: "=?ISO-8859-1?B?bmV0?=" Sender: mmxcq@qq.com Subject: Re:Hello, about whether netmap under e1000e cannot receive the package problem help look? Thank you Mime-Version: 1.0 Date: Sun, 8 Jun 2014 22:07:39 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-ReplyHash: 2870259204 X-QQ-SENDSIZE: 520 X-QQ-FName: FF736776232344EAA5EEE286AD15C800 X-QQ-LocalIP: 58.250.132.20 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 14:07:44 -0000 SGVsbG8sIEkgc29sdmVkLCB0aGUgb3JpZ2luYWwgbmV0bWFwIGNhbiBub3QgdGFrZSB0aGUg aW5pdGlhdGl2ZSB0byBzZXQgdGhlIHByb21pc2N1b3VzLCB0aGFuayB5b3UNCg0KSSBmb3Vu ZCBhIHByb2JsZW0gdGhhdCBJIHVzZWQgTGlicGNhcCBjYXB0dXJlLCBsaWtlIENQVSBsb2Fk IGlzIG5vdCBoaWdoLA0KDQpCdXQgSSB1c2UgbmV0bWFwK2xpYnBjYXAgQ1BVIGxvYWQgaXMg dmVyeSBoaWdoLCBpcyB0aGlzIG5vcm1hbD8gVGhhbmsgeW91DQoNCg0KDQoNCi0tLS0tLS0t LS0tLS0tLS0tLSBPcmlnaW5hbCAtLS0tLS0tLS0tLS0tLS0tLS0NCkZyb206ICAibHllcnd0 eXIiOzxseWVyd3R5ckBnbWFpbC5jb20+Ow0KRGF0ZTogIFN1biwgSnVuIDgsIDIwMTQgMDY6 MzggUE0NClRvOiAgIm5ldCI8bmV0QGZyZWVic2Qub3JnPjsgDQoNClN1YmplY3Q6ICBIZWxs bywgYWJvdXQgd2hldGhlciBuZXRtYXAgdW5kZXIgZTEwMDBlIGNhbm5vdCByZWNlaXZlIHRo ZSBwYWNrYWdlIHByb2JsZW0gaGVscCBsb29rPyBUaGFuayB5b3UNCg0KDQoNCkx1aWdpIFJp enpvIEhlbGxvLCB0aGFuayB5b3UgZm9yIHlvdXIgc2VsZmxlc3MgY29udHJpYnV0aW9ucyBu ZXRtYXAgdGhlIGZhc3QgcGFja2V0IEkvTyBmcmFtZXdvcmssDQoNCkkgaGF2ZSBhIGxpdHRs ZSBwcm9ibGVtIG5vdywgdGhlIHRoaW5nIGlzIHN1Y2gsIGRvIG5vdCBrbm93IGlmIHlvdSBj YW4gaGVscCBtZSB0byBoYXZlIGEgbG9vaz8gVGhhbmsgeW91DQoNCkkgIGRvIHRoZSB0ZXN0 IGluIHRoZSBMaW51eCBrZXJuZWwgdmVyc2lvbiAzLjAuODAgYmVsb3csIEkgZGlzY292ZXJl ZCB0aGF0ICBuZXRtYXAgY2Fubm90IHJlY2VpdmUgcGFja2V0cywgbWFjaGluZSBJIGlzIHRo ZSBmb2xsb3dpbmcNCg0KSSB1c2UgdGhlIGxzbW9kIGNvbW1hbmQgb3V0cHV0LCB0aGF0IEkg c2hvdWxkIGhhdmUgYmVlbiBsb2FkZWQgd2l0aCBuZXRtYXBfbGluLmtvIGFuZCBlMTAwMGUu a28NCg0KDQpbcm9vdEBsb2NhbGhvc3Qgfl0jIGxzbW9kDQpNb2R1bGUgICAgICAgICAgICAg ICAgICBTaXplICBVc2VkIGJ5DQplMTAwMGUgICAgICAgICAgICAgICAgIDIwOTk2ICAwIA0K bmV0bWFwX2xpbiAgICAgICAgICAgIDI1Mzk4NCAgMSBlMTAwMGUNCg0KDQoNCg0KDQpUaGVu IEkgdXNlIHRjcGR1bXAgLWkgZXRoMSB0byBjYXB0dXJlIGNhbiBub3JtYWxseSByZWNlaXZl IGFsbCBwYWNrZXRzDQoNCkJ1dCBJIHRjcGR1bXAgLWkgbmV0bWFwOmV0aDENCg0KLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLw0KDQpbcm9vdEBjZW50b3MgZXhh bXBsZXNdIyB0Y3BkdW1wIC1pIG5ldG1hcDpldGgxDQoNClBjYXBfbmV0bWFwX2FjdGl2YXRl IGRldmljZSBuZXRtYXA6ZXRoMSBwcml2IDB4MWJhYzYwMCBGRCAzIHBvcnRzIDAuLjANCg0K UGNhcF9uZXRtYXBfaW9jdGwgZXRoMSBJT0NUTCAweDg5MTMgcmV0dXJucyAwDQoNClBjYXBf bmV0bWFwX2lvY3RsIGV0aDEgSU9DVEwgMHg4OTE0IHJldHVybnMgMA0KDQpUY3BkdW1wOiB2 ZXJib3NlIG91dHB1dCBzdXBwcmVzc2VkLCB1c2UgLXYgb3IgLXZ2IGZvciBmdWxsIHByb3Rv Y29sIGRlY29kZQ0KDQpMaXN0ZW5pbmcgb24gbmV0bWFwOmV0aDEsIGxpbmstdHlwZSBFTjEw TUIgKEV0aGVybmV0KSwgY2FwdHVyZSBzaXplIDY1NTM1IGJ5dGVzDQoNCjA4OjI1OjI2LjEy MjM2MSBJUCAxOTIuMTY4LjE4OC4yLm5ldGJpb3MtbnMgPiAxOTIuMTY4LjE4OC4yNTUubmV0 Ymlvcy1uczogTkJUIFVEUCBQQUNLRVQgKDEzNyk6IFFVRVJZOyBSRVFVRVNUOyBCUk9BRENB U1QNCg0KMDg6MjU6MjYuODcxOTIzIElQIDE5Mi4xNjguMTg4LjIubmV0Ymlvcy1ucyA+IDE5 Mi4xNjguMTg4LjI1NS5uZXRiaW9zLW5zOiBOQlQgVURQIFBBQ0tFVCAoMTM3KTogUVVFUlk7 IFJFUVVFU1Q7IEJST0FEQ0FTVA0KDQowODoyNToyNy42MjI0ODQgSVAgMTkyLjE2OC4xODgu Mi5uZXRiaW9zLW5zID4gMTkyLjE2OC4xODguMjU1Lm5ldGJpb3MtbnM6IE5CVCBVRFAgUEFD S0VUICgxMzcpOiBRVUVSWTsgUkVRVUVTVDsgQlJPQURDQVNUDQoNCk5vdCByZWNlaXZpbmcg YW55IHVzZWZ1bCBwYWNrYWdlLCBJIGFtIHZlcnkgY29uZnVzZWQgZG8gbm90IGtub3cgaXMg d2hhdCByZWFzb24NCg0KLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLw0KDQpJIHVzZSB0aGUgZm9sbG93aW5nIG5ldG1hcC9leGFtcGxlIHRvb2xzIHBr dC1nZW4gdGVzdCBkb2VzIG5vdCByZWNlaXZlIGFsbCB0aGUgcGFja2FnZXMgYmVsb3cgaXMg bXkgb3BlcmF0aW9uIHJlc3VsdHMNCg0KW3Jvb3RAY2VudG9zIGV4YW1wbGVzXSMuL3BrdC1n ZW4gLWkgZXRoMSAtZiBSDQoNCjMwLjUyNDAyOSBtYWluIFsxNjI0XSBpbnRlcmZhY2UgaXMg ZXRoMQ0KDQozMC41MjQ1MjcgZXh0cmFjdF9pcF9yYW5nZSBbMjc1XSByYW5nZSBpcyAxMC4w LjAuMTowIHRvIDEwLjAuMC4xOjANCg0KMzAuNTI0NTUwIGV4dHJhY3RfaXBfcmFuZ2UgWzI3 NV0gcmFuZ2UgaXMgMTAuMS4wLjE6MCB0byAxMC4xLjAuMTowDQoNCjMwLjY3Njc4MiBtYWlu IFsxODA3XSBtYXBwZWQgMzM0OTgwS0IgYXQgMHg3ZmVkZTljOWIwMDANCg0KUmVjZWl2aW5n IGZyb20gbmV0bWFwOmV0aDE6IDEgcXVldWVzLCAxIHRocmVhZHMgYW5kIDEgY3B1cy4NCg0K MzAuNjc2ODk5IG1haW4gWzE4ODddIFdhaXQgMiBzZWNzIGZvciBQSFkgcmVzZXQNCg0KMzIu Njc2OTkyIG1haW4gWzE4ODldIFJlYWR5Li4uDQoNCjMyLjY3NzA0NCBubV9vcGVuIFs0NTdd IG92ZXJyaWRpbmcgaWZuYW1lIGV0aDEgcmluZ2lkIDB4MCBmbGFncyAweDENCg0KMzMuNjc4 MTkwIHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywg cG9sbCByZXR1cm5zIDAwDQoNCjMzLjY3ODI2NSBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMg KDAgUEtUUyBpbiAxMDAxMTIxIFVTRUMpDQoNCjM0LjY3OTI5MiByZWNlaXZlcl9ib2R5IFsx MTcwXSB3YWl0aW5nIGZvciBpbml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQoz NC42Nzk0MzcgbWFpbl90aHJlYWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTE3MyBV U0VDKQ0KDQozNS42ODAzNDcgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5p dGlhbCBwYWNrZXRzLCBwb2xsIHJldHVybnMgMDANCg0KMzUuNjgwNTk1IG1haW5fdGhyZWFk IFsxNDIxXSAwIFBQUyAoMCBQS1RTIGluIDEwMDExNTggVVNFQykNCg0KMzYuNjgxMzg2IHJl Y2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCBy ZXR1cm5zIDAwDQoNCjM2LjY4MTYzNCBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtU UyBpbiAxMDAxMDM4IFVTRUMpDQoNCjM3LjY4MjQ2NyByZWNlaXZlcl9ib2R5IFsxMTcwXSB3 YWl0aW5nIGZvciBpbml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQozNy42ODI3 MDcgbWFpbl90aHJlYWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTA3NCBVU0VDKQ0K DQozOC42ODM1MDIgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBw YWNrZXRzLCBwb2xsIHJldHVybnMgMDANCg0KMzguNjgzODcwIG1haW5fdGhyZWFkIFsxNDIx XSAwIFBQUyAoMCBQS1RTIGluIDEwMDExNjMgVVNFQykNCg0KMzkuNjg0NTU2IHJlY2VpdmVy X2JvZHkgWzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5z IDAwDQoNCjM5LjY4NTAyMiBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAx MDAxMTUyIFVTRUMpDQoNCjQwLjY4NTU5MSByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5n IGZvciBpbml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0MC42ODYwNTYgbWFp bl90aHJlYWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTAzNCBVU0VDKQ0KDQo0MS42 ODY2MzIgcmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRz LCBwb2xsIHJldHVybnMgMDANCg0KNDEuNjg3MTkzIG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQ UyAoMCBQS1RTIGluIDEwMDExMzcgVVNFQykNCg0KNDIuNjg3NjY0IHJlY2VpdmVyX2JvZHkg WzExNzBdIHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoN CjQyLjY4ODM1MyBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMTYw IFVTRUMpDQoNCjQzLjY4ODcwNyByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBp bml0aWFsIHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0My42ODk1MDIgbWFpbl90aHJl YWQgWzE0MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTE0OSBVU0VDKQ0KDQo0NC42ODk3NTEg cmVjZWl2ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xs IHJldHVybnMgMDANCg0KNDQuNjkwNTQ5IG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQ S1RTIGluIDEwMDEwNDcgVVNFQykNCg0KNDUuNjkwNzg0IHJlY2VpdmVyX2JvZHkgWzExNzBd IHdhaXRpbmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQ1LjY5 MTY0MSBtYWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMDkyIFVTRUMp DQoNCjQ2LjY5MTgyNSByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFs IHBhY2tldHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0Ni42OTI4MDcgbWFpbl90aHJlYWQgWzE0 MjFdIDAgUFBTICgwIFBLVFMgaW4gMTAwMTE2NiBVU0VDKQ0KDQo0Ny42OTI4NTcgcmVjZWl2 ZXJfYm9keSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVy bnMgMDANCg0KNDcuNjkzOTYyIG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGlu IDEwMDExNTQgVVNFQykNCg0KNDguNjkzODkyIHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRp bmcgZm9yIGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjQ4LjY5NDk5NiBt YWluX3RocmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMDM1IFVTRUMpDQoNCjQ5 LjY5NDkyNiByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aWFsIHBhY2tl dHMsIHBvbGwgcmV0dXJucyAwMA0KDQo0OS42OTYxMzIgbWFpbl90aHJlYWQgWzE0MjFdIDAg UFBTICgwIFBLVFMgaW4gMTAwMTEzNSBVU0VDKQ0KDQo1MC42OTU5NTggcmVjZWl2ZXJfYm9k eSBbMTE3MF0gd2FpdGluZyBmb3IgaW5pdGlhbCBwYWNrZXRzLCBwb2xsIHJldHVybnMgMDAN Cg0KNTAuNjk3Mjk1IG1haW5fdGhyZWFkIFsxNDIxXSAwIFBQUyAoMCBQS1RTIGluIDEwMDEx NjQgVVNFQykNCg0KNTEuNjk2OTk3IHJlY2VpdmVyX2JvZHkgWzExNzBdIHdhaXRpbmcgZm9y IGluaXRpYWwgcGFja2V0cywgcG9sbCByZXR1cm5zIDAwDQoNCjUxLjY5ODQ1MSBtYWluX3Ro cmVhZCBbMTQyMV0gMCBQUFMgKDAgUEtUUyBpbiAxMDAxMTU1IFVTRUMpDQoNCjUyLjY5ODA0 NiByZWNlaXZlcl9ib2R5IFsxMTcwXSB3YWl0aW5nIGZvciBpbml0aQ0KDQovLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8v Ly8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8NCg0KSSBkb24ndCBrbm93IElz IGl0IHJpZ2h0PyBFMTAwMGUgdGhlIGRyaXZlciB3aGVyZSBJIGRpZG4ndCB1c2UgdG8gYmFn IGlzIG5vdCBub3JtYWwNCg0KQnV0IEknbSBhbHNvIGEgbWFjaGluZSB0aGlzIG1hY2hpbmUg aXMgaW5zdGFsbGVkIGFib3ZlIHRoZSBkcml2ZXIgaXMgRTEwMDAgY2VudG9zNi4yIExpbnV4 IGtlcm5lbCB2ZXJzaW9uIGlzIDMuMC44MA0KDQpJbiB0aGlzIG1hY2hpbmUgYWJvdmUgSSBj YW4gdXNlIHRoZSBub3JtYWwgbmV0bWFwIGJhZy4NCg0KVGhhbmsgeW91LCBzaW5jZXJlbHkg d2FpdGluZyBmb3IgeW91ciByZXBseSwgdGhhbmsgeW91 From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 04:07:11 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F73CBC9 for ; Mon, 9 Jun 2014 04:07:11 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 416182740 for ; Mon, 9 Jun 2014 04:07:11 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id j107so8277895qga.20 for ; Sun, 08 Jun 2014 21:07:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=MkyFvEffeWWXmNtRr8yDYHg3ZlKnCCnpnhvtCWFors0=; b=Z9XkqPGnIBydIrJy9+zqmHSGiFVa0gqI1fOTIBEdzY7yEyhBGGK7f9daQlqTVLMdjy /i66ih42KUNdRfTPa/lY/D7znUHwnNUMIYKIvFLUs+BnDB/V8f9T9fs3URAGXW6YzdJ1 2G+FlietX2ja8kdqvKvGj5YSFavkcNGzuEM+EzX9569yIB6fWiPDzAm4xXUMagCfTVS3 VRzFUz2gvCdML9VQKCVuOh4cUQ0c7sOVDT4MtxHM0g/UssCC/lB1nsQQen+oyD1445q2 nVUIy8X/0CYiJ4nnEUyMLGOYqZ+Ob/2Q3mqOUdEcQ1Z7ktM3HiiWuC3EBEAGrOlnoaGm QnlQ== MIME-Version: 1.0 X-Received: by 10.140.22.209 with SMTP id 75mr27439695qgn.4.1402286830084; Sun, 08 Jun 2014 21:07:10 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Sun, 8 Jun 2014 21:07:10 -0700 (PDT) In-Reply-To: References: <533BF902.7010506@sfc.wide.ad.jp> Date: Sun, 8 Jun 2014 21:07:10 -0700 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: hiren panchasara To: "net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 04:07:11 -0000 On Sun, Jun 1, 2014 at 12:31 AM, hiren panchasara wrote: > On Tue, Apr 8, 2014 at 9:14 PM, hiren panchasara > wrote: >> On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd wrote: >>> Hi! Cool! can you file a FreeBSD PR with this? >> >> I'm testing this patch right now. >> >> I will make sure it doesn't get lost. :-) > > Committed as r266941. I wish to MFC this change and following actual dctcp changes (which I'll commit to -head in coming weeks) to 10. Let me know if there is any objection to it. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 08:32:09 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9632D4E for ; Mon, 9 Jun 2014 08:32:09 +0000 (UTC) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 668862B0A for ; Mon, 9 Jun 2014 08:32:09 +0000 (UTC) Received: from [10.64.24.24] (dc1-prod.netflix.com [69.53.236.251]) by lauren.room52.net (Postfix) with ESMTPSA id 370FE7E824; Mon, 9 Jun 2014 18:31:59 +1000 (EST) Message-ID: <539570F8.2010105@freebsd.org> Date: Mon, 09 Jun 2014 01:31:52 -0700 From: Lawrence Stewart User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: hiren panchasara Subject: Re: ECN marking implenetation for dummynet References: <533BF902.7010506@sfc.wide.ad.jp> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 08:32:09 -0000 On 6/8/2014 9:07 PM, hiren panchasara wrote: > On Sun, Jun 1, 2014 at 12:31 AM, hiren panchasara > wrote: >> On Tue, Apr 8, 2014 at 9:14 PM, hiren panchasara >> wrote: >>> On Tue, Apr 8, 2014 at 8:46 PM, Adrian Chadd wrote: >>>> Hi! Cool! can you file a FreeBSD PR with this? >>> >>> I'm testing this patch right now. >>> >>> I will make sure it doesn't get lost. :-) >> >> Committed as r266941. > > I wish to MFC this change and following actual dctcp changes (which > I'll commit to -head in coming weeks) to 10. > > Let me know if there is any objection to it. Please don't MFC the DCTCP stuff until I've had a chance to consider the KPI changes. Prodding me to do so is likely a good idea. Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 08:47:46 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAA744A0; Mon, 9 Jun 2014 08:47:46 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B67F2C9B; Mon, 9 Jun 2014 08:47:46 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i17so338995qcy.36 for ; Mon, 09 Jun 2014 01:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ei7snnPSd+ah/nW61+h1ntiUAIp8ijfv8336fuorI4M=; b=c0f/kqeZi7loz/MCQ+3q/C3imW5vBAaBLixtcbnWonJSuKnX8ioXJZJjSGICEzbDXO OWWknz8IflahSNMNeygPGQ/rl5kYdwWnntoSbRlmloR0uGkRf2H5sXfrKrm0vVejXD+l jBVM6JwX1PUefWb0YD7t54MQKhngCJAoJ8KceMP1JD2R9sDVkAG7Va6H9NDn2i79ary0 b26bQ8tkl7UL6MQYdWHobOc2AixfFdfm7SNoL7qdT8vsWSUjo3Aadx5DcBa1toJyIG+X mmgTo09kEh7cK9B15KxfIu71hx1PrImnkCnzDZYVvbYoJDcpQvMwdpg0yzkYbp1DCjFN 0+tw== MIME-Version: 1.0 X-Received: by 10.140.35.212 with SMTP id n78mr29096233qgn.87.1402303665540; Mon, 09 Jun 2014 01:47:45 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Mon, 9 Jun 2014 01:47:45 -0700 (PDT) In-Reply-To: <539570F8.2010105@freebsd.org> References: <533BF902.7010506@sfc.wide.ad.jp> <539570F8.2010105@freebsd.org> Date: Mon, 9 Jun 2014 01:47:45 -0700 Message-ID: Subject: Re: ECN marking implenetation for dummynet From: hiren panchasara To: Lawrence Stewart Content-Type: text/plain; charset=UTF-8 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 08:47:46 -0000 On Mon, Jun 9, 2014 at 1:31 AM, Lawrence Stewart wrote: > On 6/8/2014 9:07 PM, hiren panchasara wrote: >> I wish to MFC this change and following actual dctcp changes (which >> I'll commit to -head in coming weeks) to 10. >> >> Let me know if there is any objection to it. > > > Please don't MFC the DCTCP stuff until I've had a chance to consider the KPI > changes. Prodding me to do so is likely a good idea. I am in no rush to MFC. But I didn't quite get what you are talking about wrt KPI changes. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 10:08:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FC3E570 for ; Mon, 9 Jun 2014 10:08:55 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C36E323E4 for ; Mon, 9 Jun 2014 10:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=hXDxkRQrHZj/enlHHqQfandVqnhcmAFZwJu8cDBCVcU=; b=iHOS9ZOMenwgfDCKYFp4gSVNHtGG3xu+pVEvy6RKCIK4GPRTsVYApbXFsr9jZrif0surecWJlL8Jf1TLdH6NYiXeTDVhCbAzv3FCy8lYndlO+JGIdVbTwkvLoRjwwqBuFerCx4QTwearaD9BQOSxNOvco4UhU0hlysqOU7ZTJ9bx7v9C54NXbJ3Z+zNGnpeNMNP2RvBnIW/jCkyXHd/UkqwZHs07Bp3m12pXk88QRjcsB4zg59DNIcBkkw36/IGHKCbfcV5+lcdtar/Rmva3nPsEFrIkNrROCVNZq3OSD5BxYJCks7yV0xgBvSEbyl7RuB+UHT+jbIShKbDeUMLcKzrH+a3GaNkWjYmcTBZmq0cvvwgnJcAQ+vZS+0zmsClyvaLvkcsXdFLH3o7gJP5OKp4hH0Xa50mJyzn1hd9HIMEeND6m1aCal62oVWe8vKN6tP9tvsNu/ZW7YgA1gc7Y8tTCuUo3G/Q27NunC6k1+6drbL5fD9R43GnH7T4Ok1vWZwuRtjA5tBomt0/N9M0oQxRH14TArQhxeckHmwT53HkmzsJP+pBkEOd3IufCDYF5KtdVNBnuo4Nzdj55aHqxoAUsRn6BFp7mL0Zk9/65WcWBv9z6KafCOnADSv19dAbuxDiEyAxlZ8HXXxV4TyIYA0pD8ZYVPOHq8j3hL9xhO2c=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WtwVP-000A7I-8A; Mon, 09 Jun 2014 14:08:43 +0400 Date: Mon, 9 Jun 2014 14:08:41 +0400 From: Eygene Ryabinkin To: None Secure Subject: Re: Does FreeBSD have the ability to properly forward UDP traffic ? Message-ID: References: <1402122333.57974.YahooMailNeo@web162101.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="X0vpKvTpCy87tk9a" Content-Disposition: inline In-Reply-To: <1402122333.57974.YahooMailNeo@web162101.mail.bf1.yahoo.com> Sender: rea@codelabs.ru Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 10:08:55 -0000 --X0vpKvTpCy87tk9a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Good day. Fri, Jun 06, 2014 at 11:25:33PM -0700, None Secure via freebsd-net wrote: > I would like very much to use sshuttle for an informal VPN. >=20 > However, sshuttle sets up a lot of complexity in order to route DNS > requests over the ssh tunnel ... it uses divert rules for dns > traffic, and I don't think they even tested it because it fails to > start or utilize natd. sshuttle doesn't need natd, divert(4) sockets are in use. It is the mechanism that is used to forward packets from, say, ipfw to the userland and back to ipfw. You have divert rule in ipfw that puts a packet to the divert(4) socket that is identified by it's numeric port (that has nothing to do with the TCP/UDP ports, it is just an identified for different divert(4) listeners). User-space application listent to this socket and can read and write packets, thus consuming input ones and generating output ones. Sshuttle listens to the divert socket: {{{ if dnsport: divertsock =3D socket.socket(socket.AF_INET, socket.SOCK_RAW, IPPROTO_DIVERT) divertsock.bind(('0.0.0.0', port)) # IP field is ignored nslist =3D resolvconf_nameservers() for ip in nslist: # relabel and then catch outgoing DNS requests ipfw('add', sport, 'divert', sport, 'udp', 'from', 'any', 'to', '%s/32' % ip, '53', 'not', 'ipttl', '42') # relabel DNS responses ipfw('add', sport, 'divert', sport, 'udp', 'from', 'any', str(dnsport), 'to', 'any', 'not', 'ipttl', '42') def do_wait(): while 1: r,w,x =3D select.select([sys.stdin, divertsock], [], []) if divertsock in r: _handle_diversion(divertsock, dnsport) if sys.stdin in r: return else: do_wait =3D None return do_wait def _handle_diversion(divertsock, dnsport): p,tag =3D divertsock.recvfrom(4096) src,dst =3D _udp_unpack(p) debug3('got diverted packet from %r to %r\n' % (src, dst)) if dst[1] =3D=3D 53: # outgoing DNS debug3('...packet is a DNS request.\n') _real_dns_server[0] =3D dst dst =3D ('127.0.0.1', dnsport) elif src[1] =3D=3D dnsport: if islocal(src[0]): debug3('...packet is a DNS response.\n') src =3D _real_dns_server[0] else: log('weird?! unexpected divert from %r to %r\n' % (src, dst)) assert(0) newp =3D _udp_repack(p, src, dst) divertsock.sendto(newp, tag) }}} so it doesn't need natd at all. > The stated reason by sshuttle project is that you can't just forward > UDP traffic properly with BSD, like you can with linux - they say it > doesn't keep track of port numbers or connections properly. Are you referring to the following comment from https://github.com/apenwarr/sshuttle/blob/master/firewall.py {{{ # This part is much crazier than it is on Linux, because MacOS (at least # 10.6, and probably other versions, and maybe FreeBSD too) doesn't # correctly fixup the dstip/dstport for UDP packets when it puts them # through a 'fwd' rule. It also doesn't fixup the srcip/srcport in the # response packet. In Linux iptables, all that happens magically for us, # so we just redirect the packets and relax. # # On MacOS, we have to fix the ports ourselves. For that, we use a # 'divert' socket, which receives raw packets and lets us mangle them. # # Here's how it works. Let's say the local DNS server is 1.1.1.1:53, # and the remote DNS server is 2.2.2.2:53, and the local transproxy port # is 10.0.0.1:12300, and a client machine is making a request from # 10.0.0.5:9999. We see a packet like this: # 10.0.0.5:9999 -> 1.1.1.1:53 # Since the destip:port matches one of our local nameservers, it will # match a 'fwd' rule, thus grabbing it on the local machine. However, # the local kernel will then see a packet addressed to *:53 and # not know what to do with it; there's nobody listening on port 53. Thu= s, # we divert it, rewriting it into this: # 10.0.0.5:9999 -> 10.0.0.1:12300 # This gets proxied out to the server, which sends it to 2.2.2.2:53, # and the answer comes back, and the proxy sends it back out like this: # 10.0.0.1:12300 -> 10.0.0.5:9999 # But that's wrong! The original machine expected an answer from # 1.1.1.1:53, so we have to divert the *answer* and rewrite it: # 1.1.1.1:53 -> 10.0.0.5:9999 # # See? Easy stuff. }}} ipfw(8) 'fwd' rule isn't touching the internals of the packet, it really just forwards it "as is". This has nothing to do with UDP, 'fwd' is just intended to be used for transparent proxying, not for "real" forwarding that implies rewriting packet contents. sshuttle folks say about UDP just because of the following property of SSH tunnelling: when you're doing it, you have something that listens to localhost:N on TCP. So you can forward packets to localhost:N without any particular problems, they will be consumed by the proper application (SSH) and be forwarded to the remote end. But for DNS packets sshuttle does the following: it just forwards them to the local host and some local port where sshuttle's own DNS proxy lives. So, in reality, there is no forwarding or tunnelling of DNS/UDP requests in the normal sense: sshuttle wants to proxy them via itself. And for this it has to mangle the packets via divert sockets on FreeBSD. What I don't quite understand is that why sshuttle needs this dance of diverting and can't just consume incoming DNS packet like SSH's tunneling port will, reinterepret it and put the answer back with the proper src/dst fields inside the UDP packet. But probably there's some explanation: I'll try to think of it later. > Or is it possible to properly forward UDP traffic with ipfw rules, > and not use natd/divert ? What is your real problem with sshuttle? If you concerned that no natd(8) is spawned, then you shouldn't worry about that. If something doesn't work for you, please, be specific, what doesn't work, which FreeBSD and sshuttle versions you're using and what is your configuration. Hope that helps. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --X0vpKvTpCy87tk9a Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOVh6lfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Pu5WAD/RzMECcPfreYGH2fzllTJanwU kWX8BqBW3G03ScSIx/gA/3eSXHbjDCMzpfTVcAugSPtYpJLrhdrlyWE1TMsDYH0I =JlhY -----END PGP SIGNATURE----- --X0vpKvTpCy87tk9a-- From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 14:18:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 808ECE1E for ; Mon, 9 Jun 2014 14:18:46 +0000 (UTC) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 496E528D9 for ; Mon, 9 Jun 2014 14:18:46 +0000 (UTC) Received: by mail-ob0-f171.google.com with SMTP id wn1so5982365obc.16 for ; Mon, 09 Jun 2014 07:18:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=8b8iE1epzcUeNL0OUbeE9264AxIFaaAtiRzhxeR2xos=; b=AhthiJRqpEFCLExMHALfqYBG1pqRoHT95MaJ2BWfd0YaKzS1p7SoXgS9c2w22Z7A56 aHmkp/y+gQLzOERh+4fyLPti2p9B4MBnmtI+do0COKNu90BOU9Q9O+5BGSDiP5o78aFN iXGQd9Qp24DqquShkGtpEE8uJclmbL4ErxHe5R+eMcbFFMNi2CuVog2XaA4clnPh7/QC dzFdzBDden/7gMuC8po/RhNvPGMMFv8hU8xHwGIoof4vd3t/i4YcoADxitqbaGRgaM1a xrSS70OKkGzyc9jqdHNkdc5IO2zDBABuBFhd6e4CClaoHM+8byShFmf+qqsVy/C2pcMC eayA== MIME-Version: 1.0 X-Received: by 10.60.55.169 with SMTP id t9mr2559880oep.84.1402323525574; Mon, 09 Jun 2014 07:18:45 -0700 (PDT) Received: by 10.76.101.18 with HTTP; Mon, 9 Jun 2014 07:18:45 -0700 (PDT) Date: Mon, 9 Jun 2014 22:18:45 +0800 Message-ID: Subject: how to capture packets when using netmap? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 14:18:46 -0000 Hi all I'm using FreeBSD 10 release and with "device netmap" added (recompile kernel to use netmap) but it seems didn't support libpcap (I use tcpdump to capture when netmap-bridge is on and 2 hosts ping each other | topo: host1 --- [em0] FreeBSD [em1] --- host2 ) I know there is a netmap-libpcap, but I don't know how to generate it to BSD system... this is my try: *1> download source code, extract* *2> cd src dir* *3> configure* root@switch:/usr/src/tools/tools/netmap/netmap-libpcap # ./configure checking build system type... x86_64-unknown-freebsd10.0 checking host system type... x86_64-unknown-freebsd10.0 checking target system type... x86_64-unknown-freebsd10.0 checking for gcc... no checking for cc... cc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether cc accepts -g... yes checking for cc option to accept ISO C89... none needed checking for inline... inline checking for __attribute__... yes checking whether __attribute__((unused)) can be used without warnings... yes checking whether __attribute__((format)) can be used without warnings... yes checking how to run the C preprocessor... cc -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking sys/bitypes.h usability... no checking sys/bitypes.h presence... no checking for sys/bitypes.h... no checking for int8_t... yes checking for u_int8_t... yes checking for int16_t... yes checking for u_int16_t... yes checking for int32_t... yes checking for u_int32_t... yes checking for int64_t... yes checking for u_int64_t... yes checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... no checking for _LARGEFILE_SOURCE value needed for large files... no checking sys/ioccom.h usability... yes checking sys/ioccom.h presence... yes checking for sys/ioccom.h... yes checking sys/sockio.h usability... yes checking sys/sockio.h presence... yes checking for sys/sockio.h... yes checking limits.h usability... yes checking limits.h presence... yes checking for limits.h... yes checking paths.h usability... yes checking paths.h presence... yes checking for paths.h... yes checking linux/types.h usability... no checking linux/types.h presence... no checking for linux/types.h... no checking linux/if_packet.h usability... no checking linux/if_packet.h presence... no checking for linux/if_packet.h... no checking netpacket/packet.h usability... no checking netpacket/packet.h presence... no checking for netpacket/packet.h... no checking netpacket/if_packet.h usability... no checking netpacket/if_packet.h presence... no checking for netpacket/if_packet.h... no checking for net/pfvar.h... yes checking whether net/pfvar.h defines PF_NAT through PF_NORDR... yes checking for netinet/if_ether.h... no configure: Rechecking with some additional includes checking for netinet/if_ether.h... yes checking for ANSI ioctl definitions... yes checking for strerror... yes checking for strlcpy... yes checking for vsnprintf... yes checking for snprintf... yes checking for library containing gethostbyname... none required checking for library containing socket... none required checking for library containing putmsg... no checking for ether_hostton... yes checking whether ether_hostton is declared... yes checking if --disable-protochain option is specified... enabled checking packet capture type... bpf checking net/if_media.h usability... yes checking net/if_media.h presence... yes checking for net/if_media.h... yes checking whether the system supports zerocopy BPF... yes checking for struct BPF_TIMEVAL... no checking for getifaddrs... yes checking ifaddrs.h usability... yes checking ifaddrs.h presence... yes checking for ifaddrs.h... yes checking for socklen_t... yes checking for getaddrinfo... yes checking whether to build optimizer debugging code... no checking whether to build parser debugging code... no checking whether we have DAG API headers... no (/usr/local/include) checking whether we have the DAG API... no checking whether we have Septel API... no checking whether we have Myricom Sniffer API... no (/opt/snf) checking for flex... flex checking for flex 2.4 or higher... yes checking for bison... no configure: WARNING: don't have both flex and bison; reverting to lex/yacc checking for capable lex... yes checking for ranlib... ranlib checking for ar... ar checking whether ln -s works... yes checking if sockaddr struct has the sa_len member... yes checking if sockaddr_storage struct exists... yes checking if dl_hp_ppa_info_t struct has dl_module_id_1 member... no checking if unaligned accesses fail... no checking for USB sniffing support... no checking whether the platform could support netfilter sniffing... no checking for net/netmap_user.h... yes configure: netmap is supported configure: no Bluetooth sniffing support implemented for freebsd10.0 configure: no canusb support implemented for freebsd10.0 configure: no CAN sniffing support implemented for freebsd10.0 checking for pkg-config... no configure: no hardware timestamp support implemented for freebsd10.0 checking for a BSD-compatible install... /usr/bin/install -c configure: creating ./config.status config.status: creating Makefile config.status: creating pcap-filter.manmisc config.status: creating pcap-linktype.manmisc config.status: creating pcap-tstamp.manmisc config.status: creating pcap-savefile.manfile config.status: creating pcap.3pcap config.status: creating pcap_compile.3pcap config.status: creating pcap_datalink.3pcap config.status: creating pcap_dump_open.3pcap config.status: creating pcap_get_tstamp_precision.3pcap config.status: creating pcap_list_datalinks.3pcap config.status: creating pcap_list_tstamp_types.3pcap config.status: creating pcap_open_dead.3pcap config.status: creating pcap_open_offline.3pcap config.status: creating pcap_set_tstamp_precision.3pcap config.status: creating pcap_set_tstamp_type.3pcap config.status: creating config.h config.status: executing default-1 commands *4>make* root@switch:/usr/src/tools/tools/netmap/netmap-libpcap # make cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-bpf.c cc -fpic -I. -DHAVE_CONFIG_H -D_U_="__attribute__((unused))" -g -O2 -c ./pcap-netmap.c ./pcap-netmap.c:117:9: warning: implicit declaration of function 'nm_dispatch' is invalid in C99 [-Wimplicit-function-declaration] ret = nm_dispatch((void *)d, cnt, (void *)pcap_netmap_fi... ^ ./pcap-netmap.c:131:9: warning: implicit declaration of function 'nm_inject' is invalid in C99 [-Wimplicit-function-declaration] return nm_inject(d, buf, size); ^ ./pcap-netmap.c:139:15: error: variable has incomplete type 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:139:9: note: forward declaration of 'struct ifreq' struct ifreq ifr; ^ ./pcap-netmap.c:140:19: error: incomplete definition of type 'struct nm_desc' int error, fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:152:7: error: use of undeclared identifier 'SIOCSIFFLAGS' case SIOCSIFFLAGS: ^ ./pcap-netmap.c:157:10: warning: implicit declaration of function 'ioctl' is invalid in C99 [-Wimplicit-function-declaration] error = ioctl(fd, what, &ifr); ^ ./pcap-netmap.c:159:4: error: incomplete definition of type 'struct nm_desc' d->req.nr_name, what, error); ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:163:7: error: use of undeclared identifier 'SIOCGIFFLAGS' case SIOCGIFFLAGS: ^ ./pcap-netmap.c:177:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:178:18: error: use of undeclared identifier 'IFF_PPROMISC' if (if_flags & IFF_PPROMISC) { ^ ./pcap-netmap.c:179:17: error: use of undeclared identifier 'IFF_PPROMISC' if_flags &= ~IFF_PPROMISC; ^ ./pcap-netmap.c:180:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ ./pcap-netmap.c:183:2: warning: implicit declaration of function 'nm_close' is invalid in C99 [-Wimplicit-function-declaration] nm_close(d); ^ ./pcap-netmap.c:195:22: warning: implicit declaration of function 'nm_open' is invalid in C99 [-Wimplicit-function-declaration] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ./pcap-netmap.c:195:18: warning: incompatible integer to pointer conversion initializing 'struct nm_desc *' with an expression of type 'int' [-Wint-conversion] struct nm_desc *d = nm_open(p->opt.source, NULL, 0, NULL); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./pcap-netmap.c:210:36: error: incomplete definition of type 'struct nm_desc' __FUNCTION__, p->opt.source, d, d->fd, ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:213:11: error: incomplete definition of type 'struct nm_desc' p->fd = d->fd; ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:27: error: incomplete definition of type 'struct nm_desc' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ~^ ./pcap-netmap.c:71:9: note: forward declaration of 'struct nm_desc' struct nm_desc *d; /* pointer returned by nm_open() */ ^ ./pcap-netmap.c:214:45: error: use of undeclared identifier 'NETMAP_SW_RING' if (p->opt.promisc && !(d->req.nr_ringid & NETMAP_SW_RING)) { ^ ./pcap-netmap.c:215:24: error: use of undeclared identifier 'SIOCGIFFLAGS' pcap_netmap_ioctl(p, SIOCGIFFLAGS, &if_flags); /* fetch flags */ ^ ./pcap-netmap.c:216:20: error: use of undeclared identifier 'IFF_PPROMISC' if (!(if_flags & IFF_PPROMISC)) { ^ ./pcap-netmap.c:218:16: error: use of undeclared identifier 'IFF_PPROMISC' if_flags |= IFF_PPROMISC; ^ ./pcap-netmap.c:219:25: error: use of undeclared identifier 'SIOCSIFFLAGS' pcap_netmap_ioctl(p, SIOCSIFFLAGS, &if_flags); ^ 6 warnings and 17 errors generated. *** Error code 1 Stop. make: stopped in /usr/src/tools/tools/netmap/netmap-libpcap and I've no idea to fix the problem...could anybody give me advice? From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 21:25:13 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D8A93D3 for ; Mon, 9 Jun 2014 21:25:13 +0000 (UTC) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id DF259245D for ; Mon, 9 Jun 2014 21:25:12 +0000 (UTC) Received: from lgwl-lstewart2.corp.netflix.com (unknown [69.53.237.72]) by lauren.room52.net (Postfix) with ESMTPSA id 3CE017E8C2; Tue, 10 Jun 2014 07:25:10 +1000 (EST) Message-ID: <53953725.7060604@freebsd.org> Date: Mon, 09 Jun 2014 14:25:09 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: hiren panchasara Subject: Re: ECN marking implenetation for dummynet References: <533BF902.7010506@sfc.wide.ad.jp> <539570F8.2010105@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.8 required=5.0 tests=DATE_IN_PAST_12_24, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 21:25:13 -0000 On 06/09/14 18:47, hiren panchasara wrote: > On Mon, Jun 9, 2014 at 1:31 AM, Lawrence Stewart wrote: >> On 6/8/2014 9:07 PM, hiren panchasara wrote: > >>> I wish to MFC this change and following actual dctcp changes (which >>> I'll commit to -head in coming weeks) to 10. >>> >>> Let me know if there is any objection to it. >> >> >> Please don't MFC the DCTCP stuff until I've had a chance to consider the KPI >> changes. Prodding me to do so is likely a good idea. > > I am in no rush to MFC. But I didn't quite get what you are talking > about wrt KPI changes. Unless Midori's DCTCP work has changed significantly since I first reviewed it in 2013, it changes the modular congestion control KPI. I'd like to review the work again but don't want to delay it getting into head because I'm super busy. Just asking that it not be MFCed before I've given it a review. Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 00:02:54 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 750183F6; Tue, 10 Jun 2014 00:02:54 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 52A3C20EA; Tue, 10 Jun 2014 00:02:53 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5A02kIq066075 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 9 Jun 2014 17:02:46 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5A02kjk066074; Mon, 9 Jun 2014 17:02:46 -0700 (PDT) (envelope-from jmg) Date: Mon, 9 Jun 2014 17:02:46 -0700 From: John-Mark Gurney To: current@FreeBSD.org, net@FreeBSD.org Subject: dhclient sucks cpu usage... Message-ID: <20140610000246.GW31367@funkthat.com> Mail-Followup-To: current@FreeBSD.org, net@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 09 Jun 2014 17:02:46 -0700 (PDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 00:02:54 -0000 So, after finding out that nc has a stupidly small buffer size (2k even though there is space for 16k), I was still not getting as good as performance using nc between machines, so I decided to generate some flame graphs to try to identify issues... (Thanks to who included a full set of modules, including dtraceall on memstick!) So, the first one is: https://www.funkthat.com/~jmg/em.stack.svg As I was browsing around, the em_handle_que was consuming quite a bit of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows me that the taskqueue for em was consuming about 50% cpu... Also pretty high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump or anything, but I think dhclient uses bpf to be able to inject packets and listen in on them, so I kill off dhclient, and instantly, the taskqueue thread for em drops down to 40% CPU... (transfer rate only marginally improves, if it does) I decide to run another flame graph w/o dhclient running: https://www.funkthat.com/~jmg/em.stack.nodhclient.svg and now _rxeof drops from 17.22% to 11.94%, pretty significant... So, if you care about performance, don't run dhclient... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 03:13:38 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6AE097F; Tue, 10 Jun 2014 03:13:38 +0000 (UTC) Received: from torment.daemoninthecloset.org (torment.daemoninthecloset.org [94.242.209.234]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "torment.daemoninthecloset.org", Issuer "daemoninthecloset.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 648EE2F9E; Tue, 10 Jun 2014 03:13:38 +0000 (UTC) Received: from sage.daemoninthecloset.org (cpe-72-177-8-109.austin.res.rr.com [72.177.8.109]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "sage.daemoninthecloset.org", Issuer "daemoninthecloset.org" (not verified)) by torment.daemoninthecloset.org (Postfix) with ESMTPS id 03F6F42C2546; Tue, 10 Jun 2014 05:04:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at daemoninthecloset.org X-Virus-Scanned: amavisd-new at daemoninthecloset.org Date: Mon, 9 Jun 2014 22:03:56 -0500 (CDT) From: Bryan Venteicher To: John-Mark Gurney Message-ID: <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> In-Reply-To: <20140610000246.GW31367@funkthat.com> References: <20140610000246.GW31367@funkthat.com> Subject: Re: dhclient sucks cpu usage... MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.10.20] X-Mailer: Zimbra 8.0.2_GA_5569 (ZimbraWebClient - GC33 ([unknown])/8.0.2_GA_5569) Thread-Topic: dhclient sucks cpu usage... Thread-Index: fmMcJNl9h8hOdeWB79fYswL50E8uIA== Cc: current@FreeBSD.org, net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 03:13:38 -0000 Hi, ----- Original Message ----- > So, after finding out that nc has a stupidly small buffer size (2k > even though there is space for 16k), I was still not getting as good > as performance using nc between machines, so I decided to generate some > flame graphs to try to identify issues... (Thanks to who included a > full set of modules, including dtraceall on memstick!) > > So, the first one is: > https://www.funkthat.com/~jmg/em.stack.svg > > As I was browsing around, the em_handle_que was consuming quite a bit > of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows > me that the taskqueue for em was consuming about 50% cpu... Also pretty > high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is > consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump > or anything, but I think dhclient uses bpf to be able to inject packets > and listen in on them, so I kill off dhclient, and instantly, the taskqueue > thread for em drops down to 40% CPU... (transfer rate only marginally > improves, if it does) > > I decide to run another flame graph w/o dhclient running: > https://www.funkthat.com/~jmg/em.stack.nodhclient.svg > > and now _rxeof drops from 17.22% to 11.94%, pretty significant... > > So, if you care about performance, don't run dhclient... > Yes, I've noticed the same issue. It can absolutely kill performance in a VM guest. It is much more pronounced on only some of my systems, and I hadn't tracked it down yet. I wonder if this is fallout from the callout work, or if there was some bpf change. I've been using the kludgey workaround patch below. diff --git a/sys/net/bpf.c b/sys/net/bpf.c index cb3ed27..9751986 100644 --- a/sys/net/bpf.c +++ b/sys/net/bpf.c @@ -2013,9 +2013,11 @@ bpf_gettime(struct bintime *bt, int tstype, struct mbuf *m) return (BPF_TSTAMP_EXTERN); } } +#if 0 if (quality == BPF_TSTAMP_NORMAL) binuptime(bt); else +#endif getbinuptime(bt); return (quality); > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 09:19:52 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88F9F442; Tue, 10 Jun 2014 09:19:52 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4AB3F2CD8; Tue, 10 Jun 2014 09:19:52 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WuEIY-0004ZN-DK; Tue, 10 Jun 2014 09:08:38 +0400 Message-ID: <5396CD41.2080300@FreeBSD.org> Date: Tue, 10 Jun 2014 13:17:53 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Bryan Venteicher , John-Mark Gurney Subject: Re: dhclient sucks cpu usage... References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> In-Reply-To: <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: current@FreeBSD.org, net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 09:19:52 -0000 On 10.06.2014 07:03, Bryan Venteicher wrote: > Hi, > > ----- Original Message ----- >> So, after finding out that nc has a stupidly small buffer size (2k >> even though there is space for 16k), I was still not getting as good >> as performance using nc between machines, so I decided to generate some >> flame graphs to try to identify issues... (Thanks to who included a >> full set of modules, including dtraceall on memstick!) >> >> So, the first one is: >> https://www.funkthat.com/~jmg/em.stack.svg >> >> As I was browsing around, the em_handle_que was consuming quite a bit >> of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows >> me that the taskqueue for em was consuming about 50% cpu... Also pretty >> high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is >> consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump >> or anything, but I think dhclient uses bpf to be able to inject packets >> and listen in on them, so I kill off dhclient, and instantly, the taskqueue >> thread for em drops down to 40% CPU... (transfer rate only marginally >> improves, if it does) >> >> I decide to run another flame graph w/o dhclient running: >> https://www.funkthat.com/~jmg/em.stack.nodhclient.svg >> >> and now _rxeof drops from 17.22% to 11.94%, pretty significant... >> >> So, if you care about performance, don't run dhclient... >> > Yes, I've noticed the same issue. It can absolutely kill performance > in a VM guest. It is much more pronounced on only some of my systems, > and I hadn't tracked it down yet. I wonder if this is fallout from > the callout work, or if there was some bpf change. > > I've been using the kludgey workaround patch below. Hm, pretty interesting. dhclient should setup proper filter (and it looks like it does so: 13:10 [0] m@ptichko s netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient ) see "match" count. And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for each consumer on interface). It should not introduce significant performance penalties. > > diff --git a/sys/net/bpf.c b/sys/net/bpf.c > index cb3ed27..9751986 100644 > --- a/sys/net/bpf.c > +++ b/sys/net/bpf.c > @@ -2013,9 +2013,11 @@ bpf_gettime(struct bintime *bt, int tstype, struct mbuf *m) > return (BPF_TSTAMP_EXTERN); > } > } > +#if 0 > if (quality == BPF_TSTAMP_NORMAL) > binuptime(bt); > else > +#endif bpf_getttime() is called IFF packet filter matches some traffic. Can you show your "netstat -B" output ? > getbinuptime(bt); > > return (quality); > > >> -- >> John-Mark Gurney Voice: +1 415 225 5579 >> >> "All that I will do, has been done, All that I have, has not." >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 13:53:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6136FAB8 for ; Tue, 10 Jun 2014 13:53:08 +0000 (UTC) Received: from main.vmx.hu (178-248-200-106.giganet.hu [178.248.200.106]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2593E2851 for ; Tue, 10 Jun 2014 13:53:07 +0000 (UTC) Received: from csernovicsgyula by main.vmx.hu with local (Exim 4.80) (envelope-from ) id 1WuMTZ-0007hC-IT for freebsd-net@freebsd.org; Tue, 10 Jun 2014 15:52:33 +0200 To: freebsd-net@freebsd.org Subject: RFQ From: Chris Tellis Reply-To: betabattery@gmail.com MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Message-Id: Date: Tue, 10 Jun 2014 15:52:33 +0200 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 13:53:08 -0000 Hello, Please quote us for the below listed items. - Solar Panel Module 240/250w - Sealed Lead Acid Battery 12v 100-200aH - 11 SQF-2 Grundfos Solar Pump Also, advice on your acceptable methods of payment and stock availability. Thank You Chris Tellis Beta Battery From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 15:06:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CA19DDE for ; Tue, 10 Jun 2014 15:06:38 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 579E92FE6 for ; Tue, 10 Jun 2014 15:06:38 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id at20so6838944iec.25 for ; Tue, 10 Jun 2014 08:06:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=E+N0oGZZuy4APN3VHO8hh8EVmgdO2UMnT8IxMORv3XU=; b=MqznJbmPmOE5coLiL0JRP67PeKsM+9raDSPNdlNSgErLpB5Q0eLvPlJeAS+/yJkm5A tTcglz3Ul2snA7QzrZlteCgzq7GISeh3KIT1ElyHwfGeYywF+VB86Xp8rSHVRhEB23Zu enR+7kYlYVLjES9ValthaNjYoNvWyzwMDjjVI3c67zdC4Ntj1vIATMDziVrtzGqcZidJ Gs9lrZW2TyaTQOzmEJq77eXdsTZ9QzH3N6V+WuZcWaxZSOHjcEiqYlECbviIYv64kQMI B6dIrmhJjUvEMruLk1DwmMcW9GzIajChLCQ7a52h0AujBf7R7rgRd6Rc32WHtb0oFiqu glvw== X-Received: by 10.50.50.141 with SMTP id c13mr28637301igo.0.1402412797487; Tue, 10 Jun 2014 08:06:37 -0700 (PDT) Received: from [10.1.68.236] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id mh1sm9910531igb.20.2014.06.10.08.06.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 10 Jun 2014 08:06:36 -0700 (PDT) Message-ID: <53971EFB.50107@gmail.com> Date: Tue, 10 Jun 2014 11:06:35 -0400 From: John Jasen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: recommendations on supported 40GbE adapters? X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 15:06:38 -0000 Hello -- frequent listener, first time caller. If this should be redirected to another freebsd-* mailing list, please let me know. I've been experimenting with multiple dual port Mellanox adapters in a system, to see if the entire solution would be suitable for a router replacement project I'm doing. In general, I've been impressed with the Mellanox drivers and their responsiveness, however, I have encountered a few issues where the cards seem to 'fall asleep' or, when I'm playing with the sysctl settings for the mlxen drivers, I panic the kernel. Of course, before I finish spec'ing out a solution to be purchased, I should solicit real-world feedback on the available 40GbE adapters and drivers. >From what I've gleaned, Mellanox, Emulex, and Chelsio all support the development of FreeBSD drivers for their 40GbE cards. Are there any other vendors I should be considering? If anyone else has tried 40GbE cards, I am most interested in your experiences -- especially in stability, performance and performance tuning. Thanks in advance! -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 16:24:51 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53EF160C; Tue, 10 Jun 2014 16:24:51 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 22E0227F6; Tue, 10 Jun 2014 16:24:50 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5AGOiDM079182 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 10 Jun 2014 09:24:44 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5AGOhY5079181; Tue, 10 Jun 2014 09:24:43 -0700 (PDT) (envelope-from jmg) Date: Tue, 10 Jun 2014 09:24:43 -0700 From: John-Mark Gurney To: "Alexander V. Chernikov" Subject: Re: dhclient sucks cpu usage... Message-ID: <20140610162443.GD31367@funkthat.com> Mail-Followup-To: "Alexander V. Chernikov" , Bryan Venteicher , current@FreeBSD.org, net@FreeBSD.org References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5396CD41.2080300@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 10 Jun 2014 09:24:44 -0700 (PDT) Cc: Bryan Venteicher , current@FreeBSD.org, net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 16:24:51 -0000 Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 13:17 +0400: > On 10.06.2014 07:03, Bryan Venteicher wrote: > >Hi, > > > >----- Original Message ----- > >>So, after finding out that nc has a stupidly small buffer size (2k > >>even though there is space for 16k), I was still not getting as good > >>as performance using nc between machines, so I decided to generate some > >>flame graphs to try to identify issues... (Thanks to who included a > >>full set of modules, including dtraceall on memstick!) > >> > >>So, the first one is: > >>https://www.funkthat.com/~jmg/em.stack.svg > >> > >>As I was browsing around, the em_handle_que was consuming quite a bit > >>of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows > >>me that the taskqueue for em was consuming about 50% cpu... Also pretty > >>high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is > >>consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump > >>or anything, but I think dhclient uses bpf to be able to inject packets > >>and listen in on them, so I kill off dhclient, and instantly, the > >>taskqueue > >>thread for em drops down to 40% CPU... (transfer rate only marginally > >>improves, if it does) > >> > >>I decide to run another flame graph w/o dhclient running: > >>https://www.funkthat.com/~jmg/em.stack.nodhclient.svg > >> > >>and now _rxeof drops from 17.22% to 11.94%, pretty significant... > >> > >>So, if you care about performance, don't run dhclient... > >> > >Yes, I've noticed the same issue. It can absolutely kill performance > >in a VM guest. It is much more pronounced on only some of my systems, > >and I hadn't tracked it down yet. I wonder if this is fallout from > >the callout work, or if there was some bpf change. > > > >I've been using the kludgey workaround patch below. > Hm, pretty interesting. > dhclient should setup proper filter (and it looks like it does so: > 13:10 [0] m@ptichko s netstat -B > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient > ) > see "match" count. > And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for > each consumer on interface). > It should not introduce significant performance penalties. Don't forget that it has to process the returning ack's... So, you're looking around 10k+ pps that you have to handle and pass through the filter... That's a lot of packets to process... Just for a bit more "double check", instead of using the HD as a source, I used /dev/zero... I ran a netstat -w 1 -I em0 when running the test, and I was getting ~50.7MiB/s w/ dhclient running and then I killed dhclient and it instantly jumped up to ~57.1MiB/s.. So I launched dhclient again, and it dropped back to ~50MiB/s... and some of this slowness is due to nc using small buffers which I will fix shortly.. And with witness disabled it goes from 58MiB/s to 65.7MiB/s.. In both cases, that's a 13% performance improvement by running w/o dhclient... This is using the latest memstick image, r266655 on a (Lenovo T61): FreeBSD 11.0-CURRENT #0 r266655: Sun May 25 18:55:02 UTC 2014 root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 WARNING: WITNESS option enabled, expect reduced performance. CPU: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz (1995.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x6fb Family=0x6 Model=0xf Stepping=11 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2014019584 (1920 MB) -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 17:35:15 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07456AF9; Tue, 10 Jun 2014 17:35:15 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 920242EBA; Tue, 10 Jun 2014 17:35:14 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WuM1x-0009rk-1x; Tue, 10 Jun 2014 17:24:01 +0400 Message-ID: <5397415B.5070409@FreeBSD.org> Date: Tue, 10 Jun 2014 21:33:15 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Bryan Venteicher , current@FreeBSD.org, net@FreeBSD.org Subject: Re: dhclient sucks cpu usage... References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> <20140610162443.GD31367@funkthat.com> In-Reply-To: <20140610162443.GD31367@funkthat.com> Content-Type: multipart/mixed; boundary="------------070504090202070208030308" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:35:15 -0000 This is a multi-part message in MIME format. --------------070504090202070208030308 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 10.06.2014 20:24, John-Mark Gurney wrote: > Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 13:17 +0400: >> On 10.06.2014 07:03, Bryan Venteicher wrote: >>> Hi, >>> >>> ----- Original Message ----- >>>> So, after finding out that nc has a stupidly small buffer size (2k >>>> even though there is space for 16k), I was still not getting as good >>>> as performance using nc between machines, so I decided to generate some >>>> flame graphs to try to identify issues... (Thanks to who included a >>>> full set of modules, including dtraceall on memstick!) >>>> >>>> So, the first one is: >>>> https://www.funkthat.com/~jmg/em.stack.svg >>>> >>>> As I was browsing around, the em_handle_que was consuming quite a bit >>>> of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows >>>> me that the taskqueue for em was consuming about 50% cpu... Also pretty >>>> high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is >>>> consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump >>>> or anything, but I think dhclient uses bpf to be able to inject packets >>>> and listen in on them, so I kill off dhclient, and instantly, the >>>> taskqueue >>>> thread for em drops down to 40% CPU... (transfer rate only marginally >>>> improves, if it does) >>>> >>>> I decide to run another flame graph w/o dhclient running: >>>> https://www.funkthat.com/~jmg/em.stack.nodhclient.svg >>>> >>>> and now _rxeof drops from 17.22% to 11.94%, pretty significant... >>>> >>>> So, if you care about performance, don't run dhclient... >>>> >>> Yes, I've noticed the same issue. It can absolutely kill performance >>> in a VM guest. It is much more pronounced on only some of my systems, >>> and I hadn't tracked it down yet. I wonder if this is fallout from >>> the callout work, or if there was some bpf change. >>> >>> I've been using the kludgey workaround patch below. >> Hm, pretty interesting. >> dhclient should setup proper filter (and it looks like it does so: >> 13:10 [0] m@ptichko s netstat -B >> Pid Netif Flags Recv Drop Match Sblen Hblen Command >> 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient >> ) >> see "match" count. >> And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for >> each consumer on interface). >> It should not introduce significant performance penalties. > Don't forget that it has to process the returning ack's... So, you're Well, it can be still captured with the proper filter like "ip && udp && port 67 or port 68". We're using tcpdump on high packet ratios (>1M) and it does not influence process _much_. We should probably convert its rwlock to rmlock and use per-cpu counters for statistics, but that's a different story. > looking around 10k+ pps that you have to handle and pass through the > filter... That's a lot of packets to process... > > Just for a bit more "double check", instead of using the HD as a > source, I used /dev/zero... I ran a netstat -w 1 -I em0 when > running the test, and I was getting ~50.7MiB/s w/ dhclient running and > then I killed dhclient and it instantly jumped up to ~57.1MiB/s.. So I > launched dhclient again, and it dropped back to ~50MiB/s... dhclient uses different BPF sockets for reading and writing (and it moves write socket to privileged child process via fork(). The problem we're facing with is the fact that dhclient does not set _any_ read filter on write socket: 21:27 [0] zfscurr0# netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 1529 em0 --fs--l 86774 86769 86784 4044 3180 dhclient --------------------------------------- ^^^^^ -------------------------- 1526 em0 -ifs--l 86789 0 1 0 0 dhclient so all traffic is pushed down introducing contention on BPF descriptor mutex. (That's why I've asked for netstat -B output.) Please try an attached patch to fix this. This is not the right way to fix this, we'd better change BPF behavior not to attach to interface readers for write-only consumers. This have been partially implemented as net.bpf.optimize_writers hack, but it does not work for all direct BPF consumers (which are not using pcap(3) API). > > and some of this slowness is due to nc using small buffers which I will > fix shortly.. > > And with witness disabled it goes from 58MiB/s to 65.7MiB/s.. In > both cases, that's a 13% performance improvement by running w/o > dhclient... > > This is using the latest memstick image, r266655 on a (Lenovo T61): > FreeBSD 11.0-CURRENT #0 r266655: Sun May 25 18:55:02 UTC 2014 > root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > WARNING: WITNESS option enabled, expect reduced performance. > CPU: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz (1995.05-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x6fb Family=0x6 Model=0xf Stepping=11 > Features=0xbfebfbff > Features2=0xe3bd > AMD Features=0x20100800 > AMD Features2=0x1 > TSC: P-state invariant, performance statistics > real memory = 2147483648 (2048 MB) > avail memory = 2014019584 (1920 MB) > --------------070504090202070208030308 Content-Type: text/x-patch; name="dhclient_fix.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dhclient_fix.diff" Index: sbin/dhclient/bpf.c =================================================================== --- sbin/dhclient/bpf.c (revision 266306) +++ sbin/dhclient/bpf.c (working copy) @@ -131,6 +131,11 @@ struct bpf_insn dhcp_bpf_wfilter[] = { int dhcp_bpf_wfilter_len = sizeof(dhcp_bpf_wfilter) / sizeof(struct bpf_insn); +struct bpf_insn dhcp_bpf_dfilter[] = { + BPF_STMT(BPF_RET+BPF_K, 0) +}; +int dhcp_bpf_dfilter_len = sizeof(dhcp_bpf_dfilter) / sizeof(struct bpf_insn); + void if_register_send(struct interface_info *info) { @@ -160,6 +165,12 @@ if_register_send(struct interface_info *info) if (ioctl(info->wfdesc, BIOCSETWF, &p) < 0) error("Can't install write filter program: %m"); + /* Set deny-all read filter for write socket */ + p.bf_len = dhcp_bpf_dfilter_len; + p.bf_insns = dhcp_bpf_dfilter; + if (ioctl(info->wfdesc, BIOCSETFNR, &p) < 0) + error("Can't install write filter program: %m"); + if (ioctl(info->wfdesc, BIOCLOCK, NULL) < 0) error("Cannot lock bpf"); --------------070504090202070208030308-- From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 18:11:55 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E46A4BC1; Tue, 10 Jun 2014 18:11:54 +0000 (UTC) Received: from torment.daemoninthecloset.org (torment.daemoninthecloset.org [94.242.209.234]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "torment.daemoninthecloset.org", Issuer "daemoninthecloset.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 739F222EA; Tue, 10 Jun 2014 18:11:54 +0000 (UTC) Received: from sage.daemoninthecloset.org (cpe-72-177-8-109.austin.res.rr.com [72.177.8.109]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "sage.daemoninthecloset.org", Issuer "daemoninthecloset.org" (not verified)) by torment.daemoninthecloset.org (Postfix) with ESMTPS id 5A65142C2546; Tue, 10 Jun 2014 20:12:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at daemoninthecloset.org X-Virus-Scanned: amavisd-new at daemoninthecloset.org Date: Tue, 10 Jun 2014 13:11:36 -0500 (CDT) From: Bryan Venteicher To: "Alexander V. Chernikov" Message-ID: <1520746932.4518.1402423896830.JavaMail.root@daemoninthecloset.org> In-Reply-To: <5396CD41.2080300@FreeBSD.org> References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> Subject: Re: dhclient sucks cpu usage... MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.51.1.18] X-Mailer: Zimbra 8.0.2_GA_5569 (ZimbraWebClient - GC35 (Mac)/8.0.2_GA_5569) Thread-Topic: dhclient sucks cpu usage... Thread-Index: bTpFmXGUUIkbUiV5bNA/D31nBl/aNQ== Cc: John-Mark Gurney , current@FreeBSD.org, net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:11:55 -0000 ----- Original Message ----- > On 10.06.2014 07:03, Bryan Venteicher wrote: > > Hi, > > > > ----- Original Message ----- > >> So, after finding out that nc has a stupidly small buffer size (2k > >> even though there is space for 16k), I was still not getting as good > >> as performance using nc between machines, so I decided to generate some > >> flame graphs to try to identify issues... (Thanks to who included a > >> full set of modules, including dtraceall on memstick!) > >> > >> So, the first one is: > >> https://www.funkthat.com/~jmg/em.stack.svg > >> > >> As I was browsing around, the em_handle_que was consuming quite a bit > >> of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows > >> me that the taskqueue for em was consuming about 50% cpu... Also pretty > >> high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is > >> consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump > >> or anything, but I think dhclient uses bpf to be able to inject packets > >> and listen in on them, so I kill off dhclient, and instantly, the > >> taskqueue > >> thread for em drops down to 40% CPU... (transfer rate only marginally > >> improves, if it does) > >> > >> I decide to run another flame graph w/o dhclient running: > >> https://www.funkthat.com/~jmg/em.stack.nodhclient.svg > >> > >> and now _rxeof drops from 17.22% to 11.94%, pretty significant... > >> > >> So, if you care about performance, don't run dhclient... > >> > > Yes, I've noticed the same issue. It can absolutely kill performance > > in a VM guest. It is much more pronounced on only some of my systems, > > and I hadn't tracked it down yet. I wonder if this is fallout from > > the callout work, or if there was some bpf change. > > > > I've been using the kludgey workaround patch below. > Hm, pretty interesting. > dhclient should setup proper filter (and it looks like it does so: > 13:10 [0] m@ptichko s netstat -B > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient > ) > see "match" count. > And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for > each consumer on interface). > It should not introduce significant performance penalties. > It will be a bit before I'm able to capture that. Here's a Flamegraph from earlier in the year showing an absurd amount of time spent in bpf_mtap(): http://people.freebsd.org/~bryanv/vtnet/vtnet-bpf-10.svg > > > > diff --git a/sys/net/bpf.c b/sys/net/bpf.c > > index cb3ed27..9751986 100644 > > --- a/sys/net/bpf.c > > +++ b/sys/net/bpf.c > > @@ -2013,9 +2013,11 @@ bpf_gettime(struct bintime *bt, int tstype, struct > > mbuf *m) > > return (BPF_TSTAMP_EXTERN); > > } > > } > > +#if 0 > > if (quality == BPF_TSTAMP_NORMAL) > > binuptime(bt); > > else > > +#endif > bpf_getttime() is called IFF packet filter matches some traffic. > Can you show your "netstat -B" output ? > > getbinuptime(bt); > > > > return (quality); > > > > > >> -- > >> John-Mark Gurney Voice: +1 415 225 5579 > >> > >> "All that I will do, has been done, All that I have, has not." > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 18:18:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4593E9D for ; Tue, 10 Jun 2014 18:18:56 +0000 (UTC) Received: from mx02.buh.bitdefender.com (mx02.buh.bitdefender.com [91.199.104.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.bitdefender.com", Issuer "GeoTrust SSL CA - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFD6A2334 for ; Tue, 10 Jun 2014 18:18:55 +0000 (UTC) Received: (qmail 30573 invoked from network); 10 Jun 2014 21:12:11 +0300 Received: from unknown (HELO mx-robo.bitdefender.biz) (10.17.80.60) by mx02.buh.bitdefender.com with AES256-GCM-SHA384 encrypted SMTP; 10 Jun 2014 21:12:11 +0300 Received: (qmail 9614 invoked from network); 10 Jun 2014 21:12:04 +0300 Received: from unknown (HELO ?172.21.32.65?) (172.21.32.65) by mx-robo.bitdefender.biz with SMTP; 10 Jun 2014 21:12:04 +0300 Message-ID: <53974A75.3040608@users.sourceforge.net> Date: Tue, 10 Jun 2014 21:12:05 +0300 From: Catalin Salgau User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:32.0) Gecko/20100101 Thunderbird/32.0a1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: BPF after BIOCSDIRECTION Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:18:56 -0000 Hello. I was looking to use BIOCSDIRECTION/BIOCSSEESENT, but I find that bpf(4) does not make it very clear if setting BPF_D_IN would apply to all local packets or just those on the current bpf device. As a side question, would anybody happen to know if Linux-land has a similar feature for their LSF thing? Thanks! From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 18:23:23 2014 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE421202; Tue, 10 Jun 2014 18:23:23 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46DCD23F3; Tue, 10 Jun 2014 18:23:23 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WuMmW-000ARB-Gw; Tue, 10 Jun 2014 18:12:08 +0400 Message-ID: <53974CA3.7010701@FreeBSD.org> Date: Tue, 10 Jun 2014 22:21:23 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Bryan Venteicher Subject: Re: dhclient sucks cpu usage... References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> <1520746932.4518.1402423896830.JavaMail.root@daemoninthecloset.org> In-Reply-To: <1520746932.4518.1402423896830.JavaMail.root@daemoninthecloset.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: John-Mark Gurney , current@FreeBSD.org, net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:23:23 -0000 On 10.06.2014 22:11, Bryan Venteicher wrote: > > ----- Original Message ----- >> On 10.06.2014 07:03, Bryan Venteicher wrote: >>> Hi, >>> >>> ----- Original Message ----- >>>> So, after finding out that nc has a stupidly small buffer size (2k >>>> even though there is space for 16k), I was still not getting as good >>>> as performance using nc between machines, so I decided to generate some >>>> flame graphs to try to identify issues... (Thanks to who included a >>>> full set of modules, including dtraceall on memstick!) >>>> >>>> So, the first one is: >>>> https://www.funkthat.com/~jmg/em.stack.svg >>>> >>>> As I was browsing around, the em_handle_que was consuming quite a bit >>>> of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows >>>> me that the taskqueue for em was consuming about 50% cpu... Also pretty >>>> high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is >>>> consuming ~3.18% (under ether_nh_input).. I know I'm not running tcpdump >>>> or anything, but I think dhclient uses bpf to be able to inject packets >>>> and listen in on them, so I kill off dhclient, and instantly, the >>>> taskqueue >>>> thread for em drops down to 40% CPU... (transfer rate only marginally >>>> improves, if it does) >>>> >>>> I decide to run another flame graph w/o dhclient running: >>>> https://www.funkthat.com/~jmg/em.stack.nodhclient.svg >>>> >>>> and now _rxeof drops from 17.22% to 11.94%, pretty significant... >>>> >>>> So, if you care about performance, don't run dhclient... >>>> >>> Yes, I've noticed the same issue. It can absolutely kill performance >>> in a VM guest. It is much more pronounced on only some of my systems, >>> and I hadn't tracked it down yet. I wonder if this is fallout from >>> the callout work, or if there was some bpf change. >>> >>> I've been using the kludgey workaround patch below. >> Hm, pretty interesting. >> dhclient should setup proper filter (and it looks like it does so: >> 13:10 [0] m@ptichko s netstat -B >> Pid Netif Flags Recv Drop Match Sblen Hblen Command >> 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient >> ) >> see "match" count. >> And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for >> each consumer on interface). >> It should not introduce significant performance penalties. >> > > It will be a bit before I'm able to capture that. Here's a Flamegraph from > earlier in the year showing an absurd amount of time spent in bpf_mtap(): Can you briefly describe test setup? (Actually I'm interested in overall pps rate, bpf filter used and match ratio). For example, for some random box at $work: 22:17 [0] m@sas1-fw1 netstat -I vlan802 -w1 input (vlan802) output packets errs idrops bytes packets errs bytes colls 430418 0 0 337712454 396282 0 333207773 0 CPU: 0.4% user, 0.0% nice, 1.2% system, 15.9% interrupt, 82.5% idle 2:17 [0] sas1-fw1# tcpdump -i vlan802 -lnps0 icmp and host X.X.X.X tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vlan802, link-type EN10MB (Ethernet), capture size 65535 bytes 22:17:14.866085 IP X.X.X.X > Y.Y.Y.Y: ICMP echo request, id 6730, seq 1, length 64 22:17 [0] m@sas1-fw1 s netstat -B 2>/dev/null | grep tcpdump 98520 vlan802 ---s--- 27979422 0 40 0 0 tcpdump CPU: 0.9% user, 0.0% nice, 2.7% system, 17.6% interrupt, 78.8% idle (Actually the load is floating due to bursty traffic in 14-20% rate but I can't see much difference with tcpdump turned on/off). > > http://people.freebsd.org/~bryanv/vtnet/vtnet-bpf-10.svg > > >>> diff --git a/sys/net/bpf.c b/sys/net/bpf.c >>> index cb3ed27..9751986 100644 >>> --- a/sys/net/bpf.c >>> +++ b/sys/net/bpf.c >>> @@ -2013,9 +2013,11 @@ bpf_gettime(struct bintime *bt, int tstype, struct >>> mbuf *m) >>> return (BPF_TSTAMP_EXTERN); >>> } >>> } >>> +#if 0 >>> if (quality == BPF_TSTAMP_NORMAL) >>> binuptime(bt); >>> else >>> +#endif >> bpf_getttime() is called IFF packet filter matches some traffic. >> Can you show your "netstat -B" output ? >>> getbinuptime(bt); >>> >>> return (quality); >>> >>> >>>> -- >>>> John-Mark Gurney Voice: +1 415 225 5579 >>>> >>>> "All that I will do, has been done, All that I have, has not." >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 18:49:27 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B98F1131; Tue, 10 Jun 2014 18:49:27 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 93E812642; Tue, 10 Jun 2014 18:49:27 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5AInLnC081441 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 10 Jun 2014 11:49:21 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5AInK72081440; Tue, 10 Jun 2014 11:49:20 -0700 (PDT) (envelope-from jmg) Date: Tue, 10 Jun 2014 11:49:20 -0700 From: John-Mark Gurney To: "Alexander V. Chernikov" Subject: Re: dhclient sucks cpu usage... Message-ID: <20140610184920.GJ31367@funkthat.com> Mail-Followup-To: "Alexander V. Chernikov" , Bryan Venteicher , current@freebsd.org, net@freebsd.org References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> <1520746932.4518.1402423896830.JavaMail.root@daemoninthecloset.org> <53974CA3.7010701@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53974CA3.7010701@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 10 Jun 2014 11:49:21 -0700 (PDT) Cc: Bryan Venteicher , current@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:49:27 -0000 Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 22:21 +0400: > On 10.06.2014 22:11, Bryan Venteicher wrote: > > > >----- Original Message ----- > >>On 10.06.2014 07:03, Bryan Venteicher wrote: > >>>Hi, > >>> > >>>----- Original Message ----- > >>>>So, after finding out that nc has a stupidly small buffer size (2k > >>>>even though there is space for 16k), I was still not getting as good > >>>>as performance using nc between machines, so I decided to generate some > >>>>flame graphs to try to identify issues... (Thanks to who included a > >>>>full set of modules, including dtraceall on memstick!) > >>>> > >>>>So, the first one is: > >>>>https://www.funkthat.com/~jmg/em.stack.svg > >>>> > >>>>As I was browsing around, the em_handle_que was consuming quite a bit > >>>>of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows > >>>>me that the taskqueue for em was consuming about 50% cpu... Also pretty > >>>>high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is > >>>>consuming ~3.18% (under ether_nh_input).. I know I'm not running > >>>>tcpdump > >>>>or anything, but I think dhclient uses bpf to be able to inject packets > >>>>and listen in on them, so I kill off dhclient, and instantly, the > >>>>taskqueue > >>>>thread for em drops down to 40% CPU... (transfer rate only marginally > >>>>improves, if it does) > >>>> > >>>>I decide to run another flame graph w/o dhclient running: > >>>>https://www.funkthat.com/~jmg/em.stack.nodhclient.svg > >>>> > >>>>and now _rxeof drops from 17.22% to 11.94%, pretty significant... > >>>> > >>>>So, if you care about performance, don't run dhclient... > >>>> > >>>Yes, I've noticed the same issue. It can absolutely kill performance > >>>in a VM guest. It is much more pronounced on only some of my systems, > >>>and I hadn't tracked it down yet. I wonder if this is fallout from > >>>the callout work, or if there was some bpf change. > >>> > >>>I've been using the kludgey workaround patch below. > >>Hm, pretty interesting. > >>dhclient should setup proper filter (and it looks like it does so: > >>13:10 [0] m@ptichko s netstat -B > >> Pid Netif Flags Recv Drop Match Sblen Hblen Command > >> 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient > >>) > >>see "match" count. > >>And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for > >>each consumer on interface). > >>It should not introduce significant performance penalties. > >> > > > >It will be a bit before I'm able to capture that. Here's a Flamegraph from > >earlier in the year showing an absurd amount of time spent in bpf_mtap(): > Can you briefly describe test setup? For mine, one machine is sink: nc -l 2387 > /dev/null The machine w/ dhclient is source: nc carbon 2387 < /dev/zero > (Actually I'm interested in overall pps rate, bpf filter used and match > ratio). the overal rate is ~26k pps both in and out (so total ~52kpps)... So, netstat -B; sleep 5; netstat -B gives: Pid Netif Flags Recv Drop Match Sblen Hblen Command 919 em0 --fs--l 6275907 6275938 6275961 4060 2236 dhclient 937 em0 -ifs--l 6275992 0 1 0 0 dhclient Pid Netif Flags Recv Drop Match Sblen Hblen Command 919 em0 --fs--l 6539717 6539752 6539775 4060 2236 dhclient 937 em0 -ifs--l 6539806 0 1 0 0 dhclient -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 18:56:31 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52759557; Tue, 10 Jun 2014 18:56:31 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 288F02735; Tue, 10 Jun 2014 18:56:30 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5AIuQjr081571 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 10 Jun 2014 11:56:27 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5AIuQfV081570; Tue, 10 Jun 2014 11:56:26 -0700 (PDT) (envelope-from jmg) Date: Tue, 10 Jun 2014 11:56:26 -0700 From: John-Mark Gurney To: "Alexander V. Chernikov" Subject: Re: dhclient sucks cpu usage... Message-ID: <20140610185626.GK31367@funkthat.com> Mail-Followup-To: "Alexander V. Chernikov" , Bryan Venteicher , current@freebsd.org, net@freebsd.org References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> <20140610162443.GD31367@funkthat.com> <5397415B.5070409@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5397415B.5070409@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 10 Jun 2014 11:56:27 -0700 (PDT) Cc: Bryan Venteicher , current@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:56:31 -0000 Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 21:33 +0400: > On 10.06.2014 20:24, John-Mark Gurney wrote: > >Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 13:17 > >+0400: > >>On 10.06.2014 07:03, Bryan Venteicher wrote: > >>>Hi, > >>> > >>>----- Original Message ----- > >>>>So, after finding out that nc has a stupidly small buffer size (2k > >>>>even though there is space for 16k), I was still not getting as good > >>>>as performance using nc between machines, so I decided to generate some > >>>>flame graphs to try to identify issues... (Thanks to who included a > >>>>full set of modules, including dtraceall on memstick!) > >>>> > >>>>So, the first one is: > >>>>https://www.funkthat.com/~jmg/em.stack.svg > >>>> > >>>>As I was browsing around, the em_handle_que was consuming quite a bit > >>>>of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows > >>>>me that the taskqueue for em was consuming about 50% cpu... Also pretty > >>>>high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is > >>>>consuming ~3.18% (under ether_nh_input).. I know I'm not running > >>>>tcpdump > >>>>or anything, but I think dhclient uses bpf to be able to inject packets > >>>>and listen in on them, so I kill off dhclient, and instantly, the > >>>>taskqueue > >>>>thread for em drops down to 40% CPU... (transfer rate only marginally > >>>>improves, if it does) > >>>> > >>>>I decide to run another flame graph w/o dhclient running: > >>>>https://www.funkthat.com/~jmg/em.stack.nodhclient.svg > >>>> > >>>>and now _rxeof drops from 17.22% to 11.94%, pretty significant... > >>>> > >>>>So, if you care about performance, don't run dhclient... > >>>> > >>>Yes, I've noticed the same issue. It can absolutely kill performance > >>>in a VM guest. It is much more pronounced on only some of my systems, > >>>and I hadn't tracked it down yet. I wonder if this is fallout from > >>>the callout work, or if there was some bpf change. > >>> > >>>I've been using the kludgey workaround patch below. > >>Hm, pretty interesting. > >>dhclient should setup proper filter (and it looks like it does so: > >>13:10 [0] m@ptichko s netstat -B > >> Pid Netif Flags Recv Drop Match Sblen Hblen Command > >> 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient > >>) > >>see "match" count. > >>And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for > >>each consumer on interface). > >>It should not introduce significant performance penalties. > >Don't forget that it has to process the returning ack's... So, you're > Well, it can be still captured with the proper filter like "ip && udp && > port 67 or port 68". > We're using tcpdump on high packet ratios (>1M) and it does not > influence process _much_. > We should probably convert its rwlock to rmlock and use per-cpu counters > for statistics, but that's a different story. > >looking around 10k+ pps that you have to handle and pass through the > >filter... That's a lot of packets to process... > > > >Just for a bit more "double check", instead of using the HD as a > >source, I used /dev/zero... I ran a netstat -w 1 -I em0 when > >running the test, and I was getting ~50.7MiB/s w/ dhclient running and > >then I killed dhclient and it instantly jumped up to ~57.1MiB/s.. So I > >launched dhclient again, and it dropped back to ~50MiB/s... > dhclient uses different BPF sockets for reading and writing (and it > moves write socket to privileged child process via fork(). > The problem we're facing with is the fact that dhclient does not set > _any_ read filter on write socket: > 21:27 [0] zfscurr0# netstat -B > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 1529 em0 --fs--l 86774 86769 86784 4044 3180 dhclient > --------------------------------------- ^^^^^ -------------------------- > 1526 em0 -ifs--l 86789 0 1 0 0 dhclient > > so all traffic is pushed down introducing contention on BPF descriptor > mutex. > > (That's why I've asked for netstat -B output.) > > Please try an attached patch to fix this. This is not the right way to > fix this, we'd better change BPF behavior not to attach to interface > readers for write-only consumers. > This have been partially implemented as net.bpf.optimize_writers hack, > but it does not work for all direct BPF consumers (which are not using > pcap(3) API). Ok, looks like this patch helps the issue... netstat -B; sleep 5; netstat -B: Pid Netif Flags Recv Drop Match Sblen Hblen Command 958 em0 --fs--l 3880000 14 35 3868 2236 dhclient 976 em0 -ifs--l 3880014 0 1 0 0 dhclient Pid Netif Flags Recv Drop Match Sblen Hblen Command 958 em0 --fs--l 4178525 14 35 3868 2236 dhclient 976 em0 -ifs--l 4178539 0 1 0 0 dhclient and now the rate only drops from ~66MiB/s to ~63MiB/s when dhclient is running... Still a significant drop (5%), but better than before... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Tue Jun 10 21:46:46 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62D14200; Tue, 10 Jun 2014 21:46:46 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D73BB2684; Tue, 10 Jun 2014 21:46:45 +0000 (UTC) Received: from secured.by.ipfw.ru ([95.143.220.47] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WuPxM-000CaH-5U; Tue, 10 Jun 2014 21:35:32 +0400 Message-ID: <53977CB7.9010904@FreeBSD.org> Date: Wed, 11 Jun 2014 01:46:31 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Bryan Venteicher , current@freebsd.org, net@freebsd.org Subject: Re: dhclient sucks cpu usage... References: <20140610000246.GW31367@funkthat.com> <100488220.4292.1402369436876.JavaMail.root@daemoninthecloset.org> <5396CD41.2080300@FreeBSD.org> <20140610162443.GD31367@funkthat.com> <5397415B.5070409@FreeBSD.org> <20140610185626.GK31367@funkthat.com> In-Reply-To: <20140610185626.GK31367@funkthat.com> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------040100020408040101050505" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 21:46:46 -0000 This is a multi-part message in MIME format. --------------040100020408040101050505 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 10.06.2014 22:56, John-Mark Gurney wrote: > Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 21:33 +0400: >> On 10.06.2014 20:24, John-Mark Gurney wrote: >>> Alexander V. Chernikov wrote this message on Tue, Jun 10, 2014 at 13:17 >>> +0400: >>>> On 10.06.2014 07:03, Bryan Venteicher wrote: >>>>> Hi, >>>>> >>>>> ----- Original Message ----- >>>>>> So, after finding out that nc has a stupidly small buffer size (2k >>>>>> even though there is space for 16k), I was still not getting as good >>>>>> as performance using nc between machines, so I decided to generate some >>>>>> flame graphs to try to identify issues... (Thanks to who included a >>>>>> full set of modules, including dtraceall on memstick!) >>>>>> >>>>>> So, the first one is: >>>>>> https://www.funkthat.com/~jmg/em.stack.svg >>>>>> >>>>>> As I was browsing around, the em_handle_que was consuming quite a bit >>>>>> of cpu usage for only doing ~50MB/sec over gige.. Running top -SH shows >>>>>> me that the taskqueue for em was consuming about 50% cpu... Also pretty >>>>>> high for only 50MB/sec... Looking closer, you'll see that bpf_mtap is >>>>>> consuming ~3.18% (under ether_nh_input).. I know I'm not running >>>>>> tcpdump >>>>>> or anything, but I think dhclient uses bpf to be able to inject packets >>>>>> and listen in on them, so I kill off dhclient, and instantly, the >>>>>> taskqueue >>>>>> thread for em drops down to 40% CPU... (transfer rate only marginally >>>>>> improves, if it does) >>>>>> >>>>>> I decide to run another flame graph w/o dhclient running: >>>>>> https://www.funkthat.com/~jmg/em.stack.nodhclient.svg >>>>>> >>>>>> and now _rxeof drops from 17.22% to 11.94%, pretty significant... >>>>>> >>>>>> So, if you care about performance, don't run dhclient... >>>>>> >>>>> Yes, I've noticed the same issue. It can absolutely kill performance >>>>> in a VM guest. It is much more pronounced on only some of my systems, >>>>> and I hadn't tracked it down yet. I wonder if this is fallout from >>>>> the callout work, or if there was some bpf change. >>>>> >>>>> I've been using the kludgey workaround patch below. >>>> Hm, pretty interesting. >>>> dhclient should setup proper filter (and it looks like it does so: >>>> 13:10 [0] m@ptichko s netstat -B >>>> Pid Netif Flags Recv Drop Match Sblen Hblen Command >>>> 1224 em0 -ifs--l 41225922 0 11 0 0 dhclient >>>> ) >>>> see "match" count. >>>> And BPF itself adds the cost of read rwlock (+ bgp_filter() calls for >>>> each consumer on interface). >>>> It should not introduce significant performance penalties. >>> Don't forget that it has to process the returning ack's... So, you're >> Well, it can be still captured with the proper filter like "ip && udp && >> port 67 or port 68". >> We're using tcpdump on high packet ratios (>1M) and it does not >> influence process _much_. >> We should probably convert its rwlock to rmlock and use per-cpu counters >> for statistics, but that's a different story. >>> looking around 10k+ pps that you have to handle and pass through the >>> filter... That's a lot of packets to process... >>> >>> Just for a bit more "double check", instead of using the HD as a >>> source, I used /dev/zero... I ran a netstat -w 1 -I em0 when >>> running the test, and I was getting ~50.7MiB/s w/ dhclient running and >>> then I killed dhclient and it instantly jumped up to ~57.1MiB/s.. So I >>> launched dhclient again, and it dropped back to ~50MiB/s... >> dhclient uses different BPF sockets for reading and writing (and it >> moves write socket to privileged child process via fork(). >> The problem we're facing with is the fact that dhclient does not set >> _any_ read filter on write socket: >> 21:27 [0] zfscurr0# netstat -B >> Pid Netif Flags Recv Drop Match Sblen Hblen Command >> 1529 em0 --fs--l 86774 86769 86784 4044 3180 dhclient >> --------------------------------------- ^^^^^ -------------------------- >> 1526 em0 -ifs--l 86789 0 1 0 0 dhclient >> >> so all traffic is pushed down introducing contention on BPF descriptor >> mutex. >> >> (That's why I've asked for netstat -B output.) >> >> Please try an attached patch to fix this. This is not the right way to >> fix this, we'd better change BPF behavior not to attach to interface >> readers for write-only consumers. >> This have been partially implemented as net.bpf.optimize_writers hack, >> but it does not work for all direct BPF consumers (which are not using >> pcap(3) API). > > Ok, looks like this patch helps the issue... > > netstat -B; sleep 5; netstat -B: > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 958 em0 --fs--l 3880000 14 35 3868 2236 dhclient > 976 em0 -ifs--l 3880014 0 1 0 0 dhclient > Pid Netif Flags Recv Drop Match Sblen Hblen Command > 958 em0 --fs--l 4178525 14 35 3868 2236 dhclient > 976 em0 -ifs--l 4178539 0 1 0 0 dhclient > > and now the rate only drops from ~66MiB/s to ~63MiB/s when dhclient is > running... Still a significant drop (5%), but better than before... Interesting. Can you provide some traces (pmc or dtrace ones)? I'm unsure if this will help, but it's worth trying: please revert my previous patch, apply an attached kernel patch, reboot, set net.bpf.optimize_writers to 1 and try again? > --------------040100020408040101050505 Content-Type: text/x-patch; name="bpf_optimize.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="bpf_optimize.diff" Index: sys/net/bpf.c =================================================================== --- sys/net/bpf.c (revision 266306) +++ sys/net/bpf.c (working copy) @@ -44,7 +44,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include +#include #include #include #include @@ -643,6 +643,20 @@ bpf_attachd(struct bpf_d *d, struct bpf_if *bp) } /* + * Check if filter looks like 'set snaplen' + * program used by pcap-bpf.c:pcap_activate_bpf() + */ +static void +bpf_check_upgrade(u_long cmd, struct bpf_insn *fcode, u_int flen, int *is_snap) +{ + + if (cmd == BIOCSETF && flen == 1 && fcode[0].code == (BPF_RET | BPF_K)) + *is_snap = 1; + else + *is_snap = 0; +} + +/* * Add d to the list of active bp filters. * Reuqires bpf_attachd() to be called before */ @@ -1728,7 +1742,7 @@ bpf_setf(struct bpf_d *d, struct bpf_program *fp, #endif size_t size; u_int flen; - int need_upgrade; + int is_snaplen, need_upgrade; #ifdef COMPAT_FREEBSD32 switch (cmd) { @@ -1755,6 +1769,7 @@ bpf_setf(struct bpf_d *d, struct bpf_program *fp, #ifdef BPF_JITTER jfunc = ofunc = NULL; #endif + is_snaplen = 0; need_upgrade = 0; /* @@ -1773,6 +1788,8 @@ bpf_setf(struct bpf_d *d, struct bpf_program *fp, free(fcode, M_BPF); return (EINVAL); } + /* Try to guess if this is snaplen cmd */ + bpf_check_upgrade(cmd, fcode, flen, &is_snaplen); #ifdef BPF_JITTER /* Filter is copied inside fcode and is perfectly valid. */ jfunc = bpf_jitter(fcode, flen); @@ -1807,11 +1824,27 @@ bpf_setf(struct bpf_d *d, struct bpf_program *fp, * Do not require upgrade by first BIOCSETF * (used to set snaplen) by pcap_open_live(). */ - if (d->bd_writer != 0 && --d->bd_writer == 0) - need_upgrade = 1; - CTR4(KTR_NET, "%s: filter function set by pid %d, " - "bd_writer counter %d, need_upgrade %d", - __func__, d->bd_pid, d->bd_writer, need_upgrade); + if (d->bd_writer != 0) { + if (is_snaplen == 0) { + /* + * We're probably using bpf directly. + * Upgrade immediately. + */ + need_upgrade = 1; + } else if (--d->bd_writer == 0) { + /* + * First snaplen filter has already + * been set. This is probably catch-all + * filter + */ + need_upgrade = 1; + } + CTR5(KTR_NET, + "%s: filter function set by pid %d, " + "bd_writer counter %d, snap %d upgrade %d", + __func__, d->bd_pid, d->bd_writer, + is_snaplen, need_upgrade); + } } } BPFD_UNLOCK(d); @@ -1842,6 +1875,7 @@ bpf_setif(struct bpf_d *d, struct ifreq *ifr) { struct bpf_if *bp; struct ifnet *theywant; + BPFIF_TRACKER; BPF_LOCK_ASSERT(); @@ -2038,6 +2072,7 @@ bpf_tap(struct bpf_if *bp, u_char *pkt, u_int pktl #endif u_int slen; int gottime; + BPFIF_TRACKER; gottime = BPF_TSTAMP_NONE; @@ -2105,6 +2140,7 @@ bpf_mtap(struct bpf_if *bp, struct mbuf *m) #endif u_int pktlen, slen; int gottime; + BPFIF_TRACKER; /* Skip outgoing duplicate packets. */ if ((m->m_flags & M_PROMISC) != 0 && m->m_pkthdr.rcvif == NULL) { @@ -2158,6 +2194,7 @@ bpf_mtap2(struct bpf_if *bp, void *data, u_int dle struct bpf_d *d; u_int pktlen, slen; int gottime; + BPFIF_TRACKER; /* Skip outgoing duplicate packets. */ if ((m->m_flags & M_PROMISC) != 0 && m->m_pkthdr.rcvif == NULL) { @@ -2477,7 +2514,7 @@ bpfattach2(struct ifnet *ifp, u_int dlt, u_int hdr LIST_INIT(&bp->bif_wlist); bp->bif_ifp = ifp; bp->bif_dlt = dlt; - rw_init(&bp->bif_lock, "bpf interface lock"); + rm_init(&bp->bif_lock, "bpf interface lock"); KASSERT(*driverp == NULL, ("bpfattach2: driverp already initialized")); *driverp = bp; @@ -2582,7 +2619,7 @@ bpf_ifdetach(void *arg __unused, struct ifnet *ifp LIST_REMOVE(bp, bif_next); - rw_destroy(&bp->bif_lock); + rm_destroy(&bp->bif_lock); free(bp, M_BPF); nmatched++; @@ -2696,6 +2733,7 @@ bpf_zero_counters(void) { struct bpf_if *bp; struct bpf_d *bd; + BPFIF_TRACKER; BPF_LOCK(); LIST_FOREACH(bp, &bpf_iflist, bif_next) { @@ -2760,6 +2798,7 @@ bpf_stats_sysctl(SYSCTL_HANDLER_ARGS) int index, error; struct bpf_if *bp; struct bpf_d *bd; + BPFIF_TRACKER; /* * XXX This is not technically correct. It is possible for non Index: sys/net/bpf.h =================================================================== --- sys/net/bpf.h (revision 266306) +++ sys/net/bpf.h (working copy) @@ -1260,7 +1260,7 @@ struct bpf_if { u_int bif_dlt; /* link layer type */ u_int bif_hdrlen; /* length of link header */ struct ifnet *bif_ifp; /* corresponding interface */ - struct rwlock bif_lock; /* interface lock */ + struct rmlock bif_lock; /* interface lock */ LIST_HEAD(, bpf_d) bif_wlist; /* writer-only list */ int flags; /* Interface flags */ #endif Index: sys/net/bpfdesc.h =================================================================== --- sys/net/bpfdesc.h (revision 266306) +++ sys/net/bpfdesc.h (working copy) @@ -152,10 +152,11 @@ struct xbpf_d { u_int64_t bd_spare[4]; }; -#define BPFIF_RLOCK(bif) rw_rlock(&(bif)->bif_lock) -#define BPFIF_RUNLOCK(bif) rw_runlock(&(bif)->bif_lock) -#define BPFIF_WLOCK(bif) rw_wlock(&(bif)->bif_lock) -#define BPFIF_WUNLOCK(bif) rw_wunlock(&(bif)->bif_lock) +#define BPFIF_TRACKER struct rm_priotracker tracker +#define BPFIF_RLOCK(bif) rm_rlock(&(bif)->bif_lock, &tracker) +#define BPFIF_RUNLOCK(bif) rm_runlock(&(bif)->bif_lock, &tracker) +#define BPFIF_WLOCK(bif) rm_wlock(&(bif)->bif_lock) +#define BPFIF_WUNLOCK(bif) rm_wunlock(&(bif)->bif_lock) #define BPFIF_FLAG_DYING 1 /* Reject new bpf consumers */ Index: sys/kern/subr_witness.c =================================================================== --- sys/kern/subr_witness.c (revision 266306) +++ sys/kern/subr_witness.c (working copy) @@ -550,7 +550,7 @@ static struct witness_order_list_entry order_lists * BPF */ { "bpf global lock", &lock_class_mtx_sleep }, - { "bpf interface lock", &lock_class_rw }, + { "bpf interface lock", &lock_class_rm }, { "bpf cdev lock", &lock_class_mtx_sleep }, { NULL, NULL }, /* --------------040100020408040101050505-- From owner-freebsd-net@FreeBSD.ORG Wed Jun 11 08:31:36 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9833A9FE for ; Wed, 11 Jun 2014 08:31:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7FA0E297A for ; Wed, 11 Jun 2014 08:31:36 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5B8VaWX040034 for ; Wed, 11 Jun 2014 09:31:36 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 185967] [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 Date: Wed, 11 Jun 2014 08:31:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: olivier@cochard.me X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 08:31:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185967 olivier@cochard.me changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |olivier@cochard.me --- Comment #8 from olivier@cochard.me --- The patch didn't solve Lagg regression between 9.2 and 10-stable (r267244). My configuration only use one lagg member (the second will be added later): ifconfig_em0="up" cloned_interfaces="lagg0" ifconfig_lagg0="laggproto lacp laggport em0 SYNCDHCP" ifconfig_lagg0_ipv6="inet6 accept_rtadv" And it's works great on 9.2: [root@R1]~# uname -a FreeBSD R1 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255918M: Sat Oct 26 22:41:39 CEST 2013 root@orange.bsdrp.net:/usr/obj/BSDRP.amd64/usr/local/BSDRP/BSDRP/FreeBSD/src/sys/amd64 amd64 [root@R1]~# ifconfig lagg0 lagg0: flags=8843 metric 0 mtu 1500 options=9b ether aa:aa:00:01:01:01 inet6 fe80::a8aa:ff:fe01:101%lagg0 prefixlen 64 scopeid 0x8 inet 10.0.12.1 netmask 0xffffff00 broadcast 10.0.12.255 inet6 2001:db8:12:0:a8aa:ff:fe01:101 prefixlen 64 autoconf nd6 options=23 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=1c [root@R1]~# uname -a FreeBSD R1 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255918M: Sat Oct 26 22:41:39 CEST 2013 root@orange.bsdrp.net:/usr/obj/BSDRP.amd64/usr/local/BSDRP/BSDRP/FreeBSD/src/sys/amd64 amd64 But once upgraded to 10-stable AND DISABLING the strict mode, it didn't works anymore: [root@R1]~# uname -a FreeBSD R1 10.0-STABLE FreeBSD 10.0-STABLE #0 r267244M: Mon Jun 9 03:57:44 CEST 2014 root@orange.bsdrp.net:/usr/obj/BSDRP.amd64/usr/local/BSDRP/BSDRP/FreeBSD/src/sys/amd64 amd64 [root@R1]~# echo "net.link.lagg.0.lacp.lacp_strict_mode=0" >> /etc/sysctl.conf [root@R1]~# ifconfig lagg0 lagg0: flags=8843 metric 0 mtu 1500 options=9b ether aa:aa:00:01:01:01 inet6 fe80::a8aa:ff:fe01:101%lagg0 prefixlen 64 scopeid 0x7 inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 nd6 options=23 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=0<> [root@R1]~# sysctl net.link.lagg.0. net.link.lagg.0.use_flowid: 1 net.link.lagg.0.flowid_shift: 16 net.link.lagg.0.count: 1 net.link.lagg.0.active: 0 net.link.lagg.0.flapping: 0 net.link.lagg.0.lacp.lacp_strict_mode: 0 net.link.lagg.0.lacp.debug.rx_test: 0 net.link.lagg.0.lacp.debug.tx_test: 0 In debug mode, I've got this output: lacp_select_tx_port: no active aggregator -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 10:02:09 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4AB9F5F7 for ; Thu, 12 Jun 2014 10:02:09 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 316EF27D3 for ; Thu, 12 Jun 2014 10:02:09 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5CA29AU022457 for ; Thu, 12 Jun 2014 11:02:09 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10gigabit networking problems with Emulex OCE 11102 CNA Date: Thu, 12 Jun 2014 10:02:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 10:02:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 borjam@sarenet.es changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |borjam@sarenet.es --- Comment #5 from borjam@sarenet.es --- I wouldn't classify this bug as non serious. It's critical, as the oce card is currently unusable with the driver included in -STABLE and -CURRENT (10.0.664.0). I am experiencing easy to reproduce panics linked to network traffic. I include summaries of two crash reports. PANIC 1: mierda dumped core - see /var/crash/vmcore.0 Thu Jun 12 12:01:47 CEST 2014 FreeBSD mierda 10.0-STABLE FreeBSD 10.0-STABLE #0 r265018: Sun Apr 27 19:01:41 UTC 2014 root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 panic: sbdrop GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: sbdrop cpuid = 12 KDB: stack backtrace: #0 0xffffffff809102b0 at kdb_backtrace+0x60 #1 0xffffffff808d5815 at panic+0x155 #2 0xffffffff80948383 at sbcut_internal+0x273 #3 0xffffffff80a6d68c at tcp_do_segment+0x175c #4 0xffffffff80a6b2e4 at tcp_input+0xd04 #5 0xffffffff80a01087 at ip_input+0x97 #6 0xffffffff809a0332 at netisr_dispatch_src+0x62 #7 0xffffffff80997916 at ether_demux+0x126 #8 0xffffffff809985be at ether_nh_input+0x35e #9 0xffffffff809a0332 at netisr_dispatch_src+0x62 #10 0xffffffff80a6fd38 at tcp_lro_flush+0x198 #11 0xffffffff81855692 at oce_rq_handler+0x212 #12 0xffffffff81857a8c at oce_intr+0xdc #13 0xffffffff8091e8f5 at taskqueue_run_locked+0xe5 #14 0xffffffff8091f388 at taskqueue_thread_loop+0xa8 #15 0xffffffff808a63aa at fork_exit+0x9a #16 0xffffffff80cafd3e at fork_trampoline+0xe PANIC 2: mierda dumped core - see /var/crash/vmcore.1 Thu Jun 12 12:20:20 CEST 2014 FreeBSD mierda 10.0-STABLE FreeBSD 10.0-STABLE #0 r265018: Sun Apr 27 19:01:41 UTC 2014 root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 panic: sbsndptr: sockbuf 0xfffff800280549b0 and mbuf 0xfffff80178df1d00 clashing GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: sbsndptr: sockbuf 0xfffff800280549b0 and mbuf 0xfffff80178df1d00 clashing cpuid = 12 KDB: stack backtrace: #0 0xffffffff809102b0 at kdb_backtrace+0x60 #1 0xffffffff808d5815 at panic+0x155 #2 0xffffffff80948500 at sbdroprecord_locked+0 #3 0xffffffff80a7123c at tcp_output+0xdbc #4 0xffffffff80a6f02f at tcp_do_segment+0x30ff #5 0xffffffff80a6b2e4 at tcp_input+0xd04 #6 0xffffffff80a01087 at ip_input+0x97 #7 0xffffffff809a0332 at netisr_dispatch_src+0x62 #8 0xffffffff80997916 at ether_demux+0x126 #9 0xffffffff809985be at ether_nh_input+0x35e #10 0xffffffff809a0332 at netisr_dispatch_src+0x62 #11 0xffffffff81855ab9 at oce_rx+0x3c9 #12 0xffffffff81855536 at oce_rq_handler+0xb6 #13 0xffffffff81857a8c at oce_intr+0xdc #14 0xffffffff8091e8f5 at taskqueue_run_locked+0xe5 #15 0xffffffff8091f388 at taskqueue_thread_loop+0xa8 #16 0xffffffff808a63aa at fork_exit+0x9a #17 0xffffffff80cafd3e at fork_trampoline+0xe Uptime: 4m11s Dumping 947 out of 24540 MB:..2%..11%..21%..31%..41%..51%..61%..71%..82%..92% -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 10:35:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6C46D30 for ; Thu, 12 Jun 2014 10:35:54 +0000 (UTC) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A5202AAA for ; Thu, 12 Jun 2014 10:35:54 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id x3so370091qcv.38 for ; Thu, 12 Jun 2014 03:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=x6hHOpj7g2xWSHB4YRpgEigLMeArKTAQ6RUAxg52fAU=; b=PJlW8yNlce4yqgCJGXFI7IvI4/lZeJcNcHRqNezZ/uvwTOEthJ6V1E+iEQ1QG5UHQx ivR54/CUIrHbm7KU/RKlFsb7yKnXNhvTasyTw8Dfjr2WjL6B/0oDBcYOPGAI2uI2OZ+f ycPBToxnTMtDAlslHUeZ0FBPeuBttTwPRxGm2CLgyE25J1iBR55lywsa+DaHvtpLJ/9F ehf5x2AkzU5hcL6XPvf+QpYFzgN8bdeslbdzbuCI4TyGHC386Ht1JsyJaH+vcVy4zlYg rAExi9f+HEjLpMFwpLtOsPm4u4HS/r1uvs1pqrb0oE22rl3Lyz4jpl98J7iBlgkIeru9 yv3g== X-Received: by 10.224.166.9 with SMTP id k9mr59273457qay.25.1402569353532; Thu, 12 Jun 2014 03:35:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.178.137 with HTTP; Thu, 12 Jun 2014 03:35:13 -0700 (PDT) From: Carlos Ferreira Date: Thu, 12 Jun 2014 11:35:13 +0100 Message-ID: Subject: netmap To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 10:35:54 -0000 Hello! First of all, to Luigi and the dev team, great piece of work that netmap is! This is a piece of software that I was looking for quite some time. Your team effort is appreciated! Now the question. I know that this is a FreeBSD mailing list but I was wondering, since you have a PKGBUILD file for ArchLinux, could someone in your team do the same for OpenWRT? Netmap seems to be a piece of software which would come in handy, to develop applications and protocols for the OpenWRT distro. Since OpenWRT uses uClib, I don't really know if it is possible to compile it. Help is appreciated! Once again, thank you! -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 10:41:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAADBF1B for ; Thu, 12 Jun 2014 10:41:36 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 524302B70 for ; Thu, 12 Jun 2014 10:41:36 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id cc10so5206214wib.0 for ; Thu, 12 Jun 2014 03:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=sQwWPt3blK/kJ3JbnFWgryJBo0SFVV8EVokHfmwajao=; b=j3y3zet6dZGi7xQlCxoFjy/BOU3Pzy8rRZ7RrtjEewQ0LZgyG2cFO7wK3onjlDhC08 wdz7XjURwuWrJ6Vld+/aWgybbuGZ+mvQkk/Gip4CwgZ/RvwMPBkxcW3CmRYvIAj8OjTK Y13KVBRm5I1Kr/isB6PgH/UD4TA+62bdNmYK0ZR8CU+ohZxElJnl6xWY2axW1YsRtoa7 jFmuV0d4gqgO1c476pUhG9YzFblZNeFebwg3kf9zOY/4g5QrEA9mRq4pFPzDJywWaO/P SU6nVjoaeSikuI9PPMOo2qQ22/uwlV6p8n+I3MZT32q29ELX8sZplr1jrwVEnWaten95 PaIg== MIME-Version: 1.0 X-Received: by 10.180.183.131 with SMTP id em3mr5221815wic.56.1402569694421; Thu, 12 Jun 2014 03:41:34 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Thu, 12 Jun 2014 03:41:34 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 Jun 2014 12:41:34 +0200 X-Google-Sender-Auth: kr99IquxTVyq_336VMpmoQK1ab4 Message-ID: Subject: Re: netmap From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 10:41:36 -0000 On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira wrote: > Hello! > > First of all, to Luigi and the dev team, great piece of work that netmap > is! This is a piece of software that I was looking for quite some time. > Your team effort is appreciated! > > Now the question. > I know that this is a FreeBSD mailing list but I was wondering, since you > have a PKGBUILD file for ArchLinux, could someone in your team do the sam= e > for OpenWRT? Netmap seems to be a piece of software which would come in > handy, to develop applications and protocols for the OpenWRT distro. > Since OpenWRT uses uClib, I don't really know if it is possible to compil= e > it. > Help is appreciated! > > =E2=80=8Bi expect it will be trivial to compile netmap for openwrt, because it is just a piece of kernel code and the userspace has little or no dependencies. Another story is to write specific netmap extensions for the driver in use -- that might require a little bit of work though not much, knowing the driver. I am afraid we currently we do not have the time and manpower to set up a build environment and give it a try. cheers luigi From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 10:49:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB49215B for ; Thu, 12 Jun 2014 10:49:35 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A63F2C10 for ; Thu, 12 Jun 2014 10:49:35 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id cm18so1377865qab.28 for ; Thu, 12 Jun 2014 03:49:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=TERW5ZRSgQUUzaRVv1tK0CMC4XFi1aFf9YotmJ6sxxY=; b=CBMcDBCOWCzkZ4A6/F05VPLWJ20NqHIKrvMX/NuqxNuQmS8Ua+w92G6f4MUrq3LFCj XgVDNPV7vET/E4ovtUa9df7YR5OIQYErUTmyRgPLtZh6yoA2Sj/eR37uPix7CvXBbk1S NCjRDTlcnyod1s9ZRMduvLX//kKkAQuPYKkHJeAfNX9NQNM2QScp+qWrV+iVh3idZE6O dbyckeFoGal2NDQ9kyDpJ9cMAPUUdlngLWbWp/5nDURVpuRLQdxxel3Oh5A+0ONsPUyE Ao7aAHh/wOwFvtbmeV9xztw7AIuaOi+Nca62dFhBiTPcAGbSNU44OO1VkIZFb5CsYtIr XZoQ== X-Received: by 10.140.85.102 with SMTP id m93mr56419700qgd.26.1402570174618; Thu, 12 Jun 2014 03:49:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.178.137 with HTTP; Thu, 12 Jun 2014 03:48:54 -0700 (PDT) In-Reply-To: References: From: Carlos Ferreira Date: Thu, 12 Jun 2014 11:48:54 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 10:49:35 -0000 First of all, thank you for the quick answer! I will try it myself to compile just the netmap module without the drivers and report the results back to you. Once again, thank you! On 12 June 2014 11:41, Luigi Rizzo wrote: > > > > On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira > wrote: > >> Hello! >> >> First of all, to Luigi and the dev team, great piece of work that netmap >> is! This is a piece of software that I was looking for quite some time. >> Your team effort is appreciated! >> >> Now the question. >> I know that this is a FreeBSD mailing list but I was wondering, since yo= u >> have a PKGBUILD file for ArchLinux, could someone in your team do the sa= me >> for OpenWRT? Netmap seems to be a piece of software which would come in >> handy, to develop applications and protocols for the OpenWRT distro. >> Since OpenWRT uses uClib, I don't really know if it is possible to compi= le >> it. >> Help is appreciated! >> >> > =E2=80=8Bi expect it will be trivial to compile netmap for openwrt, > because it is just a piece of kernel code and the userspace > has little or no dependencies. > Another story is to write specific netmap extensions for the > driver in use -- that might require a little bit of work > though not much, knowing the driver. > > I am afraid we currently we do not have the time and manpower > to set up a build environment and give it a try. > > cheers > luigi > > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 15:15:48 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1B192DF for ; Thu, 12 Jun 2014 15:15:48 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 990B32767 for ; Thu, 12 Jun 2014 15:15:48 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5CFFmSm063822 for ; Thu, 12 Jun 2014 16:15:48 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10gigabit networking problems with Emulex OCE 11102 CNA Date: Thu, 12 Jun 2014 15:15:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 15:15:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #6 from borjam@sarenet.es --- Indeed, just fetching the source from the Emulex website, compiling and loading the module, it seems to be solid now. dev.oce.0.%pnpinfo: vendor=0x19a2 device=0x0700 subvendor=0x10df subdevice=0xe629 class=0x020000 dev.oce.0.%parent: pci3 dev.oce.0.component_revision: ///10.0.747.0/// dev.oce.0.firmware_version: 2.103.397.34 I'm close saturating the interface bandwidth using iperf and now I am unable to make it crash. Please, replace the driver!! :) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 16:11:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4772B9C8 for ; Thu, 12 Jun 2014 16:11:48 +0000 (UTC) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1368E2CEA for ; Thu, 12 Jun 2014 16:11:48 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id tr6so1281215ieb.24 for ; Thu, 12 Jun 2014 09:11:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=Vkc2qe/V/1E+1MrtUY5dwPLY2IIaCAi8q1oKrbBDxMM=; b=VgZAI/0kC/UmJr9WQVY970MViU0PufEvKRRm07QYf/B+LW6LjPZX+FiaFp0u2dcJ+I CsvbV2NyXtZ5SezZ+x73xB8Q5OCCUKw4m+WKKSBZoajSwxzkhnfjILwwhkkIuCRMPBL9 14VgXC6gzUbD6txYw5CcNJ6G1MEtDLmQLw14hYB3adzkH7bKSO+8Ls35VMmx8M6Q1kv0 L8WhYnUr5nCQ9g6otjBcJQ92ADI61sSsGvnGj6JxfCsYmJidUiMeG6G1TCjBCQ9pvaf+ qOchsC11QiUpZdpiIjbX7w9yCAWzbzbXn6xG1FmOZ2IXWfxYKVhDOlAtq5bNToqyR1z6 1LzQ== X-Received: by 10.50.114.197 with SMTP id ji5mr8382442igb.48.1402589507494; Thu, 12 Jun 2014 09:11:47 -0700 (PDT) Received: from [10.1.68.236] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id m1sm6372638ige.22.2014.06.12.09.11.46 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 12 Jun 2014 09:11:46 -0700 (PDT) Message-ID: <5399D141.1020600@gmail.com> Date: Thu, 12 Jun 2014 12:11:45 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: em driver: netif hangs the system if interface is cabled and configured but there is no link X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 16:11:48 -0000 I'm configuring a system that's destined to be a multi-homed server, using Intel dual port 1GbE cards that rely on the em driver. em0 has link, and only needed configuration. In an attempt to be ahead of the game, I pre-configured em2, plugged in my side of the cable to be ready when the other side plugged theirs in, and rebooted the box. In this state, it appears the box will hang as netif works through interfaces defined in rc.conf. I'm not sure if permanently, but I'm willing to call 20 minutes 'permanent' for the purposes of this exercise. I eventually was able to narrow it down to the configurations for em2, and em2 not having link WHILE a cable was plugged in. I was able to replicate the expected condition by unplugging the cable, and was able to replicate the failure condition on em1 and em3 by moving the cable and configurations. Any thoughts? Am I missing something? -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 17:02:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97B3D843 for ; Thu, 12 Jun 2014 17:02:39 +0000 (UTC) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6181E21FA for ; Thu, 12 Jun 2014 17:02:39 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id wp4so1650638obc.12 for ; Thu, 12 Jun 2014 10:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vBrpSjUkhYJyzaHv/iO6ASDCnO94Mkf8X41UICU2DFw=; b=sZRdBzRAuYG2O56S7xl4oNN2lhDUUNiWPz0+iZJZuAge8HhVIVrokKzxLNFBdNu14M jylBxAXaI84HlAV7jdMoeJieptjLAt1FqisgxR3w8kIc+6R2dt8pSB/X+HgbfcrSuXn3 8gZPZlk1+5rlmCfoX4Xjbkbf62EbZL3bq56Jd0FwKRicmeOHBAs2Ki1RCOmxy61CYpBw nEmAFmOLtpJA6P+dMpP7ZKSGaaRSdzdWdi3Sg0yBghuFgGRc33d0u28Tv24onWdmZ+9m 9wRRiB05TjswuXCUCzhcJzyFBRly+BZqXaqVAFSNieBcfIUEn8DjBh42yobNbWsSkILq MZXw== MIME-Version: 1.0 X-Received: by 10.182.20.105 with SMTP id m9mr46568246obe.36.1402592558772; Thu, 12 Jun 2014 10:02:38 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Thu, 12 Jun 2014 10:02:38 -0700 (PDT) In-Reply-To: <5399D141.1020600@gmail.com> References: <5399D141.1020600@gmail.com> Date: Thu, 12 Jun 2014 19:02:38 +0200 Message-ID: Subject: Re: em driver: netif hangs the system if interface is cabled and configured but there is no link From: Andreas Nilsson To: John Jasem Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 17:02:39 -0000 On Thu, Jun 12, 2014 at 6:11 PM, John Jasem wrote: > I'm configuring a system that's destined to be a multi-homed server, > using Intel dual port 1GbE cards that rely on the em driver. > > em0 has link, and only needed configuration. > > In an attempt to be ahead of the game, I pre-configured em2, plugged in > my side of the cable to be ready when the other side plugged theirs in, > and rebooted the box. > > In this state, it appears the box will hang as netif works through > interfaces defined in rc.conf. I'm not sure if permanently, but I'm > willing to call 20 minutes 'permanent' for the purposes of this exercise. > > I eventually was able to narrow it down to the configurations for em2, > and em2 not having link WHILE a cable was plugged in. I was able to > replicate the expected condition by unplugging the cable, and was able > to replicate the failure condition on em1 and em3 by moving the cable > and configurations. > > Any thoughts? Am I missing something? > > -- John Jasen (jjasen@gmail.com) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > If it is a dual port card, shouldn't it be em0 and em1 ? Best regards Andreas From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 17:11:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7AA8D7D for ; Thu, 12 Jun 2014 17:11:38 +0000 (UTC) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2F17232C for ; Thu, 12 Jun 2014 17:11:38 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id rd18so1410501iec.35 for ; Thu, 12 Jun 2014 10:11:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=lCsWUho8M0+oa98YX+Ieqb0l44JSZILhsTAv1YXsXHE=; b=i8nLzjaH4ePuZSwo4C6q4Y8Saq/1Cxjud3w0G8cwtTI9ljH5LrlrswIKQ9ElhnU+TE nz0cmvBY1GzzBnBAkO2rPDTUe7v4HsCfXRZsD4xUHDYzlQ8f892e/Ro4goiN3SvBHFR4 jjcMK9DctKxh3hRZeb+rAsbapuxGdQtYMtRecCoXAUGLNKJg7JManXynWaUcEcLOYlw4 gLiMrvjYA6c6s/aN9STznJeG+lmQSkI0Iii2UYm740YSfBqc2M0p1tPuB29gJVCVHlsi IWWuOApj3iQt4toVK5P4Pzlz3d4x/l7tH/RLGkVjWNwkuZpuJ0azT02u7nkZvza03nOq fHAA== X-Received: by 10.42.98.73 with SMTP id r9mr12571821icn.87.1402593098134; Thu, 12 Jun 2014 10:11:38 -0700 (PDT) Received: from [10.1.68.236] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id 3sm49377550igy.13.2014.06.12.10.11.37 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 12 Jun 2014 10:11:37 -0700 (PDT) Message-ID: <5399DF48.3000008@gmail.com> Date: Thu, 12 Jun 2014 13:11:36 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: em driver: netif hangs the system if interface is cabled and configured but there is no link References: <5399D141.1020600@gmail.com> In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 17:11:38 -0000 On 06/12/2014 01:02 PM, Andreas Nilsson wrote: > If it is a dual port card, shouldn't it be em0 and em1 ? Yes. I do have two dual port cards however. -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 20:23:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40AE3D32 for ; Thu, 12 Jun 2014 20:23:53 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 178D72535 for ; Thu, 12 Jun 2014 20:23:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=2N4ckM/U/6CzlER9p6spa4dr5WtcsGenAewlfgSYqCk=; b=EiayeDT21i4OMmNbqjD1MRPfhGD/67sAFW2+meLj84gNIVk6jP0xF7xdpEcbbYXTgHq3xm32iFb7HzaNEcd7pksxnPwh9Z0ZhVlunU+L0Ip5Ov053fYn38/wJjxviWZVHcznp/hZmF2AfKydL+BKdeZ6d9dyEer/Jc/1F0pQPD0=; Received: from thebighonker.lerctr.org ([192.147.25.65]:24901) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WvBXK-000Gxq-CA for freebsd-net@freebsd.org; Thu, 12 Jun 2014 15:23:51 -0500 Date: Thu, 12 Jun 2014 15:23:49 -0500 From: Larry Rosenman To: freebsd-net@freebsd.org Subject: IPv6: "xxx::x already configured" in logs... why? Message-ID: <20140612202349.GA65079@thebighonker.lerctr.org> Mail-Followup-To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -3.6 (---) X-LERCTR-Spam-Score: -3.6 (---) X-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651 X-LERCTR-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 20:23:53 -0000 I just started using IPv6 behind my (new to me) Cisco 1841. I see lots of: Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured in my /var/log/messages. The rc.conf: cloned_interfaces="lagg0:sticky " create_args_lagg0="laggport bce0 laggport bce1 laggproto loadbalance" ifconfig_lagg0="192.147.25.65/24" ifconfig_bce0="up" ifconfig_bce1="up" defaultrouter="192.147.25.1" #################################################### # IPv6 AutoConf: ifconfig_lagg0_ipv6="inet6 accept_rtadv" rtsold_enable="YES" #################################################### hostname=thebighonker.lerctr.org ifconfig_lagg0_alias0="inet 192.147.25.45/24" ifconfig_lagg0_alias1="inet 192.147.25.11/24" ifconfig_lagg0_alias2="inet 192.147.25.66/24" sshd_enable="YES" ntpd_enable="YES" powerd_enable="YES" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev="AUTO" zfs_enable="YES" # ------------ hostid_enable="YES" postgresql_enable="YES" fah_enable="NO" fah2_enable="NO" linux_enable="YES" smartd_enable="YES" spamd_enable="YES" devcpu_enable="YES" clamav_clamd_enable="YES" clamav_freshclam_enable="YES" sendmail_enable="NONE" exim_enable="YES" exim_flags="-bd -q5m" named_enable="YES" named_program="/usr/local/sbin/named" bsdstats_enable="YES" spamd_flags="-c -Q -m 20" apache22_enable="YES" syslogd_flags="-n -a 192.147.25.0/24:* -a 209.198.148.248/29:*" bacula_fd_enable="YES" sshblock_enable="YES" dovecot_enable="YES" monthly_stats_enable="YES" boinc_client_enable="YES" microcode_update_enable="YES" ---- sysctl.conf: # $FreeBSD: stable/10/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 vm.lowmem_period=0 vfs.usermount=1 vfs.zfs.super_owner=1 kern.elf32.fallback_brand=3 kern.ipc.shm_allow_removed=1 net.inet6.ip6.dad_count=0 ===== Ideas? (I may be an idiot, so any criticism welcomed). if you need the 1841's config, I can supply that as well. It's using a Hurricane electric Tunnel. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-net@FreeBSD.ORG Thu Jun 12 20:40:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB0AFF4D for ; Thu, 12 Jun 2014 20:40:22 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B09812665 for ; Thu, 12 Jun 2014 20:40:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date; bh=rhgkQAT0FG0ht/WOIkIchXyorzD/AmuLErNWvLXe+k8=; b=kHmYKzRVuBLJLZjwOPWU7Zr8HfKPtn03ejkkwo08cQ4PUFNIUedWts5qaNn21Wb+yzgr0hPlKZj+QhkPVFlmLm22UlQP6pRRQ4BIT3wXh6jMEaSG1ROGYlEsD5vhiOJQTXp+oeRx/plRY9yNm+kQXug+o63Dw800RKMwGM7YdBs=; Received: from thebighonker.lerctr.org ([192.147.25.65]:56792) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WvBnJ-000HDM-14 for freebsd-net@freebsd.org; Thu, 12 Jun 2014 15:40:22 -0500 Date: Thu, 12 Jun 2014 15:40:19 -0500 From: Larry Rosenman To: freebsd-net@freebsd.org Subject: Re: IPv6: "xxx::x already configured" in logs... why? Message-ID: <20140612204019.GA66089@thebighonker.lerctr.org> Mail-Followup-To: freebsd-net@freebsd.org References: <20140612202349.GA65079@thebighonker.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140612202349.GA65079@thebighonker.lerctr.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -3.6 (---) X-LERCTR-Spam-Score: -3.6 (---) X-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651 X-LERCTR-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 20:40:22 -0000 On Thu, Jun 12, 2014 at 03:23:49PM -0500, Larry Rosenman wrote: > I just started using IPv6 behind my (new to me) Cisco 1841. > > I see lots of: > Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured FreeBSD version: FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #32 r267121M: Thu Jun 5 13:27:02 CDT 2014 root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 I've also seen the same on -CURRENT behind the same router with similar config. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-net@FreeBSD.ORG Fri Jun 13 05:54:50 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFF95B16 for ; Fri, 13 Jun 2014 05:54:50 +0000 (UTC) Received: from outbound-03.safaricombusiness.co.ke (outbound-03.safaricombusiness.co.ke [41.203.208.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0C4C823BA for ; Fri, 13 Jun 2014 05:54:49 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArAIAHmRmlMpy9CKZGdsb2JhbABEFoMNqgqFKpExNIQXFgIYCw0GFSiCCoIyATQESgoqOgETCiQBhS2Cd6MehxyPBZkdiz0BgjMIEAIBhRcEigE0hgqJcocOkAU6 X-IPAS-Result: ArAIAHmRmlMpy9CKZGdsb2JhbABEFoMNqgqFKpExNIQXFgIYCw0GFSiCCoIyATQESgoqOgETCiQBhS2Cd6MehxyPBZkdiz0BgjMIEAIBhRcEigE0hgqJcocOkAU6 X-IronPort-AV: E=Sophos;i="5.01,469,1400014800"; d="scan'208";a="633012391" Received: from 3g-relay-01.safaricombusiness.co.ke ([41.203.208.138]) by smtp01.safaricombusiness.co.ke with ESMTP; 13 Jun 2014 08:51:01 +0300 From: Compuline Technologies To: Message-Id: <20140613085103.76319121@compulinetechnologies.com> Subject: Payroll 2014 with New NSSF rates @ 24,000 only !!! Date: Fri, 13 Jun 2014 08:51:03 +0300 Reply-To: info@compulinetechnologies.com Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 05:54:50 -0000 Dear client Dont miss this new offer PAYROLL 2014 With new NSSF Rates @ Ksh 24,000 ONLY !!! QUICKBOOKS PRO 2014 @ ksh 30,000 For the clients who are already using our Payroll the updates are available @ a fee of ksh 9,000 kindly contact the undersigned to get the updates. Mr. James. Mungai I.T Consultant Compuline Technologies 0721339494 The 2014 BestPay Human Resource and Payroll Processing System is a Kenyan human resource and payroll software that is used by many different organizations. The software incorporates all the features needed to run a payroll that fulfills the requirements of the Kenya Revenue Authority (K.R.A.) as per the Employers Guide to Pay As You Earn document which is issued by K.R.A every year. BestPay is a Windows based system which uses graphical features such as icons for ease of use. It is a user-friendly payroll program that runs on PCs with Windows XP,Linux and Windows 7& 8 platforms. Its main objective is to computerize the payroll and the human resource functions of your organization, providing the management with clear, concise, up-to- date reports that would give an accurate picture of the activities carried out within the organization. This would result in greater efficiency and accuracy in the information processed. The software also conforms with the new PAYE submission guidelines from the K.R.A. with regards to monthly and quarterly PAYE returns by employers on behalf of the employees. The system automatically generates the P10D report conforming with the KRA format. The employer can also generate the monthly return that can be uploaded to the KRA website on a monthly basis. SYSTEM FEATURES . Multi companies . Company details, company address, registration details . Allows user definable employee categories, departments, grades etc. . Permits the user to input the employee details in the main employee details screen. . Terminated employees - date of leaving, reasons for leaving (dismissal,resignation, termination, retirement) . User definable medical schemes . Employer training reports . Pension details showing employer and employee contributions. . Terminated employees reports . Multi access Other Features include: . User definable payroll, earning/deduction codes . Flexibility in handling of tax tables. . Batch posting of transactions; by employee, by class of employees. . Exempting of staff from selected statutory deductions (e.g. in case of an expatriate or casual employees) . Loan processing- tracking down loan details, editing loan transactions, producing and printing loan repayment schedule. . Instant viewing of an employee pay slip. . Statutory deductions PAYE, NSSF, NHIF . Year end income tax reports - KRA approved P9 forms, P10A, P10, P10D,Previous years P9 etc . Supports direct bank salary remittance (SFI) and compatible with QuickBooks Accounting Software . Audit trail Reports Include: . Pay slips . Payment lists ie. payment by cash, payment by cheque, payment by bank transfer . Payroll analysis payments, deductions, negative pay, by department,by cost center, by pay point . Benefits details . Statutory deduction reports e.g NSSF, NHIF,PAYE,HELB,Pension schemes . Printable Labels and Coinage Analysis Our expertise in Computers and Software Design provided us with the knowledge to Develop Solutions and Implement Customised and Offshelf Softwares that answer the needs of today's clients. . looking forward to doing business with you. This email, any attachment and response string are confidential and may be legally privileged. Any opinions expressed in this mail do not necessarily reflect the opinions of Compuline Technologies. If you are not the intended recipient, please email the sender and delete this message and any attachments immediately. Please do not copy or forward this message or attachment. Internet communications are not secure and therefore Compuline Technologies does not accept legal responsibility for the contents of this message as it has been transmitted over a public network From owner-freebsd-net@FreeBSD.ORG Fri Jun 13 09:56:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1702867 for ; Fri, 13 Jun 2014 09:56:35 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 903C22761 for ; Fri, 13 Jun 2014 09:56:35 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id w8so3067409qac.22 for ; Fri, 13 Jun 2014 02:56:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=gejE7To1GFlwjM4fyjp1+6qvnjsM0K6KTmiZi07hxhI=; b=ihTbzx1V1QF8rRR3w8gDCi3OakgSa3goT6tEdPCe/IahfOnehyI6PzxuyloFfPWExB /1j1XavifO5pGIFwH8Gc51zeg6VLtqYKR6X7H3PmU4N6IHgfPNtCMV7//ejVn/JE5Z0z 4d2XyNDS9l5DbG5oFfeMVf59w/JqYl8ki5WnKHW2OSfQTy7karA8Zs+w0GPGuGeao/MF H8UbQkQ7E9wfRe12kApStMl4cGTBbK9Pm/D5uC3dZzhws3Xw2JM8MEXw9H+R1BgrCVrQ VM2Tw/GhK1c75GirrZU9fVHAJquEY8NFBY5UG7ba1aPXsVcLxVzqttbWtJR+kpnU6Fmt xS1A== X-Received: by 10.229.79.2 with SMTP id n2mr1895536qck.11.1402653394700; Fri, 13 Jun 2014 02:56:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.178.137 with HTTP; Fri, 13 Jun 2014 02:55:54 -0700 (PDT) In-Reply-To: References: From: Carlos Ferreira Date: Fri, 13 Jun 2014 10:55:54 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 09:56:35 -0000 Hello Luigi (and to all) I was able to successfully compile the netmap module for OpenWRT but without drivers. According to the information in the README file which comes with the source, the drivers are not necessary but with some "reduced performance". I would like to ask how much degradation should be expected. I would like to run some tests to see if everything is ok and if the port was successful= . Thank you for the help! On 12 June 2014 11:48, Carlos Ferreira wrote: > First of all, thank you for the quick answer! > > I will try it myself to compile just the netmap module without the driver= s > and report the results back to you. > > Once again, thank you! > > > On 12 June 2014 11:41, Luigi Rizzo wrote: > >> >> >> >> On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira >> wrote: >> >>> Hello! >>> >>> First of all, to Luigi and the dev team, great piece of work that netma= p >>> is! This is a piece of software that I was looking for quite some time. >>> Your team effort is appreciated! >>> >>> Now the question. >>> I know that this is a FreeBSD mailing list but I was wondering, since y= ou >>> have a PKGBUILD file for ArchLinux, could someone in your team do the >>> same >>> for OpenWRT? Netmap seems to be a piece of software which would come in >>> handy, to develop applications and protocols for the OpenWRT distro. >>> Since OpenWRT uses uClib, I don't really know if it is possible to >>> compile >>> it. >>> Help is appreciated! >>> >>> >> =E2=80=8Bi expect it will be trivial to compile netmap for openwrt, >> because it is just a piece of kernel code and the userspace >> has little or no dependencies. >> Another story is to write specific netmap extensions for the >> driver in use -- that might require a little bit of work >> though not much, knowing the driver. >> >> I am afraid we currently we do not have the time and manpower >> to set up a build environment and give it a try. >> >> cheers >> luigi >> >> > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Fri Jun 13 19:22:06 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1DEADB3 for ; Fri, 13 Jun 2014 19:22:06 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 88A2F2D27 for ; Fri, 13 Jun 2014 19:22:06 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5DJM660020018 for ; Fri, 13 Jun 2014 20:22:06 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 175734] no ethernet detected on system with EG20T PCH chipset ATOM E6xx series Date: Fri, 13 Jun 2014 19:22:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: n_hibma@FreeBSD.org X-Bugzilla-Status: Timeout X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 19:22:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=175734 Nick Hibma changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Timeout CC| |n_hibma@FreeBSD.org Resolution|--- |Overcome By Events --- Comment #3 from Nick Hibma --- We've successfully run FBSD10-STABLE on a EG20T chipset platform and used its networking as well. You might want to try again. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Jun 13 22:39:53 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCA674A2 for ; Fri, 13 Jun 2014 22:39:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4B6F2D88 for ; Fri, 13 Jun 2014 22:39:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5DMdrjp081633 for ; Fri, 13 Jun 2014 23:39:53 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] cpu affinity not working in FreeBSD 10-STABLE Date: Fri, 13 Jun 2014 22:39:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc component assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 22:39:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hiren@FreeBSD.org Component|bin |kern Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Jun 14 10:12:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEDC98DF for ; Sat, 14 Jun 2014 10:12:25 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 5470B2243 for ; Sat, 14 Jun 2014 10:12:24 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 9E1B77300A; Sat, 14 Jun 2014 12:15:23 +0200 (CEST) Date: Sat, 14 Jun 2014 12:15:23 +0200 From: Luigi Rizzo To: Carlos Ferreira Subject: Re: netmap Message-ID: <20140614101523.GD22187@onelab2.iet.unipi.it> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2014 10:12:26 -0000 On Fri, Jun 13, 2014 at 10:55:54AM +0100, Carlos Ferreira wrote: > Hello Luigi (and to all) > > I was able to successfully compile the netmap module for OpenWRT but > without drivers. According to the information in the README file which > comes with the source, the drivers are not necessary but with some "reduced > performance". > > I would like to ask how much degradation should be expected. I would like > to run some tests to see if everything is ok and if the port was successful. At 10Gbit we are easily talking about a 5x loss of efficiency. At 1Gbit probably the gap is a bit smaller, also because a lot of low-end device have severe bus bandwidth limitations. I would be interested in seeing what kind of performance you get on your openwrt box with the following commands 1. just a sender on a software switch pkt-gen -i vale0:a -f tx 2. sender and receiver on a software switch pkt-gen -i vale0:a -f tx & pkt-gen -i vale0:b -f rx & 3. sender and receiver on a netmap pipe pkt-gen -i vale0:x{0 -f tx & pkt-gen -i vale0:x}0 -f rx & 4. sending on a physical device (make sure something is attached to the output port. Also this is tricky because many openwrt boxes have a switch on the output so you need to use a unicast destination MAC address) pkt-gen -i eth0 -f tx -D 00:11:22:33:44:55 cheers luigi > > Thank you for the help! > > > > On 12 June 2014 11:48, Carlos Ferreira wrote: > > > First of all, thank you for the quick answer! > > > > I will try it myself to compile just the netmap module without the drivers > > and report the results back to you. > > > > Once again, thank you! > > > > > > On 12 June 2014 11:41, Luigi Rizzo wrote: > > > >> > >> > >> > >> On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira > >> wrote: > >> > >>> Hello! > >>> > >>> First of all, to Luigi and the dev team, great piece of work that netmap > >>> is! This is a piece of software that I was looking for quite some time. > >>> Your team effort is appreciated! > >>> > >>> Now the question. > >>> I know that this is a FreeBSD mailing list but I was wondering, since you > >>> have a PKGBUILD file for ArchLinux, could someone in your team do the > >>> same > >>> for OpenWRT? Netmap seems to be a piece of software which would come in > >>> handy, to develop applications and protocols for the OpenWRT distro. > >>> Since OpenWRT uses uClib, I don't really know if it is possible to > >>> compile > >>> it. > >>> Help is appreciated! > >>> > >>> > >> ???i expect it will be trivial to compile netmap for openwrt, > >> because it is just a piece of kernel code and the userspace > >> has little or no dependencies. > >> Another story is to write specific netmap extensions for the > >> driver in use -- that might require a little bit of work > >> though not much, knowing the driver. > >> > >> I am afraid we currently we do not have the time and manpower > >> to set up a build environment and give it a try. > >> > >> cheers > >> luigi > >> > >> > > > > > > -- > > > > Carlos Miguel Ferreira > > Researcher at Telecommunications Institute > > Aveiro - Portugal > > Work E-mail - cmf@av.it.pt > > Skype & GTalk -> carlosmf.pt@gmail.com > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Jun 14 17:35:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67D7F68E for ; Sat, 14 Jun 2014 17:35:37 +0000 (UTC) Received: from mail-ob0-f175.google.com (mail-ob0-f175.google.com [209.85.214.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3501720D5 for ; Sat, 14 Jun 2014 17:35:36 +0000 (UTC) Received: by mail-ob0-f175.google.com with SMTP id wm4so3350157obc.6 for ; Sat, 14 Jun 2014 10:35:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=zYrfNcbzdv6RB8ZB1V3GQHeig6y8oF9xw51b3mgaOZk=; b=Z9/Un9DE1yBwm6baYkFx91MgqD1XAnpLEl9lSb+QMvSn2AQlID3c+Tr7PLqcP5TQoi nRQmhn+Xt9iUr3h8kOr9iRKJ2piEcPWTAEMeHEOtaTREonmbzpYtuKTFiNhDo9/dvykW pVGg7QtALEfOFD6BbZB5VWNJmjUgQV4sxaYX8xzGAasy0XzlR2rwIBS6e6D3dEwkZzLE DiIbvKkiuOk+8dUKIp3tcgWToNYmRGSr406b7qeKxfqPwcV4EErY5PLu97wrhjNUYIIA /p71NTsf1pdiOL9EoKOWfUsA3sc0ZVXKhrz+s6zmmQ0RLaRoo2wzKCnJF3AaY9gOllPe ZlYQ== X-Gm-Message-State: ALoCoQkl04IMc4AbHRQei29tcI90tAqB+VBKwcm1fK5GFIRleOs5pl6PPHbKKEHTHREWIu/7Hha5 MIME-Version: 1.0 X-Received: by 10.182.245.164 with SMTP id xp4mr10043683obc.23.1402767329861; Sat, 14 Jun 2014 10:35:29 -0700 (PDT) Received: by 10.60.11.134 with HTTP; Sat, 14 Jun 2014 10:35:29 -0700 (PDT) Date: Sat, 14 Jun 2014 10:35:29 -0700 Message-ID: Subject: ipfw table matching algorithm question From: Michael Sierchio To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2014 17:35:37 -0000 Luigi - Does table entry matching use a longest prefix match? - M From owner-freebsd-net@FreeBSD.ORG Sat Jun 14 19:02:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 147F8B55 for ; Sat, 14 Jun 2014 19:02:50 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CAC362708 for ; Sat, 14 Jun 2014 19:02:49 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WvpIk-000ME4-Mn; Sat, 14 Jun 2014 18:51:26 +0400 Message-ID: <539C9BD5.70302@FreeBSD.org> Date: Sat, 14 Jun 2014 23:00:37 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Michael Sierchio , "freebsd-net@freebsd.org" Subject: Re: ipfw table matching algorithm question References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2014 19:02:50 -0000 On 14.06.2014 21:35, Michael Sierchio wrote: > Luigi - > > Does table entry matching use a longest prefix match? I'm not Luigi, but the answer is "yes" anyway :) > > - M > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 02:30:57 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E96356F3 for ; Sun, 15 Jun 2014 02:30:57 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D027424F7 for ; Sun, 15 Jun 2014 02:30:57 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5F2UvwN088796 for ; Sun, 15 Jun 2014 03:30:57 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191040] [em] netif hangs the system if interface is cabled and configured but there is no link Date: Sun, 15 Jun 2014 02:30:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 02:30:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191040 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|em driver: netif hangs the |[em] netif hangs the system |system if interface is |if interface is cabled and |cabled and configured but |configured but there is no |there is no link |link --- Comment #1 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 04:43:25 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7CB2F4D2 for ; Sun, 15 Jun 2014 04:43:25 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5730A2F86 for ; Sun, 15 Jun 2014 04:43:25 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5F4hPvX030532 for ; Sun, 15 Jun 2014 05:43:25 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Sun, 15 Jun 2014 04:43:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 04:43:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|cpu affinity not working in |[em] cpu affinity not |FreeBSD 10-STABLE |working in FreeBSD | |10-STABLE --- Comment #2 from Mark Linimon --- Update Summary. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 10:09:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 848173E5 for ; Sun, 15 Jun 2014 10:09:15 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5828325D0 for ; Sun, 15 Jun 2014 10:09:14 +0000 (UTC) Received: from jre-mbp.elischer.org (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5FA94VT059589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 15 Jun 2014 03:09:06 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <539D70BB.70203@freebsd.org> Date: Sun, 15 Jun 2014 18:08:59 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, Michael Sierchio Subject: Re: ipfw table matching algorithm question References: <539C9BD5.70302@FreeBSD.org> In-Reply-To: <539C9BD5.70302@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 10:09:15 -0000 On 6/15/14, 3:00 AM, Alexander V. Chernikov wrote: > On 14.06.2014 21:35, Michael Sierchio wrote: >> Luigi - >> >> Does table entry matching use a longest prefix match? > I'm not Luigi, but the answer is "yes" anyway :) this may be about to change, because tables are getting a rewrite, but IP-based tables use the same code that the routing tables use. >> >> - M >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 12:01:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B84FDF89; Sun, 15 Jun 2014 12:01:07 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2AA972292; Sun, 15 Jun 2014 12:01:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s5FC10QY065827; Sun, 15 Jun 2014 22:01:01 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sun, 15 Jun 2014 22:01:00 +1000 (EST) From: Ian Smith To: Julian Elischer Subject: Re: ipfw table matching algorithm question In-Reply-To: <539D70BB.70203@freebsd.org> Message-ID: <20140615215526.U609@sola.nimnet.asn.au> References: <539C9BD5.70302@FreeBSD.org> <539D70BB.70203@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org, Michael Sierchio X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 12:01:07 -0000 On Sun, 15 Jun 2014 18:08:59 +0800, Julian Elischer wrote: > On 6/15/14, 3:00 AM, Alexander V. Chernikov wrote: > > On 14.06.2014 21:35, Michael Sierchio wrote: > > > Luigi - > > > > > > Does table entry matching use a longest prefix match? > > I'm not Luigi, but the answer is "yes" anyway :) > > this may be about to change, because tables are getting a rewrite, > but IP-based tables use the same code that the routing tables use. It'd be a bit anti-POLA for the longest prefix match behaviour to change, though, especially with some tablearg usage. Alexander? cheers, Ian From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 12:07:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23818218; Sun, 15 Jun 2014 12:07:01 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA3C52373; Sun, 15 Jun 2014 12:07:00 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Ww5Hr-0007b6-4R; Sun, 15 Jun 2014 11:55:35 +0400 Message-ID: <539D8BDD.2080104@FreeBSD.org> Date: Sun, 15 Jun 2014 16:04:45 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Ian Smith , Julian Elischer Subject: Re: ipfw table matching algorithm question References: <539C9BD5.70302@FreeBSD.org> <539D70BB.70203@freebsd.org> <20140615215526.U609@sola.nimnet.asn.au> In-Reply-To: <20140615215526.U609@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Michael Sierchio X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 12:07:01 -0000 On 15.06.2014 16:01, Ian Smith wrote: > On Sun, 15 Jun 2014 18:08:59 +0800, Julian Elischer wrote: > > On 6/15/14, 3:00 AM, Alexander V. Chernikov wrote: > > > On 14.06.2014 21:35, Michael Sierchio wrote: > > > > Luigi - > > > > > > > > Does table entry matching use a longest prefix match? > > > I'm not Luigi, but the answer is "yes" anyway :) > > > > this may be about to change, because tables are getting a rewrite, > > but IP-based tables use the same code that the routing tables use. > > It'd be a bit anti-POLA for the longest prefix match behaviour to > change, though, especially with some tablearg usage. Alexander? Well, "cidr" table are LPM by their nature. Additional algorithms for matching may be introduced (dxr, hashed tables for host-only prefixes) but it won't influence user-visible behavior for given type. > > cheers, Ian > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 12:36:25 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBB4AA58; Sun, 15 Jun 2014 12:36:25 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 363692641; Sun, 15 Jun 2014 12:36:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s5FCaKls066941; Sun, 15 Jun 2014 22:36:20 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sun, 15 Jun 2014 22:36:20 +1000 (EST) From: Ian Smith To: "Alexander V. Chernikov" Subject: Re: ipfw table matching algorithm question In-Reply-To: <539D8BDD.2080104@FreeBSD.org> Message-ID: <20140615221726.P609@sola.nimnet.asn.au> References: <539C9BD5.70302@FreeBSD.org> <539D70BB.70203@freebsd.org> <20140615215526.U609@sola.nimnet.asn.au> <539D8BDD.2080104@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Michael Sierchio , freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 12:36:25 -0000 On Sun, 15 Jun 2014 16:04:45 +0400, Alexander V. Chernikov wrote: > On 15.06.2014 16:01, Ian Smith wrote: > > On Sun, 15 Jun 2014 18:08:59 +0800, Julian Elischer wrote: > > > On 6/15/14, 3:00 AM, Alexander V. Chernikov wrote: > > > > On 14.06.2014 21:35, Michael Sierchio wrote: > > > > > Luigi - > > > > > > > > > > Does table entry matching use a longest prefix match? > > > > I'm not Luigi, but the answer is "yes" anyway :) > > > > > > this may be about to change, because tables are getting a rewrite, > > > but IP-based tables use the same code that the routing tables use. > > > > It'd be a bit anti-POLA for the longest prefix match behaviour to > > change, though, especially with some tablearg usage. Alexander? > Well, "cidr" table are LPM by their nature. > Additional algorithms for matching may be introduced (dxr, hashed tables for > host-only prefixes) > but it won't influence user-visible behavior for given type. Thanks for the confirmation .. nothing too scary so far, then .. but in what manner do you see integrating DXR into firewall table usage? cheers, Ian From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 12:40:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17828B03; Sun, 15 Jun 2014 12:40:20 +0000 (UTC) Received: from golf.bouncedomains.com (golf.bouncedomains.com [103.18.252.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C80EA266A; Sun, 15 Jun 2014 12:40:19 +0000 (UTC) Received: from [103.18.252.45] (port=59465 helo=[192.168.77.151]) by golf.bouncedomains.com with esmtpa (Exim 4.82) (envelope-from ) id 1Ww9IG-0006Xp-QB; Sun, 15 Jun 2014 22:12:17 +1000 User-Agent: Microsoft-MacOutlook/14.3.8.130913 Date: Sun, 15 Jun 2014 21:09:56 +1000 Subject: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? From: Mark van der Meulen To: Message-ID: Thread-Topic: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? Mime-version: 1.0 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - golf.bouncedomains.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - fivenynes.com X-Get-Message-Sender-Via: golf.bouncedomains.com: authenticated_id: mark@fivenynes.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-bugs@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 12:40:20 -0000 Hi List, I=B9m wondering if anyone can help me with this problem or at least help point me in the direction of where to start looking? I have FreeBSD 9 based servers which are crashing every 4-10 days and producing crash dumps similar to this one: http://pastebin.com/F82Jc08C All crash dumps seem to involve the net graph code and the current process is always ng_queueX. In summary, we have 4 x FreeBSD server running as LNS(MPD5) for around 2000 subscribers with 3 of the servers running a modified version of BSDRP, the fourth running a FreeBSD 9 install with what I thought was the latest stable source for the kernel because I fetched it from stable/9 however it shows up as 9.3-BETA in uname(the linked crash dump is from that server). 3 x LNS running modified BSDRP: DELL PowerEdge 2950, 2 x Xeon E5320, 4GB RAM, igb Quad Port NIC in LAGG, Quagga, MPD5, IPFW for Host Access Control, NTPD, BSNMPD 1 x LNS running latest FreeBSD 9 code: HP ProLiant DL380, 2 x Xeon X5465, 36GB RAM, em Quad Port NIC in LAGG, BIRD, MPD5, IPFW for Host Access Control, NTPD, BSNMPD The reason I built the fresh server on FreeBSD 9 is because I cannot save crash dumps for BSDRP easily. In short the problem is this =AD servers with 10-50 clients will run indefinitely(as long as we have had them, which is probably about 1.5 years) without errors and serve clients fine, however any with over 300 clients appear to only stay online for 4-10 days maximum before crashing and rebooting. I have attached the crash file from the latest crash on the LNS running the latest FreeBSD 9 code however unsure what to do with it and where to look? When these devices crash they are often doing in excess of 200Mbps(anywhere between 200Mbps and 450Mbps), very little load(3-4.5 on the first 3, less than 2 on the fourth). Things I=B9ve done to attempt resolution: - Replaced bce network cards with em network cards. This produced far less errors on the interfaces(was many before, now none) and I think caused the machines to stay up longer between reboots as before it would happen up to once a day. - Replaced em network cards with igb network cards. All this did was lower load and give us a little more time between reboots. - Tried an implementation using FreeBSD 10(this lasted less than 4 hours before reboots when under load) - Replaced memory - Increased memory on LNS4 to 36GB. - Various kernel rebuilds - Tweaked various kernel settings. This appears to have helped a little and given us more time between reboots. - Disabled IPv6 - Disabled IPFW - Disabled BSNMPD - Disabled Netflow - Versions 5.6 and 5.7 of MPD5 Anyone able to help me work out what the crash dump means? It only happens on servers running MPD5 (eg. Exact same boxes, exact same code pushing 800Mbps+ of routing and no crashes) and I can see the crash relates to net graph, however unsure where to go from there=8A Thanks, Mark Relevant Current Settings: net.inet.ip.fastforwarding=3D1 net.inet.ip.fw.default_to_accept=3D1 net.bpf.zerocopy_enable=3D1 net.inet.raw.maxdgram=3D16384 net.inet.raw.recvspace=3D16384 hw.intr_storm_threshold=3D64000 net.inet.ip.fastforwarding=3D1 net.inet.ip.fw.default_to_accept=3D1 net.inet.ip.intr_queue_maxlen=3D10240 net.inet.ip.redirect=3D0 net.inet.ip.sourceroute=3D0 net.inet.ip.rtexpire=3D2 net.inet.ip.rtminexpire=3D2 net.inet.ip.rtmaxcache=3D256 net.inet.ip.accept_sourceroute=3D0 net.inet.ip.process_options=3D0 net.inet.icmp.log_redirect=3D0 net.inet.icmp.drop_redirect=3D1 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.blackhole=3D2 net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.sendbuf_auto=3D1 net.inet.tcp.recvbuf_auto=3D1 net.inet.udp.recvspace=3D262144 net.inet.udp.blackhole=3D0 net.inet.udp.maxdgram=3D57344 net.route.netisr_maxqlen=3D4096 net.local.stream.recvspace=3D65536 net.local.stream.sendspace=3D65536 net.graph.maxdata=3D65536 net.graph.maxalloc=3D65536 net.graph.maxdgram=3D2096000 net.graph.recvspace=3D2096000 kern.ipc.somaxconn=3D32768 kern.ipc.nmbclusters=3D524288 kern.ipc.maxsockbuf=3D26214400 kern.ipc.shmmax=3D=B32147483648" kern.ipc.nmbjumbop=3D=B353200" kern.ipc.maxpipekva=3D=B3536870912" kern.random.sys.harvest.ethernet=3D"0" kern.random.sys.harvest.interrupt=3D"0" vm.kmem_size=3D=B34096M=B2 # Only on box with over 12G RAM. Otherwise 2G. vm.kmem_size_max=3D=B38192M" # Only on box with over 12G RAM. hw.igb.rxd=3D"4096" hw.igb.txd=3D"4096" hw.em.rxd=3D"4096" hw.em.txd=3D"4096" hw.igb.max_interrupt_rate=3D=B332000" hw.igb.rx_process_limit=3D"4096" hw.em.rx_process_limit=3D"500" net.link.ifqmaxlen=3D"20480" net.isr.dispatch=3D"direct" net.isr.direct_force=3D"1" net.isr.direct=3D"1" net.isr.maxthreads=3D"8" net.isr.numthreads=3D"4" net.isr.bindthreads=3D"1" net.isr.maxqlimit=3D"20480" net.isr.defaultqlimit=3D"8192" =20 From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 12:46:00 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45319F12; Sun, 15 Jun 2014 12:46:00 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0593D272A; Sun, 15 Jun 2014 12:46:00 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Ww5tc-00080f-M8; Sun, 15 Jun 2014 12:34:36 +0400 Message-ID: <539D9502.5080102@FreeBSD.org> Date: Sun, 15 Jun 2014 16:43:46 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Ian Smith Subject: Re: ipfw table matching algorithm question References: <539C9BD5.70302@FreeBSD.org> <539D70BB.70203@freebsd.org> <20140615215526.U609@sola.nimnet.asn.au> <539D8BDD.2080104@FreeBSD.org> <20140615221726.P609@sola.nimnet.asn.au> In-Reply-To: <20140615221726.P609@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Michael Sierchio , freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 12:46:00 -0000 On 15.06.2014 16:36, Ian Smith wrote: > On Sun, 15 Jun 2014 16:04:45 +0400, Alexander V. Chernikov wrote: > > On 15.06.2014 16:01, Ian Smith wrote: > > > On Sun, 15 Jun 2014 18:08:59 +0800, Julian Elischer wrote: > > > > On 6/15/14, 3:00 AM, Alexander V. Chernikov wrote: > > > > > On 14.06.2014 21:35, Michael Sierchio wrote: > > > > > > Luigi - > > > > > > > > > > > > Does table entry matching use a longest prefix match? > > > > > I'm not Luigi, but the answer is "yes" anyway :) > > > > > > > > this may be about to change, because tables are getting a rewrite, > > > > but IP-based tables use the same code that the routing tables use. > > > > > > It'd be a bit anti-POLA for the longest prefix match behaviour to > > > change, though, especially with some tablearg usage. Alexander? > > Well, "cidr" table are LPM by their nature. > > Additional algorithms for matching may be introduced (dxr, hashed tables for > > host-only prefixes) > > but it won't influence user-visible behavior for given type. > > Thanks for the confirmation .. nothing too scary so far, then .. but in > what manner do you see integrating DXR into firewall table usage? DXR (or any other similar algorithm) returns next hop index in case of match. The only thing that is needed - is to add another indirection table w idx -> u32 values (or even IPv6 nexthop values). You can take a look at current radix bindings here: http://svnweb.freebsd.org/base/projects/ipfw/sys/netpfil/ipfw/ip_fw_table_algo.c?revision=267469&view=markup > > cheers, Ian > From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 13:30:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A11B976F; Sun, 15 Jun 2014 13:30:51 +0000 (UTC) Received: from zimbra.nitronet.pl (zimbra.nitronet.pl [79.98.150.2]) by mx1.freebsd.org (Postfix) with ESMTP id 57F712A96; Sun, 15 Jun 2014 13:30:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id EFFC660E81; Sun, 15 Jun 2014 15:35:22 +0200 (CEST) Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Ct2To1CK5UO9; Sun, 15 Jun 2014 15:35:20 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id 8B44063584; Sun, 15 Jun 2014 15:35:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.nitronet.pl Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vc4PaLflHz7H; Sun, 15 Jun 2014 15:35:20 +0200 (CEST) Received: from hC35A6B23.cli.nitronet.pl (hC35A6B23.cli.nitronet.pl [195.90.107.35]) by zimbra.nitronet.pl (Postfix) with ESMTPSA id 6AB1560E81; Sun, 15 Jun 2014 15:35:20 +0200 (CEST) Date: Sun, 15 Jun 2014 15:30:26 +0200 From: =?utf-8?Q?Pawe=C5=82_Tyll?= X-Priority: 3 (Normal) Message-ID: <1251795590.20140615153026@ofca.me> To: Mark van der Meulen Subject: Re: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, freebsd-bugs@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 13:30:51 -0000 Hello Mark, Sunday, June 15, 2014, 1:09:56 PM, you wrote: > I=B9m wondering if anyone can help me with this problem or at least help > point me in the direction of where to start looking? I have FreeBSD 9 > based servers which are crashing every 4-10 days and producing crash dumps > similar to this one: http://pastebin.com/F82Jc08C Just for the fun of it, add this to the template definition: set iface down-script mpd-down.sh mpd-down.sh contents being: #!/bin/sh /bin/sleep 1 I've seen panics related to queuing with dummynet (not dummynet's fault :P), and this awful hack worked around the real problem. Not sure if you're suffering from the same race condition tough :) From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 14:39:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0CF9FFE; Sun, 15 Jun 2014 14:39:33 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D32B2FE6; Sun, 15 Jun 2014 14:39:33 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id cc10so2854531wib.6 for ; Sun, 15 Jun 2014 07:39:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=negTkOTcAck1Pxn3npaKYpwgOqHurYW9M+1GUZTojfY=; b=Rc3bAabT4Q1lSfiw7swT4BqdWFnw8K1Dwgl3iv6rfflvvbz+Z8zutZnPGFfBQSfOz2 96W1/DmsjQPWieQYlZ2hErjHcv+xBXyeoSbzxRvh70/q3eiabsqmaSZzyTy4dutkZBIF LolruHvLPpuYzkV0AhbkV04wkVT/UOZpB0T/PGFFASMeUQL9ujIxLonvjWtkFw23cOdq uxER9ELJPdY5eYmqmDJ96AdAqHjEZsFVasqltU96JGvsBf+hfy1s2U9Mgl8jt6Nz3+8b d99mkJJkschimo3pu0lSx2no06THU4xpgaTMjnA2z2GmwmM/bXr9gym7PqCUJK5cR2Mt Um3Q== X-Received: by 10.194.161.168 with SMTP id xt8mr20559495wjb.35.1402843171634; Sun, 15 Jun 2014 07:39:31 -0700 (PDT) Received: from [192.168.2.30] ([2.176.164.233]) by mx.google.com with ESMTPSA id b44sm26985828eem.45.2014.06.15.07.39.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 15 Jun 2014 07:39:30 -0700 (PDT) Message-ID: <539DB018.5020702@gmail.com> Date: Sun, 15 Jun 2014 19:09:20 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Mark van der Meulen Subject: Re: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org, freebsd-bugs@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 14:39:34 -0000 On 6/15/2014 3:39 PM, Mark van der Meulen wrote: > Hi List, > > Iđm wondering if anyone can help me with this problem or at least help > point me in the direction of where to start looking? I have FreeBSD 9 > based servers which are crashing every 4-10 days and producing crash dumps > similar to this one: http://pastebin.com/F82Jc08C > > All crash dumps seem to involve the net graph code and the current process > is always ng_queueX. > > In summary, we have 4 x FreeBSD server running as LNS(MPD5) for around > 2000 subscribers with 3 of the servers running a modified version of > BSDRP, the fourth running a FreeBSD 9 install with what I thought was the > latest stable source for the kernel because I fetched it from stable/9 > however it shows up as 9.3-BETA in uname(the linked crash dump is from > that server). > > 3 x LNS running modified BSDRP: DELL PowerEdge 2950, 2 x Xeon E5320, 4GB > RAM, igb Quad Port NIC in LAGG, Quagga, MPD5, IPFW for Host Access > Control, NTPD, BSNMPD > 1 x LNS running latest FreeBSD 9 code: HP ProLiant DL380, 2 x Xeon X5465, > 36GB RAM, em Quad Port NIC in LAGG, BIRD, MPD5, IPFW for Host Access > Control, NTPD, BSNMPD > > The reason I built the fresh server on FreeBSD 9 is because I cannot save > crash dumps for BSDRP easily. In short the problem is this ­ servers with > 10-50 clients will run indefinitely(as long as we have had them, which is > probably about 1.5 years) without errors and serve clients fine, however > any with over 300 clients appear to only stay online for 4-10 days maximum > before crashing and rebooting. I have attached the crash file from the > latest crash on the LNS running the latest FreeBSD 9 code however unsure > what to do with it and where to look? > > When these devices crash they are often doing in excess of > 200Mbps(anywhere between 200Mbps and 450Mbps), very little load(3-4.5 on > the first 3, less than 2 on the fourth). > > Things Iđve done to attempt resolution: > > - Replaced bce network cards with em network cards. This produced far less > errors on the interfaces(was many before, now none) and I think caused the > machines to stay up longer between reboots as before it would happen up to > once a day. > - Replaced em network cards with igb network cards. All this did was lower > load and give us a little more time between reboots. > - Tried an implementation using FreeBSD 10(this lasted less than 4 hours > before reboots when under load) > - Replaced memory > - Increased memory on LNS4 to 36GB. > - Various kernel rebuilds > - Tweaked various kernel settings. This appears to have helped a little > and given us more time between reboots. > - Disabled IPv6 > - Disabled IPFW > - Disabled BSNMPD > - Disabled Netflow > - Versions 5.6 and 5.7 of MPD5 > > Anyone able to help me work out what the crash dump means? It only happens > on servers running MPD5 (eg. Exact same boxes, exact same code pushing > 800Mbps+ of routing and no crashes) and I can see the crash relates to net > graph, however unsure where to go from thereŠ > > Thanks, > > Mark > > > Relevant Current Settings: > > net.inet.ip.fastforwarding=1 > net.inet.ip.fw.default_to_accept=1 > net.bpf.zerocopy_enable=1 > net.inet.raw.maxdgram=16384 > net.inet.raw.recvspace=16384 > hw.intr_storm_threshold=64000 > net.inet.ip.fastforwarding=1 > net.inet.ip.fw.default_to_accept=1 > net.inet.ip.intr_queue_maxlen=10240 > net.inet.ip.redirect=0 > net.inet.ip.sourceroute=0 > net.inet.ip.rtexpire=2 > net.inet.ip.rtminexpire=2 > net.inet.ip.rtmaxcache=256 > net.inet.ip.accept_sourceroute=0 > net.inet.ip.process_options=0 > net.inet.icmp.log_redirect=0 > net.inet.icmp.drop_redirect=1 > net.inet.tcp.drop_synfin=1 > net.inet.tcp.blackhole=2 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.recvbuf_max=16777216 > net.inet.tcp.sendbuf_auto=1 > net.inet.tcp.recvbuf_auto=1 > net.inet.udp.recvspace=262144 > net.inet.udp.blackhole=0 > net.inet.udp.maxdgram=57344 > net.route.netisr_maxqlen=4096 > net.local.stream.recvspace=65536 > net.local.stream.sendspace=65536 > net.graph.maxdata=65536 > net.graph.maxalloc=65536 > net.graph.maxdgram=2096000 > net.graph.recvspace=2096000 > kern.ipc.somaxconn=32768 > kern.ipc.nmbclusters=524288 > kern.ipc.maxsockbuf=26214400 > kern.ipc.shmmax=ģ2147483648" > kern.ipc.nmbjumbop=ģ53200" > kern.ipc.maxpipekva=ģ536870912" > kern.random.sys.harvest.ethernet="0" > kern.random.sys.harvest.interrupt="0" > vm.kmem_size=ģ4096Mē # Only on box with over 12G RAM. Otherwise 2G. > > > vm.kmem_size_max=ģ8192M" # Only on box with over 12G RAM. > hw.igb.rxd="4096" > hw.igb.txd="4096" > hw.em.rxd="4096" > hw.em.txd="4096" > hw.igb.max_interrupt_rate=ģ32000" > > hw.igb.rx_process_limit="4096" > hw.em.rx_process_limit="500" > net.link.ifqmaxlen="20480" > net.isr.dispatch="direct" > net.isr.direct_force="1" > net.isr.direct="1" > net.isr.maxthreads="8" > net.isr.numthreads="4" > net.isr.bindthreads="1" > net.isr.maxqlimit="20480" > net.isr.defaultqlimit="8192" > > The following workarounds have worked for some people. They may not solve your problem, but are worth giving a try: 1. Increases netgraph limits: net.graph.maxdata=262140 # /boot/loader.conf net.graph.maxalloc=262140 # /boot.loader.conf 2. Remove FLOWTABLE kernel option. It would also help if you put your kernel and core dump somewhere for download so we can have a closer look at panic trace. -- Best regards. Hooman Fazaeli From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 15:34:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B806AE20 for ; Sun, 15 Jun 2014 15:34:29 +0000 (UTC) Received: from emma.bazalt43.ru (emma.bazalt43.ru [91.193.143.8]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63D522458 for ; Sun, 15 Jun 2014 15:34:28 +0000 (UTC) Received: from [0.0.0.0] (oneex.me [91.193.143.91]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by emma.bazalt43.ru (Postfix) with ESMTPSA id 44F99172FA; Sun, 15 Jun 2014 21:25:56 +0600 (YEKT) Message-ID: <539DBB0D.1080502@hostelnet.ru> Date: Sun, 15 Jun 2014 21:26:05 +0600 From: noc@hostelnet.ru Reply-To: noc@hostelnet.ru User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Mark van der Meulen , freebsd-net@freebsd.org Subject: Re: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 15:34:29 -0000 15.06.2014 17:09, Mark van der Meulen ÐŋÐļŅˆÐĩŅ‚: > Hi List, > > IÂđm wondering if anyone can help me with this problem or at least help > point me in the direction of where to start looking? I have FreeBSD 9 > based servers which are crashing every 4-10 days and producing crash dumps > similar to this one: http://pastebin.com/F82Jc08C > Since we have similar situation - try use 8-stable. We used FreeBSD 9-stable (from r259072 to r262224) with 150-300 PPPoE tunnels (mpd 5.7) and it randomly crashing 1-2 times in month. Some times ago one server was rolled back to 8-stable (r259629) - it never crashed since december 2013, while on server with 9-stable already got 8 crash dumps. Both systems have same hardware, GENERIC kernels, same pkgs versions and same configs: /boot/loader.conf: net.isr.maxthreads=2 net.isr.numthreads=2 net.graph.maxdata=65536 net.graph.maxalloc=65536 net.isr.defaultqlimit=4096 net.link.ifqmaxlen=10240 /etc/sysctl.conf: net.inet.ip.fastforwarding=1 net.inet.ip.redirect=0 kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.point_to_point=0 kern.random.sys.harvest.interrupt=0 net.inet.raw.maxdgram=16384 net.inet.raw.recvspace=16384 net.inet.ip.intr_queue_maxlen=10240 net.route.netisr_maxqlen=4096 From owner-freebsd-net@FreeBSD.ORG Sun Jun 15 16:01:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8EF5808 for ; Sun, 15 Jun 2014 16:01:59 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A33082686 for ; Sun, 15 Jun 2014 16:01:59 +0000 (UTC) Received: from jre-mbp.elischer.org (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5FG1qmM060518 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 15 Jun 2014 09:01:56 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <539DC36A.4030309@freebsd.org> Date: Mon, 16 Jun 2014 00:01:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: FreeBSD 9 w/ MPD5 crashes as LNS with 300+ tunnels. Netgraph issue? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 16:02:00 -0000 On 6/15/14, 7:09 PM, Mark van der Meulen wrote: > Hi List, > > Iđm wondering if anyone can help me with this problem or at least help > point me in the direction of where to start looking? I have FreeBSD 9 > based servers which are crashing every 4-10 days and producing crash dumps > similar to this one: http://pastebin.com/F82Jc08C > > All crash dumps seem to involve the net graph code and the current process > is always ng_queueX. I looked at your trace. I see that you have good access to gdb .. can you show the exact C statements having the problems? and hte values of the variables concerned? Since I dont' have your sources I dont' knwo if line 3587 on your system matches line 3587 on fxp.watson.org http://fxr.watson.org/fxr/source/netgraph/ng_base.c?v=FREEBSD9#L3587 if it does, then can you look to see why it got into that clause. There are six different subconditions that could have made it go there. it woudl be instructive to know which triggered. That line in my sources is a TRAP_ERROR() which is defined to nothing, so it woudl be nice to see exactly where your gdb says it is. if it IS there and you have a remote(serial) gdb set up, you could try doing what the comment in the sources says: /* Set this to kdb_enter("X") to catch all errors as they occur */ #ifndef TRAP_ERROR #define TRAP_ERROR() #endif if you do NOT have serial set up, you could run the server on a byhve instance on a freebsd 10 system and set that up for serial debugging. but that may be quite a learning curve.. (I've never fully done that myself yet). You don't say how similar the traces are. Or how reproducible.. have you seen exactly the same trace more than once? Julian > In summary, we have 4 x FreeBSD server running as LNS(MPD5) for around > 2000 subscribers with 3 of the servers running a modified version of > BSDRP, the fourth running a FreeBSD 9 install with what I thought was the > latest stable source for the kernel because I fetched it from stable/9 > however it shows up as 9.3-BETA in uname(the linked crash dump is from > that server). > > 3 x LNS running modified BSDRP: DELL PowerEdge 2950, 2 x Xeon E5320, 4GB > RAM, igb Quad Port NIC in LAGG, Quagga, MPD5, IPFW for Host Access > Control, NTPD, BSNMPD > 1 x LNS running latest FreeBSD 9 code: HP ProLiant DL380, 2 x Xeon X5465, > 36GB RAM, em Quad Port NIC in LAGG, BIRD, MPD5, IPFW for Host Access > Control, NTPD, BSNMPD > > The reason I built the fresh server on FreeBSD 9 is because I cannot save > crash dumps for BSDRP easily. In short the problem is this ­ servers with > 10-50 clients will run indefinitely(as long as we have had them, which is > probably about 1.5 years) without errors and serve clients fine, however > any with over 300 clients appear to only stay online for 4-10 days maximum > before crashing and rebooting. I have attached the crash file from the > latest crash on the LNS running the latest FreeBSD 9 code however unsure > what to do with it and where to look? > > When these devices crash they are often doing in excess of > 200Mbps(anywhere between 200Mbps and 450Mbps), very little load(3-4.5 on > the first 3, less than 2 on the fourth). > > Things Iđve done to attempt resolution: > > - Replaced bce network cards with em network cards. This produced far less > errors on the interfaces(was many before, now none) and I think caused the > machines to stay up longer between reboots as before it would happen up to > once a day. > - Replaced em network cards with igb network cards. All this did was lower > load and give us a little more time between reboots. > - Tried an implementation using FreeBSD 10(this lasted less than 4 hours > before reboots when under load) > - Replaced memory > - Increased memory on LNS4 to 36GB. > - Various kernel rebuilds > - Tweaked various kernel settings. This appears to have helped a little > and given us more time between reboots. > - Disabled IPv6 > - Disabled IPFW > - Disabled BSNMPD > - Disabled Netflow > - Versions 5.6 and 5.7 of MPD5 > > Anyone able to help me work out what the crash dump means? It only happens > on servers running MPD5 (eg. Exact same boxes, exact same code pushing > 800Mbps+ of routing and no crashes) and I can see the crash relates to net > graph, however unsure where to go from thereS( > > Thanks, > > Mark > > > Relevant Current Settings: > > net.inet.ip.fastforwarding=1 > net.inet.ip.fw.default_to_accept=1 > net.bpf.zerocopy_enable=1 > net.inet.raw.maxdgram=16384 > net.inet.raw.recvspace=16384 > hw.intr_storm_threshold=64000 > net.inet.ip.fastforwarding=1 > net.inet.ip.fw.default_to_accept=1 > net.inet.ip.intr_queue_maxlen=10240 > net.inet.ip.redirect=0 > net.inet.ip.sourceroute=0 > net.inet.ip.rtexpire=2 > net.inet.ip.rtminexpire=2 > net.inet.ip.rtmaxcache=256 > net.inet.ip.accept_sourceroute=0 > net.inet.ip.process_options=0 > net.inet.icmp.log_redirect=0 > net.inet.icmp.drop_redirect=1 > net.inet.tcp.drop_synfin=1 > net.inet.tcp.blackhole=2 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.recvbuf_max=16777216 > net.inet.tcp.sendbuf_auto=1 > net.inet.tcp.recvbuf_auto=1 > net.inet.udp.recvspace=262144 > net.inet.udp.blackhole=0 > net.inet.udp.maxdgram=57344 > net.route.netisr_maxqlen=4096 > net.local.stream.recvspace=65536 > net.local.stream.sendspace=65536 > net.graph.maxdata=65536 > net.graph.maxalloc=65536 > net.graph.maxdgram=2096000 > net.graph.recvspace=2096000 > kern.ipc.somaxconn=32768 > kern.ipc.nmbclusters=524288 > kern.ipc.maxsockbuf=26214400 > kern.ipc.shmmax=ģ2147483648" > kern.ipc.nmbjumbop=ģ53200" > kern.ipc.maxpipekva=ģ536870912" > kern.random.sys.harvest.ethernet="0" > kern.random.sys.harvest.interrupt="0" > vm.kmem_size=ģ4096Mē # Only on box with over 12G RAM. Otherwise 2G. > > > vm.kmem_size_max=ģ8192M" # Only on box with over 12G RAM. > hw.igb.rxd="4096" > hw.igb.txd="4096" > hw.em.rxd="4096" > hw.em.txd="4096" > hw.igb.max_interrupt_rate=ģ32000" > > hw.igb.rx_process_limit="4096" > hw.em.rx_process_limit="500" > net.link.ifqmaxlen="20480" > net.isr.dispatch="direct" > net.isr.direct_force="1" > net.isr.direct="1" > net.isr.maxthreads="8" > net.isr.numthreads="4" > net.isr.bindthreads="1" > net.isr.maxqlimit="20480" > net.isr.defaultqlimit="8192" > > > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Jun 16 08:00:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F187192 for ; Mon, 16 Jun 2014 08:00:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8933231F for ; Mon, 16 Jun 2014 08:00:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5G80Cvs071836 for ; Mon, 16 Jun 2014 09:00:12 +0100 (BST) (envelope-from bz-noreply@freebsd.org) Message-Id: <201406160800.s5G80Cvs071836@kenobi.freebsd.org> From: bz-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 16 Jun 2014 08:00:12 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 08:00:13 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Jun 16 13:25:51 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1759538D for ; Mon, 16 Jun 2014 13:25:51 +0000 (UTC) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id D26E920F5 for ; Mon, 16 Jun 2014 13:25:50 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:1c4b:cac4:59c6:f7d9]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 784134AC0F for ; Mon, 16 Jun 2014 17:25:49 +0400 (MSK) Date: Mon, 16 Jun 2014 17:25:46 +0400 From: Lev Serebryakov Reply-To: lev@FreeBSD.org Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1863738251.20140616172546@serebryakov.spb.ru> To: freebsd-net@FreeBSD.org Subject: LEDBAT (RFC-6817)i n FreeBSD as mod_cc(9)? MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 13:25:51 -0000 Hello, Freebsd-net. It looks like, that some TCP connections could benefit from LEDBAT (RFC-6871) cognestion control algorithm (not all, of course, it should not be default). Also, it looks like Apple implements one (http://www.opensource.apple.com/source/xnu/xnu-1699.32.7/bsd/netinet/tcp_ledbat.c), but it uses much more "callbacks" from TCP/SCTP core to CC module, that FreeBSD has. Does somebody evaluate, is it possible to bring LEDBAT to FreeBSD? -- // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Mon Jun 16 15:31:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C6FAD03 for ; Mon, 16 Jun 2014 15:31:41 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD9F32D42 for ; Mon, 16 Jun 2014 15:31:40 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id w8so7511519qac.25 for ; Mon, 16 Jun 2014 08:31:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=4LP7xE1RmlL0u5Tsti6YopJ1p7zCzS6fMxShU7qlBb8=; b=BM6Tfbg/ZTjgEbQ/ltxhrSMhUlIzL1HuD4ce11Ircxn3v1pdg6k8J1fCQ7I9OPZF6/ bxwU6KP4orprhTjSqJnm/AFsNkSWX1/4hCtVs+k0LF3d0bQoS8EjEo7+OokZp3oINawc pJ6A0UH7xcC2uhvKFlTZE/n/bTHj748qxSrIEB3uw83f2bw9rdjwgGSEUo8HCIvlfGZS 6/g+IJsy2v7MsYFZoMOMgfbPA1X3ML8T6s+DpJ4YBkxwaW9wbXswOC699TBYfoUporL3 CSfSVggS/B4FB3wUIJYV9DLZkVVo/g1oj+TTZmmHY1kYil/KrlGO+mYbBncE3+kpyzM2 KsbA== X-Received: by 10.224.166.9 with SMTP id k9mr27357292qay.25.1402932699944; Mon, 16 Jun 2014 08:31:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Mon, 16 Jun 2014 08:30:59 -0700 (PDT) In-Reply-To: <20140614101523.GD22187@onelab2.iet.unipi.it> References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Mon, 16 Jun 2014 16:30:59 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 15:31:41 -0000 Ok, thanks for the enlightenment regarding the loss of performance. One question, just to be sure. Does the kernel module contains the VALA switch code? Or do I need to compile extra code to have the switch working? Also, where can I find the documentation to use the Vala Switch? Once again, thank you for the support. On 14 June 2014 11:15, Luigi Rizzo wrote: > On Fri, Jun 13, 2014 at 10:55:54AM +0100, Carlos Ferreira wrote: > > Hello Luigi (and to all) > > > > I was able to successfully compile the netmap module for OpenWRT but > > without drivers. According to the information in the README file which > > comes with the source, the drivers are not necessary but with some > "reduced > > performance". > > > > I would like to ask how much degradation should be expected. I would like > > to run some tests to see if everything is ok and if the port was > successful. > > At 10Gbit we are easily talking about a 5x loss of efficiency. > > At 1Gbit probably the gap is a bit smaller, also because a > lot of low-end device have severe bus bandwidth limitations. > > I would be interested in seeing what kind of performance you > get on your openwrt box with the following commands > > 1. just a sender on a software switch > pkt-gen -i vale0:a -f tx > > 2. sender and receiver on a software switch > pkt-gen -i vale0:a -f tx & > pkt-gen -i vale0:b -f rx & > > 3. sender and receiver on a netmap pipe > pkt-gen -i vale0:x{0 -f tx & > pkt-gen -i vale0:x}0 -f rx & > > 4. sending on a physical device (make sure something is attached > to the output port. Also this is tricky because many openwrt boxes > have a switch on the output so you need to use a unicast destination > MAC address) > > pkt-gen -i eth0 -f tx -D 00:11:22:33:44:55 > > cheers > luigi > > > > > Thank you for the help! > > > > > > > > On 12 June 2014 11:48, Carlos Ferreira wrote: > > > > > First of all, thank you for the quick answer! > > > > > > I will try it myself to compile just the netmap module without the > drivers > > > and report the results back to you. > > > > > > Once again, thank you! > > > > > > > > > On 12 June 2014 11:41, Luigi Rizzo wrote: > > > > > >> > > >> > > >> > > >> On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira < > carlosmf.pt@gmail.com> > > >> wrote: > > >> > > >>> Hello! > > >>> > > >>> First of all, to Luigi and the dev team, great piece of work that > netmap > > >>> is! This is a piece of software that I was looking for quite some > time. > > >>> Your team effort is appreciated! > > >>> > > >>> Now the question. > > >>> I know that this is a FreeBSD mailing list but I was wondering, > since you > > >>> have a PKGBUILD file for ArchLinux, could someone in your team do the > > >>> same > > >>> for OpenWRT? Netmap seems to be a piece of software which would come > in > > >>> handy, to develop applications and protocols for the OpenWRT distro. > > >>> Since OpenWRT uses uClib, I don't really know if it is possible to > > >>> compile > > >>> it. > > >>> Help is appreciated! > > >>> > > >>> > > >> ???i expect it will be trivial to compile netmap for openwrt, > > >> because it is just a piece of kernel code and the userspace > > >> has little or no dependencies. > > >> Another story is to write specific netmap extensions for the > > >> driver in use -- that might require a little bit of work > > >> though not much, knowing the driver. > > >> > > >> I am afraid we currently we do not have the time and manpower > > >> to set up a build environment and give it a try. > > >> > > >> cheers > > >> luigi > > >> > > >> > > > > > > > > > -- > > > > > > Carlos Miguel Ferreira > > > Researcher at Telecommunications Institute > > > Aveiro - Portugal > > > Work E-mail - cmf@av.it.pt > > > Skype & GTalk -> carlosmf.pt@gmail.com > > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > > > > > > > > > > -- > > > > Carlos Miguel Ferreira > > Researcher at Telecommunications Institute > > Aveiro - Portugal > > Work E-mail - cmf@av.it.pt > > Skype & GTalk -> carlosmf.pt@gmail.com > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Mon Jun 16 22:53:28 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95EE552F for ; Mon, 16 Jun 2014 22:53:28 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D8FA2786 for ; Mon, 16 Jun 2014 22:53:28 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5GMrS6d059673 for ; Mon, 16 Jun 2014 23:53:28 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 184311] [bge] [panic] kernel panic with bge(4) on SunFire X2100 Date: Mon, 16 Jun 2014 22:53:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: shurd@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 22:53:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184311 Stephen Hurd changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |shurd@FreeBSD.org --- Comment #3 from Stephen Hurd --- Part of the problem may be that the device is being detected incorrectly. I'm pretty sure the 5750 was never sold to customers, and it was a PCI device, not PCI-E. Possibly this is a actually 5751? Could one of you post the pciconf information? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 16 23:43:07 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70CB11C2 for ; Mon, 16 Jun 2014 23:43:07 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 578862BA9 for ; Mon, 16 Jun 2014 23:43:07 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5GNh71a036334 for ; Tue, 17 Jun 2014 00:43:07 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 184311] [bge] [panic] kernel panic with bge(4) on SunFire X2100 Date: Mon, 16 Jun 2014 23:43:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: 000.fbsd@quip.cz X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 23:43:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184311 Miroslav Lachman <000.fbsd@quip.cz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |000.fbsd@quip.cz --- Comment #4 from Miroslav Lachman <000.fbsd@quip.cz> --- Are you talking about SunFire X2100 or SunFire X2100 M2? Because "X2100" (withou M2) has only one bge NIC identified as BCM5721 and one nVidia CK804 NIC. "X2100 M2" has two bge BCM5714 and two nfe NVIDIA nForce MCP55 Networking Adapter X2100 M2 - FreeBSD 8.4-RELEASE-p9 i386 GENERIC from /var/run/dmesg.boot bge0: mem 0xfdff0000-0xfdffffff,0xfdfe0000-0xfdfeffff irq 17 at device 4.0 on pci6 bge0: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus2: on bge0 brgphy0: PHY 1 on miibus2 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:1b:24:xx:xx:xx bge0: [ITHREAD] bge1: mem 0xfdfc0000-0xfdfcffff,0xfdfb0000-0xfdfbffff irq 18 at device 4.1 on pci6 bge1: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus3: on bge1 brgphy1: PHY 1 on miibus3 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:1b:24:xx:xx:xx bge1: [ITHREAD] pciconf bge0@pci0:6:4:0: class=0x020000 card=0x534c108e chip=0x167814e4 rev=0xa3 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5715C 10/100/100 PCIe Ethernet Controller' class = network subclass = ethernet bge1@pci0:6:4:1: class=0x020000 card=0x534c108e chip=0x167814e4 rev=0xa3 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5715C 10/100/100 PCIe Ethernet Controller' class = network subclass = ethernet Note that pciconf says BCM5715C while dmesg shows BCM5714 X2100 M2 - FreeBSD 9.2-RELEASE-p5 amd64 GENERIC bge0: mem 0xfdff0000-0xfdffffff,0xfdfe0000-0xfdfeffff irq 17 at device 4.0 on pci6 bge0: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus2: on bge0 brgphy0: PHY 1 on miibus2 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:1e:68:xx:xx:xx bge1: mem 0xfdfc0000-0xfdfcffff,0xfdfb0000-0xfdfbffff irq 18 at device 4.1 on pci6 bge1: CHIP ID 0x00009003; ASIC REV 0x09; CHIP REV 0x90; PCI-X 133 MHz miibus3: on bge1 brgphy1: PHY 1 on miibus3 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:1e:68:xx:xx:xx pciconf bge0@pci0:6:4:0: class=0x020000 card=0x534c108e chip=0x167814e4 rev=0xa3 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme BCM5715 Gigabit Ethernet' class = network subclass = ethernet bge1@pci0:6:4:1: class=0x020000 card=0x534c108e chip=0x167814e4 rev=0xa3 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme BCM5715 Gigabit Ethernet' class = network subclass = ethernet I am running "X2100 M2" without any problems on FreeBSD 8.4 and 9.2 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 00:07:32 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9354343A for ; Tue, 17 Jun 2014 00:07:32 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7AC1C2D4A for ; Tue, 17 Jun 2014 00:07:32 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5H07WBs097362 for ; Tue, 17 Jun 2014 01:07:32 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 184311] [bge] [panic] kernel panic with bge(4) on SunFire X2100 Date: Tue, 17 Jun 2014 00:07:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: shurd@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 00:07:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184311 --- Comment #5 from Stephen Hurd --- Both of the log bits in the reports contained the lines: bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus1: on bge0 brgphy0: PHY 1 on miibus1 Which is troubling since there shouldn't be any 5750s out there, and there absolutely aren't any on PCI-E busses. > Note that pciconf says BCM5715C while dmesg shows BCM5714 It also says 10/100/100 (missing zero there!). There shouldn't be any differences between the 5715 and 5714 as far as a driver behaviour goes, so this is likely a cosmetic issue. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 05:15:16 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96F89C6A for ; Tue, 17 Jun 2014 05:15:16 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EC5B2456 for ; Tue, 17 Jun 2014 05:15:16 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5H5FGTx055281 for ; Tue, 17 Jun 2014 06:15:16 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 175734] no ethernet detected on system with EG20T PCH chipset ATOM E6xx series Date: Tue, 17 Jun 2014 05:15:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kevlo@FreeBSD.org X-Bugzilla-Status: Timeout X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 05:15:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=175734 Kevin Lo changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kevlo@FreeBSD.org --- Comment #4 from Kevin Lo --- (In reply to Nick Hibma from comment #3) > We've successfully run FBSD10-STABLE on a EG20T chipset platform and used > its networking as well. You might want to try again. Really? Which device driver you use? Marius is right, there's no FreeBSD support for this device. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 07:41:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2C4840B; Tue, 17 Jun 2014 07:41:36 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 764732F00; Tue, 17 Jun 2014 07:41:36 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id hw13so7493016qab.20 for ; Tue, 17 Jun 2014 00:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JmtO1EEbH5b1x4yRj6O8P7NWFNJf1edNEDc7iKkfVHw=; b=ag4U6fujG4xjwWrAYNsLF02PbTrAAhK3WIc9163tDD6WVBGYKL68exr7Q8bZulclxx CE9i0xBHsOP+Vd0Nrlq72CwY4sFHZpnpSrDc62330ou6f0VA28VxoIAvVnw/K2GE8RC+ bNAhAAS64RYJt5aZZY+5bfUV7nwmnANxNvvlG5EJ3EwtHCg1bCMdSDQ15uzKBTay7fPY FEHzf+Qn65iOfOC3ZWisYfiVBtYHtkMxMriV5c5xIdz0pRPHQ7gN/8HZiq9xvaYPdpoN ubqJ57fM4RSQsF/0B3yHzyOMy3gc4qOvg0VhhF/+jYKdWHrdCOr7U9KkezIAIgTxAs0d 9QZg== MIME-Version: 1.0 X-Received: by 10.224.166.73 with SMTP id l9mr33135594qay.34.1402990895597; Tue, 17 Jun 2014 00:41:35 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Tue, 17 Jun 2014 00:41:35 -0700 (PDT) In-Reply-To: <1863738251.20140616172546@serebryakov.spb.ru> References: <1863738251.20140616172546@serebryakov.spb.ru> Date: Tue, 17 Jun 2014 00:41:35 -0700 Message-ID: Subject: Re: LEDBAT (RFC-6817)i n FreeBSD as mod_cc(9)? From: hiren panchasara To: Lev Serebryakov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 07:41:36 -0000 On Mon, Jun 16, 2014 at 6:25 AM, Lev Serebryakov wrote: > Hello, Freebsd-net. > > It looks like, that some TCP connections could benefit from LEDBAT > (RFC-6871) cognestion control algorithm (not all, of course, it should not > be default). > > Also, it looks like Apple implements one > (http://www.opensource.apple.com/source/xnu/xnu-1699.32.7/bsd/netinet/tcp_ledbat.c), > but it uses much more "callbacks" from TCP/SCTP core to CC module, that > FreeBSD has. > > Does somebody evaluate, is it possible to bring LEDBAT to FreeBSD? I'd guess there is nothing wrong in having this as a cc module. Someone has to do the necessary legwork :-) cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 08:48:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75C581C9 for ; Tue, 17 Jun 2014 08:48:50 +0000 (UTC) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 33F712494 for ; Tue, 17 Jun 2014 08:48:49 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:1c4b:cac4:59c6:f7d9]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 0483F4AC0C; Tue, 17 Jun 2014 12:48:46 +0400 (MSK) Date: Tue, 17 Jun 2014 12:48:43 +0400 From: Lev Serebryakov Reply-To: lev@FreeBSD.org Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1819611615.20140617124843@serebryakov.spb.ru> To: hiren panchasara Subject: Re: LEDBAT (RFC-6817)i n FreeBSD as mod_cc(9)? In-Reply-To: References: <1863738251.20140616172546@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 08:48:50 -0000 Hello, hiren. You wrote 17 =D0=B8=D1=8E=D0=BD=D1=8F 2014 =D0=B3., 11:41:35: >> It looks like, that some TCP connections could benefit from LEDBAT >> (RFC-6871) cognestion control algorithm (not all, of course, it should = not >> be default). >> >> Also, it looks like Apple implements one >> (http://www.opensource.apple.com/source/xnu/xnu-1699.32.7/bsd/netinet/= tcp_ledbat.c), >> but it uses much more "callbacks" from TCP/SCTP core to CC module, that >> FreeBSD has. >> >> Does somebody evaluate, is it possible to bring LEDBAT to FreeBSD? hp> I'd guess there is nothing wrong in having this as a cc module. hp> Someone has to do the necessary legwork :-) The problem is it seems that "someone" needs to extend set of hooks in mod_cc substantially. It is more than "write one more module" and, IMHO, should be discussed with net-related people :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 09:13:54 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83B7E8F7 for ; Tue, 17 Jun 2014 09:13:54 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AA93273B for ; Tue, 17 Jun 2014 09:13:54 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5H9DsuO001982 for ; Tue, 17 Jun 2014 10:13:54 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 175734] no ethernet detected on system with EG20T PCH chipset ATOM E6xx series Date: Tue, 17 Jun 2014 09:13:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: n_hibma@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 09:13:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=175734 Nick Hibma changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Timeout |In Discussion Resolution|Overcome By Events |--- --- Comment #5 from Nick Hibma --- My bad: We had some problems with this EG20T platform and I thought we resolved them all. We managed to get the 82574L (if_em) based adapters to work (invalid EEPROM checksum). I hadn't noticed the none2@pci0:2:0:1: class=0x020000 card=0x00000000 chip=0x88028086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Platform Controller Hub EG20T Gigabit Ethernet Controller' class = network subclass = ethernet which isn't wired up at all externally. Thanks, Kevin, for correcting me! Reopened. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 10:27:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8935D336 for ; Tue, 17 Jun 2014 10:27:27 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 496DC2E08 for ; Tue, 17 Jun 2014 10:27:27 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id r5so6041646qcx.39 for ; Tue, 17 Jun 2014 03:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Es8ix9szurAwy+WfiSLTQDAc1PVuUrGEbRnqO4kSEsc=; b=Iam/dGKccoNYo/dClNHVEKr2abGOJ4PxLN8ufUlwiS637w+8d7dflJ4ZBNbkBzaPQm 6X/6jKeP/4wxgSEWBYmfVGUYETf0bGWIkvkgXC9OXs8pxW/Za/BgrkV9B13hQsKcmOcF BarBHl/xzvw8lS9jHkBrE8WQZF5s9aW4DiMJ7yZLTGpEzjO/ogtb4uDywmOYeEcRT7iF LHDDviGMNVEpveEadccc609ZPfJD309YyiLfOamQp0Vlp92kXveBJht7X2XbHX7N9dWT wgWmTxM/OXM0CIob4Nt4tFEl+dkRv/JJ8fYvCSfIMHoJ45PO6MjLkWejlzpYQrDZxHxv Mdcg== MIME-Version: 1.0 X-Received: by 10.224.74.196 with SMTP id v4mr11607380qaj.104.1403000846445; Tue, 17 Jun 2014 03:27:26 -0700 (PDT) Received: by 10.229.232.74 with HTTP; Tue, 17 Jun 2014 03:27:26 -0700 (PDT) Date: Tue, 17 Jun 2014 14:57:26 +0430 Message-ID: Subject: netmap pktgen From: Mahnaz Talebi To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 10:27:27 -0000 Hi, I read in " http://typo3.change-project.eu/fileadmin/publications/Deliverables/CHANGE_Deliverable_D6_3.pdf", that: "Packets generated by the Netmap PktGen were always dropped by the Linux Kernel after being processed by the PREROUTING chain of the mangle table of Iptables." Is it true? Is there a way to avoid this? I want to forward netmap generated packets via a DUT device to receiver. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 10:54:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9DC9396C for ; Tue, 17 Jun 2014 10:54:27 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34A9A20D9 for ; Tue, 17 Jun 2014 10:54:27 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n3so5564046wiv.14 for ; Tue, 17 Jun 2014 03:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OSKmxNiQM/3STZGoWUJZTN9yvD+LC9XsZQU7lbJ5B/A=; b=sVkBF+VUiCUZiJj1NpRrooiXeLnGlE37qsWEIN6c32adhnUJ2driRIKdINtl3NHA8M LC7ZVSMas7SYBLcnhrbJ/2D+cVLWSQFQ1m4CTKcBjAebcD1W6QCjXj0CEjZ/uTbpxgv2 Lvjj5lYCzxvYsOL+EtgGeDLzGtzvoo30O10SmqZsRBv6bPVEvJ6jHlygs8gFqBrjMoTz 5PDKLWaYMasI3IEOpO5CcXolveOnZTKqxhRu0tmSKZZCC8fMhBOsgQ+RGgQ40eXL3/2s vyUJbpn8Ha8eGPDjJeRmMiPqIpG/GAJExKShll7YZ6AwUs7ECdDZeAhezOniWDYvAZzE Mlfw== MIME-Version: 1.0 X-Received: by 10.180.221.163 with SMTP id qf3mr35243445wic.56.1403002465421; Tue, 17 Jun 2014 03:54:25 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Tue, 17 Jun 2014 03:54:25 -0700 (PDT) In-Reply-To: References: Date: Tue, 17 Jun 2014 12:54:25 +0200 X-Google-Sender-Auth: jCgieXNAL7z6hp0TFAVsmW-0OpA Message-ID: Subject: Re: netmap pktgen From: Luigi Rizzo To: Mahnaz Talebi Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 10:54:27 -0000 On Tue, Jun 17, 2014 at 12:27 PM, Mahnaz Talebi wrote: > Hi, > > I read in " > > http://typo3.change-project.eu/fileadmin/publications/Deliverables/CHANGE= _Deliverable_D6_3.pdf > ", > that: "Packets generated > by the Netmap PktGen were always dropped by the Linux Kernel after being > processed by the > PREROUTING chain of the mangle table of Iptables." > Is it true? > Is there a way to avoid this? I want to forward netmap generated packets > via a DUT device to receiver. > =E2=80=8BGeneral comment:=E2=80=8B =E2=80=8Bthis type of questions sounds very much like the "studies on the internet say.." line from Dilbert strips. You are taking a random sentence from a random document, the conclusions drawn by the author (assuming they are correct, i have no idea) may depend so much on testing conditions that it is pointless to reason how they apply to your case. Please run _your_ experiment in the configuration that you care about, then if you have any issues report them and your configuration. That is something we can discuss about. =E2=80=8Bcheers luigi=E2=80=8B From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 11:01:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 163CDA8F for ; Tue, 17 Jun 2014 11:01:18 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5ED02196 for ; Tue, 17 Jun 2014 11:01:17 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id k15so9030636qaq.2 for ; Tue, 17 Jun 2014 04:01:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=X+OuG6QIoXcsNtgQn8CFusibRd4tvYtZ/9UcCA3bUxQ=; b=nHHu8SSbg3yOMZY7kVksXzURqL/sw8+N3QyXksISMeymclsQbmNxyrKH62BynM2qM/ AMiGEfqYgDKSLaq5RR7guEAzo4KusT4/EQeCRBt4n0v3qOXQt1lqzyhAScjo4zPw99Pb DuITMVj/q0xQbbpmRG+IayVLmqVpjjKjDH769Tarq8ZRuxOIWthqh/Q/ekFpsoVMfsVv C2eFc/5S1i92kw1ZkaPvd6WDJoS9PfKUNaZ1/UdmWS5Jo1NBEIlyswMhCz0JzDxCE9ae 7U4DHc+HiQ0HTHM1RvQlHMmikUi2tFOBS8WiDA5aqT+JorZaOLcJuFTTqBut5rT5Id5t fFxg== X-Received: by 10.140.41.213 with SMTP id z79mr33138547qgz.20.1403002876921; Tue, 17 Jun 2014 04:01:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Tue, 17 Jun 2014 04:00:36 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Tue, 17 Jun 2014 12:00:36 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 11:01:18 -0000 Just a remainder, so that the question doesn't get lost in time. On 16 June 2014 16:30, Carlos Ferreira wrote: > Ok, thanks for the enlightenment regarding the loss of performance. > > One question, just to be sure. Does the kernel module contains the VALA > switch code? Or do I need to compile extra code to have the switch working? > Also, where can I find the documentation to use the Vala Switch? > > Once again, thank you for the support. > > > On 14 June 2014 11:15, Luigi Rizzo wrote: > >> On Fri, Jun 13, 2014 at 10:55:54AM +0100, Carlos Ferreira wrote: >> > Hello Luigi (and to all) >> > >> > I was able to successfully compile the netmap module for OpenWRT but >> > without drivers. According to the information in the README file which >> > comes with the source, the drivers are not necessary but with some >> "reduced >> > performance". >> > >> > I would like to ask how much degradation should be expected. I would >> like >> > to run some tests to see if everything is ok and if the port was >> successful. >> >> At 10Gbit we are easily talking about a 5x loss of efficiency. >> >> At 1Gbit probably the gap is a bit smaller, also because a >> lot of low-end device have severe bus bandwidth limitations. >> >> I would be interested in seeing what kind of performance you >> get on your openwrt box with the following commands >> >> 1. just a sender on a software switch >> pkt-gen -i vale0:a -f tx >> >> 2. sender and receiver on a software switch >> pkt-gen -i vale0:a -f tx & >> pkt-gen -i vale0:b -f rx & >> >> 3. sender and receiver on a netmap pipe >> pkt-gen -i vale0:x{0 -f tx & >> pkt-gen -i vale0:x}0 -f rx & >> >> 4. sending on a physical device (make sure something is attached >> to the output port. Also this is tricky because many openwrt boxes >> have a switch on the output so you need to use a unicast destination >> MAC address) >> >> pkt-gen -i eth0 -f tx -D 00:11:22:33:44:55 >> >> cheers >> luigi >> >> > >> > Thank you for the help! >> > >> > >> > >> > On 12 June 2014 11:48, Carlos Ferreira wrote: >> > >> > > First of all, thank you for the quick answer! >> > > >> > > I will try it myself to compile just the netmap module without the >> drivers >> > > and report the results back to you. >> > > >> > > Once again, thank you! >> > > >> > > >> > > On 12 June 2014 11:41, Luigi Rizzo wrote: >> > > >> > >> >> > >> >> > >> >> > >> On Thu, Jun 12, 2014 at 12:35 PM, Carlos Ferreira < >> carlosmf.pt@gmail.com> >> > >> wrote: >> > >> >> > >>> Hello! >> > >>> >> > >>> First of all, to Luigi and the dev team, great piece of work that >> netmap >> > >>> is! This is a piece of software that I was looking for quite some >> time. >> > >>> Your team effort is appreciated! >> > >>> >> > >>> Now the question. >> > >>> I know that this is a FreeBSD mailing list but I was wondering, >> since you >> > >>> have a PKGBUILD file for ArchLinux, could someone in your team do >> the >> > >>> same >> > >>> for OpenWRT? Netmap seems to be a piece of software which would >> come in >> > >>> handy, to develop applications and protocols for the OpenWRT distro. >> > >>> Since OpenWRT uses uClib, I don't really know if it is possible to >> > >>> compile >> > >>> it. >> > >>> Help is appreciated! >> > >>> >> > >>> >> > >> ???i expect it will be trivial to compile netmap for openwrt, >> > >> because it is just a piece of kernel code and the userspace >> > >> has little or no dependencies. >> > >> Another story is to write specific netmap extensions for the >> > >> driver in use -- that might require a little bit of work >> > >> though not much, knowing the driver. >> > >> >> > >> I am afraid we currently we do not have the time and manpower >> > >> to set up a build environment and give it a try. >> > >> >> > >> cheers >> > >> luigi >> > >> >> > >> >> > > >> > > >> > > -- >> > > >> > > Carlos Miguel Ferreira >> > > Researcher at Telecommunications Institute >> > > Aveiro - Portugal >> > > Work E-mail - cmf@av.it.pt >> > > Skype & GTalk -> carlosmf.pt@gmail.com >> > > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > >> > >> > >> > >> > -- >> > >> > Carlos Miguel Ferreira >> > Researcher at Telecommunications Institute >> > Aveiro - Portugal >> > Work E-mail - cmf@av.it.pt >> > Skype & GTalk -> carlosmf.pt@gmail.com >> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 11:55:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D5B9D77 for ; Tue, 17 Jun 2014 11:55:24 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D575D2649 for ; Tue, 17 Jun 2014 11:55:23 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id x12so6982299wgg.10 for ; Tue, 17 Jun 2014 04:55:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=o9gxSGErIX0MJjdZPYm46KvN3PEmvkU8q3CmQB4Q/cQ=; b=WzaJKRSfcOKS+xDvC/eilxqXQKZcsUQ92vSdLjRp6QXKbNXbVho6a6CG5GAspX1eZo 3sbKyP1USZsQaNz+x7W9Ti1ziFUSK6hxkxm1rRtsWnFRHRLTf3lpo1Xi+ZZ687qI+Jx0 cxfFewmhwQoe3DTzvMrIqvggx9lMTFERH8oXE9iZi3zxriAC70EGKPoJupTdy2pB9Idx 478W9RXZhqqSbj/mu6hzzscEL8PhA9sllTl181qrW6fPhBgE4iWHS1m1HBkyC5VEIuP8 SeD/HVGFnnI0UNkcXQ8FyW2ezgVLZ3ysN6Rd9oTgljpJNJd6yGJLyJH3kdnSbjchn/uU nLdg== MIME-Version: 1.0 X-Received: by 10.180.83.131 with SMTP id q3mr35666013wiy.31.1403006121905; Tue, 17 Jun 2014 04:55:21 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.194.246.130 with HTTP; Tue, 17 Jun 2014 04:55:21 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> Date: Tue, 17 Jun 2014 13:55:21 +0200 X-Google-Sender-Auth: t1TQrhjgklmulXmju6Bc9GF3Wpk Message-ID: Subject: Re: netmap From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 11:55:24 -0000 On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira wrote: > Ok, thanks for the enlightenment regarding the loss of performance. > > One question, just to be sure. Does the kernel module contains the VALA > switch code? Or do I need to compile extra code to have the switch workin= g? > Also, where can I find the documentation to use the Vala Switch? > =E2=80=8BVALE is part of the netmap kernel module, the only thing you need to know to use it is port names: you can have multiple switch instances with multiple ports each, valeX:Y means port Y on switch X, X and Y are arbitrary strings with the constraint that the whole name must fit 15 characters. Details in the netmap manpage cheers luigi From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 14:04:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0008C4EF for ; Tue, 17 Jun 2014 14:04:11 +0000 (UTC) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE5892319 for ; Tue, 17 Jun 2014 14:04:11 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id c9so10163448qcz.9 for ; Tue, 17 Jun 2014 07:04:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=++xkXSOi71yNi+U7ixo3IBqghh5tCAXteEydBaXJSvE=; b=tHjcAWqjwh6cx+eRl0McgbSMNgdh4Os5jUVMmgDbrcJ8FuqUiyhO/5Wz1UmDL1W6+A YjGLUJ9Ynemj3hHp2CL4Xe7X8N4rbG0iXXSuc1j/uYsD8l9XSy/0t0cuVuvL5Orefz6p xrpaDa6rLP8sWOJIPdz2WL9deeKjN2e+aHc/Fd2tNYY5M3HAv9aEL06PwXSLVYMP91kJ B86PrqMjwdhAYpYaHQen665XuO2psG58RvgK0M3MLN3CcPzLobHgg5+RTLlRkuB1LUwk IQ3IXWTmvDcX+BD1gQEps57x00z13jE65wwweFmGtb+c2lODaZhwBdcjDKCQ5QRgrMO+ 76zw== X-Received: by 10.224.11.137 with SMTP id t9mr37216245qat.4.1403013850526; Tue, 17 Jun 2014 07:04:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Tue, 17 Jun 2014 07:03:30 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Tue, 17 Jun 2014 15:03:30 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 14:04:12 -0000 Great! :) I will give you the results as soon as I can get them :) On 17 June 2014 12:55, Luigi Rizzo wrote: > On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira > wrote: > >> Ok, thanks for the enlightenment regarding the loss of performance. >> >> One question, just to be sure. Does the kernel module contains the VALA >> switch code? Or do I need to compile extra code to have the switch worki= ng? >> Also, where can I find the documentation to use the Vala Switch? >> > > =E2=80=8BVALE is part of the netmap kernel module, the only thing you nee= d > to know to use it is port names: > you can have multiple switch instances with multiple ports each, > > valeX:Y means port Y on switch X, X and Y are arbitrary strings > with the constraint that the whole name must fit 15 characters. > > Details in the netmap manpage > > cheers > luigi > > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 16:14:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F9F98EB; Tue, 17 Jun 2014 16:14:32 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E41F2FAC; Tue, 17 Jun 2014 16:14:31 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 896A99DCAA7; Tue, 17 Jun 2014 18:07:46 +0200 (CEST) From: Borja Marcos Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Unusable Emulex "oce" driver version in -STABLE, 9.3 and CURRENT Date: Tue, 17 Jun 2014 18:07:45 +0200 Message-Id: <8D93EDF7-AC0E-4707-AC62-55CBD5F3358D@sarenet.es> To: Stable Stable Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 16:14:32 -0000 Hello, This should be fixed before the release of 9.3, and of course as soon as = possible for the rest of the branches. At least under 10-STABLE the "oce" driver can cause a panic when there's = a lot of network traffic. I have been able to reproduce it using "iperf". Sometimes the panic can be triggered within seconds of = starting an iperf test. There's a new driver available for download on the Emulex site, version = 10.0.747.0, and it works perfectly. The source code compiles perfectly under FreeBSD 10-STABLE (at least recent versions since April) and the = kernel module loads without trouble. There's a bug report covering this but seems to be forgotten :/ https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183391 Thanks! Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 23:38:42 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41EB7843 for ; Tue, 17 Jun 2014 23:38:42 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28C442B91 for ; Tue, 17 Jun 2014 23:38:42 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5HNcggc028579 for ; Wed, 18 Jun 2014 00:38:42 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10gigabit networking problems with Emulex OCE 11102 CNA Date: Tue, 17 Jun 2014 23:38:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 23:38:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koobs@FreeBSD.org Version|unspecified |9.2-RELEASE --- Comment #7 from Kubilay Kocak --- Please attach: * /var/run/dmesg.boot output * pciconf pciconf -lveV -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 17 23:59:08 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA099C72 for ; Tue, 17 Jun 2014 23:59:08 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D08EB2D2C for ; Tue, 17 Jun 2014 23:59:08 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5HNx8cq083297 for ; Wed, 18 Jun 2014 00:59:08 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Tue, 17 Jun 2014 23:59:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 23:59:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mlavkin@gmail.com Summary|[oce] 10gigabit networking |[oce] 10GbE networking |problems with Emulex OCE |problems with Emulex OCE |11102 CNA |11102 CNA --- Comment #8 from Kubilay Kocak --- Pataki, Borjan, MLavkin, We need a confirmation that this issue (panic) is still present on 9.3-BETA3 in order to get this into 9.3-RELEASE. We have a very short window until -RELEASE, if one of you could provide that confirmation, it will go a long way. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 02:20:30 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68E1F2C5 for ; Wed, 18 Jun 2014 02:20:30 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50192277F for ; Wed, 18 Jun 2014 02:20:30 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5I2KUgb006007 for ; Wed, 18 Jun 2014 03:20:30 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 02:20:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pataki.antal@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 02:20:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #9 from pataki.antal@gmail.com --- (In reply to Kubilay Kocak from comment #7) > Please attach: > > * /var/run/dmesg.boot output > * pciconf pciconf -lveV Hi! I am so sorry, but we didn't kept the unusable system from 2013 october until today.... The system is upgraded to 9.2-STABLE and the Emulex cards fired out and replaced with Intel X520 cards. Like this, the system is operating good - in a production environment, so it is not possible to change the hardware or the operating system. So I can't send you the dmesg.boot from 2013 october with the emulex card on fbsd 9.2-RELEASE, I am sorry. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 02:23:27 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE5D7390 for ; Wed, 18 Jun 2014 02:23:27 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A559F2812 for ; Wed, 18 Jun 2014 02:23:27 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5I2NR6q069371 for ; Wed, 18 Jun 2014 03:23:27 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 02:23:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pataki.antal@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 02:23:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #10 from pataki.antal@gmail.com --- Like my previous reply, I can't confirm this, because we workarounded the bug in a way. (upgraded to freebsd 9.2-stable and the emulex cards are changed to intel x520 cards) I am so sorry, we couldn't wait for about 8 months for any solution, we needed to use the servers in production - we spent many thousands of dollars for that, not for keeping them in the corner. I hope you understood. (In reply to Kubilay Kocak from comment #8) > Pataki, Borjan, MLavkin, > > We need a confirmation that this issue (panic) is still present on 9.3-BETA3 > in order to get this into 9.3-RELEASE. > > We have a very short window until -RELEASE, if one of you could provide that > confirmation, it will go a long way. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 07:44:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63E75F8; Wed, 18 Jun 2014 07:44:37 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1FA762275; Wed, 18 Jun 2014 07:44:36 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 429329DD121; Wed, 18 Jun 2014 09:35:30 +0200 (CEST) Subject: Re: Unusable Emulex "oce" driver version in -STABLE, 9.3 and CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <8D93EDF7-AC0E-4707-AC62-55CBD5F3358D@sarenet.es> Date: Wed, 18 Jun 2014 09:35:26 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8D93EDF7-AC0E-4707-AC62-55CBD5F3358D@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org, Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 07:44:37 -0000 On Jun 17, 2014, at 6:07 PM, Borja Marcos wrote: >=20 > Hello, >=20 > This should be fixed before the release of 9.3, and of course as soon = as possible for the rest of the branches. >=20 > At least under 10-STABLE the "oce" driver can cause a panic when = there's a lot of network traffic. I have been able to reproduce it > using "iperf". Sometimes the panic can be triggered within seconds of = starting an iperf test. >=20 > There's a new driver available for download on the Emulex site, = version 10.0.747.0, and it works perfectly. The source code compiles = perfectly > under FreeBSD 10-STABLE (at least recent versions since April) and the = kernel module loads without trouble. >=20 > There's a bug report covering this but seems to be forgotten :/ >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183391 In my case, the problem manifests itself running 10. Emulex provides a = more recent driver which compiles perfectly and works perfectly (if we = can safely assume that a weekend saturating a couple of interfaces = runnning benchmarks with no glitches is a good hint). I am now installing the latest beta of 9.3 to see if I can reproduce the = panic. I will follow-up in a couple of hours if possible. Borja. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 12:17:11 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 302119F for ; Wed, 18 Jun 2014 12:17:11 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 154C72A56 for ; Wed, 18 Jun 2014 12:17:11 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5ICHBT2046867 for ; Wed, 18 Jun 2014 13:17:11 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 12:17:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 12:17:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #11 from borjam@sarenet.es --- Alright. I installed 9.3-BETA3 on a test machine. On 9.3 it DOES NOT PANIC but, anyway, the interface freezes and it doesn't work properly. It's cross connected to another machine using an Intel 10 Gbps adapter (ix driver). So I did: kldload oce ifconfig oce0 inet 10.0.0.2 netmask 255.255.255.0 I started iperf -s on both machines. (iperf -s) (/usr/ports/benchmarks/iperf3) iperf3 -N -c 10.0.0.2 -P 4 -t 120 iperf3 -N -c 10.0.0.1 -P 4 -t 120 The test started and it froze, unable to move more packets. It managed to connect and it started sending traffic. root@pruebas:~ # iperf3 -N -c 10.0.0.1 -P 4 -t 120 Connecting to host 10.0.0.1, port 5201 [ 4] local 10.0.0.2 port 58691 connected to 10.0.0.1 port 5201 [ 6] local 10.0.0.2 port 25671 connected to 10.0.0.1 port 5201 [ 8] local 10.0.0.2 port 30969 connected to 10.0.0.1 port 5201 [ 10] local 10.0.0.2 port 36802 connected to 10.0.0.1 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 150 MBytes 1.26 Gbits/sec [ 6] 0.00-1.00 sec 150 MBytes 1.25 Gbits/sec [ 8] 0.00-1.00 sec 150 MBytes 1.26 Gbits/sec [ 10] 0.00-1.00 sec 150 MBytes 1.26 Gbits/sec [SUM] 0.00-1.00 sec 599 MBytes 5.02 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 6] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 8] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 10] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [SUM] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [ 6] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [ 8] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [ 10] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [SUM] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - Trying pings from 10.0.0.1 (Intel) to 10.0.0.2 (Emulex) and running tcpdump on the Emulex I saw the ARP requests coming *but* no replies. I tried to ifconfig oce0 down and up. The ifconfig oce0 down took several seconds, showing an abnormal usage of CPU time in "top". The ifconfig oce0 up is stuck in an endless loop, taking 100% CPU. It has been for several minutes. (top) 913 root 1 20 0 18344K 1932K CPU18 18 0:00 100.00% ifconfig The second test I've done was similar, just iperf3 -s (server) on the Emulex machine, client on the Intel. Traffic froze in seconds. I am going to reboot, compile the new Emulex driver and try again. Followup will come in a couple of hours or so. root@pruebas:~ # uname -a FreeBSD pruebas 9.3-BETA3 FreeBSD 9.3-BETA3 #0 r267433: Fri Jun 13 01:48:48 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 (from /var/log/messages after loading the oce.ko module) Jun 18 13:50:13 pruebas kernel: oce0: mem 0xdf478000-0xdf47bfff,0xdf480000-0xdf49ffff,0xdf4a0000-0xdf4bffff irq 38 at device 0.0 on pci3 Jun 18 13:50:13 pruebas kernel: oce1: mem 0xdf47c000-0xdf47ffff,0xdf4c0000-0xdf4dffff,0xdf4e0000-0xdf4fffff irq 45 at device 0.1 on pci3 (sysctl dev.oce) root@pruebas:~ # sysctl dev.oce dev.oce.0.%desc: Emulex CNA NIC function:///10.0.664.0/// dev.oce.0.%driver: oce dev.oce.0.%location: slot=0 function=0 dev.oce.0.%pnpinfo: vendor=0x19a2 device=0x0700 subvendor=0x10df subdevice=0xe629 class=0x020000 dev.oce.0.%parent: pci3 dev.oce.0.component_revision: ///10.0.664.0/// dev.oce.0.firmware_version: 2.103.397.34 dev.oce.0.max_rsp_handled: 64 dev.oce.0.speed: 10000 dev.oce.0.loop_back: 0 dev.oce.0.fw_upgrade: dev.oce.0.sfp_vpd_dump: 0 dev.oce.0.stats.rx.total_pkts: 448304 dev.oce.0.stats.rx.total_bytes: 30414856 dev.oce.0.stats.rx.total_frags: 448304 dev.oce.0.stats.rx.total_mcast_pkts: 0 dev.oce.0.stats.rx.total_ucast_pkts: 448242 dev.oce.0.stats.rx.total_rxcp_errs: 0 dev.oce.0.stats.rx.pause_frames: 0 dev.oce.0.stats.rx.priority_pause_frames: 0 dev.oce.0.stats.rx.control_frames: 0 dev.oce.0.stats.rx.queue0.rx_pkts: 448304 dev.oce.0.stats.rx.queue0.rx_bytes: 30414856 dev.oce.0.stats.rx.queue0.rx_frags: 448304 dev.oce.0.stats.rx.queue0.rx_mcast_pkts: 0 dev.oce.0.stats.rx.queue0.rx_ucast_pkts: 448242 dev.oce.0.stats.rx.queue0.rxcp_err: 0 dev.oce.0.stats.rx.err.crc_errs: 0 dev.oce.0.stats.rx.err.pbuf_errors: 0 dev.oce.0.stats.rx.err.erx_errors: 0 dev.oce.0.stats.rx.err.alignment_errors: 0 dev.oce.0.stats.rx.err.in_range_errors: 0 dev.oce.0.stats.rx.err.out_range_errors: 0 dev.oce.0.stats.rx.err.frame_too_long: 0 dev.oce.0.stats.rx.err.address_match_errors: 0 dev.oce.0.stats.rx.err.dropped_too_small: 0 dev.oce.0.stats.rx.err.dropped_too_short: 0 dev.oce.0.stats.rx.err.dropped_header_too_small: 0 dev.oce.0.stats.rx.err.dropped_tcp_length: 0 dev.oce.0.stats.rx.err.dropped_runt: 0 dev.oce.0.stats.rx.err.ip_checksum_errs: 0 dev.oce.0.stats.rx.err.tcp_checksum_errs: 0 dev.oce.0.stats.rx.err.udp_checksum_errs: 0 dev.oce.0.stats.rx.err.fifo_overflow_drop: 0 dev.oce.0.stats.rx.err.input_fifo_overflow_drop: 0 dev.oce.0.stats.tx.total_tx_pkts: 38169 dev.oce.0.stats.tx.total_tx_bytes: 1269173326 dev.oce.0.stats.tx.total_tx_reqs: 38169 dev.oce.0.stats.tx.total_tx_stops: 3 dev.oce.0.stats.tx.total_tx_wrbs: 474180 dev.oce.0.stats.tx.total_tx_compl: 37745 dev.oce.0.stats.tx.total_ipv6_ext_hdr_tx_drop: 0 dev.oce.0.stats.tx.pauseframes: 0 dev.oce.0.stats.tx.priority_pauseframes: 0 dev.oce.0.stats.tx.controlframes: 0 dev.oce.0.stats.tx.queue0.tx_pkts: 38169 dev.oce.0.stats.tx.queue0.tx_bytes: 1269173326 dev.oce.0.stats.tx.queue0.tx_reqs: 38169 dev.oce.0.stats.tx.queue0.tx_stops: 3 dev.oce.0.stats.tx.queue0.tx_wrbs: 474180 dev.oce.0.stats.tx.queue0.tx_compl: 37745 dev.oce.0.stats.tx.queue0.ipv6_ext_hdr_tx_drop: 0 dev.oce.1.%desc: Emulex CNA NIC function:///10.0.664.0/// dev.oce.1.%driver: oce dev.oce.1.%location: slot=0 function=1 dev.oce.1.%pnpinfo: vendor=0x19a2 device=0x0700 subvendor=0x10df subdevice=0xe629 class=0x020000 dev.oce.1.%parent: pci3 dev.oce.1.component_revision: ///10.0.664.0/// dev.oce.1.firmware_version: 2.103.397.34 dev.oce.1.max_rsp_handled: 64 dev.oce.1.speed: 10000 dev.oce.1.loop_back: 0 dev.oce.1.fw_upgrade: dev.oce.1.sfp_vpd_dump: 0 dev.oce.1.stats.rx.total_pkts: 0 dev.oce.1.stats.rx.total_bytes: 0 dev.oce.1.stats.rx.total_frags: 0 dev.oce.1.stats.rx.total_mcast_pkts: 0 dev.oce.1.stats.rx.total_ucast_pkts: 0 dev.oce.1.stats.rx.total_rxcp_errs: 0 dev.oce.1.stats.rx.pause_frames: 0 dev.oce.1.stats.rx.priority_pause_frames: 0 dev.oce.1.stats.rx.control_frames: 0 dev.oce.1.stats.rx.queue0.rx_pkts: 0 dev.oce.1.stats.rx.queue0.rx_bytes: 0 dev.oce.1.stats.rx.queue0.rx_frags: 0 dev.oce.1.stats.rx.queue0.rx_mcast_pkts: 0 dev.oce.1.stats.rx.queue0.rx_ucast_pkts: 0 dev.oce.1.stats.rx.queue0.rxcp_err: 0 dev.oce.1.stats.rx.err.crc_errs: 0 dev.oce.1.stats.rx.err.pbuf_errors: 0 dev.oce.1.stats.rx.err.erx_errors: 0 dev.oce.1.stats.rx.err.alignment_errors: 0 dev.oce.1.stats.rx.err.in_range_errors: 0 dev.oce.1.stats.rx.err.out_range_errors: 0 dev.oce.1.stats.rx.err.frame_too_long: 0 dev.oce.1.stats.rx.err.address_match_errors: 0 dev.oce.1.stats.rx.err.dropped_too_small: 0 dev.oce.1.stats.rx.err.dropped_too_short: 0 dev.oce.1.stats.rx.err.dropped_header_too_small: 0 dev.oce.1.stats.rx.err.dropped_tcp_length: 0 dev.oce.1.stats.rx.err.dropped_runt: 0 dev.oce.1.stats.rx.err.ip_checksum_errs: 0 dev.oce.1.stats.rx.err.tcp_checksum_errs: 0 dev.oce.1.stats.rx.err.udp_checksum_errs: 0 dev.oce.1.stats.rx.err.fifo_overflow_drop: 0 dev.oce.1.stats.rx.err.input_fifo_overflow_drop: 0 dev.oce.1.stats.tx.total_tx_pkts: 0 dev.oce.1.stats.tx.total_tx_bytes: 0 dev.oce.1.stats.tx.total_tx_reqs: 0 dev.oce.1.stats.tx.total_tx_stops: 0 dev.oce.1.stats.tx.total_tx_wrbs: 0 dev.oce.1.stats.tx.total_tx_compl: 0 dev.oce.1.stats.tx.total_ipv6_ext_hdr_tx_drop: 0 dev.oce.1.stats.tx.pauseframes: 0 dev.oce.1.stats.tx.priority_pauseframes: 0 dev.oce.1.stats.tx.controlframes: 0 dev.oce.1.stats.tx.queue0.tx_pkts: 0 dev.oce.1.stats.tx.queue0.tx_bytes: 0 dev.oce.1.stats.tx.queue0.tx_reqs: 0 dev.oce.1.stats.tx.queue0.tx_stops: 0 dev.oce.1.stats.tx.queue0.tx_wrbs: 0 dev.oce.1.stats.tx.queue0.tx_compl: 0 dev.oce.1.stats.tx.queue0.ipv6_ext_hdr_tx_drop: 0 Machine information: root@pruebas:~ # pciconf -lveV hostb0@pci0:0:0:0: class=0x060000 card=0x02f11028 chip=0x34038086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5500 I/O Hub to ESI Port' class = bridge subclass = HOST-PCI pcib1@pci0:0:1:0: class=0x060400 card=0x02f11028 chip=0x34088086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub PCI Express Root Port 1' class = bridge subclass = PCI-PCI pcib2@pci0:0:3:0: class=0x060400 card=0x02f11028 chip=0x340a8086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub PCI Express Root Port 3' class = bridge subclass = PCI-PCI pcib3@pci0:0:7:0: class=0x060400 card=0x02f11028 chip=0x340e8086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub PCI Express Root Port 7' class = bridge subclass = PCI-PCI pcib4@pci0:0:9:0: class=0x060400 card=0x02f11028 chip=0x34108086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub PCI Express Root Port 9' class = bridge subclass = PCI-PCI pcib5@pci0:0:10:0: class=0x060400 card=0x02f11028 chip=0x34118086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub PCI Express Root Port 10' class = bridge subclass = PCI-PCI none0@pci0:0:20:0: class=0x080000 card=0x00000000 chip=0x342e8086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub System Management Registers' class = base peripheral subclass = interrupt controller none1@pci0:0:20:1: class=0x080000 card=0x00000000 chip=0x34228086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers' class = base peripheral subclass = interrupt controller none2@pci0:0:20:2: class=0x080000 card=0x00000000 chip=0x34238086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 I/O Hub Control Status and RAS Registers' class = base peripheral subclass = interrupt controller none3@pci0:0:22:0: class=0x088000 card=0x02f11028 chip=0x34308086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none4@pci0:0:22:1: class=0x088000 card=0x02f11028 chip=0x34318086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none5@pci0:0:22:2: class=0x088000 card=0x02f11028 chip=0x34328086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none6@pci0:0:22:3: class=0x088000 card=0x02f11028 chip=0x34338086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none7@pci0:0:22:4: class=0x088000 card=0x02f11028 chip=0x34298086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none8@pci0:0:22:5: class=0x088000 card=0x02f11028 chip=0x342a8086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none9@pci0:0:22:6: class=0x088000 card=0x02f11028 chip=0x342b8086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral none10@pci0:0:22:7: class=0x088000 card=0x02f11028 chip=0x342c8086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = '5520/5500/X58 Chipset QuickData Technology Device' class = base peripheral uhci0@pci0:0:26:0: class=0x0c0300 card=0x02f11028 chip=0x3a378086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB uhci1@pci0:0:26:1: class=0x0c0300 card=0x02f11028 chip=0x3a388086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB ehci0@pci0:0:26:7: class=0x0c0320 card=0x02f11028 chip=0x3a3c8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB2 EHCI Controller' class = serial bus subclass = USB uhci2@pci0:0:29:0: class=0x0c0300 card=0x02f11028 chip=0x3a348086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB uhci3@pci0:0:29:1: class=0x0c0300 card=0x02f11028 chip=0x3a358086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB uhci4@pci0:0:29:2: class=0x0c0300 card=0x02f11028 chip=0x3a368086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB uhci5@pci0:0:29:3: class=0x0c0300 card=0x02f11028 chip=0x3a398086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB UHCI Controller' class = serial bus subclass = USB ehci1@pci0:0:29:7: class=0x0c0320 card=0x02f11028 chip=0x3a3a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JI (ICH10 Family) USB2 EHCI Controller' class = serial bus subclass = USB pcib6@pci0:0:30:0: class=0x060401 card=0x02f11028 chip=0x244e8086 rev=0x90 hdr=0x01 vendor = 'Intel Corporation' device = '82801 PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0x02f11028 chip=0x3a168086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82801JIR (ICH10R) LPC Interface Controller' class = bridge subclass = PCI-ISA bce0@pci0:1:0:0: class=0x020000 card=0x02f11028 chip=0x163b14e4 rev=0x20 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme II BCM5716 Gigabit Ethernet' class = network subclass = ethernet PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom NetXtreme II Ethernet Controller' VPD ro PN = 'BCM95716C1' VPD ro EC = '220197-3' VPD ro SN = '0123456789' VPD ro MN = '1028' VPD ro V0 = '6.2.12' bce1@pci0:1:0:1: class=0x020000 card=0x02f11028 chip=0x163b14e4 rev=0x20 hdr=0x00 vendor = 'Broadcom Corporation' device = 'NetXtreme II BCM5716 Gigabit Ethernet' class = network subclass = ethernet PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom NetXtreme II Ethernet Controller' VPD ro PN = 'BCM95716C1' VPD ro EC = '220197-3' VPD ro SN = '0123456789' VPD ro MN = '1028' VPD ro V0 = '6.2.12' mps0@pci0:2:0:0: class=0x010700 card=0x1f1e1028 chip=0x00721000 rev=0x02 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]' class = mass storage subclass = SAS oce0@pci0:3:0:0: class=0x020000 card=0xe62910df chip=0x070019a2 rev=0x02 hdr=0x00 vendor = 'Emulex Corporation' device = 'OneConnect 10Gb NIC' class = network subclass = ethernet PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error oce1@pci0:3:0:1: class=0x020000 card=0xe62910df chip=0x070019a2 rev=0x02 hdr=0x00 vendor = 'Emulex Corporation' device = 'OneConnect 10Gb NIC' class = network subclass = ethernet PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error vgapci0@pci0:6:3:0: class=0x030000 card=0x02f11028 chip=0x0532102b rev=0x0a hdr=0x00 vendor = 'Matrox Graphics, Inc.' device = 'MGA G200eW WPCM450' class = display subclass = VGA hostb1@pci0:255:0:0: class=0x060000 card=0x80868086 chip=0x2c708086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QuickPath Architecture Generic Non-core Registers' class = bridge subclass = HOST-PCI hostb2@pci0:255:0:1: class=0x060000 card=0x80868086 chip=0x2d818086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QuickPath Architecture System Address Decoder' class = bridge subclass = HOST-PCI hostb3@pci0:255:2:0: class=0x060000 card=0x80868086 chip=0x2d908086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Link 0' class = bridge subclass = HOST-PCI hostb4@pci0:255:2:1: class=0x060000 card=0x80868086 chip=0x2d918086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Physical 0' class = bridge subclass = HOST-PCI hostb5@pci0:255:2:2: class=0x060000 card=0x80868086 chip=0x2d928086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Mirror Port Link 0' class = bridge subclass = HOST-PCI hostb6@pci0:255:2:3: class=0x060000 card=0x80868086 chip=0x2d938086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Mirror Port Link 1' class = bridge subclass = HOST-PCI hostb7@pci0:255:2:4: class=0x060000 card=0x80868086 chip=0x2d948086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Link 1' class = bridge subclass = HOST-PCI hostb8@pci0:255:2:5: class=0x060000 card=0x80868086 chip=0x2d958086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Physical 1' class = bridge subclass = HOST-PCI hostb9@pci0:255:3:0: class=0x060000 card=0x80868086 chip=0x2d988086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Registers' class = bridge subclass = HOST-PCI hostb10@pci0:255:3:1: class=0x060000 card=0x80868086 chip=0x2d998086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Target Address Decoder' class = bridge subclass = HOST-PCI hostb11@pci0:255:3:2: class=0x060000 card=0x80868086 chip=0x2d9a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller RAS Registers' class = bridge subclass = HOST-PCI hostb12@pci0:255:3:4: class=0x060000 card=0x80868086 chip=0x2d9c8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Test Registers' class = bridge subclass = HOST-PCI hostb13@pci0:255:4:0: class=0x060000 card=0x80868086 chip=0x2da08086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Control' class = bridge subclass = HOST-PCI hostb14@pci0:255:4:1: class=0x060000 card=0x80868086 chip=0x2da18086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Address' class = bridge subclass = HOST-PCI hostb15@pci0:255:4:2: class=0x060000 card=0x80868086 chip=0x2da28086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Rank' class = bridge subclass = HOST-PCI hostb16@pci0:255:4:3: class=0x060000 card=0x80868086 chip=0x2da38086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control' class = bridge subclass = HOST-PCI hostb17@pci0:255:5:0: class=0x060000 card=0x80868086 chip=0x2da88086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Control' class = bridge subclass = HOST-PCI hostb18@pci0:255:5:1: class=0x060000 card=0x80868086 chip=0x2da98086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Address' class = bridge subclass = HOST-PCI hostb19@pci0:255:5:2: class=0x060000 card=0x80868086 chip=0x2daa8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Rank' class = bridge subclass = HOST-PCI hostb20@pci0:255:5:3: class=0x060000 card=0x80868086 chip=0x2dab8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control' class = bridge subclass = HOST-PCI hostb21@pci0:255:6:0: class=0x060000 card=0x80868086 chip=0x2db08086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Control' class = bridge subclass = HOST-PCI hostb22@pci0:255:6:1: class=0x060000 card=0x80868086 chip=0x2db18086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Address' class = bridge subclass = HOST-PCI hostb23@pci0:255:6:2: class=0x060000 card=0x80868086 chip=0x2db28086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Rank' class = bridge subclass = HOST-PCI hostb24@pci0:255:6:3: class=0x060000 card=0x80868086 chip=0x2db38086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control' class = bridge subclass = HOST-PCI hostb25@pci0:254:0:0: class=0x060000 card=0x80868086 chip=0x2c708086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QuickPath Architecture Generic Non-core Registers' class = bridge subclass = HOST-PCI hostb26@pci0:254:0:1: class=0x060000 card=0x80868086 chip=0x2d818086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QuickPath Architecture System Address Decoder' class = bridge subclass = HOST-PCI hostb27@pci0:254:2:0: class=0x060000 card=0x80868086 chip=0x2d908086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Link 0' class = bridge subclass = HOST-PCI hostb28@pci0:254:2:1: class=0x060000 card=0x80868086 chip=0x2d918086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Physical 0' class = bridge subclass = HOST-PCI hostb29@pci0:254:2:2: class=0x060000 card=0x80868086 chip=0x2d928086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Mirror Port Link 0' class = bridge subclass = HOST-PCI hostb30@pci0:254:2:3: class=0x060000 card=0x80868086 chip=0x2d938086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Mirror Port Link 1' class = bridge subclass = HOST-PCI hostb31@pci0:254:2:4: class=0x060000 card=0x80868086 chip=0x2d948086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Link 1' class = bridge subclass = HOST-PCI hostb32@pci0:254:2:5: class=0x060000 card=0x80868086 chip=0x2d958086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series QPI Physical 1' class = bridge subclass = HOST-PCI hostb33@pci0:254:3:0: class=0x060000 card=0x80868086 chip=0x2d988086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Registers' class = bridge subclass = HOST-PCI hostb34@pci0:254:3:1: class=0x060000 card=0x80868086 chip=0x2d998086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Target Address Decoder' class = bridge subclass = HOST-PCI hostb35@pci0:254:3:2: class=0x060000 card=0x80868086 chip=0x2d9a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller RAS Registers' class = bridge subclass = HOST-PCI hostb36@pci0:254:3:4: class=0x060000 card=0x80868086 chip=0x2d9c8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Test Registers' class = bridge subclass = HOST-PCI hostb37@pci0:254:4:0: class=0x060000 card=0x80868086 chip=0x2da08086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Control' class = bridge subclass = HOST-PCI hostb38@pci0:254:4:1: class=0x060000 card=0x80868086 chip=0x2da18086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Address' class = bridge subclass = HOST-PCI hostb39@pci0:254:4:2: class=0x060000 card=0x80868086 chip=0x2da28086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Rank' class = bridge subclass = HOST-PCI hostb40@pci0:254:4:3: class=0x060000 card=0x80868086 chip=0x2da38086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control' class = bridge subclass = HOST-PCI hostb41@pci0:254:5:0: class=0x060000 card=0x80868086 chip=0x2da88086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Control' class = bridge subclass = HOST-PCI hostb42@pci0:254:5:1: class=0x060000 card=0x80868086 chip=0x2da98086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Address' class = bridge subclass = HOST-PCI hostb43@pci0:254:5:2: class=0x060000 card=0x80868086 chip=0x2daa8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Rank' class = bridge subclass = HOST-PCI hostb44@pci0:254:5:3: class=0x060000 card=0x80868086 chip=0x2dab8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control' class = bridge subclass = HOST-PCI hostb45@pci0:254:6:0: class=0x060000 card=0x80868086 chip=0x2db08086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Control' class = bridge subclass = HOST-PCI hostb46@pci0:254:6:1: class=0x060000 card=0x80868086 chip=0x2db18086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Address' class = bridge subclass = HOST-PCI hostb47@pci0:254:6:2: class=0x060000 card=0x80868086 chip=0x2db28086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Rank' class = bridge subclass = HOST-PCI hostb48@pci0:254:6:3: class=0x060000 card=0x80868086 chip=0x2db38086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control' class = bridge subclass = HOST-PCI root@pruebas:~ # (/var/run/dmesg.boot) root@pruebas:~ # cat /var/run/dmesg.boot Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.3-BETA3 #0 r267433: Fri Jun 13 01:48:48 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 gcc version 4.2.1 20070831 patched [FreeBSD] CPU: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz (2266.79-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206c2 Family = 0x6 Model = 0x2c Stepping = 2 Features=0xbfebfbff Features2=0x29ee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 25769803776 (24576 MB) avail memory = 24828514304 (23678 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 32 cpu1 (AP): APIC ID: 33 cpu2 (AP): APIC ID: 34 cpu3 (AP): APIC ID: 35 cpu4 (AP): APIC ID: 36 cpu5 (AP): APIC ID: 37 cpu6 (AP): APIC ID: 48 cpu7 (AP): APIC ID: 49 cpu8 (AP): APIC ID: 50 cpu9 (AP): APIC ID: 51 cpu10 (AP): APIC ID: 52 cpu11 (AP): APIC ID: 53 cpu12 (AP): APIC ID: 0 cpu13 (AP): APIC ID: 1 cpu14 (AP): APIC ID: 2 cpu15 (AP): APIC ID: 3 cpu16 (AP): APIC ID: 4 cpu17 (AP): APIC ID: 5 cpu18 (AP): APIC ID: 16 cpu19 (AP): APIC ID: 17 cpu20 (AP): APIC ID: 18 cpu21 (AP): APIC ID: 19 cpu22 (AP): APIC ID: 20 cpu23 (AP): APIC ID: 21 ioapic1: Changing APIC ID to 1 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 32-55 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 cpu12: on acpi0 cpu13: on acpi0 cpu14: on acpi0 cpu15: on acpi0 cpu16: on acpi0 cpu17: on acpi0 cpu18: on acpi0 cpu19: on acpi0 cpu20: on acpi0 cpu21: on acpi0 cpu22: on acpi0 cpu23: on acpi0 atrtc0: port 0x70-0x7f irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x5f irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 450 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 Event timer "HPET3" frequency 14318180 Hz quality 440 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 bce0: mem 0xda000000-0xdbffffff irq 36 at device 0.0 on pci1 miibus0: on bce0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bce0: Ethernet address: 78:2b:cb:1b:4a:61 bce0: ASIC (0x57092008); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.2.3); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.11) Coal (RX:6,6,18,18; TX:20,20,80,80) bce1: mem 0xdc000000-0xddffffff irq 48 at device 0.1 on pci1 miibus1: on bce1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bce1: Ethernet address: 78:2b:cb:1b:4a:62 bce1: ASIC (0x57092008); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.2.3); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.11) Coal (RX:6,6,18,18; TX:20,20,80,80) pcib2: at device 3.0 on pci0 pci2: on pcib2 mps0: port 0xfc00-0xfcff mem 0xdf2b0000-0xdf2bffff,0xdf2c0000-0xdf2fffff irq 32 at device 0.0 on pci2 mps0: Firmware: 14.00.00.00, Driver: 16.00.00.00-fbsd mps0: IOCCapabilities: 1285c pcib3: at device 7.0 on pci0 pci3: on pcib3 pci3: at device 0.0 (no driver attached) pci3: at device 0.1 (no driver attached) pcib4: at device 9.0 on pci0 pci4: on pcib4 pcib5: at device 10.0 on pci0 pci5: on pcib5 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) uhci0: port 0xec40-0xec5f irq 17 at device 26.0 on pci0 usbus0 on uhci0 uhci1: port 0xec60-0xec7f irq 18 at device 26.1 on pci0 usbus1 on uhci1 ehci0: mem 0xdf0de000-0xdf0de3ff irq 19 at device 26.7 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci0 uhci2: port 0xec80-0xec9f irq 21 at device 29.0 on pci0 usbus3 on uhci2 uhci3: port 0xeca0-0xecbf irq 20 at device 29.1 on pci0 usbus4 on uhci3 uhci4: port 0xecc0-0xecdf irq 21 at device 29.2 on pci0 usbus5 on uhci4 uhci5: port 0xece0-0xecff irq 20 at device 29.3 on pci0 usbus6 on uhci5 ehci1: mem 0xdf0df000-0xdf0df3ff irq 21 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: mem 0xd9800000-0xd9ffffff,0xde7fc000-0xde7fffff,0xde800000-0xdeffffff irq 19 at device 3.0 on pci6 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 qpi0: on motherboard pcib7: pcibus 255 on qpi0 pci255: on pcib7 pcib8: pcibus 254 on qpi0 pci254: on pcib8 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff,0xce000-0xcf7ff,0xcf800-0xd07ff,0xec000-0xeffff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est1 attach returned 6 p4tcc1: on cpu1 est2: on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est2 attach returned 6 p4tcc2: on cpu2 est3: on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est3 attach returned 6 p4tcc3: on cpu3 est4: on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est4 attach returned 6 p4tcc4: on cpu4 est5: on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est5 attach returned 6 p4tcc5: on cpu5 est6: on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est6 attach returned 6 p4tcc6: on cpu6 est7: on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est7 attach returned 6 p4tcc7: on cpu7 est8: on cpu8 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est8 attach returned 6 p4tcc8: on cpu8 est9: on cpu9 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est9 attach returned 6 p4tcc9: on cpu9 est10: on cpu10 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est10 attach returned 6 p4tcc10: on cpu10 est11: on cpu11 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est11 attach returned 6 p4tcc11: on cpu11 est12: on cpu12 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est12 attach returned 6 p4tcc12: on cpu12 est13: on cpu13 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est13 attach returned 6 p4tcc13: on cpu13 est14: on cpu14 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est14 attach returned 6 p4tcc14: on cpu14 est15: on cpu15 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est15 attach returned 6 p4tcc15: on cpu15 est16: on cpu16 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est16 attach returned 6 p4tcc16: on cpu16 est17: on cpu17 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est17 attach returned 6 p4tcc17: on cpu17 est18: on cpu18 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est18 attach returned 6 p4tcc18: on cpu18 est19: on cpu19 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est19 attach returned 6 p4tcc19: on cpu19 est20: on cpu20 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est20 attach returned 6 p4tcc20: on cpu20 est21: on cpu21 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est21 attach returned 6 p4tcc21: on cpu21 est22: on cpu22 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est22 attach returned 6 p4tcc22: on cpu22 est23: on cpu23 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 12 device_attach: est23 attach returned 6 p4tcc23: on cpu23 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 480Mbps High Speed USB v2.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 ugen5.1: at usbus5 uhub5: on usbus5 ugen6.1: at usbus6 uhub6: on usbus6 ugen7.1: at usbus7 uhub7: on usbus7 da0 at mps0 bus 0 scbus0 target 8 lun 0 da0: Fixed Direct Access SCSI-5 device da0: Serial Number 6SD2R026 da0: 600.000MB/s transfers da0: Command Queueing enabled da0: 140014MB (286749480 512 byte sectors: 255H 63S/T 17849C) da1 at mps0 bus 0 scbus0 target 9 lun 0 da1: Fixed Direct Access SCSI-5 device da1: Serial Number 6SD2QE3N da1: 600.000MB/s transfers da1: Command Queueing enabled da1: 140014MB (286749480 512 byte sectors: 255H 63S/T 17849C) da2 at mps0 bus 0 scbus0 target 11 lun 0 da2: Fixed Direct Access SCSI-6 device da2: Serial Number OCZ-806BA2P5056GW75I da2: 600.000MB/s transfers da2: Command Queueing enabled da2: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da2: quirks=0x8<4K> da3 at mps0 bus 0 scbus0 target 12 lun 0 da3: Fixed Direct Access SCSI-6 device da3: Serial Number OCZ-XF296AZZ71QSP1ZW da3: 600.000MB/s transfers da3: Command Queueing enabled da3: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da3: quirks=0x8<4K> da4 at mps0 bus 0 scbus0 target 13 lun 0 da4: Fixed Direct Access SCSI-6 device da4: Serial Number OCZ-XLEQGOUM0W4X8YEK da4: 600.000MB/s transfers da4: Command Queueing enabled da4: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da4: quirks=0x8<4K> da5 at mps0 bus 0 scbus0 target 14 lun 0 da5: Fixed Direct Access SCSI-6 device da5: Serial Number OCZ-Y95766VPB527422Z da5: 600.000MB/s transfers da5: Command Queueing enabled da5: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da5: quirks=0x8<4K> da6 at mps0 bus 0 scbus0 target 15 lun 0 da6: Fixed Direct Access SCSI-6 device da6: Serial Number OCZ-P41W6HED0801IAL3 da6: 600.000MB/s transfers da6: Command Queueing enabled da6: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da6: quirks=0x8<4K> da7 at mps0 bus 0 scbus0 target 16 lun 0 da7: Fixed Direct Access SCSI-6 device da7: Serial Number OCZ-YB60WW78PP6BK4TM da7: 600.000MB/s transfers da7: Command Queueing enabled da7: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da7: quirks=0x8<4K> da8 at mps0 bus 0 scbus0 target 17 lun 0 da8: Fixed Direct Access SCSI-6 device da8: Serial Number OCZ-G8477U602103SBQ3 da8: 600.000MB/s transfers da8: Command Queueing enabled da8: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da8: quirks=0x8<4K> da9 at mps0 bus 0 scbus0 target 18 lun 0 da9: Fixed Direct Access SCSI-6 device da9: Serial Number OCZ-5Z928G69IA2N6WN7 da9: 600.000MB/s transfers da9: Command Queueing enabled da9: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da9: quirks=0x8<4K> da10 at mps0 bus 0 scbus0 target 19 lun 0 da10: Fixed Direct Access SCSI-6 device da10: Serial Number OCZ-K008HT13IQ31VK3F da10: 600.000MB/s transfers da10: Command Queueing enabled da10: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da10: quirks=0x8<4K> da11 at mps0 bus 0 scbus0 target 20 lun 0 da11: Fixed Direct Access SCSI-6 device da11: Serial Number OCZ-I276ZFJW8I19B288 da11: 600.000MB/s transfers da11: Command Queueing enabled da11: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da11: quirks=0x8<4K> da12 at mps0 bus 0 scbus0 target 21 lun 0 da12: Fixed Direct Access SCSI-6 device da12: Serial Number OCZ-JHK0L1B3K0UJC91U da12: 600.000MB/s transfers da12: Command Queueing enabled da12: 488386MB (1000215216 512 byte sectors: 255H 63S/T 62260C) da12: quirks=0x8<4K> SMP: AP CPU #1 Launched! SMP: AP CPU #23 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #22 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #18 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #19 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #20 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #21 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #17 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #16 Launched! Timecounter "TSC-low" frequency 1133396472 Hz quality 1000 uhub6: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered Root mount waiting for: usbus7 usbus2 Root mount waiting for: usbus7 usbus2 uhub2: 4 ports with 4 removable, self powered uhub7: 4 ports with 4 removable, self powered Root mount waiting for: usbus7 usbus2 ugen2.2: at usbus2 uhub8: on usbus2 uhub8: MTT enabled uhub8: 4 ports with 4 removable, self powered Root mount waiting for: usbus7 usbus2 ugen2.3: at usbus2 uhub9: on usbus2 ugen7.2: at usbus7 umass0: on usbus7 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:1:0:-1: Attached to scbus1 da13 at umass-sim0 bus 0 scbus1 target 0 lun 0 da13: Removable Direct Access SCSI-4 device da13: Serial Number 3ENXRQO0QDYMK5QO da13: 40.000MB/s transfers da13: 15480MB (31703040 512 byte sectors: 255H 63S/T 1973C) da13: quirks=0x2 uhub9: 3 ports with 2 removable, bus powered Root mount waiting for: usbus2 ugen2.4: at usbus2 ukbd0: on usbus2 kbd2 at ukbd0 uhid0: on usbus2 Trying to mount root from ufs:/dev/da13p2 [rw]... WARNING: / was not properly dismounted ugen3.2: at usbus3 ukbd1: on usbus3 kbd3 at ukbd1 ums0: on usbus3 ums0: 3 buttons and [Z] coordinates ID=0 root@pruebas:~ # -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 12:20:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8EEB5536; Wed, 18 Jun 2014 12:20:29 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A4B42B10; Wed, 18 Jun 2014 12:20:28 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 123D99DC848; Wed, 18 Jun 2014 14:20:25 +0200 (CEST) Subject: Re: Unusable Emulex "oce" driver version in -STABLE, 9.3 and CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Wed, 18 Jun 2014 14:20:23 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <0BF09067-7B0A-4161-8A74-DDD38D195796@sarenet.es> References: <8D93EDF7-AC0E-4707-AC62-55CBD5F3358D@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org, Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 12:20:29 -0000 On Jun 18, 2014, at 9:35 AM, Borja Marcos wrote: > In my case, the problem manifests itself running 10. Emulex provides a = more recent driver which compiles perfectly and works perfectly (if we = can safely assume that a weekend saturating a couple of interfaces = runnning benchmarks with no glitches is a good hint). >=20 > I am now installing the latest beta of 9.3 to see if I can reproduce = the panic. I will follow-up in a couple of hours if possible. Just a follow-up.=20 I ran the tests on 9.3-BETA3. It doesn't panic, but the interface = freezes and it's unable to transmit packets. So, it's still unusable. Now I'm installing the new driver and trying again. Will take time, I = have installed the system to a pendrive.=20 Borja. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 14:07:45 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62DB32B0 for ; Wed, 18 Jun 2014 14:07:45 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49E66276A for ; Wed, 18 Jun 2014 14:07:45 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IE7jgi064722 for ; Wed, 18 Jun 2014 15:07:45 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 14:07:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 14:07:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #12 from borjam@sarenet.es --- I've compiled the latest driver supplied by Emulex. Jun 18 17:47:22 pruebas kernel: oce0: mem 0xdf478000-0xdf47bfff,0xdf480000-0xdf49ffff,0xdf4a0000-0xdf4bffff irq 38 at device 0.0 on pci3 Jun 18 17:47:22 pruebas kernel: oce1: mem 0xdf47c000-0xdf47ffff,0xdf4c0000-0xdf4dffff,0xdf4e0000-0xdf4fffff irq 45 at device 0.1 on pci3 Running with this latest driver on 9.3-BETA3 it doesn't crash or freeze. At least I managed to keep a lot of traffic running for a couple of minutes. The driver doesn't compile cleanly, I just commented out a couple of ifdefs and probably I broke something. Some adult please check this ;) --- oce_if.c 2014-06-18 17:33:34.000000000 +0200 +++ oce_if.c.orig 2013-12-05 14:26:55.000000000 +0100 @@ -1232,7 +1232,7 @@ } -// #if __FreeBSD_version >= 1000000 +#if __FreeBSD_version >= 1000000 static __inline void drbr_stats_update(struct ifnet *ifp, int len, int mflags) { @@ -1242,7 +1242,7 @@ ifp->if_omcasts++; #endif } -// #endif +#endif static int oce_multiq_transmit(struct ifnet *ifp, struct mbuf *m, struct oce_wq *wq) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 14:09:02 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC8BE5DE for ; Wed, 18 Jun 2014 14:09:02 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B52127AE for ; Wed, 18 Jun 2014 14:09:02 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IE92WW065572 for ; Wed, 18 Jun 2014 15:09:02 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 14:09:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 14:09:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #13 from borjam@sarenet.es --- I downloaded the source code from here. http://www.emulex.com/downloads/emulex/drivers/freebsd/freebsd-91-amd64/drivers/ To summarize: Compiles cleanly and works on 10-STABLE Can be compiled and works on 9.3-BETA3. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 14:13:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7ED0F946; Wed, 18 Jun 2014 14:13:02 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39AD22865; Wed, 18 Jun 2014 14:13:01 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id B276A9DC69E; Wed, 18 Jun 2014 16:12:58 +0200 (CEST) Subject: Re: Unusable Emulex "oce" driver version in -STABLE, 9.3 and CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <0BF09067-7B0A-4161-8A74-DDD38D195796@sarenet.es> Date: Wed, 18 Jun 2014 16:12:57 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <417B6D80-80AD-4135-982F-CA3F05E96988@sarenet.es> References: <8D93EDF7-AC0E-4707-AC62-55CBD5F3358D@sarenet.es> <0BF09067-7B0A-4161-8A74-DDD38D195796@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org, Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 14:13:02 -0000 On Jun 18, 2014, at 2:20 PM, Borja Marcos wrote: > Just a follow-up.=20 >=20 > I ran the tests on 9.3-BETA3. It doesn't panic, but the interface = freezes and it's unable to transmit packets. So, it's still unusable. >=20 > Now I'm installing the new driver and trying again. Will take time, I = have installed the system to a pendrive.=20 Tried with the new driver on 9.3-BETA3. = http://www.emulex.com/downloads/emulex/drivers/freebsd/freebsd-91-amd64/dr= ivers/ It doesn't compile cleanly but commenting out an ifdef/endif made it = compile. It works, at least with 2 minutes of traffic I didn't see it freeze or = hang. I can load and unload the extension, no problems. I've updated the bug report. I am posting this just in case someone is = not following freebsd-net. Borja. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 14:35:39 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A2923EA for ; Wed, 18 Jun 2014 14:35:39 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0158B2AF1 for ; Wed, 18 Jun 2014 14:35:39 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IEZcTw046843 for ; Wed, 18 Jun 2014 15:35:38 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 14:35:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-BETA3 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 14:35:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Version|9.2-RELEASE |9.3-BETA3 --- Comment #14 from Kubilay Kocak --- Thank you Borjam, extremely helpful :) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 14:36:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9D06510 for ; Wed, 18 Jun 2014 14:36:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C13312B04 for ; Wed, 18 Jun 2014 14:36:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IEaDVG052612 for ; Wed, 18 Jun 2014 15:36:13 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 14:36:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-BETA3 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 14:36:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #15 from Kubilay Kocak --- Updating Version to 9.3-BETA3 to reflect latest reproduction test. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 15:00:10 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8B9A4B64 for ; Wed, 18 Jun 2014 15:00:10 +0000 (UTC) Received: from mta1.tenable.com (mta1.tenable.com [63.148.88.66]) by mx1.freebsd.org (Postfix) with ESMTP id 544F22D1A for ; Wed, 18 Jun 2014 15:00:09 +0000 (UTC) X-ASG-Debug-ID: 1403102885-081a2a38920a840001-oFaieN Received: from imail.tenable.com ([172.20.100.238]) by mta1.tenable.com with ESMTP id eyxW1inNEzUvTZx9 for ; Wed, 18 Jun 2014 07:48:05 -0700 (PDT) X-Barracuda-Envelope-From: dsmith@tenable.com X-Barracuda-RBL-Trusted-Forwarder: 172.20.100.238 Received: from COL-MBS-001.corp.tenablesecurity.com ([fe80::908:e78e:9ee4:a586]) by col-cas-002.corp.tenablesecurity.com ([fe80::d9c:cce0:cd61:87e3%11]) with mapi id 14.03.0181.006; Wed, 18 Jun 2014 10:48:05 -0400 From: Doug Smith X-Barracuda-RBL-IP: 0.0.0.0 To: "net@freebsd.org" Subject: =?Windows-1252?Q?netmap_make_error_with_CentO6.5.x86=5F64:__=91struct_eth?= =?Windows-1252?Q?tool=5Fops=92_has_no_member_named_=91set=5Fchannels=92?= Thread-Topic: =?Windows-1252?Q?netmap_make_error_with_CentO6.5.x86=5F64:__=91struct_eth?= =?Windows-1252?Q?tool=5Fops=92_has_no_member_named_=91set=5Fchannels=92?= X-ASG-Orig-Subj: =?Windows-1252?Q?netmap_make_error_with_CentO6.5.x86=5F64:__=91struct_eth?= =?Windows-1252?Q?tool=5Fops=92_has_no_member_named_=91set=5Fchannels=92?= Thread-Index: Ac+LA+MvTzK4R9nTS4OobGFH7nPc7Q== Date: Wed, 18 Jun 2014 14:48:03 +0000 Message-ID: <98DC5C21DE51CC42AA340F1B868792B13E836ACD@col-mbs-001.corp.tenablesecurity.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.20.100.125] MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[172.20.100.238] X-Barracuda-Start-Time: 1403102885 X-Barracuda-URL: http://172.20.210.66:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at tenable.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.6748 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 15:00:10 -0000 Hi, I was trying to create the netmap code with Centos6.5.x86_64 system but the= make command fails with the following error message: error: =91struct ethtool_ops=92 has no member named =91set_= channels=92 The 2.6.32-431.17.1.el6.x86_64' kernel is being used. Any help would be greatly appreciated. # cd /usr/local/src/netmap/LINUX # make KSRC=3D/usr/src/kernels/2.6.32-431.17.1.el6.x86_64 make -C /usr/src/kernels/2.6.32-431.17.1.el6.x86_64 M=3D/usr/local/src/netm= ap/LINUX CONFIG_NETMAP=3Dm CONFIG_E1000=3Dm CONFIG_E1000E=3Dm CONFIG_IXGBE= =3Dm CONFIG_IGB=3Dm CONFIG_BNX2X=3Dm CONFIG_MLX4=3Dm CONFIG_VIRTIO_NET=3Dm = \ EXTRA_CFLAGS=3D'-I/usr/local/src/netmap/LINUX -I/usr/local/= src/netmap/LINUX/../sys -I/usr/local/src/netmap/LINUX/../sys/dev -DCONFIG_N= ETMAP -Wno-unused-but-set-variable' \ O_DRIVERS=3D"e1000/ e1000e/" modules make[1]: Entering directory `/usr/src/kernels/2.6.32-431.17.1.el6.x86_64' CC [M] /usr/local/src/netmap/LINUX/netmap.o /usr/local/src/netmap/LINUX/../sys/dev/netmap/netmap.c: In function =91netm= ap_attach=92: /usr/local/src/netmap/LINUX/../sys/dev/netmap/netmap.c:2255: error: =91stru= ct ethtool_ops=92 has no member named =91set_channels=92 make[2]: *** [/usr/local/src/netmap/LINUX/netmap.o] Error 1 make[1]: *** [_module_/usr/local/src/netmap/LINUX] Error 2 make[1]: Leaving directory `/usr/src/kernels/2.6.32-431.17.1.el6.x86_64' make: *** [build] Error 2 Thanks, Doug From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 15:26:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E41E5ED6 for ; Wed, 18 Jun 2014 15:26:49 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB4F92FAD for ; Wed, 18 Jun 2014 15:26:49 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IFQnTp037273 for ; Wed, 18 Jun 2014 16:26:49 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 18 Jun 2014 15:26:50 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-BETA3 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 15:26:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #16 from borjam@sarenet.es --- Remember, remember, the 10-STABLE of November, nonetheless :) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 18 16:36:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEE325AD for ; Wed, 18 Jun 2014 16:36:26 +0000 (UTC) Received: from udns.ultimateDNS.NET (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70C4726F4 for ; Wed, 18 Jun 2014 16:36:26 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s5IGacNU052075 for ; Wed, 18 Jun 2014 09:36:44 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s5IGaXGU052074; Wed, 18 Jun 2014 09:36:33 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from 2602:d1:b4d6:e600:4261:86ff:fef6:aa2a (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Wed, 18 Jun 2014 09:36:33 -0700 (PDT) Message-ID: Date: Wed, 18 Jun 2014 09:36:33 -0700 (PDT) Subject: 6rd and DNS (bind/nsd) on FreeBSD From: "Chris H" To: "freebsd-net" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 16:36:26 -0000 Greetings, I manage a /29 at $home. While I manage _real_ IPv6 on many networks at $work. I'm stuck with 6rd at $home. I don't much care for 6rd. It's still pretty much 6to4. But it's all I have to work with, given the CPE's limitations. So, as I'm new to it, I'm not quite sure _which_ address to advertise, and listen on, on the DNS. eg; I'm told I have 6 6rd addresses: 2602:0000:0000:0000:0000:0000:xxxx:xxxx or 2602::xxxx:xxxx but the address returned, has the IPv4 bits packed into it. So which of the to addresses should be advertised in the zone files. Should that address also be the one used to "listen" on? Apologies for seemingly such a dimwitted question. But even after reading RFC 5569, I'm still unclear. Thank you. --Chris From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 03:11:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17D9D3E9 for ; Thu, 19 Jun 2014 03:11:56 +0000 (UTC) Received: from yoshi.brtsvcs.net (yoshi.brtsvcs.net [IPv6:2607:f2f8:a450::66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE7B02DF7 for ; Thu, 19 Jun 2014 03:11:55 +0000 (UTC) Received: from chombo.houseloki.net (c-76-115-19-22.hsd1.or.comcast.net [76.115.19.22]) by yoshi.brtsvcs.net (Postfix) with ESMTPSA id 8B83CE6048; Wed, 18 Jun 2014 20:11:54 -0700 (PDT) Received: from [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf] (unknown [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf]) by chombo.houseloki.net (Postfix) with ESMTPSA id 4ED5BFF2; Wed, 18 Jun 2014 20:11:52 -0700 (PDT) Message-ID: <53A254F7.4040801@bluerosetech.com> Date: Wed, 18 Jun 2014 20:11:51 -0700 From: Darren Pilgrim Reply-To: freebsd-net User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Chris H , freebsd-net Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 03:11:56 -0000 On 6/18/2014 9:36 AM, Chris H wrote: > Greetings, > I manage a /29 at $home. While I manage _real_ IPv6 on many networks > at $work. I'm stuck with 6rd at $home. I don't much care for 6rd. It's > still pretty much 6to4. But it's all I have to work with, given the > CPE's limitations. So, as I'm new to it, I'm not quite sure _which_ > address to advertise, and listen on, on the DNS. > eg; I'm told I have 6 6rd addresses: > 2602:0000:0000:0000:0000:0000:xxxx:xxxx > or > 2602::xxxx:xxxx > > but the address returned, has the IPv4 bits packed into it. > So which of the to addresses should be advertised in the zone files. > Should that address also be the one used to "listen" on? > > Apologies for seemingly such a dimwitted question. But even > after reading RFC 5569, I'm still unclear. FreeBSD doesn't support 6rd. Ironically, pfSense does. From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 03:16:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD28D4CA for ; Thu, 19 Jun 2014 03:16:19 +0000 (UTC) Received: from mario.brtsvcs.net (mario.brtsvcs.net [IPv6:2607:fc50:0:a400::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B37C82E29 for ; Thu, 19 Jun 2014 03:16:19 +0000 (UTC) Received: from chombo.houseloki.net (unknown [IPv6:2601:7:880:bd0:21c:c0ff:fe7f:96ee]) by mario.brtsvcs.net (Postfix) with ESMTPSA id 2630A2C1630; Wed, 18 Jun 2014 20:16:18 -0700 (PDT) Received: from [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf] (unknown [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf]) by chombo.houseloki.net (Postfix) with ESMTPSA id 50A12FF5; Wed, 18 Jun 2014 20:16:16 -0700 (PDT) Message-ID: <53A25600.1090103@bluerosetech.com> Date: Wed, 18 Jun 2014 20:16:16 -0700 From: Darren Pilgrim Reply-To: freebsd-net User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Chris H , freebsd-net Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 03:16:19 -0000 Chris, On 6/18/2014 8:11 PM, Mail Delivery System wrote: > : host mx99.ultimatedns.net[209.180.214.225] > said: 550 5.0.0 SPAM and BULK mail REJECTED (in reply to MAIL FROM > command) You might need to adjust your mail filters. :) On 6/18/2014 9:36 AM, Chris H wrote: > Greetings, I manage a /29 at $home. While I manage _real_ IPv6 on > many networks at $work. I'm stuck with 6rd at $home. I don't much > care for 6rd. It's still pretty much 6to4. But it's all I have to > work with, given the CPE's limitations. So, as I'm new to it, I'm not > quite sure _which_ address to advertise, and listen on, on the DNS. > eg; I'm told I have 6 6rd addresses: > 2602:0000:0000:0000:0000:0000:xxxx:xxxx or 2602::xxxx:xxxx > > but the address returned, has the IPv4 bits packed into it. So which > of the to addresses should be advertised in the zone files. Should > that address also be the one used to "listen" on? > > Apologies for seemingly such a dimwitted question. But even after > reading RFC 5569, I'm still unclear. > > Thank you. > > --Chris > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, > send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 05:12:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDC5B893 for ; Thu, 19 Jun 2014 05:12:05 +0000 (UTC) Received: from udns.ultimateDNS.NET (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F5AA26B1 for ; Thu, 19 Jun 2014 05:12:05 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s5J5CHGs013666; Wed, 18 Jun 2014 22:12:23 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s5J5CC5G013663; Wed, 18 Jun 2014 22:12:12 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from 2602:d1:b4d6:e600:4261:86ff:fef6:aa2a (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Wed, 18 Jun 2014 22:12:12 -0700 (PDT) Message-ID: <71fc86db55a80c38420515eb7181758e.authenticated@ultimatedns.net> In-Reply-To: <53A254F7.4040801@bluerosetech.com> References: <53A254F7.4040801@bluerosetech.com> Date: Wed, 18 Jun 2014 22:12:12 -0700 (PDT) Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD From: "Chris H" To: "freebsd-net" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: list_freebsd@bluerosetech.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 05:12:05 -0000 > On 6/18/2014 9:36 AM, Chris H wrote: >> Greetings, >> I manage a /29 at $home. While I manage _real_ IPv6 on many networks >> at $work. I'm stuck with 6rd at $home. I don't much care for 6rd. It's >> still pretty much 6to4. But it's all I have to work with, given the >> CPE's limitations. So, as I'm new to it, I'm not quite sure _which_ >> address to advertise, and listen on, on the DNS. >> eg; I'm told I have 6 6rd addresses: >> 2602:0000:0000:0000:0000:0000:xxxx:xxxx >> or >> 2602::xxxx:xxxx >> >> but the address returned, has the IPv4 bits packed into it. >> So which of the to addresses should be advertised in the zone files. >> Should that address also be the one used to "listen" on? >> >> Apologies for seemingly such a dimwitted question. But even >> after reading RFC 5569, I'm still unclear. > > FreeBSD doesn't support 6rd. Ironically, pfSense does. Are you sure? There are even a couple of 6rd ports: net/stf-6rd-kmod and net/u6rd or am I to understand that _without_ those ports, FreeBSD doesn't support 6rd. Thanks for the reply. --Chris > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 05:23:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D28FAD7 for ; Thu, 19 Jun 2014 05:23:16 +0000 (UTC) Received: from udns.ultimateDNS.NET (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44A62277A for ; Thu, 19 Jun 2014 05:23:16 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s5J5NVmk014080 for ; Wed, 18 Jun 2014 22:23:37 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s5J5NQIo014074; Wed, 18 Jun 2014 22:23:26 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from 2602:d1:b4d6:e600:4261:86ff:fef6:aa2a (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Wed, 18 Jun 2014 22:23:26 -0700 (PDT) Message-ID: <29ca3349b776931af8fa773d379e0c23.authenticated@ultimatedns.net> In-Reply-To: <53A25600.1090103@bluerosetech.com> References: <53A25600.1090103@bluerosetech.com> Date: Wed, 18 Jun 2014 22:23:26 -0700 (PDT) Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD From: "Chris H" To: "freebsd-net" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 05:23:16 -0000 > Chris, > > On 6/18/2014 8:11 PM, Mail Delivery System wrote: >> : host mx99.ultimatedns.net[209.180.214.225] >> said: 550 5.0.0 SPAM and BULK mail REJECTED (in reply to MAIL FROM >> command) > > You might need to adjust your mail filters. :) Thanks for the heads up. Appears the IP block yoshi.brtsvcs.net lives in, was on the list (not because of yoshi.brtsvcs.net). I'm afraid I was hammered by ~500 spammer/hour. So I got a little overzealous adding IP blocks. I thought I had cleaned it all up. But appears I need to go over it again. Interestingly enough, bluerosetech.com doesn't have any A glue. Thanks again, and sorry. Nothing personal. :) --Chris > > > On 6/18/2014 9:36 AM, Chris H wrote: >> Greetings, I manage a /29 at $home. While I manage _real_ IPv6 on >> many networks at $work. I'm stuck with 6rd at $home. I don't much >> care for 6rd. It's still pretty much 6to4. But it's all I have to >> work with, given the CPE's limitations. So, as I'm new to it, I'm not >> quite sure _which_ address to advertise, and listen on, on the DNS. >> eg; I'm told I have 6 6rd addresses: >> 2602:0000:0000:0000:0000:0000:xxxx:xxxx or 2602::xxxx:xxxx >> >> but the address returned, has the IPv4 bits packed into it. So which >> of the to addresses should be advertised in the zone files. Should >> that address also be the one used to "listen" on? >> >> Apologies for seemingly such a dimwitted question. But even after >> reading RFC 5569, I'm still unclear. >> >> Thank you. >> >> --Chris >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, >> send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 08:48:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 882181AD for ; Thu, 19 Jun 2014 08:48:17 +0000 (UTC) Received: from mario.brtsvcs.net (mario.brtsvcs.net [IPv6:2607:fc50:0:a400::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E26426B9 for ; Thu, 19 Jun 2014 08:48:17 +0000 (UTC) Received: from chombo.houseloki.net (c-76-115-19-22.hsd1.or.comcast.net [76.115.19.22]) by mario.brtsvcs.net (Postfix) with ESMTPSA id 7BD722C1630; Thu, 19 Jun 2014 01:48:15 -0700 (PDT) Received: from [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf] (unknown [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf]) by chombo.houseloki.net (Postfix) with ESMTPSA id E14FCCF; Thu, 19 Jun 2014 01:48:12 -0700 (PDT) Message-ID: <53A2A3CC.5030407@bluerosetech.com> Date: Thu, 19 Jun 2014 01:48:12 -0700 From: Darren Pilgrim Reply-To: freebsd-net User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Chris H , freebsd-net Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD References: <53A254F7.4040801@bluerosetech.com> <71fc86db55a80c38420515eb7181758e.authenticated@ultimatedns.net> In-Reply-To: <71fc86db55a80c38420515eb7181758e.authenticated@ultimatedns.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 08:48:17 -0000 On 6/18/2014 10:12 PM, Chris H wrote: >> FreeBSD doesn't support 6rd. Ironically, pfSense does. > > Are you sure? > There are even a couple of 6rd ports: > net/stf-6rd-kmod > and > net/u6rd > or am I to understand that _without_ those ports, FreeBSD doesn't > support 6rd. Yes, if you bring in third-party code you can do 6rd on FreeBSD. I meant that FreeBSD does not have 6rd support in the base like it does for 6to4 and 6in4. There have been patches available for years that add 6rd feature suppport to if_stf, but for some reason they've never made it into FreeBSD despite making into downstream projects. From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 09:12:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C967C2B for ; Thu, 19 Jun 2014 09:12:12 +0000 (UTC) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3B4A290A for ; Thu, 19 Jun 2014 09:12:10 +0000 (UTC) Received: from alph.d.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=56) by mail.allbsd.org (8.14.8/8.14.8) with ESMTP id s5J9Bp0n089603 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Jun 2014 18:12:02 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.8/8.14.8) with ESMTP id s5J9BntA063515; Thu, 19 Jun 2014 18:11:50 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Thu, 19 Jun 2014 18:11:29 +0900 (JST) Message-Id: <20140619.181129.1279477227287764712.hrs@allbsd.org> To: ler@lerctr.org Subject: Re: IPv6: "xxx::x already configured" in logs... why? From: Hiroki Sato In-Reply-To: <20140612202349.GA65079@thebighonker.lerctr.org> References: <20140612202349.GA65079@thebighonker.lerctr.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Thu_Jun_19_18_11_29_2014_347)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Thu, 19 Jun 2014 18:12:02 +0900 (JST) X-Spam-Status: No, score=-97.9 required=13.0 tests=CONTENT_TYPE_PRESENT, RDNS_NONE,SPF_SOFTFAIL,USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 09:12:12 -0000 ----Security_Multipart(Thu_Jun_19_18_11_29_2014_347)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Larry Rosenman wrote in <20140612202349.GA65079@thebighonker.lerctr.org>: le> I just started using IPv6 behind my (new to me) Cisco 1841. le> le> I see lots of: le> Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured le> le> in my /var/log/messages. le> le> The rc.conf: le> le> cloned_interfaces="lagg0:sticky " le> create_args_lagg0="laggport bce0 laggport bce1 laggproto loadbalance" le> ifconfig_lagg0="192.147.25.65/24" le> ifconfig_bce0="up" le> ifconfig_bce1="up" le> defaultrouter="192.147.25.1" le> #################################################### le> # IPv6 AutoConf: le> ifconfig_lagg0_ipv6="inet6 accept_rtadv" le> rtsold_enable="YES" le> #################################################### le> hostname=thebighonker.lerctr.org le> ifconfig_lagg0_alias0="inet 192.147.25.45/24" le> ifconfig_lagg0_alias1="inet 192.147.25.11/24" le> ifconfig_lagg0_alias2="inet 192.147.25.66/24" le> sshd_enable="YES" le> ntpd_enable="YES" le> powerd_enable="YES" le> # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable le> dumpdev="AUTO" le> zfs_enable="YES" le> # ------------ le> le> le> hostid_enable="YES" le> le> postgresql_enable="YES" le> fah_enable="NO" le> fah2_enable="NO" le> linux_enable="YES" le> smartd_enable="YES" le> spamd_enable="YES" le> devcpu_enable="YES" le> clamav_clamd_enable="YES" le> clamav_freshclam_enable="YES" le> sendmail_enable="NONE" le> exim_enable="YES" le> exim_flags="-bd -q5m" le> named_enable="YES" le> named_program="/usr/local/sbin/named" le> bsdstats_enable="YES" le> spamd_flags="-c -Q -m 20" le> apache22_enable="YES" le> syslogd_flags="-n -a 192.147.25.0/24:* -a 209.198.148.248/29:*" le> bacula_fd_enable="YES" le> sshblock_enable="YES" le> dovecot_enable="YES" le> monthly_stats_enable="YES" le> boinc_client_enable="YES" le> microcode_update_enable="YES" le> ---- le> sysctl.conf: le> le> # $FreeBSD: stable/10/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $ le> # le> # This file is read when going to multi-user and its contents piped thru le> # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. le> # le> le> # Uncomment this to prevent users from seeing information about processes that le> # are being run under another UID. le> #security.bsd.see_other_uids=0 le> vm.lowmem_period=0 le> vfs.usermount=1 le> vfs.zfs.super_owner=1 le> kern.elf32.fallback_brand=3 le> kern.ipc.shm_allow_removed=1 le> net.inet6.ip6.dad_count=0 le> le> ===== le> le> Ideas? (I may be an idiot, so any criticism welcomed). le> le> if you need the 1841's config, I can supply that as well. It's using a Hurricane le> electric Tunnel. How frequent were the log message added into /var/log/messages? And when did it start to happen after boot. Just after lagg0 is configured? -- Hiroki ----Security_Multipart(Thu_Jun_19_18_11_29_2014_347)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlOiqUEACgkQTyzT2CeTzy14QQCfQwaPp+Npq/VZXP3ZZmfMurr3 qKEAoJYfD0ubKVJRkK/L3QIZ+uciILHi =XEtX -----END PGP SIGNATURE----- ----Security_Multipart(Thu_Jun_19_18_11_29_2014_347)---- From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 10:35:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F22F584 for ; Thu, 19 Jun 2014 10:35:16 +0000 (UTC) Received: from zibbi.meraka.csir.co.za (zibbi.meraka.csir.co.za [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id CA01320D6 for ; Thu, 19 Jun 2014 10:35:15 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id 96F90B828; Thu, 19 Jun 2014 12:35:13 +0200 (SAST) Date: Thu, 19 Jun 2014 12:35:13 +0200 From: John Hay To: freebsd-net@freebsd.org Subject: network.subr vlan handling broken Message-ID: <20140619103513.GA92393@zibbi.meraka.csir.co.za> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 10:35:16 -0000 Hi Guys, freebsd-rc did not react, so I'm just checking on -net too. I found after upgrading that vlan handling broke. I tried the following: vlans_bce1="6" ipv4_addrs_bce1_6="inet 10.239.100.2/24" ifconfig_bce1_6_aliases="inet 10.239.100.2/24" ifconfig_bce1_6_alias0="inet 10.239.100.2/24" I traced it down to ifalias_af_common_handler being called with the mangled interfcace name _if and it then calls ifconfig with it. Here is my fix. Any reason not to commit it? My diff is against 10-stable, but head looks the same. ################# --- /etc/network.subr.orig 2014-06-01 17:30:38.000000000 +0000 +++ /etc/network.subr 2014-06-01 18:03:08.030175024 +0000 @@ -1151,7 +1151,7 @@ inet|inet6|ipx|link|ether) case $_tmpargs in ${_af}\ *) - eval ifalias_af_common_handler $_if $_af $_action $_tmpargs && _ret=0 + eval ifalias_af_common_handler $1 $_af $_action $_tmpargs && _ret=0 ;; esac _tmpargs=$_c @@ -1163,7 +1163,7 @@ # Process the last component case $_tmpargs in ${_af}\ *) - ifalias_af_common_handler $_if $_af $_action $_tmpargs && _ret=0 + ifalias_af_common_handler $1 $_af $_action $_tmpargs && _ret=0 ;; esac ################# While looking through the code I saw that ltr is called with different styling. Is there a reason for it? Which is the prefered style? ltr ${_if} "${_punct}" '_' _if ltr "$_if" "$_punct" "_" _if My own preference would be the first. Regards John -- John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 11:57:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E04FA95 for ; Thu, 19 Jun 2014 11:57:58 +0000 (UTC) Received: from mailout.glevia.com (ns368239.ip-94-23-31.eu [94.23.31.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 145E12764 for ; Thu, 19 Jun 2014 11:57:57 +0000 (UTC) Received: from guest201.guestnet.ripe.net (guest201.guestnet.ripe.net [193.0.10.232]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mailout.glevia.com (Postfix) with ESMTPSA id 1186020AFDF8 for ; Thu, 19 Jun 2014 11:56:56 +0000 (UTC) Message-ID: <53A2D036.8000007@glevia.com> Date: Thu, 19 Jun 2014 13:57:42 +0200 From: Massimiliano Stucchi User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD References: <53A254F7.4040801@bluerosetech.com> In-Reply-To: <53A254F7.4040801@bluerosetech.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 11:57:58 -0000 On 19/06/14 05:11, Darren Pilgrim wrote: > > FreeBSD doesn't support 6rd. Ironically, pfSense does. This is not entirely true. 6RD is about establishing a 6to4 tunnel to a well-defined tunnel server in your provider's infrastructure, so as long as you have the details about the tunnel server's IP and the prefix you're assigned, you can easily do it manually (or, better, write a tiny script to do it for you). Cheers! -- Massimiliano Stucchi From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 12:40:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A04D9E34 for ; Thu, 19 Jun 2014 12:40:59 +0000 (UTC) Received: from smtpbg299.qq.com (smtpbg299.qq.com [184.105.67.99]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8497E2B44 for ; Thu, 19 Jun 2014 12:40:58 +0000 (UTC) X-QQ-FEAT: 8B5boNG7rimZblfzaTJ84oNo3B1b89xYTkU5ZAr4WuT+kfACaP2CC2JTmZlfu iVRUkUpF16iOtfojVJ/3s8hoLeVl2KqRQkcfs2/lG7qeiHTM36pBZhr7Frol4YL24EPxpVc GBAwnWwFT/yqPBd/lmhRthO8D6JoDibvX56svfuymRSDxPGDqw== X-QQ-SSF: 000000000000000000000000000000Z X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 166.111.3.31 In-Reply-To: References: X-QQ-STYLE: X-QQ-mid: webmail196t1403181648t5791748 From: "=?gb18030?B?1cXqzw==?=" To: "=?gb18030?B?ZnJlZWJzZC1uZXQ=?=" Subject: pow function in kernel space Mime-Version: 1.0 Date: Thu, 19 Jun 2014 20:40:48 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-ReplyHash: 2403348011 X-QQ-SENDSIZE: 520 X-QQ-FName: 1B5AB7C1B0484A20AC1B2568866F03D2 X-QQ-LocalIP: 112.95.241.173 Content-Type: text/plain; charset="gb18030" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 12:40:59 -0000 aG93IGNhbiBJIGltcGxlbWVudCwgaW4gYW4gZWZmaWNpZW50LCB3YXkgdGhlIHBvdygpIGZ1 bmN0aW9uIGluIGtlcm5lbCBzcGFjZSA/ICBJcyB0aGVyZSBhbnkgZnVuY3Rpb24gSSBjYW4g dXNlIG8gciBob3cgSSBjYW4gZXZhbHVhdGUgcG93IGZ1bmN0aW9uIGluIGtlcm5lbCBtb2Rl bD8NCiAgDQogVGhhbmtzIQ0KIA0KIA0KICAtLS0tLS0tLS0tLS0tLS0tLS0NCiAgDQrXo7rD o6wNCiANCtXF6s8NCiANCiANCiANCi0tLS0tLS0tLS0gDQogDQpIYW5aaGFuZw0KIA0KU2No b29sIG9mIENvbXB1dGVyIFNjaWVuY2UNClRzaW5naHVhIFVuaXZlcnNpdHksIEJlaWppbmcs IDEwMDA4NCwgUC5SLkNISU5BDQogDQpNb2JpbGU6ICArODYgMTU2LTUyNi01OTc4Mg0KRS1t YWlsOiAgIHpoYW5naGFuQGNzbmV0MS5jcy50c2luZ2h1YS5lZHUuY24NCg0KICANCiAgDQoN CiANCg0KIC0tLS0tLS0tLS0tLS0tLS0tLSBPcmlnaW5hbCAtLS0tLS0tLS0tLS0tLS0tLS0N CiAgRnJvbTogICJmcmVlYnNkLW5ldC1yZXF1ZXN0Ijs8ZnJlZWJzZC1uZXQtcmVxdWVzdEBm cmVlYnNkLm9yZz47DQogRGF0ZTogIFdlZCwgSnVuIDE4LCAyMDE0IDA4OjAwIFBNDQogVG86 ICAiZnJlZWJzZC1uZXQiPGZyZWVic2QtbmV0QGZyZWVic2Qub3JnPjsgDQogDQogU3ViamVj dDogIGZyZWVic2QtbmV0IERpZ2VzdCwgVm9sIDU4NSwgSXNzdWUgMw0KDQogDQoNClNlbmQg ZnJlZWJzZC1uZXQgbWFpbGluZyBsaXN0IHN1Ym1pc3Npb25zIHRvDQpmcmVlYnNkLW5ldEBm cmVlYnNkLm9yZw0KDQpUbyBzdWJzY3JpYmUgb3IgdW5zdWJzY3JpYmUgdmlhIHRoZSBXb3Js ZCBXaWRlIFdlYiwgdmlzaXQNCmh0dHA6Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2ZyZWVic2QtbmV0DQpvciwgdmlhIGVtYWlsLCBzZW5kIGEgbWVzc2FnZSB3aXRo IHN1YmplY3Qgb3IgYm9keSAnaGVscCcgdG8NCmZyZWVic2QtbmV0LXJlcXVlc3RAZnJlZWJz ZC5vcmcNCg0KWW91IGNhbiByZWFjaCB0aGUgcGVyc29uIG1hbmFnaW5nIHRoZSBsaXN0IGF0 DQpmcmVlYnNkLW5ldC1vd25lckBmcmVlYnNkLm9yZw0KDQpXaGVuIHJlcGx5aW5nLCBwbGVh c2UgZWRpdCB5b3VyIFN1YmplY3QgbGluZSBzbyBpdCBpcyBtb3JlIHNwZWNpZmljDQp0aGFu ICJSZTogQ29udGVudHMgb2YgZnJlZWJzZC1uZXQgZGlnZXN0Li4uIg0KDQoNClRvZGF5J3Mg VG9waWNzOg0KDQogICAxLiBSZTogbmV0bWFwIChDYXJsb3MgRmVycmVpcmEpDQogICAyLiBV bnVzYWJsZSBFbXVsZXggIm9jZSIgZHJpdmVyIHZlcnNpb24gaW4gLVNUQUJMRSwgOS4zICBh bmQNCiAgICAgIENVUlJFTlQgKEJvcmphIE1hcmNvcykNCiAgIDMuIFtCdWcgMTgzMzkxXSBb b2NlXSAxMGdpZ2FiaXQgbmV0d29ya2luZyBwcm9ibGVtcyB3aXRoIEVtdWxleA0KICAgICAg T0NFIDExMTAyIENOQSAoYnVnemlsbGEtbm9yZXBseUBmcmVlYnNkLm9yZykNCiAgIDQuIFtC dWcgMTgzMzkxXSBbb2NlXSAxMEdiRSBuZXR3b3JraW5nIHByb2JsZW1zIHdpdGggRW11bGV4 IE9DRQ0KICAgICAgMTExMDIgQ05BIChidWd6aWxsYS1ub3JlcGx5QGZyZWVic2Qub3JnKQ0K ICAgNS4gW0J1ZyAxODMzOTFdIFtvY2VdIDEwR2JFIG5ldHdvcmtpbmcgcHJvYmxlbXMgd2l0 aCBFbXVsZXggT0NFDQogICAgICAxMTEwMiBDTkEgKGJ1Z3ppbGxhLW5vcmVwbHlAZnJlZWJz ZC5vcmcpDQogICA2LiBbQnVnIDE4MzM5MV0gW29jZV0gMTBHYkUgbmV0d29ya2luZyBwcm9i bGVtcyB3aXRoIEVtdWxleCBPQ0UNCiAgICAgIDExMTAyIENOQSAoYnVnemlsbGEtbm9yZXBs eUBmcmVlYnNkLm9yZykNCiAgIDcuIFJlOiBVbnVzYWJsZSBFbXVsZXggIm9jZSIgZHJpdmVy IHZlcnNpb24gaW4gLVNUQUJMRSwgOS4zICBhbmQNCiAgICAgIENVUlJFTlQgKEJvcmphIE1h cmNvcykNCg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCk1lc3NhZ2U6IDENCkRhdGU6IFR1ZSwg MTcgSnVuIDIwMTQgMTU6MDM6MzAgKzAxMDANCkZyb206IENhcmxvcyBGZXJyZWlyYSA8Y2Fy bG9zbWYucHRAZ21haWwuY29tPg0KVG86IEx1aWdpIFJpenpvIDxyaXp6b0BpZXQudW5pcGku aXQ+DQpDYzogImZyZWVic2QtbmV0QGZyZWVic2Qub3JnIiA8ZnJlZWJzZC1uZXRAZnJlZWJz ZC5vcmc+DQpTdWJqZWN0OiBSZTogbmV0bWFwDQpNZXNzYWdlLUlEOg0KPENBSnBZWTZYQmc4 PWc3c2pWYnUrNitFRzEtdlcxZHltc3ZEOHBMLUVCMDhCPV96PXVzQUBtYWlsLmdtYWlsLmNv bT4NCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOA0KDQpHcmVhdCEg OikNCkkgd2lsbCBnaXZlIHlvdSB0aGUgcmVzdWx0cyBhcyBzb29uIGFzIEkgY2FuIGdldCB0 aGVtIDopDQoNCg0KDQpPbiAxNyBKdW5lIDIwMTQgMTI6NTUsIEx1aWdpIFJpenpvIDxyaXp6 b0BpZXQudW5pcGkuaXQ+IHdyb3RlOg0KDQo+IE9uIE1vbiwgSnVuIDE2LCAyMDE0IGF0IDU6 MzAgUE0sIENhcmxvcyBGZXJyZWlyYSA8Y2FybG9zbWYucHRAZ21haWwuY29tPg0KPiB3cm90 ZToNCj4NCj4+IE9rLCB0aGFua3MgZm9yIHRoZSBlbmxpZ2h0ZW5tZW50IHJlZ2FyZGluZyB0 aGUgbG9zcyBvZiBwZXJmb3JtYW5jZS4NCj4+DQo+PiBPbmUgcXVlc3Rpb24sIGp1c3QgdG8g YmUgc3VyZS4gRG9lcyB0aGUga2VybmVsIG1vZHVsZSBjb250YWlucyB0aGUgVkFMQQ0KPj4g c3dpdGNoIGNvZGU/IE9yIGRvIEkgbmVlZCB0byBjb21waWxlIGV4dHJhIGNvZGUgdG8gaGF2 ZSB0aGUgc3dpdGNoIHdvcmtpbmc/DQo+PiBBbHNvLCB3aGVyZSBjYW4gSSBmaW5kIHRoZSBk b2N1bWVudGF0aW9uIHRvIHVzZSB0aGUgVmFsYSBTd2l0Y2g/DQo+Pg0KPg0KPiA/VkFMRSBp cyBwYXJ0IG9mIHRoZSBuZXRtYXAga2VybmVsIG1vZHVsZSwgdGhlIG9ubHkgdGhpbmcgeW91 IG5lZWQNCj4gdG8ga25vdyB0byB1c2UgaXQgaXMgcG9ydCBuYW1lczoNCj4geW91IGNhbiBo YXZlIG11bHRpcGxlIHN3aXRjaCBpbnN0YW5jZXMgd2l0aCBtdWx0aXBsZSBwb3J0cyBlYWNo LA0KPg0KPiB2YWxlWDpZIG1lYW5zIHBvcnQgWSBvbiBzd2l0Y2ggWCwgWCBhbmQgWSBhcmUg YXJiaXRyYXJ5IHN0cmluZ3MNCj4gd2l0aCB0aGUgY29uc3RyYWludCB0aGF0IHRoZSB3aG9s ZSBuYW1lIG11c3QgZml0IDE1IGNoYXJhY3RlcnMuDQo+DQo+IERldGFpbHMgaW4gdGhlIG5l dG1hcCBtYW5wYWdlDQo+DQo+IGNoZWVycw0KPiBsdWlnaQ0KPg0KPg0KDQoNCi0tIA0KDQpD YXJsb3MgTWlndWVsIEZlcnJlaXJhDQpSZXNlYXJjaGVyIGF0IFRlbGVjb21tdW5pY2F0aW9u cyBJbnN0aXR1dGUNCkF2ZWlybyAtIFBvcnR1Z2FsDQpXb3JrIEUtbWFpbCAtIGNtZkBhdi5p dC5wdA0KU2t5cGUgJiBHVGFsayAtPiBjYXJsb3NtZi5wdEBnbWFpbC5jb20NCkxpbmtlZElu IC0+IGh0dHA6Ly93d3cubGlua2VkaW4uY29tL2luL2Nhcmxvc21mZXJyZWlyYQ0KDQoNCi0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQpNZXNzYWdlOiAyDQpEYXRlOiBUdWUs IDE3IEp1biAyMDE0IDE4OjA3OjQ1ICswMjAwDQpGcm9tOiBCb3JqYSBNYXJjb3MgPGJvcmph bUBzYXJlbmV0LmVzPg0KVG86IFN0YWJsZSBTdGFibGUgPGZyZWVic2Qtc3RhYmxlQGZyZWVi c2Qub3JnPg0KQ2M6IGZyZWVic2QtbmV0QGZyZWVic2Qub3JnDQpTdWJqZWN0OiBVbnVzYWJs ZSBFbXVsZXggIm9jZSIgZHJpdmVyIHZlcnNpb24gaW4gLVNUQUJMRSwgOS4zICBhbmQNCkNV UlJFTlQNCk1lc3NhZ2UtSUQ6IDw4RDkzRURGNy1BQzBFLTQ3MDctQUM2Mi01NUNCRDVGMzM1 OERAc2FyZW5ldC5lcz4NCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11cy1h c2NpaQ0KDQoNCkhlbGxvLA0KDQpUaGlzIHNob3VsZCBiZSBmaXhlZCBiZWZvcmUgdGhlIHJl bGVhc2Ugb2YgOS4zLCBhbmQgb2YgY291cnNlIGFzIHNvb24gYXMgcG9zc2libGUgZm9yIHRo ZSByZXN0IG9mIHRoZSBicmFuY2hlcy4NCg0KQXQgbGVhc3QgdW5kZXIgMTAtU1RBQkxFIHRo ZSAib2NlIiBkcml2ZXIgY2FuIGNhdXNlIGEgcGFuaWMgd2hlbiB0aGVyZSdzIGEgbG90IG9m IG5ldHdvcmsgdHJhZmZpYy4gSSBoYXZlIGJlZW4gYWJsZSB0byByZXByb2R1Y2UgaXQNCnVz aW5nICJpcGVyZiIuIFNvbWV0aW1lcyB0aGUgcGFuaWMgY2FuIGJlIHRyaWdnZXJlZCB3aXRo aW4gc2Vjb25kcyBvZiBzdGFydGluZyBhbiBpcGVyZiB0ZXN0Lg0KDQpUaGVyZSdzIGEgbmV3 IGRyaXZlciBhdmFpbGFibGUgZm9yIGRvd25sb2FkIG9uIHRoZSBFbXVsZXggc2l0ZSwgdmVy c2lvbiAxMC4wLjc0Ny4wLCBhbmQgaXQgd29ya3MgcGVyZmVjdGx5LiBUaGUgc291cmNlIGNv ZGUgY29tcGlsZXMgcGVyZmVjdGx5DQp1bmRlciBGcmVlQlNEIDEwLVNUQUJMRSAoYXQgbGVh c3QgcmVjZW50IHZlcnNpb25zIHNpbmNlIEFwcmlsKSBhbmQgdGhlIGtlcm5lbCBtb2R1bGUg bG9hZHMgd2l0aG91dCB0cm91YmxlLg0KDQpUaGVyZSdzIGEgYnVnIHJlcG9ydCBjb3Zlcmlu ZyB0aGlzIGJ1dCBzZWVtcyB0byBiZSBmb3Jnb3R0ZW4gOi8NCg0KaHR0cHM6Ly9idWdzLmZy ZWVic2Qub3JnL2J1Z3ppbGxhL3Nob3dfYnVnLmNnaT9pZD0xODMzOTENCg0KDQoNClRoYW5r cyENCg0KDQoNCg0KQm9yamEuDQoNCg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0NCg0KTWVzc2FnZTogMw0KRGF0ZTogVHVlLCAxNyBKdW4gMjAxNCAyMzozODo0MiArMDAw MA0KRnJvbTogYnVnemlsbGEtbm9yZXBseUBmcmVlYnNkLm9yZw0KVG86IGZyZWVic2QtbmV0 QEZyZWVCU0Qub3JnDQpTdWJqZWN0OiBbQnVnIDE4MzM5MV0gW29jZV0gMTBnaWdhYml0IG5l dHdvcmtpbmcgcHJvYmxlbXMgd2l0aCBFbXVsZXgNCk9DRSAxMTEwMiBDTkENCk1lc3NhZ2Ut SUQ6DQo8YnVnLTE4MzM5MS0yNDcyLXplalNYMUhsRW1AaHR0cHMuYnVncy5mcmVlYnNkLm9y Zy9idWd6aWxsYS8+DQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9IlVURi04 Ig0KDQpodHRwczovL2J1Z3MuZnJlZWJzZC5vcmcvYnVnemlsbGEvc2hvd19idWcuY2dpP2lk PTE4MzM5MQ0KDQpLdWJpbGF5IEtvY2FrIDxrb29ic0BGcmVlQlNELm9yZz4gY2hhbmdlZDoN Cg0KICAgICAgICAgICBXaGF0ICAgIHxSZW1vdmVkICAgICAgICAgICAgICAgICAgICAgfEFk ZGVkDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICAgICAgICAgICAgICAgIENDfCAgICAg ICAgICAgICAgICAgICAgICAgICAgICB8a29vYnNARnJlZUJTRC5vcmcNCiAgICAgICAgICAg IFZlcnNpb258dW5zcGVjaWZpZWQgICAgICAgICAgICAgICAgIHw5LjItUkVMRUFTRQ0KDQot LS0gQ29tbWVudCAjNyBmcm9tIEt1YmlsYXkgS29jYWsgPGtvb2JzQEZyZWVCU0Qub3JnPiAt LS0NClBsZWFzZSBhdHRhY2g6DQoNCiogL3Zhci9ydW4vZG1lc2cuYm9vdCBvdXRwdXQNCiog cGNpY29uZiBwY2ljb25mIC1sdmVWDQoNCi0tIA0KWW91IGFyZSByZWNlaXZpbmcgdGhpcyBt YWlsIGJlY2F1c2U6DQpZb3UgYXJlIHRoZSBhc3NpZ25lZSBmb3IgdGhlIGJ1Zy4NCg0KDQot LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KTWVzc2FnZTogNA0KRGF0ZTogVHVl LCAxNyBKdW4gMjAxNCAyMzo1OTowOSArMDAwMA0KRnJvbTogYnVnemlsbGEtbm9yZXBseUBm cmVlYnNkLm9yZw0KVG86IGZyZWVic2QtbmV0QEZyZWVCU0Qub3JnDQpTdWJqZWN0OiBbQnVn IDE4MzM5MV0gW29jZV0gMTBHYkUgbmV0d29ya2luZyBwcm9ibGVtcyB3aXRoIEVtdWxleCBP Q0UNCjExMTAyIENOQQ0KTWVzc2FnZS1JRDoNCjxidWctMTgzMzkxLTI0NzItTjFxV2JZOGl5 VkBodHRwcy5idWdzLmZyZWVic2Qub3JnL2J1Z3ppbGxhLz4NCkNvbnRlbnQtVHlwZTogdGV4 dC9wbGFpbjsgY2hhcnNldD0iVVRGLTgiDQoNCmh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy9i dWd6aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTgzMzkxDQoNCkt1YmlsYXkgS29jYWsgPGtvb2Jz QEZyZWVCU0Qub3JnPiBjaGFuZ2VkOg0KDQogICAgICAgICAgIFdoYXQgICAgfFJlbW92ZWQg ICAgICAgICAgICAgICAgICAgICB8QWRkZWQNCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAg ICAgICAgICAgICAgICAgQ0N8ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHxtbGF2a2lu QGdtYWlsLmNvbQ0KICAgICAgICAgICAgU3VtbWFyeXxbb2NlXSAxMGdpZ2FiaXQgbmV0d29y a2luZyAgfFtvY2VdIDEwR2JFIG5ldHdvcmtpbmcNCiAgICAgICAgICAgICAgICAgICB8cHJv YmxlbXMgd2l0aCBFbXVsZXggT0NFICAgIHxwcm9ibGVtcyB3aXRoIEVtdWxleCBPQ0UNCiAg ICAgICAgICAgICAgICAgICB8MTExMDIgQ05BICAgICAgICAgICAgICAgICAgIHwxMTEwMiBD TkENCg0KLS0tIENvbW1lbnQgIzggZnJvbSBLdWJpbGF5IEtvY2FrIDxrb29ic0BGcmVlQlNE Lm9yZz4gLS0tDQpQYXRha2ksIEJvcmphbiwgTUxhdmtpbiwNCg0KV2UgbmVlZCBhIGNvbmZp cm1hdGlvbiB0aGF0IHRoaXMgaXNzdWUgKHBhbmljKSBpcyBzdGlsbCBwcmVzZW50IG9uIDku My1CRVRBMyBpbg0Kb3JkZXIgdG8gZ2V0IHRoaXMgaW50byA5LjMtUkVMRUFTRS4gDQoNCldl IGhhdmUgYSB2ZXJ5IHNob3J0IHdpbmRvdyB1bnRpbCAtUkVMRUFTRSwgaWYgb25lIG9mIHlv dSBjb3VsZCBwcm92aWRlIHRoYXQNCmNvbmZpcm1hdGlvbiwgaXQgd2lsbCBnbyBhIGxvbmcg d2F5Lg0KDQotLSANCllvdSBhcmUgcmVjZWl2aW5nIHRoaXMgbWFpbCBiZWNhdXNlOg0KWW91 IGFyZSB0aGUgYXNzaWduZWUgZm9yIHRoZSBidWcuDQoNCg0KLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tDQoNCk1lc3NhZ2U6IDUNCkRhdGU6IFdlZCwgMTggSnVuIDIwMTQgMDI6 MjA6MzAgKzAwMDANCkZyb206IGJ1Z3ppbGxhLW5vcmVwbHlAZnJlZWJzZC5vcmcNClRvOiBm cmVlYnNkLW5ldEBGcmVlQlNELm9yZw0KU3ViamVjdDogW0J1ZyAxODMzOTFdIFtvY2VdIDEw R2JFIG5ldHdvcmtpbmcgcHJvYmxlbXMgd2l0aCBFbXVsZXggT0NFDQoxMTEwMiBDTkENCk1l c3NhZ2UtSUQ6DQo8YnVnLTE4MzM5MS0yNDcyLVVTSjJwV3JFYWJAaHR0cHMuYnVncy5mcmVl YnNkLm9yZy9idWd6aWxsYS8+DQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9 IlVURi04Ig0KDQpodHRwczovL2J1Z3MuZnJlZWJzZC5vcmcvYnVnemlsbGEvc2hvd19idWcu Y2dpP2lkPTE4MzM5MQ0KDQotLS0gQ29tbWVudCAjOSBmcm9tIHBhdGFraS5hbnRhbEBnbWFp bC5jb20gLS0tDQooSW4gcmVwbHkgdG8gS3ViaWxheSBLb2NhayBmcm9tIGNvbW1lbnQgIzcp DQo+IFBsZWFzZSBhdHRhY2g6DQo+IA0KPiAqIC92YXIvcnVuL2RtZXNnLmJvb3Qgb3V0cHV0 DQo+ICogcGNpY29uZiBwY2ljb25mIC1sdmVWDQoNCkhpIQ0KDQpJIGFtIHNvIHNvcnJ5LCBi dXQgd2UgZGlkbid0IGtlcHQgdGhlIHVudXNhYmxlIHN5c3RlbSBmcm9tIDIwMTMgb2N0b2Jl ciB1bnRpbA0KdG9kYXkuLi4uDQpUaGUgc3lzdGVtIGlzIHVwZ3JhZGVkIHRvIDkuMi1TVEFC TEUgYW5kIHRoZSBFbXVsZXggY2FyZHMgZmlyZWQgb3V0IGFuZA0KcmVwbGFjZWQgd2l0aCBJ bnRlbCBYNTIwIGNhcmRzLg0KTGlrZSB0aGlzLCB0aGUgc3lzdGVtIGlzIG9wZXJhdGluZyBn b29kIC0gaW4gYSBwcm9kdWN0aW9uIGVudmlyb25tZW50LCBzbyBpdCBpcw0Kbm90IHBvc3Np YmxlIHRvIGNoYW5nZSB0aGUgaGFyZHdhcmUgb3IgdGhlIG9wZXJhdGluZyBzeXN0ZW0uDQoN ClNvIEkgY2FuJ3Qgc2VuZCB5b3UgdGhlIGRtZXNnLmJvb3QgZnJvbSAyMDEzIG9jdG9iZXIg d2l0aCB0aGUgZW11bGV4IGNhcmQgb24NCmZic2QgOS4yLVJFTEVBU0UsIEkgYW0gc29ycnku DQoNCi0tIA0KWW91IGFyZSByZWNlaXZpbmcgdGhpcyBtYWlsIGJlY2F1c2U6DQpZb3UgYXJl IHRoZSBhc3NpZ25lZSBmb3IgdGhlIGJ1Zy4NCg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0NCg0KTWVzc2FnZTogNg0KRGF0ZTogV2VkLCAxOCBKdW4gMjAxNCAwMjoyMzoy NyArMDAwMA0KRnJvbTogYnVnemlsbGEtbm9yZXBseUBmcmVlYnNkLm9yZw0KVG86IGZyZWVi c2QtbmV0QEZyZWVCU0Qub3JnDQpTdWJqZWN0OiBbQnVnIDE4MzM5MV0gW29jZV0gMTBHYkUg bmV0d29ya2luZyBwcm9ibGVtcyB3aXRoIEVtdWxleCBPQ0UNCjExMTAyIENOQQ0KTWVzc2Fn ZS1JRDoNCjxidWctMTgzMzkxLTI0NzItQVVCTklWN3o2VUBodHRwcy5idWdzLmZyZWVic2Qu b3JnL2J1Z3ppbGxhLz4NCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD0iVVRG LTgiDQoNCmh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy9idWd6aWxsYS9zaG93X2J1Zy5jZ2k/ aWQ9MTgzMzkxDQoNCi0tLSBDb21tZW50ICMxMCBmcm9tIHBhdGFraS5hbnRhbEBnbWFpbC5j b20gLS0tDQpMaWtlIG15IHByZXZpb3VzIHJlcGx5LCBJIGNhbid0IGNvbmZpcm0gdGhpcywg YmVjYXVzZSB3ZSB3b3JrYXJvdW5kZWQgdGhlIGJ1Zw0KaW4gYSB3YXkuDQoodXBncmFkZWQg dG8gZnJlZWJzZCA5LjItc3RhYmxlIGFuZCB0aGUgZW11bGV4IGNhcmRzIGFyZSBjaGFuZ2Vk IHRvIGludGVsIHg1MjANCmNhcmRzKQ0KDQpJIGFtIHNvIHNvcnJ5LCB3ZSBjb3VsZG4ndCB3 YWl0IGZvciBhYm91dCA4IG1vbnRocyBmb3IgYW55IHNvbHV0aW9uLCB3ZSBuZWVkZWQNCnRv IHVzZSB0aGUgc2VydmVycyBpbiBwcm9kdWN0aW9uIC0gd2Ugc3BlbnQgbWFueSB0aG91c2Fu ZHMgb2YgZG9sbGFycyBmb3IgdGhhdCwNCm5vdCBmb3Iga2VlcGluZyB0aGVtIGluIHRoZSBj b3JuZXIuIEkgaG9wZSB5b3UgdW5kZXJzdG9vZC4NCg0KKEluIHJlcGx5IHRvIEt1YmlsYXkg S29jYWsgZnJvbSBjb21tZW50ICM4KQ0KPiBQYXRha2ksIEJvcmphbiwgTUxhdmtpbiwNCj4g DQo+IFdlIG5lZWQgYSBjb25maXJtYXRpb24gdGhhdCB0aGlzIGlzc3VlIChwYW5pYykgaXMg c3RpbGwgcHJlc2VudCBvbiA5LjMtQkVUQTMNCj4gaW4gb3JkZXIgdG8gZ2V0IHRoaXMgaW50 byA5LjMtUkVMRUFTRS4gDQo+IA0KPiBXZSBoYXZlIGEgdmVyeSBzaG9ydCB3aW5kb3cgdW50 aWwgLVJFTEVBU0UsIGlmIG9uZSBvZiB5b3UgY291bGQgcHJvdmlkZSB0aGF0DQo+IGNvbmZp cm1hdGlvbiwgaXQgd2lsbCBnbyBhIGxvbmcgd2F5Lg0KDQotLSANCllvdSBhcmUgcmVjZWl2 aW5nIHRoaXMgbWFpbCBiZWNhdXNlOg0KWW91IGFyZSB0aGUgYXNzaWduZWUgZm9yIHRoZSBi dWcuDQoNCg0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCk1lc3NhZ2U6IDcN CkRhdGU6IFdlZCwgMTggSnVuIDIwMTQgMDk6MzU6MjYgKzAyMDANCkZyb206IEJvcmphIE1h cmNvcyA8Ym9yamFtQHNhcmVuZXQuZXM+DQpUbzogQm9yamEgTWFyY29zIDxib3JqYW1Ac2Fy ZW5ldC5lcz4NCkNjOiBmcmVlYnNkLW5ldEBmcmVlYnNkLm9yZywgU3RhYmxlIFN0YWJsZQ0K PGZyZWVic2Qtc3RhYmxlQGZyZWVic2Qub3JnPg0KU3ViamVjdDogUmU6IFVudXNhYmxlIEVt dWxleCAib2NlIiBkcml2ZXIgdmVyc2lvbiBpbiAtU1RBQkxFLCA5LjMgIGFuZA0KQ1VSUkVO VA0KTWVzc2FnZS1JRDogPEUwNjU2QkRBLTU2QjEtNEI3Ny1COTQ5LUYxOTczQjgyMzg1RkBz YXJlbmV0LmVzPg0KQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PXVzLWFzY2lp DQoNCg0KT24gSnVuIDE3LCAyMDE0LCBhdCA2OjA3IFBNLCBCb3JqYSBNYXJjb3Mgd3JvdGU6 DQoNCj4gDQo+IEhlbGxvLA0KPiANCj4gVGhpcyBzaG91bGQgYmUgZml4ZWQgYmVmb3JlIHRo ZSByZWxlYXNlIG9mIDkuMywgYW5kIG9mIGNvdXJzZSBhcyBzb29uIGFzIHBvc3NpYmxlIGZv ciB0aGUgcmVzdCBvZiB0aGUgYnJhbmNoZXMuDQo+IA0KPiBBdCBsZWFzdCB1bmRlciAxMC1T VEFCTEUgdGhlICJvY2UiIGRyaXZlciBjYW4gY2F1c2UgYSBwYW5pYyB3aGVuIHRoZXJlJ3Mg YSBsb3Qgb2YgbmV0d29yayB0cmFmZmljLiBJIGhhdmUgYmVlbiBhYmxlIHRvIHJlcHJvZHVj ZSBpdA0KPiB1c2luZyAiaXBlcmYiLiBTb21ldGltZXMgdGhlIHBhbmljIGNhbiBiZSB0cmln Z2VyZWQgd2l0aGluIHNlY29uZHMgb2Ygc3RhcnRpbmcgYW4gaXBlcmYgdGVzdC4NCj4gDQo+ IFRoZXJlJ3MgYSBuZXcgZHJpdmVyIGF2YWlsYWJsZSBmb3IgZG93bmxvYWQgb24gdGhlIEVt dWxleCBzaXRlLCB2ZXJzaW9uIDEwLjAuNzQ3LjAsIGFuZCBpdCB3b3JrcyBwZXJmZWN0bHku IFRoZSBzb3VyY2UgY29kZSBjb21waWxlcyBwZXJmZWN0bHkNCj4gdW5kZXIgRnJlZUJTRCAx MC1TVEFCTEUgKGF0IGxlYXN0IHJlY2VudCB2ZXJzaW9ucyBzaW5jZSBBcHJpbCkgYW5kIHRo ZSBrZXJuZWwgbW9kdWxlIGxvYWRzIHdpdGhvdXQgdHJvdWJsZS4NCj4gDQo+IFRoZXJlJ3Mg YSBidWcgcmVwb3J0IGNvdmVyaW5nIHRoaXMgYnV0IHNlZW1zIHRvIGJlIGZvcmdvdHRlbiA6 Lw0KPiANCj4gaHR0cHM6Ly9idWdzLmZyZWVic2Qub3JnL2J1Z3ppbGxhL3Nob3dfYnVnLmNn aT9pZD0xODMzOTENCg0KSW4gbXkgY2FzZSwgdGhlIHByb2JsZW0gbWFuaWZlc3RzIGl0c2Vs ZiBydW5uaW5nIDEwLiBFbXVsZXggcHJvdmlkZXMgYSBtb3JlIHJlY2VudCBkcml2ZXIgd2hp Y2ggY29tcGlsZXMgcGVyZmVjdGx5IGFuZCB3b3JrcyBwZXJmZWN0bHkgKGlmIHdlIGNhbiBz YWZlbHkgYXNzdW1lIHRoYXQgYSB3ZWVrZW5kIHNhdHVyYXRpbmcgYSBjb3VwbGUgb2YgaW50 ZXJmYWNlcyBydW5ubmluZyBiZW5jaG1hcmtzIHdpdGggbm8gZ2xpdGNoZXMgaXMgYSBnb29k IGhpbnQpLg0KDQpJIGFtIG5vdyBpbnN0YWxsaW5nIHRoZSBsYXRlc3QgYmV0YSBvZiA5LjMg dG8gc2VlIGlmIEkgY2FuIHJlcHJvZHVjZSB0aGUgcGFuaWMuIEkgd2lsbCBmb2xsb3ctdXAg aW4gYSBjb3VwbGUgb2YgaG91cnMgaWYgcG9zc2libGUuDQoNCg0KDQoNCg0KQm9yamEuDQoN Cg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KU3ViamVjdDogRGlnZXN0 IEZvb3Rlcg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXw0KZnJlZWJzZC1uZXRAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0DQpodHRwOi8vbGlz dHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLW5ldA0KVG8gdW5zdWJz Y3JpYmUsIHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtbmV0LXVuc3Vic2NyaWJlQGZyZWVi c2Qub3JnIg0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KRW5kIG9mIGZy ZWVic2QtbmV0IERpZ2VzdCwgVm9sIDU4NSwgSXNzdWUgMw0KKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKg0KLg== From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 13:13:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA31A572 for ; Thu, 19 Jun 2014 13:13:58 +0000 (UTC) Received: from udns.ultimateDNS.NET (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7BC522ED7 for ; Thu, 19 Jun 2014 13:13:58 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s5JDECGs049527; Thu, 19 Jun 2014 06:14:18 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s5JDE7ij049523; Thu, 19 Jun 2014 06:14:07 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from 2602:d1:b4d6:e600:4261:86ff:fef6:aa2a (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Thu, 19 Jun 2014 06:14:07 -0700 (PDT) Message-ID: <575890bf103e4c9039f0bbbfa1bd5a01.authenticated@ultimatedns.net> In-Reply-To: <53A2A3CC.5030407@bluerosetech.com> References: <53A254F7.4040801@bluerosetech.com> <71fc86db55a80c38420515eb7181758e.authenticated@ultimatedns.net> <53A2A3CC.5030407@bluerosetech.com> Date: Thu, 19 Jun 2014 06:14:07 -0700 (PDT) Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD From: "Chris H" To: "freebsd-net" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: list_freebsd@bluerosetech.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 13:13:58 -0000 > On 6/18/2014 10:12 PM, Chris H wrote: >>> FreeBSD doesn't support 6rd. Ironically, pfSense does. > > >> Are you sure? >> There are even a couple of 6rd ports: >> net/stf-6rd-kmod >> and >> net/u6rd >> or am I to understand that _without_ those ports, FreeBSD doesn't >> support 6rd. > > Yes, if you bring in third-party code you can do 6rd on FreeBSD. I > meant that FreeBSD does not have 6rd support in the base like it does > for 6to4 and 6in4. There have been patches available for years that add > 6rd feature suppport to if_stf, but for some reason they've never made > it into FreeBSD despite making into downstream projects. Ahh. That's what I was afraid you were saying. :/ Well. I've got the source for the CPE, and given all the recent work in arm@ I think it might be time to replace the OS on it, and put FreeBSD on it, with _real_ IPv6. Or, if I can figure out how to wire tip, and ring (a POTS line) to USB. Create my own CPE. :) Thanks for taking the time to reply, Darren. --Chris > From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 13:37:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E971396A for ; Thu, 19 Jun 2014 13:37:05 +0000 (UTC) Received: from udns.ultimateDNS.NET (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9A0E2145 for ; Thu, 19 Jun 2014 13:37:05 +0000 (UTC) Received: from udns.ultimateDNS.NET (localhost [127.0.0.1]) by udns.ultimateDNS.NET (8.14.5/8.14.5) with ESMTP id s5JDbLrh050505; Thu, 19 Jun 2014 06:37:27 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: (from www@localhost) by udns.ultimateDNS.NET (8.14.5/8.14.5/Submit) id s5JDbE6m050499; Thu, 19 Jun 2014 06:37:14 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) Received: from 2602:d1:b4d6:e600:4261:86ff:fef6:aa2a (UDNSMS authenticated user chrish) by ultimatedns.net with HTTP; Thu, 19 Jun 2014 06:37:16 -0700 (PDT) Message-ID: <637607d1a0e0efb9c27f6cf44540d794.authenticated@ultimatedns.net> In-Reply-To: <53A2D036.8000007@glevia.com> References: <53A254F7.4040801@bluerosetech.com> <53A2D036.8000007@glevia.com> Date: Thu, 19 Jun 2014 06:37:16 -0700 (PDT) Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD From: "Chris H" To: "Massimiliano Stucchi" User-Agent: UDNSMS/2.0.3 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 13:37:06 -0000 > On 19/06/14 05:11, Darren Pilgrim wrote: > >> >> FreeBSD doesn't support 6rd. Ironically, pfSense does. > > This is not entirely true. 6RD is about establishing a 6to4 tunnel to a > well-defined tunnel server in your provider's infrastructure, so as long > as you have the details about the tunnel server's IP and the prefix > you're assigned, you can easily do it manually (or, better, write a tiny > script to do it for you). Yes. I have both. That's good news. Thanks for the info. --Chris > > Cheers! > > -- > > Massimiliano Stucchi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 14:08:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7814D99E; Thu, 19 Jun 2014 14:08:11 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DAE8246A; Thu, 19 Jun 2014 14:08:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=S+9cB91dmElRDhQxPf0jKFtaIishZBQPhRFNEkuHUUY=; b=sbwC3sKrPdH9OA5d1uLNSy4q0wVkxNCEWAbNSnpwxe3mSlc+wq6uxuG5we64OFeOVrFso4oioxLBpwnF1mutexa+Hkm1mspq+tQohC5a/plVQK03ALzZStNAec7xWuZe3f+9ZJubJ86kN18HswnpLChP+hZxDK6YQDDyXxrKs4g=; Received: from thebighonker.lerctr.org ([192.147.25.65]:58743) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Wxd0V-000H4y-3d; Thu, 19 Jun 2014 09:08:09 -0500 Date: Thu, 19 Jun 2014 09:08:01 -0500 From: Larry Rosenman To: freebsd-net@freebsd.org Subject: Re: IPv6: "xxx::x already configured" in logs... why? Message-ID: <20140619140801.GA65420@thebighonker.lerctr.org> Mail-Followup-To: freebsd-net@freebsd.org, Hiroki Sato References: <20140612202349.GA65079@thebighonker.lerctr.org> <20140619.181129.1279477227287764712.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140619.181129.1279477227287764712.hrs@allbsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 Cc: Hiroki Sato X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 14:08:11 -0000 On Thu, Jun 19, 2014 at 06:11:29PM +0900, Hiroki Sato wrote: > Larry Rosenman wrote > in <20140612202349.GA65079@thebighonker.lerctr.org>: > > le> I just started using IPv6 behind my (new to me) Cisco 1841. > le> > le> I see lots of: > le> Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured > le> > le> in my /var/log/messages. > le> > le> The rc.conf: > le> > le> cloned_interfaces="lagg0:sticky " > le> create_args_lagg0="laggport bce0 laggport bce1 laggproto loadbalance" > le> ifconfig_lagg0="192.147.25.65/24" > le> ifconfig_bce0="up" > le> ifconfig_bce1="up" > le> defaultrouter="192.147.25.1" > le> #################################################### > le> # IPv6 AutoConf: > le> ifconfig_lagg0_ipv6="inet6 accept_rtadv" > le> rtsold_enable="YES" > le> #################################################### > le> hostname=thebighonker.lerctr.org > le> ifconfig_lagg0_alias0="inet 192.147.25.45/24" > le> ifconfig_lagg0_alias1="inet 192.147.25.11/24" > le> ifconfig_lagg0_alias2="inet 192.147.25.66/24" > le> sshd_enable="YES" > le> ntpd_enable="YES" > le> powerd_enable="YES" > le> # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable > le> dumpdev="AUTO" > le> zfs_enable="YES" > le> # ------------ > le> > le> > le> hostid_enable="YES" > le> > le> postgresql_enable="YES" > le> fah_enable="NO" > le> fah2_enable="NO" > le> linux_enable="YES" > le> smartd_enable="YES" > le> spamd_enable="YES" > le> devcpu_enable="YES" > le> clamav_clamd_enable="YES" > le> clamav_freshclam_enable="YES" > le> sendmail_enable="NONE" > le> exim_enable="YES" > le> exim_flags="-bd -q5m" > le> named_enable="YES" > le> named_program="/usr/local/sbin/named" > le> bsdstats_enable="YES" > le> spamd_flags="-c -Q -m 20" > le> apache22_enable="YES" > le> syslogd_flags="-n -a 192.147.25.0/24:* -a 209.198.148.248/29:*" > le> bacula_fd_enable="YES" > le> sshblock_enable="YES" > le> dovecot_enable="YES" > le> monthly_stats_enable="YES" > le> boinc_client_enable="YES" > le> microcode_update_enable="YES" > le> ---- > le> sysctl.conf: > le> > le> # $FreeBSD: stable/10/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $ > le> # > le> # This file is read when going to multi-user and its contents piped thru > le> # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. > le> # > le> > le> # Uncomment this to prevent users from seeing information about processes that > le> # are being run under another UID. > le> #security.bsd.see_other_uids=0 > le> vm.lowmem_period=0 > le> vfs.usermount=1 > le> vfs.zfs.super_owner=1 > le> kern.elf32.fallback_brand=3 > le> kern.ipc.shm_allow_removed=1 > le> net.inet6.ip6.dad_count=0 > le> > le> ===== > le> > le> Ideas? (I may be an idiot, so any criticism welcomed). > le> > le> if you need the 1841's config, I can supply that as well. It's using a Hurricane > le> electric Tunnel. > > How frequent were the log message added into /var/log/messages? And > when did it start to happen after boot. Just after lagg0 is > configured? > > -- Hiroki Looks like: Jun 12 07:00:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:00:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:02:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:02:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:05:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:05:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:08:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:08:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:11:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:11:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:13:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:13:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:16:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:16:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:19:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:19:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:22:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:22:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:25:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:25:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:28:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:28:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:31:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:31:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:35:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:35:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:38:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:38:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:40:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:40:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:43:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:43:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:46:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:46:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:49:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:49:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:52:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:52:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:55:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:55:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:58:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 07:58:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:01:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:01:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:03:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:03:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:06:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:06:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:09:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:09:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:12:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:12:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:14:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:14:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:17:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:17:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:20:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:20:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:23:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:23:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:26:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:26:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:31:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:31:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:34:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:34:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:37:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:37:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:40:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:40:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:43:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:43:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:46:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:46:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:49:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:49:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:51:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:51:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:55:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:55:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:57:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 08:57:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:00:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:00:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:03:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:03:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:06:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:06:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:09:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:09:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:12:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:12:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:15:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:15:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:18:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:18:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:21:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:21:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:24:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:24:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:27:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:27:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:30:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:30:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:33:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:33:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:37:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:37:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:39:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:39:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:42:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:42:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:45:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:45:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:48:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:48:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:51:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:51:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:53:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:53:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:56:35 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:56:35 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:59:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 09:59:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:02:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:02:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:05:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:05:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:07:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:07:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:10:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:10:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:14:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:14:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:17:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:17:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:20:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:20:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:23:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:23:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:26:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:26:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:28:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:28:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:31:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:31:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:34:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:34:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:37:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:37:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:40:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:40:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:43:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:43:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:46:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:46:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:48:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:48:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:51:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:51:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:54:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:54:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:57:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 10:57:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:00:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:00:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:03:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:03:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:06:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:06:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:09:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:09:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:11:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:11:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:14:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:14:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:17:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:17:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:20:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:20:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:23:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:23:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:26:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:26:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:28:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:28:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:31:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:31:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:34:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:34:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:37:38 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:37:38 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:40:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:40:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:43:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:43:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:46:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:46:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:49:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:49:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:52:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:52:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:55:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:55:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:57:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 11:57:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:00:39 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:00:39 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:03:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:03:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:06:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:06:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:09:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:09:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:12:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:12:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:15:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:15:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:17:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:17:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:20:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:20:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:23:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:23:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:25:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:25:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:31:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:31:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:34:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:34:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:40:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:40:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:44:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:44:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:46:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:46:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:49:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:49:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:52:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:52:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:55:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:55:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:58:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 12:58:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:01:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:01:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:04:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:04:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:07:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:07:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:10:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:10:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:13:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:13:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:16:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:16:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:19:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:19:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:21:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:21:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:24:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:24:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:26:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:26:59 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:29:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:29:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:32:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:32:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:35:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:35:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:38:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:38:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:41:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:41:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:44:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:44:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:46:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:46:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:49:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:49:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:52:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:52:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:55:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:55:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:58:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 13:58:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:01:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:01:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:03:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:03:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:06:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:06:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:09:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:09:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:12:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:12:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:14:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:14:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:17:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:17:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:20:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:20:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:22:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:22:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:26:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:26:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:29:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:29:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:32:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:32:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:35:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:35:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:38:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:38:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:41:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:41:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:44:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:44:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:47:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:47:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:50:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:50:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:52:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:52:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:56:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:56:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:59:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 14:59:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:02:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:02:14 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:04:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:04:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:07:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:07:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:10:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:10:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:13:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:13:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:16:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:19:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:19:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:22:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:22:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:25:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:25:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:28:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:28:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:31:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:31:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:34:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:34:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:37:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:37:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:40:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:40:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:43:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:43:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:45:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:45:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:48:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:48:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:51:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:51:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:53:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:53:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:56:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:56:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:59:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 15:59:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:02:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:02:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:05:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:05:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:08:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:08:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:10:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:10:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:13:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:13:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:16:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:16:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:19:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:19:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:21:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:21:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:24:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:24:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:28:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:28:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:30:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:30:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:33:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:33:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:36:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:36:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:38:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:38:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:41:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:41:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:44:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:44:36 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:47:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:47:51 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:50:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:50:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:53:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:53:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:55:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:55:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:58:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 16:58:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:01:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:01:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:03:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:03:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:06:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:06:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:09:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:09:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:12:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:12:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:15:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:15:34 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:18:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:18:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:21:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:21:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:24:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:24:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:27:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:27:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:30:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:30:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:32:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:32:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:35:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:35:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:38:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:38:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:42:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:42:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:44:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:44:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:47:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:47:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:50:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:50:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:53:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:53:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:56:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:56:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:59:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 17:59:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:02:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:02:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:04:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:04:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:07:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:07:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:10:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:10:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:12:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:12:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:15:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:15:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:17:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:17:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:21:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:21:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:23:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:23:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:26:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:26:47 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:29:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:29:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:31:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:31:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:35:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:35:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:41:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:41:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:43:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:43:49 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:46:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:46:53 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:50:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:50:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:53:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:53:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:56:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:56:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:59:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 18:59:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:01:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:01:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:04:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:04:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:08:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:08:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:11:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:11:10 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:14:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:14:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:17:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:17:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:20:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:20:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:22:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:22:50 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:25:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:25:40 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:28:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:28:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:31:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:31:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:34:44 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:34:44 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:37:58 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:41:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:41:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:43:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:43:55 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:47:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:47:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:50:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:50:19 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:53:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:53:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:55:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:55:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:58:39 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 19:58:39 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:01:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:01:18 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:04:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:04:22 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:07:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:07:16 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:09:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:09:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:12:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:12:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:15:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:15:25 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:18:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:18:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:21:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:21:52 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:25:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:25:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:27:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:27:54 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:30:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:30:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:33:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:33:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:36:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:36:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:39:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:39:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:42:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:42:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:45:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:45:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:48:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:48:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:51:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:51:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:53:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:53:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:56:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:56:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:59:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 20:59:23 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:02:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:02:17 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:05:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:05:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:07:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:07:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:10:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:10:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:14:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:14:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:17:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:17:28 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:20:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:20:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:23:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:23:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:26:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:26:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:28:57 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:32:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:32:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:35:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:35:04 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:37:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:37:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:40:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:40:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:44:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:44:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:47:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:47:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:50:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:50:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:53:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:53:21 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:56:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:56:32 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:59:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 21:59:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:02:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:02:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:05:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:05:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:08:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:08:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:11:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:11:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:14:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:14:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:16:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:16:43 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:19:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:19:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:22:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:22:30 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:25:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:25:31 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:28:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:28:12 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:31:00 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:31:00 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:34:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:34:07 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:36:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:36:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:39:38 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:39:38 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:42:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:42:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:45:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:45:26 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:48:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:48:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:51:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:51:13 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:54:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:54:02 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:57:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:57:08 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:59:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 22:59:46 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:03:06 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:03:06 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:05:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:05:48 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:09:00 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:09:00 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:11:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:11:41 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:14:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:14:27 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:17:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:17:15 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:20:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:20:11 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:22:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:22:42 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:25:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:25:37 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:28:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:28:09 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:31:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:31:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:34:35 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:34:35 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:37:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:37:20 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:39:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:39:56 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:42:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:42:45 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:45:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:45:24 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:48:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:48:33 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:51:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:51:29 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:54:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:54:03 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:57:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Jun 12 23:57:05 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured With boot at: un 12 06:57:28 thebighonker syslogd: kernel boot file is /boot/kernel/kernel Jun 12 06:57:28 thebighonker kernel: Copyright (c) 1992-2014 The FreeBSD Project. Jun 12 06:57:28 thebighonker kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Jun 12 06:57:28 thebighonker kernel: The Regents of the University of California. All rights reserved. Jun 12 06:57:28 thebighonker kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Jun 12 06:57:28 thebighonker kernel: FreeBSD 10.0-STABLE #32 r267121M: Thu Jun 5 13:27:02 CDT 2014 Jun 12 06:57:28 thebighonker kernel: root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 Jun 12 06:57:28 thebighonker kernel: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Jun 12 06:57:28 thebighonker kernel: CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2500.14-MHz K8-class CPU) Jun 12 06:57:28 thebighonker kernel: Origin = "GenuineIntel" Id = 0x10676 Family = 0x6 Model = 0x17 Stepping = 6 Jun 12 06:57:28 thebighonker kernel: Features=0xbfebfbff Jun 12 06:57:28 thebighonker kernel: Features2=0xce3bd Jun 12 06:57:28 thebighonker kernel: AMD Features=0x20100800 Jun 12 06:57:28 thebighonker kernel: AMD Features2=0x1 Jun 12 06:57:28 thebighonker kernel: TSC: P-state invariant, performance statistics Jun 12 06:57:28 thebighonker kernel: real memory = 17179869184 (16384 MB) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 19:04:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAC8BE79 for ; Thu, 19 Jun 2014 19:04:51 +0000 (UTC) Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9B1C2045 for ; Thu, 19 Jun 2014 19:04:51 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id v10so2105776pde.20 for ; Thu, 19 Jun 2014 12:04:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:message-id:date:from:user-agent :mime-version:to:subject:references:in-reply-to:content-type; bh=1VwGTqa4kLB5h7xHLYD9eo4ZJNB4wrYUYhVKstMholM=; b=ibb8PM+haCqg7GdSGlooc/HrszKez2VPMKYWjXavEemd7FxlpNZ3CrHJoFh2bw9gWw 7KYdqTUEiUAavc6n0olMbwViXvtRGmEGdqqvKwOSODCuy0Fyup/xBVpTJ1H2XW7BMrRl MynyoS7udz5CuUtOlJYojrQUbcmSqLCyrGJNEpjYnH5NM+8NDC7sqvjJjCVSFUhenSsi myNLrKLtxUYR2SfHUeduZ7LrWScKJ5FttXT4xcZKw8KysoG9TaZgr63UuaGJG4xCqHbu OJc0miv46ojnF5W0Uq9iCeGumQ6pBmcfuZUJAooiF+H+8Y5awSjNsZPF4f6symMynyKZ pQuw== X-Gm-Message-State: ALoCoQkIOaM3hIyZ5m9/AcrrYE+r6YrCt2H14d8r2rOJeO9RSq1MrdB41rS6kQ5v3Zq8XLxl/K0n X-Received: by 10.69.25.105 with SMTP id ip9mr7929174pbd.145.1403204685607; Thu, 19 Jun 2014 12:04:45 -0700 (PDT) Received: from zont-osx.local (c-69-181-251-166.hsd1.ca.comcast.net. [69.181.251.166]) by mx.google.com with ESMTPSA id og3sm9809858pbc.48.2014.06.19.12.04.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 19 Jun 2014 12:04:44 -0700 (PDT) Sender: Andrey Zonov Message-ID: <53A33448.9060909@FreeBSD.org> Date: Thu, 19 Jun 2014 12:04:40 -0700 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: =?gb18030?Q?=D5=C5=EA=CF?= , freebsd-net Subject: Re: pow function in kernel space References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3wgIVNaK9MV0pm5avAlQTXbwjp8uPhndb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 19:04:52 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3wgIVNaK9MV0pm5avAlQTXbwjp8uPhndb Content-Type: text/plain; charset=gb18030 Content-Transfer-Encoding: quoted-printable There is no floating point types in kernel, so there is no pow() in kerne= l. On 6/19/14, 5:40 AM, =D5=C5=EA=CF wrote: > how can I implement, in an efficient, way the pow() function in kernel = space ? Is there any function I can use o r how I can evaluate pow funct= ion in kernel model? > =20 > Thanks! > =20 > =20 > ------------------ > =20 > =D7=A3=BA=C3=A3=AC > =20 > =D5=C5=EA=CF > =20 > =20 > =20 > ----------=20 > =20 > HanZhang > =20 --=20 Andrey Zonov --3wgIVNaK9MV0pm5avAlQTXbwjp8uPhndb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTozRLAAoJEBWLemxX/CvTnokIAJMYsr/hij5/bqTNT+KYsAZQ JlrRIgnkZnOr+esLZFF/9B98tvlftrUUFHUAF+wzWvgUHTOLNvMnNtJD3hG44XhR aQe6lETIk8wIuQFJmq4rtp2MHe3Y7WBBqHZS8mKlhirJstlNdtLjoLhPuMenNftr JAA9QBEKpf6Sjg9rulWcFHr/TXhFvDlsrKdo73vWzL+ht3sRkFg7g7nWVCUUX211 ETOVQr++CosrNFtO1P+9POvn+fc+xN0mZ40hfZd0IISiyof7sdu7x/eL9s5nepBL OuKw7rwy1DoenX2sCFQTVEBl9L2gNbqZKWVfRm6yx7yrWQjcVL9/1bKbcEOXp1Q= =Xs8Z -----END PGP SIGNATURE----- --3wgIVNaK9MV0pm5avAlQTXbwjp8uPhndb-- From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 20:07:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73EBDF97 for ; Thu, 19 Jun 2014 20:07:05 +0000 (UTC) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "vps.rulingia.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D134025DE for ; Thu, 19 Jun 2014 20:07:04 +0000 (UTC) Received: from server.rulingia.com (c220-239-242-83.belrs5.nsw.optusnet.com.au [220.239.242.83]) by vps.rulingia.com (8.14.7/8.14.7) with ESMTP id s5JK6MOT035340 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 20 Jun 2014 06:06:28 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.9/8.14.9) with ESMTP id s5JK6QFi003822 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Jun 2014 06:06:27 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.9/8.14.9/Submit) id s5JK6PKU003821; Fri, 20 Jun 2014 06:06:25 +1000 (EST) (envelope-from peter) Date: Fri, 20 Jun 2014 06:06:25 +1000 From: Peter Jeremy To: =?utf-8?B?5byg5pmX?= Subject: Re: pow function in kernel space Message-ID: <20140619200625.GB3631@server.rulingia.com> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="T4sUOijqQbZv57TR" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 20:07:05 -0000 --T4sUOijqQbZv57TR Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2014-Jun-19 20:40:48 +0800, =E5=BC=A0=E6=99=97 = wrote: >how can I implement, in an efficient, way the pow() function in kernel spa= ce ? Is there any function I can use o r how I can evaluate pow function i= n kernel model? Since the kernel only offers integer arithmetic, one approach would be square and multiply. What are you trying to do? Maybe we can offer an alternative to pow(3). --=20 Peter Jeremy --T4sUOijqQbZv57TR Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQJ8BAEBCgBmBQJTo0LBXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs09oIP/1QbHPJ7urM3D6SkHbP74mBb 6Pg6vDGcnjy7QbFxFe11+qjfkIKdd3N2M5VbF+5QqI2nA1hCcGEsd6E83SkhE3v9 SqrgS+sq/JZ7/pgeYPEVqtJWucpDkYmkItNV3krZlHVGF6GK7j4yEsCzm0dMOo65 jJuFUv2pKJAzGRcKNPCWUYMQ8F80EjkGVzo4fm+8MsBG8c0Xy/SaRDCJtonBTIHj jEAcGyh8TqWgs8dX1tsQzLGT/tuzkF0eqO5p5RZfW/CQwGSjlz/bw/12i+mBRlXm rzVDSvFSI0dBczJcRoviDzbPU9UXDprFev2pfOb5TL9GTinkfJqL+IjaTwp2Fcu5 PkmlN2Cypa4jl1igXeiVNqjuuRBTmyJ2jpywn8UtO/HD1AX6dUJ8RuOxXB1IXpau HwPXcdT1Iq2u5QW1U3Q4yWl+VPm0yTYDIMZL7Bfji+xyRvLIcznbNEIDOzncJg5g SToMTQGrOu7paAy6s7cDfZ/T/HKeoP4fNbldBPhG+yBzWKIjt+cnmebuSnS1mi66 0FTHXfqhQdQ6XHr7Jn1wqnCUThsMBHhzHCruuNsyJVMo5MQw13Tq97TbHSXTsrtW 8SJqxQNnZMhSgMS57U9Y/PViK2/O/C9IaIWuDancpm/0hPVPyffhlMNuIl7hr+pu wmdAsp6U2XlvVSrn2mJ4 =4POu -----END PGP SIGNATURE----- --T4sUOijqQbZv57TR-- From owner-freebsd-net@FreeBSD.ORG Thu Jun 19 21:35:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96D3C44C for ; Thu, 19 Jun 2014 21:35:18 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67BB52D57 for ; Thu, 19 Jun 2014 21:35:18 +0000 (UTC) Received: from 24-104-73-2-ip-static.hfc.comcastbusiness.net ([24.104.73.2]:21563 helo=[10.10.11.7]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1WxjzH-0003bG-6m; Thu, 19 Jun 2014 17:35:15 -0400 From: "George Neville-Neil" To: "Eggert, Lars" Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 Date: Thu, 19 Jun 2014 14:35:13 -0700 Message-ID: In-Reply-To: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> References: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=_MailMate_22E6F6CB-E9F5-4CCA-A2C4-34006FF62074_="; micalg=pgp-sha1; protocol="application/pgp-signature" X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "freebsd-net@freebsd.org" , "varis81@hotmail.com" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 21:35:18 -0000 This is an OpenPGP/MIME signed message (RFC 3156 and 4880). --=_MailMate_22E6F6CB-E9F5-4CCA-A2C4-34006FF62074_= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On 4 Feb 2014, at 1:38, Eggert, Lars wrote: > Hi, > > below are two patches that implement RFC6937 ("Proportional Rate Reduct= ion for TCP") and draft-ietf-tcpm-newcwv-00 ("Updating TCP to support Rat= e-Limited Traffic"). They were done by Aris Angelogiannopoulos for his MS= thesis, which is at https://eggert.org/students/angelogiannopoulos-thesi= s.pdf. > > The patches should apply to -CURRENT as of Sep 17, 2013. (Sorry for the= delay in sending them, we'd been trying to get some feedback from commit= ters first, without luck.) > > Please note that newcwv is still a work in progress in the IETF, and th= e patch has some limitations with regards to the "pipeACK Sampling Period= " mentioned in the Internet-Draft. Aris says this in his thesis about wha= t exactly he implemented: > > "The second implementation choice, is in regards with the measurement o= f pipeACK. This variable is the most important introduced by the method a= nd is used to compute the phase that the sender currently lies in. In ord= er to compute pipeACK the approach suggested by the Internet Draft (ID) i= s followed [ncwv]. During initialization, pipeACK is set to the maximum p= ossible value. A helper variable prevHighACK is introduced that is initia= lized to the initial sequence number (iss). prevHighACK holds the value o= f the highest acknowledged byte so far. pipeACK is measured once per RTT = meaning that when an ACK covering prevHighACK is received, pipeACK become= s the difference between the current ACK and prevHighACK. This is called = a pipeACK sample. A newer version of the draft suggests that multiple pi= peACK samples can be used during the pipeACK sampling period." > > Lars > > > [prr.patch] > > [newcwv.patch] Apologies for not looking at this as yet. It is now closer to the top of= my list. Best, George --=_MailMate_22E6F6CB-E9F5-4CCA-A2C4-34006FF62074_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlOjV5EACgkQYdh2wUQKM9I7JQCgnd+ECKQvkv4N8VxyAvpitcR8 XygAn1FRxWHSImmlhPcgLoYXbT7u4cag =agEK -----END PGP SIGNATURE----- --=_MailMate_22E6F6CB-E9F5-4CCA-A2C4-34006FF62074_=-- From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 01:00:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 694B3451 for ; Fri, 20 Jun 2014 01:00:00 +0000 (UTC) Received: from smtpbgau1.qq.com (smtpbgau1.qq.com [54.206.16.166]) by mx1.freebsd.org (Postfix) with SMTP id 9CEC02CF4 for ; Fri, 20 Jun 2014 00:59:59 +0000 (UTC) X-QQ-FEAT: pXQxZIU7IhB/WoBZ2XJf9ENVGb9fbtMen2Zg2rAwP+IkVLXjnU3pvzOVL0MJi McagJhYmDOsoHEZxbiwy8vsxizOLkEeJrEO1cER73r/ConIFo85oOCvWszJ91zhYkJStUYj 8m80vFwPnTs3NMrh4iqOzB4FSawx+wOlmQ7ZN7y5PWql6/9JIA== X-QQ-SSF: 000000000000000000000000000000Z X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 166.111.3.31 In-Reply-To: <20140619200625.GB3631@server.rulingia.com> References: <20140619200625.GB3631@server.rulingia.com> X-QQ-STYLE: X-QQ-mid: webmail196t1403225989t7810587 From: "=?gb18030?B?1cXqzw==?=" To: "=?gb18030?B?UGV0ZXIgSmVyZW15?=" Subject: =?gb18030?B?u9i4tKO6IHBvdyBmdW5jdGlvbiBpbiBrZXJuZWwg?= =?gb18030?B?c3BhY2U=?= Mime-Version: 1.0 Date: Fri, 20 Jun 2014 08:59:49 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-ReplyHash: 1572003806 X-QQ-SENDSIZE: 520 X-QQ-Bgrelay: 1 X-Mailman-Approved-At: Fri, 20 Jun 2014 01:13:48 +0000 Content-Type: text/plain; charset="gb18030" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: =?gb18030?B?ZnJlZWJzZC1uZXQ=?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:00:00 -0000 SGk6DQogIA0KIEkgYW0gZG9pbmcgYSByZXNlYXJjaCBvbiBEMlRDUChodHRwOi8vZGwuYWNt Lm9yZy9jaXRhdGlvbi5jZm0/aWQ9MjM0MjM4OCksIEkganVzdCB3YW50IHRvIGltcGxlbWVu dCBpdCBpbnRvIHRoZSBsaW51eCBrZXJuZWwuIFdoZW4gY2FsY3VsYXRpbmcgdGhlIHBlbmFs dHkgZnVuY3Rpb24sIGl0IGlzIHAgPSBhXmQsIHdoZXJlIDA8IGEgPCAxIGFuZCAgMDwgZCA8 IDEuIFNpbmNlIHRoZSBrZXJuZWwgb25seSBvZmZlcnMgaW50ZWdlciwgICBzbyBpbiBteSBj b2RlLCBzbyBJIGxldCBhIG11bHRpcGx5IDJeMTAuIEJ1dCBJIGhhdmUgbm8gaWRlYSBvZiBj YWxjdWxhdGluZyBhXmQgd2hlbiAwPCBkIDwgMS4gTWF5IGJlIEkgd2FudCBhICBhcHByb3hp bWF0ZSBhbGdvcml0aG0gb3Igb3RoZXIgbWV0aG9kcy4gQ2FuIHlvdSBoZWxwIG1lID8NCiB0 aGFua3N+DQogIA0KDQogIA0KICAtLS0tLS0tLS0tLS0tLS0tLS0NCiAgDQrXo7rDo6wNCiAN CtXF6s8NCiANCiANCiANCi0tLS0tLS0tLS0gDQogDQpIYW5aaGFuZw0KIA0KU2Nob29sIG9m IENvbXB1dGVyIFNjaWVuY2UNClRzaW5naHVhIFVuaXZlcnNpdHksIEJlaWppbmcsIDEwMDA4 NCwgUC5SLkNISU5BDQogDQpNb2JpbGU6ICArODYgMTU2LTUyNi01OTc4Mg0KRS1tYWlsOiAg IHpoYW5naGFuQGNzbmV0MS5jcy50c2luZ2h1YS5lZHUuY24NCg0KICANCiAgDQoNCiANCg0K IC0tLS0tLS0tLS0tLS0tLS0tLSDUrcq808q8/iAtLS0tLS0tLS0tLS0tLS0tLS0NCiAgt6K8 /sjLOiAiUGV0ZXIgSmVyZW15Ijs8cGV0ZXJAcnVsaW5naWEuY29tPjsNCiC3osvNyrG85Dog MjAxNMTqNtTCMjDI1SjQx8bazuUpIMHos780OjA2DQogytW8/sjLOiAi1cXqzyI8emdhbmdo YW5oYW5AZm94bWFpbC5jb20+OyANCiCzrcvNOiAiZnJlZWJzZC1uZXQiPGZyZWVic2QtbmV0 QGZyZWVic2Qub3JnPjsgDQog1vfM4jogUmU6IHBvdyBmdW5jdGlvbiBpbiBrZXJuZWwgc3Bh Y2UNCg0KIA0KDQpPbiAyMDE0LUp1bi0xOSAyMDo0MDo0OCArMDgwMCwg1cXqzyA8emdhbmdo YW5oYW5AZm94bWFpbC5jb20+IHdyb3RlOg0KPmhvdyBjYW4gSSBpbXBsZW1lbnQsIGluIGFu IGVmZmljaWVudCwgd2F5IHRoZSBwb3coKSBmdW5jdGlvbiBpbiBrZXJuZWwgc3BhY2UgPyAg SXMgdGhlcmUgYW55IGZ1bmN0aW9uIEkgY2FuIHVzZSBvIHIgaG93IEkgY2FuIGV2YWx1YXRl IHBvdyBmdW5jdGlvbiBpbiBrZXJuZWwgbW9kZWw/DQoNClNpbmNlIHRoZSBrZXJuZWwgb25s eSBvZmZlcnMgaW50ZWdlciBhcml0aG1ldGljLCBvbmUgYXBwcm9hY2ggd291bGQgYmUNCnNx dWFyZSBhbmQgbXVsdGlwbHkuICBXaGF0IGFyZSB5b3UgdHJ5aW5nIHRvIGRvPyAgTWF5YmUg d2UgY2FuIG9mZmVyDQphbiBhbHRlcm5hdGl2ZSB0byBwb3coMykuDQoNCi0tIA0KUGV0ZXIg SmVyZW15 From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 01:00:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F310E466 for ; Fri, 20 Jun 2014 01:00:56 +0000 (UTC) Received: from smtpproxy19.qq.com (smtpproxy19.qq.com [184.105.206.84]) by mx1.freebsd.org (Postfix) with SMTP id 8CB3E2CFE for ; Fri, 20 Jun 2014 01:00:56 +0000 (UTC) X-QQ-FEAT: hl2cjxplFwu9lsDLel29IFOLgjLtA4n3t9U7xIfIwY36XSH/AXxu1nbQOBvl0 Vdi41om2hPCpH7D9wbJqfYp8fYvyVrVM90nYyyqDrZmfei2nk6pa1W1O6m9ZQVxsK+3e9+U 3jkEvhcBv2bjQI6ofAeQp0Ncscu3fG/RXET5ScRh/rtgFg204Q== X-QQ-SSF: 000000000000000000000000000000Z X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 166.111.3.31 In-Reply-To: <53A33448.9060909@FreeBSD.org> References: <53A33448.9060909@FreeBSD.org> X-QQ-STYLE: X-QQ-mid: webmail196t1403226019t1301496 From: "=?gb18030?B?1cXqzw==?=" To: "=?gb18030?B?QW5kcmV5IFpvbm92?=" Subject: =?gb18030?B?u9i4tKO6IHBvdyBmdW5jdGlvbiBpbiBrZXJuZWwg?= =?gb18030?B?c3BhY2U=?= Mime-Version: 1.0 Date: Fri, 20 Jun 2014 09:00:18 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-ReplyHash: 669706092 X-QQ-SENDSIZE: 520 X-QQ-Bgrelay: 1 X-Mailman-Approved-At: Fri, 20 Jun 2014 01:18:59 +0000 Content-Type: text/plain; charset="gb18030" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: =?gb18030?B?ZnJlZWJzZC1uZXQ=?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:00:57 -0000 SGk6DQogIA0KIEkgYW0gZG9pbmcgYSByZXNlYXJjaCBvbiBEMlRDUChodHRwOi8vZGwuYWNt Lm9yZy9jaXRhdGlvbi5jZm0/aWQ9MjM0MjM4OCksIEkganVzdCB3YW50IHRvIGltcGxlbWVu dCBpdCBpbnRvIHRoZSBsaW51eCBrZXJuZWwuIFdoZW4gY2FsY3VsYXRpbmcgdGhlIHBlbmFs dHkgZnVuY3Rpb24sIGl0IGlzIHAgPSBhXmQsIHdoZXJlIDA8IGEgPCAxIGFuZCAgMDwgZCA8 IDEuIFNpbmNlIHRoZSBrZXJuZWwgb25seSBvZmZlcnMgaW50ZWdlciwgICBzbyBpbiBteSBj b2RlLCBzbyBJIGxldCBhIG11bHRpcGx5IDJeMTAuIEJ1dCBJIGhhdmUgbm8gaWRlYSBvZiBj YWxjdWxhdGluZyBhXmQgd2hlbiAwPCBkIDwgMS4gTWF5IGJlIEkgd2FudCBhICBhcHByb3hp bWF0ZSBhbGdvcml0aG0gb3Igb3RoZXIgbWV0aG9kcy4gQ2FuIHlvdSBoZWxwIG1lID8NCiB0 aGFua3N+DQogIA0KICANCiAgDQogIC0tLS0tLS0tLS0tLS0tLS0tLQ0KICANCtejusOjrA0K IA0K1cXqzw0KIA0KIA0KIA0KLS0tLS0tLS0tLSANCiANCkhhblpoYW5nDQogDQpTY2hvb2wg b2YgQ29tcHV0ZXIgU2NpZW5jZQ0KVHNpbmdodWEgVW5pdmVyc2l0eSwgQmVpamluZywgMTAw MDg0LCBQLlIuQ0hJTkENCiANCk1vYmlsZTogICs4NiAxNTYtNTI2LTU5NzgyDQpFLW1haWw6 ICAgemhhbmdoYW5AY3NuZXQxLmNzLnRzaW5naHVhLmVkdS5jbg0KDQogIA0KICANCg0KIA0K DQogLS0tLS0tLS0tLS0tLS0tLS0tINStyrzTyrz+IC0tLS0tLS0tLS0tLS0tLS0tLQ0KICC3 orz+yMs6ICJBbmRyZXkgWm9ub3YiOzx6b250QEZyZWVCU0Qub3JnPjsNCiC3osvNyrG85Dog MjAxNMTqNtTCMjDI1SjQx8bazuUpIMHos78zOjA0DQogytW8/sjLOiAi1cXqzyI8emdhbmdo YW5oYW5AZm94bWFpbC5jb20+OyAiZnJlZWJzZC1uZXQiPGZyZWVic2QtbmV0QGZyZWVic2Qu b3JnPjsgDQogDQog1vfM4jogUmU6IHBvdyBmdW5jdGlvbiBpbiBrZXJuZWwgc3BhY2UNCg0K IA0KDQpUaGVyZSBpcyBubyBmbG9hdGluZyBwb2ludCB0eXBlcyBpbiBrZXJuZWwsIHNvIHRo ZXJlIGlzIG5vIHBvdygpIGluIGtlcm5lbC4NCg0KT24gNi8xOS8xNCwgNTo0MCBBTSwg1cXq zyB3cm90ZToNCj4gaG93IGNhbiBJIGltcGxlbWVudCwgaW4gYW4gZWZmaWNpZW50LCB3YXkg dGhlIHBvdygpIGZ1bmN0aW9uIGluIGtlcm5lbCBzcGFjZSA/ICBJcyB0aGVyZSBhbnkgZnVu Y3Rpb24gSSBjYW4gdXNlIG8gciBob3cgSSBjYW4gZXZhbHVhdGUgcG93IGZ1bmN0aW9uIGlu IGtlcm5lbCBtb2RlbD8NCj4gICANCj4gIFRoYW5rcyENCj4gIA0KPiAgDQo+ICAgLS0tLS0t LS0tLS0tLS0tLS0tDQo+ICAgDQo+INejusOjrA0KPiAgDQo+INXF6s8NCj4gIA0KPiAgDQo+ ICANCj4gLS0tLS0tLS0tLSANCj4gIA0KPiBIYW5aaGFuZw0KPiAgDQoNCg0KLS0gDQpBbmRy ZXkgWm9ub3Y= From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 01:25:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96B6CB73; Fri, 20 Jun 2014 01:25:49 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 236482F37; Fri, 20 Jun 2014 01:25:49 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5K1Pcex061881 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 19 Jun 2014 18:25:39 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5K1Pc0G061880; Thu, 19 Jun 2014 18:25:38 -0700 (PDT) (envelope-from jmg) Date: Thu, 19 Jun 2014 18:25:38 -0700 From: John-Mark Gurney To: ???? Subject: Re: ?????? pow function in kernel space Message-ID: <20140620012538.GQ31367@funkthat.com> Mail-Followup-To: ???? , Andrey Zonov , freebsd-net References: <53A33448.9060909@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 19 Jun 2014 18:25:39 -0700 (PDT) Cc: freebsd-net , Andrey Zonov X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:25:49 -0000 ???? wrote this message on Fri, Jun 20, 2014 at 09:00 +0800: > I am doing a research on D2TCP(http://dl.acm.org/citation.cfm?id=2342388), I just want to implement it into the linux kernel. When calculating the penalty function, it is p = a^d, where 0< a < 1 and 0< d < 1. Since the kernel only offers integer, so in my code, so I let a multiply 2^10. But I have no idea of calculating a^d when 0< d < 1. May be I want a approximate algorithm or other methods. Can you help me ? This is not a linux kernel list, please ask on a proper list... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 01:29:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1D61CCC for ; Fri, 20 Jun 2014 01:29:00 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 55D8E2F65 for ; Fri, 20 Jun 2014 01:29:00 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id x13so2984079wgg.3 for ; Thu, 19 Jun 2014 18:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=GzFo7RoHeXv4okTtkyQzy2SJIafz45mEpx0vPXT9tqs=; b=t45xddb0IrJ6Skg1LD6l8RPqQ49Ei65xMKnCPJ2Hlo1jPZhWMeEkip07mEvJ+cZY3H 13acKImDFYBOK/rxuh7MvNfGvA2s8+vCcpolwwtB7jN6Vpnoo6x6M76mJeS8T8/QP7Ku 9ancdEh201P4iMPGuu48LjeOkUX8K2M8SHm+X8OqvFJJBU4LH0IQvRu77hzB9Cm4vBch Jyz1kft+ba1iBg7LBHRzSTw80461HHcKnmVJqMT532u9X37xPlrV4/XH6+2gOLlLgGwM S99sMpGron4EdWJ9XH5IF+Te8HVDxq+/V3h5rPlPKlGTLzzoa/tBv5eW9dNaf03pACUp KPuw== X-Received: by 10.180.72.15 with SMTP id z15mr211159wiu.46.1403227738725; Thu, 19 Jun 2014 18:28:58 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id r9sm38143280wia.17.2014.06.19.18.28.57 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 19 Jun 2014 18:28:58 -0700 (PDT) Date: Fri, 20 Jun 2014 03:28:55 +0200 From: Mateusz Guzik To: =?utf-8?B?5byg5pmX?= Subject: Re: =?utf-8?B?5Zue5aSN?= =?utf-8?B?77ya?= pow function in kernel space Message-ID: <20140620012855.GB5830@dft-labs.eu> References: <20140619200625.GB3631@server.rulingia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net , Peter Jeremy X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:29:00 -0000 On Fri, Jun 20, 2014 at 08:59:49AM +0800, åž æ™— wrote: > Hi: > > I am doing a research on D2TCP(http://dl.acm.org/citation.cfm?id=2342388), I just want to implement it into the linux kernel. When calculating the penalty function, it is p = a^d, where 0< a < 1 and 0< d < 1. Since the kernel only offers integer, so in my code, so I let a multiply 2^10. But I have no idea of calculating a^d when 0< d < 1. May be I want a approximate algorithm or other methods. Can you help me ? > thanks~ > First of all this is a FreeBSD list and FreeBSD is not Linux. It is possible to use FPU in kernel mode, but it requires some additional work. Both FreeBSD and Linux kernels provide functions for this purpose, see kernel_fpu_begin and kernel_fpu_end on Linux. I suggest you send further questions to some Linux-related resource, although I can't recommend any in particular. -- Mateusz Guzik From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 02:19:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AA3D798 for ; Fri, 20 Jun 2014 02:19:20 +0000 (UTC) Received: from gpo3.cc.swin.edu.au (gpo3.cc.swin.edu.au [136.186.1.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DACBB24E9 for ; Fri, 20 Jun 2014 02:19:18 +0000 (UTC) Received: from [136.186.229.37] (garmitage.caia.swin.edu.au [136.186.229.37]) by gpo3.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s5K2JErm019560 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 20 Jun 2014 12:19:15 +1000 Message-ID: <53A39A22.5090802@swin.edu.au> Date: Fri, 20 Jun 2014 12:19:14 +1000 From: grenville armitage User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121107 Thunderbird/16.0.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: LEDBAT (RFC-6817)i n FreeBSD as mod_cc(9)? References: <1863738251.20140616172546@serebryakov.spb.ru> <1819611615.20140617124843@serebryakov.spb.ru> In-Reply-To: <1819611615.20140617124843@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 02:19:20 -0000 On 06/17/2014 18:48, Lev Serebryakov wrote: > Hello, hiren. > You wrote 17 =D0=B8=D1=8E=D0=BD=D1=8F 2014 =D0=B3., 11:41:35: > >>> It looks like, that some TCP connections could benefit from LEDBAT= >>> (RFC-6871) cognestion control algorithm (not all, of course, it sho= uld not >>> be default). I wonder what you think of using cc_cdg ? We did some initial exploratio= n of using cc_cdg (v0.1) in LEDBAT-like contexts in http://caia.swin.edu.= au/cv/garmitage/things/CameraReady-CDGscaveneger-LCN2013-preprint.pdf) [..] > The problem is it seems that "someone" needs to extend set of hooks = in > mod_cc substantially. It is more than "write one more module" and, IM= HO, > should be discussed with net-related people :) I'm also curious about implementing LEDBAT -- what sort of functionality = do you sense is missing from current mod_cc and h_ertt ? cheers, gja From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 03:56:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29DB01EB for ; Fri, 20 Jun 2014 03:56:15 +0000 (UTC) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E127A2DA6 for ; Fri, 20 Jun 2014 03:56:13 +0000 (UTC) Received: from alph.d.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=56) by mail.allbsd.org (8.14.8/8.14.8) with ESMTP id s5K3tpvV015364 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Jun 2014 12:56:02 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.8/8.14.8) with ESMTP id s5K3tlDH074362; Fri, 20 Jun 2014 12:55:51 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Fri, 20 Jun 2014 12:48:03 +0900 (JST) Message-Id: <20140620.124803.562992300308664968.hrs@allbsd.org> To: jhay@meraka.org.za Subject: Re: network.subr vlan handling broken From: Hiroki Sato In-Reply-To: <20140619103513.GA92393@zibbi.meraka.csir.co.za> References: <20140619103513.GA92393@zibbi.meraka.csir.co.za> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart0(Fri_Jun_20_12_48_03_2014_705)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Fri, 20 Jun 2014 12:56:05 +0900 (JST) X-Spam-Status: No, score=-97.7 required=13.0 tests=CONTENT_TYPE_PRESENT, QENCPTR2,RDNS_NONE,SPF_SOFTFAIL,USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 03:56:15 -0000 ----Security_Multipart0(Fri_Jun_20_12_48_03_2014_705)-- Content-Type: Multipart/Mixed; boundary="--Next_Part(Fri_Jun_20_12_48_03_2014_919)--" Content-Transfer-Encoding: 7bit ----Next_Part(Fri_Jun_20_12_48_03_2014_919)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit John Hay wrote in <20140619103513.GA92393@zibbi.meraka.csir.co.za>: jh> Hi Guys, jh> jh> freebsd-rc did not react, so I'm just checking on -net too. jh> jh> I found after upgrading that vlan handling broke. I tried the following: jh> jh> vlans_bce1="6" jh> ipv4_addrs_bce1_6="inet 10.239.100.2/24" jh> ifconfig_bce1_6_aliases="inet 10.239.100.2/24" jh> ifconfig_bce1_6_alias0="inet 10.239.100.2/24" jh> jh> I traced it down to ifalias_af_common_handler being called with the jh> mangled interfcace name _if and it then calls ifconfig with it. Here jh> is my fix. Any reason not to commit it? My diff is against 10-stable, jh> but head looks the same. Can you try the attached patch? It seemed broken after list_vars() was introduced. Replacing $_if with $1 also fixes it, but $_if should be used for ifname as the other parts do. jh> While looking through the code I saw that ltr is called with different jh> styling. Is there a reason for it? Which is the prefered style? jh> jh> ltr ${_if} "${_punct}" '_' _if jh> ltr "$_if" "$_punct" "_" _if I do not think there is a reason. I think there is no consensus about the style but I am using {} only when boundary between the variable name and the subsequent characters can be ambiguous. -- Hiroki ----Next_Part(Fri_Jun_20_12_48_03_2014_919)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="network.subr-20140620-1.diff" Index: network.subr =================================================================== --- network.subr (revision 267636) +++ network.subr (working copy) @@ -1077,7 +1077,7 @@ ifalias_af_common() { local _ret _if _af _action alias ifconfig_args _aliasn _c _tmpargs _iaf - local _punct=".-/+" + local _vif _punct=".-/+" _ret=1 _aliasn= @@ -1086,11 +1086,11 @@ _action=$3 # Normalize $_if before using it in a pattern to list_vars() - ltr "$_if" "$_punct" "_" _if + ltr "$_if" "$_punct" "_" _vif # ifconfig_IF_aliasN which starts with $_af - for alias in `list_vars ifconfig_${_if}_alias[0-9]\* | - sort_lite -nk1.$((9+${#_if}+7))` + for alias in `list_vars ifconfig_${_vif}_alias[0-9]\* | + sort_lite -nk1.$((9+${#_vif}+7))` do eval ifconfig_args=\"\$$alias\" _iaf= @@ -1118,8 +1118,8 @@ # backward compatibility: ipv6_ifconfig_IF_aliasN. case $_af in inet6) - for alias in `list_vars ipv6_ifconfig_${_if}_alias[0-9]\* | - sort_lite -nk1.$((14+${#_if}+7))` + for alias in `list_vars ipv6_ifconfig_${_vif}_alias[0-9]\* | + sort_lite -nk1.$((14+${#_vif}+7))` do eval ifconfig_args=\"\$$alias\" case ${_action}:"${ifconfig_args}" in @@ -1129,7 +1129,7 @@ alias:*) _aliasn="${_aliasn} inet6 ${ifconfig_args}" warn "\$${alias} is obsolete. " \ - "Use ifconfig_$1_aliasN instead." + "Use ifconfig_${_vif}_aliasN instead." ;; esac done ----Next_Part(Fri_Jun_20_12_48_03_2014_919)---- ----Security_Multipart0(Fri_Jun_20_12_48_03_2014_705)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlOjrvMACgkQTyzT2CeTzy2fTQCfR95aWy54z905YblPX/X90QgX CPQAn1hs2/nSNJaAdC0Wp3meUPyADJM6 =ir9k -----END PGP SIGNATURE----- ----Security_Multipart0(Fri_Jun_20_12_48_03_2014_705)---- From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 04:09:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBD5867C for ; Fri, 20 Jun 2014 04:09:02 +0000 (UTC) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69DF52E95 for ; Fri, 20 Jun 2014 04:09:02 +0000 (UTC) Received: from alph.d.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=56) by mail.allbsd.org (8.14.8/8.14.8) with ESMTP id s5K48ijK016375 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Jun 2014 13:08:54 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.8/8.14.8) with ESMTP id s5K48hKl074487; Fri, 20 Jun 2014 13:08:44 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Fri, 20 Jun 2014 13:08:12 +0900 (JST) Message-Id: <20140620.130812.1612764660960146940.hrs@allbsd.org> To: ler@lerctr.org Subject: Re: IPv6: "xxx::x already configured" in logs... why? From: Hiroki Sato In-Reply-To: <20140619140801.GA65420@thebighonker.lerctr.org> References: <20140612202349.GA65079@thebighonker.lerctr.org> <20140619.181129.1279477227287764712.hrs@allbsd.org> <20140619140801.GA65420@thebighonker.lerctr.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Fri_Jun_20_13_08_12_2014_519)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Fri, 20 Jun 2014 13:08:56 +0900 (JST) X-Spam-Status: No, score=-97.9 required=13.0 tests=CONTENT_TYPE_PRESENT, RDNS_NONE,SPF_SOFTFAIL,USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 04:09:03 -0000 ----Security_Multipart(Fri_Jun_20_13_08_12_2014_519)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Larry Rosenman wrote in <20140619140801.GA65420@thebighonker.lerctr.org>: le> > le> Ideas? (I may be an idiot, so any criticism welcomed). le> > le> le> > le> if you need the 1841's config, I can supply that as well. It's using a Hurricane le> > le> electric Tunnel. le> > le> > How frequent were the log message added into /var/log/messages? And le> > when did it start to happen after boot. Just after lagg0 is le> > configured? le> > le> > -- Hiroki le> Looks like: le> le> Jun 12 07:00:01 thebighonker kernel: in6_ifadd: 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured Thank you. Three more questions: 1. output of "ifconfig lagg0", "ifconfig bce0", and "ifconfig bce1". 2. output of "netstat -s -i". 3. output of "ndp -p". The cause of the message is that the automatically-configured address is not recognized as "configured" one and FreeBSD IPv6 stack is trying to add it every time a Router Advertisement message is received. I am still not sure why it happened, but the above three would help for further investigation. -- Hiroki ----Security_Multipart(Fri_Jun_20_13_08_12_2014_519)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlOjs6wACgkQTyzT2CeTzy1PCgCgu0ik8r+qWFPcYnPimpgFCsk3 Y1oAnRn0prWi32YIc+uR3y2QLOy3sguJ =nnzI -----END PGP SIGNATURE----- ----Security_Multipart(Fri_Jun_20_13_08_12_2014_519)---- From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 08:14:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B7328F8 for ; Fri, 20 Jun 2014 08:14:05 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C601622EE for ; Fri, 20 Jun 2014 08:14:04 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id k14so3293194wgh.6 for ; Fri, 20 Jun 2014 01:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:hackerspace:user-agent; bh=M9mda/aiqMsvcw78p6+/u5Du+FuyRyeTa6Y/8nwsyQg=; b=vpkSj2bY7mPMRfJKiBEtDBVmlJGuvebhX4vABVhyEKzwT21UoTU61LQLMlpyTgDbTL 1dF9vN2cKT9TeFUEiB3tl0kYwVMHHQIcAhiriwPmkouoqslw8wIGWt4vgSh5OKIg9pUe oLiq5DhlDsEqOSasEPY1YnS07MtOgL4GM5t9W5I1erLTd+uCz8rrDpwi0Jse/f4vTBuF uNFlPG9vKx8ggWjZm+M0H0WCF2pZZ0VaUo1vS42mi5ho2KBMeWBLbkMM+CZQIQ1BF2Tf 34cQ0LDfjsoWiWanxqB13TuYEqZaYsjxMZHvsZEZWEmt4ChE78PZIKWMXcJzM5KteMYx 8+Dg== X-Received: by 10.180.126.8 with SMTP id mu8mr2293585wib.10.1403252042026; Fri, 20 Jun 2014 01:14:02 -0700 (PDT) Received: from gmail.com (host86-149-215-142.range86-149.btcentralplus.com. [86.149.215.142]) by mx.google.com with ESMTPSA id cy4sm2513085wib.5.2014.06.20.01.14.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Jun 2014 01:14:01 -0700 (PDT) Sender: Tom Jones Date: Fri, 20 Jun 2014 09:13:58 +0100 From: Tom Jones To: freebsd-net@freebsd.org Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 Message-ID: <20140620081356.GA54735@gmail.com> References: <259C9434-C6FE-42EA-823D-ECB024DBF3D7@netapp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hackerspace: 57North Hacklab User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 08:14:05 -0000 On Thu, Jun 19, 2014 at 02:35:13PM -0700, George Neville-Neil wrote: > On 4 Feb 2014, at 1:38, Eggert, Lars wrote: > > > Hi, > > > > below are two patches that implement RFC6937 ("Proportional Rate Reduction for TCP") and draft-ietf-tcpm-newcwv-00 ("Updating TCP to support Rate-Limited Traffic"). They were done by Aris Angelogiannopoulos for his MS thesis, which is at https://eggert.org/students/angelogiannopoulos-thesis.pdf. > > > > The patches should apply to -CURRENT as of Sep 17, 2013. (Sorry for the delay in sending them, we'd been trying to get some feedback from committers first, without luck.) > > > > Please note that newcwv is still a work in progress in the IETF, and the patch has some limitations with regards to the "pipeACK Sampling Period" mentioned in the Internet-Draft. Aris says this in his thesis about what exactly he implemented: > > > > "The second implementation choice, is in regards with the measurement of pipeACK. This variable is the most important introduced by the method and is used to compute the phase that the sender currently lies in. In order to compute pipeACK the approach suggested by the Internet Draft (ID) is followed [ncwv]. During initialization, pipeACK is set to the maximum possible value. A helper variable prevHighACK is introduced that is initialized to the initial sequence number (iss). prevHighACK holds the value of the highest acknowledged byte so far. pipeACK is measured once per RTT meaning that when an ACK covering prevHighACK is received, pipeACK becomes the difference between the current ACK and prevHighACK. This is called a pipeACK sample. A newer version of the draft suggests that multiple pipeACK samples can be used during the pipeACK sampling period." > > > > Lars > > > > > > [prr.patch] > > > > [newcwv.patch] > > Apologies for not looking at this as yet. It is now closer to the top of my list. > > Best, > George Hi George, I have a set of patches for draft-ietf-tcpm-newcwv-06 that I was going to bring to the list in the next few days. My implementation is a port of the Linux implementation by a colleague of mine at the University of Aberdeen. Feeback on how Aris hooked in his newcwv calls would be helpful. -- Tom @adventureloop adventurist.me #pragma summon cthulhu :wq From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 12:45:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A08E854; Fri, 20 Jun 2014 12:45:57 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27E4A2D9D; Fri, 20 Jun 2014 12:45:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=+ZyZUjPzLdvLGIIqB2V/4VykFXaShGgsmpys9rqpLO4=; b=DUxvFu6x2SvWEDy89jEiJxO/QuXXg2a7l50WOL+0lgAtEYkOvDKwbUIFwtD5ZkGyuKy6Nc0OxVlizuEuln89M9/5I0BdHAgPcasfENg53LtC1LNRTY1gVR524h2L+Op+vDjnb7lecnc2Ki+mV7tZdf59p7Jkpp2afOjhUpOgTLw=; Received: from localhost.lerctr.org ([127.0.0.1]:18657 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WxyCY-0007w9-8z; Fri, 20 Jun 2014 07:45:56 -0500 Received: from ool-4b7f879a.static.optonline.net ([75.127.135.154]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 20 Jun 2014 07:45:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 20 Jun 2014 07:45:54 -0500 From: Larry Rosenman To: Hiroki Sato Subject: Re: IPv6: "xxx::x already configured" in logs... =?UTF-8?Q?why=3F?= In-Reply-To: <20140620.130812.1612764660960146940.hrs@allbsd.org> References: <20140612202349.GA65079@thebighonker.lerctr.org> <20140619.181129.1279477227287764712.hrs@allbsd.org> <20140619140801.GA65420@thebighonker.lerctr.org> <20140620.130812.1612764660960146940.hrs@allbsd.org> Message-ID: <0c5bc901eba87f35e58390b686cee5ab@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 12:45:57 -0000 On 2014-06-19 23:08, Hiroki Sato wrote: > Larry Rosenman wrote > in <20140619140801.GA65420@thebighonker.lerctr.org>: > > le> > le> Ideas? (I may be an idiot, so any criticism welcomed). > le> > le> > le> > le> if you need the 1841's config, I can supply that as well. > It's using a Hurricane > le> > le> electric Tunnel. > le> > > le> > How frequent were the log message added into /var/log/messages? > And > le> > when did it start to happen after boot. Just after lagg0 is > le> > configured? > le> > > le> > -- Hiroki > le> Looks like: > le> > le> Jun 12 07:00:01 thebighonker kernel: in6_ifadd: > 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a is already configured > > Thank you. Three more questions: > > 1. output of "ifconfig lagg0", "ifconfig bce0", and "ifconfig bce1". > > 2. output of "netstat -s -i". > > 3. output of "ndp -p". > > The cause of the message is that the automatically-configured address > is not recognized as "configured" one and FreeBSD IPv6 stack is > trying to add it every time a Router Advertisement message is > received. I am still not sure why it happened, but the above three > would help for further investigation. > > -- Hiroki I updated the system from r267121M to r267653M and rebooted. The issue does NOT occur at this level. thebighonker.lerctr.org /etc/rc.d $ ifconfig lagg0 lagg0: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:23:7d:9e:6e:8a inet 192.147.25.65 netmask 0xffffff00 broadcast 192.147.25.255 inet6 fe80::223:7dff:fe9e:6e8a%lagg0 prefixlen 64 scopeid 0x4 inet 192.147.25.45 netmask 0xffffff00 broadcast 192.147.25.255 inet 192.147.25.11 netmask 0xffffff00 broadcast 192.147.25.255 inet 192.147.25.66 netmask 0xffffff00 broadcast 192.147.25.255 inet6 2001:470:1f0f:3ad:223:7dff:fe9e:6e8a prefixlen 64 autoconf nd6 options=23 media: Ethernet autoselect status: active laggproto loadbalance lagghash l2,l3,l4 laggport: bce1 flags=4 laggport: bce0 flags=4 thebighonker.lerctr.org /etc/rc.d $ ifconfig bce0 bce0: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:23:7d:9e:6e:8a inet6 fe80::223:7dff:fe9e:6e8a%bce0 prefixlen 64 tentative scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active thebighonker.lerctr.org /etc/rc.d $ ifconfig bce1 bce1: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:23:7d:9e:6e:8a inet6 fe80::223:7dff:fe9e:6e8a%bce1 prefixlen 64 tentative scopeid 0x2 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active thebighonker.lerctr.org /etc/rc.d $ thebighonker.lerctr.org /etc/rc.d $ netstat -s -i ip6 on bce0: 0 total input datagrams 0 datagrams with invalid header received 0 datagrams exceeded MTU received 0 datagrams with no route received 0 datagrams with invalid dst received 0 datagrams with unknown proto received 0 truncated datagrams received 0 input datagrams discarded 0 datagrams delivered to an upper layer protocol 0 datagrams forwarded to this interface 4 datagrams sent from an upper layer protocol 0 total discarded output datagrams 0 output datagrams fragmented 0 output datagrams failed on fragment 0 output datagrams succeeded on fragment 0 incoming datagrams fragmented 0 datagrams reassembled 0 datagrams failed on reassembly 0 multicast datagrams received 4 multicast datagrams sent ip6 on bce1: 0 total input datagrams 0 datagrams with invalid header received 0 datagrams exceeded MTU received 0 datagrams with no route received 0 datagrams with invalid dst received 0 datagrams with unknown proto received 0 truncated datagrams received 0 input datagrams discarded 0 datagrams delivered to an upper layer protocol 0 datagrams forwarded to this interface 4 datagrams sent from an upper layer protocol 0 total discarded output datagrams 0 output datagrams fragmented 0 output datagrams failed on fragment 0 output datagrams succeeded on fragment 0 incoming datagrams fragmented 0 datagrams reassembled 0 datagrams failed on reassembly 0 multicast datagrams received 4 multicast datagrams sent ip6 on lo0: 45224 total input datagrams 0 datagrams with invalid header received 0 datagrams exceeded MTU received 0 datagrams with no route received 0 datagrams with invalid dst received 0 datagrams with unknown proto received 0 truncated datagrams received 0 input datagrams discarded 45224 datagrams delivered to an upper layer protocol 0 datagrams forwarded to this interface 45224 datagrams sent from an upper layer protocol 0 total discarded output datagrams 0 output datagrams fragmented 0 output datagrams failed on fragment 0 output datagrams succeeded on fragment 0 incoming datagrams fragmented 0 datagrams reassembled 0 datagrams failed on reassembly 0 multicast datagrams received 0 multicast datagrams sent ip6 on lagg0: 66698 total input datagrams 0 datagrams with invalid header received 0 datagrams exceeded MTU received 0 datagrams with no route received 0 datagrams with invalid dst received 0 datagrams with unknown proto received 0 truncated datagrams received 1 input datagram discarded 66698 datagrams delivered to an upper layer protocol 0 datagrams forwarded to this interface 70844 datagrams sent from an upper layer protocol 0 total discarded output datagrams 1505 output datagrams fragmented 0 output datagrams failed on fragment 3037 output datagrams succeeded on fragment 477 incoming datagrams fragmented 224 datagrams reassembled 0 datagrams failed on reassembly 498 multicast datagrams received 3 multicast datagrams sent icmp6 on bce0: 0 total input messages 0 total input error messages 0 input destination unreachable errors 0 input administratively prohibited errors 0 input time exceeded errors 0 input parameter problem errors 0 input packet too big errors 0 input echo requests 0 input echo replies 0 input router solicitations 0 input router advertisements 0 input neighbor solicitations 0 input neighbor advertisements 0 input redirects 0 input MLD queries 0 input MLD reports 0 input MLD dones 0 total output messages 0 total output error messages 0 output destination unreachable errors 0 output administratively prohibited errors 0 output time exceeded errors 0 output parameter problem errors 0 output packet too big errors 0 output echo requests 0 output echo replies 0 output router solicitations 0 output router advertisements 0 output neighbor solicitations 0 output neighbor advertisements 0 output redirects 0 output MLD queries 0 output MLD reports 0 output MLD dones icmp6 on bce1: 0 total input messages 0 total input error messages 0 input destination unreachable errors 0 input administratively prohibited errors 0 input time exceeded errors 0 input parameter problem errors 0 input packet too big errors 0 input echo requests 0 input echo replies 0 input router solicitations 0 input router advertisements 0 input neighbor solicitations 0 input neighbor advertisements 0 input redirects 0 input MLD queries 0 input MLD reports 0 input MLD dones 0 total output messages 0 total output error messages 0 output destination unreachable errors 0 output administratively prohibited errors 0 output time exceeded errors 0 output parameter problem errors 0 output packet too big errors 0 output echo requests 0 output echo replies 0 output router solicitations 0 output router advertisements 0 output neighbor solicitations 0 output neighbor advertisements 0 output redirects 0 output MLD queries 0 output MLD reports 0 output MLD dones icmp6 on lo0: 4 total input messages 4 total input error messages 4 input destination unreachable errors 0 input administratively prohibited errors 0 input time exceeded errors 0 input parameter problem errors 0 input packet too big errors 0 input echo requests 0 input echo replies 0 input router solicitations 0 input router advertisements 0 input neighbor solicitations 0 input neighbor advertisements 0 input redirects 0 input MLD queries 0 input MLD reports 0 input MLD dones 4 total output messages 4 total output error messages 4 output destination unreachable errors 0 output administratively prohibited errors 0 output time exceeded errors 0 output parameter problem errors 0 output packet too big errors 0 output echo requests 0 output echo replies 0 output router solicitations 0 output router advertisements 0 output neighbor solicitations 0 output neighbor advertisements 0 output redirects 0 output MLD queries 0 output MLD reports 0 output MLD dones icmp6 on lagg0: 3475 total input messages 10 total input error messages 0 input destination unreachable errors 0 input administratively prohibited errors 0 input time exceeded errors 0 input parameter problem errors 10 input packet too big errors 0 input echo requests 6 input echo replies 0 input router solicitations 495 input router advertisements 1895 input neighbor solicitations 1069 input neighbor advertisements 0 input redirects 0 input MLD queries 0 input MLD reports 0 input MLD dones 2994 total output messages 22 total output error messages 22 output destination unreachable errors 0 output administratively prohibited errors 0 output time exceeded errors 0 output parameter problem errors 0 output packet too big errors 6 output echo requests 0 output echo replies 1 output router solicitation 0 output router advertisements 1069 output neighbor solicitations 1895 output neighbor advertisements 0 output redirects 0 output MLD queries 1 output MLD report 0 output MLD dones thebighonker.lerctr.org /etc/rc.d $ thebighonker.lerctr.org /etc/rc.d $ ndp -p 2001:470:1f0f:3ad::/64 if=lagg0 flags=LAO vltime=2592000, pltime=604800, expire=29d23h57m21s, ref=1 advertised by fe80::218:18ff:fe72:488%lagg0 (reachable) fe80::%lagg0/64 if=lagg0 flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0 No advertising router fe80::%bce1/64 if=bce1 flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0 No advertising router fe80::%bce0/64 if=bce0 flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0 No advertising router fe80::%lo0/64 if=lo0 flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0 No advertising router thebighonker.lerctr.org /etc/rc.d $ The local mods are Karl Denninger's ZFS ARC patch. So, I'm not sure what to say at this point. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 15:25:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A54FE95 for ; Fri, 20 Jun 2014 15:25:57 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76D8B2C67 for ; Fri, 20 Jun 2014 15:25:55 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,514,1400025600"; d="scan'208,223";a="145786748" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 20 Jun 2014 15:25:53 +0000 Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.3.181.6; Fri, 20 Jun 2014 11:25:52 -0400 Message-ID: <53A4527F.90008@citrix.com> Date: Fri, 20 Jun 2014 17:25:51 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Subject: Ordering problem in if_detach_internal regarding if_bridge X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------030003030008040308010507" X-DLP: MIA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 15:25:57 -0000 --------------030003030008040308010507 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Hello, I've stumbled across the following panic when testing Xen netback with if_bridge: Kernel page fault with the following non-sleepable locks held: exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xfffff80006306c18) locked @ /usr/src/sys/m KDB: stack backtrace: X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe0000213490 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 trap() at trap+0xc9d/frame 0xfffffe00002136a0 trap() at trap+0x669/frame 0xfffffe00002138b0 calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 --- trap 0xc, rip = 0xffffffff8221a0ef, rsp = 0xfffffe0000213970, rbp = 0xfffffe00002139e0 --- bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000213a80 ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame 0xfffffe0000213b30 intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame 0xfffffe0000213b70 db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe0000213bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 --- trap 0, rip = 0, rsp = 0xfffffe0000213cb0, rbp = 0 --- I've tracked this down to if_detach_internal setting ifp->if_addr to NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., which causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tries to perform IF_LLADDR on an interface that's in the process of being destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event event has not fired yet). I have the following naive patch that moves the firing of the event before if_addr is set to NULL, but I'm not familiar with the ordering in if_detach_internal, so I'm not sure if this might cause problems in other parts of the code, could someone familiar with the net stuff comment on the best way to deal with it? Thanks, Roger. --------------030003030008040308010507 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="if_detach.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="if_detach.patch" >From 35430bcf6458bb5218268fd28f59bd911ea1ce2b Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Fri, 20 Jun 2014 15:16:58 +0200 Subject: [PATCH] --- sys/net/if.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/sys/net/if.c b/sys/net/if.c index 47b5b9d..1b3227e 100644 --- a/sys/net/if.c +++ b/sys/net/if.c @@ -875,6 +875,10 @@ if_detach_internal(struct ifnet *ifp, int vmove) #endif if_purgemaddrs(ifp); + /* Announce that the interface is gone. */ + rt_ifannouncemsg(ifp, IFAN_DEPARTURE); + EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); + if (!vmove) { /* * Prevent further calls into the device driver via ifnet. @@ -912,9 +916,6 @@ if_detach_internal(struct ifnet *ifp, int vmove) } } - /* Announce that the interface is gone. */ - rt_ifannouncemsg(ifp, IFAN_DEPARTURE); - EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); if (IS_DEFAULT_VNET(curvnet)) devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL); if_delgroups(ifp); -- 1.7.7.5 (Apple Git-26) --------------030003030008040308010507-- From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 19:47:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D25469F4; Fri, 20 Jun 2014 19:47:45 +0000 (UTC) Received: from zibbi.meraka.csir.co.za (zibbi.meraka.csir.co.za [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 1A9CA244E; Fri, 20 Jun 2014 19:47:45 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id 66DB1B843; Fri, 20 Jun 2014 21:47:41 +0200 (SAST) Date: Fri, 20 Jun 2014 21:47:41 +0200 From: John Hay To: Hiroki Sato Subject: Re: network.subr vlan handling broken Message-ID: <20140620194741.GA50421@zibbi.meraka.csir.co.za> References: <20140619103513.GA92393@zibbi.meraka.csir.co.za> <20140620.124803.562992300308664968.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140620.124803.562992300308664968.hrs@allbsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 19:47:46 -0000 Hi Hiroki, On Fri, Jun 20, 2014 at 12:48:03PM +0900, Hiroki Sato wrote: > John Hay wrote > in <20140619103513.GA92393@zibbi.meraka.csir.co.za>: > > jh> Hi Guys, > jh> > jh> freebsd-rc did not react, so I'm just checking on -net too. > jh> > jh> I found after upgrading that vlan handling broke. I tried the following: > jh> > jh> vlans_bce1="6" > jh> ipv4_addrs_bce1_6="inet 10.239.100.2/24" > jh> ifconfig_bce1_6_aliases="inet 10.239.100.2/24" > jh> ifconfig_bce1_6_alias0="inet 10.239.100.2/24" > jh> > jh> I traced it down to ifalias_af_common_handler being called with the > jh> mangled interfcace name _if and it then calls ifconfig with it. Here > jh> is my fix. Any reason not to commit it? My diff is against 10-stable, > jh> but head looks the same. > > Can you try the attached patch? It seemed broken after list_vars() > was introduced. Replacing $_if with $1 also fixes it, but $_if > should be used for ifname as the other parts do. I have tested ipv4 and ipv6 cases and it seems ok: ############################## vlans_re1="6 7" ipv4_addrs_re1_6="inet 10.254.254.253/24" ifconfig_re1_6_aliases="inet 10.254.254.254/24" ifconfig_re1_6_ipv6="inet6 accept_rtadv" ifconfig_re1_7_ipv6="inet6 fd99:6829:597c:2::1 prefixlen 64" root@angel:/etc # ifconfig re1.6 re1.6: flags=8843 metric 0 mtu 1500 options=3 ether 90:2b:34:df:ae:c4 inet6 fe80::922b:34ff:fedf:aec4%re1.6 prefixlen 64 scopeid 0x4 inet 10.254.254.254 netmask 0xffffff00 broadcast 10.254.254.255 inet 10.254.254.253 netmask 0xffffff00 broadcast 10.254.254.255 nd6 options=23 media: Ethernet autoselect (none) status: no carrier vlan: 6 parent interface: re1 root@angel:/etc # ifconfig re1.7 re1.7: flags=8843 metric 0 mtu 1500 options=3 ether 90:2b:34:df:ae:c4 inet6 fd99:6829:597c:2::1 prefixlen 64 inet6 fe80::922b:34ff:fedf:aec4%re1.7 prefixlen 64 scopeid 0x5 nd6 options=21 media: Ethernet autoselect (none) status: no carrier vlan: 7 parent interface: re1 root@angel:/etc # ############################## Thanks John > > jh> While looking through the code I saw that ltr is called with different > jh> styling. Is there a reason for it? Which is the prefered style? > jh> > jh> ltr ${_if} "${_punct}" '_' _if > jh> ltr "$_if" "$_punct" "_" _if > > I do not think there is a reason. > > I think there is no consensus about the style but I am using {} only > when boundary between the variable name and the subsequent characters > can be ambiguous. > > -- Hiroki > Index: network.subr > =================================================================== > --- network.subr (revision 267636) > +++ network.subr (working copy) > @@ -1077,7 +1077,7 @@ > ifalias_af_common() > { > local _ret _if _af _action alias ifconfig_args _aliasn _c _tmpargs _iaf > - local _punct=".-/+" > + local _vif _punct=".-/+" > > _ret=1 > _aliasn= > @@ -1086,11 +1086,11 @@ > _action=$3 > > # Normalize $_if before using it in a pattern to list_vars() > - ltr "$_if" "$_punct" "_" _if > + ltr "$_if" "$_punct" "_" _vif > > # ifconfig_IF_aliasN which starts with $_af > - for alias in `list_vars ifconfig_${_if}_alias[0-9]\* | > - sort_lite -nk1.$((9+${#_if}+7))` > + for alias in `list_vars ifconfig_${_vif}_alias[0-9]\* | > + sort_lite -nk1.$((9+${#_vif}+7))` > do > eval ifconfig_args=\"\$$alias\" > _iaf= > @@ -1118,8 +1118,8 @@ > # backward compatibility: ipv6_ifconfig_IF_aliasN. > case $_af in > inet6) > - for alias in `list_vars ipv6_ifconfig_${_if}_alias[0-9]\* | > - sort_lite -nk1.$((14+${#_if}+7))` > + for alias in `list_vars ipv6_ifconfig_${_vif}_alias[0-9]\* | > + sort_lite -nk1.$((14+${#_vif}+7))` > do > eval ifconfig_args=\"\$$alias\" > case ${_action}:"${ifconfig_args}" in > @@ -1129,7 +1129,7 @@ > alias:*) > _aliasn="${_aliasn} inet6 ${ifconfig_args}" > warn "\$${alias} is obsolete. " \ > - "Use ifconfig_$1_aliasN instead." > + "Use ifconfig_${_vif}_aliasN instead." > ;; > esac > done From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 19:55:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33BF2D04; Fri, 20 Jun 2014 19:55:26 +0000 (UTC) Received: from stargate.chelsio.com (99-65-72-228.uvs.sntcca.sbcglobal.net [99.65.72.228]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC42624FE; Fri, 20 Jun 2014 19:55:25 +0000 (UTC) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id s5KJFM0e001824; Fri, 20 Jun 2014 12:15:22 -0700 Received: from [10.192.166.0] (10.192.166.0) by nice.asicdesigners.com (10.192.160.7) with Microsoft SMTP Server id 14.2.247.3; Fri, 20 Jun 2014 12:15:22 -0700 Message-ID: <53A48849.8080504@chelsio.com> Date: Fri, 20 Jun 2014 12:15:21 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" , Subject: ifaddr refcount problem Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.192.166.0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 19:55:26 -0000 Revision 264905 and 266860 that followed it seem to leak ifaddr references. ifa_ifwithdstaddr and ifa_ifwithnet both install a reference on the ifaddr returned to the caller but ip_output does not release it, eventually leading to a panic when the refcount wraps over to 0 and the ifaddr is freed while it is still on various lists. I'm using this patch for now. Thoughts? Regards, Navdeep diff -r 6dfcecd314af sys/netinet/ip_output.c --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 @@ -243,6 +243,7 @@ again: ifp = ia->ia_ifp; ip->ip_ttl = 1; isbroadcast = 1; + ifa_free((void *)ia); } else if (flags & IP_ROUTETOIF) { if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { @@ -253,6 +254,7 @@ again: ifp = ia->ia_ifp; ip->ip_ttl = 1; isbroadcast = in_broadcast(dst->sin_addr, ifp); + ifa_free((void *)ia); } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && imo != NULL && imo->imo_multicast_ifp != NULL) { /* From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 21:09:36 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09E4F4A4 for ; Fri, 20 Jun 2014 21:09:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E54F12B41 for ; Fri, 20 Jun 2014 21:09:35 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5KL9Zvi009134 for ; Fri, 20 Jun 2014 22:09:35 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 185967] [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 Date: Fri, 20 Jun 2014 21:09:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kvedulv@kvedulv.de X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 21:09:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185967 kvedulv@kvedulv.de changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kvedulv@kvedulv.de --- Comment #9 from kvedulv@kvedulv.de --- This patch did solve my occasional problems with FreeBSD 10-STABLE and a Netgear GS110TP, so somebody should at least have a look at it (or scottl could just commit it). -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 21:14:19 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75F8F72C for ; Fri, 20 Jun 2014 21:14:19 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5CECA2BF4 for ; Fri, 20 Jun 2014 21:14:19 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5KLEJrJ094441 for ; Fri, 20 Jun 2014 22:14:19 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 185967] [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 Date: Fri, 20 Jun 2014 21:14:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kvedulv@kvedulv.de X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 21:14:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185967 --- Comment #10 from kvedulv@kvedulv.de --- Hi Oliver, (In reply to olivier from comment #8) > The patch didn't solve Lagg regression between 9.2 and 10-stable (r267244). > > My configuration only use one lagg member (the second will be added later): _Maybe_ #179926 is your Bug and you could try the patch over there... (Although that problem seems to have existed at least in 9.1) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 22:20:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEC0DAAD; Fri, 20 Jun 2014 22:20:47 +0000 (UTC) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 658C52139; Fri, 20 Jun 2014 22:20:47 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id x48so4364008wes.37 for ; Fri, 20 Jun 2014 15:20:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=sgnT9InKFw7pFdhjwBPkpNZmeAknx1QVulPmFxayTmY=; b=GndWXiAtc2mRH49E3tirKn/FUlyD0QDqhKM6w50zxo/FbXwAV+staqwInA6O5OSbOR Gn1Be/BwYPl4RJyTbEc/bLCpEJ2lzMMcJrT7fd1oH+wgdH5W7brG5JtwOfDCdfpa8Gj9 ZBuLgh1Q3w5Gek+BrlxQn7lfq+qo9mSUR1tfAPtE4PiCm7+FJ/NWibrqU/59uSL1VQbg UH45Amk+iTsJcbd44iEAMu0TdLlkZ4YLIGfJzgz9U5c3tLVzbCyIgqyUQQo3vSPO/BrP fp2Of/ZVfQSUV/N+rXz2m70NjnLIXBEvrlgtxs3k+llJEZG7B1HVgV0QZuIVUAwrSr7u 49OA== MIME-Version: 1.0 X-Received: by 10.194.157.195 with SMTP id wo3mr6659570wjb.130.1403302845657; Fri, 20 Jun 2014 15:20:45 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Fri, 20 Jun 2014 15:20:45 -0700 (PDT) In-Reply-To: <53A48849.8080504@chelsio.com> References: <53A48849.8080504@chelsio.com> Date: Fri, 20 Jun 2014 16:20:45 -0600 X-Google-Sender-Auth: oqAFFhIa-WT2C8IEa4gj0Tylpl0 Message-ID: Subject: Re: ifaddr refcount problem From: Alan Somers To: Navdeep Parhar Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 22:20:48 -0000 On Fri, Jun 20, 2014 at 1:15 PM, Navdeep Parhar wrote: > Revision 264905 and 266860 that followed it seem to leak ifaddr > references. ifa_ifwithdstaddr and ifa_ifwithnet both install a > reference on the ifaddr returned to the caller but ip_output does not > release it, eventually leading to a panic when the refcount wraps over > to 0 and the ifaddr is freed while it is still on various lists. Are you referring to the ifa_ref() calls at lines 1674, 1745, 1756, and 1795 of sys/net/if.c? Those long predate 264905. Why do you think that 264905 introduced the bug? If anything, I guess that it was introduced by change 262747, which removed ifa_free() from the done: block of ip_output(). > > I'm using this patch for now. Thoughts? > > Regards, > Navdeep > > > diff -r 6dfcecd314af sys/netinet/ip_output.c > --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 > +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 > @@ -243,6 +243,7 @@ again: > ifp = ia->ia_ifp; > ip->ip_ttl = 1; > isbroadcast = 1; > + ifa_free((void *)ia); > } else if (flags & IP_ROUTETOIF) { > if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && > (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { > @@ -253,6 +254,7 @@ again: > ifp = ia->ia_ifp; > ip->ip_ttl = 1; > isbroadcast = in_broadcast(dst->sin_addr, ifp); > + ifa_free((void *)ia); > } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && > imo != NULL && imo->imo_multicast_ifp != NULL) { > /* I don't think this is quite right. ip_output() references ia at lines 366, 433, 630-637, 674, and 675. All of those references have NULL checks, so your patch won't cause a use-after-free bug, but I think that the patch might cause statistics collection to fail, and it might cause the IP source address to not be set. Do you have a test case that can reproduce the panic? -Alan From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 22:36:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 597752D2; Fri, 20 Jun 2014 22:36:12 +0000 (UTC) Received: from stargate.chelsio.com (99-65-72-228.uvs.sntcca.sbcglobal.net [99.65.72.228]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3266D22AD; Fri, 20 Jun 2014 22:36:11 +0000 (UTC) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id s5KMaAvL003176; Fri, 20 Jun 2014 15:36:10 -0700 Received: from [10.192.166.0] (10.192.166.0) by nice.asicdesigners.com (10.192.160.7) with Microsoft SMTP Server id 14.2.247.3; Fri, 20 Jun 2014 15:36:09 -0700 Message-ID: <53A4B759.2060604@chelsio.com> Date: Fri, 20 Jun 2014 15:36:09 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Alan Somers Subject: Re: ifaddr refcount problem References: <53A48849.8080504@chelsio.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.192.166.0] Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 22:36:12 -0000 On 06/20/14 15:20, Alan Somers wrote: ... > > Do you have a test case that can reproduce the panic? > > -Alan > Just run some UDP traffic on head or stable/10 and watch the refcount leak on the transmitter. # netperf -H ... -t UDP_STREAM I see these two do ifa_ref (once per packet). kernel`ip_output+0x527 kernel`udp_send+0xa88 kernel`sosend_dgram+0x583 kernel`kern_sendit+0x244 kernel`sendit+0x129 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x3fb kernel`in_pcbladdr+0x160 kernel`in_pcbconnect_setup+0x18b kernel`udp_send+0x3d7 kernel`sosend_dgram+0x583 kernel`kern_sendit+0x244 kernel`sendit+0x129 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x3fb (kgdb) l *(ip_output + 0x527) 0xffffffff80ac37f7 is in ip_output (/usr/src/sys/netinet/ip_output.c:249). 244 ip->ip_ttl = 1; 245 isbroadcast = 1; 246 ifa_free((void *)ia); 247 } else if (flags & IP_ROUTETOIF) { 248 if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && 249 (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { 250 IPSTAT_INC(ips_noroute); 251 error = ENETUNREACH; 252 goto bad; 253 } And only this one does ifa_free (once per packet) kernel`in_pcbladdr+0x1fb kernel`in_pcbconnect_setup+0x18b kernel`udp_send+0x3d7 kernel`sosend_dgram+0x583 kernel`kern_sendit+0x244 kernel`sendit+0x129 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x3fb So there is a net leak of 1 refcount per packet. Regards, Navdeep From owner-freebsd-net@FreeBSD.ORG Fri Jun 20 23:21:04 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB581EB7 for ; Fri, 20 Jun 2014 23:21:04 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70EDE25D8 for ; Fri, 20 Jun 2014 23:21:04 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id f51so3998566qge.22 for ; Fri, 20 Jun 2014 16:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=01CljURC2F2Y9feTw4tMqXRwirEUJBqo83c/OOqVH/4=; b=liYo7MMjxpm9Gq7fI//6F6En87vkQMAZ/j81hcsfslLv2O9KSPOEQIxf+q25CzcAaM xTTVsOkpqEgtT6Y4W0RdJpNv0DacCsrP3XGmT/om4ZgGKJo3lPGSzF1+vjSuzZOHXBXc nkkZMtgWKT3ecsRk15IFeycuaCxZvwit6+0AFpI/IOOJFzWRve7Mt6jw13XZaz88MKy5 /WZ2xCAQ8RLB8Zx9+nN3I3NHfLaUqGxd2bae/qH2CazhFw6vOzafT1qJeMTegJPqKoc+ 5aCMh4OOajFGNPxQ/qB+Y8JBPeESgTZ7sF+wxmuoDNvLhv+QXL59saw2uvuO2zUimU0R nCow== MIME-Version: 1.0 X-Received: by 10.224.166.73 with SMTP id l9mr9751319qay.34.1403306463623; Fri, 20 Jun 2014 16:21:03 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Fri, 20 Jun 2014 16:21:03 -0700 (PDT) Date: Fri, 20 Jun 2014 16:21:03 -0700 Message-ID: Subject: getpeername returning ENOTCONN for a connected socket From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 23:21:04 -0000 Reviving an old thread where Steve found this problem: A call to getpeername on a connected tcp socket returns ENOTCONN with no prior errors being reported by previous socket calls. Please look at http://lists.freebsd.org/pipermail/freebsd-net/2011-January/027647.html for more details. Here is a proposed patch derived from $src/sys/netsmb/smb_trantcp.c:nbssn_recv()'s way of handling a similar situation: Index: sys/kern/uipc_syscalls.c =================================================================== --- sys/kern/uipc_syscalls.c (revision 267693) +++ sys/kern/uipc_syscalls.c (working copy) @@ -1755,6 +1755,12 @@ if (error != 0) return (error); so = fp->f_data; + if ((so->so_state & (SS_ISDISCONNECTED|SS_ISDISCONNECTING)) || + (so->so_rcv.sb_state & SBS_CANTRCVMORE)) { + error = ECONNRESET; + goto done; + } if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) { error = ENOTCONN; goto done; Does this look correct? cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sat Jun 21 01:02:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 955E849E for ; Sat, 21 Jun 2014 01:02:16 +0000 (UTC) Received: from luigi.brtsvcs.net (luigi.brtsvcs.net [204.109.60.246]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 723192D84 for ; Sat, 21 Jun 2014 01:02:15 +0000 (UTC) Received: from chombo.houseloki.net (c-76-115-19-22.hsd1.or.comcast.net [76.115.19.22]) by luigi.brtsvcs.net (Postfix) with ESMTPSA id 540E72D4F9F; Fri, 20 Jun 2014 17:52:47 -0700 (PDT) Received: from [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf] (unknown [IPv6:2601:7:2280:38b:31f8:c4d5:547d:8ddf]) by chombo.houseloki.net (Postfix) with ESMTPSA id 37071271; Fri, 20 Jun 2014 17:52:45 -0700 (PDT) Message-ID: <53A4D75A.6080103@bluerosetech.com> Date: Fri, 20 Jun 2014 17:52:42 -0700 From: Darren Pilgrim Reply-To: freebsd-net@freebsd.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Massimiliano Stucchi , freebsd-net@freebsd.org Subject: Re: 6rd and DNS (bind/nsd) on FreeBSD References: <53A254F7.4040801@bluerosetech.com> <53A2D036.8000007@glevia.com> In-Reply-To: <53A2D036.8000007@glevia.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 01:02:16 -0000 On 6/19/2014 4:57 AM, Massimiliano Stucchi wrote: > On 19/06/14 05:11, Darren Pilgrim wrote: > >> >> FreeBSD doesn't support 6rd. Ironically, pfSense does. > > This is not entirely true. 6RD is about establishing a 6to4 tunnel to a > well-defined tunnel server in your provider's infrastructure, so as long > as you have the details about the tunnel server's IP and the prefix > you're assigned, you can easily do it manually (or, better, write a tiny > script to do it for you). Do you have a working example? FreeBSD's stf interface doesn't support prefixes other than 2002::/16 or IPv4 mask lengths other than 0. From owner-freebsd-net@FreeBSD.ORG Sat Jun 21 16:00:17 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23C68267 for ; Sat, 21 Jun 2014 16:00:17 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 049972DC2 for ; Sat, 21 Jun 2014 16:00:17 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 235BF1936DE; Sat, 21 Jun 2014 16:00:16 +0000 (UTC) Subject: Re: getpeername returning ENOTCONN for a connected socket From: Sean Bruno Reply-To: sbruno@freebsd.org To: hiren panchasara In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Sat, 21 Jun 2014 09:00:15 -0700 Message-ID: <1403366415.39384.11.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 16:00:17 -0000 On Fri, 2014-06-20 at 16:21 -0700, hiren panchasara wrote: > Reviving an old thread where Steve found this problem: A call to > getpeername on a connected tcp socket returns ENOTCONN with no prior > errors being reported by previous socket calls. > > Please look at http://lists.freebsd.org/pipermail/freebsd-net/2011-January/027647.html > for more details. > > Here is a proposed patch derived from > $src/sys/netsmb/smb_trantcp.c:nbssn_recv()'s way of handling a similar > situation: > > Index: sys/kern/uipc_syscalls.c > =================================================================== > --- sys/kern/uipc_syscalls.c (revision 267693) > +++ sys/kern/uipc_syscalls.c (working copy) > @@ -1755,6 +1755,12 @@ > if (error != 0) > return (error); > so = fp->f_data; > + if ((so->so_state & (SS_ISDISCONNECTED|SS_ISDISCONNECTING)) || > + (so->so_rcv.sb_state & SBS_CANTRCVMORE)) { > + error = ECONNRESET; > + goto done; > + } > if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) { > error = ENOTCONN; > goto done; > > Does this look correct? > > cheers, > Hiren Has this been tested in "anger" anywhere? sean From owner-freebsd-net@FreeBSD.ORG Sat Jun 21 18:00:09 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81A7CC4; Sat, 21 Jun 2014 18:00:09 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 353B1261F; Sat, 21 Jun 2014 18:00:09 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id l6so4731865qcy.15 for ; Sat, 21 Jun 2014 11:00:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zu9A8i5tu32gkI7sIvcWNrVpu8Ifg1jsELH4LA1nPks=; b=AdLxpJ0DlltSy1Q1FqAn8hJtJpST4Dnm9EixkN2MxB83K56usNCz2+enF2towsYtdu fYD3R7kuVa1dnZICqlJVvFdNYOTXqaMJ6p929OP6GHaLQ67gFePMudpdu8pm0A/EKZyK MszGTFvocCiem40y9qtpp79ZQR6jMpfMta7lZzBKZVoN5uXJQGl9QOJSO6Po8sloGcrC Q+GH8HfBSpKYgqqhuktO9xAZt/af4HvRAvwgdbkJbnA7VUKotw5omlY+NMyLyQpNsaXE BNXWh9yLiAXzA7GlaKQBCFgReIwbswewokSPe/DKIFGZqIQugdgzj7Saq/yFhomRxLBy HSTA== MIME-Version: 1.0 X-Received: by 10.140.98.234 with SMTP id o97mr16465655qge.35.1403373608340; Sat, 21 Jun 2014 11:00:08 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Sat, 21 Jun 2014 11:00:08 -0700 (PDT) In-Reply-To: <1403366415.39384.11.camel@bruno> References: <1403366415.39384.11.camel@bruno> Date: Sat, 21 Jun 2014 11:00:08 -0700 Message-ID: Subject: Re: getpeername returning ENOTCONN for a connected socket From: hiren panchasara To: Sean Bruno Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 18:00:09 -0000 On Sat, Jun 21, 2014 at 9:00 AM, Sean Bruno wrote: > On Fri, 2014-06-20 at 16:21 -0700, hiren panchasara wrote: >> Reviving an old thread where Steve found this problem: A call to >> getpeername on a connected tcp socket returns ENOTCONN with no prior >> errors being reported by previous socket calls. >> >> Please look at http://lists.freebsd.org/pipermail/freebsd-net/2011-January/027647.html >> for more details. >> >> Here is a proposed patch derived from >> $src/sys/netsmb/smb_trantcp.c:nbssn_recv()'s way of handling a similar >> situation: >> >> Index: sys/kern/uipc_syscalls.c >> =================================================================== >> --- sys/kern/uipc_syscalls.c (revision 267693) >> +++ sys/kern/uipc_syscalls.c (working copy) >> @@ -1755,6 +1755,12 @@ >> if (error != 0) >> return (error); >> so = fp->f_data; >> + if ((so->so_state & (SS_ISDISCONNECTED|SS_ISDISCONNECTING)) || >> + (so->so_rcv.sb_state & SBS_CANTRCVMORE)) { >> + error = ECONNRESET; >> + goto done; >> + } >> if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) { >> error = ENOTCONN; >> goto done; >> >> Does this look correct? >> >> cheers, >> Hiren > > Has this been tested in "anger" anywhere? No. This patch is from code observation after looking at the problem. I should at least writeup a small module to do local testing as Steve did in original report. I'll do that and get back. I'd appreciate if someone can point me to a better way of testing this. (specially in "anger" ;-)) cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sun Jun 22 18:00:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA4B61BF for ; Sun, 22 Jun 2014 18:00:27 +0000 (UTC) Received: from emma.bazalt43.ru (emma.bazalt43.ru [91.193.143.8]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8818E2925 for ; Sun, 22 Jun 2014 18:00:26 +0000 (UTC) Received: from [0.0.0.0] (oneex.me [91.193.143.91]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by emma.bazalt43.ru (Postfix) with ESMTPSA id D7079172EC for ; Mon, 23 Jun 2014 00:00:16 +0600 (YEKT) Message-ID: <53A719C3.3040002@hostelnet.ru> Date: Mon, 23 Jun 2014 00:00:35 +0600 From: Alex Ros Reply-To: noc@hostelnet.ru User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: FreeBSD 9 as PPPoE BRAS(mpd 5.7) kernel panic Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 18:00:27 -0000 Hello. We have a problem with FreeBSD 9 stable (currently on r267070) as PPPoE BRAS by mpd 5.7. Server catches 1-3 kernel panics every month. Two last core.txt's: http://pkg.hostelnet.ru/pub/dump/core.txt.7.txt (r262224) and http://pkg.hostelnet.ru/pub/dump/core.txt.8.txt (r267070) Now trying to test hint with mpd-down script from similar trhread: http://lists.freebsd.org/pipermail/freebsd-net/2014-June/038952.html Hardware: HP DL360 with Intel 82571EB nic. Average load on server: 150-300 Mbit/s and around 150 users. FreeBSD configuration: running on GENERIC kernel IPFW with few rules (no dummynet, no nat) routing via OSPF (quagga) cat /boot/loader.conf net.isr.maxthreads=2 net.isr.numthreads=2 net.graph.maxdata=65536 net.graph.maxalloc=65536 net.isr.defaultqlimit=4096 net.link.ifqmaxlen=10240 cat /etc/sysctl.conf net.inet.ip.fastforwarding=1 net.inet.ip.redirect=0 kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.point_to_point=0 kern.random.sys.harvest.interrupt=0 net.inet.raw.maxdgram=16384 net.inet.raw.recvspace=16384 net.inet.ip.intr_queue_maxlen=10240 net.route.netisr_maxqlen=4096 Maybe someone can help to understand: it is a configuration error or a bug MPD/netgraph? From owner-freebsd-net@FreeBSD.ORG Sun Jun 22 18:14:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C6827F2 for ; Sun, 22 Jun 2014 18:14:24 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FE182A31 for ; Sun, 22 Jun 2014 18:14:24 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id o8so5372192qcw.31 for ; Sun, 22 Jun 2014 11:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=z7s66m5k2y4nYpSUff5zFa+hoTvBaz1l/bYIhdeg9u0=; b=ONnY9Z2ZVh9puNoZDtpFzR674t/K4JKxQSxAU5JIPFRqp2Pygjk1q0FL/+zRKNIBap /FOJfbK9cJuRATNi2+vAc0/sWazTyfDqc+nJOKkmSnJlo5rmXhwLWevXkzlYHzBvgrlo hfe3Ul7h8NN58CWMItRko5krdh5xxOuWbQxUgAJZlo9okvsHGlpb8lMQpaaWN9y45mge QODZKHGuIP18JvKogKR7ZDItN5AUZxqbLlO7CbRT6U8wfdO5yjoCaXStM57t2JfqVCje f8IhS/c0YoppaU3y4+MwcBJCK6tgGV0HNQhGW0IojyDBMl40ecLHQ0diiMzvzDZBCZS3 oYPw== MIME-Version: 1.0 X-Received: by 10.140.22.134 with SMTP id 6mr23971697qgn.4.1403460862904; Sun, 22 Jun 2014 11:14:22 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sun, 22 Jun 2014 11:14:22 -0700 (PDT) In-Reply-To: <53A719C3.3040002@hostelnet.ru> References: <53A719C3.3040002@hostelnet.ru> Date: Sun, 22 Jun 2014 11:14:22 -0700 X-Google-Sender-Auth: 5aWxTuWqm-8DFy4KaaGDrFO9XSo Message-ID: Subject: Re: FreeBSD 9 as PPPoE BRAS(mpd 5.7) kernel panic From: Adrian Chadd To: noc@hostelnet.ru Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 18:14:24 -0000 They're NULL pointer derferences, so it's likely a race condition with some other thread destroying something and setting the pointer value to NULL somewhere. I thought this was a reasonably well known problem? Was it ever fixed in 10/head? -a On 22 June 2014 11:00, Alex Ros wrote: > Hello. > We have a problem with FreeBSD 9 stable (currently on r267070) as PPPoE BRAS > by mpd 5.7. Server catches 1-3 kernel panics every month. > Two last core.txt's: > http://pkg.hostelnet.ru/pub/dump/core.txt.7.txt (r262224) > and > http://pkg.hostelnet.ru/pub/dump/core.txt.8.txt (r267070) > Now trying to test hint with mpd-down script from similar trhread: > http://lists.freebsd.org/pipermail/freebsd-net/2014-June/038952.html > Hardware: HP DL360 with Intel 82571EB nic. > Average load on server: 150-300 Mbit/s and around 150 users. > FreeBSD configuration: > running on GENERIC kernel > IPFW with few rules (no dummynet, no nat) > routing via OSPF (quagga) > > cat /boot/loader.conf > net.isr.maxthreads=2 > net.isr.numthreads=2 > net.graph.maxdata=65536 > net.graph.maxalloc=65536 > net.isr.defaultqlimit=4096 > net.link.ifqmaxlen=10240 > > cat /etc/sysctl.conf > net.inet.ip.fastforwarding=1 > net.inet.ip.redirect=0 > kern.random.sys.harvest.ethernet=0 > kern.random.sys.harvest.point_to_point=0 > kern.random.sys.harvest.interrupt=0 > net.inet.raw.maxdgram=16384 > net.inet.raw.recvspace=16384 > net.inet.ip.intr_queue_maxlen=10240 > net.route.netisr_maxqlen=4096 > > Maybe someone can help to understand: it is a configuration error or a bug > MPD/netgraph? > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Jun 22 19:48:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A91DCB40 for ; Sun, 22 Jun 2014 19:48:19 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C8E62099 for ; Sun, 22 Jun 2014 19:48:18 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WynkK-0007nD-IY for freebsd-net@freebsd.org; Sun, 22 Jun 2014 12:48:12 -0700 Date: Sun, 22 Jun 2014 12:48:12 -0700 (PDT) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <1403466492547-5922962.post@n5.nabble.com> Subject: Latest update of dnscrypt-proxy broke DNSSEC chain MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 19:48:19 -0000 I have {unbound + dnscrypt-proxy} running in a jail. /etc/passwd in jail has below and appears started in sockstat, but provides no log records. My setup was working before I did "pkg upgrade" in the jail. _dnscrypt-proxy:*:978:65534::0:0:dnscrypt-proxy user:/var/empty:/usr/sbin/nologin # dnscrypt-proxy -t 1 -R dnscrypt.eu-nl [NOTICE] Starting dnscrypt-proxy 1.4.0 [INFO] Initializing libsodium for optimal performance [INFO] Generating a new key pair [INFO] Done [INFO] Server certificate #808464433 received [INFO] This certificate looks valid [INFO] Chosen certificate #808464433 is valid from [2013-12-27] to [2014-12-27] [INFO] Server key fingerprint is SOME:GEN:KEY:XX:YY:ETC /etc/rc.conf: dnscrypt_proxy_enable="YES" dnscrypt_proxy_flags="-d -a 192.168.2.xx:9040 -R dnscrypt.eu-nl --logfile=/var/log/dnscrypt-proxy.log -m 2" #_unused_dnscrypt_proxy_flags # -L /var/unbound/dnscrypt-resolvers.csv # --provider-key= >From host or inside the jail, "# drill -TD -k /var/unbound/root.key" -> ; E;; Error verifying denial of existence for name com.NS: No DNSSEC signature(s) Jail's var/log/debug.log shows: unbound: [4180:0] debug: validator[module 0] operate: extstate:module_state_initial event:module_event_new unbound: [4180:0] debug: iterator[module 1] operate: extstate:module_state_initial event:module_event_pass unbound: [4180:0] debug: sending to target: <.> 192.168.2.xx#9040 unbound: [4180:0] debug: cache memory msg=71924 rrset=70715 infra=2849 val=66401 My var/unbound/unbound.conf: server: verbosity: 3 chroot: "" port: 53 # port to answer queries from do-ip4: yes # Enable IPv4, "yes" or "no". do-ip6: no # Enable IPv6, "yes" or "no". do-udp: yes # Enable UDP, "yes" or "no". do-tcp: yes auto-trust-anchor-file: "/var/unbound/root.key" val-clean-additional: yes root-hints: "/var/unbound/root.hints" hide-identity: yes hide-version: yes harden-glue: yes harden-dnssec-stripped: yes harden-short-bufsize: yes harden-large-queries: yes use-caps-for-id: yes prefetch: yes prefetch-key: yes num-threads: 1 # private-address: 127.0.1.0/28 - breaks things private-address: 192.168.1.0/24 private-address: 192.168.2.0/26 do-not-query-localhost: no forward-zone: name: "." forward-addr: 192.168.2.xx@9040 # does not work: 127.0.0.1@9040 ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/Latest-update-of-dnscrypt-proxy-broke-DNSSEC-chain-tp5922962.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sun Jun 22 23:16:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 816D6F1; Sun, 22 Jun 2014 23:16:31 +0000 (UTC) Received: from zimbra.nitronet.pl (zimbra.nitronet.pl [79.98.150.2]) by mx1.freebsd.org (Postfix) with ESMTP id 392372F70; Sun, 22 Jun 2014 23:16:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id B53D660E7F; Mon, 23 Jun 2014 01:21:02 +0200 (CEST) Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 0s0DV5HJJgwd; Mon, 23 Jun 2014 01:21:00 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra.nitronet.pl (Postfix) with ESMTP id 5B42C63583; Mon, 23 Jun 2014 01:21:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.nitronet.pl Received: from zimbra.nitronet.pl ([127.0.0.1]) by localhost (zimbra.nitronet.pl [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id CQ7jrtgmdSfd; Mon, 23 Jun 2014 01:21:00 +0200 (CEST) Received: from hC35A6B23.cli.nitronet.pl (hC35A6B23.cli.nitronet.pl [195.90.107.35]) by zimbra.nitronet.pl (Postfix) with ESMTPSA id 3322B60E7F; Mon, 23 Jun 2014 01:21:00 +0200 (CEST) Date: Mon, 23 Jun 2014 01:16:20 +0200 From: =?windows-1250?Q?Pawe=B3_Tyll?= X-Priority: 3 (Normal) Message-ID: <2510165566.20140623011620@ofca.me> To: Adrian Chadd Subject: Re: FreeBSD 9 as PPPoE BRAS(mpd 5.7) kernel panic In-Reply-To: References: <53A719C3.3040002@hostelnet.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , noc@hostelnet.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 23:16:31 -0000 Hello Adrian, Sunday, June 22, 2014, 8:14:22 PM, you wrote: > They're NULL pointer derferences, so it's likely a race condition with > some other thread destroying something and setting the pointer value > to NULL somewhere. > I thought this was a reasonably well known problem? Was it ever fixed > in 10/head? It probably wasn't, since I had similar issues on 10-STABLE r266523. It's caused (most often) by packet returning from traffic shaping queues, when in the meantime ng interface got destroyed by mpd. That's why the sleep hack works; mpd destroys interface 1s later that usual. Curious though, that hostelnet uses ng_car for traffic shaping, yet things still panic, even though whole ordeal happens inside netgraph. Lots of entry-points for a fix :) From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 02:27:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2DCFA79C for ; Mon, 23 Jun 2014 02:27:29 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ADB91209C for ; Mon, 23 Jun 2014 02:27:28 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hi2so3309676wib.17 for ; Sun, 22 Jun 2014 19:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=/JyWymNNrXj55OhDjFtEBi0KFOmrLjbw40kr3qA96HA=; b=dcswcUYc8xHDY7n3No2BGx+xlSeMq00hXSZYny3wuKKSxLqrgdJHlNHknQWb54f5r5 zj8eayv8uoHOA8+diR/tXDn/TRLL5GJnaCNVWMLgCXzsHKnGTAkMGD9MOFbAClJewgYq qayNXTdlzefcuBLTdyTM5xXTdT2qChV4nlOIoBtbLy7QlPP+wozSUtl7o4oWtH8GFpD0 fRaf63KASPISnSqFjwC+WR8t4PplJLp8/Jr6rHgy1CVjVHI1rP40ICvjNZaeKl3Ds8hF Oop+HvOlBR09fXXftnW/uHujZKeeKeylkp8yajnG3HfOMI2FyshNtoDJM3QDdlAAHLBH Gyfw== MIME-Version: 1.0 X-Received: by 10.180.210.134 with SMTP id mu6mr17687187wic.18.1403490446952; Sun, 22 Jun 2014 19:27:26 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Sun, 22 Jun 2014 19:27:26 -0700 (PDT) Reply-To: araujo@FreeBSD.org Date: Mon, 23 Jun 2014 10:27:26 +0800 Message-ID: Subject: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Marcelo Araujo To: FreeBSD Net Content-Type: multipart/mixed; boundary=001a11c37bdc896b5204fc779588 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 02:27:29 -0000 --001a11c37bdc896b5204fc779588 Content-Type: text/plain; charset=UTF-8 Hello guys, I made some changes on roundrobin protocol where from now you can via sysctl(8) set a better packets distribution among the interfaces that are part of the lagg(4) group. My motivation for this change was interfaces that use TSO, as example ixgbe(4), the performance is terrible, as we can't full fill the TSO buffer at once, the throughput drops expressively and we have much more sack between hosts. So, with this patch we can set the number of packets that will be send before switch to the next interface. In my testbed using ixgbe(4), I had a very good performance as you can see bellow: 1) Without patch: ------------------------------------------------------------ Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 32.5 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.1 port 32808 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 406 MBytes 3.40 Gbits/sec [ 3] 1.0- 2.0 sec 391 MBytes 3.28 Gbits/sec [ 3] 2.0- 3.0 sec 406 MBytes 3.41 Gbits/sec [ 3] 3.0- 4.0 sec 585 MBytes 4.91 Gbits/sec [ 3] 4.0- 5.0 sec 477 MBytes 4.00 Gbits/sec [ 3] 5.0- 6.0 sec 429 MBytes 3.60 Gbits/sec [ 3] 6.0- 7.0 sec 520 MBytes 4.36 Gbits/sec [ 3] 7.0- 8.0 sec 385 MBytes 3.23 Gbits/sec [ 3] 8.0- 9.0 sec 414 MBytes 3.48 Gbits/sec [ 3] 9.0-10.0 sec 515 MBytes 4.32 Gbits/sec [ 3] 0.0-10.0 sec 4.42 GBytes 3.80 Gbits/sec 2) With patch: ------------------------------------------------------------ Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 32.5 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.1 port 10526 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 694 MBytes 5.83 Gbits/sec [ 3] 1.0- 2.0 sec 999 MBytes 8.38 Gbits/sec [ 3] 2.0- 3.0 sec 1.17 GBytes 10.1 Gbits/sec [ 3] 3.0- 4.0 sec 1.34 GBytes 11.5 Gbits/sec [ 3] 4.0- 5.0 sec 1.15 GBytes 9.91 Gbits/sec [ 3] 5.0- 6.0 sec 1.19 GBytes 10.2 Gbits/sec [ 3] 6.0- 7.0 sec 1.08 GBytes 9.23 Gbits/sec [ 3] 7.0- 8.0 sec 1.10 GBytes 9.45 Gbits/sec [ 3] 8.0- 9.0 sec 1.27 GBytes 10.9 Gbits/sec [ 3] 9.0-10.0 sec 1.39 GBytes 12.0 Gbits/sec [ 3] 0.0-10.0 sec 11.3 GBytes 9.74 Gbits/sec So, basically we have a sysctl(8) called "net.link.lagg.rr_packets" where we can set the number of packets that will be send before the roundrobin move to the next interface. Any comment and review are very appreciated. Best Regards, -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) --001a11c37bdc896b5204fc779588 Content-Type: application/octet-stream; name="if_lagg.patch" Content-Disposition: attachment; filename="if_lagg.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hwr5uz6s0 SW5kZXg6IGlmX2xhZ2cuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBpZl9sYWdnLmMJKHJldmlzaW9uIDI2NzY2 NikKKysrIGlmX2xhZ2cuYwkod29ya2luZyBjb3B5KQpAQCAtMTg5LDYgKzE4OSwxMCBAQAogU1lT Q1RMX0lOVChfbmV0X2xpbmtfbGFnZywgT0lEX0FVVE8sIGRlZmF1bHRfZmxvd2lkX3NoaWZ0LCBD VExGTEFHX1JXLAogICAgICZkZWZfZmxvd2lkX3NoaWZ0LCAwLAogICAgICJEZWZhdWx0IHNldHRp bmcgZm9yIGZsb3dpZCBzaGlmdCBmb3IgbG9hZCBzaGFyaW5nIik7CitzdGF0aWMgaW50IGxhZ2df cnJfcGFja2V0cyA9IDA7IC8qIERlZmF1bHQgdmFsdWUgZm9yIHVzaW5nIHJyX3BhY2tldHMgKi8K K1NZU0NUTF9JTlQoX25ldF9saW5rX2xhZ2csIE9JRF9BVVRPLCBycl9wYWNrZXRzLCBDVExGTEFH X1JXLAorICAgICZsYWdnX3JyX3BhY2tldHMsIDAsCisgICAgIkhvdyBtYW55IHBhY2tldHMgdG8g YmUgc2VuZCBwZXIgaW50ZXJmYWNlIik7CiAKIHN0YXRpYyBpbnQKIGxhZ2dfbW9kZXZlbnQobW9k dWxlX3QgbW9kLCBpbnQgdHlwZSwgdm9pZCAqZGF0YSkKQEAgLTE2ODksMTQgKzE2OTMsNjggQEAK IHsKIAlzdHJ1Y3QgbGFnZ19wb3J0ICpscDsKIAl1aW50MzJfdCBwOworCXVpbnQzMl90IHBrdF9z eXNjdGxfY291bnQ7CisJaW50IGlmcF9jb3VudCA9IDE7CiAKIAlwID0gYXRvbWljX2ZldGNoYWRk XzMyKCZzYy0+c2Nfc2VxLCAxKTsKIAlwICU9IHNjLT5zY19jb3VudDsKIAlscCA9IFNMSVNUX0ZJ UlNUKCZzYy0+c2NfcG9ydHMpOwotCXdoaWxlIChwLS0pCi0JCWxwID0gU0xJU1RfTkVYVChscCwg bHBfZW50cmllcyk7CiAKIAkvKgorCSAqIElmIHRoZXJlIGlzIG5vIHJlZmVyZW5jZSBmb3IgdGhl IElGUCwgd2UgbXVzdAorIAkgKiBjb3B5IGl0IG5vdy4KKwkgKi8KKwlpZiAoc3RybGVuKHNjLT5z Y19yZWZfaWZwKSA9PSAwKQorCQlzdHJuY3B5KHNjLT5zY19yZWZfaWZwLCBscC0+bHBfaWZwLT5p Zl94bmFtZSwgc2l6ZW9mKHNjLT5zY19yZWZfaWZwKSk7CisgICAgICAgICAgICAgIAorCS8qCisJ ICogSWYgaWZwX2NvdW50IHdhcyBub3QgeWV0IGluaXRpYWxpemVkLCB3ZSBtdXN0CisJICogaW5p dGlhbGl6ZSBub3cuCisJICovCisJaWYgKHNjLT5zY19pZnBfY291bnQgPT0gMCkKKwkJc2MtPnNj X2lmcF9jb3VudCA9IDE7CisKKwkvKgorCSAqIElmIHRoZSBzeXNjdGwgcnJfcGFja2V0cyBpcyBz ZXQgdG8gMCwgd2UgbXVzdCB1c2UgdGhlCisJICogcm91bmRyb2JpbiBhcyBpdCBpcywgb3Igb3Ro ZXJ3aXNlLCB3ZSBtdXN0IGFwcGx5IHRoZQorCSAqIGdyYW51bGFyaXR5IGJldHdlZW4gdGhlIGlu dGVyZmFjZXMgdGhhdCBhcmUgcGFydCBvZiB0aGUgZ3JvdXAuCisJICovCisJaWYgKCFsYWdnX3Jy X3BhY2tldHMpIHsKKwkJd2hpbGUgKHAtLSkKKwkJCWxwID0gU0xJU1RfTkVYVChscCwgbHBfZW50 cmllcyk7CisJCWdvdG8gc2VuZF9tYnVmOworCX0gZWxzZSB7CisJCXBrdF9zeXNjdGxfY291bnQg PSBhdG9taWNfZmV0Y2hhZGRfMzIoJnNjLT5zY19wa3RfY291bnQsIDEpOworCQlpZiAocGt0X3N5 c2N0bF9jb3VudCA9PSBsYWdnX3JyX3BhY2tldHMpIHsKKwkJCWlmIChzYy0+c2NfaWZwX2NvdW50 IDw9IHNjLT5zY19jb3VudCkgeworCQkJCXdoaWxlIChpZnBfY291bnQgPCBzYy0+c2NfaWZwX2Nv dW50KSB7CisJCQkJCWxwID0gU0xJU1RfTkVYVChscCwgbHBfZW50cmllcyk7CisJCQkJCWlmcF9j b3VudCsrOworCQkJCX0KKwkJCQlzYy0+c2NfaWZwX2NvdW50Kys7CisJCQkJaWYgKHNjLT5zY19p ZnBfY291bnQgPiBzYy0+c2NfY291bnQpCisJCQkJCXNjLT5zY19pZnBfY291bnQgPSAwOworCQkJ fQorCQkJc3RybmNweShzYy0+c2NfcmVmX2lmcCwgbHAtPmxwX2lmcC0+aWZfeG5hbWUsIHNpemVv ZihzYy0+c2NfcmVmX2lmcCkpOworCQkJc2MtPnNjX3BrdF9jb3VudCA9IDA7CisJCX0KKwl9CisK KwkvKgorCSAqIENoZWNrIGlmIHRoZSBjdXJyZW50IGludGVyZmFjZSB0byBiZSBlbnF1ZXVlIGlz IG5vdCB0aGUKKwkgKiBzYW1lIHVzZWQgaW4gdGhlIGxhc3Qgcm91bmQuCisJICovCisJbHAgPSBT TElTVF9GSVJTVCgmc2MtPnNjX3BvcnRzKTsKKwlmb3IgKDs7KSB7CisJCWlmIChzdHJjbXAobHAt PmxwX2lmcC0+aWZfeG5hbWUsIHNjLT5zY19yZWZfaWZwKSA9PSAwKQorCQkJYnJlYWs7CisJCWVs c2UKKwkJCWxwID0gU0xJU1RfTkVYVChscCwgbHBfZW50cmllcyk7CisJfQorCWdvdG8gc2VuZF9t YnVmOworCitzZW5kX21idWY6CisJLyoKIAkgKiBDaGVjayB0aGUgcG9ydCdzIGxpbmsgc3RhdGUu IFRoaXMgd2lsbCByZXR1cm4gdGhlIG5leHQgYWN0aXZlCiAJICogcG9ydCBpZiB0aGUgbGluayBp cyBkb3duIG9yIHRoZSBwb3J0IGlzIE5VTEwuCiAJICovCkluZGV4OiBpZl9sYWdnLmgKPT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PQotLS0gaWZfbGFnZy5oCShyZXZpc2lvbiAyNjc2NjYpCisrKyBpZl9sYWdnLmgJKHdvcmtp bmcgY29weSkKQEAgLTIzMiw2ICsyMzIsOSBAQAogCXN0cnVjdCBzeXNjdGxfb2lkCQkqc2Nfb2lk OwkvKiBzeXNjdGwgdHJlZSBvaWQgKi8KIAlpbnQJCQkJdXNlX2Zsb3dpZDsJLyogdXNlIE1fRkxP V0lEICovCiAJaW50CQkJCWZsb3dpZF9zaGlmdDsJLyogc2hpZnQgdGhlIGZsb3dpZCAqLworCXVp bnQzMl90CQkJc2NfcGt0X2NvdW50OyAvKiB1c2UgZm9yIGNvdW50IHBhY2thdGVzIHBlciBpZnAg Ki8KKwlpbnQJCQkJc2NfaWZwX2NvdW50OyAvKiBjb3VudGVyIHJlZmVyZW5jZSBvZiBpbnRlcmZh Y2VzIG9uIHJyICovCisJY2hhcgkJCQlzY19yZWZfaWZwW0lGTkFNU0laXTsgLyogbmFtZSBvZiB0 aGUgaWZwICovCiB9OwogCiBzdHJ1Y3QgbGFnZ19wb3J0IHsK --001a11c37bdc896b5204fc779588-- From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 04:16:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 387695AD; Mon, 23 Jun 2014 04:16:21 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE5CC2935; Mon, 23 Jun 2014 04:16:20 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id w7so5562548qcr.21 for ; Sun, 22 Jun 2014 21:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ucVynsLPF2Yf6uU4pLFCwz9klU7U0Azr4wm6cjsjrWA=; b=oZBU2Hc27GSeG/N//xwk8RFYh1ML2Hebf2aC6gMKm3aSSmTpYfNbEgGttSPXOb3ltG YdyGFZc+epNqLTrEWomZdwYIaHCQxRsWfGd08wJP16BM9WukIJp7io6WwrY4k2S2uczu yBTdAsTKbFmU5MZ5WRSq0iRyi2XYRrx3dcDsk3J1j+UrSVG5InUeCZpkUEI/QYVMRQ6l +Iwc3bXFRLMr40/Bf4PSRtaca3zil5XaeS37Z+wU9H6kkq9XzzqTHBfAXZvUZpJxShy3 GUhlo4+XCKEpL/kCvvV+9knON2Bt0uupkMfufmiqvXrx173dHD7oYE0D5e99JbQjnW0l rKJg== MIME-Version: 1.0 X-Received: by 10.224.66.70 with SMTP id m6mr29823754qai.55.1403496979871; Sun, 22 Jun 2014 21:16:19 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sun, 22 Jun 2014 21:16:19 -0700 (PDT) In-Reply-To: References: Date: Sun, 22 Jun 2014 21:16:19 -0700 X-Google-Sender-Auth: kmsG9OfSNnjZMFDYFx-nAhokvUo Message-ID: Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Adrian Chadd To: araujo@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 04:16:21 -0000 ... It's an interesting idea, but doing round robin like that may introduce out of order packets. What's the actual problem you're seeing? Are the transmit queues filling up? Is the distribution with flowid/curcpu not good enough? Scott saw this happen at Netflix. He added a lagg twiddle to set which set of bits to care about in the flowid when picking an interface to choose. The ixgbe hashing was being done on the low x bits, where x is related to how many CPUs you have (2 CPUs? 1 bit. 8 CPUs? 3 bits. etc.) lagg was doing the same thing on the same low order set of bits. He modified lagg so you could pick some new starting point a few bits up in the flowid to pick a lagg interface with. That fixed the distribution issue and also kept the in-orderness of it all. 2c, -a On 22 June 2014 19:27, Marcelo Araujo wrote: > Hello guys, > > I made some changes on roundrobin protocol where from now you can via > sysctl(8) set a better packets distribution among the interfaces that are > part of the lagg(4) group. > > My motivation for this change was interfaces that use TSO, as example > ixgbe(4), the performance is terrible, as we can't full fill the TSO buffer > at once, the throughput drops expressively and we have much more sack > between hosts. > > So, with this patch we can set the number of packets that will be send > before switch to the next interface. > > In my testbed using ixgbe(4), I had a very good performance as you can see > bellow: > > 1) Without patch: > ------------------------------------------------------------ > Client connecting to 192.168.1.2, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.1.1 port 32808 connected with 192.168.1.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0- 1.0 sec 406 MBytes 3.40 Gbits/sec > [ 3] 1.0- 2.0 sec 391 MBytes 3.28 Gbits/sec > [ 3] 2.0- 3.0 sec 406 MBytes 3.41 Gbits/sec > [ 3] 3.0- 4.0 sec 585 MBytes 4.91 Gbits/sec > [ 3] 4.0- 5.0 sec 477 MBytes 4.00 Gbits/sec > [ 3] 5.0- 6.0 sec 429 MBytes 3.60 Gbits/sec > [ 3] 6.0- 7.0 sec 520 MBytes 4.36 Gbits/sec > [ 3] 7.0- 8.0 sec 385 MBytes 3.23 Gbits/sec > [ 3] 8.0- 9.0 sec 414 MBytes 3.48 Gbits/sec > [ 3] 9.0-10.0 sec 515 MBytes 4.32 Gbits/sec > [ 3] 0.0-10.0 sec 4.42 GBytes 3.80 Gbits/sec > > 2) With patch: > ------------------------------------------------------------ > Client connecting to 192.168.1.2, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.1.1 port 10526 connected with 192.168.1.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0- 1.0 sec 694 MBytes 5.83 Gbits/sec > [ 3] 1.0- 2.0 sec 999 MBytes 8.38 Gbits/sec > [ 3] 2.0- 3.0 sec 1.17 GBytes 10.1 Gbits/sec > [ 3] 3.0- 4.0 sec 1.34 GBytes 11.5 Gbits/sec > [ 3] 4.0- 5.0 sec 1.15 GBytes 9.91 Gbits/sec > [ 3] 5.0- 6.0 sec 1.19 GBytes 10.2 Gbits/sec > [ 3] 6.0- 7.0 sec 1.08 GBytes 9.23 Gbits/sec > [ 3] 7.0- 8.0 sec 1.10 GBytes 9.45 Gbits/sec > [ 3] 8.0- 9.0 sec 1.27 GBytes 10.9 Gbits/sec > [ 3] 9.0-10.0 sec 1.39 GBytes 12.0 Gbits/sec > [ 3] 0.0-10.0 sec 11.3 GBytes 9.74 Gbits/sec > > So, basically we have a sysctl(8) called "net.link.lagg.rr_packets" where > we can set the number of packets that will be send before the roundrobin > move to the next interface. > > Any comment and review are very appreciated. > > Best Regards, > > -- > Marcelo Araujo (__)araujo@FreeBSD.org > \\\'',)http://www.FreeBSD.org \/ \ ^ > Power To Server. .\. /_) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 04:42:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCBB2ADD; Mon, 23 Jun 2014 04:42:47 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B0B392B48; Mon, 23 Jun 2014 04:42:47 +0000 (UTC) Received: from Julian-MBP3.local (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5N4gfkj090730 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 22 Jun 2014 21:42:44 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53A7B03B.60509@freebsd.org> Date: Mon, 23 Jun 2014 12:42:35 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: =?UTF-8?B?UGF3ZcWCIFR5bGw=?= , Adrian Chadd Subject: Re: FreeBSD 9 as PPPoE BRAS(mpd 5.7) kernel panic References: <53A719C3.3040002@hostelnet.ru> <2510165566.20140623011620@ofca.me> In-Reply-To: <2510165566.20140623011620@ofca.me> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net , noc@hostelnet.ru X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 04:42:47 -0000 On 6/23/14, 7:16 AM, Paweł Tyll wrote: > Hello Adrian, > > Sunday, June 22, 2014, 8:14:22 PM, you wrote: >> They're NULL pointer derferences, so it's likely a race condition with >> some other thread destroying something and setting the pointer value >> to NULL somewhere. >> I thought this was a reasonably well known problem? Was it ever fixed >> in 10/head? > It probably wasn't, since I had similar issues on 10-STABLE r266523. > It's caused (most often) by packet returning from traffic shaping > queues, when in the meantime ng interface got destroyed by mpd. That's > why the sleep hack works; mpd destroys interface 1s later that usual. > > Curious though, that hostelnet uses ng_car for traffic shaping, yet > things still panic, even though whole ordeal happens inside netgraph. > > Lots of entry-points for a fix :) probably means we need to look at a netgraph framework based fix. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 06:36:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2969EC13 for ; Mon, 23 Jun 2014 06:36:18 +0000 (UTC) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "people.fsn.hu", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D241C22E3 for ; Mon, 23 Jun 2014 06:36:16 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 101811414D03; Mon, 23 Jun 2014 08:36:05 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.3 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 6.6648] X-CRM114-CacheID: sfid-20140623_08360_3AE94ADD X-CRM114-Status: Good ( pR: 6.6648 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Jun 23 08:36:05 2014 X-DSPAM-Confidence: 0.9947 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 53a7cad5619901266120378 X-DSPAM-Factors: 27, From*"Nagy, Attila" , 0.00021, ports, 0.00182, ports, 0.00182, timeout, 0.00259, timeout, 0.00259, boot, 0.00323, boot, 0.00323, Received*online.co.hu+[195.228.243.99]), 0.00516, Received*[195.228.243.99]), 0.00516, Received*online.co.hu, 0.00516, Received*(japan.t, 0.00516, Received*(japan.t+online.co.hu, 0.00516, >+on, 0.00573, couldn't+find, 0.00573, the+host, 0.00573, svn, 0.00602, Received*from+japan.t, 0.00645, Received*online.private+(japan.t, 0.00645, Received*japan.t+online.private, 0.00645, Received*japan.t, 0.00645, rev, 0.00736, rev, 0.00736, addr, 0.00736, addr, 0.00736, works+in, 0.00858, output, 0.00988, X-Spambayes-Classification: ham; 0.00 Received: from japan.t-online.private (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 74BE51414CF3 for ; Mon, 23 Jun 2014 08:36:03 +0200 (CEST) Message-ID: <53A7CAD3.9060005@fsn.hu> Date: Mon, 23 Jun 2014 08:36:03 +0200 From: "Nagy, Attila" MIME-Version: 1.0 To: FreeBSD Net Subject: Can't boot from NFS with Emulex BE3 (oce) in stable/10 (9 works) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 06:36:18 -0000 Hi, I have an Emulex BE3 in a HP BL460c G8 machine. I boot it from PXE/NFS, which works in stable/9 (r248885), but doesn't in stable/10 (r267603). The relevant output from the boot process: oce1: Interface Up Sending DHCP Discover packet from interface oce0 (d8:9d:67:61:c2:a8) Sending DHCP Discover packet from interface oce1 (d8:9d:67:61:c2:ac) uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered ugen1.2: at usbus1 ukbd0: on usbus1 kbd2 at ukbd0 ugen2.2: at usbus2 uhub3: on usbus 2 ugen0.2: at usbus0 uhub4: on usbus 0 uhub4: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered ugen2.3: at usbus2 uhub5: on usbus 2 uhub5: 2 ports with 1 removable, self powered DHCP/BOOTP timeout for server 255.255.255.255 DHCP/BOOTP timeout for server 255.255.255.255 DHCP/BOOTP timeout for server 255.255.255.255 And these lines forever. On the DHCP server I can see the DHCPDISCOVERs and also DHCPOFFERs without any response from the host. Skimming through the svn changelog I couldn't find any clues from the commit messages. Any ideas? Thanks, From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 07:31:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F694B3B; Mon, 23 Jun 2014 07:31:05 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63AF126D2; Mon, 23 Jun 2014 07:31:04 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id cc10so3539001wib.0 for ; Mon, 23 Jun 2014 00:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=p2GguCNkCOsd3NlBl4zN08xr2hgFlkwdSRye9LzMgnk=; b=SyOGn5KXdfibk1p2cX/7mz5JA6xFMwFXgHcsDJwjBXyYGPO3pnXS44uMV60Ewk/3uK xDkLTNfKZWUg2CVz1unpogq7rEvWiXj7KoKC7LwybduOSCpmBSVs/DJarBwQISxv/Ae6 xvT3K3wzJj3gX9LiA4A65s9SJjago783aLV+OG6fqaMkJXxYGXJSWwipIZDys7V6crdC nskJCfB228ekJY5SbR7rBJ/OxpfJ2i+SI3JY3ba3wcuoIFnRRTLS87GoGsyQnwcVOJzA wq2DbQzu5vCEyvMukDgU5M4o8b2uqn8vl8WSXiWVFvQ6vDTQMDObcvWJTayZg5L97het zmLA== MIME-Version: 1.0 X-Received: by 10.194.189.81 with SMTP id gg17mr1499197wjc.107.1403508662254; Mon, 23 Jun 2014 00:31:02 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Mon, 23 Jun 2014 00:31:02 -0700 (PDT) Reply-To: araujo@FreeBSD.org In-Reply-To: References: Date: Mon, 23 Jun 2014 15:31:02 +0800 Message-ID: Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Marcelo Araujo To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 07:31:05 -0000 Hello Adrian, 2014-06-23 12:16 GMT+08:00 Adrian Chadd : > ... > > It's an interesting idea, but doing round robin like that may > introduce out of order packets. > Actually, the round robin implementation as it is, causes out of order packets, but almost all the time SACK can recover it. In my tests using iperf, when we set a bigger number of packets to be sent through the same interface before switch to the next one, I can see that we have less SACK request, and I do believe because of it, I can reach a better throughput. The test is very simple: "iperf -s" and "iperf -c -i 1 -t 10". As an example: 1) without change the number of packets: 43 SACK recovery episodes 187 segment rexmits in SACK recovery episodes 270776 byte rexmits in SACK recovery episodes 172688 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 input SACK chunks 0 output SACKs 2) Set 50 packets per interface: 6 SACK recovery episodes 16 segment rexmits in SACK recovery episodes 23168 byte rexmits in SACK recovery episodes 111626 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 input SACK chunks 0 output SACKs > > What's the actual problem you're seeing? Are the transmit queues > filling up? Is the distribution with flowid/curcpu not good enough? > I have had imported Scott's patch, I do believe you are talking about r260070. I didn't pay attention to the flowid/curcpu distribution and I can't tell you if it is the root cause or not, but for my case, it didn't solve the bad performance of round robin. With all the other lagg(4) protocols, the throughput reach the limit of the NIC. It might be likely that the transmit queue isn't filled up or hang for some reason, it is something that I need check. My suspicious is how the ixgbe(4) trigger the TSO, it seems that transmit queue is not completely filled up and it might delay the transmission or lose packets, or perhaps lose the entire queue. Also any tips of how debug the TSO will be very welcome. > > Scott saw this happen at Netflix. He added a lagg twiddle to set which > set of bits to care about in the flowid when picking an interface to > choose. The ixgbe hashing was being done on the low x bits, where x is > related to how many CPUs you have (2 CPUs? 1 bit. 8 CPUs? 3 bits. > etc.) lagg was doing the same thing on the same low order set of bits. > He modified lagg so you could pick some new starting point a few bits > up in the flowid to pick a lagg interface with. That fixed the > distribution issue and also kept the in-orderness of it all. > I thought that Scott's patch is more focused on LACP, I didn't realize that it would helps the other aggregation protocols. Anyway, for round robin, with/without the r260070, don't change too much, at least in my environment. Best Regards, > > 2c, > > > -a > > On 22 June 2014 19:27, Marcelo Araujo wrote: > > Hello guys, > > > > I made some changes on roundrobin protocol where from now you can via > > sysctl(8) set a better packets distribution among the interfaces that are > > part of the lagg(4) group. > > > > My motivation for this change was interfaces that use TSO, as example > > ixgbe(4), the performance is terrible, as we can't full fill the TSO > buffer > > at once, the throughput drops expressively and we have much more sack > > between hosts. > > > > So, with this patch we can set the number of packets that will be send > > before switch to the next interface. > > > > In my testbed using ixgbe(4), I had a very good performance as you can > see > > bellow: > > > > 1) Without patch: > > ------------------------------------------------------------ > > Client connecting to 192.168.1.2, TCP port 5001 > > TCP window size: 32.5 KByte (default) > > ------------------------------------------------------------ > > [ 3] local 192.168.1.1 port 32808 connected with 192.168.1.2 port 5001 > > [ ID] Interval Transfer Bandwidth > > [ 3] 0.0- 1.0 sec 406 MBytes 3.40 Gbits/sec > > [ 3] 1.0- 2.0 sec 391 MBytes 3.28 Gbits/sec > > [ 3] 2.0- 3.0 sec 406 MBytes 3.41 Gbits/sec > > [ 3] 3.0- 4.0 sec 585 MBytes 4.91 Gbits/sec > > [ 3] 4.0- 5.0 sec 477 MBytes 4.00 Gbits/sec > > [ 3] 5.0- 6.0 sec 429 MBytes 3.60 Gbits/sec > > [ 3] 6.0- 7.0 sec 520 MBytes 4.36 Gbits/sec > > [ 3] 7.0- 8.0 sec 385 MBytes 3.23 Gbits/sec > > [ 3] 8.0- 9.0 sec 414 MBytes 3.48 Gbits/sec > > [ 3] 9.0-10.0 sec 515 MBytes 4.32 Gbits/sec > > [ 3] 0.0-10.0 sec 4.42 GBytes 3.80 Gbits/sec > > > > 2) With patch: > > ------------------------------------------------------------ > > Client connecting to 192.168.1.2, TCP port 5001 > > TCP window size: 32.5 KByte (default) > > ------------------------------------------------------------ > > [ 3] local 192.168.1.1 port 10526 connected with 192.168.1.2 port 5001 > > [ ID] Interval Transfer Bandwidth > > [ 3] 0.0- 1.0 sec 694 MBytes 5.83 Gbits/sec > > [ 3] 1.0- 2.0 sec 999 MBytes 8.38 Gbits/sec > > [ 3] 2.0- 3.0 sec 1.17 GBytes 10.1 Gbits/sec > > [ 3] 3.0- 4.0 sec 1.34 GBytes 11.5 Gbits/sec > > [ 3] 4.0- 5.0 sec 1.15 GBytes 9.91 Gbits/sec > > [ 3] 5.0- 6.0 sec 1.19 GBytes 10.2 Gbits/sec > > [ 3] 6.0- 7.0 sec 1.08 GBytes 9.23 Gbits/sec > > [ 3] 7.0- 8.0 sec 1.10 GBytes 9.45 Gbits/sec > > [ 3] 8.0- 9.0 sec 1.27 GBytes 10.9 Gbits/sec > > [ 3] 9.0-10.0 sec 1.39 GBytes 12.0 Gbits/sec > > [ 3] 0.0-10.0 sec 11.3 GBytes 9.74 Gbits/sec > > > > So, basically we have a sysctl(8) called "net.link.lagg.rr_packets" where > > we can set the number of packets that will be send before the roundrobin > > move to the next interface. > > > > Any comment and review are very appreciated. > > > > Best Regards, > > > > -- > > Marcelo Araujo (__)araujo@FreeBSD.org > > \\\'',)http://www.FreeBSD.org \/ \ ^ > > Power To Server. .\. /_) > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 08:00:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D34E750 for ; Mon, 23 Jun 2014 08:00:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53503293C for ; Mon, 23 Jun 2014 08:00:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5N80DdJ087011 for ; Mon, 23 Jun 2014 09:00:13 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201406230800.s5N80DdJ087011@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 23 Jun 2014 08:00:13 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 08:00:13 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 08:01:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F8F19A2 for ; Mon, 23 Jun 2014 08:01:34 +0000 (UTC) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "people.fsn.hu", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C757D2A00 for ; Mon, 23 Jun 2014 08:01:33 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 3F0691425472; Mon, 23 Jun 2014 10:01:24 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.3 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 8.9092] X-CRM114-CacheID: sfid-20140623_10012_316E7DFD X-CRM114-Status: Good ( pR: 8.9092 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Mon Jun 23 10:01:23 2014 X-DSPAM-Confidence: 0.8531 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 53a7ded372847190919736 X-DSPAM-Factors: 27, From*"Nagy, Attila" , 0.00021, wrote+>, 0.00174, >+I, 0.00231, boot, 0.00323, >+>, 0.00327, References*fsn.hu>, 0.00469, Received*online.co.hu+[195.228.243.99]), 0.00516, Received*[195.228.243.99]), 0.00516, In-Reply-To*fsn.hu>, 0.00516, Received*online.co.hu, 0.00516, Received*(japan.t, 0.00516, Received*(japan.t+online.co.hu, 0.00516, wrote, 0.00568, Nagy+Attila, 0.00572, firmware, 0.00572, Received*from+japan.t, 0.00644, Received*online.private+(japan.t, 0.00644, Received*japan.t+online.private, 0.00644, Received*japan.t, 0.00644, 10+0, 0.00856, 0+from, 0.00856, works+in, 0.00856, G8, 0.99000, driver+to, 0.01000, Subject*boot, 0.01000, driver, 0.01125, X-Spambayes-Classification: ham; 0.00 Received: from japan.t-online.private (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 52CDF142545E for ; Mon, 23 Jun 2014 10:01:21 +0200 (CEST) Message-ID: <53A7DED0.8000107@fsn.hu> Date: Mon, 23 Jun 2014 10:01:20 +0200 From: "Nagy, Attila" MIME-Version: 1.0 To: FreeBSD Net Subject: Re: Can't boot from NFS with Emulex BE3 (oce) in stable/10 (9 works) References: <53A7CAD3.9060005@fsn.hu> In-Reply-To: <53A7CAD3.9060005@fsn.hu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 08:01:34 -0000 On 06/23/14 08:36, Nagy, Attila wrote: > > I have an Emulex BE3 in a HP BL460c G8 machine. I boot it from > PXE/NFS, which works in stable/9 (r248885), but doesn't in stable/10 > (r267603). I've upgraded its firmware from 4.6.95.0 to 4.9.416.0 (hp.com latest) and the driver to 10.0.747.0 from emulex.com without any success. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 08:52:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D9883FC; Mon, 23 Jun 2014 08:52:36 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CB20E2EC6; Mon, 23 Jun 2014 08:52:35 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s5N8qTlP038411 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 23 Jun 2014 12:52:29 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s5N8qTD3038410; Mon, 23 Jun 2014 12:52:29 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 23 Jun 2014 12:52:29 +0400 From: Gleb Smirnoff To: Navdeep Parhar Subject: Re: ifaddr refcount problem Message-ID: <20140623085229.GQ28199@FreeBSD.org> References: <53A48849.8080504@chelsio.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53A48849.8080504@chelsio.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , asomers@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 08:52:36 -0000 Navdeep, On Fri, Jun 20, 2014 at 12:15:21PM -0700, Navdeep Parhar wrote: N> Revision 264905 and 266860 that followed it seem to leak ifaddr N> references. ifa_ifwithdstaddr and ifa_ifwithnet both install a N> reference on the ifaddr returned to the caller but ip_output does not N> release it, eventually leading to a panic when the refcount wraps over N> to 0 and the ifaddr is freed while it is still on various lists. N> N> I'm using this patch for now. Thoughts? N> N> Regards, N> Navdeep N> N> N> diff -r 6dfcecd314af sys/netinet/ip_output.c N> --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 N> +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 N> @@ -243,6 +243,7 @@ again: N> ifp = ia->ia_ifp; N> ip->ip_ttl = 1; N> isbroadcast = 1; N> + ifa_free((void *)ia); N> } else if (flags & IP_ROUTETOIF) { N> if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && N> (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { N> @@ -253,6 +254,7 @@ again: N> ifp = ia->ia_ifp; N> ip->ip_ttl = 1; N> isbroadcast = in_broadcast(dst->sin_addr, ifp); N> + ifa_free((void *)ia); N> } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && N> imo != NULL && imo->imo_multicast_ifp != NULL) { N> /* The patch shouldn't use void * casts, but instead specify explicit member: ifa_free(&ia->ia_ifa); Apart from that it, the patch looks entirely correct and plugging a leak. Thanks! -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 10:42:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7252F87C for ; Mon, 23 Jun 2014 10:42:09 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 1B4C42885 for ; Mon, 23 Jun 2014 10:42:08 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s5NAg0Sj057378 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 23 Jun 2014 11:42:01 +0100 (BST) Date: Mon, 23 Jun 2014 11:42:01 +0100 From: Karl Pielorz To: freebsd-net@freebsd.org Subject: Weird Xen networking issue with PV interfaces passing traffic to other PV's... Message-ID: <7D8C9B50517ABCB6EB0984E0@Mail-PC.tdx.co.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 10:42:09 -0000 Hi, I originally posted to freebsd-xen about this (and I've raised a PR) -pr188261 - It's been suggested I should ask in FreeBSD-NET to see if someone can look at this, and suggest how to proceed... In a nutshell - the FreeBSD PV network code won't pass packets properly if you setup two VM's on the same XenServer trying to use one of the VM's as a 'router' (i.e. passing traffic). Swap it for linux - works. Disable PV - works - anyway, the details are in the PR. If anyone has a chance to look at that and suggest what I can do to try and debug this further? - I've done packet captures from the VM's (they show a lot of retransmits) - but I'm not sure how I can go any further in look at this - or what the problem is likely to be (it's been suggested it's checksum related - which '-txcsum' seems to address for clients, but not the router). If some kind soul can have a look at this - and suggest anything? Cheers, -Karl From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 15:21:03 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBFC4D39 for ; Mon, 23 Jun 2014 15:21:03 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2ABC2369 for ; Mon, 23 Jun 2014 15:21:03 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5NFL3SK074522 for ; Mon, 23 Jun 2014 16:21:03 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 184311] [bge] [panic] kernel panic with bge(4) on SunFire X2100 Date: Mon, 23 Jun 2014 15:21:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 15:21:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184311 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhb@FreeBSD.org --- Comment #6 from John Baldwin --- Can you capture a verbose dmesg (boot -v) with the ASF tunable set (so that it works) and in the stock setup (where it breaks)? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 15:52:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F682EE8 for ; Mon, 23 Jun 2014 15:52:33 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 094A92657 for ; Mon, 23 Jun 2014 15:52:33 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EEDEFB98E; Mon, 23 Jun 2014 11:52:31 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Ordering problem in if_detach_internal regarding if_bridge Date: Mon, 23 Jun 2014 11:32:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <53A4527F.90008@citrix.com> In-Reply-To: <53A4527F.90008@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Message-Id: <201406231132.47487.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 23 Jun 2014 11:52:32 -0400 (EDT) Cc: Roger Pau =?iso-8859-15?q?Monn=E9?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 15:52:33 -0000 On Friday, June 20, 2014 11:25:51 am Roger Pau Monn=E9 wrote: > Hello, >=20 > I've stumbled across the following panic when testing Xen netback with=20 > if_bridge: >=20 > Kernel page fault with the following non-sleepable locks held: > exclusive sleep mutex if_bridge (if_bridge) r =3D 0 (0xfffff80006306c18)= =20 locked @ /usr/src/sys/m > KDB: stack backtrace: > X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe0000213490 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 > witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 > trap() at trap+0xc9d/frame 0xfffffe00002136a0 > trap() at trap+0x669/frame 0xfffffe00002138b0 > calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 > --- trap 0xc, rip =3D 0xffffffff8221a0ef, rsp =3D 0xfffffe0000213970, rbp= =3D=20 0xfffffe00002139e0 --- > bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 > ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 > netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000213a80 > ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 > ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame=20 0xfffffe0000213b30 > intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame=20 0xfffffe0000213b70 > db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe0000213bb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 > --- trap 0, rip =3D 0, rsp =3D 0xfffffe0000213cb0, rbp =3D 0 --- >=20 > I've tracked this down to if_detach_internal setting ifp->if_addr to=20 > NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., which=20 > causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tries=20 > to perform IF_LLADDR on an interface that's in the process of being=20 > destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event event= =20 > has not fired yet). >=20 > I have the following naive patch that moves the firing of the event=20 > before if_addr is set to NULL, but I'm not familiar with the ordering=20 > in if_detach_internal, so I'm not sure if this might cause problems in=20 > other parts of the code, could someone familiar with the net stuff=20 > comment on the best way to deal with it? Hmmm, I have no idea if this is ok or not. I do think the route message=20 should go out at the same time as the devctl_notify() call however. My gue= ss=20 is it is actually better to do this earlier so that we allow outside consum= ers to detach from an interface before it is destroyed. I'm not sure if it wou= ld break things, but I would be tempted to move this even earlier right after = it is removed from the global ifnet list but before the taskqueue_drain, etc. =2D-=20 John Baldwin From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 16:41:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A18E262; Mon, 23 Jun 2014 16:41:03 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C17492AD3; Mon, 23 Jun 2014 16:41:02 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Wz3NA-0005NQ-06; Mon, 23 Jun 2014 16:29:20 +0400 Message-ID: <53A8585C.3080000@FreeBSD.org> Date: Mon, 23 Jun 2014 20:39:56 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: John Baldwin , freebsd-net@freebsd.org Subject: Re: Ordering problem in if_detach_internal regarding if_bridge References: <53A4527F.90008@citrix.com> <201406231132.47487.jhb@freebsd.org> In-Reply-To: <201406231132.47487.jhb@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: =?ISO-8859-15?Q?Roger_Pau_Monn=E9?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 16:41:03 -0000 On 23.06.2014 19:32, John Baldwin wrote: > On Friday, June 20, 2014 11:25:51 am Roger Pau Monné wrote: >> Hello, >> >> I've stumbled across the following panic when testing Xen netback with >> if_bridge: >> >> Kernel page fault with the following non-sleepable locks held: >> exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xfffff80006306c18) > locked @ /usr/src/sys/m >> KDB: stack backtrace: >> X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe0000213490 >> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 >> witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 >> trap() at trap+0xc9d/frame 0xfffffe00002136a0 >> trap() at trap+0x669/frame 0xfffffe00002138b0 >> calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 >> --- trap 0xc, rip = 0xffffffff8221a0ef, rsp = 0xfffffe0000213970, rbp = > 0xfffffe00002139e0 --- >> bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 >> ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 >> netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000213a80 >> ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 >> ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame > 0xfffffe0000213b30 >> intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame > 0xfffffe0000213b70 >> db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe0000213bb0 >> fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 >> --- trap 0, rip = 0, rsp = 0xfffffe0000213cb0, rbp = 0 --- >> >> I've tracked this down to if_detach_internal setting ifp->if_addr to >> NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., which >> causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tries >> to perform IF_LLADDR on an interface that's in the process of being >> destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event event >> has not fired yet). >> >> I have the following naive patch that moves the firing of the event >> before if_addr is set to NULL, but I'm not familiar with the ordering >> in if_detach_internal, so I'm not sure if this might cause problems in >> other parts of the code, could someone familiar with the net stuff >> comment on the best way to deal with it? We should notify kernel customers only when we are really taking this interface down and every other subsystem cannot add any new state to the interface. In this patch you're sending notification before taking ifnet down, removing its L3 addresses, routes, and so on. This can easily lead to panic in, for example, BPF subsystem (since BPF state is freed in bpf_ifdetach() handler). Addintionally, this will introduce ifaddr / iface messages reversal for rtsock. It looks like we'd better fix if_bridge (and it is still using mutexes, what a shame!). Can you send me trace with line numbers? > > Hmmm, I have no idea if this is ok or not. I do think the route message > should go out at the same time as the devctl_notify() call however. My guess > is it is actually better to do this earlier so that we allow outside consumers > to detach from an interface before it is destroyed. I'm not sure if it would > break things, but I would be tempted to move this even earlier right after it > is removed from the global ifnet list but before the taskqueue_drain, etc. > From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 16:45:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 313D0355; Mon, 23 Jun 2014 16:45:01 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D1452B74; Mon, 23 Jun 2014 16:45:00 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id a1so6536748wgh.0 for ; Mon, 23 Jun 2014 09:44:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=awJJd701eNs67cWd2qVDQ/kb4bFI/Au6Bw7SiqMJW74=; b=cROcxyaElJkVX0tP9Mkz9t7SJihbFl7NAp3ODZoxhqW6kzWLzwRsJS3R71H89TuqFP Y2FJ2bCiayjnNRYcs4mKSJCZPBwi0XGJ9mFaNy7HQdJSNO3TQWLQ8ibDf/JiBElZvucl /pg/O4cTlzLNPODKDj7a6nCWN5bRHklh5hSomkByD0Qyvb04j+/QnTB4bzpmrN8WiHMD HRQzsVoCSyxGYnLaACBzfQ3DcX8m4QtoE8CNJcTBdgjkeTDGMbR+JbSPZZL984i5Kgf/ PJflAon1Ui6uR/BeCBPLq3ZWo8RCTydHCZ2mbVQe0kY7B1hf8E85nxRTn/j5xXARS6B9 8Yxg== MIME-Version: 1.0 X-Received: by 10.180.149.175 with SMTP id ub15mr27471666wib.53.1403541898246; Mon, 23 Jun 2014 09:44:58 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Mon, 23 Jun 2014 09:44:58 -0700 (PDT) In-Reply-To: <20140623085229.GQ28199@FreeBSD.org> References: <53A48849.8080504@chelsio.com> <20140623085229.GQ28199@FreeBSD.org> Date: Mon, 23 Jun 2014 10:44:58 -0600 X-Google-Sender-Auth: vBzQ-1o3lL9B1mCKnrF8NZ9R8oI Message-ID: Subject: Re: ifaddr refcount problem From: Alan Somers To: Gleb Smirnoff Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 16:45:01 -0000 On Mon, Jun 23, 2014 at 2:52 AM, Gleb Smirnoff wrote: > Navdeep, > > On Fri, Jun 20, 2014 at 12:15:21PM -0700, Navdeep Parhar wrote: > N> Revision 264905 and 266860 that followed it seem to leak ifaddr > N> references. ifa_ifwithdstaddr and ifa_ifwithnet both install a > N> reference on the ifaddr returned to the caller but ip_output does not > N> release it, eventually leading to a panic when the refcount wraps over > N> to 0 and the ifaddr is freed while it is still on various lists. > N> > N> I'm using this patch for now. Thoughts? > N> > N> Regards, > N> Navdeep > N> > N> > N> diff -r 6dfcecd314af sys/netinet/ip_output.c > N> --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 > N> +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 > N> @@ -243,6 +243,7 @@ again: > N> ifp = ia->ia_ifp; > N> ip->ip_ttl = 1; > N> isbroadcast = 1; > N> + ifa_free((void *)ia); > N> } else if (flags & IP_ROUTETOIF) { > N> if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && > N> (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { > N> @@ -253,6 +254,7 @@ again: > N> ifp = ia->ia_ifp; > N> ip->ip_ttl = 1; > N> isbroadcast = in_broadcast(dst->sin_addr, ifp); > N> + ifa_free((void *)ia); > N> } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && > N> imo != NULL && imo->imo_multicast_ifp != NULL) { > N> /* > > The patch shouldn't use void * casts, but instead specify explicit member: > > ifa_free(&ia->ia_ifa); > > Apart from that it, the patch looks entirely correct and plugging a leak. > Thanks! I still don't see how this patch would work without breaking stuff like the statistics collection at line 673 of ip_output.c. If we call ifa_free immediately after choosing our ifp, then ia won't be available at lines 630 or 673, and ip_output will never record statistics, right? -Alan > > -- > Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 16:50:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AA9C558; Mon, 23 Jun 2014 16:50:21 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9B862BD1; Mon, 23 Jun 2014 16:50:20 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Wz3WA-0005Tx-70; Mon, 23 Jun 2014 16:38:38 +0400 Message-ID: <53A85A8D.4090208@FreeBSD.org> Date: Mon, 23 Jun 2014 20:49:17 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: John Baldwin , freebsd-net@freebsd.org Subject: Re: Ordering problem in if_detach_internal regarding if_bridge References: <53A4527F.90008@citrix.com> <201406231132.47487.jhb@freebsd.org> <53A8585C.3080000@FreeBSD.org> In-Reply-To: <53A8585C.3080000@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: =?ISO-8859-15?Q?Roger_Pau_Monn=E9?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 16:50:21 -0000 On 23.06.2014 20:39, Alexander V. Chernikov wrote: > On 23.06.2014 19:32, John Baldwin wrote: >> On Friday, June 20, 2014 11:25:51 am Roger Pau Monné wrote: >>> Hello, >>> >>> I've stumbled across the following panic when testing Xen netback with >>> if_bridge: >>> >>> Kernel page fault with the following non-sleepable locks held: >>> exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xfffff80006306c18) >> locked @ /usr/src/sys/m >>> KDB: stack backtrace: >>> X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe0000213490 >>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 >>> witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 >>> trap() at trap+0xc9d/frame 0xfffffe00002136a0 >>> trap() at trap+0x669/frame 0xfffffe00002138b0 >>> calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 >>> --- trap 0xc, rip = 0xffffffff8221a0ef, rsp = 0xfffffe0000213970, rbp = >> 0xfffffe00002139e0 --- >>> bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 >>> ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 >>> netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000213a80 >>> ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 >>> ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame >> 0xfffffe0000213b30 >>> intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame >> 0xfffffe0000213b70 >>> db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe0000213bb0 >>> fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 >>> --- trap 0, rip = 0, rsp = 0xfffffe0000213cb0, rbp = 0 --- >>> >>> I've tracked this down to if_detach_internal setting ifp->if_addr to >>> NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., which >>> causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tries >>> to perform IF_LLADDR on an interface that's in the process of being >>> destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event event >>> has not fired yet). >>> >>> I have the following naive patch that moves the firing of the event >>> before if_addr is set to NULL, but I'm not familiar with the ordering >>> in if_detach_internal, so I'm not sure if this might cause problems in >>> other parts of the code, could someone familiar with the net stuff >>> comment on the best way to deal with it? > > We should notify kernel customers only when we are really taking this > interface down and every other subsystem cannot add any new state to the > interface. > > In this patch you're sending notification before taking ifnet down, > removing its L3 addresses, routes, and so on. > > This can easily lead to panic in, for example, BPF subsystem (since BPF > state is freed in bpf_ifdetach() handler). > > Addintionally, this will introduce ifaddr / iface messages reversal for > rtsock. Whoops. I misread the patch. It should be OK. > > It looks like we'd better fix if_bridge (and it is still using mutexes, > what a shame!). > > Can you send me trace with line numbers? However, these two still stands. (And I'm wondering how you're getting any traffic on down/dying interface). > >> >> Hmmm, I have no idea if this is ok or not. I do think the route message >> should go out at the same time as the devctl_notify() call however. My guess >> is it is actually better to do this earlier so that we allow outside consumers >> to detach from an interface before it is destroyed. I'm not sure if it would >> break things, but I would be tempted to move this even earlier right after it >> is removed from the global ifnet list but before the taskqueue_drain, etc. >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 17:13:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2DE25CDD; Mon, 23 Jun 2014 17:13:23 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A18E2E69; Mon, 23 Jun 2014 17:13:21 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,531,1400025600"; d="scan'208";a="146235088" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 23 Jun 2014 17:13:12 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Mon, 23 Jun 2014 13:13:12 -0400 Message-ID: <53A86016.5000102@citrix.com> Date: Mon, 23 Jun 2014 19:12:54 +0200 From: =?ISO-8859-15?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , John Baldwin , Subject: Re: Ordering problem in if_detach_internal regarding if_bridge References: <53A4527F.90008@citrix.com> <201406231132.47487.jhb@freebsd.org> <53A8585C.3080000@FreeBSD.org> <53A85A8D.4090208@FreeBSD.org> In-Reply-To: <53A85A8D.4090208@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: 8bit X-DLP: MIA1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 17:13:23 -0000 On 23/06/14 18:49, Alexander V. Chernikov wrote: > On 23.06.2014 20:39, Alexander V. Chernikov wrote: >> On 23.06.2014 19:32, John Baldwin wrote: >>> On Friday, June 20, 2014 11:25:51 am Roger Pau Monné wrote: >>>> Hello, >>>> >>>> I've stumbled across the following panic when testing Xen netback with >>>> if_bridge: >>>> >>>> Kernel page fault with the following non-sleepable locks held: >>>> exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xfffff80006306c18) >>> locked @ /usr/src/sys/m >>>> KDB: stack backtrace: >>>> X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe0000213490 >>>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 >>>> witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 >>>> trap() at trap+0xc9d/frame 0xfffffe00002136a0 >>>> trap() at trap+0x669/frame 0xfffffe00002138b0 >>>> calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 >>>> --- trap 0xc, rip = 0xffffffff8221a0ef, rsp = 0xfffffe0000213970, rbp = >>> 0xfffffe00002139e0 --- >>>> bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 >>>> ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 >>>> netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000213a80 >>>> ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 >>>> ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame >>> 0xfffffe0000213b30 >>>> intr_event_execute_handlers() at intr_event_execute_handlers+0x93/frame >>> 0xfffffe0000213b70 >>>> db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe0000213bb0 >>>> fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 >>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 >>>> --- trap 0, rip = 0, rsp = 0xfffffe0000213cb0, rbp = 0 --- >>>> >>>> I've tracked this down to if_detach_internal setting ifp->if_addr to >>>> NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., which >>>> causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tries >>>> to perform IF_LLADDR on an interface that's in the process of being >>>> destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event event >>>> has not fired yet). >>>> >>>> I have the following naive patch that moves the firing of the event >>>> before if_addr is set to NULL, but I'm not familiar with the ordering >>>> in if_detach_internal, so I'm not sure if this might cause problems in >>>> other parts of the code, could someone familiar with the net stuff >>>> comment on the best way to deal with it? >> >> We should notify kernel customers only when we are really taking this >> interface down and every other subsystem cannot add any new state to the >> interface. >> >> In this patch you're sending notification before taking ifnet down, >> removing its L3 addresses, routes, and so on. >> >> This can easily lead to panic in, for example, BPF subsystem (since BPF >> state is freed in bpf_ifdetach() handler). >> >> Addintionally, this will introduce ifaddr / iface messages reversal for >> rtsock. > Whoops. I misread the patch. > It should be OK. > >> >> It looks like we'd better fix if_bridge (and it is still using mutexes, >> what a shame!). >> >> Can you send me trace with line numbers? > However, these two still stands. > (And I'm wondering how you're getting any traffic on down/dying interface). I'm not getting the traffic from the dying interface, I'm getting the traffic from another interface on the bridge (a physical bce interface), which injects traffic into the bridge, that calls bridge_input, which tries to read ifp->if_addr->ifa_addr from the dying interface, and that leads to the panic. Line numbers: /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2410 (bridge_input) /usr/src/sys/net/if_ethersubr.c:543 (ether_input_internal) /usr/src/sys/net/netisr.c:972 (netisr_dispatch_src) /usr/src/sys/net/if_ethersubr.c:674 (ether_input) /usr/src/sys/dev/bce/if_bce.c:6861 (bce_rx_intr) Roger. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 20:37:45 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C9D7F9E for ; Mon, 23 Jun 2014 20:37:45 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E82692189 for ; Mon, 23 Jun 2014 20:37:44 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5NKbi0i051501 for ; Mon, 23 Jun 2014 21:37:44 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Mon, 23 Jun 2014 20:37:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 20:37:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhb@FreeBSD.org --- Comment #3 from John Baldwin --- Can you provide more details? The em/igb drivers create additional taskqueue threads for each queue, but 'cpuset -x' is only going to pin the interrupt thread associated with that IRQ, not other threads the em driver may create. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 23 22:54:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E96CD9C; Mon, 23 Jun 2014 22:54:29 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 99B7F2E7F; Mon, 23 Jun 2014 22:54:29 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id o8so6818657qcw.31 for ; Mon, 23 Jun 2014 15:54:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=l6HmwZbJoMZE7fpg0IS5vku24k4DXa6+XjWrkUqzE10=; b=IBJNsS09Abr/+c8Ca46jjxjM6v04addDOwLtwO9q6xaV6tPw12/F6eUR/YzrJcKY0X xrw63dBZSsk5AQMBq++IF9hwEZYcenfBzWh5PwCr7X2kxkcWwbWbPq7exKxwJkRUKxdl ndnhvhOdSf+5zq8KUcItvgAUqlHPp8H9T5VBPImYggi20EQ5wj+v6fmt8p/hjbNZd9N1 NMh1c89IDu47r6KIsjqsvu1MWj2f97sMja3Npdzc8i5Dr8hefAjdxeSc+tieP0bwQVg5 RQvj1iaa1YtRyDNVeZOO+v9y9r2BInkSfLob5zgXkmqSbOdt1wlMmuKtwwr86dnP88Yb +vpQ== MIME-Version: 1.0 X-Received: by 10.140.80.49 with SMTP id b46mr32208219qgd.102.1403564068836; Mon, 23 Jun 2014 15:54:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Mon, 23 Jun 2014 15:54:28 -0700 (PDT) In-Reply-To: References: Date: Mon, 23 Jun 2014 15:54:28 -0700 X-Google-Sender-Auth: ypWfYoEZJH30MeamQSbVWAOi9ms Message-ID: Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Adrian Chadd To: araujo@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 22:54:30 -0000 Hi, No, don't introduce out of order behaviour. Ever. You may not think it's a problem for TCP, but UDP things and VPN things will start getting very angry. There are VPN configurations out there that will drop the VPN if frames are out of order. The ixgbe driver is setting the flowid to the msix queue ID, rather than a 32 bit unique flow id hash value for the flow. That makes it hard to do traffic distribution where the flowid is available. There's an lagg option to re-hash the mbuf rather than rely on the flowid for outbound port choice - have you looked at using that? Did that make any difference? -a From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 02:24:44 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA2B3138 for ; Tue, 24 Jun 2014 02:24:44 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D10A42069 for ; Tue, 24 Jun 2014 02:24:44 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5O2Oiem047530 for ; Tue, 24 Jun 2014 03:24:44 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 02:24:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gondim@bsdinfo.com.br X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 02:24:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #4 from gondim@bsdinfo.com.br --- (In reply to John Baldwin from comment #3) > Can you provide more details? The em/igb drivers create additional > taskqueue threads for each queue, but 'cpuset -x' is only going to pin the > interrupt thread associated with that IRQ, not other threads the em driver > may create. Hi John, I thought the driver em(4) could do cpu affinity but by the way I was wrong. As I understand it, only drivers as the igb(4), ixgbe(4) having msi-x that would work with the cpu affinity. Would that be? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 02:40:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40B56359; Tue, 24 Jun 2014 02:40:55 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E7A0213D; Tue, 24 Jun 2014 02:40:54 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n3so5083417wiv.8 for ; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=CJnWdBKuKehgHB3KCFM7ocJ2UdlAiuIGAFml0TdUVis=; b=gHRUOSDQ8KVqFucVSjyeNiuOscfqEl8Wcf7rTC9f0QoTpQtn90RZlOI1ThlyLlXCDD 3eA1BCHlhki1ehUghZdnQty5npDMEpnRGV2Z1d4+nMcfCBvnSO4s5nal20ffX5ToKmG2 knpVCBSwibrDciMQ/rIcuN1JZsYfpZiPbDGOQzMVo6fxaHLvedQV6UuGx7s3OQu/OAy9 wrxQtIXYXVvGJ2p1iuIpE8rCgFaygOXcYYLHnghJp2dcbgWRyLCLrUbEln7PbY7q+BiL rowqpiUAH7uaCHfJmljzLMrDAbW8fKDusC1qLJfi7XwJlH9kkLNUCNj+sM9wSCPKwaJI X+xg== MIME-Version: 1.0 X-Received: by 10.180.19.37 with SMTP id b5mr30371361wie.16.1403577652768; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) Reply-To: araujo@FreeBSD.org In-Reply-To: References: Date: Tue, 24 Jun 2014 10:40:52 +0800 Message-ID: Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Marcelo Araujo To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 02:40:55 -0000 2014-06-24 6:54 GMT+08:00 Adrian Chadd : > Hi, > > No, don't introduce out of order behaviour. Ever. Yes, it has out of order behavior; with my patch much less. I upload two pcap files and you can see by yourself, if you don't believe in what I'm talking about. Test done using: "iperf -s" and "iperf -c -i 1 -t 10". 1) Don't change the number of packets(default round robin behavior). http://people.freebsd.org/~araujo/lagg/lagg-nop.cap 8 out of order packets. Several SACKs. 2) Set the number of packets to 50. http://people.freebsd.org/~araujo/lagg/lagg.cap 0 out of order packets. Less SACKs. > You may not think > it's a problem for TCP, but UDP things and VPN things will start > getting very angry. There are VPN configurations out there that will > drop the VPN if frames are out of order. > I'm not thinking that will be a problem for TCP, but, in somehow it will be, less throughput as I showed before, and less SACK. About the VPN, please, tell me which softwares, and let me know where I can get a sample to make a testbed. However to be very honest, I don't believe anyone here when change something at network protocols will make this extensive testbed. It is almost impossible to predict what software it will works or not, and I don't believe anyone here has all these stuff in hands. > > The ixgbe driver is setting the flowid to the msix queue ID, rather > than a 32 bit unique flow id hash value for the flow. That makes it > hard to do traffic distribution where the flowid is available. > Thanks for the explanation. > > There's an lagg option to re-hash the mbuf rather than rely on the > flowid for outbound port choice - have you looked at using that? Did > that make any difference? > Yes, I set to 0 the net.link.lagg.0.use _flowid, it make a little difference to the default round robin implementation, but yet I can't reach more than 5 Gbit/s. With my patch and set the packets to 50, it improved a bit too. So, thank you so much for all review, I don't know if you have time and a testbed to make a real test, as I'm doing. I would be happy if you or more people could make tests on that patch. Also, I have only ixgbe(4) to make tests, would appreciate if this patch could be tested with other NICs too. Best Regards, -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 08:56:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D550D55D for ; Tue, 24 Jun 2014 08:56:50 +0000 (UTC) Received: from nm41.bullet.mail.ne1.yahoo.com (nm41.bullet.mail.ne1.yahoo.com [98.138.120.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8D5A02F83 for ; Tue, 24 Jun 2014 08:56:49 +0000 (UTC) Received: from [127.0.0.1] by nm41.bullet.mail.ne1.yahoo.com with NNFMP; 24 Jun 2014 08:56:42 -0000 Received: from [98.138.226.177] by nm41.bullet.mail.ne1.yahoo.com with NNFMP; 24 Jun 2014 08:53:42 -0000 Received: from [66.196.81.174] by tm12.bullet.mail.ne1.yahoo.com with NNFMP; 24 Jun 2014 08:53:21 -0000 Received: from [98.139.212.224] by tm20.bullet.mail.bf1.yahoo.com with NNFMP; 24 Jun 2014 08:53:21 -0000 Received: from [127.0.0.1] by omp1033.mail.bf1.yahoo.com with NNFMP; 24 Jun 2014 08:53:21 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 297825.95222.bm@omp1033.mail.bf1.yahoo.com Received: (qmail 87466 invoked by uid 60001); 24 Jun 2014 08:53:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1403600001; bh=9wIe3YsxzjYdMc2jGWtGsXE8MzX6B+HMdqbWI08O4YY=; h=References:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=GJli6KdzjIUZgBVJiNwkIf2IJ8ZSbiSdwLtZwnX8AP2rSL+d3QlMW6D7PG1Gt0UzMnobABzhhaMuUnwmgsWdzt9xwcqNamUjEtMDjtbjwdyQx2c1xn8E68geUAhGwbFtrDoJeQumNrwXXFyAR0sUTJm2vlpkT5qsojoNO/cJUjc= X-YMail-OSG: OssDxbQVM1ltf1N.SgrBzzYhRFfDBigVdHSiOaVXzDht_Ux EbQ4Lg1_GNzgkkzhs4stkzWywO.hkHh9bPa0RrE_.rlH.CGE9msdBGWjqsls KcBt736WEeHbvJFcmTUAMcFgUeqROEQVWT45aoEDDqWWlF8Snv8TvUWsySE9 VeRH9AABaXmeakukzJVFNh5GnipHSwwhS5q4SkjDRzIum.Pi3t1nz4HEx4qd F_vynWBPukkSQUcKTgffFz3fMwt9o872ptbI1P2etL.U2zBOhtXUSKtzOM0B Dz7G.NUwjmaYpCt8NsQP.V1oLrTp0TnTX1lTKb2_Ix0ma4x18Zzg1pGV2r4p 9_gnNWukRuVCX.WtWS22ngnz2_UK1pj2R.iVeJotaVF9vLnNCsaXyj8ydMmi WoOuGBqhMk4WhtPkmK5nG0ep3_DewOdm_en66VT21LiSiDfBKWUfPhTCTOrh 2vw4_7JoBSIKZU3AgBLl_1q2ypew4Y9UbkU1O0YEyXBOoBzujiOAXah9Uyup D2o2hsGdDNb4t2F9SWgdJG49sAxa4han0ZZXpq8yqV1vJ Received: from [89.122.203.240] by web162506.mail.bf1.yahoo.com via HTTP; Tue, 24 Jun 2014 01:53:21 PDT X-Rocket-MIMEInfo: 002.001, SGksCgpJIGhhdmUgYSB2bXdhcmUgdmlydHVhbCBzZXJ2ZXIgcnVubmluZyBGcmVlQlNEIDEwLjAgU1RBQkxFClRoZSB2aXJ0dWFsIHNlcnZlciBoYXMgMTAwbWJwcyBwb3J0LgoKSXQgaXMgcnVubmluZyBhIFRvciByb3V0ZXIsIGNvbnN1bWluZyBhbiBhdmVyYWdlIG9mIDYtNyBUQiBvZiBtb250aGx5IHRyYWZmaWMuIEl0cyB0aGUgb25seSBwdXJwb3NlIG9mIHRoZSBzZXJ2ZXIuCgpMYXN0IG5pZ2h0IGl0IHdlbnQgZG93biwgYW5kIHZtd2FyZSBjb25zb2xlIGxvZyB3YXMgc2F5aW5nOgpbem9uZTogTWJ1Zl8BMAEBAQE- X-RocketYMMF: stere01 X-Mailer: YahooMailWebService/0.8.191.1 References: Message-ID: <1403600001.74170.YahooMailNeo@web162506.mail.bf1.yahoo.com> Date: Tue, 24 Jun 2014 01:53:21 -0700 From: Stefan Stere Reply-To: : ; Subject: [zone: Mbuf_cluster] kern.ipc.nmbclusters limit reached in Virtual machine causes downtime To: "freebsd-net@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 08:56:50 -0000 Hi, I have a vmware virtual server running FreeBSD 10.0 STABLE The virtual server has 100mbps port. It is running a Tor router, consuming an average of 6-7 TB of monthly traffic. Its the only purpose of the server. Last night it went down, and vmware console log was saying: [zone: Mbuf_cluster] kern.ipc.nmbclusters limit reached I don't know what this means - the traffic of the server is unlimited and nothing is capped in any way. What can I do to fix this? I have read on freebsd wiki that I might need to add some lines to sysctl ? can you please confirm? Thank you in advance. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 09:09:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B77B9A36; Tue, 24 Jun 2014 09:09:02 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 502AB20A7; Tue, 24 Jun 2014 09:09:01 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s5O98m6t046655 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 24 Jun 2014 13:08:48 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s5O98lQK046654; Tue, 24 Jun 2014 13:08:47 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 24 Jun 2014 13:08:47 +0400 From: Gleb Smirnoff To: Alan Somers Subject: Re: ifaddr refcount problem Message-ID: <20140624090847.GB28199@glebius.int.ru> References: <53A48849.8080504@chelsio.com> <20140623085229.GQ28199@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="jL2BoiuKMElzg3CS" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 09:09:02 -0000 --jL2BoiuKMElzg3CS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jun 23, 2014 at 10:44:58AM -0600, Alan Somers wrote: A> > On Fri, Jun 20, 2014 at 12:15:21PM -0700, Navdeep Parhar wrote: A> > N> Revision 264905 and 266860 that followed it seem to leak ifaddr A> > N> references. ifa_ifwithdstaddr and ifa_ifwithnet both install a A> > N> reference on the ifaddr returned to the caller but ip_output does not A> > N> release it, eventually leading to a panic when the refcount wraps over A> > N> to 0 and the ifaddr is freed while it is still on various lists. A> > N> A> > N> I'm using this patch for now. Thoughts? A> > N> A> > N> Regards, A> > N> Navdeep A> > N> A> > N> A> > N> diff -r 6dfcecd314af sys/netinet/ip_output.c A> > N> --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 A> > N> +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 A> > N> @@ -243,6 +243,7 @@ again: A> > N> ifp = ia->ia_ifp; A> > N> ip->ip_ttl = 1; A> > N> isbroadcast = 1; A> > N> + ifa_free((void *)ia); A> > N> } else if (flags & IP_ROUTETOIF) { A> > N> if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && A> > N> (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { A> > N> @@ -253,6 +254,7 @@ again: A> > N> ifp = ia->ia_ifp; A> > N> ip->ip_ttl = 1; A> > N> isbroadcast = in_broadcast(dst->sin_addr, ifp); A> > N> + ifa_free((void *)ia); A> > N> } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && A> > N> imo != NULL && imo->imo_multicast_ifp != NULL) { A> > N> /* A> > A> > The patch shouldn't use void * casts, but instead specify explicit member: A> > A> > ifa_free(&ia->ia_ifa); A> > A> > Apart from that it, the patch looks entirely correct and plugging a leak. A> > Thanks! A> A> I still don't see how this patch would work without breaking stuff A> like the statistics collection at line 673 of ip_output.c. If we call A> ifa_free immediately after choosing our ifp, then ia won't be A> available at lines 630 or 673, and ip_output will never record A> statistics, right? You are right, thanks. In case of IP_SENDONES/IP_ROUTETOIF we should hold the reference to ia throughout the function and free it at the end. Suggested patch, not tested. -- Totus tuus, Glebius. --jL2BoiuKMElzg3CS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="ip_output.diff" Index: sys/netinet/ip_output.c =================================================================== --- sys/netinet/ip_output.c (revision 267536) +++ sys/netinet/ip_output.c (working copy) @@ -552,8 +552,11 @@ sendit: #endif error = netisr_queue(NETISR_IP, m); goto done; - } else + } else { + if (flags & (IP_SENDONES | IP_ROUTETOIF)) + ifa_free(&ia->ia_ifa); goto again; /* Redo the routing table lookup. */ + } } /* See if local, if yes, send it to netisr with IP_FASTFWD_OURS. */ @@ -582,6 +585,8 @@ sendit: m->m_flags |= M_SKIP_FIREWALL; m->m_flags &= ~M_IP_NEXTHOP; m_tag_delete(m, fwd_tag); + if (flags & (IP_SENDONES | IP_ROUTETOIF)) + ifa_free(&ia->ia_ifa); goto again; } @@ -694,6 +699,8 @@ passout: done: if (ro == &iproute) RO_RTFREE(ro); + if (flags & (IP_SENDONES | IP_ROUTETOIF)) + ifa_free(&ia->ia_ifa); return (error); bad: m_freem(m); --jL2BoiuKMElzg3CS-- From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 14:43:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A27A9FC9; Tue, 24 Jun 2014 14:43:47 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 094AA2193; Tue, 24 Jun 2014 14:43:46 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n3so735962wiv.8 for ; Tue, 24 Jun 2014 07:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=1/lL62HjsZ3uJFQ6Hh+7zWVJ79PcYeC+I/Rko4Bywjo=; b=l4KCZ5MOOrnxykmgqzJ+TF1lBQ4jshFItoQD2jj4aeqhZacMXLriE2bb1bu9B5zPko k+Z1uyHd5JoI0qYMg2FZY60CLRCb75WwZn1yMC0RHW0833ZhQTm3GVB0OsTwiZqC70hX fRlJQvOyd1+vNq52r9e9cJuj33rGMvO83al2HLW9t61wdIqRfw8xP9OBJFPf6r5L4XrJ /zHfd5sQyVGLRNeq7IzuBUONqjAHV5TARcYVkpVMQt96gUxavgtRjnO3lbiD/HMfe4WO z786Pj+9wfA2qhrNLWsBXel2UjZDcn+rUU1bQ+yvONhysIrQ9UutShXdHcvwf1W/61Cd M6Ig== MIME-Version: 1.0 X-Received: by 10.180.20.206 with SMTP id p14mr3142619wie.26.1403621020873; Tue, 24 Jun 2014 07:43:40 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Tue, 24 Jun 2014 07:43:40 -0700 (PDT) In-Reply-To: <20140624090847.GB28199@glebius.int.ru> References: <53A48849.8080504@chelsio.com> <20140623085229.GQ28199@FreeBSD.org> <20140624090847.GB28199@glebius.int.ru> Date: Tue, 24 Jun 2014 08:43:40 -0600 X-Google-Sender-Auth: 4MuXNWXLCIJXMq1Nl6jq9gZCA7s Message-ID: Subject: Re: ifaddr refcount problem From: Alan Somers To: Gleb Smirnoff Content-Type: multipart/mixed; boundary=bcaec53d55f5594ee104fc95fcf8 Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 14:43:47 -0000 --bcaec53d55f5594ee104fc95fcf8 Content-Type: text/plain; charset=UTF-8 On Tue, Jun 24, 2014 at 3:08 AM, Gleb Smirnoff wrote: > On Mon, Jun 23, 2014 at 10:44:58AM -0600, Alan Somers wrote: > A> > On Fri, Jun 20, 2014 at 12:15:21PM -0700, Navdeep Parhar wrote: > A> > N> Revision 264905 and 266860 that followed it seem to leak ifaddr > A> > N> references. ifa_ifwithdstaddr and ifa_ifwithnet both install a > A> > N> reference on the ifaddr returned to the caller but ip_output does not > A> > N> release it, eventually leading to a panic when the refcount wraps over > A> > N> to 0 and the ifaddr is freed while it is still on various lists. > A> > N> > A> > N> I'm using this patch for now. Thoughts? > A> > N> > A> > N> Regards, > A> > N> Navdeep > A> > N> > A> > N> > A> > N> diff -r 6dfcecd314af sys/netinet/ip_output.c > A> > N> --- a/sys/netinet/ip_output.c Fri Jun 20 10:33:22 2014 -0700 > A> > N> +++ b/sys/netinet/ip_output.c Fri Jun 20 12:07:12 2014 -0700 > A> > N> @@ -243,6 +243,7 @@ again: > A> > N> ifp = ia->ia_ifp; > A> > N> ip->ip_ttl = 1; > A> > N> isbroadcast = 1; > A> > N> + ifa_free((void *)ia); > A> > N> } else if (flags & IP_ROUTETOIF) { > A> > N> if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == NULL && > A> > N> (ia = ifatoia(ifa_ifwithnet(sintosa(dst), 0))) == NULL) { > A> > N> @@ -253,6 +254,7 @@ again: > A> > N> ifp = ia->ia_ifp; > A> > N> ip->ip_ttl = 1; > A> > N> isbroadcast = in_broadcast(dst->sin_addr, ifp); > A> > N> + ifa_free((void *)ia); > A> > N> } else if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr)) && > A> > N> imo != NULL && imo->imo_multicast_ifp != NULL) { > A> > N> /* > A> > > A> > The patch shouldn't use void * casts, but instead specify explicit member: > A> > > A> > ifa_free(&ia->ia_ifa); > A> > > A> > Apart from that it, the patch looks entirely correct and plugging a leak. > A> > Thanks! > A> > A> I still don't see how this patch would work without breaking stuff > A> like the statistics collection at line 673 of ip_output.c. If we call > A> ifa_free immediately after choosing our ifp, then ia won't be > A> available at lines 630 or 673, and ip_output will never record > A> statistics, right? > > You are right, thanks. > > In case of IP_SENDONES/IP_ROUTETOIF we should hold the reference to ia > throughout the function and free it at the end. > > Suggested patch, not tested. That looks better. But I think there is one more possibility for a leak. For multicast packets, IFP_TO_IA at line 263 will call ifa_ref(), unless the the interface has no address assigned. How about this patch instead? Also, not tested. > > -- > Totus tuus, Glebius. --bcaec53d55f5594ee104fc95fcf8 Content-Type: text/plain; charset=US-ASCII; name="ip_output_v3.diff" Content-Disposition: attachment; filename="ip_output_v3.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hwtbpicn1 SW5kZXg6IHN5cy9uZXRpbmV0L2lwX291dHB1dC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9uZXRpbmV0 L2lwX291dHB1dC5jCShyZXZpc2lvbiAyNjc4MDEpCisrKyBzeXMvbmV0aW5ldC9pcF9vdXRwdXQu Ywkod29ya2luZyBjb3B5KQpAQCAtMTM2LDYgKzEzNiw3IEBACiAJc3RydWN0IHJ0ZW50cnkgKnJ0 ZTsJLyogY2FjaGUgZm9yIHJvLT5yb19ydCAqLwogCXN0cnVjdCBpbl9hZGRyIG9kc3Q7CiAJc3Ry dWN0IG1fdGFnICpmd2RfdGFnID0gTlVMTDsKKwlpbnQgaGF2ZV9pYV9yZWYgPSAwOwogI2lmZGVm IElQU0VDCiAJaW50IG5vX3JvdXRlX2J1dF9jaGVja19zcGQgPSAwOwogI2VuZGlmCkBAIC0yMzgs NiArMjM5LDcgQEAKIAkJCWVycm9yID0gRU5FVFVOUkVBQ0g7CiAJCQlnb3RvIGJhZDsKIAkJfQor CQloYXZlX2lhX3JlZiA9IDE7CiAJCWlwLT5pcF9kc3Quc19hZGRyID0gSU5BRERSX0JST0FEQ0FT VDsKIAkJZHN0LT5zaW5fYWRkciA9IGlwLT5pcF9kc3Q7CiAJCWlmcCA9IGlhLT5pYV9pZnA7CkBA IC0yNTAsNiArMjUyLDcgQEAKIAkJCWVycm9yID0gRU5FVFVOUkVBQ0g7CiAJCQlnb3RvIGJhZDsK IAkJfQorCQloYXZlX2lhX3JlZiA9IDE7CiAJCWlmcCA9IGlhLT5pYV9pZnA7CiAJCWlwLT5pcF90 dGwgPSAxOwogCQlpc2Jyb2FkY2FzdCA9IGluX2Jyb2FkY2FzdChkc3QtPnNpbl9hZGRyLCBpZnAp OwpAQCAtMjYxLDYgKzI2NCw3IEBACiAJCSAqLwogCQlpZnAgPSBpbW8tPmltb19tdWx0aWNhc3Rf aWZwOwogCQlJRlBfVE9fSUEoaWZwLCBpYSk7CisJCWhhdmVfaWFfcmVmID0gMTsKIAkJaXNicm9h ZGNhc3QgPSAwOwkvKiBmb29sIGdjYyAqLwogCX0gZWxzZSB7CiAJCS8qCkBAIC01NTIsOCArNTU2 LDExIEBACiAjZW5kaWYKIAkJCWVycm9yID0gbmV0aXNyX3F1ZXVlKE5FVElTUl9JUCwgbSk7CiAJ CQlnb3RvIGRvbmU7Ci0JCX0gZWxzZQorCQl9IGVsc2UgeworCQkJaWYgKGhhdmVfaWFfcmVmICYm IGlhKQorCQkJCWlmYV9mcmVlKCZpYS0+aWZhKTsKIAkJCWdvdG8gYWdhaW47CS8qIFJlZG8gdGhl IHJvdXRpbmcgdGFibGUgbG9va3VwLiAqLworCQl9CiAJfQogCiAJLyogU2VlIGlmIGxvY2FsLCBp ZiB5ZXMsIHNlbmQgaXQgdG8gbmV0aXNyIHdpdGggSVBfRkFTVEZXRF9PVVJTLiAqLwpAQCAtNTgy LDYgKzU4OSw4IEBACiAJCW0tPm1fZmxhZ3MgfD0gTV9TS0lQX0ZJUkVXQUxMOwogCQltLT5tX2Zs YWdzICY9IH5NX0lQX05FWFRIT1A7CiAJCW1fdGFnX2RlbGV0ZShtLCBmd2RfdGFnKTsKKwkJaWYg KGhhdmVfaWFfcmVmICYmIGlhKQorCQkJaWZhX2ZyZWUoJmlhLT5pZmEpOwogCQlnb3RvIGFnYWlu OwogCX0KIApAQCAtNjk0LDYgKzcwMyw4IEBACiBkb25lOgogCWlmIChybyA9PSAmaXByb3V0ZSkK IAkJUk9fUlRGUkVFKHJvKTsKKwlpZiAoaGF2ZV9pYV9yZWYgJiYgaWEpCisJCWlmYV9mcmVlKCZp YS0+aWZhKTsKIAlyZXR1cm4gKGVycm9yKTsKIGJhZDoKIAltX2ZyZWVtKG0pOwo= --bcaec53d55f5594ee104fc95fcf8-- From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 15:40:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CB391FB for ; Tue, 24 Jun 2014 15:40:21 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D788D27BC for ; Tue, 24 Jun 2014 15:40:20 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id u56so576446wes.22 for ; Tue, 24 Jun 2014 08:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Sll2UzaL97CeN7f/wB94NdWa3+FJRdFv0uyH8NVO6p4=; b=SAQtjdU8tjLchrwiCpRVvf53d/4P2MQOehhYQ7fR0Pbu8xnNdEZbocW7aHOzmpT0ya 7GK0lV+13ptdnOoYKadV7hcP9AVRXofO4wF3TuotNIiNyJadQUKpGRDnaa0lokh+zOCE cwnh76Cw5rhE0D9FiqyRAVBUUkvpzHBtGJgGGDxZfAoO8JjrKxrhZnT2YS5C4SHrEsV+ JyzPBmTDYK+Hvb210qRN8sGo/rebId3zozdGun5EwQqQRB9zoe6Nk7xCwZyvSUCRdDHv cd0PNus+Hp201LX3uuV6DVLcBnODDy69ym3Aoedylgv3AqJG3T7XPMTjL+JDfu9R5O5x AMaA== MIME-Version: 1.0 X-Received: by 10.180.149.175 with SMTP id ub15mr3258567wib.53.1403624415605; Tue, 24 Jun 2014 08:40:15 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Tue, 24 Jun 2014 08:40:15 -0700 (PDT) In-Reply-To: <1403600001.74170.YahooMailNeo@web162506.mail.bf1.yahoo.com> References: <1403600001.74170.YahooMailNeo@web162506.mail.bf1.yahoo.com> Date: Tue, 24 Jun 2014 09:40:15 -0600 X-Google-Sender-Auth: cfYs_iQ8kXuaopTLrFBBKv9XorQ Message-ID: Subject: Re: [zone: Mbuf_cluster] kern.ipc.nmbclusters limit reached in Virtual machine causes downtime From: Alan Somers To: Stefan Stere Content-Type: multipart/mixed; boundary=001a11c38118b10b7204fc96c60f Cc: "freebsd-net@freebsd.org" , Alon Ronen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 15:40:21 -0000 --001a11c38118b10b7204fc96c60f Content-Type: text/plain; charset=UTF-8 On Tue, Jun 24, 2014 at 2:53 AM, Stefan Stere wrote: > Hi, > > I have a vmware virtual server running FreeBSD 10.0 STABLE > The virtual server has 100mbps port. > > It is running a Tor router, consuming an average of 6-7 TB of monthly traffic. Its the only purpose of the server. > > Last night it went down, and vmware console log was saying: > [zone: Mbuf_cluster] kern.ipc.nmbclusters limit reached > > > I don't know what this means - the traffic of the server is unlimited and nothing is capped in any way. What can I do to fix this? I have read on freebsd wiki that I might need to add some lines to sysctl ? can you please confirm? Thank you in advance. This might be related to a problem that Alon Ronen discovered. The kernel can leak mbufs when experiencing memory pressure, if you're using SOCK_DGRAM or SOCK_SEQPACKET Unix domain sockets (or even SOCK_STREAM if sending ancillary data). You could try the attached patch that Alon and I are working on. Even if the patch doesn't fix your problem, it would be interesting to see the output of "vmstat -z". -Alan --001a11c38118b10b7204fc96c60f Content-Type: text/plain; charset=US-ASCII; name="uipc_send.diff" Content-Disposition: attachment; filename="uipc_send.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hwtdorqd0 SW5kZXg6IHN5cy9rZXJuL3VpcGNfdXNycmVxLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2tlcm4vdWlw Y191c3JyZXEuYwkocmV2aXNpb24gMjY3ODE4KQorKysgc3lzL2tlcm4vdWlwY191c3JyZXEuYwko d29ya2luZyBjb3B5KQpAQCAtOTcwLDEwICs5NzAsMTUgQEAKIAkJY2FzZSBTT0NLX1NUUkVBTToK IAkJCWlmIChjb250cm9sICE9IE5VTEwpIHsKIAkJCQlpZiAoc2JhcHBlbmRjb250cm9sX2xvY2tl ZCgmc28yLT5zb19yY3YsIG0sCi0JCQkJICAgIGNvbnRyb2wpKQorCQkJCSAgICBjb250cm9sKSkg ewogCQkJCQljb250cm9sID0gTlVMTDsKLQkJCX0gZWxzZQorCQkJCQltID0gTlVMTDsKKwkJCQl9 IGVsc2UKKwkJCQkJZXJyb3IgPSBFTk9CVUZTOworCQkJfSBlbHNlIHsKIAkJCQlzYmFwcGVuZF9s b2NrZWQoJnNvMi0+c29fcmN2LCBtKTsKKwkJCQltID0gTlVMTDsKKwkJCX0KIAkJCWJyZWFrOwog CiAJCWNhc2UgU09DS19TRVFQQUNLRVQ6IHsKQEAgLTk4Nyw4ICs5OTIsMTEgQEAKIAkJCSAqIGxl dmVsIHVwIHRoZSBzdGFjay4KIAkJCSAqLwogCQkJaWYgKHNiYXBwZW5kYWRkcl9ub3NwYWNlY2hl Y2tfbG9ja2VkKCZzbzItPnNvX3JjdiwKLQkJCQlmcm9tLCBtLCBjb250cm9sKSkKKwkJCQlmcm9t LCBtLCBjb250cm9sKSkgewogCQkJCWNvbnRyb2wgPSBOVUxMOworCQkJCW0gPSBOVUxMOworCQkJ fSBlbHNlCisJCQkJZXJyb3IgPSBFTk9CVUZTOwogCQkJYnJlYWs7CiAJCQl9CiAJCX0KQEAgLTEw MDksNyArMTAxNyw2IEBACiAJCQlzby0+c29fc25kLnNiX2ZsYWdzIHw9IFNCX1NUT1A7CiAJCVNP Q0tCVUZfVU5MT0NLKCZzby0+c29fc25kKTsKIAkJVU5QX1BDQl9VTkxPQ0sodW5wMik7Ci0JCW0g PSBOVUxMOwogCQlicmVhazsKIAogCWRlZmF1bHQ6Cg== --001a11c38118b10b7204fc96c60f-- From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 16:46:15 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A083AF99 for ; Tue, 24 Jun 2014 16:46:15 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 879582FBA for ; Tue, 24 Jun 2014 16:46:15 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OGkFnx009120 for ; Tue, 24 Jun 2014 17:46:15 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 181657] [bpf] [patch] BPF_COP/BPF_COPX instruction reservation (sync with NetBSD) Date: Tue, 24 Jun 2014 16:46:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 1.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: brueffer@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: adrian@freebsd.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 16:46:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=181657 Christian Brueffer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |brueffer@FreeBSD.org Assignee|freebsd-net@FreeBSD.org |adrian@freebsd.org --- Comment #2 from Christian Brueffer --- Adrian volunteered to handle this. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 17:08:02 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C003735F for ; Tue, 24 Jun 2014 17:08:02 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A77AD21E6 for ; Tue, 24 Jun 2014 17:08:02 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OH82DZ086934 for ; Tue, 24 Jun 2014 18:08:02 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 17:08:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 17:08:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #5 from John Baldwin --- Well, cpuset -x forces affinity for the ithread even if the driver did not request it. I do believe that em does not request specific affinity on its own. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 17:50:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 330E5FFA for ; Tue, 24 Jun 2014 17:50:49 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A2292671 for ; Tue, 24 Jun 2014 17:50:49 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OHomSI023605 for ; Tue, 24 Jun 2014 18:50:48 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 17:50:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gondim@bsdinfo.com.br X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 17:50:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #6 from gondim@bsdinfo.com.br --- So it's not a bug, it's a feature of the driver. Can we close this bug then. Thanks and best regards -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 18:43:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5D72FA2 for ; Tue, 24 Jun 2014 18:43:50 +0000 (UTC) Received: from mp1-smtp-5.eutelia.it (mp1-smtp-5.eutelia.it [62.94.10.165]) by mx1.freebsd.org (Postfix) with ESMTP id 5AA9F2B6B for ; Tue, 24 Jun 2014 18:43:50 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-5.eutelia.it (Eutelia) with ESMTP id 437CC1747C7 for ; Tue, 24 Jun 2014 20:43:42 +0200 (CEST) Received: from soth.ventu (adsl-ull-205-226.41-151.net24.it [151.41.226.205]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s5OIhblp043607 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Tue, 24 Jun 2014 20:43:38 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-205-226.41-151.net24.it [151.41.226.205] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s5OIhSgF051149 for ; Tue, 24 Jun 2014 20:43:28 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53A9C6D0.3090900@netfence.it> Date: Tue, 24 Jun 2014 20:43:28 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: MTU not regrowing? Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Tue, 24 Jun 2014 20:43:38 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 18:43:50 -0000 Hello. Today I experienced something weird (at least for me) on a 8.4 system: _ the system had vlan3 interface, with default MTU (1500 bytes); _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" would yield "frag needed and DF set" (forgive me if the message is not exact, I don't have it anymore); _ to make some tests I reduced MTU size with "ifconfig vlan3 mtu 500"; _ now, of course, "ping -D -s 400 somehost" would work, but "ping -D -s 500 somehost" would yield "frag needed and DF set"; _ then I raised MTU again with "ifconfig vlan3 mtu 1500" (notice ifconfig would actually report this as "mtu 1500" was shown); _ however the results were as before, i.e. "ping -D -s 400 somehost" would work, but "ping -D -s 500 somehost" would yield "frag needed and DF set"; _ no way I could ping with a packet bigger than 500 bytes until I rebooted. Is this expected behaviour? Any way to get around this? bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 18:52:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1D701C3 for ; Tue, 24 Jun 2014 18:52:27 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72FBA2C49 for ; Tue, 24 Jun 2014 18:52:27 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id i8so697976qcq.20 for ; Tue, 24 Jun 2014 11:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=joC0sVijymWPBhdukAfCfe/ODDFtqC+lx5JoCAifDqs=; b=HeO7xBmkSMYMAUJLM8K4EFQTnW8IfoNebltZQGPguN8FkfuaXplyvVm6M7Kf/Pr6bp 4UfkrvcH+Jwk76HZwYa8STl3H5TebyFZOYOGikMJBsQRt8Sl68izPCf5iufdruSlonjG eE5GmlDQFtePdffLvXZPzN8ZX2H4/FgEMe33uLM09BAfwjsGIcGcB4cyVJ5rdhwjZ75z +31Jp+iWKE+oD5W+p2MmfD52+Hd/6JChpKGr26bujq1dh9e78aXhtrbaCfDkJPCIIpLn j+eeSURoSD8PaCd18SWuneWa5qOcBi/A0JsHm29QREHwAZVg5CkokYu/KDcRxMxN//Mv i7Hg== X-Received: by 10.224.29.201 with SMTP id r9mr4683075qac.25.1403635946572; Tue, 24 Jun 2014 11:52:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Tue, 24 Jun 2014 11:51:46 -0700 (PDT) In-Reply-To: <53A9C6D0.3090900@netfence.it> References: <53A9C6D0.3090900@netfence.it> From: Carlos Ferreira Date: Tue, 24 Jun 2014 19:51:46 +0100 Message-ID: Subject: Re: MTU not regrowing? To: Andrea Venturoli Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 18:52:27 -0000 don't forget the header of the ping. the -s flag specifies the amount of data that the ping carries. By specifying a size of 500, you are creating a ping packet larger than the MTU. On 24 June 2014 19:43, Andrea Venturoli wrote: > Hello. > > Today I experienced something weird (at least for me) on a 8.4 system: > > _ the system had vlan3 interface, with default MTU (1500 bytes); > _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" > would yield "frag needed and DF set" (forgive me if the message is not > exact, I don't have it anymore); > > _ to make some tests I reduced MTU size with "ifconfig vlan3 mtu 500"; > _ now, of course, "ping -D -s 400 somehost" would work, but "ping -D -s > 500 somehost" would yield "frag needed and DF set"; > > _ then I raised MTU again with "ifconfig vlan3 mtu 1500" (notice ifconfig > would actually report this as "mtu 1500" was shown); > _ however the results were as before, i.e. "ping -D -s 400 somehost" would > work, but "ping -D -s 500 somehost" would yield "frag needed and DF set"; > > _ no way I could ping with a packet bigger than 500 bytes until I rebooted. > > Is this expected behaviour? Any way to get around this? > > bye & Thanks > av. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 18:58:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 551B2405 for ; Tue, 24 Jun 2014 18:58:42 +0000 (UTC) Received: from mail-oa0-f51.google.com (mail-oa0-f51.google.com [209.85.219.51]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D4372C9A for ; Tue, 24 Jun 2014 18:58:41 +0000 (UTC) Received: by mail-oa0-f51.google.com with SMTP id j17so855318oag.24 for ; Tue, 24 Jun 2014 11:58:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=VY/42XxlPlFaHd4ufj46AZ4E1o1NS1Fj8jyvoYX29qs=; b=lP8EHTOyjjuft71fK3QoVp/1l/5FuaOSzFLBGY8o4lKPvC+dDagMDL3XtiixXKj8si mSrkBWuZPwSMq6DTNJjmS/KhmyYqaZC1G0c1HR1P26Vs/7V+zE97dEe+bMLIce1KUgmZ QEDdBy3naziVULJozcIMKGbqto0yJeUqsYAAjt2NPv9jBywsdr/W9XWPacpbz45uCA/C C2yq4iwurqAf/J+rqGw+VYSCMD5BcySB9Z4AYV5W3qkHcUbn/sIWMqgkJ7/FVGxEdxaJ GbH3r+IviNGNODgC/ju1f+3mhRt/altR7KUMYZHLUZGOt82XdIt3HOJC2NSRtrIw7P0I Xg9A== X-Gm-Message-State: ALoCoQkAs5VVz4xWetPV2qf4riapP7o9rcM/PY64WtKzWDE4y1SX7T9m+LZG8kXh9VMoRKk2MNJC MIME-Version: 1.0 X-Received: by 10.60.57.3 with SMTP id e3mr2806999oeq.33.1403635958655; Tue, 24 Jun 2014 11:52:38 -0700 (PDT) Received: by 10.60.11.134 with HTTP; Tue, 24 Jun 2014 11:52:38 -0700 (PDT) In-Reply-To: <53A9C6D0.3090900@netfence.it> References: <53A9C6D0.3090900@netfence.it> Date: Tue, 24 Jun 2014 11:52:38 -0700 Message-ID: Subject: Re: MTU not regrowing? From: Michael Sierchio To: Andrea Venturoli Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 18:58:42 -0000 On Tue, Jun 24, 2014 at 11:43 AM, Andrea Venturoli wrote: > _ the system had vlan3 interface, with default MTU (1500 bytes); > _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" > would yield "frag needed and DF set" (forgive me if the message is not > exact, I don't have it anymore); Does your interface support hardware VLAN tagging? (i.e. does an ifconfig show VLAN_HWTAGGING?) - M =C3=97 From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 19:03:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A80251F for ; Tue, 24 Jun 2014 19:03:36 +0000 (UTC) Received: from zibbi.meraka.csir.co.za (zibbi.meraka.csir.co.za [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0F91C2D40 for ; Tue, 24 Jun 2014 19:03:36 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id C8039B843; Tue, 24 Jun 2014 21:03:33 +0200 (SAST) Date: Tue, 24 Jun 2014 21:03:33 +0200 From: John Hay To: Andrea Venturoli Subject: Re: MTU not regrowing? Message-ID: <20140624190333.GA13748@zibbi.meraka.csir.co.za> References: <53A9C6D0.3090900@netfence.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53A9C6D0.3090900@netfence.it> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 19:03:36 -0000 On Tue, Jun 24, 2014 at 08:43:28PM +0200, Andrea Venturoli wrote: > Hello. > > Today I experienced something weird (at least for me) on a 8.4 system: > > _ the system had vlan3 interface, with default MTU (1500 bytes); > _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" > would yield "frag needed and DF set" (forgive me if the message is not > exact, I don't have it anymore); > > _ to make some tests I reduced MTU size with "ifconfig vlan3 mtu 500"; > _ now, of course, "ping -D -s 400 somehost" would work, but "ping -D -s > 500 somehost" would yield "frag needed and DF set"; > > _ then I raised MTU again with "ifconfig vlan3 mtu 1500" (notice > ifconfig would actually report this as "mtu 1500" was shown); > _ however the results were as before, i.e. "ping -D -s 400 somehost" > would work, but "ping -D -s 500 somehost" would yield "frag needed and > DF set"; > > _ no way I could ping with a packet bigger than 500 bytes until I rebooted. > > Is this expected behaviour? Any way to get around this? Do a "route get somehost" and see what mtu is returned. You might be able to delete or tweak that route. John -- John Hay -- jhay@meraka.csir.co.za / jhay@meraka.org.za From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 19:32:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A96455FC for ; Tue, 24 Jun 2014 19:32:50 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 5A44E2052 for ; Tue, 24 Jun 2014 19:32:50 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id E0D196BAECA for ; Tue, 24 Jun 2014 21:32:42 +0200 (CEST) Received: from soth.ventu (adsl-ull-205-226.41-151.net24.it [151.41.226.205]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s5OJWbf5045882 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Tue, 24 Jun 2014 21:32:39 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-205-226.41-151.net24.it [151.41.226.205] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s5OJWVeM075552; Tue, 24 Jun 2014 21:32:31 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53A9D24F.5020003@netfence.it> Date: Tue, 24 Jun 2014 21:32:31 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Hay Subject: Re: MTU not regrowing? References: <53A9C6D0.3090900@netfence.it> <20140624190333.GA13748@zibbi.meraka.csir.co.za> In-Reply-To: <20140624190333.GA13748@zibbi.meraka.csir.co.za> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Tue, 24 Jun 2014 21:32:40 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 19:32:50 -0000 On 06/24/14 21:03, John Hay wrote: > Do a "route get somehost" and see what mtu is returned. You might be > able to delete or tweak that route. Thanks a lot! I learned something new :) I'll try this next time I have the chance. bye av. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 19:48:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19ED5BAC; Tue, 24 Jun 2014 19:48:41 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6B2C2522; Tue, 24 Jun 2014 19:48:40 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 63055B94A; Tue, 24 Jun 2014 15:48:39 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Ordering problem in if_detach_internal regarding if_bridge Date: Tue, 24 Jun 2014 13:06:39 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <53A4527F.90008@citrix.com> <53A85A8D.4090208@FreeBSD.org> <53A86016.5000102@citrix.com> In-Reply-To: <53A86016.5000102@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Message-Id: <201406241306.40064.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 24 Jun 2014 15:48:39 -0400 (EDT) Cc: "Alexander V. Chernikov" , Roger Pau =?iso-8859-15?q?Monn=E9?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 19:48:41 -0000 On Monday, June 23, 2014 1:12:54 pm Roger Pau Monn=E9 wrote: > On 23/06/14 18:49, Alexander V. Chernikov wrote: > > On 23.06.2014 20:39, Alexander V. Chernikov wrote: > >> On 23.06.2014 19:32, John Baldwin wrote: > >>> On Friday, June 20, 2014 11:25:51 am Roger Pau Monn=E9 wrote: > >>>> Hello, > >>>> > >>>> I've stumbled across the following panic when testing Xen netback wi= th=20 > >>>> if_bridge: > >>>> > >>>> Kernel page fault with the following non-sleepable locks held: > >>>> exclusive sleep mutex if_bridge (if_bridge) r =3D 0 (0xfffff80006306= c18)=20 > >>> locked @ /usr/src/sys/m > >>>> KDB: stack backtrace: > >>>> X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe00002= 13490 > >>>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0000213540 > >>>> witness_warn() at witness_warn+0x4a8/frame 0xfffffe0000213600 > >>>> trap() at trap+0xc9d/frame 0xfffffe00002136a0 > >>>> trap() at trap+0x669/frame 0xfffffe00002138b0 > >>>> calltrap() at calltrap+0x8/frame 0xfffffe00002138b0 > >>>> --- trap 0xc, rip =3D 0xffffffff8221a0ef, rsp =3D 0xfffffe0000213970= , rbp =3D=20 > >>> 0xfffffe00002139e0 --- > >>>> bridge_input() at bridge_input+0x5ff/frame 0xfffffe00002139e0 > >>>> ether_vlanencap() at ether_vlanencap+0x4a3/frame 0xfffffe0000213a10 > >>>> netisr_dispatch_src() at netisr_dispatch_src+0x90/frame 0xfffffe0000= 213a80 > >>>> ether_ifattach() at ether_ifattach+0x19f/frame 0xfffffe0000213ab0 > >>>> ath_dfs_get_thresholds() at ath_dfs_get_thresholds+0x81ce/frame=20 > >>> 0xfffffe0000213b30 > >>>> intr_event_execute_handlers() at intr_event_execute_handlers+0x93/fr= ame=20 > >>> 0xfffffe0000213b70 > >>>> db_dump_intr_event() at db_dump_intr_event+0x796/frame 0xfffffe00002= 13bb0 > >>>> fork_exit() at fork_exit+0x84/frame 0xfffffe0000213bf0 > >>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000213bf0 > >>>> --- trap 0, rip =3D 0, rsp =3D 0xfffffe0000213cb0, rbp =3D 0 --- > >>>> > >>>> I've tracked this down to if_detach_internal setting ifp->if_addr to= =20 > >>>> NULL before calling EVENTHANDLER_INVOKE(ifnet_departure_event..., wh= ich=20 > >>>> causes a panic in GRAB_OUR_PACKETS in the if_bridge code when it tri= es=20 > >>>> to perform IF_LLADDR on an interface that's in the process of being= =20 > >>>> destroyed (ifp->if_addr set to NULL, but the ifnet_departure_event e= vent=20 > >>>> has not fired yet). > >>>> > >>>> I have the following naive patch that moves the firing of the event= =20 > >>>> before if_addr is set to NULL, but I'm not familiar with the orderin= g=20 > >>>> in if_detach_internal, so I'm not sure if this might cause problems = in=20 > >>>> other parts of the code, could someone familiar with the net stuff=20 > >>>> comment on the best way to deal with it? > >> > >> We should notify kernel customers only when we are really taking this > >> interface down and every other subsystem cannot add any new state to t= he > >> interface. > >> > >> In this patch you're sending notification before taking ifnet down, > >> removing its L3 addresses, routes, and so on. > >> > >> This can easily lead to panic in, for example, BPF subsystem (since BPF > >> state is freed in bpf_ifdetach() handler). > >> > >> Addintionally, this will introduce ifaddr / iface messages reversal for > >> rtsock. > > Whoops. I misread the patch. > > It should be OK. > >=20 > >> > >> It looks like we'd better fix if_bridge (and it is still using mutexes, > >> what a shame!). > >> > >> Can you send me trace with line numbers? > > However, these two still stands. > > (And I'm wondering how you're getting any traffic on down/dying interfa= ce). >=20 > I'm not getting the traffic from the dying interface, I'm getting the > traffic from another interface on the bridge (a physical bce interface), > which injects traffic into the bridge, that calls bridge_input, which > tries to read ifp->if_addr->ifa_addr from the dying interface, and that > leads to the panic. >=20 > Line numbers: >=20 > /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2410 (bridge_input) > /usr/src/sys/net/if_ethersubr.c:543 (ether_input_internal) > /usr/src/sys/net/netisr.c:972 (netisr_dispatch_src) > /usr/src/sys/net/if_ethersubr.c:674 (ether_input) > /usr/src/sys/dev/bce/if_bce.c:6861 (bce_rx_intr) I think this certainly suggests moving at least the eventhandler up so that things like vlans and bridges can detach from an interface while it is still constructed. I do think it would be ideal to move all three notifications to the same place though. (So your original patch plus moving the routing socket message.) =2D-=20 John Baldwin From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 20:41:57 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9DEB94A for ; Tue, 24 Jun 2014 20:41:57 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0E652BBE for ; Tue, 24 Jun 2014 20:41:57 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OKfvwK003836 for ; Tue, 24 Jun 2014 21:41:57 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 20:41:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 20:41:57 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Triage |Issue Resolved Resolution|--- |Works As Intended --- Comment #7 from John Baldwin --- Well, if you do 'cpuset -x 264 -l 3' you should not see the interrupt or ithread run on any other CPU than CPU 3. If you do see that, let me know and we can re-open this. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 20:56:05 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00BAEB8F for ; Tue, 24 Jun 2014 20:56:04 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC6022D31 for ; Tue, 24 Jun 2014 20:56:04 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OKu4bi037170 for ; Tue, 24 Jun 2014 21:56:04 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 20:56:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gondim@bsdinfo.com.br X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 20:56:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #8 from gondim@bsdinfo.com.br --- Yes it changes the CPU. With top -PSH you can see it changing CPU. I did: cpuset -x 264 -l 3 See my top below: PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 10 root 155 ki31 0K 64K RUN 1 477.6H 100.00% idle{idle: cpu1} 0 root -92 0 0K 384K CPU3 3 219.2H 60.50% kernel{em0 que} PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 10 root 155 ki31 0K 64K RUN 1 477.6H 100.00% idle{idle: cpu1} 0 root -92 0 0K 384K CPU2 2 219.2H 61.72% kernel{em0 que} Process kernel{em0 que} running in CPU3 and CPU2. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 21:09:09 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C13E10F for ; Tue, 24 Jun 2014 21:09:09 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1356F2EA4 for ; Tue, 24 Jun 2014 21:09:09 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OL98ir078858 for ; Tue, 24 Jun 2014 22:09:08 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 21:09:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 21:09:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #9 from John Baldwin --- Ah, the "em0 que" thread is a separate helper thread created by the em driver to service a taskqueue. It is not an interrupt thread so it is not affected by 'cpuset -x'. However, you can use 'procstat -t 0' to determine the thread ID of that thread and bind that specific thread to CPU 3 using 'cpuset -l 3 -t '. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 21:35:53 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21102BB3 for ; Tue, 24 Jun 2014 21:35:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0858E2181 for ; Tue, 24 Jun 2014 21:35:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5OLZqoZ010947 for ; Tue, 24 Jun 2014 22:35:52 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190785] [em] cpu affinity not working in FreeBSD 10-STABLE Date: Tue, 24 Jun 2014 21:35:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gondim@bsdinfo.com.br X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 21:35:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190785 --- Comment #10 from gondim@bsdinfo.com.br --- Cool!!!! Thanks! Works now. :D Cheers, -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 00:01:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F94FD47 for ; Wed, 25 Jun 2014 00:01:10 +0000 (UTC) Received: from mail-in6.apple.com (mail-out6.apple.com [17.151.62.28]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 683672E35 for ; Wed, 25 Jun 2014 00:01:09 +0000 (UTC) Received: from mail-out.apple.com (bramley.apple.com [17.151.62.49]) (using TLS with cipher RC4-MD5 (128/128 bits)) (Client did not present a certificate) by mail-in6.apple.com (Apple Secure Mail Relay) with SMTP id F7.AD.27911.5411AA35; Tue, 24 Jun 2014 17:01:09 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from relay3.apple.com ([17.128.113.83]) by local.mail-out.apple.com (Oracle Communications Messaging Server 7.0.5.30.0 64bit (built Oct 22 2013)) with ESMTP id <0N7P002MB6NL2SQ1@local.mail-out.apple.com> for freebsd-net@freebsd.org; Tue, 24 Jun 2014 17:01:09 -0700 (PDT) X-AuditID: 11973e15-f79cf6d000006d07-9a-53aa1145fa89 Received: from [17.149.224.119] (Unknown_Domain [17.149.224.119]) (using TLS with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by relay3.apple.com (Apple SCV relay) with SMTP id 5B.26.08757.7411AA35; Tue, 24 Jun 2014 17:01:11 -0700 (PDT) Subject: Re: MTU not regrowing? From: Charles Swiger In-reply-to: <53A9C6D0.3090900@netfence.it> Date: Tue, 24 Jun 2014 17:01:08 -0700 Message-id: <2CA56230-440A-4D77-83BD-222FBC704F70@mac.com> References: <53A9C6D0.3090900@netfence.it> To: Andrea Venturoli X-Mailer: Apple Mail (2.1510) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrLLMWRmVeSWpSXmKPExsUiON3OUNdVcFWwwdNeEYsPO9qZHBg9Znya zxLAGMVlk5Kak1mWWqRvl8CVsWoLS8FLzopVd9qYGxhvsHcxcnJICJhIvD3xmRnCFpO4cG89 WxcjF4eQwEwmiYdX/oAleAUEJX5MvsfSxcjBwSwgL3HwvCxImFlAS+L7o1YWiPp5TBLXutcz wgw9d/UmE0Sin0libcsdNpCEMFDzhVNNbCCD2ATUJCZM5AEJcwpoS/w9sJcFxGYRUJW4d6OH DWKXtMSCPzEQJ1hJLPzZxgoSFgLae/8oE0hYBKh68e0ZTBBbZSVOn3sOdo6EwHtWia33F7NO YBSeheSDWQgfzELywQJG5lWMQrmJmTm6mXlmeokFBTmpesn5uZsYIeEruoPxzCqrQ4wCHIxK PLwXZq8MFmJNLCuuzD3EKM3BoiTOq7lhRbCQQHpiSWp2ampBalF8UWlOavEhRiYOTqkGRp73 aYHVfY+6n5XuPGHG6fAkOU12VZn+VbaujMD0n56/ntyuMamv2mpyw7F7pf67gjcN6ZNrPCo0 ZMtWn7Vk7/dKsW8wU29we8q4as4Os09xIt+WSvjoqAUcnxx3NTauPkNYc/OVvmBVa7YjhZ0z JjY6b39R8/KtXv7GbTtLDpa9s/jW8bBFiaU4I9FQi7moOBEAARPfhEACAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLLMWRmVeSWpSXmKPExsUiOPVBua674Kpgg3fHZS0+7Ghnsrh2o43d gcljxqf5LB4fl0xhDmCK4rJJSc3JLEst0rdL4Mo4feAle0EPV8XkiZ9YGxi/sHcxcnJICJhI nLt6kwnCFpO4cG89WxcjF4eQQD+TxMsbD9hAEswCWhI3/r0EKuLg4BXQk9j+Sw4kLCwgL3Hh VBMbSJhNQE1iwkQekDCngLbE5LUbwUayCKhK3LvRA1bCLCAtseBPDMRAbYllC18zQwy0klg1 pRbEFALac/8oWKMIUOPi2zOg7pKVOH3uOcsERv5ZSK6ZhXDNLCQzFzAyr2IUKErNSaw01kss KMhJ1UvOz93ECAq0hsLgHYx/llkdYhTgYFTi4b0we2WwEGtiWXFl7iFGCQ5mJRHeV6+BQrwp iZVVqUX58UWlOanFhxilOViUxHnPRiwOFhJITyxJzU5NLUgtgskycXBKNTCaff7s/f/44pNc QQcOrebOlH9/OPnV0fVb0iT2B3y86W9ntqhXcMrVjOB7ccynJX4pR1i6sVzbfFWOu1U/UfRb 1IFD+m7pwRcPLFeTvH1bOf7LoSfTuSNiZ/79/EdJ/Ihr1HF1azYmG3npE8f2NDJtq+KY7Wh8 8f3njT1syqkHO1+++F7nZhemxFKckWioxVxUnAgArwSpnTACAAA= Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 00:01:10 -0000 Hi-- On Jun 24, 2014, at 11:43 AM, Andrea Venturoli wrote: > Hello. > > Today I experienced something weird (at least for me) on a 8.4 system: > > _ the system had vlan3 interface, with default MTU (1500 bytes); > _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" would yield "frag needed and DF set" (forgive me if the message is not exact, I don't have it anymore); > > _ to make some tests I reduced MTU size with "ifconfig vlan3 mtu 500"; > _ now, of course, "ping -D -s 400 somehost" would work, but "ping -D -s 500 somehost" would yield "frag needed and DF set"; > > _ then I raised MTU again with "ifconfig vlan3 mtu 1500" (notice ifconfig would actually report this as "mtu 1500" was shown); > _ however the results were as before, i.e. "ping -D -s 400 somehost" would work, but "ping -D -s 500 somehost" would yield "frag needed and DF set"; > > _ no way I could ping with a packet bigger than 500 bytes until I rebooted. > > Is this expected behaviour? Any way to get around this? Does "ifconfig vlan3 down; ifconfig vlan3 up" do any good? Or that run against the physical NIC? What is the ethernet HW; and are you using VLAN_HWTAGGING capability? Regards, -- -Chuck From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 02:52:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B035319 for ; Wed, 25 Jun 2014 02:52:07 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 56E862D33 for ; Wed, 25 Jun 2014 02:52:06 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5P2q55w031478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Jun 2014 19:52:05 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5P2q5l0031477; Tue, 24 Jun 2014 19:52:05 -0700 (PDT) (envelope-from jmg) Date: Tue, 24 Jun 2014 19:52:05 -0700 From: John-Mark Gurney To: Andrea Venturoli Subject: Re: MTU not regrowing? Message-ID: <20140625025205.GR1560@funkthat.com> Mail-Followup-To: Andrea Venturoli , freebsd-net@freebsd.org References: <53A9C6D0.3090900@netfence.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53A9C6D0.3090900@netfence.it> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 24 Jun 2014 19:52:05 -0700 (PDT) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 02:52:07 -0000 Andrea Venturoli wrote this message on Tue, Jun 24, 2014 at 20:43 +0200: > Today I experienced something weird (at least for me) on a 8.4 system: > > _ the system had vlan3 interface, with default MTU (1500 bytes); > _ "ping -D -s 1400 somehost" would work, but "ping -D -s 1500 somehost" > would yield "frag needed and DF set" (forgive me if the message is not > exact, I don't have it anymore); > > _ to make some tests I reduced MTU size with "ifconfig vlan3 mtu 500"; > _ now, of course, "ping -D -s 400 somehost" would work, but "ping -D -s > 500 somehost" would yield "frag needed and DF set"; > > _ then I raised MTU again with "ifconfig vlan3 mtu 1500" (notice > ifconfig would actually report this as "mtu 1500" was shown); > _ however the results were as before, i.e. "ping -D -s 400 somehost" > would work, but "ping -D -s 500 somehost" would yield "frag needed and > DF set"; > > _ no way I could ping with a packet bigger than 500 bytes until I rebooted. > > Is this expected behaviour? Any way to get around this? This is expected behavior.. You need to delete the network route as that stores the MTU that is cloned to the host routes... HEAD and I believe 10 doesn't have this issue.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 09:43:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D94AC769; Wed, 25 Jun 2014 09:43:25 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB5D92150; Wed, 25 Jun 2014 09:43:24 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,544,1400025600"; d="scan'208,223";a="147116730" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 25 Jun 2014 09:43:16 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Wed, 25 Jun 2014 05:43:14 -0400 Message-ID: <53AA99AF.4020205@citrix.com> Date: Wed, 25 Jun 2014 11:43:11 +0200 From: =?ISO-8859-15?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Baldwin , Subject: Re: Ordering problem in if_detach_internal regarding if_bridge References: <53A4527F.90008@citrix.com> <53A85A8D.4090208@FreeBSD.org> <53A86016.5000102@citrix.com> <201406241306.40064.jhb@freebsd.org> In-Reply-To: <201406241306.40064.jhb@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------010303040404000907010805" X-DLP: MIA2 Cc: "Alexander V. Chernikov" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 09:43:26 -0000 --------------010303040404000907010805 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: 8bit On 24/06/14 19:06, John Baldwin wrote: > On Monday, June 23, 2014 1:12:54 pm Roger Pau Monné wrote: >> I'm not getting the traffic from the dying interface, I'm getting the >> traffic from another interface on the bridge (a physical bce interface), >> which injects traffic into the bridge, that calls bridge_input, which >> tries to read ifp->if_addr->ifa_addr from the dying interface, and that >> leads to the panic. >> >> Line numbers: >> >> /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2410 (bridge_input) >> /usr/src/sys/net/if_ethersubr.c:543 (ether_input_internal) >> /usr/src/sys/net/netisr.c:972 (netisr_dispatch_src) >> /usr/src/sys/net/if_ethersubr.c:674 (ether_input) >> /usr/src/sys/dev/bce/if_bce.c:6861 (bce_rx_intr) > > I think this certainly suggests moving at least the eventhandler up so that > things like vlans and bridges can detach from an interface while it is still > constructed. I do think it would be ideal to move all three notifications > to the same place though. (So your original patch plus moving the > routing socket message.) OK, I have updated the patch, now the devctl_notify is also done in the same block. Let me know what you think about it. Roger. --------------010303040404000907010805 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="if_detach.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="if_detach.patch" >From a3fdee18fa3f0d8707bfbf2d9324e3103e1cede8 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Fri, 20 Jun 2014 15:16:58 +0200 Subject: [PATCH] --- sys/net/if.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/sys/net/if.c b/sys/net/if.c index 47b5b9d..1c4bcab 100644 --- a/sys/net/if.c +++ b/sys/net/if.c @@ -875,6 +875,12 @@ if_detach_internal(struct ifnet *ifp, int vmove) #endif if_purgemaddrs(ifp); + /* Announce that the interface is gone. */ + rt_ifannouncemsg(ifp, IFAN_DEPARTURE); + EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); + if (IS_DEFAULT_VNET(curvnet)) + devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL); + if (!vmove) { /* * Prevent further calls into the device driver via ifnet. @@ -912,11 +918,6 @@ if_detach_internal(struct ifnet *ifp, int vmove) } } - /* Announce that the interface is gone. */ - rt_ifannouncemsg(ifp, IFAN_DEPARTURE); - EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); - if (IS_DEFAULT_VNET(curvnet)) - devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL); if_delgroups(ifp); /* -- 1.7.7.5 (Apple Git-26) --------------010303040404000907010805-- From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 10:08:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C97DF99; Wed, 25 Jun 2014 10:08:49 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A04602398; Wed, 25 Jun 2014 10:08:47 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s5PA8Z3g053604 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 25 Jun 2014 14:08:35 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s5PA8Z9g053603; Wed, 25 Jun 2014 14:08:35 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 25 Jun 2014 14:08:35 +0400 From: Gleb Smirnoff To: Alan Somers Subject: Re: ifaddr refcount problem Message-ID: <20140625100835.GK28199@glebius.int.ru> References: <53A48849.8080504@chelsio.com> <20140623085229.GQ28199@FreeBSD.org> <20140624090847.GB28199@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="SWTRyWv/ijrBap1m" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 10:08:49 -0000 --SWTRyWv/ijrBap1m Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Alan, On Tue, Jun 24, 2014 at 08:43:40AM -0600, Alan Somers wrote: A> That looks better. But I think there is one more possibility for a A> leak. For multicast packets, IFP_TO_IA at line 263 will call A> ifa_ref(), unless the the interface has no address assigned. How A> about this patch instead? Also, not tested. Again you are right, thanks. The patch needs to reset have_ia_ref in the 'goto again' case. Also, I'd suggest a tiny change to your patch so that we don't have a broken logic reading the code as "I have ia reference, but don't have ia". -- Totus tuus, Glebius. --SWTRyWv/ijrBap1m Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="ip_output_v4.diff" Index: ip_output.c =================================================================== --- ip_output.c (revision 267536) +++ ip_output.c (working copy) @@ -136,6 +136,7 @@ ip_output(struct mbuf *m, struct mbuf *opt, struct struct rtentry *rte; /* cache for ro->ro_rt */ struct in_addr odst; struct m_tag *fwd_tag = NULL; + int have_ia_ref; #ifdef IPSEC int no_route_but_check_spd = 0; #endif @@ -202,6 +203,7 @@ ip_output(struct mbuf *m, struct mbuf *opt, struct gw = dst = (struct sockaddr_in *)&ro->ro_dst; again: ia = NULL; + have_ia_ref = 0; /* * If there is a cached route, check that it is to the same * destination and is still up. If not, free it and try again. @@ -238,6 +240,7 @@ again: error = ENETUNREACH; goto bad; } + have_ia_ref = 1; ip->ip_dst.s_addr = INADDR_BROADCAST; dst->sin_addr = ip->ip_dst; ifp = ia->ia_ifp; @@ -250,6 +253,7 @@ again: error = ENETUNREACH; goto bad; } + have_ia_ref = 1; ifp = ia->ia_ifp; ip->ip_ttl = 1; isbroadcast = in_broadcast(dst->sin_addr, ifp); @@ -261,6 +265,8 @@ again: */ ifp = imo->imo_multicast_ifp; IFP_TO_IA(ifp, ia); + if (ia) + have_ia_ref = 1; isbroadcast = 0; /* fool gcc */ } else { /* @@ -552,8 +558,11 @@ sendit: #endif error = netisr_queue(NETISR_IP, m); goto done; - } else + } else { + if (have_ia_ref) + ifa_free(&ia->ifa); goto again; /* Redo the routing table lookup. */ + } } /* See if local, if yes, send it to netisr with IP_FASTFWD_OURS. */ @@ -582,6 +591,8 @@ sendit: m->m_flags |= M_SKIP_FIREWALL; m->m_flags &= ~M_IP_NEXTHOP; m_tag_delete(m, fwd_tag); + if (have_ia_ref) + ifa_free(&ia->ifa); goto again; } @@ -694,6 +705,8 @@ passout: done: if (ro == &iproute) RO_RTFREE(ro); + if (have_ia_ref) + ifa_free(&ia->ifa); return (error); bad: m_freem(m); --SWTRyWv/ijrBap1m-- From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 10:50:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A70681FD for ; Wed, 25 Jun 2014 10:50:54 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id 681902839 for ; Wed, 25 Jun 2014 10:50:53 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id D5619141485D for ; Wed, 25 Jun 2014 12:43:33 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 548011A60DC for ; Wed, 25 Jun 2014 12:43:36 +0200 (CEST) From: Nils Beyer To: freebsd-net@freebsd.org Subject: MPTCP: anything new? Date: Wed, 25 Jun 2014 12:43:36 +0200 Message-ID: <4420150.07CXDzonuv@asbach.renzel.net> Organization: VKF Renzel GmbH User-Agent: KMail/4.12.5 (FreeBSD/10.0-STABLE; KDE/4.12.5; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-7.0 required=7.0 tests=BAYES_00,UNPARSEABLE_RELAY autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 10:50:54 -0000 Hi, are there any news regarding MPTCP? Last public patch is from April last year: http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html Is there a newer patch for a more recent CURRENT/STABLE available? Thanks and regards, Nils From owner-freebsd-net@FreeBSD.ORG Wed Jun 25 13:23:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06BD7245 for ; Wed, 25 Jun 2014 13:23:57 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id AD9532784 for ; Wed, 25 Jun 2014 13:23:56 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id A1F9C6BACDF for ; Wed, 25 Jun 2014 15:23:50 +0200 (CEST) Received: from soth.ventu (adsl-ull-149-156.41-151.net24.it [151.41.156.149]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s5PDNh3m019667 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 25 Jun 2014 15:23:44 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-149-156.41-151.net24.it [151.41.156.149] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s5PDNYZ8039138; Wed, 25 Jun 2014 15:23:34 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53AACD56.2030701@netfence.it> Date: Wed, 25 Jun 2014 15:23:34 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Charles Swiger Subject: Re: MTU not regrowing? References: <53A9C6D0.3090900@netfence.it> <2CA56230-440A-4D77-83BD-222FBC704F70@mac.com> In-Reply-To: <2CA56230-440A-4D77-83BD-222FBC704F70@mac.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Wed, 25 Jun 2014 15:23:44 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 13:23:57 -0000 On 06/25/14 02:01, Charles Swiger wrote: > Does "ifconfig vlan3 down; ifconfig vlan3 up" do any good? > Or that run against the physical NIC? Can't try this now, I'll do when I can play again with this box. > What is the ethernet HW em0@pci0:6:0:0: class=0x020000 card=0x10828086 chip=0x107d8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'PRO/1000 PT' class = network subclass = ethernet > and are you using VLAN_HWTAGGING capability? Yes. bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 01:25:36 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11639FF1 for ; Thu, 26 Jun 2014 01:25:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC8B82ADA for ; Thu, 26 Jun 2014 01:25:35 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q1PZrL055176 for ; Thu, 26 Jun 2014 02:25:35 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191380] [vlan] Problem with tagged vlan after upgrading Date: Thu, 26 Jun 2014 01:25:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords component assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 01:25:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191380 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |regression Component|misc |kern Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|Problem with tagged vlan |[vlan] Problem with tagged |after upgrading |vlan after upgrading -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 01:33:21 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 277A631D for ; Thu, 26 Jun 2014 01:33:21 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0ECEE2BB1 for ; Thu, 26 Jun 2014 01:33:21 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q1XKef066244 for ; Thu, 26 Jun 2014 02:33:20 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191312] [re] re does not work with TP-Link TG-3468 v2 Date: Thu, 26 Jun 2014 01:33:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 01:33:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191312 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|re does not work with |[re] re does not work with |TP-Link TG-3468 v2 |TP-Link TG-3468 v2 --- Comment #1 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 01:36:00 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 581B9404 for ; Thu, 26 Jun 2014 01:36:00 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3F75E2BC9 for ; Thu, 26 Jun 2014 01:36:00 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q1a0s6095689 for ; Thu, 26 Jun 2014 02:36:00 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191343] [ipnat] ipnat error at boot disables active sessions Date: Thu, 26 Jun 2014 01:36:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 01:36:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191343 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|ipnat error at boot |[ipnat] ipnat error at boot |disables active sessions |disables active sessions --- Comment #3 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 01:59:32 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37623D6A for ; Thu, 26 Jun 2014 01:59:32 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1DD542DC6 for ; Thu, 26 Jun 2014 01:59:32 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q1xVRW073647 for ; Thu, 26 Jun 2014 02:59:31 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191192] [e1000] [patch] add support for Intel I218 V2 (0x15a1 PCI ID) network controller in the H97I chipset Date: Thu, 26 Jun 2014 01:59:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 01:59:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191192 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|support for Intel I218 V2 |[e1000] [patch] add support |(0x15a1 PCI ID) network |for Intel I218 V2 (0x15a1 |controller in the H97I |PCI ID) network controller |chipset |in the H97I chipset --- Comment #2 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 02:02:35 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8087EAE for ; Thu, 26 Jun 2014 02:02:35 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE5CF2E5D for ; Thu, 26 Jun 2014 02:02:35 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q22ZSV098273 for ; Thu, 26 Jun 2014 03:02:35 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191384] [ixgbe] ixgbe driver failed to identify PCI-Express slot bandwidth on 10-STABLE and 9.3-RC1, 10.0-RELEASE is partialy affected Date: Thu, 26 Jun 2014 02:02:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 02:02:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191384 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|ixgbe driver failed to |[ixgbe] ixgbe driver failed |identify PCI-Express slot |to identify PCI-Express |bandwidth on 10-STABLE and |slot bandwidth on 10-STABLE |9.3-RC1, 10.0-RELEASE is |and 9.3-RC1, 10.0-RELEASE |partialy affected |is partialy affected --- Comment #1 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 02:16:12 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED6DA3CF for ; Thu, 26 Jun 2014 02:16:12 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D4F172F7B for ; Thu, 26 Jun 2014 02:16:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5Q2GCKt067503 for ; Thu, 26 Jun 2014 03:16:12 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191312] [re] re does not work with TP-Link TG-3468 v2 Date: Thu, 26 Jun 2014 02:16:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 02:16:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191312 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hiren@FreeBSD.org, | |yongari@FreeBSD.org --- Comment #2 from Hiren Panchasara --- Afaik, "re0: Chip rev. 0x80000000" is not supported by looking at: $src/sys/pci/if_rlreg.h in -CURRENT. I am not well-versed with this code but yongari@ (cced in the ticket) may be able to help here. cheers, Hiren -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 02:22:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D9F2608 for ; Thu, 26 Jun 2014 02:22:28 +0000 (UTC) Received: from gpo3.cc.swin.edu.au (gpo3.cc.swin.edu.au [136.186.1.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B9A42077 for ; Thu, 26 Jun 2014 02:22:27 +0000 (UTC) Received: from [136.186.229.37] (garmitage.caia.swin.edu.au [136.186.229.37]) by gpo3.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s5Q2MPBg006639 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Jun 2014 12:22:25 +1000 Message-ID: <53AB83E1.7000803@swin.edu.au> Date: Thu, 26 Jun 2014 12:22:25 +1000 From: grenville armitage User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121107 Thunderbird/16.0.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: MPTCP: anything new? References: <4420150.07CXDzonuv@asbach.renzel.net> In-Reply-To: <4420150.07CXDzonuv@asbach.renzel.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Nigel Williams X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 02:22:28 -0000 I expect we'll be releasing a v0.4 patch in the next few days, as it happens. Many bug fixes wrt v0.3. It has been a long-time coming. cheers, gja On 06/25/2014 20:43, Nils Beyer wrote: > Hi, > > are there any news regarding MPTCP? Last public patch is from April last > year: > > http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html > > Is there a newer patch for a more recent CURRENT/STABLE available? > > > Thanks and regards, > Nils -- Professor Grenville Armitage Director, Centre for Advanced Internet Architectures School of Software and Electrical Engineering Faculty of Science, Engineering and Technology Swinburne University of Technology, Australia http://caia.swin.edu.au From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 07:48:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE26A56E for ; Thu, 26 Jun 2014 07:48:05 +0000 (UTC) Received: from frv191.fwdcdn.com (frv191.fwdcdn.com [212.42.77.191]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6783B29BF for ; Thu, 26 Jun 2014 07:48:04 +0000 (UTC) Received: from [10.10.1.29] (helo=frv197.fwdcdn.com) by frv191.fwdcdn.com with esmtp ID 1X048k-000GpN-TS for freebsd-net@freebsd.org; Thu, 26 Jun 2014 10:30:38 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=48c0ZOgp82BONoA2cSe4Cg0VHU/8IIN1EcJIPz4leb4=; b=IqBOoeTnGeUyD/tao+JpIcBHOsbA7cKUzLJrXXdM95290bRPckmwaYTWbAUM6EfmbMLOu/NmW26cBwPBGwRS+kiJMSjKzKrdYvVipofNk6fSXXbgQSjtgGUlUqMMFHKBmasJ19oaJHtPV10ujLSZsdmN06JmFJuNSfAy0CJPo8Y=; Received: from [10.10.10.34] (helo=frv34.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1X048a-0005BR-7O for freebsd-net@freebsd.org; Thu, 26 Jun 2014 10:30:28 +0300 Date: Thu, 26 Jun 2014 10:30:27 +0300 From: wishmaster Subject: Re[2]: MPTCP: anything new? To: grenville armitage X-Mailer: mail.ukr.net 5.0 Message-Id: <1403767523.719364127.q2e5ryg4@frv34.fwdcdn.com> In-Reply-To: <53AB83E1.7000803@swin.edu.au> References: <4420150.07CXDzonuv@asbach.renzel.net> <53AB83E1.7000803@swin.edu.au> MIME-Version: 1.0 Received: from artemrts@ukr.net by frv34.fwdcdn.com; Thu, 26 Jun 2014 10:30:27 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: freebsd-net@freebsd.org, Nigel Williams X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 07:48:05 -0000 Hi, Grenville Armitage. Sorry, for (unrelated question|offtopic), but this question related to you as developer of DIFFUSED. Why this extension (DIFFUSED) for IPFW, granted by FreeBSD Foundation, is still not in 10-RELEASE as was stated? -- --- Original message --- From: "grenville armitage" Date: 26 June 2014, 05:22:34 > > I expect we'll be releasing a v0.4 patch in the next few days, as it happens. Many bug fixes wrt v0.3. It has been a long-time coming. > > cheers, > gja > > On 06/25/2014 20:43, Nils Beyer wrote: > > Hi, > > > > are there any news regarding MPTCP? Last public patch is from April last > > year: > > > > http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html > > > > Is there a newer patch for a more recent CURRENT/STABLE available? > > > > > > Thanks and regards, > > Nils > > -- > Professor Grenville Armitage > Director, Centre for Advanced Internet Architectures > School of Software and Electrical Engineering > Faculty of Science, Engineering and Technology > Swinburne University of Technology, Australia > http://caia.swin.edu.au > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 15:52:40 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7151B87A for ; Thu, 26 Jun 2014 15:52:40 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5722D27D9 for ; Thu, 26 Jun 2014 15:52:40 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5QFqert055810 for ; Thu, 26 Jun 2014 16:52:40 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191384] [ixgbe] ixgbe driver failed to identify PCI-Express slot bandwidth on 10-STABLE and 9.3-RC1, 10.0-RELEASE is partialy affected Date: Thu, 26 Jun 2014 15:52:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: melifaro@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 15:52:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191384 Alexander V. Chernikov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |melifaro@FreeBSD.org --- Comment #2 from Alexander V. Chernikov --- As far as I remember, Intel 82598 is not capable of doing 5GT/s. Quoting product brief: "The Intel=C2=AE 82598 10 Gigabit Ethernet Controlle= r is a next-generation, PCI Express* 2.0 (2.5 Gbps) controller with balanced featu= res " http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-= adapters/82598-10-gigabit-ethernet-controller-brief.html --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 17:55:20 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A874826 for ; Thu, 26 Jun 2014 17:55:20 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7137326D9 for ; Thu, 26 Jun 2014 17:55:20 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5QHtKQm054429 for ; Thu, 26 Jun 2014 18:55:20 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191192] [e1000] [patch] add support for Intel I218 V2 (0x15a1 PCI ID) network controller in the H97I chipset Date: Thu, 26 Jun 2014 17:55:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: jfv@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 17:55:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191192 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |jfv@FreeBSD.org --- Comment #3 from Hiren Panchasara --- Unified version of the submitted diffs. Assigning to jfv@ % svn diff Index: sys/dev/e1000/e1000_api.c =================================================================== --- sys/dev/e1000/e1000_api.c (revision 267197) +++ sys/dev/e1000/e1000_api.c (working copy) @@ -293,6 +293,7 @@ case E1000_DEV_ID_PCH_LPT_I217_V: case E1000_DEV_ID_PCH_LPTLP_I218_LM: case E1000_DEV_ID_PCH_LPTLP_I218_V: + case E1000_DEV_ID_PCH_I218_V2: mac->type = e1000_pch_lpt; break; case E1000_DEV_ID_82575EB_COPPER: Index: sys/dev/e1000/e1000_hw.h =================================================================== --- sys/dev/e1000/e1000_hw.h (revision 267197) +++ sys/dev/e1000/e1000_hw.h (working copy) @@ -133,6 +133,7 @@ #define E1000_DEV_ID_PCH_LPT_I217_V 0x153B #define E1000_DEV_ID_PCH_LPTLP_I218_LM 0x155A #define E1000_DEV_ID_PCH_LPTLP_I218_V 0x1559 +#define E1000_DEV_ID_PCH_I218_V2 0x15A1 #define E1000_DEV_ID_82576 0x10C9 #define E1000_DEV_ID_82576_FIBER 0x10E6 #define E1000_DEV_ID_82576_SERDES 0x10E7 Index: sys/dev/e1000/e1000_ich8lan.c =================================================================== --- sys/dev/e1000/e1000_ich8lan.c (revision 267197) +++ sys/dev/e1000/e1000_ich8lan.c (working copy) @@ -61,7 +61,7 @@ * 82579V Gigabit Network Connection * Ethernet Connection I217-LM * Ethernet Connection I217-V - * Ethernet Connection I218-V + * Ethernet Connection I218-V2 * Ethernet Connection I218-LM */ @@ -1246,7 +1246,8 @@ /* Work-around I218 hang issue */ if ((hw->device_id == E1000_DEV_ID_PCH_LPTLP_I218_LM) || - (hw->device_id == E1000_DEV_ID_PCH_LPTLP_I218_V)) { + (hw->device_id == E1000_DEV_ID_PCH_LPTLP_I218_V) || + (hw->device_id == E1000_DEV_ID_PCH_I218_V2)) { ret_val = e1000_k1_workaround_lpt_lp(hw, link); if (ret_val) return ret_val; @@ -4597,7 +4598,8 @@ u16 phy_reg, device_id = hw->device_id; if ((device_id == E1000_DEV_ID_PCH_LPTLP_I218_LM) || - (device_id == E1000_DEV_ID_PCH_LPTLP_I218_V)) { + (device_id == E1000_DEV_ID_PCH_LPTLP_I218_V) || + (device_id == E1000_DEV_ID_PCH_I218_V2)) { u32 fextnvm6 = E1000_READ_REG(hw, E1000_FEXTNVM6); E1000_WRITE_REG(hw, E1000_FEXTNVM6, -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jun 26 20:09:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A4175AB; Thu, 26 Jun 2014 20:09:41 +0000 (UTC) Received: from gpo3.cc.swin.edu.au (gpo3.cc.swin.edu.au [136.186.1.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E5622354; Thu, 26 Jun 2014 20:09:39 +0000 (UTC) Received: from [136.186.229.37] (garmitage.caia.swin.edu.au [136.186.229.37]) by gpo3.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s5QK9a5h023453 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 27 Jun 2014 06:09:37 +1000 Message-ID: <53AC7E00.1070605@swin.edu.au> Date: Fri, 27 Jun 2014 06:09:36 +1000 From: grenville armitage User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121107 Thunderbird/16.0.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: re DIFFUSE... Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Lawrence Stewart X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 20:09:41 -0000 Hi, On 06/26/2014 17:30, wishmaster wrote: > Hi, Grenville Armitage. > > Sorry, for (unrelated question|offtopic), (I've started a new thread.) > but this question related to you as developer of DIFFUSED. > Why this extension (DIFFUSED) for IPFW, granted by FreeBSD Foundation, is still not in 10-RELEASE as was stated? Fair question. This is sitting in Lawrence Stewart's (rather deep) TODO queue. I'll let him comment further when he gets back from travels. cheers, gja From owner-freebsd-net@FreeBSD.ORG Fri Jun 27 13:28:35 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99FA2443 for ; Fri, 27 Jun 2014 13:28:35 +0000 (UTC) Received: from frv190.fwdcdn.com (frv190.fwdcdn.com [212.42.77.190]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5506D2EA8 for ; Fri, 27 Jun 2014 13:28:34 +0000 (UTC) Received: from [10.10.1.23] (helo=frv199.fwdcdn.com) by frv190.fwdcdn.com with esmtp ID 1X0Vub-0008TQ-9R for net@freebsd.org; Fri, 27 Jun 2014 16:09:53 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:Cc:To:Subject:From:Date; bh=zdVcWWdRuDmcRtZP2d0wbM0N8itvJxp+dfGBwoj5TrA=; b=mFit97smbp3iGvaAAsLy+Ox7IdeSov5umU2cOYKzBj64mR52Fyu9BmxbyGX8WsNpTNq+hfrlpcXAeO3wmr1si6z6pMKCGnLCwAzmG+GvFuRcYG0tmrFPaOwcdW+ZZ4jvgkb8j04k1hJSXjrMg2coGEax1HgRkg8B/kS5+MoYTOU=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv199.fwdcdn.com with smtp ID 1X0VuQ-0008Jr-7z for net@freebsd.org; Fri, 27 Jun 2014 16:09:42 +0300 Date: Fri, 27 Jun 2014 16:09:41 +0300 From: Vladislav Prodan Subject: sendmail+tls+sasl2-8.14.9 =?UTF-8?b?0L3QsCAxMC3QutC1IGRvbuKAmXQ=?= send mail during dual stack To: net@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1403874003.599493508.q8rrp3n5@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Fri, 27 Jun 2014 16:09:41 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: gshapiro@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 13:28:35 -0000 Hello! My sendmail+tls+sasl2-8.14.9 can’t send mail to ipv4 only post hosts, and sends to gmail (dualstack – ipv4 and ipv6) through ipv4! ----- Transcript of session follows ----- 553 5.3.5 mxs.mail.ru. config error: mail loops back to me (MX problem?) 554 5.3.5 Local configuration error 553 5.3.5 mxs.ukr.net. config error: mail loops back to me (MX problem?) 554 5.3.5 Local configuration error Before update on FreeBSD 10.0, was installed sendmail+tls+sasl2-8.14.4_2, without any problem. sendmail+tls+sasl2-8.14.9 : # sockstat | grep sendmail smmsp sendmail 26131 4 dgram -> /var/run/log root sendmail 26125 4 dgram -> /var/run/logpriv root sendmail 26125 5 tcp4 *:25 *:* root sendmail 26125 6 tcp6 *:25 *:* root sendmail 26125 7 tcp4 *:587 *:* root sendmail 26125 8 tcp6 *:587 *:* # echo '$=m' | sendmail -bt -d0.4 Version 8.14.9 Compiled with: DNSMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB NIS PICKY_HELO_CHECK PIPELINING SASLv2 SCANF SOCKETMAP STARTTLS TCPWRAPPERS USERDB XDEBUG Canonical name: otrada.od.ua UUCP nodename: mary-teresa.otrada.od.ua a.k.a.: mary-teresa.otrada.od.ua a.k.a.: [IPv6:2001:470:28:140::5] ============ SYSTEM IDENTITY (after readcf) ============ (short domain name) $w = otrada (canonical domain name) $j = otrada.od.ua (subdomain name) $m = od.ua (node name) $k = mary-teresa.otrada.od.ua ======================================================== ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter
> od.ua > #egrep -v '^dnl|^$|^#' mary-teresa.otrada.od.ua.mc divert(0) VERSIONID(`$FreeBSD: src/etc/sendmail/freebsd.mc,v 1.34 2007/04/23 22:23:54 gshapiro Exp $') OSTYPE(freebsd6) DOMAIN(generic) INPUT_MAIL_FILTER(`greylist', `S=local:/var/milter-greylist/milter-greylist.sock, F=T, T=R:30s') INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass-milter.sock, F=, T=C:15m;S:4m;R:4m;E:10m') INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock, F=, T=S:4m;R:4m') define(`confMILTER_LOG_LEVEL',`16') define(`confMILTER_MACROS_HELO', confMILTER_MACROS_HELO``, {verify}'')dnl define(`confMILTER_MACROS_ENVRCPT',`r, v, Z')dnl define(`confMILTER_MACROS_CONNECT', `j, {if_addr}') define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}') define(`confMILTER_MACROS_ENVRCPT', confMILTER_MACROS_ENVRCPT``, {greylist}'') define(`confINPUT_MAIL_FILTERS', `greylist,spamassassin,clmilter') define(`confSMTP_LOGIN_MSG', `exchange.srv.local Microsoft MAIL Service, Version: 6.0.3790.1830 ready')dnl TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl define(`confDEF_AUTH_INFO', `/etc/mail/auth-info')dnl FEATURE(`delay_checks') FEATURE(access_db, `hash -o -T /etc/mail/access') FEATURE(blacklist_recipients) FEATURE(local_lmtp) FEATURE(mailertable, `hash -o /etc/mail/mailertable') FEATURE(virtusertable, `hash -o /etc/mail/virtusertable') define(`confCR_FILE', `/etc/mail/relay-domains')dnl define(`STATUS_FILE', `/etc/mail/statistics')dnl define(`confHOST_STATUS_DIRECTORY', `/etc/mail/host_status')dnl define(`confCW_FILE', `-o /etc/mail/local-host-names') define(`confMAX_MESSAGE_SIZE',`25000000') DAEMON_OPTIONS(`Name=IPv4, Family=inet') DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O') FEATURE(`no_default_msa')dnl DAEMON_OPTIONS(`Port=587, Name=MSA-IPv4, M=Ea, Family=inet') DAEMON_OPTIONS(`Port=587, Name=MSA-IPv6, M=Ea, Family=inet6') define(`confDONT_PROBE_INTERFACES') define(`confBIND_OPTS', `WorkAroundBrokenAAAA') define(`confNO_RCPT_ACTION', `add-to-undisclosed') define(`confPRIVACY_FLAGS', `authwarnings,noexpn,novrfy') MAILER(local) MAILER(smtp) # egrep -v '^dnl|^$|^#' mary-teresa.otrada.od.ua.submit.mc divert(-1) divert(0)dnl VERSIONID(`$FreeBSD: src/etc/sendmail/freebsd.submit.mc,v 1.7.2.2 2010/01/31 19:04:52 gshapiro Exp $') define(`confCF_VERSION', `Submit')dnl define(`__OSTYPE__',`')dnl dirty hack to keep proto.m4 from complaining define(`_USE_DECNET_SYNTAX_', `1')dnl support DECnet define(`confTIME_ZONE', `USE_TZ')dnl define(`confDONT_INIT_GROUPS', `True')dnl define(`confBIND_OPTS', `WorkAroundBrokenAAAA')dnl FEATURE(`msp', `[127.0.0.1]')dnl #cat local-host-names mary-teresa.otrada.od.ua otrada.od.ua mx.otrada.od.ua localhost otrada.local # host localhost localhost.otrada.od.ua has address 127.0.0.1 # cat /etc/hosts ::1 localhost localhost.my.domain 127.0.0.1 localhost localhost.my.domain # telnet localhost 25 Trying ::1... Connected to localhost. Escape character is '^]'. 220 exchange.srv.local ESMTP Microsoft MAIL Service, Version: 6.0.3790.1830 ready ^] telnet> Working version sendmail - sendmail+tls+sasl2-8.14.4_2 : FreeBSD 10.0-STABLE #0: Fri Apr 25 12:30:14 EEST 2014 sendmail+tls+sasl2-8.14.4_2 Reliable, highly configurable mail transfer agent with util # echo '$=m' | sendmail -bt -d0.4 Version 8.14.4 Compiled with: DNSMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB NIS PICKY_HELO_CHECK PIPELINING SASLv2 SCANF SOCKETMAP STARTTLS TCPWRAPPERS USERDB XDEBUG Canonical name: otrada.od.ua UUCP nodename: mary-teresa.otrada.od.ua a.k.a.: mary-teresa.otrada.od.ua a.k.a.: [IPv6:2001:470:28:140::5] a.k.a.: [89.209.81.54] ============ SYSTEM IDENTITY (after readcf) ============ (short domain name) $w = otrada (canonical domain name) $j = otrada.od.ua (subdomain name) $m = od.ua (node name) $k = mary-teresa.otrada.od.ua ======================================================== ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter
> od.ua > -- Vladislav V. Prodan System & Network Administrator support.od.ua From owner-freebsd-net@FreeBSD.ORG Fri Jun 27 13:32:12 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9634E64D for ; Fri, 27 Jun 2014 13:32:12 +0000 (UTC) Received: from frv197.fwdcdn.com (frv197.fwdcdn.com [212.42.77.197]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FB4F2F65 for ; Fri, 27 Jun 2014 13:32:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=c6s90JfdRZteo4KwXK9Y97MlZClQGA1x3wYNJT96tAE=; b=pu4yR1yt+94OwzrU5xPk7JMXrv1w/4v2AgsXyFstQoZS1+1FUCEIkotYsOSaEwOUhQlOH9behCzsUx2VTGcJ4rjen/67qjXlIJ3x8rMQLVYI6TfA2tGw9Dgb7VWZdAonR5gq9Pp76QnT+LWutUC6McJqeQFPpl5KDQe3CRBY6us=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1X0WG9-000GU8-PV for net@freebsd.org; Fri, 27 Jun 2014 16:32:09 +0300 Date: Fri, 27 Jun 2014 16:32:09 +0300 From: Vladislav Prodan Subject: Re: sendmail+tls+sasl2-8.14.9 =?UTF-8?b?0L3QsCAxMC3QutC1IGRvbuKAmXQ=?= send mail during dual stack To: Vladislav Prodan X-Mailer: mail.ukr.net 5.0 Message-Id: <1403875921.768979030.2b1o31rv@frv35.fwdcdn.com> In-Reply-To: <1403874003.599493508.q8rrp3n5@frv35.fwdcdn.com> References: <1403874003.599493508.q8rrp3n5@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Fri, 27 Jun 2014 16:32:09 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: gshapiro@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 13:32:12 -0000 --- Original message --- From: "Vladislav Prodan" Date: 27 June 2014, 16:28:41 > > Hello! > > and sends to gmail (dualstack – ipv4 and ipv6) through ipv4! > To gmail: Return-Path: Received: from [10.0.0.10] (otrada.od.ua. [89.209.81.54]) by mx.google.com with ESMTPSA id x2sm5634910lae.1.2014.06.05.11.48.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 05 Jun 2014 11:48:03 -0700 (PDT) From: "Vladislav V. Prodan" X-Google-Original-From: "Vladislav V. Prodan" Message-ID: <5390BB60.2090407@otrada.od.ua> Date: Thu, 05 Jun 2014 21:48:00 +0300 Organization: Otrada Network User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "Vladislav V. Prodan" Subject: =?UTF-8?B?0L/RgNC+0LLQtdGA0LrQsCAy?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ... -- Vladislav V. Prodan System & Network Administrator support.od.ua From owner-freebsd-net@FreeBSD.ORG Fri Jun 27 14:17:34 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEE55D4C for ; Fri, 27 Jun 2014 14:17:34 +0000 (UTC) Received: from frv189.fwdcdn.com (frv189.fwdcdn.com [212.42.77.189]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7843E2446 for ; Fri, 27 Jun 2014 14:17:34 +0000 (UTC) Received: from [10.10.2.23] (helo=frv198.fwdcdn.com) by frv189.fwdcdn.com with esmtp ID 1X0WeB-000P9j-56 for net@freebsd.org; Fri, 27 Jun 2014 16:56:59 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=A9p8Jd7Gjd17O/9kijxvhO83vGgj+mjCXG47C2qxPF8=; b=Jb1ny2sCbXtPdfBzLrtbWcGkjg4dMxNyreNqwckPX/5Mtf64Nk62suuu1Tjgu1rmjshKhYSCmho4GicViDPlAMkeePTFnmaF78dS28tgpL8DqYGyrbqthHBHrvLWxqoVzZOQDb2yS+v2TiauGbQP4Yw6XzZ1cs9Txom6NHF9ODs=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1X0We3-000GAa-CT for net@freebsd.org; Fri, 27 Jun 2014 16:56:51 +0300 Date: Fri, 27 Jun 2014 16:56:50 +0300 From: Vladislav Prodan Subject: Sorry. Subject should read as: sendmail+tls+sasl2-8.14.9 in FreeBSD 10-STABLE don't send mail during dual stack To: net@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1403877318.701212986.s5q2br7r@frv35.fwdcdn.com> In-Reply-To: <1403874003.599493508.q8rrp3n5@frv35.fwdcdn.com> References: <1403874003.599493508.q8rrp3n5@frv35.fwdcdn.com> MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Fri, 27 Jun 2014 16:56:51 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: gshapiro@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 14:17:34 -0000 -- Vladislav V. Prodan System & Network Administrator support.od.ua From owner-freebsd-net@FreeBSD.ORG Sat Jun 28 07:45:34 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5BE8917 for ; Sat, 28 Jun 2014 07:45:34 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 423742E71 for ; Sat, 28 Jun 2014 07:45:34 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id b13so6039643wgh.35 for ; Sat, 28 Jun 2014 00:45:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=AwuvycG+/EhckcyHddA3YOgiDYELwcoFjrOKrakdYYE=; b=DJ3c7LY2oO3isG+IahRrCePFMlDITy3++wgmeL740R9jgGGxEqcE5GbzjE73zSQJkP F/EYXn365IC07FaElVRdxvDCmxk/mp9bwU4kdI3P/duts+Tufn3Ijt4oVQIk1lZpCU6L nWODMD+gVzP/kL4Oux0O56GhH+jOyS2HxpUbTfEE3OPRDTImF/veT9mdCetoHme1gW40 pXInR7JRt4SDwQnDJPBLncM1PiuXXQJgqCkxfhUxfvQj2Ugbk85XnAAXa3mviMcoYM5n eyyTiCIcmbqHNliFJAsSvGrIEsyvVKEHau8h//sELP4S4hBrgeN8awXrq7BXWzOP9lBI zrAw== MIME-Version: 1.0 X-Received: by 10.180.105.68 with SMTP id gk4mr16446707wib.24.1403941532423; Sat, 28 Jun 2014 00:45:32 -0700 (PDT) Received: by 10.194.173.132 with HTTP; Sat, 28 Jun 2014 00:45:32 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> Date: Sat, 28 Jun 2014 13:15:32 +0530 Message-ID: Subject: Re: netmap From: Prashant Upadhyaya To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Luigi Rizzo X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 07:45:34 -0000 Hi, Any further news ? Professor Luigi, one question regarding the tx with netmap. Whenever, I write a packet from user space into netmap rings, and if I want netmap to send this out immediately, do I necessarily have to do a ioctl(fd, NIOCTXSYNC, NULL) ? I have an application which receives packets, does some processing and then sends them out. If I keep doing ioctl's on every packet send, then there will be too may system calls hitting performance, the application can't afford to block it has to return back to polling for the receipt of next packet. On the receive side, I see that I don't have a problem because I can poll the ring without initiating an RXSYNC and whenever in user space I find that there is nothing on the ring (probably half way down the ring size), I do an RXSYNC to get more packets thus saving system calls. But on tx side, I have noticed that unless I do a TXSYNC, the packet does not go out, please let me know if I am missing something. Regards -Prashant On Tue, Jun 17, 2014 at 7:33 PM, Carlos Ferreira wrote: > Great! :) > I will give you the results as soon as I can get them :) > > > > On 17 June 2014 12:55, Luigi Rizzo wrote: > > > On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira > > wrote: > > > >> Ok, thanks for the enlightenment regarding the loss of performance. > >> > >> One question, just to be sure. Does the kernel module contains the VAL= A > >> switch code? Or do I need to compile extra code to have the switch > working? > >> Also, where can I find the documentation to use the Vala Switch? > >> > > > > =E2=80=8BVALE is part of the netmap kernel module, the only thing you n= eed > > to know to use it is port names: > > you can have multiple switch instances with multiple ports each, > > > > valeX:Y means port Y on switch X, X and Y are arbitrary strings > > with the constraint that the whole name must fit 15 characters. > > > > Details in the netmap manpage > > > > cheers > > luigi > > > > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Jun 28 08:24:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8EEB3FD for ; Sat, 28 Jun 2014 08:24:38 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40190213B for ; Sat, 28 Jun 2014 08:24:38 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id pn19so3623786lab.15 for ; Sat, 28 Jun 2014 01:24:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Hk6XDm+2Gz29O/p0U8iPXa5LoqGcg0EwK+gnCnxg6yk=; b=Nd+GFU/oCZnvnS3hzka+i7Y07NtMMttYlwiklu5FvBmFpdE0+xEoVwoRldqSWjaPJx 0cf8cwaC1f9HZ3b/fc0Qc8NpPem6O/1d/r8lMqvqvTuvTJqzVoEUUzwxeQZGgnTSIzTW 8TtTvjhPuP70p1xFG7FsGoGTKgsuNwS8JqpHpIh9BCllMnSklmSjVaTNnZHbu4RccBRk Eokxn4M3o77ELMTlPOMO3jFFgALFBYSnkGe7Tqm4f06eNcbHDzLv5mYqK7X4/tpI/zEt cPod0qC6z4M0BvlXVnUrRglPAijVADNYPmdEzCmeEkNj1D14apC4Hlq15S1EGpBwHd3h zflA== MIME-Version: 1.0 X-Received: by 10.112.218.74 with SMTP id pe10mr18885684lbc.3.1403943876043; Sat, 28 Jun 2014 01:24:36 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.22.100 with HTTP; Sat, 28 Jun 2014 01:24:35 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> Date: Sat, 28 Jun 2014 10:24:35 +0200 X-Google-Sender-Auth: G57mQDy0y8b-KTW4GyNyHX1qUUQ Message-ID: Subject: Re: netmap From: Luigi Rizzo To: Prashant Upadhyaya Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Carlos Ferreira X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 08:24:39 -0000 On Saturday, June 28, 2014, Prashant Upadhyaya wrote: > Hi, > > Any further news ? > > Professor Luigi, one question regarding the tx with netmap. > Whenever, I write a packet from user space into netmap rings, and if I > want netmap to send this out immediately, do I necessarily have to do > a ioctl(fd, NIOCTXSYNC, NULL) ? > Yes it is up to the application to decide when to push packets out with a txsync or select() or poll(), and unfortunately there is a tradeoff between efficiency and latency Cheers Luigi > > I have an application which receives packets, does some processing and > then sends them out. If I keep doing ioctl's on every packet send, then > there will be too may system calls hitting performance, the application > can't afford to block it has to return back to polling for the receipt of > next packet. > > On the receive side, I see that I don't have a problem because I can poll > the ring without initiating an RXSYNC and whenever in user space I find > that there is nothing on the ring (probably half way down the ring size),= I > do an RXSYNC to get more packets thus saving system calls. > > But on tx side, I have noticed that unless I do a TXSYNC, the packet does > not go out, please let me know if I am missing something. > > Regards > -Prashant > > > > On Tue, Jun 17, 2014 at 7:33 PM, Carlos Ferreira > wrote: > >> Great! :) >> I will give you the results as soon as I can get them :) >> >> >> >> On 17 June 2014 12:55, Luigi Rizzo > > wrote: >> >> > On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira > > >> > wrote: >> > >> >> Ok, thanks for the enlightenment regarding the loss of performance. >> >> >> >> One question, just to be sure. Does the kernel module contains the VA= LA >> >> switch code? Or do I need to compile extra code to have the switch >> working? >> >> Also, where can I find the documentation to use the Vala Switch? >> >> >> > >> > =E2=80=8BVALE is part of the netmap kernel module, the only thing you = need >> > to know to use it is port names: >> > you can have multiple switch instances with multiple ports each, >> > >> > valeX:Y means port Y on switch X, X and Y are arbitrary strings >> > with the constraint that the whole name must fit 15 characters. >> > >> > Details in the netmap manpage >> > >> > cheers >> > luigi >> > >> > >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> _______________________________________________ >> freebsd-net@freebsd.org >> mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org >> " >> > > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Sat Jun 28 13:08:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 74CADE5A for ; Sat, 28 Jun 2014 13:08:33 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C4CC25F8 for ; Sat, 28 Jun 2014 13:08:33 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id l6so5613113qcy.1 for ; Sat, 28 Jun 2014 06:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=DitvbK3YTvxFzKXyb87IuSSRHlb+e/L70AO97a03qcg=; b=xWD/a1z17LCWVi8niSDlSiH5L/VlNqUGDbfItdHwFPiZn1RwHh0Qd6J2OjMpIrU0WI XnE0ViqPEaXWUWUik109qasuzb+ZS5Q1d28MuX+eXQdVr5rbB9j+sR7Yem+hTwCgl6E9 E1ia5iSgy7YD5VvyMa0NkFhCPGuw1R1CSKqQjSIW4VJR2bRAjEELHGx5WWaspv5YdoAK oroPAUbPoYOPPd/ocRHvOhzGte3x9bT774FXz9tliC9wgnWbUJQs7xf9ENGrkUYr4jSl zT2Y1xsC6uUD3FkmyEJ7sH/oq2dzBPvapTgznsk/w6Tjo+QqhnVdURQ9rrVYYEgDInhP DMAw== X-Received: by 10.229.79.2 with SMTP id n2mr41805968qck.11.1403960912225; Sat, 28 Jun 2014 06:08:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Sat, 28 Jun 2014 06:07:52 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Sat, 28 Jun 2014 14:07:52 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 13:08:33 -0000 Hello to all. Unfortunately, I have been unable to do the test that Luigi requested due to the lack of hardware. All the Gateworks Cambria SBC that I have are currently allocated to another project. Only in about one week, will I have the opportunity to test it. As soon as I have news, I will post them. :) On 28 June 2014 09:24, Luigi Rizzo wrote: > > > On Saturday, June 28, 2014, Prashant Upadhyaya > wrote: > >> Hi, >> >> Any further news ? >> >> Professor Luigi, one question regarding the tx with netmap. >> Whenever, I write a packet from user space into netmap rings, and if I >> want netmap to send this out immediately, do I necessarily have to do >> a ioctl(fd, NIOCTXSYNC, NULL) ? >> > > Yes it is up to the application to decide when to push packets out with a > txsync or select() or poll(), and unfortunately there is a tradeoff betwe= en > efficiency and latency > > Cheers > Luigi > > >> >> I have an application which receives packets, does some processing and >> then sends them out. If I keep doing ioctl's on every packet send, then >> there will be too may system calls hitting performance, the application >> can't afford to block it has to return back to polling for the receipt o= f >> next packet. >> >> On the receive side, I see that I don't have a problem because I can pol= l >> the ring without initiating an RXSYNC and whenever in user space I find >> that there is nothing on the ring (probably half way down the ring size)= , I >> do an RXSYNC to get more packets thus saving system calls. >> >> But on tx side, I have noticed that unless I do a TXSYNC, the packet doe= s >> not go out, please let me know if I am missing something. >> >> Regards >> -Prashant >> >> >> >> On Tue, Jun 17, 2014 at 7:33 PM, Carlos Ferreira >> wrote: >> >>> Great! :) >>> I will give you the results as soon as I can get them :) >>> >>> >>> >>> On 17 June 2014 12:55, Luigi Rizzo wrote: >>> >>> > On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira < >>> carlosmf.pt@gmail.com> >>> > wrote: >>> > >>> >> Ok, thanks for the enlightenment regarding the loss of performance. >>> >> >>> >> One question, just to be sure. Does the kernel module contains the >>> VALA >>> >> switch code? Or do I need to compile extra code to have the switch >>> working? >>> >> Also, where can I find the documentation to use the Vala Switch? >>> >> >>> > >>> > =E2=80=8BVALE is part of the netmap kernel module, the only thing you= need >>> > to know to use it is port names: >>> > you can have multiple switch instances with multiple ports each, >>> > >>> > valeX:Y means port Y on switch X, X and Y are arbitrary strings >>> > with the constraint that the whole name must fit 15 characters. >>> > >>> > Details in the netmap manpage >>> > >>> > cheers >>> > luigi >>> > >>> > >>> >>> >>> -- >>> >>> Carlos Miguel Ferreira >>> Researcher at Telecommunications Institute >>> Aveiro - Portugal >>> Work E-mail - cmf@av.it.pt >>> Skype & GTalk -> carlosmf.pt@gmail.com >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Sat Jun 28 14:12:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2242865 for ; Sat, 28 Jun 2014 14:12:27 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EDE02A75 for ; Sat, 28 Jun 2014 14:12:27 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1X0tMZ-0008Ug-HN for freebsd-net@freebsd.org; Sat, 28 Jun 2014 07:12:19 -0700 Date: Sat, 28 Jun 2014 07:12:19 -0700 (PDT) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <1403964739522-5924518.post@n5.nabble.com> Subject: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 14:12:27 -0000 I'm using grub2 as the pxe bootloader rather than BTX's pxeboot. I get Grub to load kernel and all necessary modules and boot. It goes as far as mountroot> and stops. "?" fails to show any pxe devices. If I recall correctly, when booting from BTX and upon hitting a mountroot problem, "?" is able to show pxe device. The kernel modules {nfscl.ko, nfsclient.ko, nfscommon.ko} are loaded through grub.cfg kfreebsd_module entries. Using the BTX pxeboot system boots and mounts everything normally. At this point, I could only think of two possibilities: A) the grub.cfg entry is wrong (which I don't think so since as stated above mountroot> "?" shows no available pxe devices). Or B) the more plausible reason is that I need to load some other module which will make the pxe devices become visible to the mountroot process. Grub's normal solution to this problem (pxechainloader $path/pxeboot) has been broken for some time unfortunately. I'd like to make sure that I am loading all the necessary modules through grub.cfg (for diskless clients) before heading over to the grub mail list with this problem. Server-side loaded modules: {nfsd.ko, nfslockd.ko, nfsserver.ko, nfs_common.ko, acl_nfs4.ko} within kernel: {nfscommon, nfssvc, nfs, nfscl, nfslock}. As a side comment, under BTX as long as one has "option root-path" in dhcpd.conf and a correct fsatb entry for "/", none of the below are needed in loader.conf - system is able to boot without these and mount ro (from fstab): boot.nfsroot.server="192.168.2.1" boot.nfsroot.path="/data/amd6" vfs.root.mountfrom="nfs:192.168.2.1:/data/amd64" vfs.root.mountfrom="nfs" Regards. ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sat Jun 28 20:53:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C886660A for ; Sat, 28 Jun 2014 20:53:29 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8B6332731 for ; Sat, 28 Jun 2014 20:53:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFAMIqr1ODaFve/2dsb2JhbABag19agm6oFAEBAQEBAQaSMIZtSQoBgR91hAMBAQEDAQEBASArIAsFFhgCAg0ZAikBCSYGCAcEARoCBIgNAwkIDahalj8XhkgXgSuEOYhQAQEbNAeCd4FMBZdzhDCSNYNeIS8BAQSBBTk X-IronPort-AV: E=Sophos;i="5.01,568,1400040000"; d="scan'208";a="136360767" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 28 Jun 2014 16:52:57 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D3B5FB40FD; Sat, 28 Jun 2014 16:52:57 -0400 (EDT) Date: Sat, 28 Jun 2014 16:52:57 -0400 (EDT) From: Rick Macklem To: Beeblebrox Message-ID: <1186767237.5285620.1403988777812.JavaMail.root@uoguelph.ca> In-Reply-To: <1403964739522-5924518.post@n5.nabble.com> Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 20:53:30 -0000 Beeblebrox wrote: > I'm using grub2 as the pxe bootloader rather than BTX's pxeboot. > > I get Grub to load kernel and all necessary modules and boot. It goes > as far > as mountroot> and stops. "?" fails to show any pxe devices. If I > recall > correctly, when booting from BTX and upon hitting a mountroot > problem, "?" > is able to show pxe device. The kernel modules {nfscl.ko, > nfsclient.ko, > nfscommon.ko} are loaded through grub.cfg kfreebsd_module entries. > Using the > BTX pxeboot system boots and mounts everything normally. > > At this point, I could only think of two possibilities: A) the > grub.cfg > entry is wrong (which I don't think so since as stated above > mountroot> "?" > shows no available pxe devices). Or B) the more plausible reason is > that I > need to load some other module which will make the pxe devices become > visible to the mountroot process. Grub's normal solution to this > problem > (pxechainloader $path/pxeboot) has been broken for some time > unfortunately. > > I'd like to make sure that I am loading all the necessary modules > through > grub.cfg (for diskless clients) before heading over to the grub mail > list > with this problem. Server-side loaded modules: {nfsd.ko, nfslockd.ko, > nfsserver.ko, nfs_common.ko, acl_nfs4.ko} within kernel: {nfscommon, > nfssvc, > nfs, nfscl, nfslock}. > I suspect it is linked into the kernel, but you'll need krpc.ko as well. Btw, if your kernel is built with "options NFS_ROOT", it expects the structure called nfsv3_diskless to be filled in (FreeBSD's pxeboot does this). If nfsv3_diskless isn't being filled in by pxeboot or similar, you need to build your kernel with: options BOOTP options BOOTP_NFSROOT so that the kernel will use dhcp and NFS to acquire the information instead of expecting it to be filled into nfsv3_diskless. (The code that this uses is in sys/nfs/bootp_subr.c.) rick > As a side comment, under BTX as long as one has "option root-path" in > dhcpd.conf and a correct fsatb entry for "/", none of the below are > needed > in loader.conf - system is able to boot without these and mount ro > (from > fstab): > boot.nfsroot.server="192.168.2.1" > boot.nfsroot.path="/data/amd6" > vfs.root.mountfrom="nfs:192.168.2.1:/data/amd64" > vfs.root.mountfrom="nfs" > > Regards. > > > > ----- > FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS > -- > View this message in context: > http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518.html > Sent from the freebsd-net mailing list archive at Nabble.com. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Jun 29 11:35:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18074FCE for ; Sun, 29 Jun 2014 11:35:02 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED47021DB for ; Sun, 29 Jun 2014 11:35:01 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1X1DNr-0005hR-QU for freebsd-net@freebsd.org; Sun, 29 Jun 2014 04:34:59 -0700 Date: Sun, 29 Jun 2014 04:34:59 -0700 (PDT) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <1404041699791-5924681.post@n5.nabble.com> In-Reply-To: <1186767237.5285620.1403988777812.JavaMail.root@uoguelph.ca> References: <1403964739522-5924518.post@n5.nabble.com> <1186767237.5285620.1403988777812.JavaMail.root@uoguelph.ca> Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jun 2014 11:35:02 -0000 Thanks Rick for the insight. I complied a fresh kernel with {options BOOTP, options BOOTP_NFSROOT} as you suggested. The new kernel behaves differently but does not solve the problem. BTX's pxeboot still works normally, but when trying to use Grub the client now reboots at the end of kernel loading Instead of dropping to "mountroot>". After many trials and reboots, I was barely able to glimpse a message like "panic: bootpc-init ...." (the reboot is immediate, no keystroke prevents it). Looking through sys/nfs/bootp_subr.c did not give me much of a clue as to the reason for this. Regards. ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518p5924681.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sun Jun 29 19:50:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2AA0BB4 for ; Sun, 29 Jun 2014 19:50:14 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A3B712426 for ; Sun, 29 Jun 2014 19:50:14 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1X1L71-0000kF-Dl for freebsd-net@freebsd.org; Sun, 29 Jun 2014 12:50:07 -0700 Date: Sun, 29 Jun 2014 12:50:07 -0700 (PDT) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <1404071407420-5924767.post@n5.nabble.com> In-Reply-To: <1404041699791-5924681.post@n5.nabble.com> References: <1403964739522-5924518.post@n5.nabble.com> <1186767237.5285620.1403988777812.JavaMail.root@uoguelph.ca> <1404041699791-5924681.post@n5.nabble.com> Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jun 2014 19:50:14 -0000 >> when trying to use Grub the client now reboots at the end of kernel loading Sooo, I got curious and compiled a full debug kernel that included {options BOOTP, options BOOTP_NFSROOT} knobs. From the grub menu, without changing any grub.cfg lines (as previously posted) - hit go and ... YEAH BABY! System boots and mounts to the exported 11-current_amd64 NFS root. Two observations. * The boot process is a little slow and shows a number of error messages. Hangs for a while before moving on to the final good result. Can share if anyone needs the output (advise how to capture for PXE-boot) * My custom kernel config file obviously is missing something(s). How can I find what they may be? For example I normally have {NFSD, NFSLOCKD} loaded as modules for if/when I need them, and a part of MYKERNEL - could this be a cause? Thanks for the help & regards. ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518p5924767.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sun Jun 29 21:24:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 929F3C51 for ; Sun, 29 Jun 2014 21:24:29 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 5A1352ACF for ; Sun, 29 Jun 2014 21:24:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhgFAFB7sFODaFve/2dsb2JhbABag19agm6oHgEBAQEBAQaSdYZtUwGBHnWEAwEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASIDQMJCA2pWJV1F4ZIF4ErhDmIUgEBGzQHgneBTAWXdIQwkjeDXiEvAQEEgQU5 X-IronPort-AV: E=Sophos;i="5.01,572,1400040000"; d="scan'208";a="136000462" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 29 Jun 2014 17:24:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 422FDB405A; Sun, 29 Jun 2014 17:24:22 -0400 (EDT) Date: Sun, 29 Jun 2014 17:24:22 -0400 (EDT) From: Rick Macklem To: Beeblebrox Message-ID: <1270654408.5439540.1404077062258.JavaMail.root@uoguelph.ca> In-Reply-To: <1404071407420-5924767.post@n5.nabble.com> Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jun 2014 21:24:29 -0000 Beeblebrox wrote: > >> when trying to use Grub the client now reboots at the end of > >> kernel > loading > > Sooo, I got curious and compiled a full debug kernel that included > {options > BOOTP, options BOOTP_NFSROOT} knobs. From the grub menu, without > changing > any grub.cfg lines (as previously posted) - hit go and ... YEAH BABY! > System > boots and mounts to the exported 11-current_amd64 NFS root. > > Two observations. > * The boot process is a little slow and shows a number of error > messages. > Hangs for a while before moving on to the final good result. Can > share if > anyone needs the output (advise how to capture for PXE-boot) > * My custom kernel config file obviously is missing something(s). How > can I > find what they may be? For example I normally have {NFSD, NFSLOCKD} > loaded > as modules for if/when I need them, and a part of MYKERNEL - could > this be a > cause? > Well NFSD is only used if you start the nfsd daemon/server. NFSLOCKD (rpc.lockd, rpc.statd) will get used if you start either of those daemons. For NFSv3 mounts, I'd suggest the nolockd option, unless you have multiple clients concurrently doing byte range locking on the same file. (With "nolockd" option on the mounts, you shouldn't need to run rpc.lockd, rpc.statd and that implies NFSLOCKD shouldn't be needed, too.) Btw, there is BOOTP_DEBUG stuff in bootp_subr.c. If your debug kernel didn't include that option, trying it might tell you where it breaks? rick > Thanks for the help & regards. > > > > ----- > FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS > -- > View this message in context: > http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518p5924767.html > Sent from the freebsd-net mailing list archive at Nabble.com. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 08:00:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C05680E for ; Mon, 30 Jun 2014 08:00:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 305FA2924 for ; Mon, 30 Jun 2014 08:00:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5U80Dcb004089 for ; Mon, 30 Jun 2014 09:00:13 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201406300800.s5U80Dcb004089@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 30 Jun 2014 08:00:13 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 08:00:13 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 08:25:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA9E5184 for ; Mon, 30 Jun 2014 08:25:12 +0000 (UTC) Received: from mail-lb0-f179.google.com (mail-lb0-f179.google.com [209.85.217.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 753D62BC3 for ; Mon, 30 Jun 2014 08:25:11 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so5595878lbi.38 for ; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=RUSrV5+b2yvE+D2m1EU9IvnwQrMbfSZD9TnUaBI8800=; b=htsdOHSo78JT7A5hZHigTafJPwUL18OtMd2mi6sEm94mXEt/sF4Pn2wH+JPagY4Ijy wj1eDM6/HIAQSICPcuWMpkU7kVn2xTjkLAmwKwIe2D3J9pocee3/YVEK6yf7SYVd041V D4WktiGLk7dAAu7rg/rsN8aN1aABt/sQa9nkiqdJXnkQezwBLPC+JgHGPoNTZzZM2+Uu grnGCwhWpRjaqw5XQTpUihWVdTJqcQNlbDlZADqB6ezVzi3gPMQUUdtdqzI+l+RVvHKR Sj+eeu/eYR9VRx23Yepc/1xpA+Xud6m/ntiUGranbCVdzfZE6pE6SrurNZ785hrXdXAf WZGw== X-Gm-Message-State: ALoCoQmwwoIlz5avhT2DUo3IwzYtEU3q2qZ3btewjY2m09bBdw86YK7bc2lHH9MPrZeB3MFcKtAr MIME-Version: 1.0 X-Received: by 10.112.182.36 with SMTP id eb4mr234139lbc.99.1404116704079; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) Received: by 10.112.99.4 with HTTP; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) Date: Mon, 30 Jun 2014 10:25:04 +0200 Message-ID: Subject: ixgbe queue hang fixed in 9.3? From: Johan Kooijman To: freebsd-stable@freebsd.org, FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 08:25:13 -0000 Hey all, A while ago I started the thread "9.2 ixgbe tx queue hang" on freebsd-net. A lot more people had the same as I had. Could anybody tell me if this issue is resolved in 9.3 RC? I haven't been able to find anything about it. -- Met vriendelijke groeten / With kind regards, Johan Kooijman From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 17:04:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93325F62 for ; Mon, 30 Jun 2014 17:04:46 +0000 (UTC) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D5FE2D4D for ; Mon, 30 Jun 2014 17:04:45 +0000 (UTC) Received: by mail-we0-f173.google.com with SMTP id t60so8540972wes.4 for ; Mon, 30 Jun 2014 10:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:hackerspace:user-agent; bh=qonc10JhApLF3wDev1vMQKztDHY7NGA0CF5NdyK7ttA=; b=1KIJKYmKaJ2Q9iJjlm1cjJj4wigdHK+H6oFgo83kHRLGdFFAIt/spVWUa6pvydl767 nVyu7aT9UvNDbfof14frSjK9IUkHpfcipUFqJQz8M98zwHsREpcg8ci1gVrsVkRCSZbW E4RZlgQgERpCfXGwcxPYr4esql3Ztx1aLhsXKE1dQepvMQ/DX7vTZegiOgFdyEfT4Bq6 lkwMRWgEO6lwl6dcqP4jf2e7331GZuLM1vNz9AMtDfnnBvJ3p+UFSpECJXE0IGYZslSg qRz/Fnfl9XzmwhVkuIh75EejodH0/GxEuieeHopwEVaeBOI3UjoylcELykiHvjzUCqtd 4O8A== X-Received: by 10.180.81.1 with SMTP id v1mr30537326wix.10.1404147884031; Mon, 30 Jun 2014 10:04:44 -0700 (PDT) Received: from gmail.com ([2001:630:241:204:9c18:7fce:d679:ccea]) by mx.google.com with ESMTPSA id f6sm42468686wja.25.2014.06.30.10.04.42 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jun 2014 10:04:43 -0700 (PDT) Sender: Tom Jones Date: Mon, 30 Jun 2014 18:04:56 +0100 From: Tom Jones To: freebsd-net@freebsd.org Subject: [PATCH] Implementation of draft-ietf-tcpm-newcwv-06 Message-ID: <20140630170453.GA21404@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="SLDf9lqlvOQaIe6s" Content-Disposition: inline Hackerspace: 57North Hacklab User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 17:04:46 -0000 --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, Attached is a patch which implements draft-ietf-tcpm-newcwv-06, "Updating TCP to support Rate-Limited Traffic". The patch is a port of the Linux implementation by Raffaello Secchi with influence from Aris Angelogiannopoulos's patch set that was sent to the list earlier this year. -- Tom | I don't see how we are going to build the dystopian megapoli @adventureloop | we were promised in 80s and early 90s cyberpunk fiction if adventurist.me | you guys keep complaining. --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Content-Disposition: inline; filename="newcwv.patch" Index: sys/conf/files =================================================================== --- sys/conf/files (revision 268040) +++ sys/conf/files (working copy) @@ -3380,6 +3380,7 @@ netinet/tcp_reass.c optional inet | inet6 netinet/tcp_sack.c optional inet | inet6 netinet/tcp_subr.c optional inet | inet6 netinet/tcp_syncache.c optional inet | inet6 +netinet/tcp_newcwv.c optional inet | inet6 netinet/tcp_timer.c optional inet | inet6 netinet/tcp_timewait.c optional inet | inet6 netinet/tcp_usrreq.c optional inet | inet6 Index: sys/netinet/tcp_input.c =================================================================== --- sys/netinet/tcp_input.c (revision 268040) +++ sys/netinet/tcp_input.c (working copy) @@ -105,6 +105,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef TCPDEBUG #include #endif /* TCPDEBUG */ @@ -174,6 +175,11 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, abc_l_var &VNET_NAME(tcp_abc_l_var), 2, "Cap the max cwnd increment during slow-start to this number of segments"); +VNET_DEFINE(int, tcp_do_newcwv) = 0; +SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, newcwv, CTLFLAG_RW, + &VNET_NAME(tcp_do_newcwv), 0, + "Enable draft-ietf-tcpm-newcwv-06 (New Congestion Window Validation)"); + static SYSCTL_NODE(_net_inet_tcp, OID_AUTO, ecn, CTLFLAG_RW, 0, "TCP ECN"); VNET_DEFINE(int, tcp_do_ecn) = 0; @@ -309,6 +315,12 @@ cc_ack_received(struct tcpcb *tp, struct tcphdr *t tp->ccv->curack = th->th_ack; CC_ALGO(tp)->ack_received(tp->ccv, type); } + + /* + * update draft-ietf-newcwv-06 pipeack + */ + if(V_tcp_do_newcwv && !IN_FASTRECOVERY(tp->t_flags)) + tcp_newcwv_update_pipeack(tp); } static void inline @@ -378,6 +390,12 @@ cc_conn_init(struct tcpcb *tp) tp->snd_cwnd = 4 * tp->t_maxseg; } + /* + * Initialise NewCWV state + */ + tp->init_cwnd = tp->snd_cwnd; + tcp_newcwv_reset(tp); + if (CC_ALGO(tp)->conn_init != NULL) CC_ALGO(tp)->conn_init(tp->ccv); } @@ -426,6 +444,11 @@ cc_cong_signal(struct tcpcb *tp, struct tcphdr *th tp->t_badrxtwin = 0; break; } + + if (V_tcp_do_newcwv && + (type == CC_NDUPACK || type == CC_ECN) && + tp->pipeack <= (tp->snd_cwnd >> 1) ) + tcp_newcwv_enter_recovery(tp); if (CC_ALGO(tp)->cong_signal != NULL) { if (th != NULL) @@ -447,6 +470,13 @@ cc_post_recovery(struct tcpcb *tp, struct tcphdr * } /* XXXLAS: EXIT_RECOVERY ? */ tp->t_bytes_acked = 0; + + if(V_tcp_do_newcwv) { + if(tp->loss_flight_size) + tcp_newcwv_end_recovery(tp); + tcp_newcwv_reset(tp); + } + tp->loss_flight_size = 0; } #ifdef TCP_SIGNATURE Index: sys/netinet/tcp_newcwv.c =================================================================== --- sys/netinet/tcp_newcwv.c (revision 0) +++ sys/netinet/tcp_newcwv.c (working copy) @@ -0,0 +1,174 @@ +/* + * Copyright (c) 2014 Tom Jones + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +#include + +#include +#include +#include +#include +#include + +#include +#include +#include + +/* + * An implementation of NewCWV (draft-ietf-tcpm-newcwv-06) for FreeBSD. + * Based on the Linux implementation by Raffaello Secchi and the an initial + * implementation of draft-ietf-tcpm-newcwv-00 by Aris Angelogiannopoulos. + */ + +#define nextbin(x) (((x)+1) & 0x03) +#define prevbin(x) (((x)-1) & 0x03) + +#define NCWV_UNDEF 0xFFFFFFFF +#define NCWV_FIVEMINS (300*hz) + +void add_element(struct tcpcb *,u_int32_t); +u_int32_t remove_expired_elements(struct tcpcb *); + +void +tcp_newcwv_update_pipeack(struct tcpcb *tp) +{ + u_int32_t tmp_pipeack; + tp->psp = MAX(3 * tp->t_srtt,hz); + + if (tp->snd_una >= tp->prev_snd_nxt) { + /* get the pipeack sample */ + tmp_pipeack = tp->snd_una - tp->prev_snd_una; + + tp->prev_snd_una = tp->snd_una; + tp->prev_snd_nxt = tp->snd_nxt; + + /* create a new element at the end of current pmp */ + if(ticks > tp->time_stamp[tp->head] + (tp->psp >> 2)) + add_element(tp,tmp_pipeack); + else + tp->psample[tp->head] = tmp_pipeack; + } + + tp->pipeack = remove_expired_elements(tp); + + /* check if cwnd is validated */ + if (tp->pipeack == NCWV_UNDEF || + ((tp->pipeack << 1) >= (tp->snd_cwnd * tp->t_maxseg))) { + tp->cwnd_valid_ts = ticks; + } +} + +void +add_element(struct tcpcb *tp,u_int32_t value) +{ + tp->head = nextbin(tp->head); + tp->psample[tp->head] = value; + tp->time_stamp[tp->head] = ticks; +} + +u_int32_t +remove_expired_elements(struct tcpcb *tp) +{ + uint8_t head = tp->head; + u_int32_t tmp = tp->psample[head]; + + while(tp->psample[head] != NCWV_UNDEF) { + /* remove the element if expired */ + if (tp->time_stamp[head] < ticks - tp->psp) { + tp->psample[head] = NCWV_UNDEF; + return tmp; + } + + /* search for the max pipeack */ + if(tp->psample[head] > tmp) + tmp = tp->psample[head]; + + head = prevbin(head); + if(head == tp->head) + return tmp; + } + + return tmp; +} + +/* Initialise NewCWV state */ +void +tcp_newcwv_reset(struct tcpcb *tp) +{ + tp->prev_snd_una = tp->snd_una; + tp->prev_snd_nxt = tp->snd_nxt; + tp->cwnd_valid_ts = ticks; + tp->loss_flight_size = 0; + + tp->head = 0; + tp->psample[0] = NCWV_UNDEF; + tp->pipeack = NCWV_UNDEF; +} + +/* NewCWV actions at loss detection */ +void +tcp_newcwv_enter_recovery(struct tcpcb *tp) +{ + u_int32_t pipe; + + if(tp->pipeack == NCWV_UNDEF) + return; + + tp->prior_retrans = tp->t_sndrexmitpack; + + /* Calculate the flight size */ + u_int32_t awnd = (tp->snd_nxt - tp->snd_fack) + tp->sackhint.sack_bytes_rexmit; + tp->loss_flight_size = awnd; + + pipe = MAX(tp->pipeack,tp->loss_flight_size); + tp->snd_cwnd = MAX(pipe >> 1,1); +} + +/* NewCWV actions at the end of recovery */ +void +tcp_newcwv_end_recovery(struct tcpcb *tp) +{ + u_int32_t retrans,pipe; + + retrans = (tp->t_sndrexmitpack - tp->prior_retrans) * tp->t_maxseg; + pipe = MAX(tp->pipeack,tp->loss_flight_size) - retrans; + + /* Ensure that snd_ssthresh is non 0 */ + tp->snd_ssthresh = MAX(pipe >> 1,1); + tp->snd_cwnd = tp->snd_ssthresh; +} + +void +tcp_newcwv_datalim_closedown(struct tcpcb *tp) +{ + while ((ticks - tp->cwnd_valid_ts) > NCWV_FIVEMINS && + tp->snd_cwnd > tp->init_cwnd) { + + tp->cwnd_valid_ts += NCWV_FIVEMINS; + tp->snd_ssthresh = MAX( (3 * tp->snd_cwnd ) >> 2,tp->snd_ssthresh); + tp->snd_cwnd = MAX(tp->snd_cwnd >> 1, tp->init_cwnd); + } +} Property changes on: sys/netinet/tcp_newcwv.c ___________________________________________________________________ Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Index: sys/netinet/tcp_newcwv.h =================================================================== --- sys/netinet/tcp_newcwv.h (revision 0) +++ sys/netinet/tcp_newcwv.h (working copy) @@ -0,0 +1,39 @@ +/* + * Copyright (c) 2014 Tom Jones + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +#ifndef _NETINET_TCP_NEWCWV_H_ +#define _NETINET_TCP_NEWCWV_H_ + +#include + +void tcp_newcwv_update_pipeack(struct tcpcb *); +void tcp_newcwv_reset(struct tcpcb *); +void tcp_newcwv_enter_recovery(struct tcpcb *); +void tcp_newcwv_end_recovery(struct tcpcb *); +void tcp_newcwv_datalim_closedown(struct tcpcb *); + +#endif /* _NETINET_TCP_NEWCWV_H_ */ Property changes on: sys/netinet/tcp_newcwv.h ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: sys/netinet/tcp_output.c =================================================================== --- sys/netinet/tcp_output.c (revision 268040) +++ sys/netinet/tcp_output.c (working copy) @@ -74,6 +74,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef TCPDEBUG #include #endif @@ -691,6 +692,10 @@ send: #endif hdrlen = sizeof (struct tcpiphdr); + /* Trigger the newcwv timer */ + if(V_tcp_do_newcwv) + tcp_newcwv_datalim_closedown(tp); + /* * Compute options for segment. * We only have to care about SYN and established connection Index: sys/netinet/tcp_subr.c =================================================================== --- sys/netinet/tcp_subr.c (revision 268040) +++ sys/netinet/tcp_subr.c (working copy) @@ -800,6 +800,7 @@ tcp_newtcpcb(struct inpcb *inp) tp->t_flags = (TF_REQ_SCALE|TF_REQ_TSTMP); if (V_tcp_do_sack) tp->t_flags |= TF_SACK_PERMIT; + TAILQ_INIT(&tp->snd_holes); tp->t_inpcb = inp; /* XXX */ /* Index: sys/netinet/tcp_var.h =================================================================== --- sys/netinet/tcp_var.h (revision 268040) +++ sys/netinet/tcp_var.h (working copy) @@ -172,6 +172,20 @@ struct tcpcb { int t_sndzerowin; /* zero-window updates sent */ u_int t_badrxtwin; /* window for retransmit recovery */ u_char snd_limited; /* segments limited transmitted */ +/* NewCWV releated state */ + u_int32_t pipeack; + u_int32_t psp; /* pipeack sampling period */ + + u_int32_t head; + u_int32_t psample[4]; /* pipe ack samples */ + u_int32_t time_stamp[4]; /* time stamp samples */ + u_int32_t prev_snd_una; /* previous snd_una in this sampe */ + u_int32_t prev_snd_nxt; /* previous snd_nxt in this sampe */ + + u_int32_t loss_flight_size; /* flightsize at loss detection */ + u_int32_t prior_retrans; /* Retransmission before going into FR */ + u_int32_t cwnd_valid_ts; /*last time cwnd was found valid */ + u_int32_t init_cwnd; /* The inital cwnd */ /* SACK related state */ int snd_numholes; /* number of holes seen by sender */ TAILQ_HEAD(sackhole_head, sackhole) snd_holes; @@ -605,6 +619,7 @@ VNET_DECLARE(int, tcp_recvspace); VNET_DECLARE(int, path_mtu_discovery); VNET_DECLARE(int, tcp_do_rfc3465); VNET_DECLARE(int, tcp_abc_l_var); +VNET_DECLARE(int, tcp_do_newcwv); #define V_tcb VNET(tcb) #define V_tcbinfo VNET(tcbinfo) #define V_tcp_mssdflt VNET(tcp_mssdflt) @@ -617,6 +632,7 @@ VNET_DECLARE(int, tcp_abc_l_var); #define V_path_mtu_discovery VNET(path_mtu_discovery) #define V_tcp_do_rfc3465 VNET(tcp_do_rfc3465) #define V_tcp_abc_l_var VNET(tcp_abc_l_var) +#define V_tcp_do_newcwv VNET(tcp_do_newcwv) VNET_DECLARE(int, tcp_do_sack); /* SACK enabled/disabled */ VNET_DECLARE(int, tcp_sc_rst_sock_fail); /* RST on sock alloc failure */ --SLDf9lqlvOQaIe6s-- From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 19:29:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50F90D7C for ; Mon, 30 Jun 2014 19:29:45 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 132022B76 for ; Mon, 30 Jun 2014 19:29:45 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id i8so7529509qcq.34 for ; Mon, 30 Jun 2014 12:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=r2ozJKl0RKHRlENWg0+nUm0B97kay8ZcqGMLeEQC9I8=; b=ln0Flr08LDbwSVrE3eaD20aPwQkrYJbcX5+1tL3e/DaaW9sxpRtmmGQ+ipfciHaGYJ dL9b3pc7ieowTh7xU6NQGKIZRB5vnQNiZDlq7oxgVBWxMjLe9bT2V/02xFia7s6uojAJ QZsSLhsImFw02axiPXz3kCSUn2n73lmO14tvVCsWegOfKhgJvRNelULhVU2It+065+Xk JnB1Dzt7Ka0LjQ65Zz0xOS62Nj2/8LkIY/2fwadrZf6hGtnWr+astnqeLZy40qhllil8 zQVPy4E/uAFSqC6TeIH2r/Ukk2FQTjEDGObqOQdTH/fe0TvgaUGJaK4xKt/CDpOtEVBX Zgbg== MIME-Version: 1.0 X-Received: by 10.140.34.195 with SMTP id l61mr61139738qgl.87.1404156584196; Mon, 30 Jun 2014 12:29:44 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Mon, 30 Jun 2014 12:29:44 -0700 (PDT) In-Reply-To: <20140630170453.GA21404@gmail.com> References: <20140630170453.GA21404@gmail.com> Date: Mon, 30 Jun 2014 12:29:44 -0700 X-Google-Sender-Auth: YU4QrLTBDtn-y9PSXxK_G8J9fEI Message-ID: Subject: Re: [PATCH] Implementation of draft-ietf-tcpm-newcwv-06 From: Adrian Chadd To: Tom Jones Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 19:29:45 -0000 Hi! Cool! Would you mind throwing it into a bugzilla ticket so it's not lost and it can be assigned for some review? http://bugs.freebsd.org/submit/ Is there any benefit in hanging this state off of it as a pointer to some other struct, rather than having it in the tcpcb? Is it always going to be used? Thanks! -a On 30 June 2014 10:04, Tom Jones wrote: > Hello, > > Attached is a patch which implements draft-ietf-tcpm-newcwv-06, "Updating TCP > to support Rate-Limited Traffic". The patch is a port of the Linux > implementation by Raffaello Secchi with influence from Aris > Angelogiannopoulos's patch set that was sent to the list earlier this year. > > -- > Tom | I don't see how we are going to build the dystopian megapoli > @adventureloop | we were promised in 80s and early 90s cyberpunk fiction if > adventurist.me | you guys keep complaining. > > Index: sys/conf/files > =================================================================== > --- sys/conf/files (revision 268040) > +++ sys/conf/files (working copy) > @@ -3380,6 +3380,7 @@ netinet/tcp_reass.c optional inet | inet6 > netinet/tcp_sack.c optional inet | inet6 > netinet/tcp_subr.c optional inet | inet6 > netinet/tcp_syncache.c optional inet | inet6 > +netinet/tcp_newcwv.c optional inet | inet6 > netinet/tcp_timer.c optional inet | inet6 > netinet/tcp_timewait.c optional inet | inet6 > netinet/tcp_usrreq.c optional inet | inet6 > Index: sys/netinet/tcp_input.c > =================================================================== > --- sys/netinet/tcp_input.c (revision 268040) > +++ sys/netinet/tcp_input.c (working copy) > @@ -105,6 +105,7 @@ __FBSDID("$FreeBSD$"); > #include > #include > #include > +#include > #ifdef TCPDEBUG > #include > #endif /* TCPDEBUG */ > @@ -174,6 +175,11 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, abc_l_var > &VNET_NAME(tcp_abc_l_var), 2, > "Cap the max cwnd increment during slow-start to this number of segments"); > > +VNET_DEFINE(int, tcp_do_newcwv) = 0; > +SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, newcwv, CTLFLAG_RW, > + &VNET_NAME(tcp_do_newcwv), 0, > + "Enable draft-ietf-tcpm-newcwv-06 (New Congestion Window Validation)"); > + > static SYSCTL_NODE(_net_inet_tcp, OID_AUTO, ecn, CTLFLAG_RW, 0, "TCP ECN"); > > VNET_DEFINE(int, tcp_do_ecn) = 0; > @@ -309,6 +315,12 @@ cc_ack_received(struct tcpcb *tp, struct tcphdr *t > tp->ccv->curack = th->th_ack; > CC_ALGO(tp)->ack_received(tp->ccv, type); > } > + > + /* > + * update draft-ietf-newcwv-06 pipeack > + */ > + if(V_tcp_do_newcwv && !IN_FASTRECOVERY(tp->t_flags)) > + tcp_newcwv_update_pipeack(tp); > } > > static void inline > @@ -378,6 +390,12 @@ cc_conn_init(struct tcpcb *tp) > tp->snd_cwnd = 4 * tp->t_maxseg; > } > > + /* > + * Initialise NewCWV state > + */ > + tp->init_cwnd = tp->snd_cwnd; > + tcp_newcwv_reset(tp); > + > if (CC_ALGO(tp)->conn_init != NULL) > CC_ALGO(tp)->conn_init(tp->ccv); > } > @@ -426,6 +444,11 @@ cc_cong_signal(struct tcpcb *tp, struct tcphdr *th > tp->t_badrxtwin = 0; > break; > } > + > + if (V_tcp_do_newcwv && > + (type == CC_NDUPACK || type == CC_ECN) && > + tp->pipeack <= (tp->snd_cwnd >> 1) ) > + tcp_newcwv_enter_recovery(tp); > > if (CC_ALGO(tp)->cong_signal != NULL) { > if (th != NULL) > @@ -447,6 +470,13 @@ cc_post_recovery(struct tcpcb *tp, struct tcphdr * > } > /* XXXLAS: EXIT_RECOVERY ? */ > tp->t_bytes_acked = 0; > + > + if(V_tcp_do_newcwv) { > + if(tp->loss_flight_size) > + tcp_newcwv_end_recovery(tp); > + tcp_newcwv_reset(tp); > + } > + tp->loss_flight_size = 0; > } > > #ifdef TCP_SIGNATURE > Index: sys/netinet/tcp_newcwv.c > =================================================================== > --- sys/netinet/tcp_newcwv.c (revision 0) > +++ sys/netinet/tcp_newcwv.c (working copy) > @@ -0,0 +1,174 @@ > +/* > + * Copyright (c) 2014 Tom Jones > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + */ > + > +#include > + > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > + > +/* > + * An implementation of NewCWV (draft-ietf-tcpm-newcwv-06) for FreeBSD. > + * Based on the Linux implementation by Raffaello Secchi and the an initial > + * implementation of draft-ietf-tcpm-newcwv-00 by Aris Angelogiannopoulos. > + */ > + > +#define nextbin(x) (((x)+1) & 0x03) > +#define prevbin(x) (((x)-1) & 0x03) > + > +#define NCWV_UNDEF 0xFFFFFFFF > +#define NCWV_FIVEMINS (300*hz) > + > +void add_element(struct tcpcb *,u_int32_t); > +u_int32_t remove_expired_elements(struct tcpcb *); > + > +void > +tcp_newcwv_update_pipeack(struct tcpcb *tp) > +{ > + u_int32_t tmp_pipeack; > + tp->psp = MAX(3 * tp->t_srtt,hz); > + > + if (tp->snd_una >= tp->prev_snd_nxt) { > + /* get the pipeack sample */ > + tmp_pipeack = tp->snd_una - tp->prev_snd_una; > + > + tp->prev_snd_una = tp->snd_una; > + tp->prev_snd_nxt = tp->snd_nxt; > + > + /* create a new element at the end of current pmp */ > + if(ticks > tp->time_stamp[tp->head] + (tp->psp >> 2)) > + add_element(tp,tmp_pipeack); > + else > + tp->psample[tp->head] = tmp_pipeack; > + } > + > + tp->pipeack = remove_expired_elements(tp); > + > + /* check if cwnd is validated */ > + if (tp->pipeack == NCWV_UNDEF || > + ((tp->pipeack << 1) >= (tp->snd_cwnd * tp->t_maxseg))) { > + tp->cwnd_valid_ts = ticks; > + } > +} > + > +void > +add_element(struct tcpcb *tp,u_int32_t value) > +{ > + tp->head = nextbin(tp->head); > + tp->psample[tp->head] = value; > + tp->time_stamp[tp->head] = ticks; > +} > + > +u_int32_t > +remove_expired_elements(struct tcpcb *tp) > +{ > + uint8_t head = tp->head; > + u_int32_t tmp = tp->psample[head]; > + > + while(tp->psample[head] != NCWV_UNDEF) { > + /* remove the element if expired */ > + if (tp->time_stamp[head] < ticks - tp->psp) { > + tp->psample[head] = NCWV_UNDEF; > + return tmp; > + } > + > + /* search for the max pipeack */ > + if(tp->psample[head] > tmp) > + tmp = tp->psample[head]; > + > + head = prevbin(head); > + if(head == tp->head) > + return tmp; > + } > + > + return tmp; > +} > + > +/* Initialise NewCWV state */ > +void > +tcp_newcwv_reset(struct tcpcb *tp) > +{ > + tp->prev_snd_una = tp->snd_una; > + tp->prev_snd_nxt = tp->snd_nxt; > + tp->cwnd_valid_ts = ticks; > + tp->loss_flight_size = 0; > + > + tp->head = 0; > + tp->psample[0] = NCWV_UNDEF; > + tp->pipeack = NCWV_UNDEF; > +} > + > +/* NewCWV actions at loss detection */ > +void > +tcp_newcwv_enter_recovery(struct tcpcb *tp) > +{ > + u_int32_t pipe; > + > + if(tp->pipeack == NCWV_UNDEF) > + return; > + > + tp->prior_retrans = tp->t_sndrexmitpack; > + > + /* Calculate the flight size */ > + u_int32_t awnd = (tp->snd_nxt - tp->snd_fack) + tp->sackhint.sack_bytes_rexmit; > + tp->loss_flight_size = awnd; > + > + pipe = MAX(tp->pipeack,tp->loss_flight_size); > + tp->snd_cwnd = MAX(pipe >> 1,1); > +} > + > +/* NewCWV actions at the end of recovery */ > +void > +tcp_newcwv_end_recovery(struct tcpcb *tp) > +{ > + u_int32_t retrans,pipe; > + > + retrans = (tp->t_sndrexmitpack - tp->prior_retrans) * tp->t_maxseg; > + pipe = MAX(tp->pipeack,tp->loss_flight_size) - retrans; > + > + /* Ensure that snd_ssthresh is non 0 */ > + tp->snd_ssthresh = MAX(pipe >> 1,1); > + tp->snd_cwnd = tp->snd_ssthresh; > +} > + > +void > +tcp_newcwv_datalim_closedown(struct tcpcb *tp) > +{ > + while ((ticks - tp->cwnd_valid_ts) > NCWV_FIVEMINS && > + tp->snd_cwnd > tp->init_cwnd) { > + > + tp->cwnd_valid_ts += NCWV_FIVEMINS; > + tp->snd_ssthresh = MAX( (3 * tp->snd_cwnd ) >> 2,tp->snd_ssthresh); > + tp->snd_cwnd = MAX(tp->snd_cwnd >> 1, tp->init_cwnd); > + } > +} > > Property changes on: sys/netinet/tcp_newcwv.c > ___________________________________________________________________ > Added: svn:mime-type > ## -0,0 +1 ## > +text/plain > \ No newline at end of property > Added: svn:keywords > ## -0,0 +1 ## > +FreeBSD=%H > \ No newline at end of property > Added: svn:eol-style > ## -0,0 +1 ## > +native > \ No newline at end of property > Index: sys/netinet/tcp_newcwv.h > =================================================================== > --- sys/netinet/tcp_newcwv.h (revision 0) > +++ sys/netinet/tcp_newcwv.h (working copy) > @@ -0,0 +1,39 @@ > +/* > + * Copyright (c) 2014 Tom Jones > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + */ > + > +#ifndef _NETINET_TCP_NEWCWV_H_ > +#define _NETINET_TCP_NEWCWV_H_ > + > +#include > + > +void tcp_newcwv_update_pipeack(struct tcpcb *); > +void tcp_newcwv_reset(struct tcpcb *); > +void tcp_newcwv_enter_recovery(struct tcpcb *); > +void tcp_newcwv_end_recovery(struct tcpcb *); > +void tcp_newcwv_datalim_closedown(struct tcpcb *); > + > +#endif /* _NETINET_TCP_NEWCWV_H_ */ > > Property changes on: sys/netinet/tcp_newcwv.h > ___________________________________________________________________ > Added: svn:keywords > ## -0,0 +1 ## > +FreeBSD=%H > \ No newline at end of property > Added: svn:eol-style > ## -0,0 +1 ## > +native > \ No newline at end of property > Added: svn:mime-type > ## -0,0 +1 ## > +text/plain > \ No newline at end of property > Index: sys/netinet/tcp_output.c > =================================================================== > --- sys/netinet/tcp_output.c (revision 268040) > +++ sys/netinet/tcp_output.c (working copy) > @@ -74,6 +74,7 @@ __FBSDID("$FreeBSD$"); > #include > #include > #include > +#include > #ifdef TCPDEBUG > #include > #endif > @@ -691,6 +692,10 @@ send: > #endif > hdrlen = sizeof (struct tcpiphdr); > > + /* Trigger the newcwv timer */ > + if(V_tcp_do_newcwv) > + tcp_newcwv_datalim_closedown(tp); > + > /* > * Compute options for segment. > * We only have to care about SYN and established connection > Index: sys/netinet/tcp_subr.c > =================================================================== > --- sys/netinet/tcp_subr.c (revision 268040) > +++ sys/netinet/tcp_subr.c (working copy) > @@ -800,6 +800,7 @@ tcp_newtcpcb(struct inpcb *inp) > tp->t_flags = (TF_REQ_SCALE|TF_REQ_TSTMP); > if (V_tcp_do_sack) > tp->t_flags |= TF_SACK_PERMIT; > + > TAILQ_INIT(&tp->snd_holes); > tp->t_inpcb = inp; /* XXX */ > /* > Index: sys/netinet/tcp_var.h > =================================================================== > --- sys/netinet/tcp_var.h (revision 268040) > +++ sys/netinet/tcp_var.h (working copy) > @@ -172,6 +172,20 @@ struct tcpcb { > int t_sndzerowin; /* zero-window updates sent */ > u_int t_badrxtwin; /* window for retransmit recovery */ > u_char snd_limited; /* segments limited transmitted */ > +/* NewCWV releated state */ > + u_int32_t pipeack; > + u_int32_t psp; /* pipeack sampling period */ > + > + u_int32_t head; > + u_int32_t psample[4]; /* pipe ack samples */ > + u_int32_t time_stamp[4]; /* time stamp samples */ > + u_int32_t prev_snd_una; /* previous snd_una in this sampe */ > + u_int32_t prev_snd_nxt; /* previous snd_nxt in this sampe */ > + > + u_int32_t loss_flight_size; /* flightsize at loss detection */ > + u_int32_t prior_retrans; /* Retransmission before going into FR */ > + u_int32_t cwnd_valid_ts; /*last time cwnd was found valid */ > + u_int32_t init_cwnd; /* The inital cwnd */ > /* SACK related state */ > int snd_numholes; /* number of holes seen by sender */ > TAILQ_HEAD(sackhole_head, sackhole) snd_holes; > @@ -605,6 +619,7 @@ VNET_DECLARE(int, tcp_recvspace); > VNET_DECLARE(int, path_mtu_discovery); > VNET_DECLARE(int, tcp_do_rfc3465); > VNET_DECLARE(int, tcp_abc_l_var); > +VNET_DECLARE(int, tcp_do_newcwv); > #define V_tcb VNET(tcb) > #define V_tcbinfo VNET(tcbinfo) > #define V_tcp_mssdflt VNET(tcp_mssdflt) > @@ -617,6 +632,7 @@ VNET_DECLARE(int, tcp_abc_l_var); > #define V_path_mtu_discovery VNET(path_mtu_discovery) > #define V_tcp_do_rfc3465 VNET(tcp_do_rfc3465) > #define V_tcp_abc_l_var VNET(tcp_abc_l_var) > +#define V_tcp_do_newcwv VNET(tcp_do_newcwv) > > VNET_DECLARE(int, tcp_do_sack); /* SACK enabled/disabled */ > VNET_DECLARE(int, tcp_sc_rst_sock_fail); /* RST on sock alloc failure */ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 19:32:07 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CED3F34 for ; Mon, 30 Jun 2014 19:32:07 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34B452C0D for ; Mon, 30 Jun 2014 19:32:07 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5UJW777098984 for ; Mon, 30 Jun 2014 20:32:07 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191343] [ipnat] ipnat error at boot disables active sessions Date: Mon, 30 Jun 2014 19:32:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 19:32:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191343 Cy Schubert changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cy@FreeBSD.org Assignee|freebsd-net@FreeBSD.org |cy@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 19:33:15 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2EEB19C for ; Mon, 30 Jun 2014 19:33:15 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAD1E2C25 for ; Mon, 30 Jun 2014 19:33:15 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5UJXFtj021926 for ; Mon, 30 Jun 2014 20:33:15 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 190038] [ipf] ipf -Fa -6 clears v6 and v6 lists Date: Mon, 30 Jun 2014 19:33:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: cy@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 19:33:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190038 Cy Schubert changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cy@FreeBSD.org Assignee|freebsd-net@FreeBSD.org |cy@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 20:54:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB0CA5F4; Mon, 30 Jun 2014 20:54:07 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49AC12463; Mon, 30 Jun 2014 20:54:07 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id b13so8569501wgh.26 for ; Mon, 30 Jun 2014 13:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:hackerspace:user-agent; bh=OMo1mMktHyv3zYyyeMUSZTNqFUyFOiy/y18U+UCWrM8=; b=XOirZrzCrQ9pYpd2PApyLCr9m+rn2Wy16als9CzbgCgGLYQ9srfJV1TBvjtmNeqFIs hhYQhUUZlCAiOKKOS96UHVxVXlCnl5WnmAQDytbNqGLqx77WGRoj17ULfiwOWbfXhe8Z yu/YZYinRQeQPE/+Lfm7kTi7sUd1ymG0b2xQC5UwFNMraW9JCU5UNFu+HwDUg9WmYSvC pneSBO7ymO3+s3Dg7MgBVjOXZvRo/EjXAVrcqlJt6UFYB6c1+4+KMEkAk/7lFYangT4C i617SMWTSsBSTg+R7fus4IKs1fM9v6eED7DnmL4ZnEykdrT6xnmD+lDoaz2jrf1rtbo2 k3Lw== X-Received: by 10.194.133.41 with SMTP id oz9mr5485483wjb.131.1404161645530; Mon, 30 Jun 2014 13:54:05 -0700 (PDT) Received: from gmail.com (host86-159-226-194.range86-159.btcentralplus.com. [86.159.226.194]) by mx.google.com with ESMTPSA id cj8sm43610578wjb.5.2014.06.30.13.54.04 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jun 2014 13:54:04 -0700 (PDT) Sender: Tom Jones Date: Mon, 30 Jun 2014 21:54:01 +0100 From: Tom Jones To: Adrian Chadd Subject: Re: [PATCH] Implementation of draft-ietf-tcpm-newcwv-06 Message-ID: <20140630205359.GA2221@gmail.com> References: <20140630170453.GA21404@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hackerspace: 57North Hacklab User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 20:54:08 -0000 On Mon, Jun 30, 2014 at 12:29:44PM -0700, Adrian Chadd wrote: > Hi! > > Cool! Would you mind throwing it into a bugzilla ticket so it's not > lost and it can be assigned for some review? > > http://bugs.freebsd.org/submit/ Done. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191520 > Is there any benefit in hanging this state off of it as a pointer to > some other struct, rather than having it in the tcpcb? Is it always > going to be used? The draft recommends newcwv be used as a default, so there is a chance it will always be around. What would the impact to performance be by moving the state out into a struct? > On 30 June 2014 10:04, Tom Jones wrote: > > Hello, > > > > Attached is a patch which implements draft-ietf-tcpm-newcwv-06, "Updating TCP > > to support Rate-Limited Traffic". The patch is a port of the Linux > > implementation by Raffaello Secchi with influence from Aris > > Angelogiannopoulos's patch set that was sent to the list earlier this year. > > > > -- > > Tom | I don't see how we are going to build the dystopian megapoli > > @adventureloop | we were promised in 80s and early 90s cyberpunk fiction if > > adventurist.me | you guys keep complaining. > > > > Index: sys/conf/files > > =================================================================== > > --- sys/conf/files (revision 268040) > > +++ sys/conf/files (working copy) > > @@ -3380,6 +3380,7 @@ netinet/tcp_reass.c optional inet | inet6 > > netinet/tcp_sack.c optional inet | inet6 > > netinet/tcp_subr.c optional inet | inet6 > > netinet/tcp_syncache.c optional inet | inet6 > > +netinet/tcp_newcwv.c optional inet | inet6 > > netinet/tcp_timer.c optional inet | inet6 > > netinet/tcp_timewait.c optional inet | inet6 > > netinet/tcp_usrreq.c optional inet | inet6 > > Index: sys/netinet/tcp_input.c > > =================================================================== > > --- sys/netinet/tcp_input.c (revision 268040) > > +++ sys/netinet/tcp_input.c (working copy) > > @@ -105,6 +105,7 @@ __FBSDID("$FreeBSD$"); > > #include > > #include > > #include > > +#include > > #ifdef TCPDEBUG > > #include > > #endif /* TCPDEBUG */ > > @@ -174,6 +175,11 @@ SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, abc_l_var > > &VNET_NAME(tcp_abc_l_var), 2, > > "Cap the max cwnd increment during slow-start to this number of segments"); > > > > +VNET_DEFINE(int, tcp_do_newcwv) = 0; > > +SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, newcwv, CTLFLAG_RW, > > + &VNET_NAME(tcp_do_newcwv), 0, > > + "Enable draft-ietf-tcpm-newcwv-06 (New Congestion Window Validation)"); > > + > > static SYSCTL_NODE(_net_inet_tcp, OID_AUTO, ecn, CTLFLAG_RW, 0, "TCP ECN"); > > > > VNET_DEFINE(int, tcp_do_ecn) = 0; > > @@ -309,6 +315,12 @@ cc_ack_received(struct tcpcb *tp, struct tcphdr *t > > tp->ccv->curack = th->th_ack; > > CC_ALGO(tp)->ack_received(tp->ccv, type); > > } > > + > > + /* > > + * update draft-ietf-newcwv-06 pipeack > > + */ > > + if(V_tcp_do_newcwv && !IN_FASTRECOVERY(tp->t_flags)) > > + tcp_newcwv_update_pipeack(tp); > > } > > > > static void inline > > @@ -378,6 +390,12 @@ cc_conn_init(struct tcpcb *tp) > > tp->snd_cwnd = 4 * tp->t_maxseg; > > } > > > > + /* > > + * Initialise NewCWV state > > + */ > > + tp->init_cwnd = tp->snd_cwnd; > > + tcp_newcwv_reset(tp); > > + > > if (CC_ALGO(tp)->conn_init != NULL) > > CC_ALGO(tp)->conn_init(tp->ccv); > > } > > @@ -426,6 +444,11 @@ cc_cong_signal(struct tcpcb *tp, struct tcphdr *th > > tp->t_badrxtwin = 0; > > break; > > } > > + > > + if (V_tcp_do_newcwv && > > + (type == CC_NDUPACK || type == CC_ECN) && > > + tp->pipeack <= (tp->snd_cwnd >> 1) ) > > + tcp_newcwv_enter_recovery(tp); > > > > if (CC_ALGO(tp)->cong_signal != NULL) { > > if (th != NULL) > > @@ -447,6 +470,13 @@ cc_post_recovery(struct tcpcb *tp, struct tcphdr * > > } > > /* XXXLAS: EXIT_RECOVERY ? */ > > tp->t_bytes_acked = 0; > > + > > + if(V_tcp_do_newcwv) { > > + if(tp->loss_flight_size) > > + tcp_newcwv_end_recovery(tp); > > + tcp_newcwv_reset(tp); > > + } > > + tp->loss_flight_size = 0; > > } > > > > #ifdef TCP_SIGNATURE > > Index: sys/netinet/tcp_newcwv.c > > =================================================================== > > --- sys/netinet/tcp_newcwv.c (revision 0) > > +++ sys/netinet/tcp_newcwv.c (working copy) > > @@ -0,0 +1,174 @@ > > +/* > > + * Copyright (c) 2014 Tom Jones > > + * All rights reserved. > > + * > > + * Redistribution and use in source and binary forms, with or without > > + * modification, are permitted provided that the following conditions > > + * are met: > > + * 1. Redistributions of source code must retain the above copyright > > + * notice, this list of conditions and the following disclaimer. > > + * 2. Redistributions in binary form must reproduce the above copyright > > + * notice, this list of conditions and the following disclaimer in the > > + * documentation and/or other materials provided with the distribution. > > + * > > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > > + * SUCH DAMAGE. > > + * > > + */ > > + > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > + > > +/* > > + * An implementation of NewCWV (draft-ietf-tcpm-newcwv-06) for FreeBSD. > > + * Based on the Linux implementation by Raffaello Secchi and the an initial > > + * implementation of draft-ietf-tcpm-newcwv-00 by Aris Angelogiannopoulos. > > + */ > > + > > +#define nextbin(x) (((x)+1) & 0x03) > > +#define prevbin(x) (((x)-1) & 0x03) > > + > > +#define NCWV_UNDEF 0xFFFFFFFF > > +#define NCWV_FIVEMINS (300*hz) > > + > > +void add_element(struct tcpcb *,u_int32_t); > > +u_int32_t remove_expired_elements(struct tcpcb *); > > + > > +void > > +tcp_newcwv_update_pipeack(struct tcpcb *tp) > > +{ > > + u_int32_t tmp_pipeack; > > + tp->psp = MAX(3 * tp->t_srtt,hz); > > + > > + if (tp->snd_una >= tp->prev_snd_nxt) { > > + /* get the pipeack sample */ > > + tmp_pipeack = tp->snd_una - tp->prev_snd_una; > > + > > + tp->prev_snd_una = tp->snd_una; > > + tp->prev_snd_nxt = tp->snd_nxt; > > + > > + /* create a new element at the end of current pmp */ > > + if(ticks > tp->time_stamp[tp->head] + (tp->psp >> 2)) > > + add_element(tp,tmp_pipeack); > > + else > > + tp->psample[tp->head] = tmp_pipeack; > > + } > > + > > + tp->pipeack = remove_expired_elements(tp); > > + > > + /* check if cwnd is validated */ > > + if (tp->pipeack == NCWV_UNDEF || > > + ((tp->pipeack << 1) >= (tp->snd_cwnd * tp->t_maxseg))) { > > + tp->cwnd_valid_ts = ticks; > > + } > > +} > > + > > +void > > +add_element(struct tcpcb *tp,u_int32_t value) > > +{ > > + tp->head = nextbin(tp->head); > > + tp->psample[tp->head] = value; > > + tp->time_stamp[tp->head] = ticks; > > +} > > + > > +u_int32_t > > +remove_expired_elements(struct tcpcb *tp) > > +{ > > + uint8_t head = tp->head; > > + u_int32_t tmp = tp->psample[head]; > > + > > + while(tp->psample[head] != NCWV_UNDEF) { > > + /* remove the element if expired */ > > + if (tp->time_stamp[head] < ticks - tp->psp) { > > + tp->psample[head] = NCWV_UNDEF; > > + return tmp; > > + } > > + > > + /* search for the max pipeack */ > > + if(tp->psample[head] > tmp) > > + tmp = tp->psample[head]; > > + > > + head = prevbin(head); > > + if(head == tp->head) > > + return tmp; > > + } > > + > > + return tmp; > > +} > > + > > +/* Initialise NewCWV state */ > > +void > > +tcp_newcwv_reset(struct tcpcb *tp) > > +{ > > + tp->prev_snd_una = tp->snd_una; > > + tp->prev_snd_nxt = tp->snd_nxt; > > + tp->cwnd_valid_ts = ticks; > > + tp->loss_flight_size = 0; > > + > > + tp->head = 0; > > + tp->psample[0] = NCWV_UNDEF; > > + tp->pipeack = NCWV_UNDEF; > > +} > > + > > +/* NewCWV actions at loss detection */ > > +void > > +tcp_newcwv_enter_recovery(struct tcpcb *tp) > > +{ > > + u_int32_t pipe; > > + > > + if(tp->pipeack == NCWV_UNDEF) > > + return; > > + > > + tp->prior_retrans = tp->t_sndrexmitpack; > > + > > + /* Calculate the flight size */ > > + u_int32_t awnd = (tp->snd_nxt - tp->snd_fack) + tp->sackhint.sack_bytes_rexmit; > > + tp->loss_flight_size = awnd; > > + > > + pipe = MAX(tp->pipeack,tp->loss_flight_size); > > + tp->snd_cwnd = MAX(pipe >> 1,1); > > +} > > + > > +/* NewCWV actions at the end of recovery */ > > +void > > +tcp_newcwv_end_recovery(struct tcpcb *tp) > > +{ > > + u_int32_t retrans,pipe; > > + > > + retrans = (tp->t_sndrexmitpack - tp->prior_retrans) * tp->t_maxseg; > > + pipe = MAX(tp->pipeack,tp->loss_flight_size) - retrans; > > + > > + /* Ensure that snd_ssthresh is non 0 */ > > + tp->snd_ssthresh = MAX(pipe >> 1,1); > > + tp->snd_cwnd = tp->snd_ssthresh; > > +} > > + > > +void > > +tcp_newcwv_datalim_closedown(struct tcpcb *tp) > > +{ > > + while ((ticks - tp->cwnd_valid_ts) > NCWV_FIVEMINS && > > + tp->snd_cwnd > tp->init_cwnd) { > > + > > + tp->cwnd_valid_ts += NCWV_FIVEMINS; > > + tp->snd_ssthresh = MAX( (3 * tp->snd_cwnd ) >> 2,tp->snd_ssthresh); > > + tp->snd_cwnd = MAX(tp->snd_cwnd >> 1, tp->init_cwnd); > > + } > > +} > > > > Property changes on: sys/netinet/tcp_newcwv.c > > ___________________________________________________________________ > > Added: svn:mime-type > > ## -0,0 +1 ## > > +text/plain > > \ No newline at end of property > > Added: svn:keywords > > ## -0,0 +1 ## > > +FreeBSD=%H > > \ No newline at end of property > > Added: svn:eol-style > > ## -0,0 +1 ## > > +native > > \ No newline at end of property > > Index: sys/netinet/tcp_newcwv.h > > =================================================================== > > --- sys/netinet/tcp_newcwv.h (revision 0) > > +++ sys/netinet/tcp_newcwv.h (working copy) > > @@ -0,0 +1,39 @@ > > +/* > > + * Copyright (c) 2014 Tom Jones > > + * All rights reserved. > > + * > > + * Redistribution and use in source and binary forms, with or without > > + * modification, are permitted provided that the following conditions > > + * are met: > > + * 1. Redistributions of source code must retain the above copyright > > + * notice, this list of conditions and the following disclaimer. > > + * 2. Redistributions in binary form must reproduce the above copyright > > + * notice, this list of conditions and the following disclaimer in the > > + * documentation and/or other materials provided with the distribution. > > + * > > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > > + * SUCH DAMAGE. > > + * > > + */ > > + > > +#ifndef _NETINET_TCP_NEWCWV_H_ > > +#define _NETINET_TCP_NEWCWV_H_ > > + > > +#include > > + > > +void tcp_newcwv_update_pipeack(struct tcpcb *); > > +void tcp_newcwv_reset(struct tcpcb *); > > +void tcp_newcwv_enter_recovery(struct tcpcb *); > > +void tcp_newcwv_end_recovery(struct tcpcb *); > > +void tcp_newcwv_datalim_closedown(struct tcpcb *); > > + > > +#endif /* _NETINET_TCP_NEWCWV_H_ */ > > > > Property changes on: sys/netinet/tcp_newcwv.h > > ___________________________________________________________________ > > Added: svn:keywords > > ## -0,0 +1 ## > > +FreeBSD=%H > > \ No newline at end of property > > Added: svn:eol-style > > ## -0,0 +1 ## > > +native > > \ No newline at end of property > > Added: svn:mime-type > > ## -0,0 +1 ## > > +text/plain > > \ No newline at end of property > > Index: sys/netinet/tcp_output.c > > =================================================================== > > --- sys/netinet/tcp_output.c (revision 268040) > > +++ sys/netinet/tcp_output.c (working copy) > > @@ -74,6 +74,7 @@ __FBSDID("$FreeBSD$"); > > #include > > #include > > #include > > +#include > > #ifdef TCPDEBUG > > #include > > #endif > > @@ -691,6 +692,10 @@ send: > > #endif > > hdrlen = sizeof (struct tcpiphdr); > > > > + /* Trigger the newcwv timer */ > > + if(V_tcp_do_newcwv) > > + tcp_newcwv_datalim_closedown(tp); > > + > > /* > > * Compute options for segment. > > * We only have to care about SYN and established connection > > Index: sys/netinet/tcp_subr.c > > =================================================================== > > --- sys/netinet/tcp_subr.c (revision 268040) > > +++ sys/netinet/tcp_subr.c (working copy) > > @@ -800,6 +800,7 @@ tcp_newtcpcb(struct inpcb *inp) > > tp->t_flags = (TF_REQ_SCALE|TF_REQ_TSTMP); > > if (V_tcp_do_sack) > > tp->t_flags |= TF_SACK_PERMIT; > > + > > TAILQ_INIT(&tp->snd_holes); > > tp->t_inpcb = inp; /* XXX */ > > /* > > Index: sys/netinet/tcp_var.h > > =================================================================== > > --- sys/netinet/tcp_var.h (revision 268040) > > +++ sys/netinet/tcp_var.h (working copy) > > @@ -172,6 +172,20 @@ struct tcpcb { > > int t_sndzerowin; /* zero-window updates sent */ > > u_int t_badrxtwin; /* window for retransmit recovery */ > > u_char snd_limited; /* segments limited transmitted */ > > +/* NewCWV releated state */ > > + u_int32_t pipeack; > > + u_int32_t psp; /* pipeack sampling period */ > > + > > + u_int32_t head; > > + u_int32_t psample[4]; /* pipe ack samples */ > > + u_int32_t time_stamp[4]; /* time stamp samples */ > > + u_int32_t prev_snd_una; /* previous snd_una in this sampe */ > > + u_int32_t prev_snd_nxt; /* previous snd_nxt in this sampe */ > > + > > + u_int32_t loss_flight_size; /* flightsize at loss detection */ > > + u_int32_t prior_retrans; /* Retransmission before going into FR */ > > + u_int32_t cwnd_valid_ts; /*last time cwnd was found valid */ > > + u_int32_t init_cwnd; /* The inital cwnd */ > > /* SACK related state */ > > int snd_numholes; /* number of holes seen by sender */ > > TAILQ_HEAD(sackhole_head, sackhole) snd_holes; > > @@ -605,6 +619,7 @@ VNET_DECLARE(int, tcp_recvspace); > > VNET_DECLARE(int, path_mtu_discovery); > > VNET_DECLARE(int, tcp_do_rfc3465); > > VNET_DECLARE(int, tcp_abc_l_var); > > +VNET_DECLARE(int, tcp_do_newcwv); > > #define V_tcb VNET(tcb) > > #define V_tcbinfo VNET(tcbinfo) > > #define V_tcp_mssdflt VNET(tcp_mssdflt) > > @@ -617,6 +632,7 @@ VNET_DECLARE(int, tcp_abc_l_var); > > #define V_path_mtu_discovery VNET(path_mtu_discovery) > > #define V_tcp_do_rfc3465 VNET(tcp_do_rfc3465) > > #define V_tcp_abc_l_var VNET(tcp_abc_l_var) > > +#define V_tcp_do_newcwv VNET(tcp_do_newcwv) > > > > VNET_DECLARE(int, tcp_do_sack); /* SACK enabled/disabled */ > > VNET_DECLARE(int, tcp_sc_rst_sock_fail); /* RST on sock alloc failure */ > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Tom @adventureloop adventurist.me #pragma summon cthulhu :wq From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 22:33:34 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A0569AB for ; Mon, 30 Jun 2014 22:33:34 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E5DDE2D55 for ; Mon, 30 Jun 2014 22:33:33 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5UMXXYg071210 for ; Mon, 30 Jun 2014 23:33:33 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191520] [PATCH] Implementation of draft-ieft-tcpm-newcwv-06 Date: Mon, 30 Jun 2014 22:33:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 22:33:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191520 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Mon Jun 30 23:36:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF013A3E; Mon, 30 Jun 2014 23:36:25 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8551E226B; Mon, 30 Jun 2014 23:36:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: As4EAADZsVODaFve/2dsb2JhbABag19agm6oLgEBAQEBAQaSdYZxUwGBKXWEAwEBAQQBAQEgKyALGw4KAgINGQIpAQkmBggHBAEcBIghDatjnCUXgSuEOYhMBgEBGzQHgneBTAWXdIQwkjeDXiE1fQgXIg X-IronPort-AV: E=Sophos;i="5.01,578,1400040000"; d="scan'208";a="137115814" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 30 Jun 2014 19:36:17 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EFC8DB3F12; Mon, 30 Jun 2014 19:36:17 -0400 (EDT) Date: Mon, 30 Jun 2014 19:36:17 -0400 (EDT) From: Rick Macklem To: Johan Kooijman Message-ID: <283522359.5786930.1404171377970.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: ixgbe queue hang fixed in 9.3? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: FreeBSD Net , freebsd-stable@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 23:36:25 -0000 Johan Kooijman wrote: > Hey all, > > A while ago I started the thread "9.2 ixgbe tx queue hang" on > freebsd-net. > A lot more people had the same as I had. Could anybody tell me if > this > issue is resolved in 9.3 RC? I haven't been able to find anything > about it. > Well, it depends on your definition of "resolved". r264630 (in head) is in 9.3. With this patch, I believe that the hangs will not occur. However, ixgbe (for the 82599 chips, there shouldn't be an issue for the 82598 chips) will do a lot of m_defrag() calls, which does result in cpu overheads. To get rid of the m_defrag() calls, you need to disable TSO. I think a better fix for this is to allow the network device drivers specify a maximum number of mbufs in the list for a TSO segment and have tcp_output() chop the TSO segments up at this limit. (I have an untested patch to do this and it might make it in head someday, but will probably never be MFCible.) So, if the overhead caused by lots of m_defrag() calls isn't a problem for you, it is resolved in 9.3. rick > -- > Met vriendelijke groeten / With kind regards, > Johan Kooijman > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to > "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 13:14:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2759E95; Tue, 1 Jul 2014 13:14:36 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C84402FDA; Tue, 1 Jul 2014 13:14:36 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,581,1400050800"; d="asc'?scan'208";a="131820345" Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) by mx11-out.netapp.com with ESMTP; 01 Jul 2014 06:14:27 -0700 Received: from HIOEXCMBX04-PRD.hq.netapp.com (10.122.105.37) by vmwexceht03-prd.hq.netapp.com (10.106.76.241) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 1 Jul 2014 06:14:27 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 1 Jul 2014 06:14:26 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::8485:306a:80a6:ba8f%21]) with mapi id 15.00.0847.030; Tue, 1 Jul 2014 06:14:26 -0700 From: "Eggert, Lars" To: Adrian Chadd Subject: Re: [rfc] tcp timer update for RSS Thread-Topic: [rfc] tcp timer update for RSS Thread-Index: AQHPb/ho5JPKULu5Hk2S1uPuczXO0JuL8ikA Date: Tue, 1 Jul 2014 13:14:25 +0000 Message-ID: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 13:14:37 -0000 --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ sys/netinet/ip_output.c: case IP_RSSCPUID: kernel compilation with RSS currently fails, because IP_RSSCPUID is = still used in ip_output.c. Lars --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU7K0MNZcnpRveo1xAQIpfAP/YWb4ZhTqG9iXsgKBM86HupgtRrq69T3k uy84RJMlXNdpuDEzKTlSPxUplt0j+AoqDgEpafG+zAtS3UtBgoKqBa/WTq4tlpaf m7fQMC9V85gcUaa7hovKAtfz963IW/2eZbUnHZAH7PmRJ9p49/vSH24yn4FT4r9D Gt7gTydxWuc= =9sL0 -----END PGP SIGNATURE----- --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 15:51:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20278531; Tue, 1 Jul 2014 15:51:11 +0000 (UTC) Received: from mail.CSUA.Berkeley.EDU (mail.CSUA.Berkeley.EDU [128.32.112.223]) by mx1.freebsd.org (Postfix) with ESMTP id E120C2FB3; Tue, 1 Jul 2014 15:51:10 +0000 (UTC) Received: by mail.CSUA.Berkeley.EDU (Postfix, from userid 500) id E9B0E43153; Tue, 1 Jul 2014 08:45:37 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 Received: from mx2.freebsd.org (mx2.freebsd.org [8.8.178.116]) by mail.CSUA.Berkeley.EDU (Postfix) with ESMTP id 21B7943192 for ; Tue, 1 Jul 2014 08:44:53 -0700 (PDT) Received: from hub.freebsd.org (hub.freebsd.org [8.8.178.136]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx2.freebsd.org (Postfix) with ESMTPS id 40327142D; Mon, 30 Jun 2014 23:36:30 +0000 (UTC) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:1900:2254:206c::16:88]) by hub.freebsd.org (Postfix) with ESMTP id D6B2BAC7; Mon, 30 Jun 2014 23:36:29 +0000 (UTC) Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF013A3E; Mon, 30 Jun 2014 23:36:25 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8551E226B; Mon, 30 Jun 2014 23:36:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: As4EAADZsVODaFve/2dsb2JhbABag19agm6oLgEBAQEBAQaSdYZxUwGBKXWEAwEBAQQBAQEgKyALGw4KAgINGQIpAQkmBggHBAEcBIghDatjnCUXgSuEOYhMBgEBGzQHgneBTAWXdIQwkjeDXiE1fQgXIg X-IronPort-AV: E=Sophos;i="5.01,578,1400040000"; d="scan'208";a="137115814" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 30 Jun 2014 19:36:17 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EFC8DB3F12; Mon, 30 Jun 2014 19:36:17 -0400 (EDT) Date: Mon, 30 Jun 2014 19:36:17 -0400 (EDT) From: Rick Macklem To: Johan Kooijman Message-ID: <283522359.5786930.1404171377970.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: ixgbe queue hang fixed in 9.3? MIME-Version: 1.0 X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: owner-freebsd-stable@freebsd.org Sender: owner-freebsd-stable@freebsd.org Cc: FreeBSD Net , freebsd-stable@freebsd.org X-BeenThere: freebsd-net@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 15:51:11 -0000 Johan Kooijman wrote: > Hey all, > > A while ago I started the thread "9.2 ixgbe tx queue hang" on > freebsd-net. > A lot more people had the same as I had. Could anybody tell me if > this > issue is resolved in 9.3 RC? I haven't been able to find anything > about it. > Well, it depends on your definition of "resolved". r264630 (in head) is in 9.3. With this patch, I believe that the hangs will not occur. However, ixgbe (for the 82599 chips, there shouldn't be an issue for the 82598 chips) will do a lot of m_defrag() calls, which does result in cpu overheads. To get rid of the m_defrag() calls, you need to disable TSO. I think a better fix for this is to allow the network device drivers specify a maximum number of mbufs in the list for a TSO segment and have tcp_output() chop the TSO segments up at this limit. (I have an untested patch to do this and it might make it in head someday, but will probably never be MFCible.) So, if the overhead caused by lots of m_defrag() calls isn't a problem for you, it is resolved in 9.3. rick > -- > Met vriendelijke groeten / With kind regards, > Johan Kooijman > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to > "freebsd-stable-unsubscribe@freebsd.org" > _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 15:51:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C1C75BC; Tue, 1 Jul 2014 15:51:14 +0000 (UTC) Received: from mail.CSUA.Berkeley.EDU (mail.CSUA.Berkeley.EDU [128.32.112.223]) by mx1.freebsd.org (Postfix) with ESMTP id 53CAE204B; Tue, 1 Jul 2014 15:51:14 +0000 (UTC) Received: by mail.CSUA.Berkeley.EDU (Postfix, from userid 500) id 5F32443153; Tue, 1 Jul 2014 08:45:41 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail X-Spam-Level: X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 Received: from mx2.freebsd.org (mx2.freebsd.org [8.8.178.116]) by mail.CSUA.Berkeley.EDU (Postfix) with ESMTP id 706C7431C4 for ; Tue, 1 Jul 2014 08:44:53 -0700 (PDT) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:1900:2254:206c::16:88]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx2.freebsd.org (Postfix) with ESMTPS id EFE464E85; Mon, 30 Jun 2014 08:25:14 +0000 (UTC) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:1900:2254:206c::16:88]) by hub.freebsd.org (Postfix) with ESMTP id EAD8D195; Mon, 30 Jun 2014 08:25:14 +0000 (UTC) Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAD92185 for ; Mon, 30 Jun 2014 08:25:12 +0000 (UTC) Received: from mail-la0-f42.google.com (mail-la0-f42.google.com [209.85.215.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7559E2BC4 for ; Mon, 30 Jun 2014 08:25:11 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id pn19so4646356lab.1 for ; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=RUSrV5+b2yvE+D2m1EU9IvnwQrMbfSZD9TnUaBI8800=; b=KgO1ZtHgcJqLfqxQ91n5FWCOh7ZB9eGf6f2T9kZrlw1K51RvVzh7YUi8X5xHIVM5GO qkxmJDK7lahEU19VUx3E8xGW+xlRxMkTvWJL5TMTxQIx938/CWBd9DYjTue5ER6NP4C+ FfscW/fIk4blW9wvz/4xHj9cgBL85LQfzMQpP31b7iaXoRYQf+dUag4eNy99oILlKTud /J2SsWU18DVNFn+4yagmHkRP1PDhR152jKXtngpkIwDR/b1+aVvGY/zGeMLrrRq07ucs X0/pGM3kCOwEDGkWdBXOsG4xLhy2dz+bC+xQ6zSAIuS3Mu0h+i6+2klbwY6zVN4Nrkd8 lwLQ== X-Gm-Message-State: ALoCoQnQlrQZ5/JUaXv7u71dD8FH4BPxmj6+uj5rtgOniuVR8jpyORQmqvyC3yel+nOWfcQ5gSww MIME-Version: 1.0 X-Received: by 10.112.182.36 with SMTP id eb4mr234139lbc.99.1404116704079; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) Received: by 10.112.99.4 with HTTP; Mon, 30 Jun 2014 01:25:04 -0700 (PDT) Date: Mon, 30 Jun 2014 10:25:04 +0200 Message-ID: Subject: ixgbe queue hang fixed in 9.3? From: Johan Kooijman To: freebsd-stable@freebsd.org, FreeBSD Net X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: owner-freebsd-stable@freebsd.org Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-net@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 15:51:14 -0000 Hey all, A while ago I started the thread "9.2 ixgbe tx queue hang" on freebsd-net. A lot more people had the same as I had. Could anybody tell me if this issue is resolved in 9.3 RC? I haven't been able to find anything about it. -- Met vriendelijke groeten / With kind regards, Johan Kooijman _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 17:24:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80615117; Tue, 1 Jul 2014 17:24:44 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 309262988; Tue, 1 Jul 2014 17:24:44 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id hw13so8002386qab.34 for ; Tue, 01 Jul 2014 10:24:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=qplSQQFgTXSo2B44iwRAOTcCRAvLVaFLOuZacq1Qw5Q=; b=zB2aVxo9AZO5phXTzK53rkVosWfgQh6mrfbLcBPg22pezuleQUcO+ZRraGYWgJP4sR 7dg6XGkNjcNHju9X91YI10gxmD0TR3wXPIx1a1no9CEGlz3x1mKDukRBDB+QvkC4lHHz Rkz3evY+2T11vDo4DSZDQEI6Y3gwCdaJa2DwgzyCPFATE9lPFwY1H4hkzvs/kXH+/qdm ENpfP6AVrs6XAT4pRrUPnr0xJo3B4ZePbDaRTl44kBQ9CkHHfI1Zx4VQ5/t9tbEQ215A darDVFign55N/t82hL27MlDCXKSr0IVSCvaS4aWaoCKBP2hL3neGvulGqMOZdeF2I3iC bPNQ== MIME-Version: 1.0 X-Received: by 10.224.66.70 with SMTP id m6mr77949266qai.55.1404235483405; Tue, 01 Jul 2014 10:24:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 10:24:43 -0700 (PDT) In-Reply-To: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> Date: Tue, 1 Jul 2014 10:24:43 -0700 X-Google-Sender-Auth: U4HDKadrc0dhAYamzt81UpLEu08 Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 17:24:44 -0000 On 1 July 2014 06:14, Eggert, Lars wrote: > Hi, > > [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys > sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ > sys/netinet/ip_output.c: case IP_RSSCPUID: > > kernel compilation with RSS currently fails, because IP_RSSCPUID is still= used in ip_output.c. I thought I committed a fix to remove that! I'll go fix that soon. Thanks! -a From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 17:28:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4292409; Tue, 1 Jul 2014 17:28:03 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7533729C3; Tue, 1 Jul 2014 17:28:03 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id f51so3593295qge.36 for ; Tue, 01 Jul 2014 10:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=qplSQQFgTXSo2B44iwRAOTcCRAvLVaFLOuZacq1Qw5Q=; b=Mb0czZr5rWmLi40iGX0w6f+Bz5o22Ix6mVVspx9ikZx3WqzsEDt06ugLNX7+6ANVkt IMMVsLEpH56CyHO/7mJnjObd8zsP/tSCfsKiJh7QHkoUFErOrwPOJxgkY74wHHGuF4IA 1lC74IUjt3rFjIq4iaAIqyF0yzBs7p0760JZBSd/KmPcNujxAxeYuCGxDbtO9eNSD8Zy AB5YhUcHk0xhCi35iYEf3vZGl8JgPLjOax66L0jWxj2pJFPRjZDC245Q2B8mz8y5zgb5 hvM/SJN3LVDIjCHPHteLHI/VXvxSADNxK2uxHRF4uxNnGz/24BnEfIREtJgkj5U4zfIq PWeg== MIME-Version: 1.0 X-Received: by 10.224.130.136 with SMTP id t8mr10887784qas.49.1404235676235; Tue, 01 Jul 2014 10:27:56 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 10:27:56 -0700 (PDT) In-Reply-To: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> Date: Tue, 1 Jul 2014 10:27:56 -0700 X-Google-Sender-Auth: 1YLq-_-5Urp2Vq4AEvYGmoQd-_A Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 17:28:03 -0000 On 1 July 2014 06:14, Eggert, Lars wrote: > Hi, > > [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys > sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ > sys/netinet/ip_output.c: case IP_RSSCPUID: > > kernel compilation with RSS currently fails, because IP_RSSCPUID is still= used in ip_output.c. I thought I committed a fix to remove that! I'll go fix that soon. Thanks! -a From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 18:02:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B717A7B5; Tue, 1 Jul 2014 18:02:20 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C4752D3C; Tue, 1 Jul 2014 18:02:20 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id v10so7883558qac.41 for ; Tue, 01 Jul 2014 11:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=k29Cqf3KrMEaMdlBk4kIeGaY3fzC2J0rdhfzdWPmX5w=; b=QtawPZXlnexi02LUPnKYHm6PhWCS1Z2ChdfG3knFSLDgXvjTWV1J45B+pClaoExHCo hBY3Pg2nkKx1xbaPQhIucshb5lWZOsYyK9Ko4S+b7qOrA+EJrTRGuNz7Kjnn+TFPi2Oi ZamKzKTde3nAtYQPOsG/52OMmExs6gTTM1WNxvMRwcfQykKu/IhTIk5AqRdxkCgbjrkk PXecvf6Ycr0aVXVe+AOYy6i2PIc4YGOxl4VAT75oeR4O7VZojKxpxJeSZ3Y7hTdnkEGF DBFn1g03+AvP82sdpyWklK8TCmHGfPmHd0Fqahl/LhZuS+QpNWLL6F6ZpFvOcVF0rxNr JOXw== MIME-Version: 1.0 X-Received: by 10.140.34.195 with SMTP id l61mr71334451qgl.87.1404237739505; Tue, 01 Jul 2014 11:02:19 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 11:02:19 -0700 (PDT) Date: Tue, 1 Jul 2014 11:02:19 -0700 X-Google-Sender-Auth: jit6IfaJ6F7YjVvAu1q_Vw0i7Nc Message-ID: Subject: [rfc] IP_BINDMULTI and IP_RSSBUCKETID support From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 18:02:20 -0000 Hi! This patch: http://people.freebsd.org/~adrian/rss/20140701-rss-1.diff Implements a few things: * It introduces IP_BINDMULTI, which allows TCP/UDP sockets to be bound to the same matching address:port. It's intended to be what Linux SO_REUSEADDR has become except I specifically wanted it to be a different option. * .. it only allows the PCBs to be created, it doesn't actually load balance between multiple sockets like this. That requires a bunch more work to happen in the general case. But! * IP_RSSBUCKETID maps an IPv4 TCP (and later UDP, and much later IPv6) socket into a specific RSS bucket rather than in the global PCB list. That way it only receives incoming connections that has to that RSS bucket ID. If userland queries the RSS setup (sysctl net.inet.rss) and creates a worker thread per RSS bucket _AND_ pins it to the correct CPU for the RSS bucket, all the transmit and receive side will stay on the same CPU. The lock contention is almost eliminated in that case. My plan: * get this into the tree and tested/debugged; * add IPv4 UDP extensions to RSS; * teach a couple of popular threaded UDP applications (eg memcached) about this; * bask in what hopefully will be a good set of benefits; * work on IP_BINDMULTI support so multiple threads/processes can listen in the same RSS bucket and get some load balancing; * add IPv6 support; * add RSS rebalancing support; * add multi-socket, multi-NIC RSS awareness. Thanks! -a From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 18:36:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E8F98FF for ; Tue, 1 Jul 2014 18:36:16 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 10B342066 for ; Tue, 1 Jul 2014 18:36:15 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id E73436BBC31 for ; Tue, 1 Jul 2014 20:36:07 +0200 (CEST) Received: from soth.ventu (adsl-ull-14-252.41-151.net24.it [151.41.252.14]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s61Ia3c8084437 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Tue, 1 Jul 2014 20:36:04 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-14-252.41-151.net24.it [151.41.252.14] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s61IZvtW045547; Tue, 1 Jul 2014 20:35:58 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53B2FF8D.6030304@netfence.it> Date: Tue, 01 Jul 2014 20:35:57 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Charles Swiger Subject: Re: MTU not regrowing? References: <53A9C6D0.3090900@netfence.it> <2CA56230-440A-4D77-83BD-222FBC704F70@mac.com> <53AACD56.2030701@netfence.it> In-Reply-To: <53AACD56.2030701@netfence.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Tue, 01 Jul 2014 20:36:05 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.74 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 18:36:16 -0000 On 06/25/14 15:23, Andrea Venturoli wrote: > On 06/25/14 02:01, Charles Swiger wrote: > >> Does "ifconfig vlan3 down; ifconfig vlan3 up" do any good? >> Or that run against the physical NIC? None of the two. John was right about the route. bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 18:37:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7DDFA97 for ; Tue, 1 Jul 2014 18:37:46 +0000 (UTC) Received: from mp1-smtp-6.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id 7A2962093 for ; Tue, 1 Jul 2014 18:37:46 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-6.eutelia.it (Eutelia) with ESMTP id B3C016BBCA9 for ; Tue, 1 Jul 2014 20:37:43 +0200 (CEST) Received: from soth.ventu (adsl-ull-14-252.41-151.net24.it [151.41.252.14]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s61IbZ0T084507 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Tue, 1 Jul 2014 20:37:36 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-14-252.41-151.net24.it [151.41.252.14] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s61IbTKt045579; Tue, 1 Jul 2014 20:37:30 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53B2FFE9.3030906@netfence.it> Date: Tue, 01 Jul 2014 20:37:29 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Hay Subject: Re: MTU not regrowing? References: <53A9C6D0.3090900@netfence.it> <20140624190333.GA13748@zibbi.meraka.csir.co.za> In-Reply-To: <20140624190333.GA13748@zibbi.meraka.csir.co.za> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Tue, 01 Jul 2014 20:37:37 +0200 (CEST) X-Spam-Score: 5.206 (*****) RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.74 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 18:37:46 -0000 On 06/24/14 21:03, John Hay wrote: > Do a "route get somehost" and see what mtu is returned. You are right, I see a route with the old, lesser MTU. > You might be able to delete or tweak that route. How do I do this? I tried "route delete", but it doesn't help. bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 01:53:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E496DA37 for ; Wed, 2 Jul 2014 01:53:25 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B261D271D for ; Wed, 2 Jul 2014 01:53:25 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 96AA8139CE for ; Tue, 1 Jul 2014 22:48:33 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1404265712; x=1405129713; bh=a5hEqZ+bFHAIwfC0YgNfrg2Q4GaEZvU78eZ r3QKFq8s=; b=V+6olW1glakRLWniJ5uEQJUo73kTTT57ybJwdMU9ta4E+NT1Hg1 W/Z22z6vuVb7JG3LU/8s4JX1otVZs/HcjVTLZsFqO9KqKraG3b17077OQyNUQcps TvdS3K6jEQzjVDmsWCNQJUhuR3wIrTI5dVqUv+FoyTZ5PyXrU+QUEzF0= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u82weiLbfT6Q for ; Tue, 1 Jul 2014 22:48:32 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id A21C8139CD for ; Tue, 1 Jul 2014 22:48:31 -0300 (BRT) Message-ID: <53B364D5.9020700@bsdinfo.com.br> Date: Tue, 01 Jul 2014 22:48:05 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: FreeBSD Net Subject: Network Intel X520-SR2 stopping Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 01:53:26 -0000 Hi all, I'm having problems with a 10GbE Intel X520-SR2 interface. After a running time, the interface does not send or receive more data. I am obliged to make a down and up the interface for it to return to work. Have changed the interface, optical cords, optical modules and problem continues. (root@rt01)[~]# netstat -inh Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll igb0 1.5K 00:1e:67:9b:03:e1 0 0 0 0 0 0 igb1 1.5K 00:1e:67:9b:03:e2 19M 0 0 27M 0 0 igb1 - 177.xx.xxx.0/ 177.xx.xxx.1 15K - - 14K - - igb1 - fe80::21e:67f fe80::21e:67ff:fe 2 - - 4 - - igb1 - 2804:xxxx:cad 2804:xxxx:cade:b1 14K - - 12K - - igb2 1.5K 00:1e:67:9b:03:e3 17M 5.8K 0 23M 0 0 igb2 - 177.xx.xxx.40 177.xx.xxx.41 10K - - 12K - - igb2 - fe80::21e:67f fe80::21e:67ff:fe 0 - - 1 - - igb2 - 2804:xxxx:bad 2804:xxxx:bad:b1f 14K - - 14K - - igb3 1.5K 00:1e:67:9b:03:e4 11K 0 0 441M 0 0 igb3 - 186.xxx.x.148 186.xxx.x.150 10K - - 94K - - igb3 - fe80::21e:67f fe80::21e:67ff:fe 0 - - 1 - - igb4 1.5K 00:1b:21:7b:ee:6c 5.8G 5.5K 0 194K 0 0 igb4 - 159.xx.xx.96/ 159.xx.xx.98 63K - - 18K - - igb4 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 - - igb4 - 2804:xxx:ffff 2804:xxx:ffff:ffd 17K - - 16K - - igb5 1.5K 00:1b:21:7b:ee:6d 6.9G 8.0K 0 9.1G 0 0 igb5 - 64.xxx.xxx.68 64.xxx.xxx.70 28K - - 5.5M - - igb5 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 135 - - igb5 - 2001:xxx:2001 2001:xxx:2001:100 17K - - 125K - - igb6 1.5K 00:1b:21:7b:ee:98 4.3G 0 0 3.8G 0 0 igb6 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 - - igb7 1.5K 00:1b:21:7b:ee:99 0 0 0 0 0 0 *ix0 1.5K 00:1b:21:89:25:28 -1.5 118 1.5T 0.1T 0 0* ix0 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 - - ix1 1.5K 00:1b:21:89:25:29 0 0 0 0 0 0 pflog 32K 0 0 0 0 0 0 pfsyn 1.5K 0 0 0 0 0 0 lo0 16K 52K 0 0 52K 0 0 lo0 - ::1/128 ::1 0 - - 41 - - lo0 - fe80::1%lo0/6 fe80::1%lo0 0 - - 0 - - lo0 - 127.0.0.0/8 127.0.0.1 48K - - 52K - - disc0 64K 0 0 0 0 0 0 # uname -a FreeBSD rt01.xxxxxx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #9 r267839: Tue Jun 24 21:06:39 BRT 2014 root@rt01.xxxxxx.xxx.xx:/usr/obj/usr/src/sys/GONDIM amd64 You have new mail in /var/mail/root # netstat -ihw 1 input (Total) output packets errs idrops bytes packets errs bytes colls 813K 0 0 522M 832K 9 615M 0 812K 0 0 523M 831K 9 616M 0 811K 0 0 521M 829K 6 614M 0 807K 0 0 519M 826K 15 615M 0 794K 0 0 514M 814K 7 610M 0 799K 0 0 520M 816K 7 612M 0 804K 0 0 523M 822K 14 613M 0 794K 0 0 518M 813K 14 610M 0 779K 0 0 510M 799K 8 601M 0 790K 0 0 514M 809K 7 603M 0 818K 0 0 525M 835K 7 617M 0 810K 0 0 522M 827K 12 614M 0 794K 0 0 514M 812K 13 603M 0 794K 0 0 518M 813K 13 608M 0 790K 0 0 516M 810K 5 606M 0 807K 0 0 525M 824K 9 614M 0 808K 0 0 529M 827K 10 619M 0 805K 0 0 524M 823K 7 614M 0 804K 0 0 526M 821K 8 615M 0 799K 0 0 520M 816K 6 612M 0 803K 0 0 522M 819K 11 610M 0 # ifconfig ix0 ix0: flags=8843 metric 0 mtu 1500 options=8404bb ether 00:1b:21:89:25:28 inet6 fe80::21b:21ff:fe89:2528%ix0 prefixlen 64 scopeid 0x9 nd6 options=21 media: Ethernet autoselect (10Gbase-SR ) status: active ix0: port 0xc020-0xc03f mem 0xec180000-0xec1fffff,0xec210000-0xec213fff irq 64 at device 0.0 on pci131 ix0: Using MSIX interrupts with 9 vectors ix0: Ethernet address: 00:1b:21:89:25:28 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: port 0xc000-0xc01f mem 0xec080000-0xec0fffff,0xec200000-0xec203fff irq 68 at device 0.1 on pci131 ix1: Using MSIX interrupts with 9 vectors ix1: Ethernet address: 00:1b:21:89:25:29 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 Could be an issue with the ixgbe driver? Thanks Gondim From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 02:48:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A038D5C4 for ; Wed, 2 Jul 2014 02:48:37 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 742A52B07 for ; Wed, 2 Jul 2014 02:48:37 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id z10so11200233pdj.29 for ; Tue, 01 Jul 2014 19:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4BEL3uv/w/SCNQZtMRDHhYvcBXRBbSWPp5oUfKoioX8=; b=V6AuPkHePG4Ke7URQpiKUAw4CzhoXKLH2Ik09uIkQ7Pr2Y+GzWx4d8aMwaducoGzah tuwrotrrGNhhiUQwEO+KdZOLRwDotN3DjtmXcibgpcLbIjpVVmbxWU+Fs+tSKPg53M1l CefYrbSzeoWMOO4i8+k2i++DP76Qfjf8Metm+Nfix2kL5xG6Gl2UOShyBHofI/glbyDQ WdihE3SxQp9T9uEt2aCo1BxOhLvbRoH7dJiuPtTwl/UcIU7Bp+7T7PnoXXydxGJXrYGL eh+T94UQoIV108MujDVbq7QRU14GgrDoQOwQ6epQiUOR7ySnL1axmsJxdoJC9xjc6P7m XtvA== MIME-Version: 1.0 X-Received: by 10.70.91.129 with SMTP id ce1mr450921pdb.68.1404269317063; Tue, 01 Jul 2014 19:48:37 -0700 (PDT) Received: by 10.70.109.225 with HTTP; Tue, 1 Jul 2014 19:48:37 -0700 (PDT) In-Reply-To: <20140529141559.GC74344@onelab2.iet.unipi.it> References: <001b01cf7b3b$dfd1cfb0$9f756f10$@gmail.com> <20140529131015.GA72798@onelab2.iet.unipi.it> <003201cf7b44$bfd6ed40$3f84c7c0$@gmail.com> <20140529141559.GC74344@onelab2.iet.unipi.it> Date: Wed, 2 Jul 2014 10:48:37 +0800 Message-ID: Subject: Re: propose a new generic purpose rule option for ipfw From: bycn82 To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 02:48:37 -0000 Hi, Back from the world cup,Let's talk about the topic of U32, You guys are right, It is almost useless if it can only filter by the 5 tuples. But in my opinion, It can do "regex match" based on the pattern. So it can do something "layer7 filtering". Sure the performance is biggest issue. Any comments? On Thu, May 29, 2014 at 10:15 PM, Luigi Rizzo wrote: > On Thu, May 29, 2014 at 09:48:58PM +0800, bycn82 wrote: > > > > > > -----Original Message----- > > From: 'Luigi Rizzo' [mailto:rizzo@iet.unipi.it] > > Sent: 29 May, 2014 21:10 > > To: bycn82 > > Cc: 'FreeBSD Net' > > Subject: Re: propose a new generic purpose rule option for ipfw > > > > > > > > On Thu, May 29, 2014 at 08:45:26PM +0800, bycn82 wrote: > > > > ... > > > > > > > > > > Sure, that is the reason why developers are providing more and more > rule options. But the my question is do we have enough options to match all > the fixed position values? > > > > > > > > we do not have an option for fixed position matching. > > > > > > > > Can I say that ???It will be useful when a user come up with a special > requirement which cannot be fulfilled by any existing rule option.??? Since > there are so many rule options already. So I don???t know when that special > requirement will appear. L that is what you said ???useless???, I accept > that . > > please re-read what i said below. 'mostly useless' != 'useless', > and i am ok importing a clean implementation. > > > As i said, feel free to submit one and i will be happy to import it if > the code is clean (btw i am still waiting for fixes to the other 'rate > limiting' option you sent), but keep in mind that 'fixed position' is > mostly useless. > > > > Which `rate limiting`, the `Packet per second`? > > > > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/189720 > > ok commented the remaining problems with a separate email. > > > More useful options would be one where you express the position as > > > > > > > > '{MAC|VLAN|IP|UDP|TCP|...|PAYLOAD}+offset' > > > > > > > > It is possible, > > > > match > > > > the can be a pattern , then that means it can match multiple > value at the same time. > > what i wrote is a completely different thing. Never mind. > > cheers > luigi > > > > > > > > > so at least you can adapt to variant headers, or one where you can look > for a pattern in the entire packet or in a portion of it. > > > > > > > cheers > > > > luigi > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 02:53:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2669969A for ; Wed, 2 Jul 2014 02:53:05 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 02DE62B9B for ; Wed, 2 Jul 2014 02:53:04 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s622r35T061277 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 1 Jul 2014 19:53:04 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s622r2Eb061276; Tue, 1 Jul 2014 19:53:02 -0700 (PDT) (envelope-from jmg) Date: Tue, 1 Jul 2014 19:53:02 -0700 From: John-Mark Gurney To: Andrea Venturoli Subject: Re: MTU not regrowing? Message-ID: <20140702025302.GI45513@funkthat.com> Mail-Followup-To: Andrea Venturoli , John Hay , freebsd-net@freebsd.org References: <53A9C6D0.3090900@netfence.it> <20140624190333.GA13748@zibbi.meraka.csir.co.za> <53B2FFE9.3030906@netfence.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53B2FFE9.3030906@netfence.it> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 01 Jul 2014 19:53:04 -0700 (PDT) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 02:53:05 -0000 Andrea Venturoli wrote this message on Tue, Jul 01, 2014 at 20:37 +0200: > On 06/24/14 21:03, John Hay wrote: > > >Do a "route get somehost" and see what mtu is returned. > > You are right, I see a route with the old, lesser MTU. > > > > > >You might be able to delete or tweak that route. > > How do I do this? > I tried "route delete", but it doesn't help. route change -mtu XXX If you delete all the routes associated w/ the interface, the network route should be recreated iirc.. Or just run the above change command on all the necessary routes that have the incorrect mtu.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 03:51:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2D2734E for ; Wed, 2 Jul 2014 03:51:51 +0000 (UTC) Received: from emma.bazalt43.ru (emma.bazalt43.ru [91.193.143.8]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F83220FE for ; Wed, 2 Jul 2014 03:51:50 +0000 (UTC) Received: from [0.0.0.0] (oneex.me [91.193.143.91]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by emma.bazalt43.ru (Postfix) with ESMTPSA id 3806B17374 for ; Wed, 2 Jul 2014 09:45:25 +0600 (YEKT) Message-ID: <53B38059.1020603@hostelnet.ru> Date: Wed, 02 Jul 2014 09:45:29 +0600 From: Alex Ros Reply-To: noc@hostelnet.ru User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: FreeBSD 9 as PPPoE BRAS(mpd 5.7) kernel panic References: <53A719C3.3040002@hostelnet.ru> In-Reply-To: <53A719C3.3040002@hostelnet.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 03:51:52 -0000 Hi. So, hint with mpd-down script (with "sleep 1") does not help - panic after 7 days uptime (rev r267703). If somebody want to look into last core.txt and vmcore, they are here: http://pkg.hostelnet.ru/pub/dump/core.txt.9.txt From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 06:14:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2CC8DF for ; Wed, 2 Jul 2014 06:14:18 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 549DB2BCA for ; Wed, 2 Jul 2014 06:14:17 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id r20so8943767wiv.8 for ; Tue, 01 Jul 2014 23:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=berentweb.com; s=google; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=P71cyAgCWlWGdisXpHNx5Plrx1pJdcjbthVuQYCLL+Q=; b=R7BBEccsJ48kVKRk4PzrlLiQMHWOiWdYMn8jp3du3J8z/6XIeJN+qFlVg2jYl8dgT8 bz2P1+O/S+R7fWFIuCgA0o4bYUg2pZUJm3is5hEEMYXLNoIBqUdxYXBkcwoZ3lki+0MC V+Rl3ZxsfKREANU7QzAl0Kk9z/vvbYBABhiao= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=P71cyAgCWlWGdisXpHNx5Plrx1pJdcjbthVuQYCLL+Q=; b=lG93+hoTQfBbXVowHvr++kPij0qexB5HQGkzy9HjdV6onZzoh0yRxbsmJ/WEaTwZl7 AdocgLWuRpAX03Htu6zYPm5ZOYe0s/yrcVGbXKp7q5VScbos+rqfdaKuAq3w62cyamc0 0/mWpm8s8mEEoLVkmxmlAieWuGNFcY8CuY8wCqBEcVmaTEgDNlUA1Bnu1LQbW+gxO2ZR 7Dxh/iFr9yPPDT1fks/NGVzMGXXy4f0g4Jg62rYx0Sinhup833GzFOueBTEQEXVhCg+Y 8e46DZBygV094WVNObCh5A8NLZbvOS+3egjwqbuIUt0mfZdW2q+G/0MqXnjXl3YZe0tK pnPA== X-Gm-Message-State: ALoCoQnSWjZjgjdl//f6UsuNOsSbEx3jStu0W7h8esS9FuXCdXW7UK6r17qIBuXJwxJ0h5mIQ9Kg X-Received: by 10.180.210.165 with SMTP id mv5mr2160111wic.70.1404281655990; Tue, 01 Jul 2014 23:14:15 -0700 (PDT) Received: from rsbsd.rsb ([83.66.210.81]) by mx.google.com with ESMTPSA id m3sm52846974wjr.49.2014.07.01.23.14.14 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 01 Jul 2014 23:14:15 -0700 (PDT) Date: Wed, 2 Jul 2014 09:14:13 +0300 From: Beeblebrox To: Rick Macklem Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. Message-ID: <20140702091413.7e27e798@rsbsd.rsb> In-Reply-To: <1270654408.5439540.1404077062258.JavaMail.root@uoguelph.ca> References: <1404071407420-5924767.post@n5.nabble.com> <1270654408.5439540.1404077062258.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 06:14:18 -0000 Hi Rick, > Btw, there is BOOTP_DEBUG stuff in bootp_subr.c. If your debug kernel > didn't include that option, trying it might tell you where it breaks? I got around to trying the BOOTP_DEBUG knob, it breaks the kernel build right from start. bootp_subr.c does have that code, but the buildkernel does not like it. I got: unknown option "BOOTP_DEBUG" \ *** Error code 1 \ Stop. Kernel Config has these debug options turned on: options KDB options DDB options GDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options DEBUG_LOCKS options DEBUG_VFS_LOCKS options DIAGNOSTIC options BOOTP_DEBUG Regards. --=20 FreeBSD_amd64_11-Current_RadeonKMS From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 07:05:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86B0A37A for ; Wed, 2 Jul 2014 07:05:57 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 451F32044 for ; Wed, 2 Jul 2014 07:05:57 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id hy10so10078298vcb.1 for ; Wed, 02 Jul 2014 00:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sr/TD8DtNinnAFdAp0FEq6MbMr9cNxOwc5potztd0Uc=; b=ZvLrf7htTvlEc3BWzX4QJYkOkz3CLyO2YYE+qJU32az6BdQmsXfZ/Py3HuKD2vIu1s CL5A/gm5zYybrz3as5uilAkkW1SkHfFunh8dOHsBlQy4YTfj7anJQV2nwphhecxC6UIn 0UYRRAYs2p+xkywdgXyCpn9jjmS51Dsw+IO2jiY8muT/2SOg+3ifZ1p+tWtCuPT2Q0bg HxaXoxj40x2lgaKyXi4FLdmOuOq1cuYSvHE2nfNWaH0t6eDoGhPq0RE4XpnZk44P4OUi pq/vQ5E40ht6cMmFMyVng9tgVSyenXgG7Suz4d8OsYh4oJawCd0yaBOZG14kvMrIGOX2 +ORg== MIME-Version: 1.0 X-Received: by 10.221.49.200 with SMTP id vb8mr34485332vcb.17.1404284755934; Wed, 02 Jul 2014 00:05:55 -0700 (PDT) Received: by 10.221.4.132 with HTTP; Wed, 2 Jul 2014 00:05:55 -0700 (PDT) In-Reply-To: <53B364D5.9020700@bsdinfo.com.br> References: <53B364D5.9020700@bsdinfo.com.br> Date: Wed, 2 Jul 2014 00:05:55 -0700 Message-ID: Subject: Re: Network Intel X520-SR2 stopping From: Jack Vogel To: Marcelo Gondim Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 07:05:57 -0000 Is only one port a problem? When it gets into the state can you do a sysctl dev.ix.X.. Jack On Tue, Jul 1, 2014 at 6:48 PM, Marcelo Gondim wrote: > Hi all, > > I'm having problems with a 10GbE Intel X520-SR2 interface. After a running > time, the interface does not send or receive more data. I am obliged to > make a down and up the interface for it to return to work. Have changed the > interface, optical cords, optical modules and problem continues. > > (root@rt01)[~]# netstat -inh > Name Mtu Network Address Ipkts Ierrs Idrop Opkts > Oerrs Coll > igb0 1.5K 00:1e:67:9b:03:e1 0 0 0 0 > 0 0 > igb1 1.5K 00:1e:67:9b:03:e2 19M 0 0 27M > 0 0 > igb1 - 177.xx.xxx.0/ 177.xx.xxx.1 15K - - 14K > - - > igb1 - fe80::21e:67f fe80::21e:67ff:fe 2 - - 4 > - - > igb1 - 2804:xxxx:cad 2804:xxxx:cade:b1 14K - - 12K > - - > igb2 1.5K 00:1e:67:9b:03:e3 17M 5.8K 0 23M > 0 0 > igb2 - 177.xx.xxx.40 177.xx.xxx.41 10K - - 12K > - - > igb2 - fe80::21e:67f fe80::21e:67ff:fe 0 - - 1 > - - > igb2 - 2804:xxxx:bad 2804:xxxx:bad:b1f 14K - - 14K > - - > igb3 1.5K 00:1e:67:9b:03:e4 11K 0 0 441M > 0 0 > igb3 - 186.xxx.x.148 186.xxx.x.150 10K - - 94K > - - > igb3 - fe80::21e:67f fe80::21e:67ff:fe 0 - - 1 > - - > igb4 1.5K 00:1b:21:7b:ee:6c 5.8G 5.5K 0 194K > 0 0 > igb4 - 159.xx.xx.96/ 159.xx.xx.98 63K - - 18K > - - > igb4 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 > - - > igb4 - 2804:xxx:ffff 2804:xxx:ffff:ffd 17K - - 16K > - - > igb5 1.5K 00:1b:21:7b:ee:6d 6.9G 8.0K 0 9.1G > 0 0 > igb5 - 64.xxx.xxx.68 64.xxx.xxx.70 28K - - 5.5M > - - > igb5 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 135 > - - > igb5 - 2001:xxx:2001 2001:xxx:2001:100 17K - - 125K > - - > igb6 1.5K 00:1b:21:7b:ee:98 4.3G 0 0 3.8G > 0 0 > igb6 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 > - - > igb7 1.5K 00:1b:21:7b:ee:99 0 0 0 0 > 0 0 > *ix0 1.5K 00:1b:21:89:25:28 -1.5 118 1.5T 0.1T > 0 0* > ix0 - fe80::21b:21f fe80::21b:21ff:fe 0 - - 0 > - - > ix1 1.5K 00:1b:21:89:25:29 0 0 0 0 > 0 0 > pflog 32K 0 0 0 0 > 0 0 > pfsyn 1.5K 0 0 0 0 > 0 0 > lo0 16K 52K 0 0 52K > 0 0 > lo0 - ::1/128 ::1 0 - - 41 > - - > lo0 - fe80::1%lo0/6 fe80::1%lo0 0 - - 0 > - - > lo0 - 127.0.0.0/8 127.0.0.1 48K - - 52K > - - > disc0 64K 0 0 0 0 > 0 0 > > # uname -a > FreeBSD rt01.xxxxxx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #9 r267839: Tue > Jun 24 21:06:39 BRT 2014 root@rt01.xxxxxx.xxx.xx:/usr/obj/usr/src/sys/GONDIM > amd64 > You have new mail in /var/mail/root > > # netstat -ihw 1 > input (Total) output > packets errs idrops bytes packets errs bytes colls > 813K 0 0 522M 832K 9 615M 0 > 812K 0 0 523M 831K 9 616M 0 > 811K 0 0 521M 829K 6 614M 0 > 807K 0 0 519M 826K 15 615M 0 > 794K 0 0 514M 814K 7 610M 0 > 799K 0 0 520M 816K 7 612M 0 > 804K 0 0 523M 822K 14 613M 0 > 794K 0 0 518M 813K 14 610M 0 > 779K 0 0 510M 799K 8 601M 0 > 790K 0 0 514M 809K 7 603M 0 > 818K 0 0 525M 835K 7 617M 0 > 810K 0 0 522M 827K 12 614M 0 > 794K 0 0 514M 812K 13 603M 0 > 794K 0 0 518M 813K 13 608M 0 > 790K 0 0 516M 810K 5 606M 0 > 807K 0 0 525M 824K 9 614M 0 > 808K 0 0 529M 827K 10 619M 0 > 805K 0 0 524M 823K 7 614M 0 > 804K 0 0 526M 821K 8 615M 0 > 799K 0 0 520M 816K 6 612M 0 > 803K 0 0 522M 819K 11 610M 0 > > # ifconfig ix0 > ix0: flags=8843 metric 0 mtu 1500 > options=8404bb MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO> > ether 00:1b:21:89:25:28 > inet6 fe80::21b:21ff:fe89:2528%ix0 prefixlen 64 scopeid 0x9 > nd6 options=21 > media: Ethernet autoselect (10Gbase-SR ) > status: active > > ix0: > port 0xc020-0xc03f mem 0xec180000-0xec1fffff,0xec210000-0xec213fff irq 64 > at device 0.0 on pci131 > ix0: Using MSIX interrupts with 9 vectors > ix0: Ethernet address: 00:1b:21:89:25:28 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix1: > port 0xc000-0xc01f mem 0xec080000-0xec0fffff,0xec200000-0xec203fff irq 68 > at device 0.1 on pci131 > ix1: Using MSIX interrupts with 9 vectors > ix1: Ethernet address: 00:1b:21:89:25:29 > ix1: PCI Express Bus: Speed 5.0GT/s Width x8 > > Could be an issue with the ixgbe driver? > > Thanks > Gondim > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 07:18:44 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2669B6A8 for ; Wed, 2 Jul 2014 07:18:44 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF655213C for ; Wed, 2 Jul 2014 07:18:43 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id l6so9456483qcy.32 for ; Wed, 02 Jul 2014 00:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=00FBYX2imJ8UYFM4/2CIdwjG0h5u/TM7CguiF3dgczI=; b=aUAVNAZnZf4wxDz1FoOIFzV6c/KsA1sWxzpurW5A5LkVrGDoa9B6M8vb03nFqkgci6 MgsB+ulhQXG/O0R5of+jaMsuY0EPZIQpPU0Oe+a66l04ltCQqlJ+bnCIUf6IZxhnUKXh AndEuCBQl1XH5Z99ezHA6tA+US4RZH7xAfqeZ8Yh302+I/sYgTTfTr+hxVQzSTBbXpE8 FCzsBbhr7YBVLBuu+Poyx1/sSe4cnfaMSa9JbJ+6/xDxaBFCRts/0LNTuGhJ2wSV9Sxx WCCHhVd6rvHKYp5DMPPXf8FYv+pQqzxqbZTxgzQCSW64NvJhSl6fbXMZKtkBNM5zzF18 Gp2g== MIME-Version: 1.0 X-Received: by 10.140.94.225 with SMTP id g88mr74467529qge.101.1404285523036; Wed, 02 Jul 2014 00:18:43 -0700 (PDT) Received: by 10.140.35.239 with HTTP; Wed, 2 Jul 2014 00:18:42 -0700 (PDT) Date: Wed, 2 Jul 2014 12:48:42 +0530 Message-ID: Subject: Please help From: Sanuri Dananja To: net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 07:18:44 -0000 Hi sir, I'm trying to setup netmap on my machine on Ubuntu 12.04 LTS and having difficulties. I don't understand much as I am not a networking student. Could you please help me with the installation? My NICs are as follows: Wired: Atheros AR8151 PCI-E Gigabit Ethernet Controller (NDIS 6.20) Wireless: Atheros AR9002WB-1NG Wireless Network Adapter Can netmap be used with wireless connection? Can it be used with my NICs? I saw this on your website: " we provide an emulated mode on top of standard drivers.". Even though I didn't understand it exactly, I felt like your system can support any NIC. Our project is a tracking service provider optimized in performance, security and reliability. My goal is to achieve at least 1 million client connections per second. Could you please help me? This is kind of urgent because we have to finish our project within next 3 months. Thank you! Regards, Sanuri From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 08:28:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 569D4EFB for ; Wed, 2 Jul 2014 08:28:03 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 32FA32782 for ; Wed, 2 Jul 2014 08:28:02 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s628RslD066242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 2 Jul 2014 01:27:54 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s628RsCj066241; Wed, 2 Jul 2014 01:27:54 -0700 (PDT) (envelope-from jmg) Date: Wed, 2 Jul 2014 01:27:54 -0700 From: John-Mark Gurney To: Beeblebrox Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. Message-ID: <20140702082754.GM45513@funkthat.com> Mail-Followup-To: Beeblebrox , Rick Macklem , freebsd-net@freebsd.org References: <1404071407420-5924767.post@n5.nabble.com> <1270654408.5439540.1404077062258.JavaMail.root@uoguelph.ca> <20140702091413.7e27e798@rsbsd.rsb> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140702091413.7e27e798@rsbsd.rsb> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 02 Jul 2014 01:27:54 -0700 (PDT) Cc: freebsd-net@freebsd.org, Rick Macklem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 08:28:03 -0000 Beeblebrox wrote this message on Wed, Jul 02, 2014 at 09:14 +0300: > Hi Rick, > > > Btw, there is BOOTP_DEBUG stuff in bootp_subr.c. If your debug kernel > > didn't include that option, trying it might tell you where it breaks? > > I got around to trying the BOOTP_DEBUG knob, it breaks the kernel build > right from start. bootp_subr.c does have that code, but the buildkernel > does not like it. I got: > unknown option "BOOTP_DEBUG" \ *** Error code 1 \ Stop. > > Kernel Config has these debug options turned on: > options KDB > options DDB > options GDB > options INVARIANTS > options INVARIANT_SUPPORT > options WITNESS > options DEBUG_LOCKS > options DEBUG_VFS_LOCKS > options DIAGNOSTIC > options BOOTP_DEBUG Looks like BOOTP_DEBUG isn't a proper option... do the conf/options file, add the line: BOOTP_DEBUG opt_bootp.h then reconfig and build your kernel... and let me know if that works for you.. if it does, I'll add the option... Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 09:05:08 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FC05DC9 for ; Wed, 2 Jul 2014 09:05:08 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 32E822AF4 for ; Wed, 2 Jul 2014 09:05:07 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 8D0A87300A; Wed, 2 Jul 2014 11:08:32 +0200 (CEST) Date: Wed, 2 Jul 2014 11:08:32 +0200 From: Luigi Rizzo To: Sanuri Dananja Subject: Re: Please help Message-ID: <20140702090832.GB25059@onelab2.iet.unipi.it> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 09:05:08 -0000 On Wed, Jul 02, 2014 at 12:48:42PM +0530, Sanuri Dananja wrote: > Hi sir, > I'm trying to setup netmap on my machine on Ubuntu 12.04 LTS and having > difficulties. I don't understand much as I am not a networking student. > Could you please help me with the installation? > My NICs are as follows: > Wired: Atheros AR8151 PCI-E Gigabit Ethernet Controller (NDIS 6.20) > Wireless: Atheros AR9002WB-1NG Wireless Network Adapter > Can netmap be used with wireless connection? > Can it be used with my NICs? > I saw this on your website: " we provide an emulated mode on top of > standard drivers.". Even though I didn't understand it exactly, I felt like > your system can support any NIC. > Our project is a tracking service provider optimized in performance, > security and reliability. My goal is to achieve at least 1 million client > connections per second. > Could you please help me? This is kind of urgent because we have to finish > our project within next 3 months. > Thank you! i don't think netmap is useful for your project, at least unless you plan to do something very special (which you probably won't given the above description). cheers luigi > Regards, > Sanuri > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 09:14:33 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0AA52F0E for ; Wed, 2 Jul 2014 09:14:33 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE96D2BBB for ; Wed, 2 Jul 2014 09:14:32 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id v10so8580739qac.18 for ; Wed, 02 Jul 2014 02:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=7sXvJMPXErCt5uWSSSLpg9IhKNB4p5T5Z1+8U1c1SF8=; b=Or9I8++xIhHwERToslsXK4b8gOLZlCctvsV600mc2BCud40KSoBiG1gWKCj+UX0zSp drf+fk1y3MxV6mUDow9rL2m6BjbyEToY1sfjLj41uXExI00ATp0QWAB+U9ihFgMaGEbK vyII3OqHlrLsqoIUdvckpn6SgRG8s3z4gvBBgngDL4h0SWHTYlA3f1gBGjS0um48EPdW 29nS5L5d0QOw+r+KbswUMWhqjliBBaSdQz5PFaBDgXxVF0aK/ZW0it1PvrVGXiSxmskI 0oGsnIEvg7epmGH/NrXpjuXhS2tAL/p9o6EgAmvdLPuNfokWbSgLq9/pG6K/SxTUseH8 2heQ== X-Received: by 10.224.93.2 with SMTP id t2mr1183094qam.41.1404292471902; Wed, 02 Jul 2014 02:14:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Wed, 2 Jul 2014 02:13:51 -0700 (PDT) In-Reply-To: <20140702090832.GB25059@onelab2.iet.unipi.it> References: <20140702090832.GB25059@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Wed, 2 Jul 2014 10:13:51 +0100 Message-ID: Subject: Re: Please help To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Sanuri Dananja , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 09:14:33 -0000 Sanuri, in order to achieve your goal, you should look for service distribution over several systems (computers), basically a Cloud architecture. Depending on the language that your system is implemented, you should look for the appropriate framework. My regards, Carlos On 2 July 2014 10:08, Luigi Rizzo wrote: > On Wed, Jul 02, 2014 at 12:48:42PM +0530, Sanuri Dananja wrote: > > Hi sir, > > I'm trying to setup netmap on my machine on Ubuntu 12.04 LTS and having > > difficulties. I don't understand much as I am not a networking student. > > Could you please help me with the installation? > > My NICs are as follows: > > Wired: Atheros AR8151 PCI-E Gigabit Ethernet Controller (NDIS 6.20) > > Wireless: Atheros AR9002WB-1NG Wireless Network Adapter > > Can netmap be used with wireless connection? > > Can it be used with my NICs? > > I saw this on your website: " we provide an emulated mode on top of > > standard drivers.". Even though I didn't understand it exactly, I felt > like > > your system can support any NIC. > > Our project is a tracking service provider optimized in performance, > > security and reliability. My goal is to achieve at least 1 million client > > connections per second. > > Could you please help me? This is kind of urgent because we have to > finish > > our project within next 3 months. > > Thank you! > > i don't think netmap is useful for your project, > at least unless you plan to do something very special > (which you probably won't given the above description). > > cheers > luigi > > > Regards, > > Sanuri > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 10:44:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75B356E4 for ; Wed, 2 Jul 2014 10:44:43 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3D1702626 for ; Wed, 2 Jul 2014 10:44:42 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqoEAI7hs1ODaFve/2dsb2JhbABag19agm6oUgEBAQaTFIZtUwGBInWEAwEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASIGQgNqiWbMxeBLIREiGEBARs0B4J3gUwFmAKEM5JBg18hNYEFOQ X-IronPort-AV: E=Sophos;i="5.01,587,1400040000"; d="scan'208";a="137664109" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 02 Jul 2014 06:44:35 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1288EB3F46; Wed, 2 Jul 2014 06:44:36 -0400 (EDT) Date: Wed, 2 Jul 2014 06:44:36 -0400 (EDT) From: Rick Macklem To: Beeblebrox Message-ID: <338810855.6193094.1404297876056.JavaMail.root@uoguelph.ca> In-Reply-To: <20140702091413.7e27e798@rsbsd.rsb> Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 10:44:43 -0000 Beeblebrox wrote: > Hi Rick, > > > Btw, there is BOOTP_DEBUG stuff in bootp_subr.c. If your debug > > kernel > > didn't include that option, trying it might tell you where it > > breaks? > > I got around to trying the BOOTP_DEBUG knob, it breaks the kernel > build > right from start. bootp_subr.c does have that code, but the > buildkernel > does not like it. I got: > unknown option "BOOTP_DEBUG" \ *** Error code 1 \ Stop. > As the other post noted, you can try adding the option. Or, you can just add a line like: #define BOOTP_DEBUG 1 at the beginning of bootp_subr.c. I haven't tried it, so there might be other build issues. rick > Kernel Config has these debug options turned on: > options KDB > options DDB > options GDB > options INVARIANTS > options INVARIANT_SUPPORT > options WITNESS > options DEBUG_LOCKS > options DEBUG_VFS_LOCKS > options DIAGNOSTIC > options BOOTP_DEBUG > > Regards. > > -- > FreeBSD_amd64_11-Current_RadeonKMS > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 11:41:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 675F3A36 for ; Wed, 2 Jul 2014 11:41:12 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BA892B00 for ; Wed, 2 Jul 2014 11:41:12 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s62BfACu069067 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 2 Jul 2014 04:41:10 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s62BfAW2069066; Wed, 2 Jul 2014 04:41:10 -0700 (PDT) (envelope-from jmg) Date: Wed, 2 Jul 2014 04:41:10 -0700 From: John-Mark Gurney To: Rick Macklem Subject: Re: PXE boot using Grub bootloader fails at mountroot; no PXE devs. Message-ID: <20140702114110.GP45513@funkthat.com> Mail-Followup-To: Rick Macklem , Beeblebrox , freebsd-net@freebsd.org References: <20140702091413.7e27e798@rsbsd.rsb> <338810855.6193094.1404297876056.JavaMail.root@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <338810855.6193094.1404297876056.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 02 Jul 2014 04:41:10 -0700 (PDT) Cc: freebsd-net@freebsd.org, Beeblebrox X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 11:41:12 -0000 Rick Macklem wrote this message on Wed, Jul 02, 2014 at 06:44 -0400: > Beeblebrox wrote: > > Hi Rick, > > > > > Btw, there is BOOTP_DEBUG stuff in bootp_subr.c. If your debug > > > kernel > > > didn't include that option, trying it might tell you where it > > > breaks? > > > > I got around to trying the BOOTP_DEBUG knob, it breaks the kernel > > build > > right from start. bootp_subr.c does have that code, but the > > buildkernel > > does not like it. I got: > > unknown option "BOOTP_DEBUG" \ *** Error code 1 \ Stop. > > > As the other post noted, you can try adding the option. > Or, you can just add a line like: > #define BOOTP_DEBUG 1 > at the beginning of bootp_subr.c. > > I haven't tried it, so there might be other build issues. I was going to suggest that, but decided against it so that we might fix it for others to use... > > Kernel Config has these debug options turned on: > > options KDB > > options DDB > > options GDB > > options INVARIANTS > > options INVARIANT_SUPPORT > > options WITNESS > > options DEBUG_LOCKS > > options DEBUG_VFS_LOCKS > > options DIAGNOSTIC > > options BOOTP_DEBUG -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 11:49:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CA21E19 for ; Wed, 2 Jul 2014 11:49:21 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 07CD62B5E for ; Wed, 2 Jul 2014 11:49:20 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 06C89139CC for ; Wed, 2 Jul 2014 08:49:41 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:mime-version:user-agent:from:from:date:date:message-id; s=dkim; t=1404301779; x=1405165780; bh=JqFuT9uWe67YNB1aSWfSKbEo +tGXuqRgKmoWEhiC4mY=; b=jOc1E3prM0AIKxcdzMj+ThWNYxEymwAzr9YmWy+t SLVdvlWp6Ffm1RQajnFiOzAoUhU63VJAL/hYKQKoiKZUX1MmOsORl2V2I8O+Oeer OL50AcgWqDEmiLluHJdzWMg2VAtpweoesJ0GCOP69JB5sg0SEE3TD6CFE7iiVx0M 8/8= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qAaGoRn9W5Nh for ; Wed, 2 Jul 2014 08:49:39 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id E2B6E139CB for ; Wed, 2 Jul 2014 08:49:38 -0300 (BRT) Message-ID: <53B3F1B6.9010606@bsdinfo.com.br> Date: Wed, 02 Jul 2014 08:49:10 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 CC: FreeBSD Net Subject: Re: Network Intel X520-SR2 stopping References: <53B364D5.9020700@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 11:49:21 -0000 Hi Jack, The problem occurs with both ports. When it happens the problem will make a sysctl dev.ix.0. and post here. Another doubt: this interface uses Intel SFP+ optical module and is connected to one Datacom equipment with XFPmodule. Incompatibility may exist between them that is causing this? Gondim Em 02/07/2014 04:05, Jack Vogel escreveu: > Is only one port a problem? When it gets into the state can you > do a sysctl dev.ix.X.. > > Jack > > > > On Tue, Jul 1, 2014 at 6:48 PM, Marcelo Gondim > wrote: > > Hi all, > > I'm having problems with a 10GbE Intel X520-SR2 interface. After a > running time, the interface does not send or receive more data. I > am obliged to make a down and up the interface for it to return to > work. Have changed the interface, optical cords, optical modules > and problem continues. > > (root@rt01)[~]# netstat -inh > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll > igb0 1.5K 00:1e:67:9b:03:e1 0 0 0 > 0 0 0 > igb1 1.5K 00:1e:67:9b:03:e2 19M 0 0 > 27M 0 0 > igb1 - 177.xx.xxx.0/ 177.xx.xxx.1 15K - - > 14K - - > igb1 - fe80::21e:67f fe80::21e:67ff:fe 2 - - > 4 - - > igb1 - 2804:xxxx:cad 2804:xxxx:cade:b1 14K - - > 12K - - > igb2 1.5K 00:1e:67:9b:03:e3 17M 5.8K 0 > 23M 0 0 > igb2 - 177.xx.xxx.40 177.xx.xxx.41 10K - - > 12K - - > igb2 - fe80::21e:67f fe80::21e:67ff:fe 0 - - > 1 - - > igb2 - 2804:xxxx:bad 2804:xxxx:bad:b1f 14K - - > 14K - - > igb3 1.5K 00:1e:67:9b:03:e4 11K 0 0 > 441M 0 0 > igb3 - 186.xxx.x.148 186.xxx.x.150 10K - - > 94K - - > igb3 - fe80::21e:67f fe80::21e:67ff:fe 0 - - > 1 - - > igb4 1.5K 00:1b:21:7b:ee:6c 5.8G 5.5K 0 > 194K 0 0 > igb4 - 159.xx.xx.96/ 159.xx.xx.98 63K - - > 18K - - > igb4 - fe80::21b:21f fe80::21b:21ff:fe 0 - - > 0 - - > igb4 - 2804:xxx:ffff 2804:xxx:ffff:ffd 17K - - > 16K - - > igb5 1.5K 00:1b:21:7b:ee:6d 6.9G 8.0K 0 > 9.1G 0 0 > igb5 - 64.xxx.xxx.68 64.xxx.xxx.70 28K - - > 5.5M - - > igb5 - fe80::21b:21f fe80::21b:21ff:fe 0 - - > 135 - - > igb5 - 2001:xxx:2001 2001:xxx:2001:100 17K - - > 125K - - > igb6 1.5K 00:1b:21:7b:ee:98 4.3G 0 0 > 3.8G 0 0 > igb6 - fe80::21b:21f fe80::21b:21ff:fe 0 - - > 0 - - > igb7 1.5K 00:1b:21:7b:ee:99 0 0 0 > 0 0 0 > *ix0 1.5K 00:1b:21:89:25:28 -1.5 118 1.5T > 0.1T 0 0* > ix0 - fe80::21b:21f fe80::21b:21ff:fe 0 - - > 0 - - > ix1 1.5K 00:1b:21:89:25:29 0 0 0 > 0 0 0 > pflog 32K 0 0 0 > 0 0 0 > pfsyn 1.5K 0 0 0 > 0 0 0 > lo0 16K 52K 0 0 > 52K 0 0 > lo0 - ::1/128 ::1 0 - - > 41 - - > lo0 - fe80::1%lo0/6 fe80::1%lo0 0 - - > 0 - - > lo0 - 127.0.0.0/8 127.0.0.1 > 48K - - 52K - - > disc0 64K 0 0 0 > 0 0 0 > > # uname -a > FreeBSD rt01.xxxxxx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #9 > r267839: Tue Jun 24 21:06:39 BRT 2014 > root@rt01.xxxxxx.xxx.xx:/usr/obj/usr/src/sys/GONDIM amd64 > You have new mail in /var/mail/root > > # netstat -ihw 1 > input (Total) output > packets errs idrops bytes packets errs bytes colls > 813K 0 0 522M 832K 9 615M 0 > 812K 0 0 523M 831K 9 616M 0 > 811K 0 0 521M 829K 6 614M 0 > 807K 0 0 519M 826K 15 615M 0 > 794K 0 0 514M 814K 7 610M 0 > 799K 0 0 520M 816K 7 612M 0 > 804K 0 0 523M 822K 14 613M 0 > 794K 0 0 518M 813K 14 610M 0 > 779K 0 0 510M 799K 8 601M 0 > 790K 0 0 514M 809K 7 603M 0 > 818K 0 0 525M 835K 7 617M 0 > 810K 0 0 522M 827K 12 614M 0 > 794K 0 0 514M 812K 13 603M 0 > 794K 0 0 518M 813K 13 608M 0 > 790K 0 0 516M 810K 5 606M 0 > 807K 0 0 525M 824K 9 614M 0 > 808K 0 0 529M 827K 10 619M 0 > 805K 0 0 524M 823K 7 614M 0 > 804K 0 0 526M 821K 8 615M 0 > 799K 0 0 520M 816K 6 612M 0 > 803K 0 0 522M 819K 11 610M 0 > > # ifconfig ix0 > ix0: flags=8843 metric 0 > mtu 1500 > options=8404bb > ether 00:1b:21:89:25:28 > inet6 fe80::21b:21ff:fe89:2528%ix0 prefixlen 64 scopeid 0x9 > nd6 options=21 > media: Ethernet autoselect (10Gbase-SR ) > status: active > > ix0: 2.5.15> port 0xc020-0xc03f mem > 0xec180000-0xec1fffff,0xec210000-0xec213fff irq 64 at device 0.0 > on pci131 > ix0: Using MSIX interrupts with 9 vectors > ix0: Ethernet address: 00:1b:21:89:25:28 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix1: 2.5.15> port 0xc000-0xc01f mem > 0xec080000-0xec0fffff,0xec200000-0xec203fff irq 68 at device 0.1 > on pci131 > ix1: Using MSIX interrupts with 9 vectors > ix1: Ethernet address: 00:1b:21:89:25:29 > ix1: PCI Express Bus: Speed 5.0GT/s Width x8 > > Could be an issue with the ixgbe driver? > > Thanks > Gondim > From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 16:39:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9048F18 for ; Wed, 2 Jul 2014 16:39:41 +0000 (UTC) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx1.freebsd.org (Postfix) with ESMTP id 90BD3286E for ; Wed, 2 Jul 2014 16:39:41 +0000 (UTC) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 02 Jul 2014 09:39:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,589,1400050800"; d="scan'208";a="564254438" Received: from orsmsx103.amr.corp.intel.com ([10.22.225.130]) by fmsmga002.fm.intel.com with ESMTP; 02 Jul 2014 09:39:16 -0700 Received: from orsmsx115.amr.corp.intel.com (10.22.240.11) by ORSMSX103.amr.corp.intel.com (10.22.225.130) with Microsoft SMTP Server (TLS) id 14.3.123.3; Wed, 2 Jul 2014 09:39:07 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.92]) by ORSMSX115.amr.corp.intel.com ([169.254.10.193]) with mapi id 14.03.0123.003; Wed, 2 Jul 2014 09:39:07 -0700 From: "Pieper, Jeffrey E" To: Marcelo Gondim Subject: RE: Network Intel X520-SR2 stopping Thread-Topic: Network Intel X520-SR2 stopping Thread-Index: AQHPlZh6VHUnosl1P06zGcow0dLU/5uM0kmAgABPJAD//9rPAA== Date: Wed, 2 Jul 2014 16:39:06 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> References: <53B364D5.9020700@bsdinfo.com.br> <53B3F1B6.9010606@bsdinfo.com.br> In-Reply-To: <53B3F1B6.9010606@bsdinfo.com.br> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 16:39:41 -0000 Is there any way that you can try a reproduce this using a B2B configurati= on, or something that doesn't use XFP as a link partner? I'm thinking that = you are correct regarding an incompatibility issue between SFP+ and XFP. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Marcelo Gondim Sent: Wednesday, July 02, 2014 4:49 AM Cc: FreeBSD Net Subject: Re: Network Intel X520-SR2 stopping Hi Jack, The problem occurs with both ports. When it happens the problem will make a sysctl dev.ix.0. and post here. Another doubt: this interface uses Intel SFP+ optical module and is=20 connected to one Datacom equipment with XFPmodule. Incompatibility may=20 exist between them that is causing this? Gondim Em 02/07/2014 04:05, Jack Vogel escreveu: > Is only one port a problem? When it gets into the state can you > do a sysctl dev.ix.X.. > > Jack > > > > On Tue, Jul 1, 2014 at 6:48 PM, Marcelo Gondim > wrote: > > Hi all, > > I'm having problems with a 10GbE Intel X520-SR2 interface. After a > running time, the interface does not send or receive more data. I > am obliged to make a down and up the interface for it to return to > work. Have changed the interface, optical cords, optical modules > and problem continues. > > (root@rt01)[~]# netstat -inh > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll > igb0 1.5K 00:1e:67:9b:03:e1 0 0 0 =20 > 0 0 0 > igb1 1.5K 00:1e:67:9b:03:e2 19M 0 0 =20 > 27M 0 0 > igb1 - 177.xx.xxx.0/ 177.xx.xxx.1 15K - - =20 > 14K - - > igb1 - fe80::21e:67f fe80::21e:67ff:fe 2 - - =20 > 4 - - > igb1 - 2804:xxxx:cad 2804:xxxx:cade:b1 14K - - =20 > 12K - - > igb2 1.5K 00:1e:67:9b:03:e3 17M 5.8K 0 =20 > 23M 0 0 > igb2 - 177.xx.xxx.40 177.xx.xxx.41 10K - - =20 > 12K - - > igb2 - fe80::21e:67f fe80::21e:67ff:fe 0 - - =20 > 1 - - > igb2 - 2804:xxxx:bad 2804:xxxx:bad:b1f 14K - - =20 > 14K - - > igb3 1.5K 00:1e:67:9b:03:e4 11K 0 0 =20 > 441M 0 0 > igb3 - 186.xxx.x.148 186.xxx.x.150 10K - - =20 > 94K - - > igb3 - fe80::21e:67f fe80::21e:67ff:fe 0 - - =20 > 1 - - > igb4 1.5K 00:1b:21:7b:ee:6c 5.8G 5.5K 0 =20 > 194K 0 0 > igb4 - 159.xx.xx.96/ 159.xx.xx.98 63K - - =20 > 18K - - > igb4 - fe80::21b:21f fe80::21b:21ff:fe 0 - - =20 > 0 - - > igb4 - 2804:xxx:ffff 2804:xxx:ffff:ffd 17K - - =20 > 16K - - > igb5 1.5K 00:1b:21:7b:ee:6d 6.9G 8.0K 0 =20 > 9.1G 0 0 > igb5 - 64.xxx.xxx.68 64.xxx.xxx.70 28K - - > 5.5M - - > igb5 - fe80::21b:21f fe80::21b:21ff:fe 0 - - =20 > 135 - - > igb5 - 2001:xxx:2001 2001:xxx:2001:100 17K - - > 125K - - > igb6 1.5K 00:1b:21:7b:ee:98 4.3G 0 0 =20 > 3.8G 0 0 > igb6 - fe80::21b:21f fe80::21b:21ff:fe 0 - - =20 > 0 - - > igb7 1.5K 00:1b:21:7b:ee:99 0 0 0 =20 > 0 0 0 > *ix0 1.5K 00:1b:21:89:25:28 -1.5 118 1.5T =20 > 0.1T 0 0* > ix0 - fe80::21b:21f fe80::21b:21ff:fe 0 - - =20 > 0 - - > ix1 1.5K 00:1b:21:89:25:29 0 0 0 =20 > 0 0 0 > pflog 32K 0 0 0 =20 > 0 0 0 > pfsyn 1.5K 0 0 0 =20 > 0 0 0 > lo0 16K 52K 0 0 =20 > 52K 0 0 > lo0 - ::1/128 ::1 0 - - =20 > 41 - - > lo0 - fe80::1%lo0/6 fe80::1%lo0 0 - - =20 > 0 - - > lo0 - 127.0.0.0/8 127.0.0.1 =20 > 48K - - 52K - - > disc0 64K 0 0 0 =20 > 0 0 0 > > # uname -a > FreeBSD rt01.xxxxxx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #9 > r267839: Tue Jun 24 21:06:39 BRT 2014 > root@rt01.xxxxxx.xxx.xx:/usr/obj/usr/src/sys/GONDIM amd64 > You have new mail in /var/mail/root > > # netstat -ihw 1 > input (Total) output > packets errs idrops bytes packets errs bytes colls > 813K 0 0 522M 832K 9 615M 0 > 812K 0 0 523M 831K 9 616M 0 > 811K 0 0 521M 829K 6 614M 0 > 807K 0 0 519M 826K 15 615M 0 > 794K 0 0 514M 814K 7 610M 0 > 799K 0 0 520M 816K 7 612M 0 > 804K 0 0 523M 822K 14 613M 0 > 794K 0 0 518M 813K 14 610M 0 > 779K 0 0 510M 799K 8 601M 0 > 790K 0 0 514M 809K 7 603M 0 > 818K 0 0 525M 835K 7 617M 0 > 810K 0 0 522M 827K 12 614M 0 > 794K 0 0 514M 812K 13 603M 0 > 794K 0 0 518M 813K 13 608M 0 > 790K 0 0 516M 810K 5 606M 0 > 807K 0 0 525M 824K 9 614M 0 > 808K 0 0 529M 827K 10 619M 0 > 805K 0 0 524M 823K 7 614M 0 > 804K 0 0 526M 821K 8 615M 0 > 799K 0 0 520M 816K 6 612M 0 > 803K 0 0 522M 819K 11 610M 0 > > # ifconfig ix0 > ix0: flags=3D8843 metric 0 > mtu 1500 > options=3D8404bb > ether 00:1b:21:89:25:28 > inet6 fe80::21b:21ff:fe89:2528%ix0 prefixlen 64 scopeid 0x9 > nd6 options=3D21 > media: Ethernet autoselect (10Gbase-SR ) > status: active > > ix0: 2.5.15> port 0xc020-0xc03f mem > 0xec180000-0xec1fffff,0xec210000-0xec213fff irq 64 at device 0.0 > on pci131 > ix0: Using MSIX interrupts with 9 vectors > ix0: Ethernet address: 00:1b:21:89:25:28 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix1: 2.5.15> port 0xc000-0xc01f mem > 0xec080000-0xec0fffff,0xec200000-0xec203fff irq 68 at device 0.1 > on pci131 > ix1: Using MSIX interrupts with 9 vectors > ix1: Ethernet address: 00:1b:21:89:25:29 > ix1: PCI Express Bus: Speed 5.0GT/s Width x8 > > Could be an issue with the ixgbe driver? > > Thanks > Gondim > _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 17:15:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DB44B35 for ; Wed, 2 Jul 2014 17:15:38 +0000 (UTC) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id 7BEE62C11 for ; Wed, 2 Jul 2014 17:15:36 +0000 (UTC) Received: (qmail 68570 invoked from network); 2 Jul 2014 17:08:54 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 2 Jul 2014 17:08:54 -0000 Date: Wed, 02 Jul 2014 19:07:39 +0200 (CEST) Message-Id: <20140702.190739.74686622.sthaug@nethelp.no> To: jeffrey.e.pieper@intel.com Subject: Re: Network Intel X520-SR2 stopping From: sthaug@nethelp.no In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, gondim@bsdinfo.com.br X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 17:15:38 -0000 > Is there any way that you can try a reproduce this using a B2B configuration, or something that doesn't use XFP as a link partner? I'm thinking that you are correct regarding an incompatibility issue between SFP+ and XFP. Why do you believe that? The optical signals are the same for SFP+ and XFP. We have lots of 10G SFP+ / XFP links in production. It just works... Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 17:17:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDD8BE96 for ; Wed, 2 Jul 2014 17:17:20 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx1.freebsd.org (Postfix) with ESMTP id C5CE32C4A for ; Wed, 2 Jul 2014 17:17:20 +0000 (UTC) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 02 Jul 2014 10:15:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,589,1400050800"; d="scan'208";a="567381444" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by orsmga002.jf.intel.com with ESMTP; 02 Jul 2014 10:15:24 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.92]) by ORSMSX101.amr.corp.intel.com ([169.254.8.157]) with mapi id 14.03.0123.003; Wed, 2 Jul 2014 10:15:24 -0700 From: "Pieper, Jeffrey E" To: "sthaug@nethelp.no" Subject: RE: Network Intel X520-SR2 stopping Thread-Topic: Network Intel X520-SR2 stopping Thread-Index: AQHPlZh6VHUnosl1P06zGcow0dLU/5uM0kmAgABPJAD//9rPAIAAfi2A//+LYrA= Date: Wed, 2 Jul 2014 17:15:24 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8F8658@ORSMSX111.amr.corp.intel.com> References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> In-Reply-To: <20140702.190739.74686622.sthaug@nethelp.no> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" , "gondim@bsdinfo.com.br" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 17:17:21 -0000 It depends on the spec of the XFP module, as well as the cabling. The spec = of both types of modules needs to match. I asked the OP to test B2B to elim= inate the XFP as the problem. If you have any suggestions, I'd be happy to= hear them.=20 Jeff -----Original Message----- From: sthaug@nethelp.no [mailto:sthaug@nethelp.no]=20 Sent: Wednesday, July 02, 2014 10:08 AM To: Pieper, Jeffrey E Cc: gondim@bsdinfo.com.br; freebsd-net@freebsd.org Subject: Re: Network Intel X520-SR2 stopping > Is there any way that you can try a reproduce this using a B2B configura= tion, or something that doesn't use XFP as a link partner? I'm thinking tha= t you are correct regarding an incompatibility issue between SFP+ and XFP. Why do you believe that? The optical signals are the same for SFP+ and XFP. We have lots of 10G SFP+ / XFP links in production. It just works... Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 17:31:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B87142F for ; Wed, 2 Jul 2014 17:31:10 +0000 (UTC) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id A90A82DA3 for ; Wed, 2 Jul 2014 17:31:08 +0000 (UTC) Received: (qmail 69912 invoked from network); 2 Jul 2014 17:31:08 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 2 Jul 2014 17:31:08 -0000 Date: Wed, 02 Jul 2014 19:29:53 +0200 (CEST) Message-Id: <20140702.192953.41693242.sthaug@nethelp.no> To: jeffrey.e.pieper@intel.com Subject: Re: Network Intel X520-SR2 stopping From: sthaug@nethelp.no In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8F8658@ORSMSX111.amr.corp.intel.com> References: <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> <2A35EA60C3C77D438915767F458D65687E8F8658@ORSMSX111.amr.corp.intel.com> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, gondim@bsdinfo.com.br X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 17:31:10 -0000 > It depends on the spec of the XFP module, as well as the cabling. The spec of both types of modules needs to match. I asked the OP to test B2B to eliminate the XFP as the problem. If you have any suggestions, I'd be happy to hear them. Obviously transceivers and cabling need to match - SR against SR, LR against LR, etc. However, whether the 10GBase-X signal is generated by an XFP transceiver or an SFP+ transceiver is irrelevant, as long as it follows spec ... Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 19:24:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2C9A4E3 for ; Wed, 2 Jul 2014 19:24:03 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EE35287D for ; Wed, 2 Jul 2014 19:24:02 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id D42C9139CE for ; Wed, 2 Jul 2014 16:24:22 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1404329041; x=1405193042; bh=ScMwVx8noNYwJ2pwfZ+L3XAbbjuaunV2yJf LfpMWFEQ=; b=ZHfg9eRvhMqRph8chwaj1v3oSkHMMIThVhNwVDRvoAo/3s+dvHe Zdzu39bII7o6diLcZvDOZqcHNj47O0Fa7/M3rYm813VOVRR8Jqy+cmnfCMKyozd7 WlzJEUpyVJzAh3eKFFOWndxfR6sLHm02GByT2vSlIcvUTzp1WkhWRMw8= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d8wK4biXXdad for ; Wed, 2 Jul 2014 16:24:01 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 0B781139CD; Wed, 2 Jul 2014 16:24:00 -0300 (BRT) Message-ID: <53B45C33.1010501@bsdinfo.com.br> Date: Wed, 02 Jul 2014 16:23:31 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: sthaug@nethelp.no, jeffrey.e.pieper@intel.com Subject: Re: Network Intel X520-SR2 stopping References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> In-Reply-To: <20140702.190739.74686622.sthaug@nethelp.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 19:24:03 -0000 Em 02/07/2014 14:07, sthaug@nethelp.no escreveu: >> Is there any way that you can try a reproduce this using a B2B configuration, or something that doesn't use XFP as a link partner? I'm thinking that you are correct regarding an incompatibility issue between SFP+ and XFP. > Why do you believe that? The optical signals are the same for SFP+ > and XFP. > > We have lots of 10G SFP+ / XFP links in production. It just works... > > Steinar Haug, Nethelp consulting, sthaug@nethelp.no > I think I found the problem. Our SFP+ optical module is 850nm MMF and our transport operator is using an XFP 1310nmMMF. I am waiting for them to exchange the module and see the result. Once the module is changed, I post here. Thanks and best regards, Gondim From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 23:50:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3F2B95F for ; Wed, 2 Jul 2014 23:50:41 +0000 (UTC) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FCBA20A3 for ; Wed, 2 Jul 2014 23:50:41 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id ij19so11075576vcb.9 for ; Wed, 02 Jul 2014 16:50:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=4BGYnRQVwXeSmv/5Nru8WUKT+J8/Tx0UD5lwwPoxrbU=; b=gweAsgmB077R4shH3hkHE6iT7+NAk1ZWO3imYGZ7v3FC1srTiDom8uSLH2GsRvse/k AX+ouvzzml3Wie46JIg04QNuutiCove0LL0AaeTxqa49QhTU7q849vgJ52z+kFna9Qkf iI3Vxc0mLFJh55ozTq1T/U4Y21JeO4uLAvf/v5lphSDRHAt/864c+QJ9hwzcY79piqJO eFx9xN9korwbXdb35vnXWPIvFWMCiAJ0YT+IiZy9us4LBbdaKvzpO1Ez/sTb+AqGsozm om4vTSqNZwq1BXzPe0+8SULz/EgfgAzkqV0sMAYgNcygtN/PuqQPtmp1ZbxmYD1XeC6s Lx8w== X-Gm-Message-State: ALoCoQkkjr1GNYqyn7RLAbLFTkhXcZBxvgtNwutM9qtc5CLq05kXczuQWm5SuI6rxU3HP4gTWRuN MIME-Version: 1.0 X-Received: by 10.58.119.75 with SMTP id ks11mr716354veb.20.1404345034201; Wed, 02 Jul 2014 16:50:34 -0700 (PDT) Received: by 10.58.11.167 with HTTP; Wed, 2 Jul 2014 16:50:34 -0700 (PDT) X-Originating-IP: [72.177.8.109] Date: Wed, 2 Jul 2014 18:50:34 -0500 Message-ID: Subject: Add netbw option to systat From: Bryan Venteicher To: freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary=001a11c21fc0e8049204fd3e8e11 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 23:50:41 -0000 --001a11c21fc0e8049204fd3e8e11 Content-Type: text/plain; charset=UTF-8 Awhile back, DragonlyFlyBSD added a netbw option to systat that I've ported to FreeBSD and found handy at various times: netbw Display aggregate and per-connection TCP receive and transmit rates. Only active TCP connections are shown. Leading to output such as: tcp accepts connects rcv 1.192G snd 15.77K rexmit 192.168.10.80:22 192.168.10.20:23103 rcv snd 415.7 [ NTSX ] 192.168.10.80:22 192.168.10.20:46560 rcv 19.80M snd 14.47K [ NTSX ] 192.168.10.80:22 192.168.10.20:60699 rcv snd 886.3 [ NTSX ] 192.168.10.81:5201 192.168.10.51:60844 rcv 293.2M snd [R TSX ] 192.168.10.81:5201 192.168.10.51:60845 rcv 293.5M snd [R TSX ] 192.168.10.81:5201 192.168.10.51:60846 rcv 293.2M snd [R TSX ] 192.168.10.81:5201 192.168.10.51:60847 rcv 292.9M snd [R TSX ] It uses the sequences number from the 'struct tcpcb' to derive the rates, which is usually good but certainly not perfect (i.e., don't set the interval too long). I'd like to commit this if anybody else thinks they'd find it useful. http://people.freebsd.org/~bryanv/patches/systat-netbw.patch --001a11c21fc0e8049204fd3e8e11 Content-Type: text/x-patch; charset=US-ASCII; name="systat-netbw.patch" Content-Disposition: attachment; filename="systat-netbw.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hx5alovy0 RnJvbSBkMGE0ZjI4MmYzZTM2ZWI1M2NiMGE1MGE2NGFhNDU5N2U1MmI3ZDQyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBCcnlhbiBWZW50ZWljaGVyIDxicnlhbnZAZGFlbW9uaW50aGVj bG9zZXQub3JnPgpEYXRlOiBUdWUsIDEgSnVsIDIwMTQgMDA6NTE6MjkgLTA1MDAKU3ViamVjdDog W1BBVENIXSBBZGQgJ25ldGJ3JyBkaXNwbGF5IHRvIHN5c3RhdAoKLS0tCiB1c3IuYmluL3N5c3Rh dC9NYWtlZmlsZSB8ICAgMiArLQogdXNyLmJpbi9zeXN0YXQvY21kdGFiLmMgfCAgIDMgKwogdXNy LmJpbi9zeXN0YXQvZXh0ZXJuLmggfCAgIDcgKwogdXNyLmJpbi9zeXN0YXQvbmV0YncuYyAgfCA0 NzYgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrCiB1c3Iu YmluL3N5c3RhdC9zeXN0YXQuMSB8ICAgNCArCiA1IGZpbGVzIGNoYW5nZWQsIDQ5MSBpbnNlcnRp b25zKCspLCAxIGRlbGV0aW9uKC0pCiBjcmVhdGUgbW9kZSAxMDA2NDQgdXNyLmJpbi9zeXN0YXQv bmV0YncuYwoKZGlmZiAtLWdpdCBhL3Vzci5iaW4vc3lzdGF0L01ha2VmaWxlIGIvdXNyLmJpbi9z eXN0YXQvTWFrZWZpbGUKaW5kZXggMWJiMmRhMC4uYTE3ZTRkZCAxMDA2NDQKLS0tIGEvdXNyLmJp bi9zeXN0YXQvTWFrZWZpbGUKKysrIGIvdXNyLmJpbi9zeXN0YXQvTWFrZWZpbGUKQEAgLTcsNyAr Nyw3IEBAIFBST0c9CXN5c3RhdAogU1JDUz0JY21kcy5jIGNtZHRhYi5jIGRldnMuYyBmZXRjaC5j IGlvc3RhdC5jIGtleWJvYXJkLmMgbWFpbi5jIFwKIAluZXRjbWRzLmMgbmV0c3RhdC5jIHBpZ3Mu YyBzd2FwLmMgaWNtcC5jIFwKIAltb2RlLmMgaXAuYyB0Y3AuYyBcCi0Jdm1zdGF0LmMgY29udnRi bC5jIGlmY21kcy5jIGlmc3RhdC5jCisJdm1zdGF0LmMgY29udnRibC5jIGlmY21kcy5jIGlmc3Rh dC5jIG5ldGJ3LmMKIAogLmlmICR7TUtfSU5FVDZfU1VQUE9SVH0gIT0gIm5vIgogU1JDUys9CWlj bXA2LmMgaXA2LmMKZGlmZiAtLWdpdCBhL3Vzci5iaW4vc3lzdGF0L2NtZHRhYi5jIGIvdXNyLmJp bi9zeXN0YXQvY21kdGFiLmMKaW5kZXggYzljOWU3ZC4uMGUyMjVlYyAxMDA2NDQKLS0tIGEvdXNy LmJpbi9zeXN0YXQvY21kdGFiLmMKKysrIGIvdXNyLmJpbi9zeXN0YXQvY21kdGFiLmMKQEAgLTU1 LDYgKzU1LDkgQEAgc3RydWN0CWNtZHRhYiBjbWR0YWJbXSA9IHsKIAl7ICJuZXRzdGF0IiwJc2hv d25ldHN0YXQsCWZldGNobmV0c3RhdCwJbGFiZWxuZXRzdGF0LAogCSAgaW5pdG5ldHN0YXQsCW9w ZW5uZXRzdGF0LAljbG9zZW5ldHN0YXQsCWNtZG5ldHN0YXQsCiAJICAwLAkJQ0ZfTE9BREFWIH0s CisJeyAibmV0YnciLAlzaG93bmV0YncsCWZldGNobmV0YncsCWxhYmVsbmV0YncsCisJICBpbml0 bmV0YncsCW9wZW5uZXRidywJY2xvc2VuZXRidywJTlVMTCwKKwkgIDAsCQkwIH0sCiAJeyAiaWNt cCIsCXNob3dpY21wLAlmZXRjaGljbXAsCWxhYmVsaWNtcCwKIAkgIGluaXRpY21wLAlvcGVuaWNt cCwJY2xvc2VpY21wLAljbWRtb2RlLAogCSAgcmVzZXRpY21wLAlDRl9MT0FEQVYgfSwKZGlmZiAt LWdpdCBhL3Vzci5iaW4vc3lzdGF0L2V4dGVybi5oIGIvdXNyLmJpbi9zeXN0YXQvZXh0ZXJuLmgK aW5kZXggMTdmZmZjMS4uMzhkNDA4NCAxMDA2NDQKLS0tIGEvdXNyLmJpbi9zeXN0YXQvZXh0ZXJu LmgKKysrIGIvdXNyLmJpbi9zeXN0YXQvZXh0ZXJuLmgKQEAgLTc2LDYgKzc2LDcgQEAgdm9pZAkg Y2xvc2Vpb3N0YXQoV0lORE9XICopOwogdm9pZAkgY2xvc2VpcChXSU5ET1cgKik7CiB2b2lkCSBj bG9zZWlwNihXSU5ET1cgKik7CiB2b2lkCSBjbG9zZWtyZShXSU5ET1cgKik7Cit2b2lkCSBjbG9z ZW5ldGJ3KFdJTkRPVyAqKTsKIHZvaWQJIGNsb3NlbmV0c3RhdChXSU5ET1cgKik7CiB2b2lkCSBj bG9zZXBpZ3MoV0lORE9XICopOwogdm9pZAkgY2xvc2Vzd2FwKFdJTkRPVyAqKTsKQEAgLTgzLDYg Kzg0LDcgQEAgdm9pZAkgY2xvc2V0Y3AoV0lORE9XICopOwogaW50CSBjbWRpZnN0YXQoY29uc3Qg Y2hhciAqLCBjb25zdCBjaGFyICopOwogaW50CSBjbWRpb3N0YXQoY29uc3QgY2hhciAqLCBjb25z dCBjaGFyICopOwogaW50CSBjbWRrcmUoY29uc3QgY2hhciAqLCBjb25zdCBjaGFyICopOworaW50 CSBjbWRuZXRidyhjb25zdCBjaGFyICosIGNvbnN0IGNoYXIgKik7CiBpbnQJIGNtZG5ldHN0YXQo Y29uc3QgY2hhciAqLCBjb25zdCBjaGFyICopOwogc3RydWN0CSBjbWR0YWIgKmxvb2t1cChjb25z dCBjaGFyICopOwogdm9pZAkgY29tbWFuZChjb25zdCBjaGFyICopOwpAQCAtOTgsNiArMTAwLDcg QEAgdm9pZAkgZmV0Y2hpcCh2b2lkKTsKIHZvaWQJIGZldGNoaXA2KHZvaWQpOwogdm9pZAkgZmV0 Y2hpb3N0YXQodm9pZCk7CiB2b2lkCSBmZXRjaGtyZSh2b2lkKTsKK3ZvaWQJIGZldGNobmV0Ynco dm9pZCk7CiB2b2lkCSBmZXRjaG5ldHN0YXQodm9pZCk7CiB2b2lkCSBmZXRjaHBpZ3Modm9pZCk7 CiB2b2lkCSBmZXRjaHN3YXAodm9pZCk7CkBAIC0xMTEsNiArMTE0LDcgQEAgaW50CSBpbml0aXAo dm9pZCk7CiBpbnQJIGluaXRpcDYodm9pZCk7CiBpbnQJIGluaXRpb3N0YXQodm9pZCk7CiBpbnQJ IGluaXRrcmUodm9pZCk7CitpbnQJIGluaXRuZXRidyh2b2lkKTsKIGludAkgaW5pdG5ldHN0YXQo dm9pZCk7CiBpbnQJIGluaXRwaWdzKHZvaWQpOwogaW50CSBpbml0c3dhcCh2b2lkKTsKQEAgLTEy NCw2ICsxMjgsNyBAQCB2b2lkCSBsYWJlbGlwKHZvaWQpOwogdm9pZAkgbGFiZWxpcDYodm9pZCk7 CiB2b2lkCSBsYWJlbGlvc3RhdCh2b2lkKTsKIHZvaWQJIGxhYmVsa3JlKHZvaWQpOwordm9pZAkg bGFiZWxuZXRidyh2b2lkKTsKIHZvaWQJIGxhYmVsbmV0c3RhdCh2b2lkKTsKIHZvaWQJIGxhYmVs cGlncyh2b2lkKTsKIHZvaWQJIGxhYmVscyh2b2lkKTsKQEAgLTEzOSw2ICsxNDQsNyBAQCBXSU5E T1cJKm9wZW5pcCh2b2lkKTsKIFdJTkRPVwkqb3BlbmlwNih2b2lkKTsKIFdJTkRPVwkqb3Blbmlv c3RhdCh2b2lkKTsKIFdJTkRPVwkqb3BlbmtyZSh2b2lkKTsKK1dJTkRPVwkqb3Blbm5ldGJ3KHZv aWQpOwogV0lORE9XCSpvcGVubmV0c3RhdCh2b2lkKTsKIFdJTkRPVwkqb3BlbnBpZ3Modm9pZCk7 CiBXSU5ET1cJKm9wZW5zd2FwKHZvaWQpOwpAQCAtMTU2LDYgKzE2Miw3IEBAIHZvaWQJIHNob3dp cCh2b2lkKTsKIHZvaWQJIHNob3dpcDYodm9pZCk7CiB2b2lkCSBzaG93aW9zdGF0KHZvaWQpOwog dm9pZAkgc2hvd2tyZSh2b2lkKTsKK3ZvaWQJIHNob3duZXRidyh2b2lkKTsKIHZvaWQJIHNob3du ZXRzdGF0KHZvaWQpOwogdm9pZAkgc2hvd3BpZ3Modm9pZCk7CiB2b2lkCSBzaG93c3dhcCh2b2lk KTsKZGlmZiAtLWdpdCBhL3Vzci5iaW4vc3lzdGF0L25ldGJ3LmMgYi91c3IuYmluL3N5c3RhdC9u ZXRidy5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAuLjc4NWFmNWYKLS0tIC9k ZXYvbnVsbAorKysgYi91c3IuYmluL3N5c3RhdC9uZXRidy5jCkBAIC0wLDAgKzEsNDc2IEBACisv KgorICogQ29weXJpZ2h0IChjKSAyMDEzIFRoZSBEcmFnb25GbHkgUHJvamVjdC4gIEFsbCByaWdo dHMgcmVzZXJ2ZWQuCisgKgorICogVGhpcyBjb2RlIGlzIGRlcml2ZWQgZnJvbSBzb2Z0d2FyZSBj b250cmlidXRlZCB0byBUaGUgRHJhZ29uRmx5IFByb2plY3QKKyAqIGJ5IE1hdHRoZXcgRGlsbG9u IDxkaWxsb25AYmFja3BsYW5lLmNvbT4KKyAqCisgKiBSZWRpc3RyaWJ1dGlvbiBhbmQgdXNlIGlu IHNvdXJjZSBhbmQgYmluYXJ5IGZvcm1zLCB3aXRoIG9yIHdpdGhvdXQKKyAqIG1vZGlmaWNhdGlv biwgYXJlIHBlcm1pdHRlZCBwcm92aWRlZCB0aGF0IHRoZSBmb2xsb3dpbmcgY29uZGl0aW9ucwor ICogYXJlIG1ldDoKKyAqCisgKiAxLiBSZWRpc3RyaWJ1dGlvbnMgb2Ygc291cmNlIGNvZGUgbXVz dCByZXRhaW4gdGhlIGFib3ZlIGNvcHlyaWdodAorICogICAgbm90aWNlLCB0aGlzIGxpc3Qgb2Yg Y29uZGl0aW9ucyBhbmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVyLgorICogMi4gUmVkaXN0cmli dXRpb25zIGluIGJpbmFyeSBmb3JtIG11c3QgcmVwcm9kdWNlIHRoZSBhYm92ZSBjb3B5cmlnaHQK KyAqICAgIG5vdGljZSwgdGhpcyBsaXN0IG9mIGNvbmRpdGlvbnMgYW5kIHRoZSBmb2xsb3dpbmcg ZGlzY2xhaW1lciBpbgorICogICAgdGhlIGRvY3VtZW50YXRpb24gYW5kL29yIG90aGVyIG1hdGVy aWFscyBwcm92aWRlZCB3aXRoIHRoZQorICogICAgZGlzdHJpYnV0aW9uLgorICogMy4gTmVpdGhl ciB0aGUgbmFtZSBvZiBUaGUgRHJhZ29uRmx5IFByb2plY3Qgbm9yIHRoZSBuYW1lcyBvZiBpdHMK KyAqICAgIGNvbnRyaWJ1dG9ycyBtYXkgYmUgdXNlZCB0byBlbmRvcnNlIG9yIHByb21vdGUgcHJv ZHVjdHMgZGVyaXZlZAorICogICAgZnJvbSB0aGlzIHNvZnR3YXJlIHdpdGhvdXQgc3BlY2lmaWMs IHByaW9yIHdyaXR0ZW4gcGVybWlzc2lvbi4KKyAqCisgKiBUSElTIFNPRlRXQVJFIElTIFBST1ZJ REVEIEJZIFRIRSBDT1BZUklHSFQgSE9MREVSUyBBTkQgQ09OVFJJQlVUT1JTCisgKiBgYEFTIElT JycgQU5EIEFOWSBFWFBSRVNTIE9SIElNUExJRUQgV0FSUkFOVElFUywgSU5DTFVESU5HLCBCVVQg Tk9UCisgKiBMSU1JVEVEIFRPLCBUSEUgSU1QTElFRCBXQVJSQU5USUVTIE9GIE1FUkNIQU5UQUJJ TElUWSBBTkQgRklUTkVTUworICogRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFIEFSRSBESVNDTEFJ TUVELiAgSU4gTk8gRVZFTlQgU0hBTEwgVEhFCisgKiBDT1BZUklHSFQgSE9MREVSUyBPUiBDT05U UklCVVRPUlMgQkUgTElBQkxFIEZPUiBBTlkgRElSRUNULCBJTkRJUkVDVCwKKyAqIElOQ0lERU5U QUwsIFNQRUNJQUwsIEVYRU1QTEFSWSBPUiBDT05TRVFVRU5USUFMIERBTUFHRVMgKElOQ0xVRElO RywKKyAqIEJVVCBOT1QgTElNSVRFRCBUTywgUFJPQ1VSRU1FTlQgT0YgU1VCU1RJVFVURSBHT09E UyBPUiBTRVJWSUNFUzsKKyAqIExPU1MgT0YgVVNFLCBEQVRBLCBPUiBQUk9GSVRTOyBPUiBCVVNJ TkVTUyBJTlRFUlJVUFRJT04pIEhPV0VWRVIgQ0FVU0VECisgKiBBTkQgT04gQU5ZIFRIRU9SWSBP RiBMSUFCSUxJVFksIFdIRVRIRVIgSU4gQ09OVFJBQ1QsIFNUUklDVCBMSUFCSUxJVFksCisgKiBP UiBUT1JUIChJTkNMVURJTkcgTkVHTElHRU5DRSBPUiBPVEhFUldJU0UpIEFSSVNJTkcgSU4gQU5Z IFdBWSBPVVQKKyAqIE9GIFRIRSBVU0UgT0YgVEhJUyBTT0ZUV0FSRSwgRVZFTiBJRiBBRFZJU0VE IE9GIFRIRSBQT1NTSUJJTElUWSBPRgorICogU1VDSCBEQU1BR0UuCisgKi8KKyNpbmNsdWRlIDxz eXMvcGFyYW0uaD4KKyNpbmNsdWRlIDxzeXMvcXVldWUuaD4KKyNpbmNsdWRlIDxzeXMvdHJlZS5o PgorI2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4KKyNpbmNsdWRlIDxzeXMvc29ja2V0dmFyLmg+Cisj aW5jbHVkZSA8c3lzL3Byb3Rvc3cuaD4KKyNpbmNsdWRlIDxzeXMvc3lzY3RsLmg+CisjaW5jbHVk ZSA8c3lzL3RpbWUuaD4KKworI2luY2x1ZGUgPG5ldGluZXQvaW4uaD4KKyNpbmNsdWRlIDxhcnBh L2luZXQuaD4KKyNpbmNsdWRlIDxuZXQvcm91dGUuaD4KKyNpbmNsdWRlIDxuZXRpbmV0L2luX3N5 c3RtLmg+CisjaW5jbHVkZSA8bmV0aW5ldC9pcC5oPgorI2lmZGVmIElORVQ2CisjaW5jbHVkZSA8 bmV0aW5ldC9pcDYuaD4KKyNlbmRpZgorI2luY2x1ZGUgPG5ldGluZXQvaW5fcGNiLmg+CisjaW5j bHVkZSA8bmV0aW5ldC9pcF9pY21wLmg+CisjaW5jbHVkZSA8bmV0aW5ldC9pY21wX3Zhci5oPgor I2luY2x1ZGUgPG5ldGluZXQvaXBfdmFyLmg+CisjaW5jbHVkZSA8bmV0aW5ldC90Y3AuaD4KKyNp bmNsdWRlIDxuZXRpbmV0L3RjcGlwLmg+CisjaW5jbHVkZSA8bmV0aW5ldC90Y3Bfc2VxLmg+Cisj aW5jbHVkZSA8bmV0aW5ldC90Y3BfZnNtLmg+CisjaW5jbHVkZSA8bmV0aW5ldC90Y3BfdGltZXIu aD4KKyNpbmNsdWRlIDxuZXRpbmV0L3RjcF92YXIuaD4KKyNpbmNsdWRlIDxuZXRpbmV0L3RjcF9k ZWJ1Zy5oPgorI2luY2x1ZGUgPG5ldGluZXQvdWRwLmg+CisjaW5jbHVkZSA8bmV0aW5ldC91ZHBf dmFyLmg+CisKKyNpbmNsdWRlIDxlcnIuaD4KKyNpbmNsdWRlIDxlcnJuby5oPgorI2luY2x1ZGUg PG5ldGRiLmg+CisjaW5jbHVkZSA8c3RkbGliLmg+CisjaW5jbHVkZSA8c3RyaW5nLmg+CisjaW5j bHVkZSA8bmxpc3QuaD4KKyNpbmNsdWRlIDxwYXRocy5oPgorI2luY2x1ZGUgInN5c3RhdC5oIgor I2luY2x1ZGUgImV4dGVybi5oIgorCisvKiBSZWR1Y2UgZGlmZiBmcm9tIERyYWdvbmZseUJTRC4g Ki8KK3VuaW9uIGluX2RlcGVuZGFkZHIgeworCXN0cnVjdCBpbl9hZGRyXzRpbjYgaWQ0Nl9hZGRy OworCXN0cnVjdCBpbjZfYWRkciBpZDZfYWRkcjsKK307Cit0eXBlZGVmIGludDMyX3QgdGNwX3Nl cV9kaWZmX3Q7CisjZGVmaW5lIHM2X2FkZHIxNiBfX3U2X2FkZHIuX191Nl9hZGRyMTYKKworc3Ry dWN0IG15dGNwY2IgeworCVJCX0VOVFJZKG15dGNwY2IpCXJiX25vZGU7CisJaW50CQkJc2VxOwor CXN0cnVjdCB4dGNwY2IJCXh0Y3A7CisJc3RydWN0IHh0Y3BjYgkJbGFzdF94dGNwOworfTsKKwor c3RhdGljIGludAorbXl0Y3BjYl9jbXAoc3RydWN0IG15dGNwY2IgKnRjcDEsIHN0cnVjdCBteXRj cGNiICp0Y3AyKQoreworCWludCByOworCisJLyoKKwkgKiBMb3cgbG9jYWwgb3IgZm9yZWlnbiBw b3J0IGNvbWVzIGZpcnN0IChsb2NhbCBoYXMgcHJpb3JpdHkpLgorCSAqLworCWlmIChudG9ocyh0 Y3AxLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9scG9ydCkgPj0gMTAyNCAmJgorCSAg ICBudG9ocyh0Y3AyLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9scG9ydCkgPj0gMTAy NCkgeworCQlpZiAobnRvaHModGNwMS0+eHRjcC54dF9pbnAuaW5wX2luYy5pbmNfaWUuaWVfZnBv cnQpIDwKKwkJICAgIG50b2hzKHRjcDItPnh0Y3AueHRfaW5wLmlucF9pbmMuaW5jX2llLmllX2Zw b3J0KSkKKwkJCXJldHVybigtMSk7CisJCWlmIChudG9ocyh0Y3AxLT54dGNwLnh0X2lucC5pbnBf aW5jLmluY19pZS5pZV9mcG9ydCkgPgorCQkgICAgbnRvaHModGNwMi0+eHRjcC54dF9pbnAuaW5w X2luYy5pbmNfaWUuaWVfZnBvcnQpKQorCQkJcmV0dXJuKDEpOworCX0KKworCWlmIChudG9ocyh0 Y3AxLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9scG9ydCkgPAorCSAgICBudG9ocyh0 Y3AyLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9scG9ydCkpCisJCXJldHVybigtMSk7 CisJaWYgKG50b2hzKHRjcDEtPnh0Y3AueHRfaW5wLmlucF9pbmMuaW5jX2llLmllX2xwb3J0KSA+ CisJICAgIG50b2hzKHRjcDItPnh0Y3AueHRfaW5wLmlucF9pbmMuaW5jX2llLmllX2xwb3J0KSkK KwkJcmV0dXJuKDEpOworCWlmIChudG9ocyh0Y3AxLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19p ZS5pZV9mcG9ydCkgPAorCSAgICBudG9ocyh0Y3AyLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19p ZS5pZV9mcG9ydCkpCisJCXJldHVybigtMSk7CisJaWYgKG50b2hzKHRjcDEtPnh0Y3AueHRfaW5w LmlucF9pbmMuaW5jX2llLmllX2Zwb3J0KSA+CisJICAgIG50b2hzKHRjcDItPnh0Y3AueHRfaW5w LmlucF9pbmMuaW5jX2llLmllX2Zwb3J0KSkKKwkJcmV0dXJuKDEpOworCisJLyoKKwkgKiBTb3J0 IElQVjQgdnMgSVBWNiBhZGRyZXNzZXMKKwkgKi8KKwlpZiAoKHRjcDEtPnh0Y3AueHRfaW5wLmlu cF92ZmxhZyAmIChJTlBfSVBWNHxJTlBfSVBWNikpIDwKKwkgICAgKHRjcDItPnh0Y3AueHRfaW5w LmlucF92ZmxhZyAmIChJTlBfSVBWNHxJTlBfSVBWNikpKQorCQlyZXR1cm4oLTEpOworCWlmICgo dGNwMS0+eHRjcC54dF9pbnAuaW5wX3ZmbGFnICYgKElOUF9JUFY0fElOUF9JUFY2KSkgPgorCSAg ICAodGNwMi0+eHRjcC54dF9pbnAuaW5wX3ZmbGFnICYgKElOUF9JUFY0fElOUF9JUFY2KSkpCisJ CXJldHVybigxKTsKKworCS8qCisJICogTG9jYWwgYW5kIGZvcmVpZ24gYWRkcmVzc2VzCisJICov CisJaWYgKHRjcDEtPnh0Y3AueHRfaW5wLmlucF92ZmxhZyAmIElOUF9JUFY0KSB7CisJCWlmIChu dG9obCh0Y3AxLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9sYWRkci5zX2FkZHIpIDwK KwkJICAgIG50b2hsKHRjcDItPnh0Y3AueHRfaW5wLmlucF9pbmMuaW5jX2llLmllX2xhZGRyLnNf YWRkcikpCisJCQlyZXR1cm4oLTEpOworCQlpZiAobnRvaGwodGNwMS0+eHRjcC54dF9pbnAuaW5w X2luYy5pbmNfaWUuaWVfbGFkZHIuc19hZGRyKSA+CisJCSAgICBudG9obCh0Y3AyLT54dGNwLnh0 X2lucC5pbnBfaW5jLmluY19pZS5pZV9sYWRkci5zX2FkZHIpKQorCQkJcmV0dXJuKDEpOworCQlp ZiAobnRvaGwodGNwMS0+eHRjcC54dF9pbnAuaW5wX2luYy5pbmNfaWUuaWVfZmFkZHIuc19hZGRy KSA8CisJCSAgICBudG9obCh0Y3AyLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9mYWRk ci5zX2FkZHIpKQorCQkJcmV0dXJuKC0xKTsKKwkJaWYgKG50b2hsKHRjcDEtPnh0Y3AueHRfaW5w LmlucF9pbmMuaW5jX2llLmllX2ZhZGRyLnNfYWRkcikgPgorCQkgICAgbnRvaGwodGNwMi0+eHRj cC54dF9pbnAuaW5wX2luYy5pbmNfaWUuaWVfZmFkZHIuc19hZGRyKSkKKwkJCXJldHVybigxKTsK Kwl9IGVsc2UgaWYgKHRjcDEtPnh0Y3AueHRfaW5wLmlucF92ZmxhZyAmIElOUF9JUFY2KSB7CisJ CXIgPSBiY21wKCZ0Y3AxLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZTZfZmFkZHIsCisJ CQkgJnRjcDItPnh0Y3AueHRfaW5wLmlucF9pbmMuaW5jX2llLmllNl9mYWRkciwKKwkJCSBzaXpl b2YodGNwMS0+eHRjcC54dF9pbnAuaW5wX2luYy5pbmNfaWUuaWU2X2ZhZGRyKSk7CisJCWlmIChy KQorCQkJcmV0dXJuKHIpOworCX0gZWxzZSB7CisJCXIgPSBiY21wKCZ0Y3AxLT54dGNwLnh0X2lu cC5pbnBfaW5jLmluY19pZS5pZTZfZmFkZHIsCisJCQkgJnRjcDItPnh0Y3AueHRfaW5wLmlucF9p bmMuaW5jX2llLmllNl9mYWRkciwKKwkJCSBzaXplb2YodGNwMS0+eHRjcC54dF9pbnAuaW5wX2lu Yy5pbmNfaWUuaWU2X2ZhZGRyKSk7CisJCWlmIChyKQorCQkJcmV0dXJuKHIpOworCX0KKwlyZXR1 cm4oMCk7Cit9CisKK3N0cnVjdCBteXRjcGNiX3RyZWU7CitzdGF0aWMgUkJfSEVBRChteXRjcGNi X3RyZWUsIG15dGNwY2IpOworUkJfUFJPVE9UWVBFX1NUQVRJQyhteXRjcGNiX3RyZWUsIG15dGNw Y2IsIHJiX25vZGUsIG15dGNwY2JfY21wKTsKK1JCX0dFTkVSQVRFX1NUQVRJQyhteXRjcGNiX3Ry ZWUsIG15dGNwY2IsIHJiX25vZGUsIG15dGNwY2JfY21wKTsKKworc3RhdGljIHN0cnVjdCBteXRj cGNiX3RyZWUgbXl0Y3BfdHJlZTsKK3N0YXRpYyBzdHJ1Y3QgdGltZXZhbCB0dl9jdXJyOworc3Rh dGljIHN0cnVjdCB0aW1ldmFsIHR2X2xhc3Q7CitzdGF0aWMgc3RydWN0IHRjcHN0YXQgdGNwX2N1 cnI7CitzdGF0aWMgc3RydWN0IHRjcHN0YXQgdGNwX2xhc3Q7CitzdGF0aWMgaW50IHRjcF9wY2Jf c2VxOworCitzdGF0aWMgY29uc3QgY2hhciAqbnVtdG9rKGRvdWJsZSB2YWx1ZSk7CitzdGF0aWMg dm9pZCBuZXRid2xpbmUoaW50IHJvdywgc3RydWN0IG15dGNwY2IgKmVsbSwgZG91YmxlIGRlbHRh X3RpbWUpOworc3RhdGljIGNvbnN0IGNoYXIgKm5ldGFkZHJzdHIodV9jaGFyIHZmbGFncywgdW5p b24gaW5fZGVwZW5kYWRkciAqZGVwYWRkciwKKwkJICAgICAgIHVfaW50MTZfdCBwb3J0KTsKK3N0 YXRpYyB2b2lkIHVwZGF0ZXBjYihzdHJ1Y3QgeHRjcGNiICp4dGNwKTsKKworI2RlZmluZSBERUxU QVJBVEUoZmllbGQpCVwKKwkoKGRvdWJsZSkodGNwX2N1cnIuZmllbGQgLSB0Y3BfbGFzdC5maWVs ZCkgLyBkZWx0YV90aW1lKQorCisjZGVmaW5lIERFTFRBRUxNKGZpZWxkKQkJXAorCSgoZG91Ymxl KSh0Y3Bfc2VxX2RpZmZfdCkoZWxtLT54dGNwLmZpZWxkIC0JCVwKKwkJCQkgIGVsbS0+bGFzdF94 dGNwLmZpZWxkKSAvCVwKKwkgZGVsdGFfdGltZSkKKworI2RlZmluZSBERUxUQUVMTVNDQUxFKGZp ZWxkLCBzY2FsZSkJCVwKKwkoKGRvdWJsZSkoKHRjcF9zZXFfZGlmZl90KShlbG0tPnh0Y3AuZmll bGQgLQkJXAorCQkJCSAgIGVsbS0+bGFzdF94dGNwLmZpZWxkKSA8PCBzY2FsZSkgLyBcCisJIGRl bHRhX3RpbWUpCisKK1dJTkRPVyAqCitvcGVubmV0Yncodm9pZCkKK3sKKwlSQl9JTklUKCZteXRj cF90cmVlKTsKKwlyZXR1cm4gKHN1YndpbihzdGRzY3IsIExJTkVTLTAtMSwgMCwgMCwgMCkpOwor fQorCit2b2lkCitjbG9zZW5ldGJ3KFdJTkRPVyAqdykKK3sKKwlzdHJ1Y3QgbXl0Y3BjYiAqbXl0 Y3A7CisKKwl3aGlsZSAoKG15dGNwID0gUkJfUk9PVCgmbXl0Y3BfdHJlZSkpICE9IE5VTEwpIHsK KwkJUkJfUkVNT1ZFKG15dGNwY2JfdHJlZSwgJm15dGNwX3RyZWUsIG15dGNwKTsKKwkJZnJlZSht eXRjcCk7CisJfQorCisJaWYgKHcgIT0gTlVMTCkgeworCQl3Y2xlYXIodyk7CisJCXdyZWZyZXNo KHcpOworCQlkZWx3aW4odyk7CisJfQorfQorCitpbnQKK2luaXRuZXRidyh2b2lkKQoreworCXJl dHVybigxKTsKK30KKwordm9pZAorZmV0Y2huZXRidyh2b2lkKQoreworCXN0cnVjdCB4aW5wZ2Vu ICppbnBnOworCXN0cnVjdCB4dGNwY2IgKnh0cDsKKwljaGFyICpjdXIsICplbmQ7CisJc2l6ZV90 IGxlbjsKKworCS8qCisJICogRXh0cmFjdCBQQ0IgbGlzdAorCSAqLworCWlucGcgPSAoc3RydWN0 IHhpbnBnZW4gKilzeXNjdGxfZHlucmVhZCgibmV0LmluZXQudGNwLnBjYmxpc3QiLCAmbGVuKTsK KwlpZiAoaW5wZyA9PSBOVUxMKQorCQlyZXR1cm47CisJLyoKKwkgKiBXZSBjdXJyZW50bHkgZG8g bm8gcmVxdWlyZSBhIGNvbnNpc3RlbnQgcGNiIGxpc3QuCisJICogVHJ5IHRvIGJlIHJvYnVzdCBp biBjYXNlIG9mIHN0cnVjdCBzaXplIGNoYW5nZXMKKwkgKi8KKwljdXIgPSAoKGNoYXIgKilpbnBn KSArIGlucGctPnhpZ19sZW47CisJLyogVGhlcmUgaXMgYWxzbyBhIHRyYWlsaW5nIHN0cnVjdCB4 aW5wZ2VuICovCisJZW5kID0gKChjaGFyICopaW5wZykgKyBsZW4gLSBpbnBnLT54aWdfbGVuOwor CWlmIChlbmQgPD0gY3VyKSB7CisJCWZyZWUoaW5wZyk7CisJCXJldHVybjsKKwl9CisJKyt0Y3Bf cGNiX3NlcTsKKwl4dHAgPSAoc3RydWN0IHh0Y3BjYiAqKWN1cjsKKwl3aGlsZSAoY3VyICsgeHRw LT54dF9sZW4gPD0gZW5kKSB7CisJCXVwZGF0ZXBjYih4dHApOworCQljdXIgKz0geHRwLT54dF9s ZW47CisJCXh0cCA9IChzdHJ1Y3QgeHRjcGNiICopY3VyOworCX0KKwlmcmVlKGlucGcpOworCisJ LyoKKwkgKiBHZW5lcmFsIHN0YXRzCisJICovCisJdGNwX2xhc3QgPSB0Y3BfY3VycjsKKwlsZW4g PSBzaXplb2YodGNwX2N1cnIpOworCWlmIChzeXNjdGxieW5hbWUoIm5ldC5pbmV0LnRjcC5zdGF0 cyIsICZ0Y3BfY3VyciwgJmxlbiwgTlVMTCwgMCkgPCAwKQorCQlyZXR1cm47CisJdHZfbGFzdCA9 IHR2X2N1cnI7CisJZ2V0dGltZW9mZGF5KCZ0dl9jdXJyLCBOVUxMKTsKK30KKwordm9pZAorbGFi ZWxuZXRidyh2b2lkKQoreworCXdtb3ZlKHduZCwgMCwgMCk7CisJd2NscnRvYm90KHduZCk7Cisj aWYgMAorCW12d2FkZHN0cih3bmQsIDAsIExBRERSLCAiTG9jYWwgQWRkcmVzcyIpOworCW12d2Fk ZHN0cih3bmQsIDAsIEZBRERSLCAiRm9yZWlnbiBBZGRyZXNzIik7CisJbXZ3YWRkc3RyKHduZCwg MCwgUFJPVE8sICJQcm90byIpOworCW12d2FkZHN0cih3bmQsIDAsIFJDVkNDLCAiUmVjdi1RIik7 CisJbXZ3YWRkc3RyKHduZCwgMCwgU05EQ0MsICJTZW5kLVEiKTsKKwltdndhZGRzdHIod25kLCAw LCBTVEFURSwgIihzdGF0ZSkiKTsKKyNlbmRpZgorfQorCit2b2lkCitzaG93bmV0Yncodm9pZCkK K3sKKwlkb3VibGUgZGVsdGFfdGltZTsKKwlzdHJ1Y3QgbXl0Y3BjYiAqZWxtLCAqdGVsbTsKKwlp bnQgcm93OworCisJZGVsdGFfdGltZSA9IChkb3VibGUpKHR2X2N1cnIudHZfc2VjIC0gdHZfbGFz dC50dl9zZWMpIC0gMS4wICsKKwkJICAgICAodHZfY3Vyci50dl91c2VjICsgMTAwMDAwMCAtIHR2 X2xhc3QudHZfdXNlYykgLyAxZTY7CisJaWYgKGRlbHRhX3RpbWUgPCAwLjEpCisJCXJldHVybjsK KworCW12d3ByaW50dyh3bmQsIDAsIDAsCisJCSAgInRjcCBhY2NlcHRzICVzIGNvbm5lY3RzICVz ICIKKwkJICAiICAgICAgICAgcmN2ICVzIHNuZCAlcyByZXhtaXQgJXMiLAorCQkgIG51bXRvayhE RUxUQVJBVEUodGNwc19hY2NlcHRzKSksCisJCSAgbnVtdG9rKERFTFRBUkFURSh0Y3BzX2Nvbm5l Y3RzKSAtIERFTFRBUkFURSh0Y3BzX2FjY2VwdHMpKSwKKwkJICBudW10b2soREVMVEFSQVRFKHRj cHNfcmN2Ynl0ZSkpLAorCQkgIG51bXRvayhERUxUQVJBVEUodGNwc19zbmRieXRlKSksCisJCSAg bnVtdG9rKERFTFRBUkFURSh0Y3BzX3NuZHJleG1pdGJ5dGUpKSk7CisKKwlyb3cgPSAyOworCVJC X0ZPUkVBQ0hfU0FGRShlbG0sIG15dGNwY2JfdHJlZSwgJm15dGNwX3RyZWUsIHRlbG0pIHsKKwkJ aWYgKGVsbS0+c2VxID09IHRjcF9wY2Jfc2VxICYmCisJCSAgICAoZWxtLT54dGNwLnh0X3NvY2tl dC5zb19yY3Yuc2JfY2MgfHwKKwkJICAgICBlbG0tPnh0Y3AueHRfc29ja2V0LnNvX3NuZC5zYl9j YyB8fAorCQkgICAgIERFTFRBRUxNKHh0X3RwLnNuZF9tYXgpIHx8CisJCSAgICAgREVMVEFFTE0o eHRfdHAucmN2X254dCkKKwkJICAgICkpIHsKKwkJCWlmIChyb3cgPCBMSU5FUyAtIDMpCisJCQkJ bmV0YndsaW5lKHJvdywgZWxtLCBkZWx0YV90aW1lKTsKKwkJCSsrcm93OworCQl9IGVsc2UgaWYg KGVsbS0+c2VxICE9IHRjcF9wY2Jfc2VxKSB7CisJCQlSQl9SRU1PVkUobXl0Y3BjYl90cmVlLCAm bXl0Y3BfdHJlZSwgZWxtKTsKKwkJCWZyZWUoZWxtKTsKKwkJfQorCX0KKwl3bW92ZSh3bmQsIHJv dywgMCk7CisJd2NscnRvYm90KHduZCk7CisJbXZ3cHJpbnR3KHduZCwgTElORVMtMiwgMCwKKwkJ ICAiUmF0ZS9zZWMsICIKKwkJICAiUj1yeHBlbmQgVD10eHBlbmQgTj1ub2RlbGF5IFQ9dHN0bXAg IgorCQkgICJTPXNhY2sgWD13aW5zY2FsZSBGPWZhc3RyZWMiKTsKK30KKworc3RhdGljIHZvaWQK K25ldGJ3bGluZShpbnQgcm93LCBzdHJ1Y3QgbXl0Y3BjYiAqZWxtLCBkb3VibGUgZGVsdGFfdGlt ZSkKK3sKKwltdndwcmludHcod25kLCByb3csIDAsCisJCSAgIiVzICVzICIKKwkJICAvKiJyeGIg JXMgdHhiICVzICIqLworCQkgICJyY3YgJXMgc25kICVzICIKKwkJICAiWyVjJWMlYyVjJWMlYyVj XSIsCisJCSAgbmV0YWRkcnN0cigKKwkJICAgIGVsbS0+eHRjcC54dF9pbnAuaW5wX3ZmbGFnLAor CQkgICAgKHVuaW9uIGluX2RlcGVuZGFkZHIgKikmZWxtLT54dGNwLnh0X2lucC5pbnBfaW5jLmlu Y19pZS4KKwkJCWllX2RlcGVuZGxhZGRyLAorCQkgICAgbnRvaHMoZWxtLT54dGNwLnh0X2lucC5p bnBfaW5jLmluY19pZS5pZV9scG9ydCkpLAorCQkgIG5ldGFkZHJzdHIoCisJCSAgICBlbG0tPnh0 Y3AueHRfaW5wLmlucF92ZmxhZywKKwkJICAgICh1bmlvbiBpbl9kZXBlbmRhZGRyKikmZWxtLT54 dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS4KKwkJCWllX2RlcGVuZGZhZGRyLAorCQkgICAgbnRv aHMoZWxtLT54dGNwLnh0X2lucC5pbnBfaW5jLmluY19pZS5pZV9mcG9ydCkpLAorCQkvKgorCQkg IG51bXRvayhlbG0tPnh0Y3AueHRfc29ja2V0LnNvX3Jjdi5zYl9jYyksCisJCSAgbnVtdG9rKGVs bS0+eHRjcC54dF9zb2NrZXQuc29fc25kLnNiX2NjKSwKKwkJKi8KKwkJICBudW10b2soREVMVEFF TE0oeHRfdHAucmN2X254dCkpLAorCQkgIG51bXRvayhERUxUQUVMTSh4dF90cC5zbmRfbWF4KSks CisJCSAgKGVsbS0+eHRjcC54dF9zb2NrZXQuc29fcmN2LnNiX2NjID4gMTUwMDAgPworCQkgICAn UicgOiAnICcpLAorCQkgIChlbG0tPnh0Y3AueHRfc29ja2V0LnNvX3NuZC5zYl9jYyA+IDE1MDAw ID8KKwkJICAgJ1QnIDogJyAnKSwKKwkJICAoKGVsbS0+eHRjcC54dF90cC50X2ZsYWdzICYgVEZf Tk9ERUxBWSkgPworCQkgICAnTicgOiAnICcpLAorCQkgICgoZWxtLT54dGNwLnh0X3RwLnRfZmxh Z3MgJiBURl9SQ1ZEX1RTVE1QKSA/CisJCSAgICdUJyA6ICcgJyksCisJCSAgKChlbG0tPnh0Y3Au eHRfdHAudF9mbGFncyAmIFRGX1NBQ0tfUEVSTUlUKSA/CisJCSAgICdTJyA6ICcgJyksCisJCSAg KChlbG0tPnh0Y3AueHRfdHAudF9mbGFncyAmIFRGX1JDVkRfU0NBTEUpID8KKwkJICAgJ1gnIDog JyAnKSwKKwkJICAoKGVsbS0+eHRjcC54dF90cC50X2ZsYWdzICYgVEZfRkFTVFJFQ09WRVJZKSA/ CisJCSAgICdGJyA6ICcgJykKKwkpOworCXdjbHJ0b2VvbCh3bmQpOworfQorCisjaWYgMAoraW50 CitjbWRuZXRidyhjb25zdCBjaGFyICpjbWQgX191bnVzZWQsIGNoYXIgKmFyZ3MgX191bnVzZWQp Cit7CisJZmV0Y2huZXRidygpOworCXNob3duZXRidygpOworCXJlZnJlc2goKTsKKworCXJldHVy biAoMCk7Cit9CisjZW5kaWYKKworI2RlZmluZSBNQVhJTkRFWEVTIDgKKworc3RhdGljIGNvbnN0 IGNoYXIgKgorbnVtdG9rKGRvdWJsZSB2YWx1ZSkKK3sKKwlzdGF0aWMgY2hhciBidWZbTUFYSU5E RVhFU11bMzJdOworCXN0YXRpYyBpbnQgbmV4dGk7CisJc3RhdGljIGNvbnN0IGNoYXIgKnN1ZmZp eGVzW10gPSB7ICIgIiwgIksiLCAiTSIsICJHIiwgIlQiLCBOVUxMIH07CisJaW50IHN1ZmZpeCA9 IDA7CisJY29uc3QgY2hhciAqZm10OworCisJd2hpbGUgKHZhbHVlID49IDEwMDAuMCAmJiBzdWZm aXhlc1tzdWZmaXgrMV0pIHsKKwkJdmFsdWUgLz0gMTAwMC4wOworCQkrK3N1ZmZpeDsKKwl9CisJ bmV4dGkgPSAobmV4dGkgKyAxKSAlIE1BWElOREVYRVM7CisJaWYgKHZhbHVlIDwgMC4wMDEpIHsK KwkJZm10ID0gIiAgICAgICI7CisJfSBlbHNlIGlmICh2YWx1ZSA8IDEuMCkgeworCQlmbXQgPSAi JTUuM2YlcyI7CisJfSBlbHNlIGlmICh2YWx1ZSA8IDEwLjApIHsKKwkJZm10ID0gIiU1LjNmJXMi OworCX0gZWxzZSBpZiAodmFsdWUgPCAxMDAuMCkgeworCQlmbXQgPSAiJTUuMmYlcyI7CisJfSBl bHNlIHsKKwkJZm10ID0gIiU1LjFmJXMiOworCX0KKwlzbnByaW50ZihidWZbbmV4dGldLCBzaXpl b2YoYnVmW25leHRpXSksCisJCSBmbXQsIHZhbHVlLCBzdWZmaXhlc1tzdWZmaXhdKTsKKwlyZXR1 cm4gKGJ1ZltuZXh0aV0pOworfQorCitzdGF0aWMgY29uc3QgY2hhciAqCituZXRhZGRyc3RyKHVf Y2hhciB2ZmxhZ3MsIHVuaW9uIGluX2RlcGVuZGFkZHIgKmRlcGFkZHIsIHVfaW50MTZfdCBwb3J0 KQoreworCXN0YXRpYyBjaGFyIGJ1ZltNQVhJTkRFWEVTXVs2NF07CisJc3RhdGljIGludCBuZXh0 YTsKKwljaGFyIGJ1ZmlwWzY0XTsKKworCW5leHRhID0gKG5leHRhICsgMSkgJSBNQVhJTkRFWEVT OworCisJaWYgKHZmbGFncyAmIElOUF9JUFY0KSB7CisJCXNucHJpbnRmKGJ1ZmlwLCBzaXplb2Yo YnVmaXApLAorCQkJICIlZC4lZC4lZC4lZCIsCisJCQkgKG50b2hsKGRlcGFkZHItPmlkNDZfYWRk ci5pYTQ2X2FkZHI0LnNfYWRkcikgPj4gMjQpICYKKwkJCSAgMjU1LAorCQkJIChudG9obChkZXBh ZGRyLT5pZDQ2X2FkZHIuaWE0Nl9hZGRyNC5zX2FkZHIpID4+IDE2KSAmCisJCQkgIDI1NSwKKwkJ CSAobnRvaGwoZGVwYWRkci0+aWQ0Nl9hZGRyLmlhNDZfYWRkcjQuc19hZGRyKSA+PiA4KSAmCisJ CQkgIDI1NSwKKwkJCSAobnRvaGwoZGVwYWRkci0+aWQ0Nl9hZGRyLmlhNDZfYWRkcjQuc19hZGRy KSA+PiAwKSAmCisJCQkgIDI1NSk7CisJCXNucHJpbnRmKGJ1ZltuZXh0YV0sIHNpemVvZihidWZb bmV4dGFdKSwKKwkJCSAiJTE1czolLTVkIiwgYnVmaXAsIHBvcnQpOworCX0gZWxzZSBpZiAodmZs YWdzICYgSU5QX0lQVjYpIHsKKwkJc25wcmludGYoYnVmaXAsIHNpemVvZihidWZpcCksCisJCQkg IiUwNHg6JTA0eDolMDR4OiUwNHg6JTA0eDolMDR4OiUwNHg6JTA0eCIsCisJCQkgbnRvaHMoZGVw YWRkci0+aWQ2X2FkZHIuczZfYWRkcjE2WzBdKSwKKwkJCSBudG9ocyhkZXBhZGRyLT5pZDZfYWRk ci5zNl9hZGRyMTZbMV0pLAorCQkJIG50b2hzKGRlcGFkZHItPmlkNl9hZGRyLnM2X2FkZHIxNlsy XSksCisJCQkgbnRvaHMoZGVwYWRkci0+aWQ2X2FkZHIuczZfYWRkcjE2WzNdKSwKKwkJCSBudG9o cyhkZXBhZGRyLT5pZDZfYWRkci5zNl9hZGRyMTZbNF0pLAorCQkJIG50b2hzKGRlcGFkZHItPmlk Nl9hZGRyLnM2X2FkZHIxNls1XSksCisJCQkgbnRvaHMoZGVwYWRkci0+aWQ2X2FkZHIuczZfYWRk cjE2WzZdKSwKKwkJCSBudG9ocyhkZXBhZGRyLT5pZDZfYWRkci5zNl9hZGRyMTZbN10pKTsKKwkJ c25wcmludGYoYnVmW25leHRhXSwgc2l6ZW9mKGJ1ZltuZXh0YV0pLAorCQkJICIlMzlzOiUtNWQi LCBidWZpcCwgcG9ydCk7CisJfSBlbHNlIHsKKwkJc25wcmludGYoYnVmaXAsIHNpemVvZihidWZp cCksICI8dW5rbm93bj4iKTsKKwkJc25wcmludGYoYnVmW25leHRhXSwgc2l6ZW9mKGJ1ZltuZXh0 YV0pLAorCQkJICIlMTVzOiUtNWQiLCBidWZpcCwgcG9ydCk7CisJfQorCXJldHVybiAoYnVmW25l eHRhXSk7Cit9CisKK3N0YXRpYyB2b2lkCit1cGRhdGVwY2Ioc3RydWN0IHh0Y3BjYiAqeHRjcCkK K3sKKwlzdHJ1Y3QgbXl0Y3BjYiBkdW1teTsKKwlzdHJ1Y3QgbXl0Y3BjYiAqZWxtOworCisJZHVt bXkueHRjcCA9ICp4dGNwOworCWlmICgoZWxtID0gUkJfRklORChteXRjcGNiX3RyZWUsICZteXRj cF90cmVlLCAmZHVtbXkpKSA9PSBOVUxMKSB7CisJCWVsbSA9IG1hbGxvYyhzaXplb2YoKmVsbSkp OworCQliemVybyhlbG0sIHNpemVvZigqZWxtKSk7CisJCWVsbS0+eHRjcCA9ICp4dGNwOworCQll bG0tPmxhc3RfeHRjcCA9ICp4dGNwOworCQlSQl9JTlNFUlQobXl0Y3BjYl90cmVlLCAmbXl0Y3Bf dHJlZSwgZWxtKTsKKwl9IGVsc2UgeworCQllbG0tPmxhc3RfeHRjcCA9IGVsbS0+eHRjcDsKKwkJ ZWxtLT54dGNwID0gKnh0Y3A7CisJfQorCWVsbS0+c2VxID0gdGNwX3BjYl9zZXE7Cit9CmRpZmYg LS1naXQgYS91c3IuYmluL3N5c3RhdC9zeXN0YXQuMSBiL3Vzci5iaW4vc3lzdGF0L3N5c3RhdC4x CmluZGV4IDVmYzMyNTcuLjBhYjE3MzggMTAwNjQ0Ci0tLSBhL3Vzci5iaW4vc3lzdGF0L3N5c3Rh dC4xCisrKyBiL3Vzci5iaW4vc3lzdGF0L3N5c3RhdC4xCkBAIC05NCw2ICs5NCw3IEBAIHRvIGJl IG9uZSBvZjoKIC5JYyBpb3N0YXQgLAogLkljIGlwICwKIC5JYyBpcDYgLAorLkljIG5ldGJ3ICwK IC5JYyBuZXRzdGF0ICwKIC5JYyBwaWdzICwKIC5JYyBzd2FwICwKQEAgLTQ0MSw2ICs0NDIsOSBA QCBEaXNwbGF5IHN0YXRpc3RpY3MgYXZlcmFnZWQgb3ZlciB0aGUgcmVmcmVzaCBpbnRlcnZhbCAo dGhlIGRlZmF1bHQpLgogLkl0IENtIHplcm8KIFJlc2V0IHJ1bm5pbmcgc3RhdGlzdGljcyB0byB6 ZXJvLgogLkVsCisuSXQgSWMgbmV0YncKK0Rpc3BsYXkgYWdncmVnYXRlIGFuZCBwZXItY29ubmVj dGlvbiBUQ1AgcmVjZWl2ZSBhbmQgdHJhbnNtaXQgcmF0ZXMuCitPbmx5IGFjdGl2ZSBUQ1AgY29u bmVjdGlvbnMgYXJlIHNob3duLgogLkl0IEljIG5ldHN0YXQKIERpc3BsYXksIGluIHRoZSBsb3dl ciB3aW5kb3csIG5ldHdvcmsgY29ubmVjdGlvbnMuCiBCeSBkZWZhdWx0LAotLSAKMS44LjUuNAoK --001a11c21fc0e8049204fd3e8e11-- From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 23:50:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF8EC9EA for ; Wed, 2 Jul 2014 23:50:59 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A517320AB for ; Wed, 2 Jul 2014 23:50:59 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id D822F1602CD; Wed, 2 Jul 2014 16:50:58 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id 291551602B1 for ; Wed, 2 Jul 2014 16:50:56 -0700 (MST) Message-ID: <53B49AE0.4030902@pinyon.org> Date: Wed, 02 Jul 2014 16:50:56 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: NFS client READ performance on -current X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 23:51:00 -0000 Greetings! It's been 14 years. OMG do I love poudriere and zfs. But apropos of this post from last January: http://lists.freebsd.org/pipermail/freebsd-net/2014-January/037547.html I am going to capitalize READ to emphasize that all I am looking at here is reading on the client. In some of posts I have read people have been confusing read vs. write performance and losing the thread. I have fresh current server and client, each with otherwise quiescent em NICs. I also have debian kernel 3.14-1 server and client. The freebsd hardware is large enough, 16GB RAM + vishera 8 cores. (The debian systems are much smaller/slower) Upshot is that with a -current NFSV4 (4.1) serving a -current client and a linux client, I see today client READ performance of: -current -> -current: 2.7MB/s READ nice and steady -current -> linux: 80MB/s READ very bursty And for the weird (to me) result: linux -> -current: 55.74MB/s READ So to be very clear, the freebsd -current server seems capable of of supporting excellent client READ bandwidth. But something is dramatically awry with the freebsd client. (scp, rsync over ssh, all are near wire speed when I get rid of disk overhead). The -current server has got an 11TB raidz2 behind it but I'm only READing, not writing, so all of the discussion about ZFS + NFS sync write problems seems immaterial, possibly. Here's the mount info: -current client's output from nfsstat -m (terp is current server, ari is linux server): terp:/raid/library on /mnt/terp/library nfsv4,minorversion=1,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 ari:/d1/library on /mnt/ari nfsv4,minorversion=0,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 Linux client's excerpt from mount output: terp:/raid/library on /mnt/terp/library type nfs4 (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.0.10.3,local_lock=none,addr=10.0.10.4) BTW, linux -> linux is about 80MB/s I tried nfsv3 -current -> -current and still got 2.7MB/s READ. I have spent about 5 hours strolling through the archives of the last several years of discussion about this issue in various forums, but I didn't see a reference once since Jaunary's thread on freebsd-net, and I didn't find a bug entry, so I thought I'd keep it alive, so to speak. I've tried all of the various sysctl tweaks that have been suggested over time, that would make sense for reads, to no effect. Russell From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 00:04:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06254252 for ; Thu, 3 Jul 2014 00:04:08 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id BF4AB2287 for ; Thu, 3 Jul 2014 00:04:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqoEAHQItFODaFve/2dsb2JhbABag19agm6oUgEBAQaTFIZtUwGBInWEAwEBAQMBAQEBIAQnIAsFFhgCAg0ZAikBCSYGCAcEARwEiBkIDapZmzAXgSyERIhhAQEbNAeCd4FMBZgChDOSQYNfIS8GgQU5 X-IronPort-AV: E=Sophos;i="5.01,591,1400040000"; d="scan'208";a="137915737" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 02 Jul 2014 20:04:06 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 177C6B3F47; Wed, 2 Jul 2014 20:04:06 -0400 (EDT) Date: Wed, 2 Jul 2014 20:04:06 -0400 (EDT) From: Rick Macklem To: "Russell L. Carter" Message-ID: <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> In-Reply-To: <53B49AE0.4030902@pinyon.org> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 00:04:08 -0000 Russell L. Carter wrote: > Greetings! > It's been 14 years. OMG do I love poudriere and zfs. > > But apropos of this post from last January: > > http://lists.freebsd.org/pipermail/freebsd-net/2014-January/037547.html > > I am going to capitalize READ to emphasize that all I am looking at > here is reading on the client. In some of posts I have read people > have been confusing read vs. write performance and losing the thread. > > I have fresh current server and client, each with otherwise quiescent > em NICs. I also have debian kernel 3.14-1 server and client. The > freebsd hardware is large enough, 16GB RAM + vishera 8 cores. (The > debian systems are much smaller/slower) > > Upshot is that with a -current NFSV4 (4.1) serving a -current client > and a linux client, I see today client READ performance of: > > -current -> -current: 2.7MB/s READ nice and steady > -current -> linux: 80MB/s READ very bursty > > And for the weird (to me) result: > > linux -> -current: 55.74MB/s READ > > So to be very clear, the freebsd -current server seems capable of of > supporting excellent client READ bandwidth. But something is > dramatically awry with the freebsd client. (scp, rsync over ssh, all > are near wire speed when I get rid of disk overhead). The -current > server has got an 11TB raidz2 behind it but I'm only READing, not > writing, so all of the discussion about ZFS + NFS sync write problems > seems immaterial, possibly. > > Here's the mount info: > > -current client's output from nfsstat -m (terp is current server, ari > is linux server): > > terp:/raid/library on /mnt/terp/library > nfsv4,minorversion=1,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 > > ari:/d1/library on /mnt/ari > nfsv4,minorversion=0,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 > > Linux client's excerpt from mount output: > > terp:/raid/library on /mnt/terp/library type nfs4 > (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.0.10.3,local_lock=none,addr=10.0.10.4) > > BTW, linux -> linux is about 80MB/s > > I tried nfsv3 -current -> -current and still got 2.7MB/s READ. > > I have spent about 5 hours strolling through the archives of the last > several years of discussion about this issue in various forums, but I > didn't see a reference once since Jaunary's thread on freebsd-net, > and > I didn't find a bug entry, so I thought I'd keep it alive, so to > speak. > > I've tried all of the various sysctl tweaks that have been suggested > over time, that would make sense for reads, to no effect. > Please try disabling TSO on the FreeBSD systems and also try an rsize=32768 to see if either of those have any effect. There was also a recent case where disabling msix interrupt handling for the network interface fixed a slow NFS I/O rate issue. (I don't know what the loader conf variable was, but look for something with msix in it for the net device driver. It might be hw.em.enable_msix=0 or something like that?) rick > Russell > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 00:17:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8C2AA6A for ; Thu, 3 Jul 2014 00:17:11 +0000 (UTC) Received: from mail-oa0-f50.google.com (mail-oa0-f50.google.com [209.85.219.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82EA223B3 for ; Thu, 3 Jul 2014 00:17:10 +0000 (UTC) Received: by mail-oa0-f50.google.com with SMTP id n16so13104465oag.37 for ; Wed, 02 Jul 2014 17:17:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=exCRB1/pluZyf+Ky5JLihU7cvYnkoGDa+fFYRNLcIy4=; b=Fys67d+AAAgHDpsIuoJgoBMhtuhyOEgTEBT4e5OoqR0+jde2qKz5RvoJhanniVV8Pe 18/Ouz6G+yWx6XXbHiuuHIWa+IgNh2o1EqS9iHzuy14Rmh/5EWLTxWScM/cDMG7Q7Dhv bxScheah1WGt+ONeCcIc/jvakFMhs0ra9HjcQp8pEZYazrtsWuXLGOp4xPDb0+nfLT6u 5E/xTS3ZJlySw4Mz8VfwzulzU7KC7snM3PKUGKMeKciVzgA2IGH7hRSNrpcr+wEp+e58 nJBHRZu0VIkMunJedqmc/LSYC474yrUDVbLU1Hk0Qc3cuIitXh69RvcfDQbgMvFCtWx/ hcGw== X-Gm-Message-State: ALoCoQlT4aROZxz5cGiHrutHQ0Sv2iZD2gFrq/mwzKjvPwcNmsI8wKa23FF14t+cytvaszAxol3/ MIME-Version: 1.0 X-Received: by 10.60.70.106 with SMTP id l10mr1184477oeu.54.1404346629709; Wed, 02 Jul 2014 17:17:09 -0700 (PDT) Received: by 10.60.11.134 with HTTP; Wed, 2 Jul 2014 17:17:09 -0700 (PDT) In-Reply-To: <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> References: <53B49AE0.4030902@pinyon.org> <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> Date: Wed, 2 Jul 2014 17:17:09 -0700 Message-ID: Subject: Re: NFS client READ performance on -current From: Michael Sierchio To: Rick Macklem Content-Type: text/plain; charset=UTF-8 Cc: "Russell L. Carter" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 00:17:11 -0000 On Wed, Jul 2, 2014 at 5:04 PM, Rick Macklem wrote: > Please try disabling TSO on the FreeBSD systems and also > try an rsize=32768 to see if either of those have any effect. ifconfig -tso sysctl net.inet.tcp.tso=0 Maybe do the same for lro? ifconfig -tso -lro sysctl net.inet.tcp.tso=0 dev..0.enable_lro=0 - M From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 00:38:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8A6A47C for ; Thu, 3 Jul 2014 00:38:32 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F4F42598 for ; Thu, 3 Jul 2014 00:38:32 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id 61DA11602CD; Wed, 2 Jul 2014 17:38:31 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id 2C9BC1601EA for ; Wed, 2 Jul 2014 17:38:29 -0700 (MST) Message-ID: <53B4A605.8000604@pinyon.org> Date: Wed, 02 Jul 2014 17:38:29 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current References: <53B49AE0.4030902@pinyon.org> <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 00:38:32 -0000 On 07/02/14 17:17, Michael Sierchio wrote: > sysctl net.inet.tcp.tso=0 Woot! 2.7MB/s -> 76MB/s Is this a FAQ somewhere? I spent a long time looking today and didn't find it (I do recall seeing a mention of TSO but not seeing how to do it or that it fixed the problem) On the second suggestion: root@feyerabend> sysctl -a | grep dev.em | grep enable_ root@feyerabend> No problem, though. 76MB/s is A-OK. Thanks! I'm very happy now. Russell From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 00:54:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D42228A2 for ; Thu, 3 Jul 2014 00:54:42 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96C4626FF for ; Thu, 3 Jul 2014 00:54:42 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id m20so10834711qcx.13 for ; Wed, 02 Jul 2014 17:54:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=27Qt267w8MXpNz2OjL5NeQOKYzBCT3SWIB0IG7fQhgA=; b=0zbsVYXlb/aMwK97xAkv0dlsadlTraGDRfzY9St7B8UejpYdYJWdzpAy13nM27wacg kWn6Zp+BX9+rxMVyjtF0I/4NwvyP51BeC0vi5qeWf+KWO7a8/FBjhdeNF8iyUdQIBT+h Qkq5GJXZ15LfUG0bMQqyrl02RZs/FD76IUcbfx7XbM8pro+noO1ZPK8SP/zLU/L7eIs7 iDOS0vLqt4Z/h709AZyGqzCI9aujsbNZ11HBVeAkl/KOdQACQ9MHBMxM+uiRVNumbYWn sTqAQ4rUbKhW1eM8HpTaiu6JH3it/ECNIyMQkm7jjjFxTDDdms5R5Q4p1Bzx4lLUJ0cC 11gg== MIME-Version: 1.0 X-Received: by 10.224.166.73 with SMTP id l9mr2327255qay.34.1404348881565; Wed, 02 Jul 2014 17:54:41 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Wed, 2 Jul 2014 17:54:41 -0700 (PDT) In-Reply-To: References: Date: Wed, 2 Jul 2014 17:54:41 -0700 Message-ID: Subject: Re: Add netbw option to systat From: hiren panchasara To: Bryan Venteicher Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 00:54:42 -0000 On Wed, Jul 2, 2014 at 4:50 PM, Bryan Venteicher wrote: > Awhile back, DragonlyFlyBSD added a netbw option to systat that I've ported > to FreeBSD and found handy at various times: > > netbw Display aggregate and per-connection TCP receive and transmit > rates. Only active TCP connections are shown. > > Leading to output such as: > > tcp accepts connects rcv 1.192G snd 15.77K rexmit > > 192.168.10.80:22 192.168.10.20:23103 rcv snd 415.7 [ NTSX ] > 192.168.10.80:22 192.168.10.20:46560 rcv 19.80M snd 14.47K [ NTSX ] > 192.168.10.80:22 192.168.10.20:60699 rcv snd 886.3 [ NTSX ] > 192.168.10.81:5201 192.168.10.51:60844 rcv 293.2M snd [R TSX ] > 192.168.10.81:5201 192.168.10.51:60845 rcv 293.5M snd [R TSX ] > 192.168.10.81:5201 192.168.10.51:60846 rcv 293.2M snd [R TSX ] > 192.168.10.81:5201 192.168.10.51:60847 rcv 292.9M snd [R TSX ] > > It uses the sequences number from the 'struct tcpcb' to derive the rates, > which is usually good but certainly not perfect (i.e., don't set the > interval too long). > > I'd like to commit this if anybody else thinks they'd find it useful. > > http://people.freebsd.org/~bryanv/patches/systat-netbw.patch I like the idea. A few things about the patch: 1) You may want to remove the code hidden behind "#if 0" at 2 places. 2) I am not entirely clear on why/if we need the last column with flags but if we keep it (for compatibility of any other reason), It would be nice to have those flags explained in the manpage: + mvwprintw(wnd, LINES-2, 0, + "Rate/sec, " + "R=rxpend T=txpend N=nodelay T=tstmp " + "S=sack X=winscale F=fastrec"); 3) I feel that the header line for o/p (specially 'tcp accepts and connects' terminology) can be improved but I do not have a better suggestion :-) It looks okay me otherwise and thanks for your work. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 02:09:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C5795FB for ; Thu, 3 Jul 2014 02:09:15 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 43FC42CB5 for ; Thu, 3 Jul 2014 02:09:14 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgQFACq5tFODaFve/2dsb2JhbABag19agm+oUAECAQEBBpMUhm5TAYEgdYQDAQEBAwEBAQEgBCcgCwUWGAICDRkCKQEJJgYIBwQBGQMEiBkIDatUm3MXgSyERIhaAQYBARs0B4J3gUwFmAKEM5JCg18hNXwBCBci X-IronPort-AV: E=Sophos;i="5.01,592,1400040000"; d="scan'208";a="137299358" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 02 Jul 2014 22:09:13 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A1A3AB4069; Wed, 2 Jul 2014 22:09:13 -0400 (EDT) Date: Wed, 2 Jul 2014 22:09:13 -0400 (EDT) From: Rick Macklem To: "Russell L. Carter" Message-ID: <509953671.6648047.1404353353654.JavaMail.root@uoguelph.ca> In-Reply-To: <53B4A605.8000604@pinyon.org> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 02:09:15 -0000 Russell L. Carter wrote: > > > On 07/02/14 17:17, Michael Sierchio wrote: > > sysctl net.inet.tcp.tso=0 > > Woot! 2.7MB/s -> 76MB/s > > Is this a FAQ somewhere? I spent a long time looking > today and didn't find it (I do recall seeing a mention > of TSO but not seeing how to do it or that it fixed > the problem) > Unfortunately, this isn't good news for me. I had thought the worst of the TSO related problems had been resolved in head. The one I knew about was a case where a read/write just under 64K would result in a TSO segment (including ethernet header) just over 64K (which can't fit in 32 mbuf clusters, while 32 is the limit for transmit segments for some TSO enabled interfaces). I think this is fixed in head by r264630, which reduces the maximum tso segment length by a small amount. Admittedly, if the network interface is limited to less than 35 transmit segments and does not use m_defrag() to compact the TSO segment into 32mbuf clusters, it would still be broken. The m_defrag() calls will result in overhead, but I don't think they would cause that much effect. Could you please post the dmesg stuff for the network interface, so I can tell what driver is being used? I'll take a look at it, in case it needs to be changed to use m_defrag(). Thanks for letting us know this fixed the problem, rick > On the second suggestion: > > root@feyerabend> sysctl -a | grep dev.em | grep enable_ > root@feyerabend> > > No problem, though. 76MB/s is A-OK. > > Thanks! I'm very happy now. > > Russell > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 03:21:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EFD439E for ; Thu, 3 Jul 2014 03:21:43 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 17A622392 for ; Thu, 3 Jul 2014 03:21:42 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id A0E201602D1; Wed, 2 Jul 2014 20:21:41 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id 56E591602B1 for ; Wed, 2 Jul 2014 20:21:39 -0700 (MST) Message-ID: <53B4CC43.1050205@pinyon.org> Date: Wed, 02 Jul 2014 20:21:39 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current References: <509953671.6648047.1404353353654.JavaMail.root@uoguelph.ca> In-Reply-To: <509953671.6648047.1404353353654.JavaMail.root@uoguelph.ca> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 03:21:43 -0000 On 07/02/14 19:09, Rick Macklem wrote: > Could you please post the dmesg stuff for the network interface, > so I can tell what driver is being used? I'll take a look at it, > in case it needs to be changed to use m_defrag(). em0: port 0xd020-0xd03f mem 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 on pci2 em0: Using an MSI interrupt em0: Ethernet address: 00:15:17:bc:29:ba 001.000007 [2323] netmap_attach success for em0 tx 1/1024 rx 1/1024 queues/slots This is one of those dual nic cards, so there is em1 as well... Best, Russell > > Thanks for letting us know this fixed the problem, rick > From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 07:17:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5420E923 for ; Thu, 3 Jul 2014 07:17:42 +0000 (UTC) Received: from mail1.sysgo.com (mail1.sysgo.com [176.9.26.183]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13C432485 for ; Thu, 3 Jul 2014 07:17:41 +0000 (UTC) Date: Thu, 3 Jul 2014 09:07:33 +0200 From: Thomas Mueller To: "Russell L. Carter" Subject: Re: NFS client READ performance on -current Message-ID: <20140703090733.5a648056@tmu.ulm.sysgo.com> In-Reply-To: <53B4A605.8000604@pinyon.org> References: <53B49AE0.4030902@pinyon.org> <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> <53B4A605.8000604@pinyon.org> Organization: SYSGO AG X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 07:17:42 -0000 On Wed, 02 Jul 2014 17:38:29 -0700, Russell L. Carter wrote: > On the second suggestion: > > root@feyerabend> sysctl -a | grep dev.em | grep enable_ > root@feyerabend> tmu:~$ sysctl -a | grep msix hw.em.enable_msix: 1 hw.igb.enable_msix: 1 hw.ix.enable_msix: 1 [...] -- Thomas Mueller From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 14:08:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1D7FD19 for ; Thu, 3 Jul 2014 14:08:07 +0000 (UTC) Received: from nm28-vm3.bullet.mail.ne1.yahoo.com (nm28-vm3.bullet.mail.ne1.yahoo.com [98.138.91.158]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ADB022C8B for ; Thu, 3 Jul 2014 14:08:07 +0000 (UTC) Received: from [98.138.226.177] by nm28.bullet.mail.ne1.yahoo.com with NNFMP; 03 Jul 2014 14:06:20 -0000 Received: from [98.138.87.10] by tm12.bullet.mail.ne1.yahoo.com with NNFMP; 03 Jul 2014 14:06:21 -0000 Received: from [127.0.0.1] by omp1010.mail.ne1.yahoo.com with NNFMP; 03 Jul 2014 14:06:21 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 554169.9208.bm@omp1010.mail.ne1.yahoo.com Received: (qmail 19611 invoked by uid 60001); 3 Jul 2014 14:06:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1404396381; bh=MfbxOEEGVQ6O74U9ZhZN4Us9oEz/9x/FHdqus0UzWCI=; h=Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=cEIVZnVL2IA4O//gYnFrHyTEgz6gB+/oR04gK8JHCSyKIIo1+M6iF2ojRMPmFsR4sX7xAmPslzRXyarLuAoVUfZS8cWv8oxifcU5DPg/K7+Ys3rDbpl6BpQyU+FBKqYSlZGHNG1aLHBF+/HABcQUt2aMMst4W0Q9N3o0zoErq48= X-YMail-OSG: z9_8xNMVM1nIJvKJUo1zJHf22be2V4Wfclosf2JUmqTceoV _K_fjVZDw8NYgJtBScPbUS5LIBNTfP1wdBmlY9MVFSAem3VosvapiCGDLo41 808qG3x117mOgS89poopHcedYkJfwFDofGUXZZ78wJ9im3UbnHY3ODjFeIk7 Kegsu8CPNITNp3Hzdm6nTmH60E0oPbA4C8PYGpxliVADeX3nHaHS1cGx.v7l QRh_61buuEKjfQMnsJVT20rH2.CeJC7j39bLYT.hlEjFClK1KtFG0G_uXCv1 dyvShr3fTdWpXLnXHjtCVrMjDpDYPyqjdonU0gelWs5uufW.eml8V84jxLCg NJF9pjWWaMJx48LjYGHqPv.hQ.fvrRQe2G6T09onGbFWXXI_EpKq5BPEgI0M TdFtOLEjaT2Bj5yt5Ok2z58qiQqeHW_tVreYbATHWzmdPBckD4WeBQGQpEhc GFJA0aIzEAWiq1PZU6I60bcOLYg0ghNqZVK8JIiUA.7KjW9j4wHsYXlqhNN4 FHNdaMPPz Received: from [76.108.181.232] by web125802.mail.ne1.yahoo.com via HTTP; Thu, 03 Jul 2014 07:06:21 PDT X-Rocket-MIMEInfo: 002.001, SSdtIGhhdmluZyBhIHByb2JsZW0gd2l0aCBhIFN1cGVybWljcm8gc3lzdGVtIHJ1bm5pbmcgRnJlZUJTRCA5LjEuIFNvbWV0aW1lcyB3aGVuIEkgdXBncmFkZSB0aGUga2VybmVsIGluIG15IG1haW4gZHJpdmUgKGFkYTApLCANCnRoZSBzeXN0ZW0gYm9vdHMgdGhlIGtlcm5lbCBmcm9tIHRoZSAybmQgZHJpdmUuIEl0IG9ubHkgaGFwcGVucyBzb21ldGltZXMuIGFkYTAgaXMgbW91bnRlZC4gYnV0IHRoZSBzeXN0ZW0gaXMgcnVubmluZyB0aGUgb2xkIGtlcm5lbC4gDQpQdWxsaW5nIHRoZSAybmQgZml4ZWQgdGgBMAEBAQE- X-Mailer: YahooMailClassic/654 YahooMailWebService/0.8.191.1 Message-ID: <1404396381.74461.YahooMailBasic@web125802.mail.ne1.yahoo.com> Date: Thu, 3 Jul 2014 07:06:21 -0700 From: Laurie Jennings Subject: System Booting Kernel from Secondary Drive To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 14:08:08 -0000 I'm having a problem with a Supermicro system running FreeBSD 9.1. Sometimes when I upgrade the kernel in my main drive (ada0), the system boots the kernel from the 2nd drive. It only happens sometimes. ada0 is mounted. but the system is running the old kernel. Pulling the 2nd fixed the problem. What can cause this to happen? Is it a supermicro problem (it's a 5017R-MTF superserver) or is it something with FreeBSD. thanks in advance, Laurie From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 20:13:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B612B1B5 for ; Thu, 3 Jul 2014 20:13:42 +0000 (UTC) Received: from orion.unicamp.br (orion.unicamp.br [143.106.10.169]) by mx1.freebsd.org (Postfix) with ESMTP id 2A56C2FB5 for ; Thu, 3 Jul 2014 20:13:41 +0000 (UTC) Received: from [143.106.64.207] (bioq15.ib.unicamp.br [143.106.64.207]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by orion.unicamp.br (Postfix) with ESMTPSA id 2100F53483B; Thu, 3 Jul 2014 17:06:29 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unicamp.br; s=default; t=1404417991; bh=TwZlQ7Bmw1E1wi87Z+CZxaKgGKkNRcyDuMGyEkVpL9I=; h=From; b=bNT7j2z3CzPzhvrjM1YE+7W4cBJ6xLeOqnfapzPKOzyBMsFS4HeJ1xxG6TYEoj3VS LPYBAQXwUI7vjyJCftMAoem12KQiafycZ+YqJJEBXXCkE92EXfjIKOXq7emxEM1BkE wIfucGH8C+8k1CrVl5lm8OcjM/fSq4KR7LgLreyA= Message-ID: <53B5B7C5.5020004@unicamp.br> Date: Thu, 03 Jul 2014 17:06:29 -0300 From: =?ISO-8859-1?Q?=22F=E1bio_R=2E_Medeiros=22?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: [iSCSI] FreeBSD native iSCSI support don't mapping LUNs to /dev/ Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 20:13:42 -0000 Hello there! I've got some difficult on mapping a iSCSI LUN to a /dev/xx device using the native iSCSI support on FreeBSD 10 (the iscsid and iscsictl). I using a Enhance UltraStor ES3160P4 storage device. I've a storage configured with one RAID0, one volume and some LUNs instances for test purposes. I can connect to storage, see the devices, but they are not mapped to LUNs into the system. Here are the issue: root@localhost:~ # iscsictl -A -p 192.168.100.100:3260 -t storage0:dev15.ctr1 -u test -s testteste12345 iscsictl: invalid iSCSI name "storage0:dev15.ctr1"; should start with either ".iqn", "eui.", or "naa." root@localhost:~ # iscsictl -L Target name Target portal State storage0:dev15.ctr1 192.168.100.100 Connected: root@localhost:~ # I now the name of storage device isn't configured as required, but I can't change it now (there are other services running on this storage). Could this be the root of the problem? --- The debug of iscsid shows all good. The last messages are these: iscsid: 192.168.100.100 (storage0:dev15.ctr1): parameter negotiation done; transitioning to Full Feature phase iscsid: 192.168.100.100 (storage0:dev15.ctr1): handing off connection to the kernel iscsid: 192.168.100.100 (storage0:dev15.ctr1): nothing more to do; exiting --- The FreeBSD handbook shows the LUN mapped to /dev/da0 device it's mapped: Target name Target portal Stateiqn.2012-06.com.example:target0 10.10.10.10 Connected: da0 Am I forgetting anything? -- Fábio Rocha Medeiros Administrador de Redes Instituto de Biologia - UNICAMP http://www.ib.unicamp.br From owner-freebsd-net@FreeBSD.ORG Thu Jul 3 21:26:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 691C77CE for ; Thu, 3 Jul 2014 21:26:39 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E92C2604 for ; Thu, 3 Jul 2014 21:26:38 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id 816C41602CD; Thu, 3 Jul 2014 14:26:37 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id 59DFF1600DF for ; Thu, 3 Jul 2014 14:26:35 -0700 (MST) Message-ID: <53B5CA8B.1050707@pinyon.org> Date: Thu, 03 Jul 2014 14:26:35 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current References: <53B49AE0.4030902@pinyon.org> <371130768.6608219.1404345846086.JavaMail.root@uoguelph.ca> <53B4A605.8000604@pinyon.org> <20140703090733.5a648056@tmu.ulm.sysgo.com> In-Reply-To: <20140703090733.5a648056@tmu.ulm.sysgo.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 21:26:39 -0000 On 07/03/14 00:07, Thomas Mueller wrote: > On Wed, 02 Jul 2014 17:38:29 -0700, Russell L. Carter wrote: >> On the second suggestion: >> >> root@feyerabend> sysctl -a | grep dev.em | grep enable_ >> root@feyerabend> > > tmu:~$ sysctl -a | grep msix > hw.em.enable_msix: 1 > hw.igb.enable_msix: 1 > hw.ix.enable_msix: 1 > [...] > > I set hw.em.enable_msix=0 on first the server and then the server + client and NFSv4.1 read throughput is unchanged. For a 5GB transfer it settles down to a range of 58-68MB/s eventually. I then set it back to enabled for both server and client and saw the same rates. Russell From owner-freebsd-net@FreeBSD.ORG Fri Jul 4 00:51:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59A4D415 for ; Fri, 4 Jul 2014 00:51:09 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1EED227D9 for ; Fri, 4 Jul 2014 00:51:08 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAJrEtVODaFve/2dsb2JhbABag19agm+7boZsUwGBJHWEAwEBAQMBAQEBIAQnIAsFFhgRGQIEJQEJJgYIBwQBHASIGQgNrgSbRReOSwYBARsZGweCd4FMBZIhhWmENJJDg18hNX0IFyI X-IronPort-AV: E=Sophos;i="5.01,598,1400040000"; d="scan'208";a="138157957" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 03 Jul 2014 20:51:01 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7CC19B409F; Thu, 3 Jul 2014 20:51:01 -0400 (EDT) Date: Thu, 3 Jul 2014 20:51:01 -0400 (EDT) From: Rick Macklem To: "Russell L. Carter" Message-ID: <870285181.7082888.1404435061501.JavaMail.root@uoguelph.ca> In-Reply-To: <53B4CC43.1050205@pinyon.org> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_7082886_1406096233.1404435061499" X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 00:51:09 -0000 ------=_Part_7082886_1406096233.1404435061499 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Russell L. Carter wrote: > > > On 07/02/14 19:09, Rick Macklem wrote: > > > Could you please post the dmesg stuff for the network interface, > > so I can tell what driver is being used? I'll take a look at it, > > in case it needs to be changed to use m_defrag(). > > em0: port 0xd020-0xd03f > mem > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 on > pci2 > em0: Using an MSI interrupt > em0: Ethernet address: 00:15:17:bc:29:ba > 001.000007 [2323] netmap_attach success for em0 tx 1/1024 > rx > 1/1024 queues/slots > > This is one of those dual nic cards, so there is em1 as well... > Well, I took a quick look at the driver and it does use m_defrag(), but I think that the "retry:" label it does a goto after doing so might be in the wrong place. The attached untested patch might fix this. Is it convenient to build a kernel with this patch applied and then try it with TSO enabled? rick ps: It does have the transmit segment limit set to 32. I have no idea if this is a hardware limitation. > Best, > Russell > > > > > Thanks for letting us know this fixed the problem, rick > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > ------=_Part_7082886_1406096233.1404435061499 Content-Type: text/x-patch; name=em.patch Content-Disposition: attachment; filename=em.patch Content-Transfer-Encoding: base64 LS0tIHN5cy9kZXYvZTEwMDAvaWZfZW0uYy5zYXYJMjAxNC0wNy0wMyAyMDoyNzo1MC4wMDAwMDAw MDAgLTA0MDAKKysrIHN5cy9kZXYvZTEwMDAvaWZfZW0uYwkyMDE0LTA3LTAzIDIwOjI5OjUzLjAw MDAwMDAwMCAtMDQwMApAQCAtMTgxOCw3ICsxODE4LDYgQEAgZW1feG1pdChzdHJ1Y3QgdHhfcmlu ZyAqdHhyLCBzdHJ1Y3QgbWJ1ZgogCWludAkJCW5zZWdzLCBpLCBqLCBmaXJzdCwgbGFzdCA9IDA7 CiAJaW50CQkJZXJyb3IsIGRvX3RzbywgdHNvX2Rlc2MgPSAwLCByZW1hcCA9IDE7CiAKLXJldHJ5 OgogCW1faGVhZCA9ICptX2hlYWRwOwogCXR4ZF91cHBlciA9IHR4ZF9sb3dlciA9IHR4ZF91c2Vk ID0gdHhkX3NhdmVkID0gMDsKIAlkb190c28gPSAoKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFn cyAmIENTVU1fVFNPKSAhPSAwKTsKQEAgLTE5NDQsNiArMTk0Myw3IEBAIHJldHJ5OgogCXR4X2J1 ZmZlcl9tYXBwZWQgPSB0eF9idWZmZXI7CiAJbWFwID0gdHhfYnVmZmVyLT5tYXA7CiAKK3JldHJ5 OgogCWVycm9yID0gYnVzX2RtYW1hcF9sb2FkX21idWZfc2codHhyLT50eHRhZywgbWFwLAogCSAg ICAqbV9oZWFkcCwgc2VncywgJm5zZWdzLCBCVVNfRE1BX05PV0FJVCk7CiAK ------=_Part_7082886_1406096233.1404435061499-- From owner-freebsd-net@FreeBSD.ORG Fri Jul 4 07:38:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 636AADAD for ; Fri, 4 Jul 2014 07:38:37 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E19C2985 for ; Fri, 4 Jul 2014 07:38:36 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 5E597139CC for ; Fri, 4 Jul 2014 04:38:52 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1404459529; x=1405323530; bh=ygtNnRMmi0lFs3RkwY1GbbfiZ+VoNazHtUu DxBGXnlY=; b=b0uH4yysv19OcDo8NmV2lQPe56whFoK+LXcfXMWnnV90+i7SYmj 1FQpYQNLbTPRSRy6J37q8POtNCYq8te6W8amTFL+RUWdJPkeBlA2LOSRfBCk86KV tbpiUvsuycGNucLXQpK2bAHAXG4PCXWYKxXQTZaLXgupEd00IEaERcxA= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L7QhBETUVquh for ; Fri, 4 Jul 2014 04:38:49 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 9D373139CB; Fri, 4 Jul 2014 04:38:48 -0300 (BRT) Message-ID: <53B659EA.20200@bsdinfo.com.br> Date: Fri, 04 Jul 2014 04:38:18 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: sthaug@nethelp.no, jeffrey.e.pieper@intel.com Subject: Re: Network Intel X520-SR2 stopping References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> <53B45C33.1010501@bsdinfo.com.br> In-Reply-To: <53B45C33.1010501@bsdinfo.com.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 07:38:37 -0000 Hi all, Both modules are 850nm. Could there be some incompatibility between the Datacom XFP optical module and the SFP + Intel X520-SR2 optical module? I ask this because I replaced all the hardware and optical cords and nothing worked. Datacom is a DM4100. dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.15 dev.ix.0.%driver: ix dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x7a11 class=0x020000 dev.ix.0.%parent: pci131 dev.ix.0.fc: 3 dev.ix.0.enable_aim: 1 dev.ix.0.advertise_speed: 0 dev.ix.0.dropped: 0 dev.ix.0.mbuf_defrag_failed: 0 dev.ix.0.watchdog_events: 0 dev.ix.0.link_irq: 61846 dev.ix.0.queue0.interrupt_rate: 83333 dev.ix.0.queue0.irqs: 2240081588 dev.ix.0.queue0.txd_head: 1687 dev.ix.0.queue0.txd_tail: 1687 dev.ix.0.queue0.tso_tx: 5 dev.ix.0.queue0.no_tx_dma_setup: 0 dev.ix.0.queue0.no_desc_avail: 2171 dev.ix.0.queue0.tx_packets: 4721544662 dev.ix.0.queue0.rxd_head: 223 dev.ix.0.queue0.rxd_tail: 222 dev.ix.0.queue0.rx_packets: 3939926170 dev.ix.0.queue0.rx_bytes: 708714319872 dev.ix.0.queue0.rx_copies: 1428958853 dev.ix.0.queue0.lro_queued: 0 dev.ix.0.queue0.lro_flushed: 0 dev.ix.0.queue1.interrupt_rate: 100000 dev.ix.0.queue1.irqs: 2164304165 dev.ix.0.queue1.txd_head: 24 dev.ix.0.queue1.txd_tail: 24 dev.ix.0.queue1.tso_tx: 0 dev.ix.0.queue1.no_tx_dma_setup: 0 dev.ix.0.queue1.no_desc_avail: 241832 dev.ix.0.queue1.tx_packets: 4995476933 dev.ix.0.queue1.rxd_head: 1161 dev.ix.0.queue1.rxd_tail: 1160 dev.ix.0.queue1.rx_packets: 3901327164 dev.ix.0.queue1.rx_bytes: 691440627148 dev.ix.0.queue1.rx_copies: 1408409383 dev.ix.0.queue1.lro_queued: 0 dev.ix.0.queue1.lro_flushed: 0 dev.ix.0.queue2.interrupt_rate: 83333 dev.ix.0.queue2.irqs: 2167993136 dev.ix.0.queue2.txd_head: 1329 dev.ix.0.queue2.txd_tail: 1329 dev.ix.0.queue2.tso_tx: 0 dev.ix.0.queue2.no_tx_dma_setup: 0 dev.ix.0.queue2.no_desc_avail: 190120 dev.ix.0.queue2.tx_packets: 5013202508 dev.ix.0.queue2.rxd_head: 2039 dev.ix.0.queue2.rxd_tail: 2038 dev.ix.0.queue2.rx_packets: 3955460159 dev.ix.0.queue2.rx_bytes: 743382421188 dev.ix.0.queue2.rx_copies: 1422295822 dev.ix.0.queue2.lro_queued: 0 dev.ix.0.queue2.lro_flushed: 0 dev.ix.0.queue3.interrupt_rate: 71428 dev.ix.0.queue3.irqs: 2139498119 dev.ix.0.queue3.txd_head: 673 dev.ix.0.queue3.txd_tail: 673 dev.ix.0.queue3.tso_tx: 0 dev.ix.0.queue3.no_tx_dma_setup: 0 dev.ix.0.queue3.no_desc_avail: 94226 dev.ix.0.queue3.tx_packets: 5301360114 dev.ix.0.queue3.rxd_head: 416 dev.ix.0.queue3.rxd_tail: 415 dev.ix.0.queue3.rx_packets: 3951345010 dev.ix.0.queue3.rx_bytes: 723655881546 dev.ix.0.queue3.rx_copies: 1424750061 dev.ix.0.queue3.lro_queued: 0 dev.ix.0.queue3.lro_flushed: 0 dev.ix.0.queue4.interrupt_rate: 100000 dev.ix.0.queue4.irqs: 2027199532 dev.ix.0.queue4.txd_head: 764 dev.ix.0.queue4.txd_tail: 764 dev.ix.0.queue4.tso_tx: 0 dev.ix.0.queue4.no_tx_dma_setup: 0 dev.ix.0.queue4.no_desc_avail: 174621 dev.ix.0.queue4.tx_packets: 5250099331 dev.ix.0.queue4.rxd_head: 780 dev.ix.0.queue4.rxd_tail: 779 dev.ix.0.queue4.rx_packets: 3898505370 dev.ix.0.queue4.rx_bytes: 680413268286 dev.ix.0.queue4.rx_copies: 1415917068 dev.ix.0.queue4.lro_queued: 0 dev.ix.0.queue4.lro_flushed: 0 dev.ix.0.queue5.interrupt_rate: 62500 dev.ix.0.queue5.irqs: 2076140170 dev.ix.0.queue5.txd_head: 553 dev.ix.0.queue5.txd_tail: 553 dev.ix.0.queue5.tso_tx: 23 dev.ix.0.queue5.no_tx_dma_setup: 0 dev.ix.0.queue5.no_desc_avail: 178058 dev.ix.0.queue5.tx_packets: 5266432651 dev.ix.0.queue5.rxd_head: 1870 dev.ix.0.queue5.rxd_tail: 1869 dev.ix.0.queue5.rx_packets: 3876348699 dev.ix.0.queue5.rx_bytes: 684015705094 dev.ix.0.queue5.rx_copies: 1433886240 dev.ix.0.queue5.lro_queued: 0 dev.ix.0.queue5.lro_flushed: 0 dev.ix.0.queue6.interrupt_rate: 100000 dev.ix.0.queue6.irqs: 2106459361 dev.ix.0.queue6.txd_head: 181 dev.ix.0.queue6.txd_tail: 181 dev.ix.0.queue6.tso_tx: 0 dev.ix.0.queue6.no_tx_dma_setup: 0 dev.ix.0.queue6.no_desc_avail: 2112 dev.ix.0.queue6.tx_packets: 4786334585 dev.ix.0.queue6.rxd_head: 1165 dev.ix.0.queue6.rxd_tail: 1164 dev.ix.0.queue6.rx_packets: 3963283094 dev.ix.0.queue6.rx_bytes: 707926383609 dev.ix.0.queue6.rx_copies: 1443393267 dev.ix.0.queue6.lro_queued: 0 dev.ix.0.queue6.lro_flushed: 0 dev.ix.0.queue7.interrupt_rate: 62500 dev.ix.0.queue7.irqs: 2046097590 dev.ix.0.queue7.txd_head: 1928 dev.ix.0.queue7.txd_tail: 1928 dev.ix.0.queue7.tso_tx: 0 dev.ix.0.queue7.no_tx_dma_setup: 0 dev.ix.0.queue7.no_desc_avail: 151682 dev.ix.0.queue7.tx_packets: 4753275982 dev.ix.0.queue7.rxd_head: 971 dev.ix.0.queue7.rxd_tail: 970 dev.ix.0.queue7.rx_packets: 3941412326 dev.ix.0.queue7.rx_bytes: 707141933261 dev.ix.0.queue7.rx_copies: 1438797770 dev.ix.0.queue7.lro_queued: 0 dev.ix.0.queue7.lro_flushed: 0 dev.ix.0.mac_stats.crc_errs: 119 dev.ix.0.mac_stats.ill_errs: 4 dev.ix.0.mac_stats.byte_errs: 20 dev.ix.0.mac_stats.short_discards: 0 dev.ix.0.mac_stats.local_faults: 14 dev.ix.0.mac_stats.remote_faults: 69 dev.ix.0.mac_stats.rec_len_errs: 0 dev.ix.0.mac_stats.xon_txd: 201740142300 dev.ix.0.mac_stats.xon_recvd: 0 dev.ix.0.mac_stats.xoff_txd: 3736066605 dev.ix.0.mac_stats.xoff_recvd: 0 dev.ix.0.mac_stats.total_octets_rcvd: 11577768204079 dev.ix.0.mac_stats.good_octets_rcvd: 11577747489371 dev.ix.0.mac_stats.total_pkts_rcvd: 31435679394 dev.ix.0.mac_stats.good_pkts_rcvd: 18446741966445774380 dev.ix.0.mac_stats.mcast_pkts_rcvd: 1678 dev.ix.0.mac_stats.bcast_pkts_rcvd: 60437 dev.ix.0.mac_stats.rx_frames_64: 488 dev.ix.0.mac_stats.rx_frames_65_127: 22152221688 dev.ix.0.mac_stats.rx_frames_128_255: 1593120232 dev.ix.0.mac_stats.rx_frames_256_511: 689654943 dev.ix.0.mac_stats.rx_frames_512_1023: 1050762441 dev.ix.0.mac_stats.rx_frames_1024_1522: 5949659560 dev.ix.0.mac_stats.recv_undersized: 0 dev.ix.0.mac_stats.recv_fragmented: 0 dev.ix.0.mac_stats.recv_oversized: 0 dev.ix.0.mac_stats.recv_jabberd: 1 dev.ix.0.mac_stats.management_pkts_rcvd: 0 dev.ix.0.mac_stats.management_pkts_drpd: 0 dev.ix.0.mac_stats.checksum_errs: 189336386 dev.ix.0.mac_stats.good_octets_txd: 39184317469340 dev.ix.0.mac_stats.total_pkts_txd: 40087600379 dev.ix.0.mac_stats.good_pkts_txd: 18446743912615910349 dev.ix.0.mac_stats.bcast_pkts_txd: 73107 dev.ix.0.mac_stats.mcast_pkts_txd: 18446743872528672584 dev.ix.0.mac_stats.management_pkts_txd: 0 dev.ix.0.mac_stats.tx_frames_64: 18446743875376966599 dev.ix.0.mac_stats.tx_frames_65_127: 7716639581 dev.ix.0.mac_stats.tx_frames_128_255: 2192043837 dev.ix.0.mac_stats.tx_frames_256_511: 1138023553 dev.ix.0.mac_stats.tx_frames_512_1023: 1236626066 dev.ix.0.mac_stats.tx_frames_1024_1522: 24955610750 Em 02/07/2014 16:23, Marcelo Gondim escreveu: > Em 02/07/2014 14:07, sthaug@nethelp.no escreveu: >>> Is there any way that you can try a reproduce this using a B2B >>> configuration, or something that doesn't use XFP as a link partner? >>> I'm thinking that you are correct regarding an incompatibility issue >>> between SFP+ and XFP. >> Why do you believe that? The optical signals are the same for SFP+ >> and XFP. >> >> We have lots of 10G SFP+ / XFP links in production. It just works... >> >> Steinar Haug, Nethelp consulting, sthaug@nethelp.no >> > I think I found the problem. Our SFP+ optical module is 850nm MMF and > our transport operator is using an XFP 1310nmMMF. > I am waiting for them to exchange the module and see the result. Once > the module is changed, I post here. > > Thanks and best regards, > Gondim > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Jul 4 07:59:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F468BE7 for ; Fri, 4 Jul 2014 07:59:48 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB8FA2B97 for ; Fri, 4 Jul 2014 07:59:47 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id im17so1260073vcb.39 for ; Fri, 04 Jul 2014 00:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ib2GhJ/BTUGi3HItfHS+z3rYni3+SwTHoLKSTsNFoik=; b=WqM80ynfSpAwLQTWXaKWh7kIhjxbokPjkF1tUJMZDkJB2Sz0gIgZ5kDH2IpL9NeLBe Bhbuz4U2TKzuszq3sdFBWEGFYwbN0prGLTemxP84lLFm5BKuWX/L1oBqRJpZ67NjEktU pIg3WMHDKqWgwWMR4vzXZWB2yI0R5d6XtakNOEEpwJGqErzbSG9ZOyQotflJt1jzblKn cTDpFS2SsHRa3ZJrTT+WocJGHabHeGNkKcTeO+w+2/giDkF4SQmemZ3JeVuHXGuHH9iP bOwF9CXLhENS6RZOqGJp1N01jNH8PuT6IL2v2Qp9ZST4eVTNMAMOKyMPNzywGBUJnMDD SaZA== MIME-Version: 1.0 X-Received: by 10.52.135.226 with SMTP id pv2mr7074045vdb.33.1404460786311; Fri, 04 Jul 2014 00:59:46 -0700 (PDT) Received: by 10.221.4.132 with HTTP; Fri, 4 Jul 2014 00:59:46 -0700 (PDT) In-Reply-To: <53B659EA.20200@bsdinfo.com.br> References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> <53B45C33.1010501@bsdinfo.com.br> <53B659EA.20200@bsdinfo.com.br> Date: Fri, 4 Jul 2014 00:59:46 -0700 Message-ID: Subject: Re: Network Intel X520-SR2 stopping From: Jack Vogel To: Marcelo Gondim Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "Pieper, Jeffrey E" , FreeBSD Net , sthaug@nethelp.no X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 07:59:48 -0000 What does a netstat -m show, I noticed you show no_desc counts on all your queues, perhaps you don't have enough mbufs/clusters available? Does your message log show any events or messages of significance? I'm not sure about the module compatibility, Jeff would be better positioned to answer that. Jack On Fri, Jul 4, 2014 at 12:38 AM, Marcelo Gondim wrote: > Hi all, > > Both modules are 850nm. Could there be some incompatibility between the > Datacom XFP optical module and the SFP + Intel X520-SR2 optical module? > I ask this because I replaced all the hardware and optical cords and > nothing worked. Datacom is a DM4100. > > dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - > 2.5.15 > dev.ix.0.%driver: ix > dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 > dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 > subdevice=0x7a11 class=0x020000 > dev.ix.0.%parent: pci131 > dev.ix.0.fc: 3 > dev.ix.0.enable_aim: 1 > dev.ix.0.advertise_speed: 0 > dev.ix.0.dropped: 0 > dev.ix.0.mbuf_defrag_failed: 0 > dev.ix.0.watchdog_events: 0 > dev.ix.0.link_irq: 61846 > dev.ix.0.queue0.interrupt_rate: 83333 > dev.ix.0.queue0.irqs: 2240081588 > dev.ix.0.queue0.txd_head: 1687 > dev.ix.0.queue0.txd_tail: 1687 > dev.ix.0.queue0.tso_tx: 5 > dev.ix.0.queue0.no_tx_dma_setup: 0 > dev.ix.0.queue0.no_desc_avail: 2171 > dev.ix.0.queue0.tx_packets: 4721544662 > dev.ix.0.queue0.rxd_head: 223 > dev.ix.0.queue0.rxd_tail: 222 > dev.ix.0.queue0.rx_packets: 3939926170 > dev.ix.0.queue0.rx_bytes: 708714319872 > dev.ix.0.queue0.rx_copies: 1428958853 > dev.ix.0.queue0.lro_queued: 0 > dev.ix.0.queue0.lro_flushed: 0 > dev.ix.0.queue1.interrupt_rate: 100000 > dev.ix.0.queue1.irqs: 2164304165 > dev.ix.0.queue1.txd_head: 24 > dev.ix.0.queue1.txd_tail: 24 > dev.ix.0.queue1.tso_tx: 0 > dev.ix.0.queue1.no_tx_dma_setup: 0 > dev.ix.0.queue1.no_desc_avail: 241832 > dev.ix.0.queue1.tx_packets: 4995476933 > dev.ix.0.queue1.rxd_head: 1161 > dev.ix.0.queue1.rxd_tail: 1160 > dev.ix.0.queue1.rx_packets: 3901327164 > dev.ix.0.queue1.rx_bytes: 691440627148 > dev.ix.0.queue1.rx_copies: 1408409383 > dev.ix.0.queue1.lro_queued: 0 > dev.ix.0.queue1.lro_flushed: 0 > dev.ix.0.queue2.interrupt_rate: 83333 > dev.ix.0.queue2.irqs: 2167993136 > dev.ix.0.queue2.txd_head: 1329 > dev.ix.0.queue2.txd_tail: 1329 > dev.ix.0.queue2.tso_tx: 0 > dev.ix.0.queue2.no_tx_dma_setup: 0 > dev.ix.0.queue2.no_desc_avail: 190120 > dev.ix.0.queue2.tx_packets: 5013202508 > dev.ix.0.queue2.rxd_head: 2039 > dev.ix.0.queue2.rxd_tail: 2038 > dev.ix.0.queue2.rx_packets: 3955460159 > dev.ix.0.queue2.rx_bytes: 743382421188 > dev.ix.0.queue2.rx_copies: 1422295822 > dev.ix.0.queue2.lro_queued: 0 > dev.ix.0.queue2.lro_flushed: 0 > dev.ix.0.queue3.interrupt_rate: 71428 > dev.ix.0.queue3.irqs: 2139498119 > dev.ix.0.queue3.txd_head: 673 > dev.ix.0.queue3.txd_tail: 673 > dev.ix.0.queue3.tso_tx: 0 > dev.ix.0.queue3.no_tx_dma_setup: 0 > dev.ix.0.queue3.no_desc_avail: 94226 > dev.ix.0.queue3.tx_packets: 5301360114 > dev.ix.0.queue3.rxd_head: 416 > dev.ix.0.queue3.rxd_tail: 415 > dev.ix.0.queue3.rx_packets: 3951345010 > dev.ix.0.queue3.rx_bytes: 723655881546 > dev.ix.0.queue3.rx_copies: 1424750061 > dev.ix.0.queue3.lro_queued: 0 > dev.ix.0.queue3.lro_flushed: 0 > dev.ix.0.queue4.interrupt_rate: 100000 > dev.ix.0.queue4.irqs: 2027199532 > dev.ix.0.queue4.txd_head: 764 > dev.ix.0.queue4.txd_tail: 764 > dev.ix.0.queue4.tso_tx: 0 > dev.ix.0.queue4.no_tx_dma_setup: 0 > dev.ix.0.queue4.no_desc_avail: 174621 > dev.ix.0.queue4.tx_packets: 5250099331 > dev.ix.0.queue4.rxd_head: 780 > dev.ix.0.queue4.rxd_tail: 779 > dev.ix.0.queue4.rx_packets: 3898505370 > dev.ix.0.queue4.rx_bytes: 680413268286 > dev.ix.0.queue4.rx_copies: 1415917068 > dev.ix.0.queue4.lro_queued: 0 > dev.ix.0.queue4.lro_flushed: 0 > dev.ix.0.queue5.interrupt_rate: 62500 > dev.ix.0.queue5.irqs: 2076140170 > dev.ix.0.queue5.txd_head: 553 > dev.ix.0.queue5.txd_tail: 553 > dev.ix.0.queue5.tso_tx: 23 > dev.ix.0.queue5.no_tx_dma_setup: 0 > dev.ix.0.queue5.no_desc_avail: 178058 > dev.ix.0.queue5.tx_packets: 5266432651 > dev.ix.0.queue5.rxd_head: 1870 > dev.ix.0.queue5.rxd_tail: 1869 > dev.ix.0.queue5.rx_packets: 3876348699 > dev.ix.0.queue5.rx_bytes: 684015705094 > dev.ix.0.queue5.rx_copies: 1433886240 > dev.ix.0.queue5.lro_queued: 0 > dev.ix.0.queue5.lro_flushed: 0 > dev.ix.0.queue6.interrupt_rate: 100000 > dev.ix.0.queue6.irqs: 2106459361 > dev.ix.0.queue6.txd_head: 181 > dev.ix.0.queue6.txd_tail: 181 > dev.ix.0.queue6.tso_tx: 0 > dev.ix.0.queue6.no_tx_dma_setup: 0 > dev.ix.0.queue6.no_desc_avail: 2112 > dev.ix.0.queue6.tx_packets: 4786334585 > dev.ix.0.queue6.rxd_head: 1165 > dev.ix.0.queue6.rxd_tail: 1164 > dev.ix.0.queue6.rx_packets: 3963283094 > dev.ix.0.queue6.rx_bytes: 707926383609 > dev.ix.0.queue6.rx_copies: 1443393267 > dev.ix.0.queue6.lro_queued: 0 > dev.ix.0.queue6.lro_flushed: 0 > dev.ix.0.queue7.interrupt_rate: 62500 > dev.ix.0.queue7.irqs: 2046097590 > dev.ix.0.queue7.txd_head: 1928 > dev.ix.0.queue7.txd_tail: 1928 > dev.ix.0.queue7.tso_tx: 0 > dev.ix.0.queue7.no_tx_dma_setup: 0 > dev.ix.0.queue7.no_desc_avail: 151682 > dev.ix.0.queue7.tx_packets: 4753275982 > dev.ix.0.queue7.rxd_head: 971 > dev.ix.0.queue7.rxd_tail: 970 > dev.ix.0.queue7.rx_packets: 3941412326 > dev.ix.0.queue7.rx_bytes: 707141933261 > dev.ix.0.queue7.rx_copies: 1438797770 > dev.ix.0.queue7.lro_queued: 0 > dev.ix.0.queue7.lro_flushed: 0 > dev.ix.0.mac_stats.crc_errs: 119 > dev.ix.0.mac_stats.ill_errs: 4 > dev.ix.0.mac_stats.byte_errs: 20 > dev.ix.0.mac_stats.short_discards: 0 > dev.ix.0.mac_stats.local_faults: 14 > dev.ix.0.mac_stats.remote_faults: 69 > dev.ix.0.mac_stats.rec_len_errs: 0 > dev.ix.0.mac_stats.xon_txd: 201740142300 > dev.ix.0.mac_stats.xon_recvd: 0 > dev.ix.0.mac_stats.xoff_txd: 3736066605 > dev.ix.0.mac_stats.xoff_recvd: 0 > dev.ix.0.mac_stats.total_octets_rcvd: 11577768204079 > dev.ix.0.mac_stats.good_octets_rcvd: 11577747489371 > dev.ix.0.mac_stats.total_pkts_rcvd: 31435679394 > dev.ix.0.mac_stats.good_pkts_rcvd: 18446741966445774380 > dev.ix.0.mac_stats.mcast_pkts_rcvd: 1678 > dev.ix.0.mac_stats.bcast_pkts_rcvd: 60437 > dev.ix.0.mac_stats.rx_frames_64: 488 > dev.ix.0.mac_stats.rx_frames_65_127: 22152221688 > dev.ix.0.mac_stats.rx_frames_128_255: 1593120232 > dev.ix.0.mac_stats.rx_frames_256_511: 689654943 > dev.ix.0.mac_stats.rx_frames_512_1023: 1050762441 > dev.ix.0.mac_stats.rx_frames_1024_1522: 5949659560 > dev.ix.0.mac_stats.recv_undersized: 0 > dev.ix.0.mac_stats.recv_fragmented: 0 > dev.ix.0.mac_stats.recv_oversized: 0 > dev.ix.0.mac_stats.recv_jabberd: 1 > dev.ix.0.mac_stats.management_pkts_rcvd: 0 > dev.ix.0.mac_stats.management_pkts_drpd: 0 > dev.ix.0.mac_stats.checksum_errs: 189336386 > dev.ix.0.mac_stats.good_octets_txd: 39184317469340 > dev.ix.0.mac_stats.total_pkts_txd: 40087600379 > dev.ix.0.mac_stats.good_pkts_txd: 18446743912615910349 > dev.ix.0.mac_stats.bcast_pkts_txd: 73107 > dev.ix.0.mac_stats.mcast_pkts_txd: 18446743872528672584 > dev.ix.0.mac_stats.management_pkts_txd: 0 > dev.ix.0.mac_stats.tx_frames_64: 18446743875376966599 > dev.ix.0.mac_stats.tx_frames_65_127: 7716639581 > dev.ix.0.mac_stats.tx_frames_128_255: 2192043837 > dev.ix.0.mac_stats.tx_frames_256_511: 1138023553 > dev.ix.0.mac_stats.tx_frames_512_1023: 1236626066 > dev.ix.0.mac_stats.tx_frames_1024_1522: 24955610750 > > > Em 02/07/2014 16:23, Marcelo Gondim escreveu: > > Em 02/07/2014 14:07, sthaug@nethelp.no escreveu: >> >>> Is there any way that you can try a reproduce this using a B2B >>>> configuration, or something that doesn't use XFP as a link partner? I'm >>>> thinking that you are correct regarding an incompatibility issue between >>>> SFP+ and XFP. >>>> >>> Why do you believe that? The optical signals are the same for SFP+ >>> and XFP. >>> >>> We have lots of 10G SFP+ / XFP links in production. It just works... >>> >>> Steinar Haug, Nethelp consulting, sthaug@nethelp.no >>> >>> I think I found the problem. Our SFP+ optical module is 850nm MMF and >> our transport operator is using an XFP 1310nmMMF. >> I am waiting for them to exchange the module and see the result. Once the >> module is changed, I post here. >> >> Thanks and best regards, >> Gondim >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Jul 4 08:10:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 604D3EDD for ; Fri, 4 Jul 2014 08:10:28 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2836A2C86 for ; Fri, 4 Jul 2014 08:10:27 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 5F2FD139D0 for ; Fri, 4 Jul 2014 05:10:50 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1404461449; x=1405325450; bh=uqLifdCfjD+/aUHeu0RK7XMTEdH66ACnUeD vJXQnxQs=; b=jTHxzzvNghh+DnoU4Qm8OtGrXeEWZliS848G+N2ErKKPiwpk5k8 Rrg8dGSYRFc3stDx9OL9/2cRIN3z1tbatqaaEh2mdqnmHQse7vCcwwDZh0mvNnTR KORHNp/NwkUQrOnr10Lowp1ZVfUYAIOzGE/2gbbzbI0E+XXBG5o4ht1M= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kxCpk8Fl0t4k for ; Fri, 4 Jul 2014 05:10:49 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 2D6F9139CB; Fri, 4 Jul 2014 05:10:48 -0300 (BRT) Message-ID: <53B6616B.2080108@bsdinfo.com.br> Date: Fri, 04 Jul 2014 05:10:19 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Jack Vogel Subject: Re: Network Intel X520-SR2 stopping References: <53B3F1B6.9010606@bsdinfo.com.br> <2A35EA60C3C77D438915767F458D65687E8F859E@ORSMSX111.amr.corp.intel.com> <20140702.190739.74686622.sthaug@nethelp.no> <53B45C33.1010501@bsdinfo.com.br> <53B659EA.20200@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Pieper, Jeffrey E" , FreeBSD Net , sthaug@nethelp.no X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 08:10:28 -0000 Hi Jack, # netstat -m 65493/19272/84765 mbufs in use (current/cache/total) 65491/13867/79358/1014370 mbuf clusters in use (current/cache/total/max) 65491/13698 mbuf+clusters out of packet secondary zone in use (current/cache) 0/15/15/507184 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) 147355K/32612K/179967K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile Thanks for your help. Em 04/07/2014 04:59, Jack Vogel escreveu: > What does a netstat -m show, I noticed you show no_desc counts on > all your queues, perhaps you don't have enough mbufs/clusters available? > Does your message log show any events or messages of significance? > > I'm not sure about the module compatibility, Jeff would be better positioned > to answer that. > > Jack > > > > On Fri, Jul 4, 2014 at 12:38 AM, Marcelo Gondim > wrote: > >> Hi all, >> >> Both modules are 850nm. Could there be some incompatibility between the >> Datacom XFP optical module and the SFP + Intel X520-SR2 optical module? >> I ask this because I replaced all the hardware and optical cords and >> nothing worked. Datacom is a DM4100. >> >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - >> 2.5.15 >> dev.ix.0.%driver: ix >> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 >> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 >> subdevice=0x7a11 class=0x020000 >> dev.ix.0.%parent: pci131 >> dev.ix.0.fc: 3 >> dev.ix.0.enable_aim: 1 >> dev.ix.0.advertise_speed: 0 >> dev.ix.0.dropped: 0 >> dev.ix.0.mbuf_defrag_failed: 0 >> dev.ix.0.watchdog_events: 0 >> dev.ix.0.link_irq: 61846 >> dev.ix.0.queue0.interrupt_rate: 83333 >> dev.ix.0.queue0.irqs: 2240081588 >> dev.ix.0.queue0.txd_head: 1687 >> dev.ix.0.queue0.txd_tail: 1687 >> dev.ix.0.queue0.tso_tx: 5 >> dev.ix.0.queue0.no_tx_dma_setup: 0 >> dev.ix.0.queue0.no_desc_avail: 2171 >> dev.ix.0.queue0.tx_packets: 4721544662 >> dev.ix.0.queue0.rxd_head: 223 >> dev.ix.0.queue0.rxd_tail: 222 >> dev.ix.0.queue0.rx_packets: 3939926170 >> dev.ix.0.queue0.rx_bytes: 708714319872 >> dev.ix.0.queue0.rx_copies: 1428958853 >> dev.ix.0.queue0.lro_queued: 0 >> dev.ix.0.queue0.lro_flushed: 0 >> dev.ix.0.queue1.interrupt_rate: 100000 >> dev.ix.0.queue1.irqs: 2164304165 >> dev.ix.0.queue1.txd_head: 24 >> dev.ix.0.queue1.txd_tail: 24 >> dev.ix.0.queue1.tso_tx: 0 >> dev.ix.0.queue1.no_tx_dma_setup: 0 >> dev.ix.0.queue1.no_desc_avail: 241832 >> dev.ix.0.queue1.tx_packets: 4995476933 >> dev.ix.0.queue1.rxd_head: 1161 >> dev.ix.0.queue1.rxd_tail: 1160 >> dev.ix.0.queue1.rx_packets: 3901327164 >> dev.ix.0.queue1.rx_bytes: 691440627148 >> dev.ix.0.queue1.rx_copies: 1408409383 >> dev.ix.0.queue1.lro_queued: 0 >> dev.ix.0.queue1.lro_flushed: 0 >> dev.ix.0.queue2.interrupt_rate: 83333 >> dev.ix.0.queue2.irqs: 2167993136 >> dev.ix.0.queue2.txd_head: 1329 >> dev.ix.0.queue2.txd_tail: 1329 >> dev.ix.0.queue2.tso_tx: 0 >> dev.ix.0.queue2.no_tx_dma_setup: 0 >> dev.ix.0.queue2.no_desc_avail: 190120 >> dev.ix.0.queue2.tx_packets: 5013202508 >> dev.ix.0.queue2.rxd_head: 2039 >> dev.ix.0.queue2.rxd_tail: 2038 >> dev.ix.0.queue2.rx_packets: 3955460159 >> dev.ix.0.queue2.rx_bytes: 743382421188 >> dev.ix.0.queue2.rx_copies: 1422295822 >> dev.ix.0.queue2.lro_queued: 0 >> dev.ix.0.queue2.lro_flushed: 0 >> dev.ix.0.queue3.interrupt_rate: 71428 >> dev.ix.0.queue3.irqs: 2139498119 >> dev.ix.0.queue3.txd_head: 673 >> dev.ix.0.queue3.txd_tail: 673 >> dev.ix.0.queue3.tso_tx: 0 >> dev.ix.0.queue3.no_tx_dma_setup: 0 >> dev.ix.0.queue3.no_desc_avail: 94226 >> dev.ix.0.queue3.tx_packets: 5301360114 >> dev.ix.0.queue3.rxd_head: 416 >> dev.ix.0.queue3.rxd_tail: 415 >> dev.ix.0.queue3.rx_packets: 3951345010 >> dev.ix.0.queue3.rx_bytes: 723655881546 >> dev.ix.0.queue3.rx_copies: 1424750061 >> dev.ix.0.queue3.lro_queued: 0 >> dev.ix.0.queue3.lro_flushed: 0 >> dev.ix.0.queue4.interrupt_rate: 100000 >> dev.ix.0.queue4.irqs: 2027199532 >> dev.ix.0.queue4.txd_head: 764 >> dev.ix.0.queue4.txd_tail: 764 >> dev.ix.0.queue4.tso_tx: 0 >> dev.ix.0.queue4.no_tx_dma_setup: 0 >> dev.ix.0.queue4.no_desc_avail: 174621 >> dev.ix.0.queue4.tx_packets: 5250099331 >> dev.ix.0.queue4.rxd_head: 780 >> dev.ix.0.queue4.rxd_tail: 779 >> dev.ix.0.queue4.rx_packets: 3898505370 >> dev.ix.0.queue4.rx_bytes: 680413268286 >> dev.ix.0.queue4.rx_copies: 1415917068 >> dev.ix.0.queue4.lro_queued: 0 >> dev.ix.0.queue4.lro_flushed: 0 >> dev.ix.0.queue5.interrupt_rate: 62500 >> dev.ix.0.queue5.irqs: 2076140170 >> dev.ix.0.queue5.txd_head: 553 >> dev.ix.0.queue5.txd_tail: 553 >> dev.ix.0.queue5.tso_tx: 23 >> dev.ix.0.queue5.no_tx_dma_setup: 0 >> dev.ix.0.queue5.no_desc_avail: 178058 >> dev.ix.0.queue5.tx_packets: 5266432651 >> dev.ix.0.queue5.rxd_head: 1870 >> dev.ix.0.queue5.rxd_tail: 1869 >> dev.ix.0.queue5.rx_packets: 3876348699 >> dev.ix.0.queue5.rx_bytes: 684015705094 >> dev.ix.0.queue5.rx_copies: 1433886240 >> dev.ix.0.queue5.lro_queued: 0 >> dev.ix.0.queue5.lro_flushed: 0 >> dev.ix.0.queue6.interrupt_rate: 100000 >> dev.ix.0.queue6.irqs: 2106459361 >> dev.ix.0.queue6.txd_head: 181 >> dev.ix.0.queue6.txd_tail: 181 >> dev.ix.0.queue6.tso_tx: 0 >> dev.ix.0.queue6.no_tx_dma_setup: 0 >> dev.ix.0.queue6.no_desc_avail: 2112 >> dev.ix.0.queue6.tx_packets: 4786334585 >> dev.ix.0.queue6.rxd_head: 1165 >> dev.ix.0.queue6.rxd_tail: 1164 >> dev.ix.0.queue6.rx_packets: 3963283094 >> dev.ix.0.queue6.rx_bytes: 707926383609 >> dev.ix.0.queue6.rx_copies: 1443393267 >> dev.ix.0.queue6.lro_queued: 0 >> dev.ix.0.queue6.lro_flushed: 0 >> dev.ix.0.queue7.interrupt_rate: 62500 >> dev.ix.0.queue7.irqs: 2046097590 >> dev.ix.0.queue7.txd_head: 1928 >> dev.ix.0.queue7.txd_tail: 1928 >> dev.ix.0.queue7.tso_tx: 0 >> dev.ix.0.queue7.no_tx_dma_setup: 0 >> dev.ix.0.queue7.no_desc_avail: 151682 >> dev.ix.0.queue7.tx_packets: 4753275982 >> dev.ix.0.queue7.rxd_head: 971 >> dev.ix.0.queue7.rxd_tail: 970 >> dev.ix.0.queue7.rx_packets: 3941412326 >> dev.ix.0.queue7.rx_bytes: 707141933261 >> dev.ix.0.queue7.rx_copies: 1438797770 >> dev.ix.0.queue7.lro_queued: 0 >> dev.ix.0.queue7.lro_flushed: 0 >> dev.ix.0.mac_stats.crc_errs: 119 >> dev.ix.0.mac_stats.ill_errs: 4 >> dev.ix.0.mac_stats.byte_errs: 20 >> dev.ix.0.mac_stats.short_discards: 0 >> dev.ix.0.mac_stats.local_faults: 14 >> dev.ix.0.mac_stats.remote_faults: 69 >> dev.ix.0.mac_stats.rec_len_errs: 0 >> dev.ix.0.mac_stats.xon_txd: 201740142300 >> dev.ix.0.mac_stats.xon_recvd: 0 >> dev.ix.0.mac_stats.xoff_txd: 3736066605 >> dev.ix.0.mac_stats.xoff_recvd: 0 >> dev.ix.0.mac_stats.total_octets_rcvd: 11577768204079 >> dev.ix.0.mac_stats.good_octets_rcvd: 11577747489371 >> dev.ix.0.mac_stats.total_pkts_rcvd: 31435679394 >> dev.ix.0.mac_stats.good_pkts_rcvd: 18446741966445774380 >> dev.ix.0.mac_stats.mcast_pkts_rcvd: 1678 >> dev.ix.0.mac_stats.bcast_pkts_rcvd: 60437 >> dev.ix.0.mac_stats.rx_frames_64: 488 >> dev.ix.0.mac_stats.rx_frames_65_127: 22152221688 >> dev.ix.0.mac_stats.rx_frames_128_255: 1593120232 >> dev.ix.0.mac_stats.rx_frames_256_511: 689654943 >> dev.ix.0.mac_stats.rx_frames_512_1023: 1050762441 >> dev.ix.0.mac_stats.rx_frames_1024_1522: 5949659560 >> dev.ix.0.mac_stats.recv_undersized: 0 >> dev.ix.0.mac_stats.recv_fragmented: 0 >> dev.ix.0.mac_stats.recv_oversized: 0 >> dev.ix.0.mac_stats.recv_jabberd: 1 >> dev.ix.0.mac_stats.management_pkts_rcvd: 0 >> dev.ix.0.mac_stats.management_pkts_drpd: 0 >> dev.ix.0.mac_stats.checksum_errs: 189336386 >> dev.ix.0.mac_stats.good_octets_txd: 39184317469340 >> dev.ix.0.mac_stats.total_pkts_txd: 40087600379 >> dev.ix.0.mac_stats.good_pkts_txd: 18446743912615910349 >> dev.ix.0.mac_stats.bcast_pkts_txd: 73107 >> dev.ix.0.mac_stats.mcast_pkts_txd: 18446743872528672584 >> dev.ix.0.mac_stats.management_pkts_txd: 0 >> dev.ix.0.mac_stats.tx_frames_64: 18446743875376966599 >> dev.ix.0.mac_stats.tx_frames_65_127: 7716639581 >> dev.ix.0.mac_stats.tx_frames_128_255: 2192043837 >> dev.ix.0.mac_stats.tx_frames_256_511: 1138023553 >> dev.ix.0.mac_stats.tx_frames_512_1023: 1236626066 >> dev.ix.0.mac_stats.tx_frames_1024_1522: 24955610750 >> >> >> Em 02/07/2014 16:23, Marcelo Gondim escreveu: >> >> Em 02/07/2014 14:07, sthaug@nethelp.no escreveu: >>>> Is there any way that you can try a reproduce this using a B2B >>>>> configuration, or something that doesn't use XFP as a link partner? I'm >>>>> thinking that you are correct regarding an incompatibility issue between >>>>> SFP+ and XFP. >>>>> >>>> Why do you believe that? The optical signals are the same for SFP+ >>>> and XFP. >>>> >>>> We have lots of 10G SFP+ / XFP links in production. It just works... >>>> >>>> Steinar Haug, Nethelp consulting, sthaug@nethelp.no >>>> >>>> I think I found the problem. Our SFP+ optical module is 850nm MMF and >>> our transport operator is using an XFP 1310nmMMF. >>> I am waiting for them to exchange the module and see the result. Once the >>> module is changed, I post here. >>> >>> Thanks and best regards, >>> Gondim From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 03:06:04 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1B32BA7; Sun, 6 Jul 2014 03:06:04 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9F7B288A; Sun, 6 Jul 2014 03:06:01 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:52945 helo=[192.168.1.10]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X3cl6-00076R-BG; Sat, 05 Jul 2014 23:04:56 -0400 From: "George Neville-Neil" To: testing@freebsd.org, net@freebsd.org Subject: A new way to test systems in multiple machine scenarios... Date: Sat, 05 Jul 2014 23:04:55 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 03:06:05 -0000 Hi, I've coded up a system to allow you to control multiple other systems for use in testing. https://github.com/gvnn3/conductor It's BSD licensed, of course, and is only alpha quality but I'm using it in the test lab to control hosts in forwarding tests. I'll be updating the system frequently over the coming months as I build out more test scenarios, add documentation and the like. There are two main scripts, player, and conductor. You run N players, one per machine, and a single conductor. The conductor controls the players by sending down phases which are encoded in INI style configs. There are a few, simple, samples in the config/ directory of the project. Best, George NOTE: Conductor MUST run as root to be useful. Do NOT run on the open Internet. It is meant for private test labs. From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 03:52:08 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E28418D; Sun, 6 Jul 2014 03:52:08 +0000 (UTC) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F6D52BDD; Sun, 6 Jul 2014 03:52:08 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id rd3so3656631pab.18 for ; Sat, 05 Jul 2014 20:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=3cqFEQ/HxvRouvdmKBBL16okKV+iBeV03h2UtwkdR6U=; b=YGvmLqNE+DjG1x+OGNNvXpVYX1Lmxt2WZXaf1BMMaI/HEp+EQV8q4UGvMz7013S4/j stnkvOiYNXM1HgEPGjdHqCH+KVR6xXJ72Zvgj0fstH2u9OwqWn4XR7LLCCfvYPVFWBft Emb9F2y1CZfQOeYBizA/Ps/fUVMf82fsT+yQxDFpbI6T0bO1JbC7og8waT7FirnJqF/9 pegB7ZhyFDLrqsZuUO9an6A+aBcrwpaz4BQA7Ikh2FNbToaHWGnxpuszHE0l6EwOge70 +YfZuqr1XwIJgjB663cQmTqBNfoR+lklxFBCv5xaZEQL+dJN+zsOvVSwYocB03rOB0W2 p4IA== X-Received: by 10.70.102.10 with SMTP id fk10mr21886pdb.111.1404618727801; Sat, 05 Jul 2014 20:52:07 -0700 (PDT) Received: from [10.101.19.180] (mobile-166-137-212-195.mycingular.net. [166.137.212.195]) by mx.google.com with ESMTPSA id b3sm6331748pbu.8.2014.07.05.20.52.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 05 Jul 2014 20:52:06 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <19D0342C-3635-4DC1-ACB8-5697F1D579F0@gmail.com> X-Mailer: iPhone Mail (11D257) From: Garrett Cooper Subject: Re: A new way to test systems in multiple machine scenarios... Date: Sat, 5 Jul 2014 20:52:03 -0700 To: George Neville-Neil Cc: "testing@freebsd.org" , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 03:52:08 -0000 > On Jul 5, 2014, at 20:04, "George Neville-Neil" wro= te: >=20 > Hi, >=20 > I've coded up a system to allow you to control multiple other systems for u= se in testing. >=20 > https://github.com/gvnn3/conductor >=20 > It's BSD licensed, of course, and is only alpha quality but I'm using it i= n the test lab > to control hosts in forwarding tests. >=20 > I'll be updating the system frequently over the coming months as I build o= ut more test scenarios, > add documentation and the like. >=20 > There are two main scripts, player, and conductor. You run N players, one= per machine, and > a single conductor. The conductor controls the players by sending down ph= ases which are > encoded in INI style configs. There are a few, simple, samples in the con= fig/ directory > of the project. >=20 > Best, > George >=20 > NOTE: Conductor MUST run as root to be useful. Do NOT run on the open Int= ernet. It is meant > for private test labs. I took a quick glance at the code -- have you considered using multiprocessi= ng managers instead? https://docs.python.org/2/library/multiprocessing.html#managers -Garrett= From owner-freebsd-net@FreeBSD.ORG Sun Jul 6 23:27:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 656EBC3D for ; Sun, 6 Jul 2014 23:27:47 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D4242038 for ; Sun, 6 Jul 2014 23:27:46 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id 4D6C716038B; Sun, 6 Jul 2014 16:27:40 -0700 (MST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on quine.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id CF89F160209 for ; Sun, 6 Jul 2014 16:27:37 -0700 (MST) Message-ID: <53B9DB69.1070705@pinyon.org> Date: Sun, 06 Jul 2014 16:27:37 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current References: <870285181.7082888.1404435061501.JavaMail.root@uoguelph.ca> In-Reply-To: <870285181.7082888.1404435061501.JavaMail.root@uoguelph.ca> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 23:27:47 -0000 On 07/03/14 17:51, Rick Macklem wrote: > Well, I took a quick look at the driver and it does use m_defrag(), but > I think that the "retry:" label it does a goto after doing so might be in > the wrong place. > > The attached untested patch might fix this. > > Is it convenient to build a kernel with this patch applied and then try > it with TSO enabled? Patch applied to both client and server (both nics use if_em), net.inet.tcp.tso=1 With a cold 5GB transfer, I see a fairly steady mid 60s MB/s reading on the client. I'm happy BTW. The throughput is now sufficient for my application. HTH, Russell > > rick > ps: It does have the transmit segment limit set to 32. I have no idea if > this is a hardware limitation. > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 01:15:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11103548 for ; Mon, 7 Jul 2014 01:15:31 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CC96E285F for ; Mon, 7 Jul 2014 01:15:30 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAITzuVODaFve/2dsb2JhbABZg2Bagm+7f4ZsUwGBInWEAwEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIAgUEARwEiBkIDa98mS0XgSyNHwYBARs0B4J3gUwFmAqENJJEg18hNX0IFyI X-IronPort-AV: E=Sophos;i="5.01,615,1400040000"; d="scan'208";a="138638228" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 06 Jul 2014 21:15:29 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 980CEB4166; Sun, 6 Jul 2014 21:15:29 -0400 (EDT) Date: Sun, 6 Jul 2014 21:15:29 -0400 (EDT) From: Rick Macklem To: "Russell L. Carter" Message-ID: <248449295.7885770.1404695729613.JavaMail.root@uoguelph.ca> In-Reply-To: <53B9DB69.1070705@pinyon.org> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 01:15:31 -0000 Russell L. Carter wrote: > > > On 07/03/14 17:51, Rick Macklem wrote: > > > Well, I took a quick look at the driver and it does use m_defrag(), > > but > > I think that the "retry:" label it does a goto after doing so might > > be in > > the wrong place. > > > > The attached untested patch might fix this. > > > > Is it convenient to build a kernel with this patch applied and then > > try > > it with TSO enabled? > > Patch applied to both client and server (both nics use if_em), > net.inet.tcp.tso=1 > > With a cold 5GB transfer, I see a fairly steady mid 60s MB/s reading > on the client. I'm happy BTW. The throughput is now sufficient for > my application. > Thanks for testing this. I've emailed some guys that I think might be able to review and/or test/commit this. (I don't mind doing the commit, but I don't have hardware to test it on.) rick > HTH, > Russell > > > > > rick > > ps: It does have the transmit segment limit set to 32. I have no > > idea if > > this is a hardware limitation. > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 04:06:49 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01D1DFFD; Mon, 7 Jul 2014 04:06:49 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 523F22519; Mon, 7 Jul 2014 04:06:48 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id s7so2448405lbd.18 for ; Sun, 06 Jul 2014 21:06:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UCrwhexUbtFEPxDcTnsSquDf8fS7I4/MURVzLxeZgTY=; b=hseFJXb+inyVMx8OXAP6k8XPXGogvVFdWI3Ui1a16n1JxqiS9jNQy42XO+xGEbze9U NaQI16i4VqElZRH0KC0+HYAnJGfz5yJDE45iWZfgwam9RPpCjTwmqcIGSNsAKBdWnsQS lBoDMEtKpDT/s/o64G5Iv7hO1eaWS9caF8jt+ABbvzgQAGZco3aEpQfApNHa3agzkmWa 5/kEPhhxDkJ1HLhb0z4654c+2+ah7N4eis+BPoXGUpwwpVJdVvFXcZLeyqeOXM+oD8lD SQysMIL2pi9k1U0gEfDzn+hOzpgkDmOBVsqb/FJAI0CBSjTR7NK5O2JJDSMchPeVDN6z CQsQ== MIME-Version: 1.0 X-Received: by 10.152.2.3 with SMTP id 3mr20962684laq.8.1404706006074; Sun, 06 Jul 2014 21:06:46 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Sun, 6 Jul 2014 21:06:46 -0700 (PDT) In-Reply-To: References: Date: Sun, 6 Jul 2014 21:06:46 -0700 X-Google-Sender-Auth: -lzvqJltz1khunkvyvAzA0-hnFQ Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Craig Rodrigues To: George Neville-Neil Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 04:06:49 -0000 On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil wrote: > Hi, > > I've coded up a system to allow you to control multiple other systems for > use in testing. > > https://github.com/gvnn3/conductor > > Cool! The architecture you have is similar to that of the SPECsfs benchmark test ( http://www.spec.org/sfs2008/ ) which involves a "coordinator node" and multiple "client nodes" which direct NFS network traffic towards a System Under Test (SUT). Garrett Cooper actually set up the original testbed that I am using now at iXsystems. :) It would be cool to put together tools like Jenkins, Kyua, and conductor to do more advanced testing of FreeBSD before the project puts out releases. -- Craig From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 08:00:12 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0F47DFD for ; Mon, 7 Jul 2014 08:00:12 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C55C925A5 for ; Mon, 7 Jul 2014 08:00:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6780Ch5030238 for ; Mon, 7 Jul 2014 09:00:12 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201407070800.s6780Ch5030238@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 07 Jul 2014 08:00:12 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 08:00:13 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 08:11:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACC72271; Mon, 7 Jul 2014 08:11:58 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FEE5270E; Mon, 7 Jul 2014 08:11:58 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id E5CED1FE02D; Mon, 7 Jul 2014 10:11:56 +0200 (CEST) Message-ID: <53BA5657.8010309@selasky.org> Date: Mon, 07 Jul 2014 10:12:07 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org Subject: [RFC] Allow m_dup() to use JUMBO clusters Content-Type: multipart/mixed; boundary="------------030007040205090607060001" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 08:11:58 -0000 This is a multi-part message in MIME format. --------------030007040205090607060001 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm asking for some input on the attached m_dup() patch, so that existing functionality or dependencies are not broken. The background for the change is to allow m_dup() to defrag long mbuf chains that doesn't fit into a specific hardware's scatter gather entries, typically when doing TSO. In my case the HW limit is 16 entries of length 4K for doing a 64KByte TSO packet. Currently m_dup() is at best producing 32 entries of each 2K for a 64Kbytes TSO packet. By allowing m_dup() to get JUMBO clusters when allocating mbufs, we avoid creating a new function, specific to the hardware, to defrag some rare-occurring very long mbuf chains into a mbuf chain below 16 entries. Any comments? --HPS --------------030007040205090607060001 Content-Type: text/x-patch; name="uipc_mbuf.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="uipc_mbuf.c.diff" === uipc_mbuf.c ================================================================== --- uipc_mbuf.c (revision 268358) +++ uipc_mbuf.c (local) @@ -917,7 +917,15 @@ struct mbuf *n; /* Get the next new mbuf */ - if (remain >= MINCLSIZE) { + if (remain >= MJUM16BYTES) { + /* + * By allocating a bigger mbuf, we get fewer + * scatter gather entries for the hardware to + * process: + */ + n = m_getjcl(how, m->m_type, 0, MJUM16BYTES); + nsize = MJUM16BYTES; + } else if (remain >= MINCLSIZE) { n = m_getcl(how, m->m_type, 0); nsize = MCLBYTES; } else { --------------030007040205090607060001-- From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:09:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA3C22E5; Mon, 7 Jul 2014 11:09:20 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84FCA27EB; Mon, 7 Jul 2014 11:09:19 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 1F67C9DCD0F; Mon, 7 Jul 2014 13:03:31 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=utf-8 From: Borja Marcos In-Reply-To: Date: Mon, 7 Jul 2014 13:03:26 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> References: To: Luigi Rizzo X-Mailer: Apple Mail (2.1283) Cc: freebsd-net@freebsd.org, freebsd-current , Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 11:09:20 -0000 On Jul 1, 2014, at 10:24 PM, Luigi Rizzo wrote: >=20 >=20 >=20 > On Tue, Jul 1, 2014 at 8:58 PM, wrote: > El 30.06.2014 18:36, Stefano Garzarella escribi=C3=B3: >=20 > Hello, > I had problems during some experiments with Emulex and "oce" driver in > CURRENT. > I found several bugs in the "oce" driver and this patch fixes them. >=20 > At least with some cards, the driver simply does not work. It causes a = panic when there is some traffic. >=20 > The relevant bug report is here. >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183391 >=20 > The latest version available from the Emulex website works. But the = version bundled with 9.3 and at least -STABLE (which is the same version = bundled with -CURRENT) does cause panics on 10- and 9- >=20 > =E2=80=8Bi compared the code on the emulex website (10.0.747.0 ?) with = the > one in HEAD and it does not seem=E2=80=8B much different, but perhaps > you have some other version in mind ? >=20 > The bugs found by stefano exist also in the emulex version above. Anyway The "fixed" version is an instant panic when generating traffic (just = use iperf3). Version 10.0.747.0 does _not_ panic. Borja. From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:23:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16B77797; Mon, 7 Jul 2014 11:23:45 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C4412981; Mon, 7 Jul 2014 11:23:44 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so2753353lab.36 for ; Mon, 07 Jul 2014 04:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=P3h6CFQrbRbCvtOYXQiDBDwDe1XajiNHEUnvWwi7//4=; b=Dus9AcyvUTwdkc0Ap8jghOEhAR8sFRGwL1gemS4bnSnnvze7NKUJP8A5wyVONiwqcE /MUGydn3bFuCcqDT5lRuDEMvO6iq5JUD0+jHwdBo7npCm/ZcbYyDGGGxxEt2A+QihbIl bJ9SalEuuBnkh1xTLiTTQS//2cYHsllEc7csfhINPnYub7LktxJKEUrj9pLxZBV5oKTc j1r5U9BdFqI+YoTRYe7V7PIKx3fhtGkNf1LwrXfB0DoiDMSHP/G9rls9ZAcq7+RbDdAi dOPDlMTCtLEyl6uiRw/vtb9nBCuIPx7TMhnscEaYTYwBkhfJH9W/YTvtTwt/sKtZ2iQj hnVQ== MIME-Version: 1.0 X-Received: by 10.112.42.45 with SMTP id k13mr992210lbl.88.1404732221265; Mon, 07 Jul 2014 04:23:41 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.234 with HTTP; Mon, 7 Jul 2014 04:23:41 -0700 (PDT) In-Reply-To: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> Date: Mon, 7 Jul 2014 13:23:41 +0200 X-Google-Sender-Auth: 2W30VX2wv15ne_yh00tN4dbSBL0 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Luigi Rizzo To: Borja Marcos Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , freebsd-current , Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 11:23:45 -0000 On Mon, Jul 7, 2014 at 1:03 PM, Borja Marcos wrote: > > On Jul 1, 2014, at 10:24 PM, Luigi Rizzo wrote: > >> >> >> >> On Tue, Jul 1, 2014 at 8:58 PM, wrote: >> El 30.06.2014 18:36, Stefano Garzarella escribi=C3=B3: >> >> Hello, >> I had problems during some experiments with Emulex and "oce" driver in >> CURRENT. >> I found several bugs in the "oce" driver and this patch fixes them. >> >> At least with some cards, the driver simply does not work. It causes a p= anic when there is some traffic. >> >> The relevant bug report is here. >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183391 >> >> The latest version available from the Emulex website works. But the vers= ion bundled with 9.3 and at least -STABLE (which is the same version bundle= d with -CURRENT) does cause panics on 10- and 9- >> >> i compared the code on the emulex website (10.0.747.0 ?) with the >> one in HEAD and it does not seem much different, but perhaps >> you have some other version in mind ? >> >> The bugs found by stefano exist also in the emulex version above. > > Anyway > > The "fixed" version is an instant panic when generating traffic (just use= iperf3). Version 10.0.747.0 does _not_ panic. we'll try to investigate, can you tell us more about the environment you us= e ? (FreeBSD version, card model (PCI id perhaps), iperf3 invocation line, interface configuration etc.) The main differences between 10.0.747.0 and the code in head (after our fix) is the use of drbr_enqueue/dequeue versus the peek/putback in the transmit routine. Both drivers still have issues when the link flaps because the transmit queue is not cleaned up properly (unlike what happens in the linux driver and all FreeBSD drivers for different hardware), so it might well be that you are seeing some side effect of that or other problem which manifests itself differently depending on the environment. 'instant panic' by itself does not tell us anything about what could be the problem you experience (and we do not see it with either driver). cheers luigi From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 11:57:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD9365EE; Mon, 7 Jul 2014 11:57:11 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D2E92C2F; Mon, 7 Jul 2014 11:57:10 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id CD6199DC9B8; Mon, 7 Jul 2014 13:57:08 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Mon, 7 Jul 2014 13:57:07 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> To: Luigi Rizzo X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 11:57:12 -0000 On Jul 7, 2014, at 1:23 PM, Luigi Rizzo wrote: > On Mon, Jul 7, 2014 at 1:03 PM, Borja Marcos = wrote: > we'll try to investigate, can you tell us more about the environment = you use ? > (FreeBSD version, card model (PCI id perhaps), iperf3 invocation line, > interface configuration etc.) >=20 > The main differences between 10.0.747.0 and the code in head (after > our fix) is the use > of drbr_enqueue/dequeue versus the peek/putback in the transmit = routine. >=20 >=20 > Both drivers still have issues when the link flaps because the > transmit queue is not cleaned > up properly (unlike what happens in the linux driver and all FreeBSD > drivers for different > hardware), so it might well be that you are seeing some side effect of > that or other > problem which manifests itself differently depending on the = environment. >=20 > 'instant panic' by itself does not tell us anything about what could > be the problem you experience (and we do not see it with either = driver). The environment details are here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D183391 The way I produce an instant panic is: 1) Connect to another machine (cross connect cable) 2) iperf3 -s on the other machine=20 (The other machine is different, it has an "ix" card) 3) iperf3 -t 30 -P 4 -c 10.0.0.1 -N In less than 30 seconds, panic. mierda dumped core - see /var/crash/vmcore.0 Mon Jul 7 13:06:44 CEST 2014 FreeBSD mierda 10.0-STABLE FreeBSD 10.0-STABLE #2: Mon Jul 7 11:41:45 = CEST 2014 root@mierda:/usr/obj/usr/src/sys/GENERIC amd64 panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 = clashing GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 = clashing cpuid =3D 12 KDB: stack backtrace: #0 0xffffffff8092a470 at kdb_backtrace+0x60 #1 0xffffffff808ef9c5 at panic+0x155 #2 0xffffffff80962710 at sbdroprecord_locked+0 #3 0xffffffff80a8ba8c at tcp_output+0xdbc #4 0xffffffff80a8987f at tcp_do_segment+0x30ff #5 0xffffffff80a85b34 at tcp_input+0xd04 #6 0xffffffff80a1af57 at ip_input+0x97 #7 0xffffffff809ba512 at netisr_dispatch_src+0x62 #8 0xffffffff809b1ae6 at ether_demux+0x126 #9 0xffffffff809b278e at ether_nh_input+0x35e #10 0xffffffff809ba512 at netisr_dispatch_src+0x62 #11 0xffffffff81c19ab9 at oce_rx+0x3c9 #12 0xffffffff81c19536 at oce_rq_handler+0xb6 #13 0xffffffff81c1bb1c at oce_intr+0xdc #14 0xffffffff80938b35 at taskqueue_run_locked+0xe5 #15 0xffffffff809395c8 at taskqueue_thread_loop+0xa8 #16 0xffffffff808c057a at fork_exit+0x9a #17 0xffffffff80ccb51e at fork_trampoline+0xe Uptime: 51m20s Borja. From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 12:28:21 2014 Return-Path: Delivered-To: FreeBSD-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50C26681 for ; Mon, 7 Jul 2014 12:28:21 +0000 (UTC) Received: from mail-ie0-f174.google.com (mail-ie0-f174.google.com [209.85.223.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 214DA2F0D for ; Mon, 7 Jul 2014 12:28:17 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id rd18so3591225iec.33 for ; Mon, 07 Jul 2014 05:28:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=9MwomwK5F+w2658kIHfCLcZPKblxiMGP/XzPG0yB54k=; b=k2dHayedokqAGhfr2eEF/vco0OpI3UVQHdATZnw1ZylO8NrNcQGDVySSNO4O4gQR42 vqpXVe/INgxTE7lU7WyN9unbBr1pgDmgMW6mXjrGggJf4Lwp7RgeDeGf3c9dIiRGvKnF vsu2X2igHoiGC5QJokP7TEwE80p1ztoYBmlhUYtnj8AL1Kc+20SU+Qek6NIy6gYly6Xv /qi4gAurnQSMhQw53WzcYmB7wy9JRiNU+kMPU8hqYAgMEKpJPIKRRAL0oCdEi8HdM8I5 lwR5Wog9BTgsYUwtxFOqCPvDyy85PdTn5H1QmGmni6fVxti2VmVH9D1r75y04lH/+nuU rl+w== X-Gm-Message-State: ALoCoQmC8lA4olS/QDmq96Ufggkp3ElC6u1CYrk4uoAEyNv6iS2LriSH0MN84bw1XcoN8/EbV9hG MIME-Version: 1.0 X-Received: by 10.50.25.104 with SMTP id b8mr22126608igg.28.1404736090938; Mon, 07 Jul 2014 05:28:10 -0700 (PDT) Received: by 10.64.249.166 with HTTP; Mon, 7 Jul 2014 05:28:10 -0700 (PDT) Received: by 10.64.249.166 with HTTP; Mon, 7 Jul 2014 05:28:10 -0700 (PDT) Date: Mon, 7 Jul 2014 08:28:10 -0400 Message-ID: Subject: please include me in mailing list From: WEB BY DOUG To: FreeBSD-net@FreeBSD.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 12:28:21 -0000 Please include me in mailing list. Kind regards, From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 15:08:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D394FAB; Mon, 7 Jul 2014 15:08:29 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A12342F2E; Mon, 7 Jul 2014 15:08:28 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id pn19so3021350lab.29 for ; Mon, 07 Jul 2014 08:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=aaoCGmlGSMPw9gHPw5QgpDC1vOY25zouKClL6n4DTBw=; b=k2FqttjwLJEgMNUrVpIlMs+4Ki/VQqJnvI+z1+l04oIx1/FiJypVEoehkRq6Sr77XJ aBTdcDSdKFHP9S4mA3Doaky7f725DXO14mlkAZ5jTujwiQKu1nnfXIFKhTTdqELIOEgE 6dAxJP7Gl0QNXWkRBdxsQbAQCHCxcDL4IfDuDGg3kuW1BNnVW/9rQiB4NtgMA5OVcz12 XXnu6yKVYpXISK0aG6fI5z2BMVavzXR/iTna6f7qZgO73rkDgdWr5uILDX2IxdFhgaKN sQwB3caQpMXG2Ck/4Jge9pWaxXI//4kvhVS/+q2PxWNE3cu2sFos0Q2OpGNxf6kDeU6v PmmA== MIME-Version: 1.0 X-Received: by 10.112.88.202 with SMTP id bi10mr22396397lbb.4.1404745706312; Mon, 07 Jul 2014 08:08:26 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.234 with HTTP; Mon, 7 Jul 2014 08:08:26 -0700 (PDT) In-Reply-To: <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Mon, 7 Jul 2014 17:08:26 +0200 X-Google-Sender-Auth: ltrIfch7koB8C3lexHP7R4AuaZM Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Luigi Rizzo To: Borja Marcos Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , freebsd-current , Stable Stable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 15:08:29 -0000 On Mon, Jul 7, 2014 at 1:57 PM, Borja Marcos wrote: ... > The environment details are here: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 > > The way I produce an instant panic is: > > 1) Connect to another machine (cross connect cable) > > 2) iperf3 -s on the other machine > (The other machine is different, it has an "ix" card) > > 3) iperf3 -t 30 -P 4 -c 10.0.0.1 -N > > In less than 30 seconds, panic. > > > > mierda dumped core - see /var/crash/vmcore.0 > > Mon Jul 7 13:06:44 CEST 2014 > > FreeBSD mierda 10.0-STABLE FreeBSD 10.0-STABLE #2: Mon Jul 7 11:41:45 CEST 2014 root@mierda:/usr/obj/usr/src/sys/GENERIC amd64 > > panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 clashing > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 clashing > cpuid = 12 > KDB: stack backtrace: > #0 0xffffffff8092a470 at kdb_backtrace+0x60 > #1 0xffffffff808ef9c5 at panic+0x155 > #2 0xffffffff80962710 at sbdroprecord_locked+0 > #3 0xffffffff80a8ba8c at tcp_output+0xdbc > #4 0xffffffff80a8987f at tcp_do_segment+0x30ff > #5 0xffffffff80a85b34 at tcp_input+0xd04 > #6 0xffffffff80a1af57 at ip_input+0x97 > #7 0xffffffff809ba512 at netisr_dispatch_src+0x62 > #8 0xffffffff809b1ae6 at ether_demux+0x126 > #9 0xffffffff809b278e at ether_nh_input+0x35e > #10 0xffffffff809ba512 at netisr_dispatch_src+0x62 > #11 0xffffffff81c19ab9 at oce_rx+0x3c9 > #12 0xffffffff81c19536 at oce_rq_handler+0xb6 > #13 0xffffffff81c1bb1c at oce_intr+0xdc > #14 0xffffffff80938b35 at taskqueue_run_locked+0xe5 > #15 0xffffffff809395c8 at taskqueue_thread_loop+0xa8 > #16 0xffffffff808c057a at fork_exit+0x9a > #17 0xffffffff80ccb51e at fork_trampoline+0xe > Uptime: 51m20s ah, that seems a bug on the receive side, we were only looking at the transmit side so far. cheers luigi From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 15:22:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1E696BC for ; Mon, 7 Jul 2014 15:22:55 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAB4B2116 for ; Mon, 7 Jul 2014 15:22:55 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 7541A139C8 for ; Mon, 7 Jul 2014 12:22:48 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1404746567; x=1405610568; bh=N+GAUioNNa0w 28x1QuMWix6L8q+WPb7jNsEMMs1LNRU=; b=mmxBYrgi23eE1qqdOG2TpJcNU2AP FuBQ4w6vmc0FdMEf2Yez0qoqIAY/rLQE8zxt+hipK+iCKU7lKm3iY4CY99EoS1TI 5rDU2CGgS34N2JcamULHNWk8vQ6V3QnSkPXvwxeoL08MTMHQiLigjPhs51rNFpEQ Ggdqmqou2oik7zE= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UhwqARyPvWZk for ; Mon, 7 Jul 2014 12:22:47 -0300 (BRT) Received: from [192.168.88.24] (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 4BBDB139C4; Mon, 7 Jul 2014 12:22:46 -0300 (BRT) Message-ID: <53BABB3F.2010603@bsdinfo.com.br> Date: Mon, 07 Jul 2014 12:22:39 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" , Dennis Yusupoff , FreeBSD Net Subject: Re: Problem with ipfw table add 0.0.0.0/8 References: <5371084F.1060009@bsdinfo.com.br> <5371112B.2030209@bsdinfo.com.br> <5371E9E7.70400@smartspb.net> <5371F4C8.3080501@FreeBSD.org> <53720AA4.80909@smartspb.net> <537767C5.80205@FreeBSD.org> <5377F0BB.1040501@bsdinfo.com.br> <5377F2CB.5010406@bsdinfo.com.br> In-Reply-To: <5377F2CB.5010406@bsdinfo.com.br> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 15:22:56 -0000 Hi all, Already exists MFC available to fix this problem in 10-STABLE? # ipfw table 99 add 0.0.0.0/8 # ipfw table 99 list ::/8 0 Cheers, Gondim Em 17/05/2014 20:37, Marcelo Gondim escreveu: > Em 17/05/14 20:28, Marcelo Gondim escreveu: >> Em 17/05/14 10:44, Alexander V. Chernikov escreveu: >>> On 13.05.2014 16:05, Dennis Yusupoff wrote: >>>> I think that universal table for all kind of data (ipv4, ipv6, ports, >>>> etc) is a bad idea by design. At least unless you haven't any >>>> ability to >>> It is not always "universal" in kernel. >>> Actually, different radix tables are used to store both IPv4 and >>> IPv6 in single table. >>>> specify address family on add, to avoid attempts to guess what user >>>> meant. Something like "ipfw table X add DEEF.DE ipv6". >>> I'm going to add explicit table type/naming setup soon. >>> Idea is the following: >>> >>> 1) Existing table can be named and addressed by either number or name. >>> However, you still need to assign table number manually. >>> >>> 2) Table type/name can be specified explicitly via one of the >>> following commands: >>> * ipfw table 1 create [type ] [name >>> "table_name"] >>> * ipfw table name "table_name" >>> * ipfw table "table_name" type >>> >>> 3) ipfw(8) stops trying to guess appropriate type based on used >>> value. Instead, >>> it requests table type from kernel and interprets value according to >>> returned type. >>> Default type for all tables is cidr >>> >>> 4) Table(s) can be returned to default values using ipfw table >>> destroy. >>> Destroy means: >>> * flush >>> * table tries (or other structures) freed >>> * type set to cidr >>> >>> >>>> >>>> >>>> 13.05.2014 14:32, Alexander V. Chernikov ÐŋÐļŅˆÐĩŅ‚: >>>>> On 13.05.2014 13:46, Dennis Yusupoff wrote: >>>>>> May be this will help? See answer on >>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/189471 >>>>> I'll try to fix it within a few days. >>> Fixed in r266310. >> The problem still exists. >> >> # ipfw table 99 add 0.0.0.0/8 >> # ipfw table 99 list >> ::/8 0 >> >> # uname -a >> FreeBSD mail.xxxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #8 >> r266370: Sat May 17 19:57:23 BRT 2014 >> root@mail.xxxxxx.br:/usr/obj/usr/src/sys/GONDIM amd64 > Ah! Sorry! > Is still in the head. > > From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 16:58:54 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9AFC1D4; Mon, 7 Jul 2014 16:58:54 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A6B9298B; Mon, 7 Jul 2014 16:58:54 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id q108so3890199qgd.7 for ; Mon, 07 Jul 2014 09:58:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=faWabrU5IjpAkM6twdxrgKDbfNdWqxGXwsNmaYIZMmk=; b=QgNCCBx/p7Hxh5OQjsl1I1VpOyLIpWp+MMejkbs6cysGAFtKRN5lmcwrU9n438OytP KLCNdyAnCcEB93nP5hXjR2T7hd7aOy4GGfKq1OMKjShQ05XQPCgCAJC5z//2PCTc91zi F9rQTCMBy1bJ+Xfoc4UQxOokasgv02XOKcPE3H7wM2hdGnv4YhPh7JbAJSs4/5BhmDAw 3z0yHuu2j5ePAwN8jFmJ1G4xK0HGsR0PJ+DnTgSopIw7Jud2Yu6Mk1fMOzRS2bGTU3Wo ClPzb8i2Mru17i9GUatidxZqM6anWthrKhKJlrg9IVut8LS53fpicBnF8eqblvXAwzSJ 55kg== MIME-Version: 1.0 X-Received: by 10.224.30.75 with SMTP id t11mr49883892qac.7.1404752333532; Mon, 07 Jul 2014 09:58:53 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Mon, 7 Jul 2014 09:58:53 -0700 (PDT) In-Reply-To: <53BA8666.9040608@omnilan.de> References: <37229F0A-4BEA-4712-9AE1-5446A630AF9F@transactionware.com> <53BA8666.9040608@omnilan.de> Date: Mon, 7 Jul 2014 09:58:53 -0700 Message-ID: Subject: Re: r256920 missing in stable/9 and releng/9.3 From: hiren panchasara To: Harald Schmalzbauer , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Stable Mailing List , Andre Oppermann X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 16:58:54 -0000 + freebsd-net@, On Mon, Jul 7, 2014 at 4:37 AM, Harald Schmalzbauer wrote: > Bez=C3=BCglich Jan Mikkelsen's Nachricht vom 24.06.2014 04:49 (localtime)= : >> Hi, >> >> I=E2=80=99m bringing 9.3-RC1 into our local Perforce depot and moving ou= r local patches to 9.2 forward. >> >> I noticed that r256920 (changing sys/netinet/tcp_input.c) has not been M= FC=E2=80=99d. It was listed as =E2=80=9CMFC after 3 days=E2=80=9D back in O= ctober 2013. >> >> Is this patch missing for a reason? > > I'm wondering too if there's any good reason not to MFC? I also don't see any obvious reason. If nobody objects on -net@, I can do it. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 20:26:17 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E337260 for ; Mon, 7 Jul 2014 20:26:17 +0000 (UTC) Received: from mp1-smtp-2.eutelia.it (mp1-smtp-2.eutelia.it [62.94.10.162]) by mx1.freebsd.org (Postfix) with ESMTP id 131DE2D91 for ; Mon, 7 Jul 2014 20:26:16 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mp1-smtp-2.eutelia.it (Eutelia) with ESMTP id B902FE206F for ; Mon, 7 Jul 2014 22:26:09 +0200 (CEST) Received: from soth.ventu (adsl-ull-14-252.41-151.net24.it [151.41.252.14]) (authenticated bits=0) by ns2.biolchim.it (8.14.9/8.14.8) with ESMTP id s67KQ5Kp009878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 7 Jul 2014 22:26:06 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: ns2.biolchim.it: Host adsl-ull-14-252.41-151.net24.it [151.41.252.14] claimed to be soth.ventu Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.9/8.14.7) with ESMTP id s67KQ0EY095211; Mon, 7 Jul 2014 22:26:00 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <53BB0258.8080107@netfence.it> Date: Mon, 07 Jul 2014 22:26:00 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Hay , freebsd-net@freebsd.org Subject: Re: MTU not regrowing? References: <53A9C6D0.3090900@netfence.it> <20140624190333.GA13748@zibbi.meraka.csir.co.za> <53B2FFE9.3030906@netfence.it> <20140702025302.GI45513@funkthat.com> In-Reply-To: <20140702025302.GI45513@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Mon, 07 Jul 2014 22:26:07 +0200 (CEST) X-Scanned-By: MIMEDefang 2.74 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 20:26:17 -0000 On 07/02/14 04:53, John-Mark Gurney wrote: >> How do I do this? >> I tried "route delete", but it doesn't help. > > route change -mtu XXX This does not work: the route is deemed as non-existent. bye & thanks av. P.S. I'm writing this more out of curiosity, than of real need; no need to solve this, really. From owner-freebsd-net@FreeBSD.ORG Mon Jul 7 21:42:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E1966B1; Mon, 7 Jul 2014 21:42:16 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3C92F2488; Mon, 7 Jul 2014 21:42:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAEMTu1ODaFve/2dsb2JhbABZg2Bagm+8CYZyUwGBLHWEAwEBAQQBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEaAgSIIQ2vYJlgF4EsiDmEZgYBARs0B4J3gUwFmAqENIoEiECDXyE1fQgXIg X-IronPort-AV: E=Sophos;i="5.01,620,1400040000"; d="scan'208";a="138237862" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 07 Jul 2014 17:42:08 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B34C6B3F45; Mon, 7 Jul 2014 17:42:08 -0400 (EDT) Date: Mon, 7 Jul 2014 17:42:08 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Message-ID: <1613382893.8316833.1404769328724.JavaMail.root@uoguelph.ca> In-Reply-To: <53BA5657.8010309@selasky.org> Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 21:42:16 -0000 Hans Petter Selasky wrote: > Hi, > > I'm asking for some input on the attached m_dup() patch, so that > existing functionality or dependencies are not broken. The background > for the change is to allow m_dup() to defrag long mbuf chains that > doesn't fit into a specific hardware's scatter gather entries, > typically > when doing TSO. > > In my case the HW limit is 16 entries of length 4K for doing a > 64KByte > TSO packet. Currently m_dup() is at best producing 32 entries of each > 2K > for a 64Kbytes TSO packet. > > By allowing m_dup() to get JUMBO clusters when allocating mbufs, we > avoid creating a new function, specific to the hardware, to defrag > some > rare-occurring very long mbuf chains into a mbuf chain below 16 > entries. > > Any comments? > 1 - If you are using NFS with the default (64K) I/O size, then long mbuf chains of 35 entries aren't rare. They happen on every read reply/write request. 2 - When I changed NFS to use pagesize clusters for reads/writes I was able to get the system into a state where threads were persistently stuck on "btalloc". If I understand this correctly, the system was not able to allocate boundary tags because the kernel address space had been fragmented too much. --> As such, I never committed this patch to head and would caution against using pagesize clusters. I do not have a better solution at this point, but I do have an untested patch (I need to get access to some TSO enabled hardware to test it) that adds if_hw_tsomaxseg, which is a count of the maximum number of transmit segments (mbufs in chain) that a network device driver supports. I think that having the driver set if_hw_tsomaxseg == 16 is preferable to doing a copy of the data to pagesize clusters. (I'd also say that hardware that supports only 16 transmit segments for a TSO segment is not a good piece of hardware for FreeBSD.) rick > --HPS > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 00:52:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF610256; Tue, 8 Jul 2014 00:52:36 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CBA5F2424; Tue, 8 Jul 2014 00:52:36 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s680qZwT073334 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jul 2014 17:52:35 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s680qY4P073333; Mon, 7 Jul 2014 17:52:34 -0700 (PDT) (envelope-from jmg) Date: Mon, 7 Jul 2014 17:52:34 -0700 From: John-Mark Gurney To: Hans Petter Selasky Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters Message-ID: <20140708005234.GJ45513@funkthat.com> Mail-Followup-To: Hans Petter Selasky , freebsd-net@freebsd.org, freebsd-current@freebsd.org References: <53BA5657.8010309@selasky.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53BA5657.8010309@selasky.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 07 Jul 2014 17:52:35 -0700 (PDT) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 00:52:37 -0000 Hans Petter Selasky wrote this message on Mon, Jul 07, 2014 at 10:12 +0200: > I'm asking for some input on the attached m_dup() patch, so that > existing functionality or dependencies are not broken. The background > for the change is to allow m_dup() to defrag long mbuf chains that > doesn't fit into a specific hardware's scatter gather entries, typically > when doing TSO. > > In my case the HW limit is 16 entries of length 4K for doing a 64KByte > TSO packet. Currently m_dup() is at best producing 32 entries of each 2K > for a 64Kbytes TSO packet. > > By allowing m_dup() to get JUMBO clusters when allocating mbufs, we > avoid creating a new function, specific to the hardware, to defrag some > rare-occurring very long mbuf chains into a mbuf chain below 16 entries. Please no... Until we get a better allocator, we should not use jumbo (>page sized) mbufs otherwise we will quickly fail to allocate mbufs after a machine has been up for a long while causing other failures... Unless of course if the code fails to allocate the largest cluster it falls through to trying to allocate the next smaller size, that might be better... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 02:06:41 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC05BED7 for ; Tue, 8 Jul 2014 02:06:41 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A30A82BC4 for ; Tue, 8 Jul 2014 02:06:41 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6826fjT042054 for ; Tue, 8 Jul 2014 03:06:41 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191415] [carp] CARP password with "-" causes /etc/rc.d/netif to infinite loop Date: Tue, 08 Jul 2014 02:06:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 02:06:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191415 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd-rc@FreeBSD.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|CARP password with "-" |[carp] CARP password with |causes /etc/rc.d/netif to |"-" causes /etc/rc.d/netif |infinite loop |to infinite loop --- Comment #1 from Mark Linimon --- Over to -net; notify -rc. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 02:08:49 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 787E2195 for ; Tue, 8 Jul 2014 02:08:49 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5FB712BEA for ; Tue, 8 Jul 2014 02:08:49 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6828nFS044801 for ; Tue, 8 Jul 2014 03:08:49 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 191700] [carp] [bridge] CARP does not work on interface which belongs to a bridge Date: Tue, 08 Jul 2014 02:08:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 02:08:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191700 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|CARP does not work on |[carp] [bridge] CARP does |interface which belongs to |not work on interface which |a bridge |belongs to a bridge --- Comment #1 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 02:14:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DC574E7; Tue, 8 Jul 2014 02:14:46 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E14172CA0; Tue, 8 Jul 2014 02:14:45 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fb1so6438050pad.28 for ; Mon, 07 Jul 2014 19:14:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Yg7ZzN5xK2nwVyX7SfVfK/3muobXoPp/qQm81mzuVE4=; b=vqiSEpkKM4ueSil2b9ERSVoBtGfvSojK37vXogaMoJ01kA9PG7gTt0L/h/q9NhsQTt OH6hNg9dgcSVbcxNEWTPTB+Np+sn2RurBMcKcqXzO0xSSb6kJrbC+7OZXUX4DEDm3f1Q mJVKqwKiX15dk6KAS8FHcoWbvBkeYZlCB2Df2cerGaYxJLvkby8j7smF2x0nWgoQgssE daepWPUw3g0x0GyiRK2JqSWf2vGK2GHmO3IpgfOzybeNypsJV+SOJ6IDDdWPyHCQ/Avb qfS3i0murE4z6G+VrITlFnE+GwF4VfjFB/xCr0+CmbrGS7ZLKWIg5xpKwBrhCOwtR/tI gM1w== X-Received: by 10.69.18.11 with SMTP id gi11mr32173688pbd.36.1404785685381; Mon, 07 Jul 2014 19:14:45 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by mx.google.com with ESMTPSA id wp3sm53981897pbc.67.2014.07.07.19.14.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 07 Jul 2014 19:14:44 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 08 Jul 2014 11:14:39 +0900 Date: Tue, 8 Jul 2014 11:14:39 +0900 To: Hans Petter Selasky Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters Message-ID: <20140708021439.GA3965@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <53BA5657.8010309@selasky.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53BA5657.8010309@selasky.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 02:14:46 -0000 On Mon, Jul 07, 2014 at 10:12:07AM +0200, Hans Petter Selasky wrote: > Hi, > > I'm asking for some input on the attached m_dup() patch, so that > existing functionality or dependencies are not broken. The background > for the change is to allow m_dup() to defrag long mbuf chains that > doesn't fit into a specific hardware's scatter gather entries, typically > when doing TSO. > > In my case the HW limit is 16 entries of length 4K for doing a 64KByte I wonder how HW can handle a full-sized TSO packet(64KB + Ethernet header + VLAN tag). > TSO packet. Currently m_dup() is at best producing 32 entries of each 2K > for a 64Kbytes TSO packet. > > By allowing m_dup() to get JUMBO clusters when allocating mbufs, we > avoid creating a new function, specific to the hardware, to defrag some > rare-occurring very long mbuf chains into a mbuf chain below 16 entries. > I think m_dup() was used to get a copy of writable mbuf chains. If m_dup() starts to allocate jumbo mbufs it will eventually fail on long running boxes. This will break firewall(ipfw divert, pf/ipf dup-to) rules and several ethernet drivers. I don't know how many TSO requests could be queued by HW but if the number is very small, the driver may be able to pre-allocate that number of buffers (N * (64KB + Ethernet header + VLAN tag)) in driver. Upper stack will almost always generate more than 16 mbufs for TSO packets. When driver knows the length of mbuf chain of TSO packet is more than 16, you can copy the mbuf chain to the pre-allocated buffer. I recall I didn't implement TSO on txp(4) because the firmware of txp(4) controller does not support more than 16 fragment descriptors. From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 04:31:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3008626F; Tue, 8 Jul 2014 04:31:46 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E78702858; Tue, 8 Jul 2014 04:31:45 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 68E9D1FE028; Tue, 8 Jul 2014 06:31:43 +0200 (CEST) Message-ID: <53BB7433.2010306@selasky.org> Date: Tue, 08 Jul 2014 06:31:47 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters References: <53BA5657.8010309@selasky.org> <20140708021439.GA3965@michelle.fasterthan.com> In-Reply-To: <20140708021439.GA3965@michelle.fasterthan.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 04:31:46 -0000 On 07/08/14 04:14, Yonghyeon PYUN wrote: > On Mon, Jul 07, 2014 at 10:12:07AM +0200, Hans Petter Selasky wrote: >> Hi, >> >> I'm asking for some input on the attached m_dup() patch, so that >> existing functionality or dependencies are not broken. The background >> for the change is to allow m_dup() to defrag long mbuf chains that >> doesn't fit into a specific hardware's scatter gather entries, typically >> when doing TSO. >> >> In my case the HW limit is 16 entries of length 4K for doing a 64KByte > > I wonder how HW can handle a full-sized TSO packet(64KB + Ethernet > header + VLAN tag). > >> TSO packet. Currently m_dup() is at best producing 32 entries of each 2K >> for a 64Kbytes TSO packet. >> >> By allowing m_dup() to get JUMBO clusters when allocating mbufs, we >> avoid creating a new function, specific to the hardware, to defrag some >> rare-occurring very long mbuf chains into a mbuf chain below 16 entries. >> > > I think m_dup() was used to get a copy of writable mbuf chains. If > m_dup() starts to allocate jumbo mbufs it will eventually fail on > long running boxes. This will break firewall(ipfw divert, pf/ipf > dup-to) rules and several ethernet drivers. > > I don't know how many TSO requests could be queued by HW but if the > number is very small, the driver may be able to pre-allocate that > number of buffers (N * (64KB + Ethernet header + VLAN tag)) in > driver. Upper stack will almost always generate more than 16 mbufs > for TSO packets. When driver knows the length of mbuf chain of TSO > packet is more than 16, you can copy the mbuf chain to the > pre-allocated buffer. > > I recall I didn't implement TSO on txp(4) because the firmware of > txp(4) controller does not support more than 16 fragment > descriptors. Hi, Would it be better if my patch used the PAGE_SIZE clusters instead of the 16K ones? Then it should not be affected by memory defragmentation. Thanks for shedding some light into this area? --HPS From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 08:56:17 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60FD1172; Tue, 8 Jul 2014 08:56:17 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28FCA2B14; Tue, 8 Jul 2014 08:56:17 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id fp1so6864087pdb.11 for ; Tue, 08 Jul 2014 01:56:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:mime-version:subject:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Q4k3NbEB1g6W2YFBRhljXgOqAWhtzsVqsntre7gfKJ0=; b=FjiX53SKVpKO/SBJ6b3CkRxoE86+M6LP/w50jVjVvUCia38CpJPgESbZJtkWgc6gvb 5MtUZuwbsWHcgz/Nj474py5fG6/nCJY5LzZ23Z6V33gXgziYyW2QZiWiR8KvDrVlj9Xd 56AOKeELEw5qL5hbpPtHH0U1aB57OpOS1cOP4H+xxTMmEMsEx+rAww5R+ToOavRumOVX mzGoMHhQ7Ww7SLeSnVyoMIVm5FGKThlXscm0pEwLpUsRWUznwbvzkM9cM79n5WviH0fm BMCsR3SAQfDw05s3K1YgenesRYdZ8vPKhdbxnp63xGcMTNgYIbH9mTsWEYSuQ32NTapf hbfg== X-Received: by 10.66.119.39 with SMTP id kr7mr1984020pab.131.1404809776724; Tue, 08 Jul 2014 01:56:16 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:8045:a94a:de:b331? ([2601:8:ab80:7d6:8045:a94a:de:b331]) by mx.google.com with ESMTPSA id w16sm23469180pdl.36.2014.07.08.01.56.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 01:56:15 -0700 (PDT) From: Garrett Cooper X-Google-Original-From: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: A new way to test systems in multiple machine scenarios... In-Reply-To: Date: Tue, 8 Jul 2014 01:56:14 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1878.2) Cc: "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 08:56:17 -0000 On Jul 6, 2014, at 9:06 PM, Craig Rodrigues wrote: > On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil = > wrote: >=20 >> Hi, >>=20 >> I've coded up a system to allow you to control multiple other systems = for >> use in testing. >>=20 >> https://github.com/gvnn3/conductor >>=20 >>=20 > Cool! The architecture you have is similar to that of the SPECsfs > benchmark test ( http://www.spec.org/sfs2008/ ) > which involves a "coordinator node" and multiple "client nodes" which > direct NFS network > traffic towards a System Under Test (SUT). Garrett Cooper actually = set up > the original testbed > that I am using now at iXsystems. :) >=20 > It would be cool to put together tools like Jenkins, Kyua, and = conductor to > do more advanced testing > of FreeBSD before the project puts out releases. Agreed. The only thing that I have some concern about is the reinventing = of the wheel in python. multiprocessing Managers are one viable option = that=92s existed since python 2.6; there=92s a learning curve though, = and you=92ll run into problems with pickling some objects because the = pickle protocol is far from complete (example: = http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod= -when-using-pythons-multiprocessing-pool-ma ); you might run into this = problems regardless because you=92re serializing objects using pickle = instead of using dill (or using a simpler serialization method like = JSON). Fabric has a framework that=92s nice to use if you have ssh = capability. There are other frameworks that use twisted conch I think = too (another library that implements ssh access). Isilon has a framework they use, but it=92s very customized to their = infrastructure and product assumptions and it=92s in need of an overhaul = :(. -Garrett= From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 11:34:17 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AAA7C86; Tue, 8 Jul 2014 11:34:17 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23BF62838; Tue, 8 Jul 2014 11:34:17 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id ik5so5235448vcb.35 for ; Tue, 08 Jul 2014 04:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Cpgqfvy+eOW1xWLY6vvVz9fRr+x12KuzlqB9lucwcfM=; b=ejb/EZcYS/789HJPDdsyx/wdPEypEYdInI1hNNlrtf+ta0YEyKhiKDWqO3nq/3U+CS 5+ZaRkanOcM8Coexu7bBKRclBS3BXiJuP1HFIDRMtueJjoACPJlFQMGa8o3GSSSTzbn0 Wuoy4vJbuPT++pT58jcuOed7cc86KWP1iNiAUwZ7KOmcQscpPpCc7HpX6XeySLYt9lAq oXrW8KdWylFt2OqM0OUMcFHJ2r6IExQ6yW6ej1Uoqf2Xuxxwy8if2hGu7hksZVqvdqHU t3BlvfAK1rAtBqDihOiR+jJSK89La9xO2n4jeyAsyomTtDhlnCTL4LN86Ma1kL8HYDIQ HUyw== MIME-Version: 1.0 X-Received: by 10.53.7.204 with SMTP id de12mr963062vdd.41.1404819256010; Tue, 08 Jul 2014 04:34:16 -0700 (PDT) Sender: ndenev@gmail.com Received: by 10.220.136.79 with HTTP; Tue, 8 Jul 2014 04:34:15 -0700 (PDT) In-Reply-To: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> Date: Tue, 8 Jul 2014 12:34:15 +0100 X-Google-Sender-Auth: wcy4yYOB7tkSY-fVuFm9HWkVvhI Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Nikolay Denev To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Craig Rodrigues , "freebsd-testing@freebsd.org" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 11:34:17 -0000 On Tue, Jul 8, 2014 at 9:56 AM, Garrett Cooper wrote: > On Jul 6, 2014, at 9:06 PM, Craig Rodrigues wrote: > >> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil >> wrote: >> >>> Hi, >>> >>> I've coded up a system to allow you to control multiple other systems f= or >>> use in testing. >>> >>> https://github.com/gvnn3/conductor >>> >>> >> Cool! The architecture you have is similar to that of the SPECsfs >> benchmark test ( http://www.spec.org/sfs2008/ ) >> which involves a "coordinator node" and multiple "client nodes" which >> direct NFS network >> traffic towards a System Under Test (SUT). Garrett Cooper actually set = up >> the original testbed >> that I am using now at iXsystems. :) >> >> It would be cool to put together tools like Jenkins, Kyua, and conductor= to >> do more advanced testing >> of FreeBSD before the project puts out releases. > > Agreed. The only thing that I have some concern about is the reinventing = of the wheel in python. multiprocessing Managers are one viable option that= =E2=80=99s existed since python 2.6; there=E2=80=99s a learning curve thoug= h, and you=E2=80=99ll run into problems with pickling some objects because = the pickle protocol is far from complete (example: http://stackoverflow.com= /questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multi= processing-pool-ma ); you might run into this problems regardless because y= ou=E2=80=99re serializing objects using pickle instead of using dill (or us= ing a simpler serialization method like JSON). Fabric has a framework that= =E2=80=99s nice to use if you have ssh capability. There are other framewor= ks that use twisted conch I think too (another library that implements ssh = access). > > Isilon has a framework they use, but it=E2=80=99s very customized to thei= r infrastructure and product assumptions and it=E2=80=99s in need of an ove= rhaul :(. > > -Garrett > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" This also looks as an interesting option : https://codespeak.net/execnet/ I haven't used personally, but judging from py-test (it's the same author) it should be good :) --Nikolay From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 14:19:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F607BDB; Tue, 8 Jul 2014 14:19:46 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50200281E; Tue, 8 Jul 2014 14:19:46 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 43A161FE028; Tue, 8 Jul 2014 16:19:44 +0200 (CEST) Message-ID: <53BBFE08.1080804@selasky.org> Date: Tue, 08 Jul 2014 16:19:52 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters References: <53BA5657.8010309@selasky.org> <20140708021439.GA3965@michelle.fasterthan.com> <53BB7433.2010306@selasky.org> In-Reply-To: <53BB7433.2010306@selasky.org> Content-Type: multipart/mixed; boundary="------------060305060509090406020506" Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 14:19:46 -0000 This is a multi-part message in MIME format. --------------060305060509090406020506 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit > > Hi, > > Would it be better if my patch used the PAGE_SIZE clusters instead of > the 16K ones? Then it should not be affected by memory defragmentation. > Thanks for shedding some light into this area? > > --HPS > Hi, Updated patch attached. --HPS --------------060305060509090406020506 Content-Type: text/x-diff; name="uipc_mbuf.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="uipc_mbuf.c.diff" === sys/kern/uipc_mbuf.c ================================================================== --- sys/kern/uipc_mbuf.c (revision 268358) +++ sys/kern/uipc_mbuf.c (local) @@ -917,7 +917,15 @@ struct mbuf *n; /* Get the next new mbuf */ - if (remain >= MINCLSIZE) { + if (remain >= MJUMPAGESIZE) { + /* + * By allocating a bigger mbuf, we get fewer + * scatter gather entries for the hardware to + * process: + */ + n = m_getjcl(how, m->m_type, 0, MJUMPAGESIZE); + nsize = MJUMPAGESIZE; + } else if (remain >= MINCLSIZE) { n = m_getcl(how, m->m_type, 0); nsize = MCLBYTES; } else { --------------060305060509090406020506-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 17:46:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 640B5D8; Tue, 8 Jul 2014 17:46:22 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0335F2D73; Tue, 8 Jul 2014 17:46:21 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id DFD931FE028; Tue, 8 Jul 2014 19:46:18 +0200 (CEST) Message-ID: <53BC2E73.6090700@selasky.org> Date: Tue, 08 Jul 2014 19:46:27 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, FreeBSD Current Subject: [RFC] Add support for changing the flow ID of TCP connections Content-Type: multipart/mixed; boundary="------------030903070908060508030601" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 17:46:22 -0000 This is a multi-part message in MIME format. --------------030903070908060508030601 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm working on a new feature which will allow TCP connections to be timing controlled by the ethernet hardware driver, actually the mlxen driver. The main missing piece in the kernel is to allow the mbuf's flowid value to be overwritten in "struct inpcb" once the connection is established and to have a callback once the TCP connection is gone so that the assigned "flowid" can be freed by the ethernet hardware driver. The "flowid" will be used to assign the outgoing data traffic of a specific TCP connections to a hardware controlled queue, which in advance contain certain parameters about the timing for the transmitted packets. To be able to set the flowid I'm using existing functions in the kernel TCP code to lookup the "inpcb" structure based on the 4-tuple, via the "ifp->if_ioctl()" callback of the network adapter. I'm also registering a function method table so that I get a callback when the TCP connection is gone. A this point of development I would like to get some feedback from FreeBSD network guys about my attached patch proposal. The motivation for this work is to have a more reliable TCP transmissions typically for fixed-rate media content going some distance. To illustrate this I will give you an example from the world of VoIP, which is using UDP. When doing long-distance VoIP calls through various unknown networks and routers it makes a very big difference if you are sending data 20ms apart or 40ms apart, even at the exact same rate. In the one case you might experience a bunch of packet drops, and in the other case, everything is fine. Why? Because the number of packets you send per second, and the timing is important. The goal is to apply some timing rules for TCP, to increase the factor of successful transmission, and to reduce the amount of data loss. For high throughput applications we want to do this by means of hardware. While at it I would like to "typedef" the flowid used by mbufs, "struct inpcb" and many more places. Where would the right place be to put such a definition? In "sys/mbuf.h"? Comments are appreciated! --HPS --------------030903070908060508030601 Content-Type: text/x-diff; name="tcp_ratecontrol.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tcp_ratecontrol.diff" === sys/netinet/in_pcb.c ================================================================== --- sys/netinet/in_pcb.c (revision 268358) +++ sys/netinet/in_pcb.c (local) @@ -1173,6 +1173,100 @@ } /* + * in_pcb_handle_ratectlreq - this function sets the hardware flow ID + * for a given IPv4 connection based on the input arguments. + * + * Return values: + * 0: Success + * Non-zero: Failure + */ +int +in_pcb_handle_ratectlreq(struct ifnet *ifp, struct in_ratectlreq *req, + const struct in_flowid_methods *mtod, void *arg) +{ + struct inpcb *inp; + int error; + + if (ifp == NULL || req == NULL || mtod == NULL || + mtod->inf_alloc == NULL || mtod->inf_rateset == NULL || + mtod->inf_free == NULL) + return (EINVAL); + + inp = in_pcblookup(&V_tcbinfo, + req->ifreq_dst.sin_addr, req->ifreq_dst.sin_port, + req->ifreq_src.sin_addr, req->ifreq_src.sin_port, + INPLOOKUP_WLOCKPCB, ifp); + if (inp == NULL) + return (ENOENT); + + INP_WLOCK_ASSERT(inp); + + if (inp->inp_flowid_mtod == NULL) { + error = mtod->inf_alloc(arg, &inp->inp_flowid); + if (error != 0) + goto done; + inp->inp_flowid_mtod = mtod; + inp->inp_flowid_arg = arg; + /* ensure that the flow ID is not overwritten */ + inp->inp_flags |= INP_HW_FLOWID; + inp->inp_flags &= ~INP_SW_FLOWID; + inp->inp_flowtype = M_HASHTYPE_NONE; + } + error = inp->inp_flowid_mtod->inf_rateset(inp->inp_flowid_arg, + inp->inp_flowid, req->ifreq_baudrate); +done: + INP_WUNLOCK(inp); + return (error); +} + +/* + * in6_pcb_handle_ratectlreq - this function sets the hardware flow ID + * for a given IPv6 connection based on the input arguments. + * + * Return values: + * 0: Success + * Non-zero: Failure + */ +int +in6_pcb_handle_ratectlreq(struct ifnet *ifp, struct in6_ratectlreq *req, + const struct in_flowid_methods *mtod, void *arg) +{ + struct inpcb *inp; + int error; + + if (ifp == NULL || req == NULL || mtod == NULL || + mtod->inf_alloc == NULL || mtod->inf_rateset == NULL || + mtod->inf_free == NULL) + return (EINVAL); + + inp = in6_pcblookup(&V_tcbinfo, + &req->ifreq_dst.sin6_addr, req->ifreq_dst.sin6_port, + &req->ifreq_src.sin6_addr, req->ifreq_src.sin6_port, + INPLOOKUP_WLOCKPCB, ifp); + if (inp == NULL) + return (ENOENT); + + INP_WLOCK_ASSERT(inp); + + if (inp->inp_flowid_mtod == NULL) { + error = mtod->inf_alloc(arg, &inp->inp_flowid); + if (error != 0) + goto done; + inp->inp_flowid_mtod = mtod; + inp->inp_flowid_arg = arg; + /* ensure that the flow ID is not overwritten */ + inp->inp_flags |= INP_HW_FLOWID; + inp->inp_flags &= ~INP_SW_FLOWID; + inp->inp_flowtype = M_HASHTYPE_NONE; + } + error = inp->inp_flowid_mtod->inf_rateset(inp->inp_flowid_arg, + inp->inp_flowid, req->ifreq_baudrate); +done: + INP_WUNLOCK(inp); + return (error); +} + +/* * Unconditionally schedule an inpcb to be freed by decrementing its * reference count, which should occur only after the inpcb has been detached * from its socket. If another thread holds a temporary reference (acquired @@ -1192,6 +1286,12 @@ INP_WLOCK_ASSERT(inp); /* XXXRW: Do as much as possible here. */ + + /* Release flow ID, if any */ + if (inp->inp_flowid_mtod != NULL) { + inp->inp_flowid_mtod->inf_free( + inp->inp_flowid_arg, inp->inp_flowid); + } #ifdef IPSEC if (inp->inp_sp != NULL) ipsec_delete_pcbpolicy(inp); === sys/netinet/in_pcb.h ================================================================== --- sys/netinet/in_pcb.h (revision 268358) +++ sys/netinet/in_pcb.h (local) @@ -39,6 +39,7 @@ #define _NETINET_IN_PCB_H_ #include +#include #include #include #include @@ -127,6 +128,23 @@ struct icmp6_filter; +/* + * The following functions must be non-blocking, because they are + * executing in the fast-path: + */ +typedef int (in_flowid_alloc_t)(void *, m_flowid_t *); +typedef int (in_flowid_rateset_t)(void *, m_flowid_t, uint64_t); +typedef void (in_flowid_free_t)(void *, m_flowid_t); + +struct in_ratectlreq; +struct in6_ratectlreq; + +struct in_flowid_methods { + in_flowid_alloc_t *inf_alloc; + in_flowid_rateset_t *inf_rateset; + in_flowid_free_t *inf_free; +}; + /*- * struct inpcb captures the network layer state for TCP, UDP, and raw IPv4 * and IPv6 sockets. In the case of TCP, further per-connection state is @@ -177,7 +195,9 @@ u_char inp_ip_ttl; /* (i) time to live proto */ u_char inp_ip_p; /* (c) protocol proto */ u_char inp_ip_minttl; /* (i) minimum TTL or drop */ - uint32_t inp_flowid; /* (x) flow id / queue id */ + m_flowid_t inp_flowid; /* (x) flow ID / queue ID */ + const struct in_flowid_methods *inp_flowid_mtod; /* (x) flow ID callback methods */ + void *inp_flowid_arg; /* (x) argument for flow ID methods */ u_int inp_refcount; /* (i) refcount */ void *inp_pspare[5]; /* (x) route caching / general use */ uint32_t inp_flowtype; /* (x) M_HASHTYPE value */ @@ -635,6 +655,10 @@ void in_pcbdisconnect(struct inpcb *); void in_pcbdrop(struct inpcb *); void in_pcbfree(struct inpcb *); +int in_pcb_handle_ratectlreq(struct ifnet *, struct in_ratectlreq *, + const struct in_flowid_methods *, void *); +int in6_pcb_handle_ratectlreq(struct ifnet *, struct in6_ratectlreq *, + const struct in_flowid_methods *, void *); int in_pcbinshash(struct inpcb *); int in_pcbinshash_nopcbgroup(struct inpcb *); int in_pcbladdr(struct inpcb *, struct in_addr *, struct in_addr *, === sys/netinet/in_var.h ================================================================== --- sys/netinet/in_var.h (revision 268358) +++ sys/netinet/in_var.h (local) @@ -81,6 +81,14 @@ struct sockaddr_in ifra_mask; int ifra_vhid; }; + +struct in_ratectlreq { + char ifreq_name[IFNAMSIZ]; + struct sockaddr_in ifreq_dst; + struct sockaddr_in ifreq_src; + uint64_t ifreq_baudrate; /* bits per second */ +}; + /* * Given a pointer to an in_ifaddr (ifaddr), * return a pointer to the addr as a sockaddr_in. === sys/netinet6/in6_var.h ================================================================== --- sys/netinet6/in6_var.h (revision 268358) +++ sys/netinet6/in6_var.h (local) @@ -307,6 +307,13 @@ struct in6_addrlifetime ifra_lifetime; }; +struct in6_ratectlreq { + char ifreq_name[IFNAMSIZ]; + struct sockaddr_in6 ifreq_dst; + struct sockaddr_in6 ifreq_src; + uint64_t ifreq_baudrate; /* bits per second */ +}; + /* prefix type macro */ #define IN6_PREFIX_ND 1 #define IN6_PREFIX_RR 2 @@ -435,6 +442,7 @@ #define SIOCDIFADDR_IN6 _IOW('i', 25, struct in6_ifreq) #define OSIOCAIFADDR_IN6 _IOW('i', 26, struct oin6_aliasreq) #define SIOCAIFADDR_IN6 _IOW('i', 27, struct in6_aliasreq) +#define SIOCSRATECTL_IN6 _IOW('i',110, struct in6_ratectlreq) #define SIOCSIFPHYADDR_IN6 _IOW('i', 70, struct in6_aliasreq) #define SIOCGIFPSRCADDR_IN6 _IOWR('i', 71, struct in6_ifreq) === sys/sys/mbuf.h ================================================================== --- sys/sys/mbuf.h (revision 268358) +++ sys/sys/mbuf.h (local) @@ -114,6 +114,8 @@ void (*m_tag_free)(struct m_tag *); }; +typedef uint32_t m_flowid_t; + /* * Record/packet header in first mbuf of chain; valid only if M_PKTHDR is set. * Size ILP32: 48 @@ -125,7 +127,7 @@ int32_t len; /* total packet length */ /* Layer crossing persistent information. */ - uint32_t flowid; /* packet's 4-tuple system */ + m_flowid_t flowid; /* packet's 4-tuple system */ uint64_t csum_flags; /* checksum and offload features */ uint16_t fibnum; /* this packet should use this fib */ uint8_t cosqos; /* class/quality of service */ === sys/sys/sockio.h ================================================================== --- sys/sys/sockio.h (revision 268358) +++ sys/sys/sockio.h (local) @@ -128,4 +128,6 @@ #define SIOCDIFGROUP _IOW('i', 137, struct ifgroupreq) /* delete ifgroup */ #define SIOCGIFGMEMB _IOWR('i', 138, struct ifgroupreq) /* get members */ +#define SIOCSRATECTL _IOW('i', 139, struct in_ratectlreq) + #endif /* !_SYS_SOCKIO_H_ */ --------------030903070908060508030601-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 19:17:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D8DCB3A; Tue, 8 Jul 2014 19:17:05 +0000 (UTC) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0EF326B9; Tue, 8 Jul 2014 19:17:04 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id rd3so7849458pab.32 for ; Tue, 08 Jul 2014 12:17:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=S7PLf0ocR4nPH4KlKgz1VM/lnTgZ/DmKEPy4DdxrYFE=; b=NXS9uEg03+okws7yonBHA+E7AMHWdffH8uz7WHlh1Wata0ScpuAPhMvkMMSngp4hW8 m3i1+3ij/0NO0+XYwJg97+X3JQ7E7olnwd6QQYxcWOExLf7Nx8unWjQnX7W6RDKOAicN nlU8oqZxqKZPUN/rglHjqAM3q2k3FJPUfB8CduOEEYLfB7RvgsaX35oRE8yoHI4XNrgR 3kM4qs27/ijB/P1gmgiH5vcxtW3OKXUZwifOmyUrVZpBIR3r3sMBHdg9luNEpnDNCNSJ lC7TJbdBiCCgQSGTmMo7gCuVMCXTsx1NgT4+MyD7WKU+IlebpJM7DnSmKrHy6xMQhKu7 SjOA== X-Received: by 10.68.176.5 with SMTP id ce5mr26991955pbc.93.1404847024115; Tue, 08 Jul 2014 12:17:04 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id b3sm14354450pbu.8.2014.07.08.12.17.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 12:17:03 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <53BC43AE.3040409@FreeBSD.org> Date: Tue, 08 Jul 2014 12:17:02 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Hans Petter Selasky , freebsd-net@freebsd.org, FreeBSD Current Subject: Re: [RFC] Add support for changing the flow ID of TCP connections References: <53BC2E73.6090700@selasky.org> In-Reply-To: <53BC2E73.6090700@selasky.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 19:17:05 -0000 On 07/08/14 10:46, Hans Petter Selasky wrote: > Hi, > > I'm working on a new feature which will allow TCP connections to be > timing controlled by the ethernet hardware driver, actually the mlxen > driver. The main missing piece in the kernel is to allow the mbuf's > flowid value to be overwritten in "struct inpcb" once the connection is > established and to have a callback once the TCP connection is gone so > that the assigned "flowid" can be freed by the ethernet hardware driver. > > The "flowid" will be used to assign the outgoing data traffic of a > specific TCP connections to a hardware controlled queue, which in > advance contain certain parameters about the timing for the transmitted > packets. > > To be able to set the flowid I'm using existing functions in the kernel > TCP code to lookup the "inpcb" structure based on the 4-tuple, via the > "ifp->if_ioctl()" callback of the network adapter. I'm also registering > a function method table so that I get a callback when the TCP connection > is gone. > > A this point of development I would like to get some feedback from > FreeBSD network guys about my attached patch proposal. > > The motivation for this work is to have a more reliable TCP > transmissions typically for fixed-rate media content going some > distance. To illustrate this I will give you an example from the world > of VoIP, which is using UDP. When doing long-distance VoIP calls through > various unknown networks and routers it makes a very big difference if > you are sending data 20ms apart or 40ms apart, even at the exact same > rate. In the one case you might experience a bunch of packet drops, and > in the other case, everything is fine. Why? Because the number of > packets you send per second, and the timing is important. The goal is to > apply some timing rules for TCP, to increase the factor of successful > transmission, and to reduce the amount of data loss. For high throughput > applications we want to do this by means of hardware. > > > While at it I would like to "typedef" the flowid used by mbufs, "struct > inpcb" and many more places. Where would the right place be to put such > a definition? In "sys/mbuf.h"? > > > Comments are appreciated! I think we need to design this to be as generic as possible. I have quite a bit of code that does this stuff but I haven't pushed it upstream or even offered it for review (yet). cxgbe(4) hardware does throttling and traffic pacing too, but it's not limited to TCP, and it can do it per queue or per "flow" -- you can limit a tx queue or an individual flow to a packet-per-second limit or a bandwidth ceiling; this works for both plain NIC (TCP, UDP, whatever), as well as stateful TCP offload). For TCP (NIC or TOE) the chip can even rewrite the TCP timestamp to account for the extra time that the chip/driver held the packet because it was asked to slow down a flow. The per queue stuff is handled via a driver-specific tool (cxgbetool). For per-flow throttling my implementation adds a new sockopt (SO_TX_THROTTLE) that lets an application specify a throttle rate for a socket. The kernel allocates a "flow identifier" for each such socket and tcp_output (or udp_output, ..) will attach an mbuf tag containing this identifier and throttling parameters to each mbuf that it pushes out. Drivers for hardware that can throttle traffic look for this tag, the rest ignore it. - cxgbe(4) registers itself as a "flow throttling provider" with the kernel when it attaches to the chip. It tells the kernel how many flows it can handle and the range of rates it can handle. - setsockopt(SO_TX_THROTTLE, rate) makes the kernel allocate a unique identifier for the socket. This is *not* related to the RSS flowid at all. If a listening socket has SO_TX_THROTTLE, all its children will inherit the rate limiting parameters but will each get its own unique identifier. The setsockopt fails if there aren't any flow throttling providers registered, - tcp_output (and other proto_output) routines look for SO_TX_THROTTLE and attach extra metadata, in the form of a tag, to the outgoing frames. - cxgbe(4) reads this metadata and acts on it. Regards, Navdeep From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 19:19:18 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10FC9CFC; Tue, 8 Jul 2014 19:19:18 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6E2126E4; Tue, 8 Jul 2014 19:19:17 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id w7so5865225qcr.30 for ; Tue, 08 Jul 2014 12:19:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=v+r3P4E8Prmo9qRRmEOJC9b01kFHTUfw9VFSMGH2qX0=; b=WFQ1BMjcNHV9TilCwPLAqwxfIW7TofmUdV3L6+EWbbspyVmxz3uSzBW0uUzFYQvuAr 3f6vBtocgGRBPG6CAnh9lys92w2qfAyX/RTMNf+wyHT4qyEb5EoI9QzBMR6yikkguzjP Z7V2Nxz7ydZmIYi9JoNDl7k3+IbwOtc7EMfj3EnD9rBPIN0d5ajcD5Ir3i4KSa0gfu/k QWjJg0GRUxpkwVIRnDwwVW8XrK9lYzNbgHJnQ/YJFL8JTw6g109vxML4WydFtbMAWIAc VGJPdKNHprZu+g4KjzgfPDMiByXNubgfL6kCeiG5dzRf9GzdUv2PA4jLIHVibitHE9vl GKRA== MIME-Version: 1.0 X-Received: by 10.140.89.18 with SMTP id u18mr17597375qgd.87.1404847156741; Tue, 08 Jul 2014 12:19:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 8 Jul 2014 12:19:16 -0700 (PDT) In-Reply-To: <53BC2E73.6090700@selasky.org> References: <53BC2E73.6090700@selasky.org> Date: Tue, 8 Jul 2014 12:19:16 -0700 X-Google-Sender-Auth: O-NZrFR1yuJajjhnPLJr-a6qfDE Message-ID: Subject: Re: [RFC] Add support for changing the flow ID of TCP connections From: Adrian Chadd To: Hans Petter Selasky Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , FreeBSD Current X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 19:19:18 -0000 Hi! The flowid value has way, way too many possible meanings but it's always been a mostly-static value. I'm worried about overriding it with multiple meanings that cause features to not work at all together. So I'd rather leave the flowid/flowtype as it currently is so it doesn't upset packet reordering and can be used by things like RSS for scaling, and instead introduce a new connection ID to be used for your purpose. That way the existing use of flowid for packet ordering and flowid/flowtype for doing network scaling and netisr selection can work together with your connection id requirements. Having stack support for hardware/firmware packet scheduling is cool. It seems to somewhat overlap with other parts of the TCP offload though and I'm concerned about bloating out inpcb by 3 pointers for each connection where lots of connections on the same NIC will point to the same function set or NULL. I'd hit up what others in this space are doing. There's pacing support in the chelsio NIC for example and I'm not sure what Navdeep's plans are for that in upstream FreeBSD. Other than that, cool! -a On 8 July 2014 10:46, Hans Petter Selasky wrote: > Hi, > > I'm working on a new feature which will allow TCP connections to be timing > controlled by the ethernet hardware driver, actually the mlxen driver. The > main missing piece in the kernel is to allow the mbuf's flowid value to be > overwritten in "struct inpcb" once the connection is established and to have > a callback once the TCP connection is gone so that the assigned "flowid" can > be freed by the ethernet hardware driver. > > The "flowid" will be used to assign the outgoing data traffic of a specific > TCP connections to a hardware controlled queue, which in advance contain > certain parameters about the timing for the transmitted packets. > > To be able to set the flowid I'm using existing functions in the kernel TCP > code to lookup the "inpcb" structure based on the 4-tuple, via the > "ifp->if_ioctl()" callback of the network adapter. I'm also registering a > function method table so that I get a callback when the TCP connection is > gone. > > A this point of development I would like to get some feedback from FreeBSD > network guys about my attached patch proposal. > > The motivation for this work is to have a more reliable TCP transmissions > typically for fixed-rate media content going some distance. To illustrate > this I will give you an example from the world of VoIP, which is using UDP. > When doing long-distance VoIP calls through various unknown networks and > routers it makes a very big difference if you are sending data 20ms apart or > 40ms apart, even at the exact same rate. In the one case you might > experience a bunch of packet drops, and in the other case, everything is > fine. Why? Because the number of packets you send per second, and the timing > is important. The goal is to apply some timing rules for TCP, to increase > the factor of successful transmission, and to reduce the amount of data > loss. For high throughput applications we want to do this by means of > hardware. > > > While at it I would like to "typedef" the flowid used by mbufs, "struct > inpcb" and many more places. Where would the right place be to put such a > definition? In "sys/mbuf.h"? > > > Comments are appreciated! > > --HPS > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 19:39:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 854821BA; Tue, 8 Jul 2014 19:39:54 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3AB1F28C5; Tue, 8 Jul 2014 19:39:53 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEANLzu1ODaFve/2dsb2JhbABZg2Bagm+8DwiGblMBgSt1hAMBAQEEAQEBICsgCxsYAgINGQIpAQkmBggHBAEcBIghDa9qmTcXgSyNRQEBGzQHgneBTAWYCoQ0igSIQINfITWBBTk X-IronPort-AV: E=Sophos;i="5.01,626,1400040000"; d="scan'208";a="139252391" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 08 Jul 2014 15:39:46 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9AFEEB403D; Tue, 8 Jul 2014 15:39:46 -0400 (EDT) Date: Tue, 8 Jul 2014 15:39:46 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Message-ID: <1758058976.8852466.1404848386625.JavaMail.root@uoguelph.ca> In-Reply-To: <53BBFE08.1080804@selasky.org> Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: pyunyh@gmail.com, freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 19:39:54 -0000 Hans Petter Selasky wrote: > > > > Hi, > > > > Would it be better if my patch used the PAGE_SIZE clusters instead > > of > > the 16K ones? Then it should not be affected by memory > > defragmentation. > > Thanks for shedding some light into this area? > > Well, I ran into the threads stuck on "btalloc" when I used PAGE_SIZE clusters mixed with MCLBYTES clusters and from what I could figure, it was a kernel address space fragmentation issue. I would guess that PAGE_SIZE clusters aren't as bad as 16K clusters w.r.t. fragmentation, but I believe that they could still be an issue. (My testing was on a 256Mbyte i386, so I can't say if amd64 systems will have a problem, just that small 32bit arches will.) rick > > --HPS > > > > Hi, > > Updated patch attached. > > --HPS > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Jul 8 20:10:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E560487F; Tue, 8 Jul 2014 20:10:08 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 9BD7E2C0B; Tue, 8 Jul 2014 20:10:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAG5PvFODaFve/2dsb2JhbABWA4NgWoJvvBUIhm5TAYEwdYQDAQEBBAEBASArIAsbGAICDRkCKQEJJgYIBwQBHASIIQ2uPZkaF4EsiE6EcQYBARskEAcRgmaBTAWYCoQ0igSIQINfITV9CBci X-IronPort-AV: E=Sophos;i="5.01,626,1400040000"; d="scan'208";a="139260753" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 08 Jul 2014 16:10:07 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 26EEFB409B; Tue, 8 Jul 2014 16:10:07 -0400 (EDT) Date: Tue, 8 Jul 2014 16:10:07 -0400 (EDT) From: Rick Macklem To: John-Mark Gurney Message-ID: <1065824414.8871880.1404850207148.JavaMail.root@uoguelph.ca> In-Reply-To: <20140708005234.GJ45513@funkthat.com> Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: Hans Petter Selasky , freebsd-net@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:10:09 -0000 John-Mark Gurney wrote: > Hans Petter Selasky wrote this message on Mon, Jul 07, 2014 at 10:12 > +0200: > > I'm asking for some input on the attached m_dup() patch, so that > > existing functionality or dependencies are not broken. The > > background > > for the change is to allow m_dup() to defrag long mbuf chains that > > doesn't fit into a specific hardware's scatter gather entries, > > typically > > when doing TSO. > > > > In my case the HW limit is 16 entries of length 4K for doing a > > 64KByte > > TSO packet. Currently m_dup() is at best producing 32 entries of > > each 2K > > for a 64Kbytes TSO packet. > > > > By allowing m_dup() to get JUMBO clusters when allocating mbufs, we > > avoid creating a new function, specific to the hardware, to defrag > > some > > rare-occurring very long mbuf chains into a mbuf chain below 16 > > entries. > > Please no... Until we get a better allocator, we should not use jumbo > (>page sized) mbufs otherwise we will quickly fail to allocate mbufs > after a machine has been up for a long while causing other > failures... > > Unless of course if the code fails to allocate the largest cluster it > falls through to trying to allocate the next smaller size, that might > be better... > Unfortunately, for the "can't allocate boundary tags" case, the allocation request with M_NOWAIT loops instead of failing. I tried: m = m_getjcl(M_NOWAIT..M_JUMPAGESIZE); if (m == NULL) m = getjcl(M_WAITOK..MCLBYTES); when I was experimenting with MJUMPAGESIZE clusters for NFS and what happened was the thread looped in the first m_getjcl() instead of returning NULL. It is about 12 layers of function calls deep and most fail/return NULL, but somewhere one of them decides to "try again". I didn't locate the location of that and don't know if it would be safe to change it so that m_getjcl() returns NULL for this case. rick > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 04:24:11 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99340180; Wed, 9 Jul 2014 04:24:11 +0000 (UTC) Received: from stargate.chelsio.com (99-65-72-228.uvs.sntcca.sbcglobal.net [99.65.72.228]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7047F2D61; Wed, 9 Jul 2014 04:24:10 +0000 (UTC) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id s694OAc3029609; Tue, 8 Jul 2014 21:24:10 -0700 Received: from dwarf.asicdesigners.com (10.192.166.0) by nice.asicdesigners.com (10.192.160.7) with Microsoft SMTP Server (TLS) id 14.2.247.3; Tue, 8 Jul 2014 21:24:09 -0700 Date: Tue, 8 Jul 2014 21:24:06 -0700 From: Navdeep Parhar To: Gleb Smirnoff , Alan Somers Subject: Re: ifaddr refcount problem Message-ID: <20140709042406.GA64350@dwarf.asicdesigners.com> References: <53A48849.8080504@chelsio.com> <20140623085229.GQ28199@FreeBSD.org> <20140624090847.GB28199@glebius.int.ru> <20140625100835.GK28199@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20140625100835.GK28199@glebius.int.ru> User-Agent: Mutt/1.5.23 (2014-03-12) X-Originating-IP: [10.192.166.0] Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 04:24:11 -0000 I got distracted by some other issues and lost track of this thread. Can one of you please commit the fix if the discussion has reached a conclusion? Thanks! Regards, Navdeep On Wed, Jun 25, 2014 at 02:08:35PM +0400, Gleb Smirnoff wrote: > Alan, > > On Tue, Jun 24, 2014 at 08:43:40AM -0600, Alan Somers wrote: > A> That looks better. But I think there is one more possibility for a > A> leak. For multicast packets, IFP_TO_IA at line 263 will call > A> ifa_ref(), unless the the interface has no address assigned. How > A> about this patch instead? Also, not tested. > > Again you are right, thanks. > > The patch needs to reset have_ia_ref in the 'goto again' case. > > Also, I'd suggest a tiny change to your patch so that we don't > have a broken logic reading the code as "I have ia reference, > but don't have ia". > > -- > Totus tuus, Glebius. > Index: ip_output.c > =================================================================== > --- ip_output.c (revision 267536) > +++ ip_output.c (working copy) > @@ -136,6 +136,7 @@ ip_output(struct mbuf *m, struct mbuf *opt, struct > struct rtentry *rte; /* cache for ro->ro_rt */ > struct in_addr odst; > struct m_tag *fwd_tag = NULL; > + int have_ia_ref; > #ifdef IPSEC > int no_route_but_check_spd = 0; > #endif > @@ -202,6 +203,7 @@ ip_output(struct mbuf *m, struct mbuf *opt, struct > gw = dst = (struct sockaddr_in *)&ro->ro_dst; > again: > ia = NULL; > + have_ia_ref = 0; > /* > * If there is a cached route, check that it is to the same > * destination and is still up. If not, free it and try again. > @@ -238,6 +240,7 @@ again: > error = ENETUNREACH; > goto bad; > } > + have_ia_ref = 1; > ip->ip_dst.s_addr = INADDR_BROADCAST; > dst->sin_addr = ip->ip_dst; > ifp = ia->ia_ifp; > @@ -250,6 +253,7 @@ again: > error = ENETUNREACH; > goto bad; > } > + have_ia_ref = 1; > ifp = ia->ia_ifp; > ip->ip_ttl = 1; > isbroadcast = in_broadcast(dst->sin_addr, ifp); > @@ -261,6 +265,8 @@ again: > */ > ifp = imo->imo_multicast_ifp; > IFP_TO_IA(ifp, ia); > + if (ia) > + have_ia_ref = 1; > isbroadcast = 0; /* fool gcc */ > } else { > /* > @@ -552,8 +558,11 @@ sendit: > #endif > error = netisr_queue(NETISR_IP, m); > goto done; > - } else > + } else { > + if (have_ia_ref) > + ifa_free(&ia->ifa); > goto again; /* Redo the routing table lookup. */ > + } > } > > /* See if local, if yes, send it to netisr with IP_FASTFWD_OURS. */ > @@ -582,6 +591,8 @@ sendit: > m->m_flags |= M_SKIP_FIREWALL; > m->m_flags &= ~M_IP_NEXTHOP; > m_tag_delete(m, fwd_tag); > + if (have_ia_ref) > + ifa_free(&ia->ifa); > goto again; > } > > @@ -694,6 +705,8 @@ passout: > done: > if (ro == &iproute) > RO_RTFREE(ro); > + if (have_ia_ref) > + ifa_free(&ia->ifa); > return (error); > bad: > m_freem(m); From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 05:01:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B300A21 for ; Wed, 9 Jul 2014 05:01:30 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4D852FFA for ; Wed, 9 Jul 2014 05:01:29 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id a108so5864019qge.4 for ; Tue, 08 Jul 2014 22:01:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=a2AtEN5CLDeAbMdja+5oJCEaO6ccQM8uXd9kBDGKwFg=; b=AL74gXMKw+2kEXDKayN4mS2sO1Hrc0YTM7pD30vPla4Pq4HIZEeYCXGPwaQmlUuTFF eR5vpW33vqMI0s9lD7DmiLKX1p5/G4Co/KCXeOwMPLQncoKmKpM8OrZULVbK815X7Elv roueVTZcaZv8IZvH6MRhXo9xvtInLa4FFxsClK1TthmCxtk/Ur6spFLQFdgnJpVfnu72 We08zAoq2HOmdxVonPkt7Up6GFIE1RbjdyFTg24go2STtPvIZxksizYXjrt7nxKXgKAO Yn9WkmxqFA5xx7MnRGB0F0RR41UdPOJbq5Q89PuUs4d8ZCND2V9FplEEpnZoDH8NvWEI q9TQ== MIME-Version: 1.0 X-Received: by 10.224.167.7 with SMTP id o7mr66160051qay.53.1404882088849; Tue, 08 Jul 2014 22:01:28 -0700 (PDT) Received: by 10.224.7.5 with HTTP; Tue, 8 Jul 2014 22:01:28 -0700 (PDT) Date: Wed, 9 Jul 2014 13:01:28 +0800 Message-ID: Subject: A question on modifying packet. From: Niu Zhixiong To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 05:01:30 -0000 Hi, all I have only one NIC. I want to capture packets from one certain ip address and change the both src and dst addresses and forward to other destination via the same NIC. It is possible? Is there any library to help me do this? Regards, Niu Zhixiong =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF= =BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D kaiaixi@gmail.com From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 14:36:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88FDD9A; Wed, 9 Jul 2014 14:36:53 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4670B2507; Wed, 9 Jul 2014 14:36:53 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 36C4C1FE028; Wed, 9 Jul 2014 16:36:45 +0200 (CEST) Message-ID: <53BD5385.4090208@selasky.org> Date: Wed, 09 Jul 2014 16:36:53 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Navdeep Parhar , freebsd-net@freebsd.org, FreeBSD Current Subject: Re: [RFC] Add support for changing the flow ID of TCP connections References: <53BC2E73.6090700@selasky.org> <53BC43AE.3040409@FreeBSD.org> In-Reply-To: <53BC43AE.3040409@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 14:36:53 -0000 On 07/08/14 21:17, Navdeep Parhar wrote: > On 07/08/14 10:46, Hans Petter Selasky wrote: >> Hi, >> >> I'm working on a new feature which will allow TCP connections to be >> timing controlled by the ethernet hardware driver, actually the mlxen >> driver. The main missing piece in the kernel is to allow the mbuf's >> flowid value to be overwritten in "struct inpcb" once the connection is >> established and to have a callback once the TCP connection is gone so >> that the assigned "flowid" can be freed by the ethernet hardware driver. >> >> The "flowid" will be used to assign the outgoing data traffic of a >> specific TCP connections to a hardware controlled queue, which in >> advance contain certain parameters about the timing for the transmitted >> packets. >> >> To be able to set the flowid I'm using existing functions in the kernel >> TCP code to lookup the "inpcb" structure based on the 4-tuple, via the >> "ifp->if_ioctl()" callback of the network adapter. I'm also registering >> a function method table so that I get a callback when the TCP connection >> is gone. >> >> A this point of development I would like to get some feedback from >> FreeBSD network guys about my attached patch proposal. >> >> The motivation for this work is to have a more reliable TCP >> transmissions typically for fixed-rate media content going some >> distance. To illustrate this I will give you an example from the world >> of VoIP, which is using UDP. When doing long-distance VoIP calls through >> various unknown networks and routers it makes a very big difference if >> you are sending data 20ms apart or 40ms apart, even at the exact same >> rate. In the one case you might experience a bunch of packet drops, and >> in the other case, everything is fine. Why? Because the number of >> packets you send per second, and the timing is important. The goal is to >> apply some timing rules for TCP, to increase the factor of successful >> transmission, and to reduce the amount of data loss. For high throughput >> applications we want to do this by means of hardware. >> >> >> While at it I would like to "typedef" the flowid used by mbufs, "struct >> inpcb" and many more places. Where would the right place be to put such >> a definition? In "sys/mbuf.h"? >> >> >> Comments are appreciated! > > I think we need to design this to be as generic as possible. I have > quite a bit of code that does this stuff but I haven't pushed it > upstream or even offered it for review (yet). > Hi, When will the non hardware related patches be available for review? I understand there are multiple ways to reach the same goal, and I think it would be great if we could agree on a common API for applications. --HPS From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 16:31:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60690EFA; Wed, 9 Jul 2014 16:31:54 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E3FE20ED; Wed, 9 Jul 2014 16:31:54 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kq14so9365768pab.6 for ; Wed, 09 Jul 2014 09:31:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=AZ1R+HH2AMz2BYa6PfiTUQb4KUw21h7O4OCsUryvgwg=; b=bJ1VmnYoaH1t/xIghWBMTmqt0ZExVk893i8fQgKNIQ7D//CdKbw7uaDoGVE/wqtpn2 SjGb3gcO+WcccP8u3IvOFc3okgWSIwihBFkmA02RLOppIS5NEi0gSE4uDbWDBb0kStao Ac2d/3yaVJVqQwwa6aVcOes0GLfKAxLVb9qPVEq7JwCiH5J0zbQ7TQHAemtsGICXYyIP fS1xydUgCeUfzjMLpoUZr2EEhtFDevF7EenCqrktS4GAKZ3TtCczhq6qbyFSEr2WLiGt 59UWUaEDdefnKdfTqVv6qmIy/TC61gKvlLp3Xr1SbpsXO6pFrt3mxsEOV/BQO+MLk2H2 44Sw== X-Received: by 10.68.162.34 with SMTP id xx2mr16065213pbb.120.1404923513752; Wed, 09 Jul 2014 09:31:53 -0700 (PDT) Received: from ox ([24.6.44.228]) by mx.google.com with ESMTPSA id ob1sm29061125pdb.40.2014.07.09.09.31.52 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 09 Jul 2014 09:31:53 -0700 (PDT) Sender: Navdeep Parhar Date: Wed, 9 Jul 2014 09:31:46 -0700 From: Navdeep Parhar To: Hans Petter Selasky Subject: Re: [RFC] Add support for changing the flow ID of TCP connections Message-ID: <20140709163146.GA21731@ox> Mail-Followup-To: Hans Petter Selasky , freebsd-net@freebsd.org, FreeBSD Current References: <53BC2E73.6090700@selasky.org> <53BC43AE.3040409@FreeBSD.org> <53BD5385.4090208@selasky.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53BD5385.4090208@selasky.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org, FreeBSD Current X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 16:31:54 -0000 On Wed, Jul 09, 2014 at 04:36:53PM +0200, Hans Petter Selasky wrote: > On 07/08/14 21:17, Navdeep Parhar wrote: ... > > > >I think we need to design this to be as generic as possible. I have > >quite a bit of code that does this stuff but I haven't pushed it > >upstream or even offered it for review (yet). > > > > Hi, > > When will the non hardware related patches be available for review? > I understand there are multiple ways to reach the same goal, and I > think it would be great if we could agree on a common API for > applications. Here is the kernel side of the patch: http://people.freebsd.org/~np/flow_pacing_kern.diff The registration parameters and the throttling parameters are probably cxgbe-centric, because that's what it was written for. We'll need to tidy up those structs certainly. And I'd like to add pps constraints to the throttling parameters (all it does is bandwidth right now). Regards, Navdeep From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 16:51:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 880E85A8 for ; Wed, 9 Jul 2014 16:51:45 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 59E822245 for ; Wed, 9 Jul 2014 16:51:45 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-250-191.lns20.per2.internode.on.net [121.45.250.191]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s69Gpf5h055062 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 9 Jul 2014 09:51:43 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53BD7317.9030008@freebsd.org> Date: Thu, 10 Jul 2014 00:51:35 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Niu Zhixiong , freebsd-net@freebsd.org Subject: Re: A question on modifying packet. References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 16:51:45 -0000 On 7/9/14, 1:01 PM, Niu Zhixiong wrote: > Hi, all > I have only one NIC. I want to capture packets from one certain ip address > and change the both src and dst addresses and forward to other destination > via the same NIC. It is possible? Is there any library to help me do this? there is no library, but look at how natd does this using divert sockets. > > > Regards, > Niu Zhixiong >  > kaiaixi@gmail.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Wed Jul 9 17:47:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BA5D3A3; Wed, 9 Jul 2014 17:47:55 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0430277A; Wed, 9 Jul 2014 17:47:54 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so9453922pab.8 for ; Wed, 09 Jul 2014 10:47:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Nb0bvoH02a03fvc8PuhSftLkyFvoHxJG92lyT+Nhesg=; b=k/Hbw3CpuzYBTtQKg9j5NBmKr7ui4UEfrHXNpHZ8g6bW8VlCnX0zSypK9gApwqjtVH iUZQVvjKrRgAteBD+MK3rmmX41WuCx9B0qBy/BwJwzwq+rZqafQ1qZD6oP68vsDr6ssl +KloGHVT3UNBLYN4esLpP4B+AVqpwOqYHQOEgqJHY3uDgi2ibgK+wLofDvEqzy4pD27k YAc4ZSHm+x1L1qF46I6wUc/iUqOSNLO3e1MnqMBpXpGH3efsu4GYGa2/EUIdZUfhz3up EGIxemAT+gZ350fOrk9xWlKRsW9SUcgIjpYcKFrTWjYAQHeCkjE18X+8UfX0Qvtf2/U+ ZiEQ== MIME-Version: 1.0 X-Received: by 10.68.252.73 with SMTP id zq9mr940533pbc.146.1404928074443; Wed, 09 Jul 2014 10:47:54 -0700 (PDT) Received: by 10.70.74.168 with HTTP; Wed, 9 Jul 2014 10:47:54 -0700 (PDT) Received: by 10.70.74.168 with HTTP; Wed, 9 Jul 2014 10:47:54 -0700 (PDT) In-Reply-To: <53BD7317.9030008@freebsd.org> References: <53BD7317.9030008@freebsd.org> Date: Wed, 9 Jul 2014 20:47:54 +0300 Message-ID: Subject: Re: A question on modifying packet. From: Sami Halabi To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-net@freebsd.org, Niu Zhixiong X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 17:47:55 -0000 use netgraph. In theory it should work.. in practice It worked for me once, but couldn't repeat the success. See ng_patch. Here is the code i tried: kldload ng_patch kldload ng_ipfw /usr/sbin/ngctl -f- << SEQ mkpeer ipfw: patch 300 in name ipfw:300 src_dst_chg msg src_dst_chg: setconfig { count=3D2 csum_flags=3D1 ops=3D[ \ { mode=3D1 value=3D0xc0a8e609 length=3D4 offset=3D= 12 } \ { mode=3D1 value=3D0xc0a8e680 length=3D4 offset=3D= 16 } ] } SEQ /sbin/ipfw add 600 netgraph 300 log ip from any to 239.0.0.19 dst-port 1234 in via vlan999 Sami On 7/9/14, 1:01 PM, Niu Zhixiong wrote: > Hi, all > I have only one NIC. I want to capture packets from one certain ip addres= s > and change the both src and dst addresses and forward to other destinatio= n > via the same NIC. It is possible? Is there any library to help me do this= ? > there is no library, but look at how natd does this using divert sockets. > > Regards, > Niu Zhixiong > =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D= =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D > kaiaixi@gmail.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 02:41:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 069F3266 for ; Thu, 10 Jul 2014 02:41:00 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA3DD2512 for ; Thu, 10 Jul 2014 02:40:59 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id j15so864458qaq.1 for ; Wed, 09 Jul 2014 19:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=a6cWY+MPn6D61LJhKdZ6Mm07SSUSzBlGv3FRKhnOh9o=; b=Oy+VpIMtiDfxhbe3AhMv9Xm3Gba94ShR6AYa+Uc2AFOoSvnJnH7vkqlUANash7G5+j kPoL1E3ZsoT+xEt7t2rWbdtWMhfwaDRV0xhxFwzwsiaGD0F9+4270DW1eq2r0ZoByQki vaJkxW9FxJRL8Je9vi8735ypxB7td3cz+WEjRNCzvROmV7OcBZRJhI8aPudVZNn3+tF8 fB5akJ8LNdoml1oQxtB4V/PAdbb/olW9NiFqjsbI6iSxSXx6Jnp8ClF9U/5mvU3NrhJ4 jRtXTM8fKqc2qt2AciPZQjITRMizIF7VKWi7ea8rLq5zmnytEs6/Xz4q6+s8N8SycDTr OqYQ== MIME-Version: 1.0 X-Received: by 10.140.19.109 with SMTP id 100mr73227508qgg.80.1404960058791; Wed, 09 Jul 2014 19:40:58 -0700 (PDT) Received: by 10.224.7.5 with HTTP; Wed, 9 Jul 2014 19:40:58 -0700 (PDT) In-Reply-To: References: <53BD7317.9030008@freebsd.org> Date: Thu, 10 Jul 2014 10:40:58 +0800 Message-ID: Subject: Re: A question on modifying packet. From: Niu Zhixiong To: Sami Halabi , freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 02:41:00 -0000 Sry, The problem is that I receive the packet from this NIC and send this packet via the same NIC? Does netmap can help me? Regards, Niu Zhixiong =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF= =BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D kaiaixi@gmail.com On Thu, Jul 10, 2014 at 1:47 AM, Sami Halabi wrote: > use netgraph. > In theory it should work.. in practice It worked for me once, but couldn'= t > repeat the success. > > See ng_patch. Here is the code i tried: > kldload ng_patch > kldload ng_ipfw > /usr/sbin/ngctl -f- << SEQ > mkpeer ipfw: patch 300 in > name ipfw:300 src_dst_chg > msg src_dst_chg: setconfig { count=3D2 csum_flags=3D1 > ops=3D[ \ > { mode=3D1 value=3D0xc0a8e609 length=3D4 offset= =3D12 } \ > { mode=3D1 value=3D0xc0a8e680 length=3D4 offset= =3D16 } ] } > SEQ > /sbin/ipfw add 600 netgraph 300 log ip from any to 239.0.0.19 dst-port > 1234 in via vlan999 > > Sami > On 7/9/14, 1:01 PM, Niu Zhixiong wrote: > >> Hi, all >> I have only one NIC. I want to capture packets from one certain ip addre= ss >> and change the both src and dst addresses and forward to other destinati= on >> via the same NIC. It is possible? Is there any library to help me do thi= s? >> > there is no library, but look at how natd does this using divert sockets. > > >> >> Regards, >> Niu Zhixiong >> =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D= =EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D=EF=BC=8D >> kaiaixi@gmail.com >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 08:13:23 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E710258; Thu, 10 Jul 2014 08:13:23 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5022B2EA0; Thu, 10 Jul 2014 08:13:23 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id ik5so9762667vcb.35 for ; Thu, 10 Jul 2014 01:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Dqp+RhJ3d5x5SArFjWRGNewVvxqL3Hv2vjX/dsHLQoU=; b=x6mw3IOKTEURsAce49ZBoXnUk7GyOeY/0XVwF4y/BeLDpxznaV05tzoTmkcCrlXK0e OjQ+HBY/10fW3Ogf4ggf5bO1AbQvCcZ4KGoh5zwIrCr4msNB4IXcvVdln5r0Nfrp07xB jamaCo+9nIvQovJAsebUKBLn+mez7x5wZ1U7sTeooEVxSq6so8T4jl39V4MIHiNQtCL8 //F1FUfPrrYivtVoHvD+y5lds9vs5ED8mPLCeNpImwZyoshdVNlph2xT4QJz84m3KxlU 9yCdZYlL0X3WZJ+00E0ujtpDEYFK3UUj5O5dGfhFDhbESRyJ6CdcBkCPS2QSmiVcF5JS w2XQ== MIME-Version: 1.0 X-Received: by 10.220.177.133 with SMTP id bi5mr2051973vcb.26.1404980002338; Thu, 10 Jul 2014 01:13:22 -0700 (PDT) Received: by 10.221.0.147 with HTTP; Thu, 10 Jul 2014 01:13:22 -0700 (PDT) Date: Thu, 10 Jul 2014 10:13:22 +0200 Message-ID: Subject: ng_netflow From: Cristiano Deana To: FreeBSD Stable Mailing List , FreeBSD net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 08:13:23 -0000 Hi all, I have a bsd box as a router, with 4 vlan interfaces. I started collecting flow data with softflowd and analyze them (in a separate machine) with nfsen, but softflowd is taking too much cpu (for the busiest interface up to 20%), so i tried to switch ONE interface to ng_netflow. I configured the same as the man page, but results are... odd. Measure are wrong. I mean, graphs collected from softflowd show the right amounts of packet/flows/data, the ones collected with ng_netflow are wrong. packets and flows are lower then expected, traffic is MUCH LOWER than expected (1/10). Any hint to debug or anyone with similar experience? Thank you -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/ From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 08:27:39 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A1915D1 for ; Thu, 10 Jul 2014 08:27:39 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D57C2FDF for ; Thu, 10 Jul 2014 08:27:38 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id ij19so9814166vcb.37 for ; Thu, 10 Jul 2014 01:27:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=E/kTcuadlalFX0PWafm4kpMAXE/NHnJq5RjavMiAHh4=; b=Wcdlyl/M76afzgvwPK1ikgaR3PKXqSXCujnqOXr7aNU0aVSwIxRkFUKEnetfTix9/C Los+ALqH+rzQKdZkVdrxJy6So6g714ffDmf6EvG/5QBkBOTdIlS55I+9w3amnxt4/isW ODu2AQAZ6PBYKFQgtxbLj7Oco5WalfoobYe2KOtxaQo6FtyY5/LI/7AYo6kBXhx5fMQv iCqjzN6F0O0QsWAnCzYklYIxM9ELOUTNg48h+1C7MZ1tFZk7t0aO4++ZZokqaQarYWB9 HOSBCcdQwbNllBl9xBp+abN8o01l2zKgCxl13mv7wVbUnz4o2CnfFMsoK6BdnPYgQOkO cNTw== MIME-Version: 1.0 X-Received: by 10.58.150.100 with SMTP id uh4mr30120871veb.30.1404980857854; Thu, 10 Jul 2014 01:27:37 -0700 (PDT) Received: by 10.220.40.75 with HTTP; Thu, 10 Jul 2014 01:27:37 -0700 (PDT) Date: Thu, 10 Jul 2014 13:57:37 +0530 Message-ID: Subject: Query Regarding Netmap From: Robert Clove To: net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 08:27:39 -0000 Hello Sir, I am a student and i am working on a project in which we have mellanox NIC cards. Is your netmap supported on Mellanox? If yes, how to load the netmap driver,where to get the files.... Sorry for wrong ques i am not much aware of it. Regards Clove From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 10:51:49 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76F5CF67 for ; Thu, 10 Jul 2014 10:51:49 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2D812CBC for ; Thu, 10 Jul 2014 10:51:48 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so5849448lan.26 for ; Thu, 10 Jul 2014 03:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GmRwt5DieXlwHrriqXEFX2lIZ5wWjWMk76cHvm4vPqw=; b=zuDCHOWDhbhPsQwBAv7e50coEkDxsc5NT0y6dcdHNRbEjo0D/cAEQ8phQ6QSP1BLNH 7FsFAdikUHYog0qvALYNcUsJ2l36YaRbdTWTU7UVnxns8vT4/NPigWruyQxuoGoznJeg ZTYfsq9Ujn92+lwLGiK4D9vcMiNyr97WtaaX/NgqAFMFqnTkQU0U9uHkNKiYDiphLpT7 wFkgXh0EyeZJIxA/QiR4cISsKMhhBSxO345u9GaLsVzDHya7/kmfMbAAbvxAsDpJyodX 1ykCyCk+S04U87117ajAhmYER1w3U2WSbqbjQnybeEKkd12G5DVUEhv6NmGhNb3eBKvg KzOg== MIME-Version: 1.0 X-Received: by 10.112.134.40 with SMTP id ph8mr4041123lbb.34.1404989506762; Thu, 10 Jul 2014 03:51:46 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.234 with HTTP; Thu, 10 Jul 2014 03:51:46 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Jul 2014 12:51:46 +0200 X-Google-Sender-Auth: r3mflO1xAQTN1aNpMVapDDUOYZc Message-ID: Subject: Re: Query Regarding Netmap From: Luigi Rizzo To: Robert Clove Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 10:51:49 -0000 On Thu, Jul 10, 2014 at 10:27 AM, Robert Clove wrote: > Hello Sir, > > I am a student and i am working on a project in which we have mellanox NI= C > cards. > Is your netmap supported on Mellanox? > If yes, how to load the netmap driver,where to get the files.... > =E2=80=8Bhi, i had some mellanox patches to support netmap natively, but only for linux. This said, the card seems unable to go beyond ~7Mpps so you might still get similar results with the emulated netmap mode, which only uses the netmap kernel module without any driver modification. Cheers luigi =E2=80=8B > > From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 11:16:39 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16D2E5E8 for ; Thu, 10 Jul 2014 11:16:39 +0000 (UTC) Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C910B2E95 for ; Thu, 10 Jul 2014 11:16:38 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id il7so10262941vcb.12 for ; Thu, 10 Jul 2014 04:16:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9OBVd9OSh4ge7B0jQJqt43EIGhC/3O+zHdMBVEPyOnc=; b=YhvaJ6P1YUHe9K2MoUVRr4r+dlmwOzRoteSysPDg/0mPGlwvmK3H+teKPzosOTwlXY prc9lESoZ0TzSGPbhZC2hUbwrDQeaKj5txOwI2cllaaFA5EWMiN+eMEfciZWcRPo6Ru8 le4u3uBK/dYvWhhFvYrvb81LvQU21zQmI/xwNd34zwshKN1qMSZpoc6I+GQ2TCTZHkMS zu/Nv/UkiPhmZzIcN0udZj0xKUL5xDYpmpSQIiBWc3+muO78yzz9vwNbXO/BygBqDdK7 c2E+tHXq1UT/Lx+FAC4qNM0bnBC6oObqeuUWR+orxxouvXmWj5nc09KHUWi8uwSJqpCq r8ZA== MIME-Version: 1.0 X-Received: by 10.220.68.140 with SMTP id v12mr44479168vci.13.1404990997726; Thu, 10 Jul 2014 04:16:37 -0700 (PDT) Received: by 10.220.40.75 with HTTP; Thu, 10 Jul 2014 04:16:37 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Jul 2014 16:46:37 +0530 Message-ID: Subject: Re: Query Regarding Netmap From: Robert Clove To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 11:16:39 -0000 Dear Sir, Thanks for the reply. Following things i need from you: where to get the module for netmap. How to apply it? So according to you fro mellanox card i should not apply the patch on Linux just install the netmap? Regards On Thu, Jul 10, 2014 at 4:21 PM, Luigi Rizzo wrote: > > > > On Thu, Jul 10, 2014 at 10:27 AM, Robert Clove > wrote: > >> Hello Sir, >> >> I am a student and i am working on a project in which we have mellanox N= IC >> cards. >> Is your netmap supported on Mellanox? >> If yes, how to load the netmap driver,where to get the files.... >> > > =E2=80=8Bhi, > i had some mellanox patches to support netmap natively, but only for linu= x. > This said, the card seems unable to go beyond ~7Mpps so you might still > get similar results with the emulated netmap mode, which only uses > the netmap kernel module without any driver modification. > > Cheers > luigi > =E2=80=8B > >> >> From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 11:42:51 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3735CC21 for ; Thu, 10 Jul 2014 11:42:51 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A24522127 for ; Thu, 10 Jul 2014 11:42:50 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id s7so6006370lbd.18 for ; Thu, 10 Jul 2014 04:42:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=fJnRIVhXWK5efhZ8b7ZFtXTpfBDfIRKqniJZY28JnE4=; b=oPSgqk7K0E66dKXGoDj+BpOXpGBEkYg1W8kIJ2ryBWS7xp6A9n7ZyYaVyY3Jr8Thae iNDvicsWioPbc4DAKn86YCGwjyor3EcIqbEFwdVdPPjxcJex4g6xQ/YguE/mXvtDq4sh NeaaMPG7Crnd23c3wFAv18IPOODxTJx1IiSN0Z5Lt38NHEXh3+QtjtQqwNjiP82TtUpY dNZjVvFJUJa6vuXBSd2cxRYjG3ag6ixeUNB/x52S5Tt2MfzoRYxYzF8Evui3TNKtg5tm DxMGwRhLNAEz0X40bL3HT3D/x0BHEGRpWwtniQGXK54R6zmqam70yk6kFPoj4sJEbxus SuMw== MIME-Version: 1.0 X-Received: by 10.112.204.106 with SMTP id kx10mr2526506lbc.73.1404992568337; Thu, 10 Jul 2014 04:42:48 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.234 with HTTP; Thu, 10 Jul 2014 04:42:48 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Jul 2014 13:42:48 +0200 X-Google-Sender-Auth: 7BOzPSS-KILhFBfdxAlqRePF_tU Message-ID: Subject: Re: Query Regarding Netmap From: Luigi Rizzo To: Robert Clove Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 11:42:51 -0000 code.google.com/p/netmap/ On Thu, Jul 10, 2014 at 1:16 PM, Robert Clove wrote= : > Dear Sir, > > Thanks for the reply. > Following things i need from you: > where to get the module for netmap. > How to apply it? > So according to you fro mellanox card i should not apply the patch on > Linux just install the netmap? > > Regards > > > > On Thu, Jul 10, 2014 at 4:21 PM, Luigi Rizzo wrote: > >> >> >> >> On Thu, Jul 10, 2014 at 10:27 AM, Robert Clove >> wrote: >> >>> Hello Sir, >>> >>> I am a student and i am working on a project in which we have mellanox >>> NIC >>> cards. >>> Is your netmap supported on Mellanox? >>> If yes, how to load the netmap driver,where to get the files.... >>> >> >> =E2=80=8Bhi, >> i had some mellanox patches to support netmap natively, but only for >> linux. >> This said, the card seems unable to go beyond ~7Mpps so you might still >> get similar results with the emulated netmap mode, which only uses >> the netmap kernel module without any driver modification. >> >> Cheers >> luigi >> =E2=80=8B >> >>> >>> > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 14:20:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC47F90B; Thu, 10 Jul 2014 14:20:33 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BE6A32F91; Thu, 10 Jul 2014 14:20:33 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-236-203.lns20.per1.internode.on.net [121.45.236.203]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s6AEKRC4058415 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 10 Jul 2014 07:20:30 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53BEA125.6010400@freebsd.org> Date: Thu, 10 Jul 2014 22:20:21 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Cristiano Deana , FreeBSD Stable Mailing List , FreeBSD net Subject: Re: ng_netflow References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 14:20:34 -0000 On 7/10/14, 4:13 PM, Cristiano Deana wrote: > Hi all, > > I have a bsd box as a router, with 4 vlan interfaces. > I started collecting flow data with softflowd and analyze them (in a > separate machine) with nfsen, but softflowd is taking too much cpu > (for the busiest interface up to 20%), so i tried to switch ONE > interface to ng_netflow. > > I configured the same as the man page, but results are... odd. > > Measure are wrong. I mean, graphs collected from softflowd show the > right amounts of packet/flows/data, the ones collected with ng_netflow > are wrong. > packets and flows are lower then expected, traffic is MUCH LOWER than > expected (1/10). > > Any hint to debug or anyone with similar experience? Is it possible that you are working with an interface that has TSO on? if so then netgraph will be seeing huge "aggregate" packets rather than the normal packets. so teh number of packets may be out by more than a factor of 30. I have never used the ng_netflow node, but try turning off TSO if you have it on and see if that makes a difference. failing that you might just look a the source of the ng_netflow module (ng_netflow.c) for ideas. netgraph nodes are relatively simple.. they have an entry point where packets turn up and they call the same method in some other module.. > > Thank you > From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 14:39:11 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC7C4D9A; Thu, 10 Jul 2014 14:39:11 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F782179; Thu, 10 Jul 2014 14:39:11 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id hy4so10740292vcb.34 for ; Thu, 10 Jul 2014 07:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BcCFPIntKfm3CGp9Xj9xb4ix+nPAfc1AdspuVV/IA2o=; b=disdpQi9CFIhxBE7+aKd1I7Q+kL7vJvqoXIxoHlySrFSYT/xvSdKWa+9cign6XaS3f sff6CmZi6exvHjMeqhMezQ2mfPTQUR4z4BNUqTCYrNJ93P1296OX5HJKcdd0xxzSyybZ W4UmP+TMqEq+TRiPcNhzdcSN8RT9woKywPx8FtW6Czzxeuu4RVd0yomW39TAvv74xxOK np+C8qexSKUPauDT+j+Aprr4kHqJjt0CVhMqPwoVqfz2NgdPzv14VXaT8NOA2N7NlvG1 fxKrGCx+r96h0JL/QKDS6DF9TUXxH+CXaFPuxoKUucBJblBToVZ/6hts+DWn/t4zlzpc 0oMw== MIME-Version: 1.0 X-Received: by 10.58.72.165 with SMTP id e5mr609560vev.59.1405003150217; Thu, 10 Jul 2014 07:39:10 -0700 (PDT) Received: by 10.221.0.147 with HTTP; Thu, 10 Jul 2014 07:39:10 -0700 (PDT) In-Reply-To: <53BEA125.6010400@freebsd.org> References: <53BEA125.6010400@freebsd.org> Date: Thu, 10 Jul 2014 16:39:10 +0200 Message-ID: Subject: Re: ng_netflow From: Cristiano Deana To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD net , FreeBSD Stable Mailing List X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 14:39:11 -0000 On Thu, Jul 10, 2014 at 4:20 PM, Julian Elischer wrote: Hi, Julian > Is it possible that you are working with an interface that has TSO on? it's a vlan interface, the device is a em0. net.inet.tcp.tso: 1 > if so then netgraph will be seeing huge "aggregate" packets rather than the > normal packets. > so teh number of packets may be out by more than a factor of 30. My bigger problem is with traffic counter, but maybe it's just my error. > netgraph nodes are relatively simple.. Not really :) I have problems to find "easy" documentation about it. It's hard to understand what lower, upper, right, etc meaning. I think I could hav setup something wrong, maybe counting only incoming packet (or outgoing). I followed, not fully undestand, the example in ng_netflow man page: /usr/sbin/ngctl -f- <<-SEQ mkpeer fxp0: netflow lower iface0 name fxp0:lower netflow connect fxp0: netflow: upper out0 mkpeer netflow: ksocket export inet/dgram/udp msg netflow:export connect inet/10.0.0.1:4444 SEQ Thank you -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/ From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 15:22:00 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48D9AF83; Thu, 10 Jul 2014 15:22:00 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B2E325B2; Thu, 10 Jul 2014 15:21:59 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:52429 helo=[192.168.1.12]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X5GAW-0006uM-Ky; Thu, 10 Jul 2014 11:21:57 -0400 From: "George Neville-Neil" To: "Garrett Cooper" Subject: Re: A new way to test systems in multiple machine scenarios... Date: Thu, 10 Jul 2014 11:21:54 -0400 Message-ID: In-Reply-To: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: Craig Rodrigues , "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 15:22:00 -0000 On 8 Jul 2014, at 4:56, Garrett Cooper wrote: > On Jul 6, 2014, at 9:06 PM, Craig Rodrigues = > wrote: > >> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil = >> >> wrote: >> >>> Hi, >>> >>> I've coded up a system to allow you to control multiple other = >>> systems for >>> use in testing. >>> >>> https://github.com/gvnn3/conductor >>> >>> >> Cool! The architecture you have is similar to that of the SPECsfs >> benchmark test ( http://www.spec.org/sfs2008/ ) >> which involves a "coordinator node" and multiple "client nodes" which >> direct NFS network >> traffic towards a System Under Test (SUT). Garrett Cooper actually = >> set up >> the original testbed >> that I am using now at iXsystems. :) >> >> It would be cool to put together tools like Jenkins, Kyua, and = >> conductor to >> do more advanced testing >> of FreeBSD before the project puts out releases. > > Agreed. The only thing that I have some concern about is the = > reinventing of the wheel in python. multiprocessing Managers are one = > viable option that=E2=80=99s existed since python 2.6; there=E2=80=99s = a learning = > curve though, and you=E2=80=99ll run into problems with pickling some = > objects because the pickle protocol is far from complete (example: = > http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemet= hod-when-using-pythons-multiprocessing-pool-ma = > ); you might run into this problems regardless because you=E2=80=99re = > serializing objects using pickle instead of using dill (or using a = > simpler serialization method like JSON). Fabric has a framework = > that=E2=80=99s nice to use if you have ssh capability. There are other = > frameworks that use twisted conch I think too (another library that = > implements ssh access). Yes, I learned quite a bit about pickling in writing this. Conductor = aims to be quite simple so I am hoping to avoid any crazy corner cases to do with pickling. > > Isilon has a framework they use, but it=E2=80=99s very customized to th= eir = > infrastructure and product assumptions and it=E2=80=99s in need of an = > overhaul :(. This, actually, is the problem I found. Lots of folks have partial = solutions that are either proprietary, internal, not read for prime time, not quite what we want, etc. etc. I = did get one private response of another system to look at as well. I basically did this as a stake in the ground around which to build = something we could possibly move forwards with. It's not a 100% solution but it's 80% of the solution to the = problem I run into 80% of the time. Best, George From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 17:07:40 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AA32163; Thu, 10 Jul 2014 17:07:40 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 682652F09; Thu, 10 Jul 2014 17:07:39 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id el20so6511543lab.35 for ; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UFBdicxDi6WH9o4QyTnTqrgUqm6bkxYXPvhmaJCUHVw=; b=FM7Vv7Ir27tqftU/QFsP0AI9n4mocOg9YegMYRhBMy5CQvCraMoPu74kqE0uHmf1pp qwxJHlhZGAjDqGwAAVM5p+OOWWpPJbpjHyqIuOhPVIKKx+6FxWEKQvRsC5FwBWH7Wtek 7LIa2+c2ZDz6dDhDI7I9Oay2sntpHK6PyGBccjV50/o3awq3mGxkme9E4HPpWJFMWbJs 6XV2AhPnYo4H0Y8rz3YKFAW7oN4TsWNna1hMKoYoWKJewbC0+GDCZ3EvkWWmNxXrIvt4 zLfPG1YuU2gwIxBVCSf6661fZcch0ZL4n0X5qAeQEW358p5bEzzDzjUeyqNe8dFOAFgx n42Q== MIME-Version: 1.0 X-Received: by 10.112.34.110 with SMTP id y14mr6566246lbi.36.1405012057304; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) In-Reply-To: References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> Date: Thu, 10 Jul 2014 10:07:37 -0700 X-Google-Sender-Auth: iMXwwAWS_9CLuTrjaRl5gM_G_-c Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Craig Rodrigues To: George Neville-Neil Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Garrett Cooper , "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 17:07:40 -0000 On Thu, Jul 10, 2014 at 8:21 AM, George Neville-Neil wrote: > This, actually, is the problem I found. Lots of folks have partial > solutions that are either proprietary, > internal, not read for prime time, not quite what we want, etc. etc. I > did get one private > response of another system to look at as well. > > I basically did this as a stake in the ground around which to build > something we could possibly move forwards > with. It's not a 100% solution but it's 80% of the solution to the > problem I run into 80% of the time. > You are absolutely right. Many companies have internal testing frameworks that are home-grown and proprietary. It's great for the individual company, but doesn't help people in the larger community. Since you put this stuff in github, I encourage folks to hack on the code and send pull requests to George to help improve it! -- Craig From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 19:37:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00D40B95 for ; Thu, 10 Jul 2014 19:37:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD9DA2C9B for ; Thu, 10 Jul 2014 19:37:35 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CCD04B91C; Thu, 10 Jul 2014 15:37:33 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Add netbw option to systat Date: Thu, 10 Jul 2014 13:14:58 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407101314.58558.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Jul 2014 15:37:33 -0400 (EDT) Cc: Bryan Venteicher , hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 19:37:36 -0000 On Wednesday, July 02, 2014 8:54:41 pm hiren panchasara wrote: > On Wed, Jul 2, 2014 at 4:50 PM, Bryan Venteicher > wrote: > > Awhile back, DragonlyFlyBSD added a netbw option to systat that I've ported > > to FreeBSD and found handy at various times: > > > > netbw Display aggregate and per-connection TCP receive and transmit > > rates. Only active TCP connections are shown. > > > > Leading to output such as: > > > > tcp accepts connects rcv 1.192G snd 15.77K rexmit > > > > 192.168.10.80:22 192.168.10.20:23103 rcv snd 415.7 [ NTSX ] > > 192.168.10.80:22 192.168.10.20:46560 rcv 19.80M snd 14.47K [ NTSX ] > > 192.168.10.80:22 192.168.10.20:60699 rcv snd 886.3 [ NTSX ] > > 192.168.10.81:5201 192.168.10.51:60844 rcv 293.2M snd [R TSX ] > > 192.168.10.81:5201 192.168.10.51:60845 rcv 293.5M snd [R TSX ] > > 192.168.10.81:5201 192.168.10.51:60846 rcv 293.2M snd [R TSX ] > > 192.168.10.81:5201 192.168.10.51:60847 rcv 292.9M snd [R TSX ] > > > > It uses the sequences number from the 'struct tcpcb' to derive the rates, > > which is usually good but certainly not perfect (i.e., don't set the > > interval too long). > > > > I'd like to commit this if anybody else thinks they'd find it useful. > > > > http://people.freebsd.org/~bryanv/patches/systat-netbw.patch > > I like the idea. I also like the idea. > A few things about the patch: > 1) You may want to remove the code hidden behind "#if 0" at 2 places. > 2) I am not entirely clear on why/if we need the last column with > flags but if we keep it (for compatibility of any other reason), It > would be nice to have those flags explained in the manpage: > > + mvwprintw(wnd, LINES-2, 0, > + "Rate/sec, " > + "R=rxpend T=txpend N=nodelay T=tstmp " > + "S=sack X=winscale F=fastrec"); > 3) I feel that the header line for o/p (specially 'tcp accepts and > connects' terminology) can be improved but I do not have a better > suggestion :-) 4) Should numtok() just be humanize_number? Or rather, would it simplify the code to use humanize_number? (It might not, but if it does, I think that would be preferable.) -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 19:37:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15222B96 for ; Thu, 10 Jul 2014 19:37:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2BAE2C9C for ; Thu, 10 Jul 2014 19:37:36 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D684CB926; Thu, 10 Jul 2014 15:37:35 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org, Laurie Jennings Subject: Re: System Booting Kernel from Secondary Drive Date: Thu, 10 Jul 2014 13:16:00 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1404396381.74461.YahooMailBasic@web125802.mail.ne1.yahoo.com> In-Reply-To: <1404396381.74461.YahooMailBasic@web125802.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407101316.00803.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Jul 2014 15:37:35 -0400 (EDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 19:37:37 -0000 On Thursday, July 03, 2014 10:06:21 am Laurie Jennings via freebsd-net wrote: > I'm having a problem with a Supermicro system running FreeBSD 9.1. Sometimes when I upgrade the kernel in my main drive (ada0), > the system boots the kernel from the 2nd drive. It only happens sometimes. ada0 is mounted. but the system is running the old kernel. > Pulling the 2nd fixed the problem. > > What can cause this to happen? Is it a supermicro problem (it's a 5017R-MTF superserver) or is it something with FreeBSD. Are you using a software RAID between the two disks? -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 19:37:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 246AEB97 for ; Thu, 10 Jul 2014 19:37:38 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2BDD2C9D for ; Thu, 10 Jul 2014 19:37:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EA743B94E; Thu, 10 Jul 2014 15:37:36 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current Date: Thu, 10 Jul 2014 13:25:45 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <870285181.7082888.1404435061501.JavaMail.root@uoguelph.ca> In-Reply-To: <870285181.7082888.1404435061501.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201407101325.46156.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Jul 2014 15:37:37 -0400 (EDT) Cc: "Russell L. Carter" , Rick Macklem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 19:37:38 -0000 On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > Russell L. Carter wrote: > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > Could you please post the dmesg stuff for the network interface, > > > so I can tell what driver is being used? I'll take a look at it, > > > in case it needs to be changed to use m_defrag(). > > > > em0: port 0xd020-0xd03f > > mem > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 on > > pci2 > > em0: Using an MSI interrupt > > em0: Ethernet address: 00:15:17:bc:29:ba > > 001.000007 [2323] netmap_attach success for em0 tx 1/1024 > > rx > > 1/1024 queues/slots > > > > This is one of those dual nic cards, so there is em1 as well... > > > Well, I took a quick look at the driver and it does use m_defrag(), but > I think that the "retry:" label it does a goto after doing so might be in > the wrong place. > > The attached untested patch might fix this. > > Is it convenient to build a kernel with this patch applied and then try > it with TSO enabled? > > rick > ps: It does have the transmit segment limit set to 32. I have no idea if > this is a hardware limitation. I think the retry is not in the wrong place, but the overhead of all those pullups is apparently quite severe. It would be interesting to test the following in addition to your change to see if it improves performance further: Index: if_em.c =================================================================== --- if_em.c (revision 268495) +++ if_em.c (working copy) @@ -1959,7 +1959,9 @@ retry: if (error == EFBIG && remap) { struct mbuf *m; - m = m_defrag(*m_headp, M_NOWAIT); + m = m_collapse(*m_headp, M_NOWAIT, EM_MAX_SCATTER); + if (m == NULL) + m = m_defrag(*m_headp, M_NOWAIT); if (m == NULL) { adapter->mbuf_alloc_failed++; m_freem(*m_headp); -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 19:37:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3FD31B98; Thu, 10 Jul 2014 19:37:39 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 15B812C9E; Thu, 10 Jul 2014 19:37:39 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0D06DB9A1; Thu, 10 Jul 2014 15:37:38 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: r256920 missing in stable/9 and releng/9.3 Date: Thu, 10 Jul 2014 13:36:00 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <37229F0A-4BEA-4712-9AE1-5446A630AF9F@transactionware.com> <53BA8666.9040608@omnilan.de> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201407101336.00965.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Jul 2014 15:37:38 -0400 (EDT) Cc: Harald Schmalzbauer , Andre Oppermann , FreeBSD Stable Mailing List , hiren panchasara , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 19:37:39 -0000 On Monday, July 07, 2014 12:58:53 pm hiren panchasara wrote: > + freebsd-net@, >=20 > On Mon, Jul 7, 2014 at 4:37 AM, Harald Schmalzbauer > wrote: > > Bez=C3=BCglich Jan Mikkelsen's Nachricht vom 24.06.2014 04:49 (localtim= e): > >> Hi, > >> > >> I=E2=80=99m bringing 9.3-RC1 into our local Perforce depot and moving = our local=20 patches to 9.2 forward. > >> > >> I noticed that r256920 (changing sys/netinet/tcp_input.c) has not been= =20 MFC=E2=80=99d. It was listed as =E2=80=9CMFC after 3 days=E2=80=9D back in = October 2013. > >> > >> Is this patch missing for a reason? > > > > I'm wondering too if there's any good reason not to MFC? >=20 > I also don't see any obvious reason. >=20 > If nobody objects on -net@, I can do it. I think this looks fine to merge. =2D-=20 John Baldwin From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 21:07:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD671A46; Thu, 10 Jul 2014 21:07:06 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B4D624B8; Thu, 10 Jul 2014 21:07:06 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id dc16so140945qab.36 for ; Thu, 10 Jul 2014 14:07:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=o4MueEeyIjUUvJ8q4Wk9ID+pw13T/9qwdr4w4OGb3lw=; b=eh8TlAGTjoXT5XhRhPcFW5n727F90FNHwU7QoQR60D9E8IJ8PfEVeb002e6LeJDkjj lBD2ZQo8Zpbsj7NGAMs9yAJfja09vRfzzrmS11Xx3chXgED99EMfWpezx/lNsttEFJ0b bswPowC+TGocS26FaDOK5DZZek+SRZtTmtT7FMNpAtO/0ehC/opbJAO7JxV7XrMYGGwv L8FSudcyoHJVf2n8uzz8FA3t2YaCNnVUjfdjHOayawO978DmfQhtMl4rkdCRgQKXH6sa REaIhLgPdjTmnDEqwAQc9s/6ZBGTN11bTIb0mTlEXzD3XlU8Ykz8PajwERF6rKqNdX3q TmQA== MIME-Version: 1.0 X-Received: by 10.140.89.18 with SMTP id u18mr38622944qgd.87.1405026425058; Thu, 10 Jul 2014 14:07:05 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Thu, 10 Jul 2014 14:07:05 -0700 (PDT) In-Reply-To: <201407101336.00965.jhb@freebsd.org> References: <37229F0A-4BEA-4712-9AE1-5446A630AF9F@transactionware.com> <53BA8666.9040608@omnilan.de> <201407101336.00965.jhb@freebsd.org> Date: Thu, 10 Jul 2014 14:07:05 -0700 Message-ID: Subject: Re: r256920 missing in stable/9 and releng/9.3 From: hiren panchasara To: John Baldwin Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , Andre Oppermann , "freebsd-net@freebsd.org" , FreeBSD Stable Mailing List , Harald Schmalzbauer X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 21:07:06 -0000 On Thu, Jul 10, 2014 at 10:36 AM, John Baldwin wrote: > On Monday, July 07, 2014 12:58:53 pm hiren panchasara wrote: >> + freebsd-net@, >> >> On Mon, Jul 7, 2014 at 4:37 AM, Harald Schmalzbauer >> wrote: >> > Bez=C3=BCglich Jan Mikkelsen's Nachricht vom 24.06.2014 04:49 (localti= me): >> >> Hi, >> >> >> >> I=E2=80=99m bringing 9.3-RC1 into our local Perforce depot and moving= our local > patches to 9.2 forward. >> >> >> >> I noticed that r256920 (changing sys/netinet/tcp_input.c) has not bee= n > MFC=E2=80=99d. It was listed as =E2=80=9CMFC after 3 days=E2=80=9D back i= n October 2013. >> >> >> >> Is this patch missing for a reason? >> > >> > I'm wondering too if there's any good reason not to MFC? >> >> I also don't see any obvious reason. >> >> If nobody objects on -net@, I can do it. > > I think this looks fine to merge. Thanks John for confirming. Committed as r268506 to stable/9. It's a bit too late for 9.3-R. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 22:08:32 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0285E72D; Thu, 10 Jul 2014 22:08:32 +0000 (UTC) Received: from caravan.chchile.org (caravan.chchile.org [178.32.125.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Caravan", Issuer "Mail Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C31DB2AB7; Thu, 10 Jul 2014 22:08:31 +0000 (UTC) Received: by caravan.chchile.org (Postfix, from userid 1000) id 29B3CB28B0; Thu, 10 Jul 2014 22:01:16 +0000 (UTC) Date: Fri, 11 Jul 2014 00:01:16 +0200 From: Jeremie Le Hen To: freebsd-net@FreeBSD.org Subject: Description change proposal for the no_prefer_iface flag Message-ID: <20140710220115.GA91832@caravan.chchile.org> Mail-Followup-To: freebsd-net@FreeBSD.org, ume@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Cc: ume@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 22:08:32 -0000 Hi Hajimu-san, net@, I plan to commit the following change to the no_prefer_iface flag description in the ifconfig(8) manpage. I find it easier to understand and more explicit than the current one. Comments? Index: sbin/ifconfig/ifconfig.8 =================================================================== --- sbin/ifconfig/ifconfig.8 (revision 268508) +++ sbin/ifconfig/ifconfig.8 (working copy) @@ -679,9 +679,11 @@ Clear a flag .Cm nud . .It Cm no_prefer_iface -Set a flag to not prefer address on the interface as candidates of the -source address for outgoing packets, even when the interface is -outgoing interface. +Set a flag to not honor rule 5 of source address selection in RFC 3484. +In practice this means the address on the outgoing interface which will +be used will not be preferred, effectively yielding the decision to the +address selection policy table, configurable with +.Xr ip6addrctl 8 . .It Cm -no_prefer_iface Clear a flag .Cm no_prefer_iface . -- Jeremie Le Hen Scientists say the world is made up of Protons, Neutrons and Electrons. They forgot to mention Morons. From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 22:31:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BD4C19C; Thu, 10 Jul 2014 22:31:50 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3C78B2CEF; Thu, 10 Jul 2014 22:31:49 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqcEAEAQv1ODaFve/2dsb2JhbABZg2Bagm+9dAqGb1MBgSV1hAMBAQEDAQEBASAEJyALBRYOCgICDRkCKQEJJgYIBwQBHASIGQgNrHOZLReBLI1BBgEBGwEzB4J3gUwFmBSENJJNg18hNX0IFyI X-IronPort-AV: E=Sophos;i="5.01,640,1400040000"; d="scan'208";a="139312936" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 10 Jul 2014 18:31:43 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 25977B4045; Thu, 10 Jul 2014 18:31:43 -0400 (EDT) Date: Thu, 10 Jul 2014 18:31:43 -0400 (EDT) From: Rick Macklem To: John Baldwin Message-ID: <1610703198.9975909.1405031503143.JavaMail.root@uoguelph.ca> In-Reply-To: <201407101325.46156.jhb@freebsd.org> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: "Russell L. Carter" , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 22:31:50 -0000 John Baldwin wrote: > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > Russell L. Carter wrote: > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > Could you please post the dmesg stuff for the network > > > > interface, > > > > so I can tell what driver is being used? I'll take a look at > > > > it, > > > > in case it needs to be changed to use m_defrag(). > > > > > > em0: port > > > 0xd020-0xd03f > > > mem > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 > > > on > > > pci2 > > > em0: Using an MSI interrupt > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > 001.000007 [2323] netmap_attach success for em0 tx > > > 1/1024 > > > rx > > > 1/1024 queues/slots > > > > > > This is one of those dual nic cards, so there is em1 as well... > > > > > Well, I took a quick look at the driver and it does use m_defrag(), > > but > > I think that the "retry:" label it does a goto after doing so might > > be in > > the wrong place. > > > > The attached untested patch might fix this. > > > > Is it convenient to build a kernel with this patch applied and then > > try > > it with TSO enabled? > > > > rick > > ps: It does have the transmit segment limit set to 32. I have no > > idea if > > this is a hardware limitation. > > I think the retry is not in the wrong place, but the overhead of all > those > pullups is apparently quite severe. The m_defrag() call after the first failure will just barely squeeze the just under 64K TSO segment into 32 mbuf clusters. Then I think any m_pullup() done during the retry will allocate an mbuf (at a glance it seems to always do this when the old mbuf is a cluster) and prepend that to the list. --> Now the list is > 32 mbufs again and the bus_dmammap_load_mbuf_sg() will fail again on the retry, this time fatally, I think? I can't see any reason to re-do all the stuff using m_pullup() and Russell reported that moving the "retry:" fixed his problem, from what I understood. > It would be interesting to test > the > following in addition to your change to see if it improves > performance > further: > > Index: if_em.c > =================================================================== > --- if_em.c (revision 268495) > +++ if_em.c (working copy) > @@ -1959,7 +1959,9 @@ retry: > if (error == EFBIG && remap) { > struct mbuf *m; > > - m = m_defrag(*m_headp, M_NOWAIT); > + m = m_collapse(*m_headp, M_NOWAIT, EM_MAX_SCATTER); > + if (m == NULL) > + m = m_defrag(*m_headp, M_NOWAIT); Since a just under 64K TSO segment barely fits in 32 mbuf clusters, I'm at least 99% sure the m_collapse() will fail, but it can't hurt to try it. (If it supported 33 or 34, I think m_collapse() would have a reasonable chance of success.) Right now the NFS and krpc code creates 2 small mbufs in front of the read/write data clusters and I think the TCP layer adds another one. Even if this was modified to put it all in one cluster, I don't think m_collapse() would succeed, since it only copies the data up and deletes an mbuf from the chain if it will all fit in the preceding one. Since the read/write data clusters are full (except the last one), they can't fit in the M_TRAILINGSPACE() of the preceding one unless it is empty from my reading of m_collapse(). rick > if (m == NULL) { > adapter->mbuf_alloc_failed++; > m_freem(*m_headp); > > > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 22:42:15 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C341344 for ; Thu, 10 Jul 2014 22:42:15 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 23A922DD2 for ; Thu, 10 Jul 2014 22:42:15 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6AMgF88074927 for ; Thu, 10 Jul 2014 22:42:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 154851] [new driver] [request]: Port brcm80211 driver from Linux to FreeBSD Date: Thu, 10 Jul 2014 22:42:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: wireless X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc component version bug_severity Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 22:42:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154851 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koobs@FreeBSD.org Component|kern |wireless Version|8.1-RELEASE |11.0-CURRENT Severity|Affects Only Me |Affects Many People -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 22:43:07 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DB753F1 for ; Thu, 10 Jul 2014 22:43:07 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65A182DEB for ; Thu, 10 Jul 2014 22:43:07 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6AMh7mM075746 for ; Thu, 10 Jul 2014 22:43:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 154851] [new driver] [request]: Port brcm80211 driver from Linux to FreeBSD Date: Thu, 10 Jul 2014 22:43:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: wireless X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 22:43:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154851 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Open -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 04:51:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67D233A0; Fri, 11 Jul 2014 04:51:21 +0000 (UTC) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 1226B2C2B; Fri, 11 Jul 2014 04:51:20 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id C32131A6429; Fri, 11 Jul 2014 14:51:02 +1000 (EST) Date: Fri, 11 Jul 2014 14:50:54 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin Subject: Re: Add netbw option to systat In-Reply-To: <201407101314.58558.jhb@freebsd.org> Message-ID: <20140711133724.E969@besplex.bde.org> References: <201407101314.58558.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=dZS5gxne c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=O_OX2y58AAAA:8 a=6I5d2MoRAAAA:8 a=4q-Fw4_naauloQYB2ZIA:9 a=OF2PrfWNs_z9Q2o2:21 a=e_RUTKJNGN7qmsxl:21 a=CjuIK1q_8ugA:10 a=UcwoUbD9TkEA:10 Cc: freebsd-net@freebsd.org, Bryan Venteicher , hiren panchasara X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 04:51:21 -0000 On Thu, 10 Jul 2014, John Baldwin wrote: > On Wednesday, July 02, 2014 8:54:41 pm hiren panchasara wrote: >> On Wed, Jul 2, 2014 at 4:50 PM, Bryan Venteicher >> wrote: >>> I'd like to commit this if anybody else thinks they'd find it useful. >>> >>> http://people.freebsd.org/~bryanv/patches/systat-netbw.patch > ... > 4) Should numtok() just be humanize_number? Or rather, would it simplify > the code to use humanize_number? (It might not, but if it does, I > think that would be preferable.) No, nothing should use dehumanize(scientificize)_number(). It is a badly designed API that doesn't even support unsigned numbers or intmax_t. But numtok() takes a floating point arg. (It is always used for rates that really do need floating point or perhaps a quotient of integers. Except under #if 0, it is called on intger args. It doesn't support this case since it always geenrates a %5.Nf format with N > 0 (except for numbers < 0.001 it prints nothing, perhaps because %f format doesn't work well for this case). systat already hads the better functions putint(), putfloat() and putlongdouble(). Unfortunately, these are static in vmstat.c. They should be used throughout systat to get a consistent format. numtok() takes a double arg and could be handled at some cost in efficiency by putlongdouble(). (putlongdouble() only exists due to design errors in libdevstat. Old parts of systat -v use floats and floats are more than adequate, but libdevstat uses long doubles. Probably downcasting the long doubles to float would work in systat -v, but putfloat() was cloned to create putlongdouble() instead.) putlongdouble() prints the output, but numtok() formats the output for printing and returns it in a static buffer with MAXINDEXES = 8 generations so that this method is not too fragile. In general, direct printing is easier to use, but here the output has be printed at a certain place in the window. putlongdouble() takes coordinates for each call and prints using move(), addch() and addstr(). It has more control over padding characters than printf() can provide or humanize_number() can dream of, and uses this to padd with '*' in some cases (mainly for numbers that cannot be formatted to fit in the desired space. Callers must pass some format info, especially the field width, in each call. numtok() basically hard-codes a field width of 6 and a format of %5.N%c where %c is the suffix. This format wastes 1 character for the suffix when the suffix is ' '. putlongdouble() and even dehumanize_number() avoid this wastage. This is more critical when the field with is < 6. N > 0 and the decimal point also waste a lot of space. putlongdouble() has complications to print 100% as 100% and not 100.0% (the latter is 2 characters wider, and especially wasteful). dehumanize_number() doesn't have any of the complications for the decimal point since it doesn't support floating point. systat -v (vmstat.c) is mostly written in KNF, but the patch has mounds of style bugs, e.g.: +void +shownetbw(void) +{ + double delta_time; + struct mytcpcb *elm, *telm; + int row; + + delta_time = (double)(tv_curr.tv_sec - tv_last.tv_sec) - 1.0 + + (tv_curr.tv_usec + 1000000 - tv_last.tv_usec) / 1e6; + if (delta_time < 0.1) + return; + + mvwprintw(wnd, 0, 0, + "tcp accepts %s connects %s " + " rcv %s snd %s rexmit %s", + numtok(DELTARATE(tcps_accepts)), + numtok(DELTARATE(tcps_connects) - DELTARATE(tcps_accepts)), + numtok(DELTARATE(tcps_rcvbyte)), + numtok(DELTARATE(tcps_sndbyte)), + numtok(DELTARATE(tcps_sndrexmitbyte))); + ... In KNF, the continuation indent is 4. This helps minimize line splitting, and lines are not split unnecessarily. Fixing this and other style bugs and pessimizations gives: @ struct mytcpcb *elm, *telm; @ double delta_time; @ int row; @ @ delta_time = tv_curr.tv_sec - tv_last.tv_sec + @ (tv_curr.tv_usec - tv_last.tv_usec) * 1e-6; @ if (delta_time < 0.1) @ return; @ mvwprintw(wnd, 0, 0, @ "tcp accepts %s connects %s rcv %s snd %s rexmit %s", @ numtok(DELTARATE(tcps_accepts)), @ numtok(DELTARATE(tcps_connects) - DELTARATE(tcps_accepts)), @ numtok(DELTARATE(tcps_rcvbyte)), @ numtok(DELTARATE(tcps_sndbyte)), @ numtok(DELTARATE(tcps_rcvbyte)), @ numtok(DELTARATE(tcps_sndrexmitbyte))); @ ... This maps fairly easily to putlongdouble(). You would have to pass window coordinates to each call. Maintaining these coordinates is fragile in a different way than the above. The above is a combination of hard-coded formats and field widths. It looks like it has free format except for the initial position and the spaces before "rcv", but the fixed format returned by numtok() turns the %s's into %.6s's modulo overflow bugs in numtok(). +static const char * +numtok(double value) +{ + static char buf[MAXINDEXES][32]; + static int nexti; + static const char *suffixes[] = { " ", "K", "M", "G", "T", NULL }; + int suffix = 0; + const char *fmt; Initialization in declaration. Unsorted declarations. + + while (value >= 1000.0 && suffixes[suffix+1]) { + value /= 1000.0; + ++suffix; + } Missing spaces around binary operator. The initialization in the declaration is a good obfuscation. Fixing some style bugs gives: @ /* Next line could be compressed to single characters. */ @ static const char * const suffixes[] = { @ " ", "K", "M", "G", "T", NULL }; @ static char buf[MAXINDEXES][32]; @ static int nexti; @ const char *fmt; @ int suffix; @ @ suffix = 0; @ while (value >= 1000 && suffixes[suffix + 1] != NULL) { @ value /= 1000; @ suffix++; @ } + nexti = (nexti + 1) % MAXINDEXES; + if (value < 0.001) { + fmt = " "; + } else if (value < 1.0) { + fmt = "%5.3f%s"; + } else if (value < 10.0) { + fmt = "%5.3f%s"; + } else if (value < 100.0) { + fmt = "%5.2f%s"; + } else { + fmt = "%5.1f%s"; + } Excessive braces. + snprintf(buf[nexti], sizeof(buf[nexti]), + fmt, value, suffixes[suffix]); The usual non-KNF indentation. Splitting the line isn't even needed. The usual null error handling for snprintf() failure. Errors are sort of handled by using a buffer of size 32 instead of 7 so that if the value is too large for the format it is not truncated at length 6 but messes up the format in the output. + return (buf[nexti]); +} Bruce From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 05:30:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70CD8A24 for ; Fri, 11 Jul 2014 05:30:06 +0000 (UTC) Received: from mail-vc0-f174.google.com (mail-vc0-f174.google.com [209.85.220.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 288D82EE4 for ; Fri, 11 Jul 2014 05:30:05 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id hy4so1087804vcb.5 for ; Thu, 10 Jul 2014 22:29:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=4Lr8xHAWrNLufTQB6373tJfBSuE2E9JhuHSuLTdGe0A=; b=BetZ6E+rVSbM6n0JSZp8tAo3CRYp7QWKRWXZsmMYRqhJcGIvY1rj8dWOn/QBi3C0t3 BfuIsBWfX6mCn1p8oYtApe1OdoL7qVFhBqgdNNL2xTpeSt3bgwAxrSH0m41AeHrqvvSC 2RnGbn8wNmtpHhaeg0XzczIQKmJN6VJJW/BWJXC6XwKZfgNP+nP/0vDLozBEvkt1eLhB wwNvfdegcn+AlYPHOGNldTenG4QU7IBvIDgr+0Oc2IuEQkaABHofSEWxBcFqTf3AYl/T bb91fCMFNa9ftvl5m0jwCE4asytW1qdZTg7chAqLVaWjogFasb7kUi1h5IDrN6iMDfgQ pUVA== X-Gm-Message-State: ALoCoQn7zOYnHzucfL9BPuqOWzqe/HnIgeXTiXVvJ5wqmbaG6d40htxkZibshbQRIz716BARMogk X-Received: by 10.58.165.106 with SMTP id yx10mr50926597veb.17.1405056599039; Thu, 10 Jul 2014 22:29:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.11.167 with HTTP; Thu, 10 Jul 2014 22:29:38 -0700 (PDT) X-Originating-IP: [72.177.8.109] In-Reply-To: References: From: Bryan Venteicher Date: Fri, 11 Jul 2014 00:29:38 -0500 Message-ID: Subject: Re: Add netbw option to systat To: hiren panchasara Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 05:30:06 -0000 On Wed, Jul 2, 2014 at 7:54 PM, hiren panchasara wrote: > On Wed, Jul 2, 2014 at 4:50 PM, Bryan Venteicher > wrote: > > Awhile back, DragonlyFlyBSD added a netbw option to systat that I've > ported > > to FreeBSD and found handy at various times: > > > > netbw Display aggregate and per-connection TCP receive and > transmit > > rates. Only active TCP connections are shown. > > > > Leading to output such as: > > > > tcp accepts connects rcv 1.192G snd 15.77K rexmi= t > > > > 192.168.10.80:22 192.168.10.20:23103 rcv snd 415.7 [ > NTSX ] > > 192.168.10.80:22 192.168.10.20:46560 rcv 19.80M snd 14.47K [ > NTSX ] > > 192.168.10.80:22 192.168.10.20:60699 rcv snd 886.3 [ > NTSX ] > > 192.168.10.81:5201 192.168.10.51:60844 rcv 293.2M snd [R > TSX ] > > 192.168.10.81:5201 192.168.10.51:60845 rcv 293.5M snd [R > TSX ] > > 192.168.10.81:5201 192.168.10.51:60846 rcv 293.2M snd [R > TSX ] > > 192.168.10.81:5201 192.168.10.51:60847 rcv 292.9M snd [R > TSX ] > > > > It uses the sequences number from the 'struct tcpcb' to derive the rate= s, > > which is usually good but certainly not perfect (i.e., don't set the > > interval too long). > > > > I'd like to commit this if anybody else thinks they'd find it useful. > > > > http://people.freebsd.org/~bryanv/patches/systat-netbw.patch > > I like the idea. > > A few things about the patch: > 1) You may want to remove the code hidden behind "#if 0" at 2 places. > =E2=80=8BThat's inherited as is from the =E2=80=8BDragonflyBSD code, and I = was trying to keep the diff relativity small with upstream. I'll remove it the hidden code. > 2) I am not entirely clear on why/if we need the last column with > flags but if we keep it (for compatibility of any other reason), It > would be nice to have those flags explained in the manpage: > > + mvwprintw(wnd, LINES-2, 0, > + "Rate/sec, " > + "R=3Drxpend T=3Dtxpend N=3Dnodelay T=3Dtstmp " > + "S=3Dsack X=3Dwinscale F=3Dfastrec"); > =E2=80=8BYes, I'll document them.=E2=80=8B > 3) I feel that the header line for o/p (specially 'tcp accepts and > connects' terminology) can be improved but I do not have a better > suggestion :-) > > It looks okay me otherwise and thanks for your work. > > cheers, > Hiren > From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 05:36:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44C5CB73 for ; Fri, 11 Jul 2014 05:36:53 +0000 (UTC) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF81D2F8F for ; Fri, 11 Jul 2014 05:36:52 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id hu12so1084425vcb.40 for ; Thu, 10 Jul 2014 22:36:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=PksXmLr3loTeI20ld97LyjSUVM3yaO9Zm/jidiP31LY=; b=GOJattoQXT9/hnxqd0MR+VqOiQVCXvLN0Nc6D46tbhrwZpKbiDCSr4NzahFcdpAMi5 0YjSbgLbfEl+btERvie5p3kbmyun6zv8d5+OJk/F5aeAzcaSWDjD061KggRpNtusK7Hz ktYPS4490jCCVDes4dpdt4RdalJIAEmNuXpQc2x31YXp1A/qS9vxZ7We82jDZ5mU9Cj4 hmomLlPTkPlSYecKhaH0gS31AZmsdRt7y1j2MOLnGYA7WlfVqF5r2QObLaooi9cvAAee kKC1fWCk56rjE5ONfb+6GmKCyZbjdVq6UvmHE+3zJUQ+K+/i6gb5XhhsupF8S//3y4Fb MqSA== X-Gm-Message-State: ALoCoQlOgY1W52SgEADHlMMF88owYY4n7qehwPJP8ly3OGBc+1ITLxrwKOE+2HRJogkIUHCAYBzl X-Received: by 10.220.2.136 with SMTP id 8mr5011352vcj.17.1405056611431; Thu, 10 Jul 2014 22:30:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.11.167 with HTTP; Thu, 10 Jul 2014 22:29:51 -0700 (PDT) X-Originating-IP: [72.177.8.109] In-Reply-To: <20140711133724.E969@besplex.bde.org> References: <201407101314.58558.jhb@freebsd.org> <20140711133724.E969@besplex.bde.org> From: Bryan Venteicher Date: Fri, 11 Jul 2014 00:29:51 -0500 Message-ID: Subject: Re: Add netbw option to systat To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-net@freebsd.org, hiren panchasara , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 05:36:53 -0000 On Thu, Jul 10, 2014 at 11:50 PM, Bruce Evans wrote: > On Thu, 10 Jul 2014, John Baldwin wrote: > > On Wednesday, July 02, 2014 8:54:41 pm hiren panchasara wrote: >> >>> On Wed, Jul 2, 2014 at 4:50 PM, Bryan Venteicher >>> wrote: >>> >>>> I'd like to commit this if anybody else thinks they'd find it useful. >>>> >>>> http://people.freebsd.org/~bryanv/patches/systat-netbw.patch >>>> >>> ... >> >> 4) Should numtok() just be humanize_number? Or rather, would it simplif= y >> the code to use humanize_number? (It might not, but if it does, I >> think that would be preferable.) >> > > No, nothing should use dehumanize(scientificize)_number(). It is a > badly designed API that doesn't even support unsigned numbers or > intmax_t. But numtok() takes a floating point arg. (It is always > used for rates that really do need floating point or perhaps a > quotient of integers. Except under #if 0, it is called on intger > args. It doesn't support this case since it always geenrates a > %5.Nf format with N > 0 (except for numbers < 0.001 it prints nothing, > perhaps because %f format doesn't work well for this case). > > systat already hads the better functions putint(), putfloat() and > putlongdouble(). Unfortunately, these are static in vmstat.c. They > should be used throughout systat to get a consistent format. > numtok() takes a double arg and could be handled at some cost in > efficiency by putlongdouble(). (putlongdouble() only exists due > to design errors in libdevstat. Old parts of systat -v use floats > and floats are more than adequate, but libdevstat uses long doubles. > Probably downcasting the long doubles to float would work in > systat -v, but putfloat() was cloned to create putlongdouble() > instead.) > > putlongdouble() prints the output, but numtok() formats the output for > printing and returns it in a static buffer with MAXINDEXES =3D 8 > generations so that this method is not too fragile. In general, direct > printing is easier to use, but here the output has be printed at a > certain place in the window. putlongdouble() takes coordinates for > each call and prints using move(), addch() and addstr(). It has more > control over padding characters than printf() can provide or > humanize_number() can dream of, and uses this to padd with '*' in some > cases (mainly for numbers that cannot be formatted to fit in the desired > space. Callers must pass some format info, especially the field width, > in each call. numtok() basically hard-codes a field width of 6 and a > format of %5.N%c where %c is the suffix. This format wastes 1 character > for the suffix when the suffix is ' '. putlongdouble() and even > dehumanize_number() avoid this wastage. This is more critical when the > field with is < 6. N > 0 and the decimal point also waste a lot of > space. putlongdouble() has complications to print 100% as 100% and > not 100.0% (the latter is 2 characters wider, and especially wasteful). > dehumanize_number() doesn't have any of the complications for the > decimal point since it doesn't support floating point. > > systat -v (vmstat.c) is mostly written in KNF, but the patch has mounds > of style bugs, e.g.: > > =E2=80=8BI was trying to keep the diff small with upstream. =E2=80=8BI'll m= ake a cleanup sweep and incorporate your comments. Thanks. +void > +shownetbw(void) > +{ > + double delta_time; > + struct mytcpcb *elm, *telm; > + int row; > + > + delta_time =3D (double)(tv_curr.tv_sec - tv_last.tv_sec) - 1.0 + > + (tv_curr.tv_usec + 1000000 - tv_last.tv_usec) / 1e6; > + if (delta_time < 0.1) > + return; > + > + mvwprintw(wnd, 0, 0, > + "tcp accepts %s connects %s " > + " rcv %s snd %s rexmit %s", > + numtok(DELTARATE(tcps_accepts)), > + numtok(DELTARATE(tcps_connects) - > DELTARATE(tcps_accepts)), > + numtok(DELTARATE(tcps_rcvbyte)), > + numtok(DELTARATE(tcps_sndbyte)), > + numtok(DELTARATE(tcps_sndrexmitbyte))); > + ... > > In KNF, the continuation indent is 4. This helps minimize line splitting= , > and lines are not split unnecessarily. Fixing this and other style bugs > and pessimizations gives: > > @ struct mytcpcb *elm, *telm; > @ double delta_time; > @ int row; > @ > @ delta_time =3D tv_curr.tv_sec - tv_last.tv_sec + > @ (tv_curr.tv_usec - tv_last.tv_usec) * 1e-6; > @ if (delta_time < 0.1) > @ return; > @ mvwprintw(wnd, 0, 0, > @ "tcp accepts %s connects %s rcv %s snd %s rexmit %s"= , > @ numtok(DELTARATE(tcps_accepts)), > @ numtok(DELTARATE(tcps_connects) - DELTARATE(tcps_accepts)), > @ numtok(DELTARATE(tcps_rcvbyte)), > @ numtok(DELTARATE(tcps_sndbyte)), > @ numtok(DELTARATE(tcps_rcvbyte)), > @ numtok(DELTARATE(tcps_sndrexmitbyte))); > @ ... > > This maps fairly easily to putlongdouble(). You would have to pass windo= w > coordinates to each call. Maintaining these coordinates is fragile in > a different way than the above. The above is a combination of hard-coded > formats and field widths. It looks like it has free format except for > the initial position and the spaces before "rcv", but the fixed format > returned by numtok() turns the %s's into %.6s's modulo overflow bugs > in numtok(). > > +static const char * > +numtok(double value) > +{ > + static char buf[MAXINDEXES][32]; > + static int nexti; > + static const char *suffixes[] =3D { " ", "K", "M", "G", "T", NULL= }; > + int suffix =3D 0; > + const char *fmt; > > Initialization in declaration. Unsorted declarations. > > + > + while (value >=3D 1000.0 && suffixes[suffix+1]) { > + value /=3D 1000.0; > + ++suffix; > + } > > Missing spaces around binary operator. The initialization in the > declaration > is a good obfuscation. > > Fixing some style bugs gives: > > @ /* Next line could be compressed to single characters. */ > @ static const char * const suffixes[] =3D { > @ " ", "K", "M", "G", "T", NULL }; > @ static char buf[MAXINDEXES][32]; > @ static int nexti; > @ const char *fmt; > @ int suffix; > @ > @ suffix =3D 0; > @ while (value >=3D 1000 && suffixes[suffix + 1] !=3D NULL) { > @ value /=3D 1000; > @ suffix++; > @ } > > + nexti =3D (nexti + 1) % MAXINDEXES; > + if (value < 0.001) { > + fmt =3D " "; > + } else if (value < 1.0) { > + fmt =3D "%5.3f%s"; > + } else if (value < 10.0) { > + fmt =3D "%5.3f%s"; > + } else if (value < 100.0) { > + fmt =3D "%5.2f%s"; > + } else { > + fmt =3D "%5.1f%s"; > + } > > Excessive braces. > > + snprintf(buf[nexti], sizeof(buf[nexti]), > + fmt, value, suffixes[suffix]); > > The usual non-KNF indentation. Splitting the line isn't even needed. > The usual null error handling for snprintf() failure. Errors are > sort of handled by using a buffer of size 32 instead of 7 so that if > the value is too large for the format it is not truncated at length 6 > but messes up the format in the output. > > + return (buf[nexti]); > +} > > Bruce > From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 06:50:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53F1F87E for ; Fri, 11 Jul 2014 06:50:55 +0000 (UTC) Received: from gpo3.cc.swin.edu.au (gpo3.cc.swin.edu.au [136.186.1.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1A64256D for ; Fri, 11 Jul 2014 06:50:53 +0000 (UTC) Received: from [136.186.229.154] (nwilliams-laptop.caia.swin.edu.au [136.186.229.154]) by gpo3.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s6B6ojeZ029310 for ; Fri, 11 Jul 2014 16:50:46 +1000 Message-ID: <53BF8945.3000802@swin.edu.au> Date: Fri, 11 Jul 2014 16:50:45 +1000 From: Nigel Williams User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Multipath TCP for FreeBSD v0.4 References: <513CB9AF.3090409@swin.edu.au> In-Reply-To: <513CB9AF.3090409@swin.edu.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 06:50:55 -0000 Hello all, A new v0.4 patch is available at [1]. This release is mostly bug-fixes and improvements to core functionality (establishing/closing connections, retransmissions etc), and also brings the implementation up to a more recent version of FreeBSD-HEAD. The full list of changes and caveats can be found in [2] and [3], but briefly: - Patched against r265307 of FreeBSD-HEAD. This is prior to some recent TCP reassembly memory management changes and the patch will be brought up to a newer revision soon (currently working on integrating those changes). - Added data-level retransmits and subflows can now stall and recover (or timeout) during a connection. - The path management and packet scheduler are still fairly rudimentary, and I haven't yet implemented coupled CC. - The patch is still under heavy development so consider this release code to be of alpha quality. This release ties up work that was partially supported by a gift from The Cisco University Research Program Fund. Future releases will be supported by a grant from the FreeBSD Foundation. P.S. I will be working on the patch full-time again so updates should be a little more frequent from this point onwards. cheers, nigel [1] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-changelog-v0.4.txt [3] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-readme-v0.4.txt On 11/03/13 03:49, Lawrence Stewart wrote: > Hi all, > > The CAIA MPTCP team is pleased to announce the initial release of our > multipath TCP implementation for FreeBSD 10-CURRENT which is available > from [1]. This release contains wire-related protocol code and a lot of > core stack infrastructure. It is capable of running regular TCP flows > and single or multi-subflow MPTCP flows (with some caveats as documented > in the readme [2]). > > We consider this code to be of alpha quality and plan to release > frequent updates going forward as we continue to flesh out additional > features and fix the rough edges. > > That being said, we welcome everyone to start playing with the code and > provide feedback, bug reports, fixes, praise and/or abuse ;) > > The "Multipath TCP for FreeBSD" project team consists of: > > Nigel Williams: lead R&D engineer > Lawrence Stewart: supporting R&D engineer > Grenville Armitage: principal investigator & overall project lead > > Many thanks go to the Cisco University Research Program Fund at > Community Foundation Silicon Valley for their support of this work. > > Have fun with it! > > Cheers, > Lawrence, Nigel & Grenville > > http://caia.swin.edu.au > > > > [1] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html > > [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-readme-v0.1.txt > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 10:25:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E140B71 for ; Fri, 11 Jul 2014 10:25:32 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id F244A2875 for ; Fri, 11 Jul 2014 10:25:31 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id A44EC14148B1 for ; Fri, 11 Jul 2014 12:24:27 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id D0F061A0C06 for ; Fri, 11 Jul 2014 12:24:31 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Fri, 11 Jul 2014 12:24:31 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Multipath TCP for FreeBSD v0.4 To: freebsd-net@freebsd.org References: <513CB9AF.3090409@swin.edu.au> <53BF8945.3000802@swin.edu.au> Lines: 58 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 10:25:32 -0000 Hi Nigel, Nigel Williams wrote: > A new v0.4 patch is available at [1]. [...] Thanks a lot for publishing the latest patch. Already tried it on two phyiscal machines with directly connected NICs. "iperf" looks nice: =============================================================================== #iperf -c 10.255.255.11 -i 1 ------------------------------------------------------------ Client connecting to 10.255.255.11, TCP port 5001 TCP window size: 32.5 KByte (default) ------------------------------------------------------------ [ 3] local 10.255.255.10 port 40171 connected with 10.255.255.11 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 167 MBytes 1.40 Gbits/sec [ 3] 1.0- 2.0 sec 171 MBytes 1.43 Gbits/sec [ 3] 2.0- 3.0 sec 171 MBytes 1.44 Gbits/sec [ 3] 3.0- 4.0 sec 171 MBytes 1.44 Gbits/sec [ 3] 4.0- 5.0 sec 171 MBytes 1.44 Gbits/sec [ 3] 5.0- 6.0 sec 169 MBytes 1.41 Gbits/sec [ 3] 6.0- 7.0 sec 168 MBytes 1.41 Gbits/sec [ 3] 7.0- 8.0 sec 169 MBytes 1.41 Gbits/sec [ 3] 8.0- 9.0 sec 168 MBytes 1.41 Gbits/sec [ 3] 9.0-10.0 sec 171 MBytes 1.43 Gbits/sec [ 3] 0.0-10.0 sec 1.66 GBytes 1.42 Gbits/sec =============================================================================== TCP networking is rather unstable after some "iperf" executions. So that new SSH connections aren't possible anymore. Everything more complex than "iperf" - like NFS and FTP usage - leads to a kernel panic (page fault). Do you want any crash dumps? If yes, where do you want them to be uploaded? FWIW: I haven't set up any special routings or PF rules at all: =============================================================================== MPTCP1 ------ ifconfig_em1="10.255.255.10/8 -tso" ifconfig_em0="192.168.1.1/24 -tso" ifconfig_em2="192.168.2.1/24 -tso" MPTCP2 ------ ifconfig_em1="10.255.255.11/8 -tso" ifconfig_em0="192.168.1.2/24 -tso" ifconfig_em2="192.168.2.2/24 -tso" =============================================================================== Thanks for all your work and regards, Nils From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 16:08:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9523BC7F for ; Fri, 11 Jul 2014 16:08:13 +0000 (UTC) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49E042962 for ; Fri, 11 Jul 2014 16:08:13 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id i17so1153340qcy.23 for ; Fri, 11 Jul 2014 09:08:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wVNrHOBOYHcV7mlDsMwJT60gi9iGrdsZrqUAfkASt1k=; b=G55jHficu5ZeRHp3v/kBgqpsifn9PB5jMmZCBnVMwp8NfrjclRMxUA+pWvU/0HPVEo oTNmU6bQaV299Ms8JSXDaRkR/xMR/DLv25aojtD56PTlI9oFN5krN814OOqL9lmHfsm4 1HW7k7pyX1q3zHMEBTkWVbE1YlJR8Rvgga49LN7qhtkav9G90qXfXFvPvcmeHMHAP114 qu3m2gAXWvDcpkvRoW4jm/nwhq8GHop/CB9PvxbGRhe9d3avRYq83ntMG1Idhr92BejD VkieQ3b3GMVNgcNMFyrB7xJETFwz7zs8CKKRb/d/zcjUHLq0Ek2TraADMy7sGvjdRuHb 18zw== X-Received: by 10.229.184.9 with SMTP id ci9mr23377513qcb.11.1405094892422; Fri, 11 Jul 2014 09:08:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Fri, 11 Jul 2014 09:07:31 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Fri, 11 Jul 2014 17:07:31 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 16:08:13 -0000 Luigi one question. Does netmap requires a processor with 64 bits? I'm having some trouble in compiling netmap, using the same Makefile I used previously, but for an Intel IXP435 CPU (Gateworks Cambria). On 28 June 2014 14:07, Carlos Ferreira wrote: > Hello to all. > Unfortunately, I have been unable to do the test that Luigi requested due > to the lack of hardware. All the Gateworks Cambria SBC that I have are > currently allocated to another project. Only in about one week, will I ha= ve > the opportunity to test it. > As soon as I have news, I will post them. :) > > > On 28 June 2014 09:24, Luigi Rizzo wrote: > >> >> >> On Saturday, June 28, 2014, Prashant Upadhyaya >> wrote: >> >>> Hi, >>> >>> Any further news ? >>> >>> Professor Luigi, one question regarding the tx with netmap. >>> Whenever, I write a packet from user space into netmap rings, and if I >>> want netmap to send this out immediately, do I necessarily have to do >>> a ioctl(fd, NIOCTXSYNC, NULL) ? >>> >> >> Yes it is up to the application to decide when to push packets out with = a >> txsync or select() or poll(), and unfortunately there is a tradeoff betw= een >> efficiency and latency >> >> Cheers >> Luigi >> >> >>> >>> I have an application which receives packets, does some processing and >>> then sends them out. If I keep doing ioctl's on every packet send, then >>> there will be too may system calls hitting performance, the application >>> can't afford to block it has to return back to polling for the receipt = of >>> next packet. >>> >>> On the receive side, I see that I don't have a problem because I can >>> poll the ring without initiating an RXSYNC and whenever in user space I >>> find that there is nothing on the ring (probably half way down the ring >>> size), I do an RXSYNC to get more packets thus saving system calls. >>> >>> But on tx side, I have noticed that unless I do a TXSYNC, the packet >>> does not go out, please let me know if I am missing something. >>> >>> Regards >>> -Prashant >>> >>> >>> >>> On Tue, Jun 17, 2014 at 7:33 PM, Carlos Ferreira >>> wrote: >>> >>>> Great! :) >>>> I will give you the results as soon as I can get them :) >>>> >>>> >>>> >>>> On 17 June 2014 12:55, Luigi Rizzo wrote: >>>> >>>> > On Mon, Jun 16, 2014 at 5:30 PM, Carlos Ferreira < >>>> carlosmf.pt@gmail.com> >>>> > wrote: >>>> > >>>> >> Ok, thanks for the enlightenment regarding the loss of performance. >>>> >> >>>> >> One question, just to be sure. Does the kernel module contains the >>>> VALA >>>> >> switch code? Or do I need to compile extra code to have the switch >>>> working? >>>> >> Also, where can I find the documentation to use the Vala Switch? >>>> >> >>>> > >>>> > =E2=80=8BVALE is part of the netmap kernel module, the only thing yo= u need >>>> > to know to use it is port names: >>>> > you can have multiple switch instances with multiple ports each, >>>> > >>>> > valeX:Y means port Y on switch X, X and Y are arbitrary strings >>>> > with the constraint that the whole name must fit 15 characters. >>>> > >>>> > Details in the netmap manpage >>>> > >>>> > cheers >>>> > luigi >>>> > >>>> > >>>> >>>> >>>> -- >>>> >>>> Carlos Miguel Ferreira >>>> Researcher at Telecommunications Institute >>>> Aveiro - Portugal >>>> Work E-mail - cmf@av.it.pt >>>> Skype & GTalk -> carlosmf.pt@gmail.com >>>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>> >>> >> >> -- >> -----------------------------------------+------------------------------= - >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------= - >> >> > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 16:13:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E143FBD for ; Fri, 11 Jul 2014 16:13:48 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F55B2A28 for ; Fri, 11 Jul 2014 16:13:47 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so1090176lan.12 for ; Fri, 11 Jul 2014 09:13:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ocd1/nJOhY4nGlb5G28VQ/7d2+S16/6BQz/D5KuUVFI=; b=uvEgQEG5l+XBtevkf/+n5BlBwinfJhU1FPMpziJDHnudcDXCDUbptE59M92K4nEagi XJe4byFWZBDv1QWtQ24aQwYD/5pC/2BPvbDo51rjp8fA75gJXTYRRak05SUnpnUVlahc 1kVu6dYGFOBcrWNZv4q/rL314/cEFig9clzRI6Kcwo320lbO/Q2xHdoqxaJQwSqFHoPF pSwcZA5T5BGLfoR+NbKDbLbBoMykmruXKNfIEB1Nt2SiNLpCbuKLp1DzxiY7KfdJefcC i52Ja89ekrGlUpf3N1a+aFNxx2bhSMAMS/brrjcL8tq0q16TEvWvzgPBWm9Wz5qL/vLd sckQ== MIME-Version: 1.0 X-Received: by 10.152.185.104 with SMTP id fb8mr5483725lac.64.1405095225521; Fri, 11 Jul 2014 09:13:45 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.234 with HTTP; Fri, 11 Jul 2014 09:13:45 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> Date: Fri, 11 Jul 2014 18:13:45 +0200 X-Google-Sender-Auth: fZnOFH_ddGaeCnoOBYf2XlaCirc Message-ID: Subject: Re: netmap From: Luigi Rizzo To: Carlos Ferreira Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 16:13:48 -0000 On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira wrote: > Luigi one question. Does netmap requires a processor with 64 bits? I'm > having some trouble in compiling netmap, using the same Makefile I used > previously, but for an Intel IXP435 CPU (Gateworks Cambria). > =E2=80=8Bit used to build and work on 32 bit archs but we have not tried th= at there i a while. Hopefully it is just a matter of casts in printfs. which OS and netmap versions are you using ? can you send me an error log ? cheers luigi =E2=80=8B > > From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 16:41:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF94EA77 for ; Fri, 11 Jul 2014 16:41:36 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B3DD02D2E for ; Fri, 11 Jul 2014 16:41:36 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A67C0B91E; Fri, 11 Jul 2014 12:41:33 -0400 (EDT) From: John Baldwin To: Rick Macklem Subject: Re: NFS client READ performance on -current Date: Fri, 11 Jul 2014 09:54:23 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1610703198.9975909.1405031503143.JavaMail.root@uoguelph.ca> In-Reply-To: <1610703198.9975909.1405031503143.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201407110954.23381.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 11 Jul 2014 12:41:33 -0400 (EDT) Cc: "Russell L. Carter" , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 16:41:37 -0000 On Thursday, July 10, 2014 6:31:43 pm Rick Macklem wrote: > John Baldwin wrote: > > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > > Russell L. Carter wrote: > > > > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > > > Could you please post the dmesg stuff for the network > > > > > interface, > > > > > so I can tell what driver is being used? I'll take a look at > > > > > it, > > > > > in case it needs to be changed to use m_defrag(). > > > > > > > > em0: port > > > > 0xd020-0xd03f > > > > mem > > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 > > > > on > > > > pci2 > > > > em0: Using an MSI interrupt > > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > > 001.000007 [2323] netmap_attach success for em0 tx > > > > 1/1024 > > > > rx > > > > 1/1024 queues/slots > > > > > > > > This is one of those dual nic cards, so there is em1 as well... > > > > > > > Well, I took a quick look at the driver and it does use m_defrag(), > > > but > > > I think that the "retry:" label it does a goto after doing so might > > > be in > > > the wrong place. > > > > > > The attached untested patch might fix this. > > > > > > Is it convenient to build a kernel with this patch applied and then > > > try > > > it with TSO enabled? > > > > > > rick > > > ps: It does have the transmit segment limit set to 32. I have no > > > idea if > > > this is a hardware limitation. > > > > I think the retry is not in the wrong place, but the overhead of all > > those > > pullups is apparently quite severe. > The m_defrag() call after the first failure will just barely squeeze > the just under 64K TSO segment into 32 mbuf clusters. Then I think any > m_pullup() done during the retry will allocate an mbuf > (at a glance it seems to always do this when the old mbuf is a cluster) > and prepend that to the list. > --> Now the list is > 32 mbufs again and the bus_dmammap_load_mbuf_sg() > will fail again on the retry, this time fatally, I think? > > I can't see any reason to re-do all the stuff using m_pullup() and Russell > reported that moving the "retry:" fixed his problem, from what I understood. Ah, I had assumed (incorrectly) that the m_pullup()s would all be nops in this case. It seems the NIC would really like to have all those things in a single segment, but it is not required, so I agree that your patch is fine. > > It would be interesting to test > > the > > following in addition to your change to see if it improves > > performance > > further: > > > > Index: if_em.c > > =================================================================== > > --- if_em.c (revision 268495) > > +++ if_em.c (working copy) > > @@ -1959,7 +1959,9 @@ retry: > > if (error == EFBIG && remap) { > > struct mbuf *m; > > > > - m = m_defrag(*m_headp, M_NOWAIT); > > + m = m_collapse(*m_headp, M_NOWAIT, EM_MAX_SCATTER); > > + if (m == NULL) > > + m = m_defrag(*m_headp, M_NOWAIT); > Since a just under 64K TSO segment barely fits in 32 mbuf clusters, > I'm at least 99% sure the m_collapse() will fail, but it can't hurt to > try it. (If it supported 33 or 34, I think m_collapse() would have a > reasonable chance of success.) > > Right now the NFS and krpc code creates 2 small mbufs in front of the > read/write data clusters and I think the TCP layer adds another one. > Even if this was modified to put it all in one cluster, I don't think > m_collapse() would succeed, since it only copies the data up and deletes > an mbuf from the chain if it will all fit in the preceding one. Since > the read/write data clusters are full (except the last one), they can't > fit in the M_TRAILINGSPACE() of the preceding one unless it is empty > from my reading of m_collapse(). Correct, ok. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 17:28:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AA7E58C for ; Fri, 11 Jul 2014 17:28:26 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66133221A for ; Fri, 11 Jul 2014 17:28:26 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id h18so37249igc.12 for ; Fri, 11 Jul 2014 10:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=KC8IXfalIp1wXlsoBr9z9i5Nc0DkcNndfzrgVXlzc78=; b=xRfraO1tusq/eVE141u6+fA6fjLSapYapSGjP6FXBR+6kD3Q7j0hmE3wYy7CRfroBD sbjjuzYdjqKMZet8VCHv9w8TuMU8YENPV243h8vM9Y72L1zzIwZjc82kvS8bDlKhUcH1 2TGyYe4TQwkVXjEXqLHNBjdgq2Jmb/2dnbaRvaXYy2x/tSV//tRiYLPrJQl35GJo7A3t 7RUnf5noFJUQqoR0TxjC6XoGdExVylLF8x2+iJsyLkEZ4iyF2Hb6lCS8BSMoov3l1OMp nuVfZPP0By+gn9xPwabp0WF3AiErSQmnpWLc0eTrS/+LsAD3hOL6+iCIcq/TubQSCHZ9 UhgQ== X-Received: by 10.50.80.116 with SMTP id q20mr6493233igx.22.1405099705606; Fri, 11 Jul 2014 10:28:25 -0700 (PDT) Received: from [10.1.68.187] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id dz3sm7710410igb.3.2014.07.11.10.28.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jul 2014 10:28:24 -0700 (PDT) Message-ID: <53C01EB5.6090701@gmail.com> Date: Fri, 11 Jul 2014 13:28:21 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: FreeBSD Net , Navdeep Parhar Subject: tuning routing using cxgbe and T580-CR cards? X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 17:28:26 -0000 In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, I've been able to use a collection of clients to generate approximately 1.5-1.6 million TCP packets per second sustained, and routinely hit 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the quick read, accepting the loss of granularity). While performance has so far been stellar, and I'm honestly speculating I will need more CPU depth and horsepower to get much faster, I'm curious if there is any gain to tweaking performance settings. I'm seeing, under multiple streams, with N targets connecting to N servers, interrupts on all CPUs peg at 99-100%, and I'm curious if tweaking configs will help, or its a free clue to get more horsepower. So, far, except for temporarily turning off pflogd, and setting the following sysctl variables, I've not done any performance tuning on the system yet. /etc/sysctl.conf net.inet.ip.fastforwarding=1 kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.point_to_point=0 kern.random.sys.harvest.interrupt=0 a) One of the first things I did in prior testing was to turn hyperthreading off. I presume this is still prudent, as HT doesn't help with interrupt handling? b) I briefly experimented with using cpuset(1) to stick interrupts to physical CPUs, but it offered no performance enhancements, and indeed, appeared to decrease performance by 10-20%. Has anyone else tried this? What were your results? c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx queues, with N being the number of CPUs detected. For a system running multiple cards, routing or firewalling, does this make sense, or would balancing tx and rx be more ideal? And would reducing queues per card based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? d) dev.cxl.$PORT.qsize_rxq: 1024 and dev.cxl.$PORT.qsize_txq: 1024. These appear to not be writeable when if_cxgbe is loaded, so I speculate they are not to be messed with, or are loader.conf variables? Is there any benefit to messing with them? e) dev.t5nex.$CARD.toe.sndbuf: 262144. These are writeable, but messing with values did not yield an immediate benefit. Am I barking up the wrong tree, trying? f) based on prior experiments with other vendors, I tried tweaks to net.isr.* settings, but did not see any benefits worth discussing. Am I correct in this speculation, based on others experience? g) Are there other settings I should be looking at, that may squeeze out a few more packets? Thanks in advance! -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 17:29:41 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5183A65A for ; Fri, 11 Jul 2014 17:29:41 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02BA72234 for ; Fri, 11 Jul 2014 17:29:40 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id m5so1123808qaj.23 for ; Fri, 11 Jul 2014 10:29:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=wXuGZlHIh1AzjHO6r4hGVEnV+lh+xg8ciA7xvZoGfbo=; b=j/gykQnvRDLRdULGDVV5156BrN7IN/l89i+lGjB+or6o5zReAUAUra1prYR2aaj80G 9F0VnunhIwhkRj56Qa93ONZhk8MA08aWP9EfIeH5Ag+qERkFKmcxoCJcqhorXgtZfwf9 oXIkb4Mp6vhYFVWOXz9Sh5UEsqgGHnzS8ASqJMD+Ngv6NDeO+UwwWF3ChKf/zqkVPqiv rl8KcIBUIzeGZ2G7j51QOoGOSBxb6ZkHnrkMA7zLVjmreDboz/CnDUo7dxVoJmR1MqpH lhmNmlSRc4pCvruaiD5UFGwiCCTiXJDGZOI2vHoru5rCFEdKd9R4vXPhx68q3q2BbtBx kOXg== X-Received: by 10.224.0.79 with SMTP id 15mr550828qaa.56.1405099780014; Fri, 11 Jul 2014 10:29:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Fri, 11 Jul 2014 10:27:36 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Fri, 11 Jul 2014 18:27:36 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: multipart/mixed; boundary=047d7bf0d49843228104fdee493e X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 17:29:41 -0000 --047d7bf0d49843228104fdee493e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I'm building for OpenWRT (trunk) for the IXP4xx target. Attached goes the output for the compile attempt. Maybe I'm missing something very basic... On 11 July 2014 17:13, Luigi Rizzo wrote: > > > > On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira > wrote: > >> Luigi one question. Does netmap requires a processor with 64 bits? I'm >> having some trouble in compiling netmap, using the same Makefile I used >> previously, but for an Intel IXP435 CPU (Gateworks Cambria). >> > > =E2=80=8Bit used to build and work on 32 bit archs but we have not tried = that > there i a while. > Hopefully it is just a matter of casts in printfs. > > which OS and netmap versions are you using ? > can you send me an error log ? > > cheers > luigi > =E2=80=8B > >> >> --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira --047d7bf0d49843228104fdee493e Content-Type: text/plain; charset=US-ASCII; name="output.txt" Content-Disposition: attachment; filename="output.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hxhs3per0 bWFrZSAuL3BhY2thZ2UvbmV0bWFwL2NvbXBpbGUgVj05OQ0KQ29sbGVjdGluZyBwYWNrYWdlIGlu Zm86IGRvbmUNCm1ha2VbMV06IEVudGVyaW5nIGRpcmVjdG9yeSAnL2hvbWUvb3BlbndydC9EZXNr dG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0Jw0KbWFrZVsyXTogRW50ZXJpbmcgZGlyZWN0b3J5 ICcvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvcGFja2FnZS9u ZXRtYXAnDQorKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysNCk1BS0UgICAgICAgICAgIAkgID0gbWFrZQ0KUEtHX05BTUUgICAg ICAgICAgID0gbmV0bWFwDQpQS0dfVkVSU0lPTiAgICAgICAgPSBIRUFEDQpQS0dfQlVJTERfRElS ICAgICAgPSAvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVp bGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcA0KUEtHX1NP VVJDRV9QUk9UTyAgID0gZ2l0DQpQS0dfU09VUkNFX1VSTCAgICAgPSBodHRwczovL2NvZGUuZ29v Z2xlLmNvbS9wL25ldG1hcC8NClBLR19TT1VSQ0UgICAgICAgICA9IG5ldG1hcC50YXIuZ3oNClBL R19TT1VSQ0VfVkVSU0lPTiA9IEhFQUQNClBLR19TT1VSQ0VfU1VCRElSICA9IG5ldG1hcA0KTUFL RV9PUFRTICAgICAgICAgID0gLUMgL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJp YS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9s aW51eC1peHA0eHhfZ2VuZXJpYy9saW51eC0zLjEwLjI4IFBBVEg9L2hvbWUvb3BlbndydC9EZXNr dG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L3N0YWdpbmdfZGlyL3Rvb2xjaGFpbi1hcm1lYl94 c2NhbGVfZ2NjLTQuOC1saW5hcm9fdUNsaWJjLTAuOS4zMy4yL2JpbjovaG9tZS9vcGVud3J0L0Rl c2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvc3RhZ2luZ19kaXIvaG9zdC9iaW46L2hvbWUv b3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L3N0YWdpbmdfZGlyL3Rvb2xj aGFpbi1hcm1lYl94c2NhbGVfZ2NjLTQuOC1saW5hcm9fdUNsaWJjLTAuOS4zMy4yL2JpbjovaG9t ZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvc3RhZ2luZ19kaXIvaG9z dC9iaW46L3Vzci9sb2NhbC9zYmluOi91c3IvbG9jYWwvYmluOi91c3IvYmluOi91c3IvYmluL3Np dGVfcGVybDovdXNyL2Jpbi92ZW5kb3JfcGVybDovdXNyL2Jpbi9jb3JlX3BlcmwgRE9NVUxUST0x DQpFWFRSQV9DRkxBR1MgICAgICAgPSAtSS9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNh bWJyaWEvb3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMz LjIvbmV0bWFwL0xJTlVYIC1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9v cGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRt YXAvTElOVVgvLi4vc3lzIC1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9v cGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRt YXAvTElOVVgvLi4vc3lzL2RldiAtRENPTkZJR19ORVRNQVAgLVduby11bnVzZWQtYnV0LXNldC12 YXJpYWJsZSANCkVYVFJBX0tDT05GSUcgICAgICA9IA0KTElOVVhfS01PRF9TVUZGSVggID0ga28N CkxJTlVYX0RJUiAgICAgICAgICA9IC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJy aWEvb3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMzLjIv bGludXgtaXhwNHh4X2dlbmVyaWMvbGludXgtMy4xMC4yOA0KTElOVVhfS0FSQ0ggICAgICAgID0g YXJtDQpUQVJHRVRfQ1JPU1MgICAgICAgPSBhcm1lYi1vcGVud3J0LWxpbnV4LXVjbGliY2dudWVh YmktDQpQV0QgICAgICAgICAgICAgICAgPSAvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1D YW1icmlhL29wZW53cnQNCisrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQorKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKw0KbWtkaXIgLXAgL2hvbWUvb3BlbndydC9EZXNrdG9w L09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VD bGliYy0wLjkuMzMuMi9uZXRtYXANCnRhciAgLS1kaXJlY3Rvcnk9Ii9ob21lL29wZW53cnQvRGVz a3RvcC9PcGVuV1JULUNhbWJyaWEvb3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2Fs ZV91Q2xpYmMtMC45LjMzLjIiIC16eGYgL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2Ft YnJpYS9vcGVud3J0L2RsL25ldG1hcC50YXIuZ3oNCiNjcCAtZnBSIC4vKiAvaG9tZS9vcGVud3J0 L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94 c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC8NCnRvdWNoIC9ob21lL29wZW53cnQvRGVza3Rv cC9PcGVuV1JULUNhbWJyaWEvb3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91 Q2xpYmMtMC45LjMzLjIvbmV0bWFwLy5wcmVwYXJlZF84MmZjMjQ4NDQ1ZTc0ODY1MzNlNjQ1YTUw MjNjZjNhZA0KKGNkIC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJyaWEvb3Blbndy dC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMzLjIvbmV0bWFwLy4v OyBpZiBbIC14IC4vY29uZmlndXJlIF07IHRoZW4gL3Vzci9iaW4vZmluZCAvaG9tZS9vcGVud3J0 L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94 c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC8gLW5hbWUgY29uZmlnLmd1ZXNzIHwgeGFyZ3Mg LXIgY2htb2QgdSt3OyAvdXNyL2Jpbi9maW5kIC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JU LUNhbWJyaWEvb3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45 LjMzLjIvbmV0bWFwLyAtbmFtZSBjb25maWcuZ3Vlc3MgfCB4YXJncyAtciAtbjEgY3AgL2hvbWUv b3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L3NjcmlwdHMvY29uZmlnLmd1 ZXNzOyAvdXNyL2Jpbi9maW5kIC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJyaWEv b3BlbndydC9idWlsZF9kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMzLjIvbmV0 bWFwLyAtbmFtZSBjb25maWcuc3ViIHwgeGFyZ3MgLXIgY2htb2QgdSt3OyAvdXNyL2Jpbi9maW5k IC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJyaWEvb3BlbndydC9idWlsZF9kaXIv dGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMzLjIvbmV0bWFwLyAtbmFtZSBjb25maWcu c3ViIHwgeGFyZ3MgLXIgLW4xIGNwIC9ob21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJy aWEvb3BlbndydC9zY3JpcHRzL2NvbmZpZy5zdWI7IEFSPWFybWViLW9wZW53cnQtbGludXgtdWNs aWJjZ251ZWFiaS1hciBBUz0iYXJtZWItb3BlbndydC1saW51eC11Y2xpYmNnbnVlYWJpLWdjYyAt YyAtT3MgLXBpcGUgLW1hcmNoPWFybXY1dGUgLW10dW5lPXhzY2FsZSAtZm5vLWNhbGxlci1zYXZl cyAtZmhvbm91ci1jb3B0cyAtV25vLWVycm9yPXVudXNlZC1idXQtc2V0LXZhcmlhYmxlIC1tc29m dC1mbG9hdCIgTEQ9YXJtZWItb3BlbndydC1saW51eC11Y2xpYmNnbnVlYWJpLWxkIE5NPWFybWVi LW9wZW53cnQtbGludXgtdWNsaWJjZ251ZWFiaS1ubSBDQz0iYXJtZWItb3BlbndydC1saW51eC11 Y2xpYmNnbnVlYWJpLWdjYyIgR0NDPSJhcm1lYi1vcGVud3J0LWxpbnV4LXVjbGliY2dudWVhYmkt Z2NjIiBDWFg9ImFybWViLW9wZW53cnQtbGludXgtdWNsaWJjZ251ZWFiaS1nKysiIFJBTkxJQj1h cm1lYi1vcGVud3J0LWxpbnV4LXVjbGliY2dudWVhYmktcmFubGliIFNUUklQPWFybWViLW9wZW53 cnQtbGludXgtdWNsaWJjZ251ZWFiaS1zdHJpcCBPQkpDT1BZPWFybWViLW9wZW53cnQtbGludXgt dWNsaWJjZ251ZWFiaS1vYmpjb3B5IE9CSkRVTVA9YXJtZWItb3BlbndydC1saW51eC11Y2xpYmNn bnVlYWJpLW9iamR1bXAgU0laRT1hcm1lYi1vcGVud3J0LWxpbnV4LXVjbGliY2dudWVhYmktc2l6 ZSBDRkxBR1M9Ii1PcyAtcGlwZSAtbWFyY2g9YXJtdjV0ZSAtbXR1bmU9eHNjYWxlIC1mbm8tY2Fs bGVyLXNhdmVzIC1maG9ub3VyLWNvcHRzIC1Xbm8tZXJyb3I9dW51c2VkLWJ1dC1zZXQtdmFyaWFi bGUgLW1zb2Z0LWZsb2F0IC1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9v cGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRt YXAvTElOVVggLUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQv YnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC9MSU5V WC8uLi9zeXMgLUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQv YnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC9MSU5V WC8uLi9zeXMvZGV2IC1EQ09ORklHX05FVE1BUCAtV25vLXVudXNlZC1idXQtc2V0LXZhcmlhYmxl ICIgQ1hYRkxBR1M9Ii1PcyAtcGlwZSAtbWFyY2g9YXJtdjV0ZSAtbXR1bmU9eHNjYWxlIC1mbm8t Y2FsbGVyLXNhdmVzIC1maG9ub3VyLWNvcHRzIC1Xbm8tZXJyb3I9dW51c2VkLWJ1dC1zZXQtdmFy aWFibGUgLW1zb2Z0LWZsb2F0IC1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJp YS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9u ZXRtYXAvTElOVVggLUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53 cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC9M SU5VWC8uLi9zeXMgLUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53 cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC9M SU5VWC8uLi9zeXMvZGV2IC1EQ09ORklHX05FVE1BUCAtV25vLXVudXNlZC1idXQtc2V0LXZhcmlh YmxlICIgQ1BQRkxBR1M9Ii1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9v cGVud3J0L3N0YWdpbmdfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL3Vz ci9pbmNsdWRlIC1JL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0 L3N0YWdpbmdfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL2luY2x1ZGUg LUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvc3RhZ2luZ19k aXIvdG9vbGNoYWluLWFybWViX3hzY2FsZV9nY2MtNC44LWxpbmFyb191Q2xpYmMtMC45LjMzLjIv dXNyL2luY2x1ZGUgLUkvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53 cnQvc3RhZ2luZ19kaXIvdG9vbGNoYWluLWFybWViX3hzY2FsZV9nY2MtNC44LWxpbmFyb191Q2xp YmMtMC45LjMzLjIvaW5jbHVkZSAiIExERkxBR1M9Ii1ML2hvbWUvb3BlbndydC9EZXNrdG9wL09w ZW5XUlQtQ2FtYnJpYS9vcGVud3J0L3N0YWdpbmdfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNs aWJjLTAuOS4zMy4yL3Vzci9saWIgLUwvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1i cmlhL29wZW53cnQvc3RhZ2luZ19kaXIvdGFyZ2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMz LjIvbGliIC1ML2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L3N0 YWdpbmdfZGlyL3Rvb2xjaGFpbi1hcm1lYl94c2NhbGVfZ2NjLTQuOC1saW5hcm9fdUNsaWJjLTAu OS4zMy4yL3Vzci9saWIgLUwvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29w ZW53cnQvc3RhZ2luZ19kaXIvdG9vbGNoYWluLWFybWViX3hzY2FsZV9nY2MtNC44LWxpbmFyb191 Q2xpYmMtMC45LjMzLjIvbGliICIgICAuL2NvbmZpZ3VyZSAtLXRhcmdldD1hcm1lYi1vcGVud3J0 LWxpbnV4IC0taG9zdD1hcm1lYi1vcGVud3J0LWxpbnV4IC0tYnVpbGQ9aTY4Ni1wYy1saW51eC1n bnUgLS1wcm9ncmFtLXByZWZpeD0iIiAtLXByb2dyYW0tc3VmZml4PSIiIC0tcHJlZml4PS91c3Ig LS1leGVjLXByZWZpeD0vdXNyIC0tYmluZGlyPS91c3IvYmluIC0tc2JpbmRpcj0vdXNyL3NiaW4g LS1saWJleGVjZGlyPS91c3IvbGliIC0tc3lzY29uZmRpcj0vZXRjIC0tZGF0YWRpcj0vdXNyL3No YXJlIC0tbG9jYWxzdGF0ZWRpcj0vdmFyIC0tbWFuZGlyPS91c3IvbWFuIC0taW5mb2Rpcj0vdXNy L2luZm8gLS1kaXNhYmxlLW5scyAgLS1kaXNhYmxlLWlwdjYgOyBmaTsgKQ0Kcm0gLWYgL2hvbWUv b3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQt YXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRtYXAvLmNvbmZpZ3VyZWRfKg0KdG91Y2gg L2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2J1aWxkX2Rpci90 YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRtYXAvLmNvbmZpZ3VyZWRfDQpF bnRlcmluZyBCdWlsZC9Db21waWxlDQpjZCAvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1D YW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4z My4yL25ldG1hcC9MSU5VWA0KbWFrZSAtQyAvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1D YW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4z My4yL2xpbnV4LWl4cDR4eF9nZW5lcmljL2xpbnV4LTMuMTAuMjggRVhUUkFfQ0ZMQUdTPSctSS9o b21lL29wZW53cnQvRGVza3RvcC9PcGVuV1JULUNhbWJyaWEvb3BlbndydC9idWlsZF9kaXIvdGFy Z2V0LWFybWViX3hzY2FsZV91Q2xpYmMtMC45LjMzLjIvbmV0bWFwL0xJTlVYIC1JL2hvbWUvb3Bl bndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJt ZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRtYXAvTElOVVgvLi4vc3lzIC1JL2hvbWUvb3Bl bndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJt ZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9uZXRtYXAvTElOVVgvLi4vc3lzL2RldiAtRENPTkZJ R19ORVRNQVAgLVduby11bnVzZWQtYnV0LXNldC12YXJpYWJsZSAnIG1vZHVsZXMNCm1ha2VbM106 IEVudGVyaW5nIGRpcmVjdG9yeSAnL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJp YS9vcGVud3J0L2J1aWxkX2Rpci90YXJnZXQtYXJtZWJfeHNjYWxlX3VDbGliYy0wLjkuMzMuMi9s aW51eC1peHA0eHhfZ2VuZXJpYy9saW51eC0zLjEwLjI4Jw0KbWFrZVs0XTogTm90aGluZyB0byBi ZSBkb25lIGZvciAnYWxsJy4NCm1ha2VbNF06IE5vdGhpbmcgdG8gYmUgZG9uZSBmb3IgJ3JlbG9j cycuDQogIENISyAgICAgaW5jbHVkZS9nZW5lcmF0ZWQvdWFwaS9saW51eC92ZXJzaW9uLmgNCiAg Q0hLICAgICBpbmNsdWRlL2dlbmVyYXRlZC91dHNyZWxlYXNlLmgNCiAgQ0MgICAgICBrZXJuZWwv Ym91bmRzLnMNCmtlcm5lbC9ib3VuZHMuYzoxOjA6IGVycm9yOiBjb2RlIG1vZGVsICdrZXJuZWwn IG5vdCBzdXBwb3J0ZWQgaW4gdGhlIDMyIGJpdCBtb2RlDQogLyoNCiBeDQprZXJuZWwvYm91bmRz LmM6MTowOiBzb3JyeSwgdW5pbXBsZW1lbnRlZDogNjQtYml0IG1vZGUgbm90IGNvbXBpbGVkIGlu DQovaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGly L3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL2xpbnV4LWl4cDR4eF9nZW5lcmlj L2xpbnV4LTMuMTAuMjgvLi9LYnVpbGQ6MzU6IHJlY2lwZSBmb3IgdGFyZ2V0ICdrZXJuZWwvYm91 bmRzLnMnIGZhaWxlZA0KbWFrZVs0XTogKioqIFtrZXJuZWwvYm91bmRzLnNdIEVycm9yIDENCk1h a2VmaWxlOjgzNTogcmVjaXBlIGZvciB0YXJnZXQgJ3ByZXBhcmUwJyBmYWlsZWQNCm1ha2VbM106 ICoqKiBbcHJlcGFyZTBdIEVycm9yIDINCm1ha2VbM106IExlYXZpbmcgZGlyZWN0b3J5ICcvaG9t ZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3Rhcmdl dC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL2xpbnV4LWl4cDR4eF9nZW5lcmljL2xpbnV4 LTMuMTAuMjgnDQpNYWtlZmlsZToxMjI6IHJlY2lwZSBmb3IgdGFyZ2V0ICcvaG9tZS9vcGVud3J0 L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGlyL3RhcmdldC1hcm1lYl94 c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC8uYnVpbHQnIGZhaWxlZA0KbWFrZVsyXTogKioq IFsvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3BlbldSVC1DYW1icmlhL29wZW53cnQvYnVpbGRfZGly L3RhcmdldC1hcm1lYl94c2NhbGVfdUNsaWJjLTAuOS4zMy4yL25ldG1hcC8uYnVpbHRdIEVycm9y IDINCm1ha2VbMl06IExlYXZpbmcgZGlyZWN0b3J5ICcvaG9tZS9vcGVud3J0L0Rlc2t0b3AvT3Bl bldSVC1DYW1icmlhL29wZW53cnQvcGFja2FnZS9uZXRtYXAnDQpwYWNrYWdlL01ha2VmaWxlOjE2 MjogcmVjaXBlIGZvciB0YXJnZXQgJ3BhY2thZ2UvbmV0bWFwL2NvbXBpbGUnIGZhaWxlZA0KbWFr ZVsxXTogKioqIFtwYWNrYWdlL25ldG1hcC9jb21waWxlXSBFcnJvciAyDQptYWtlWzFdOiBMZWF2 aW5nIGRpcmVjdG9yeSAnL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVu d3J0Jw0KL2hvbWUvb3BlbndydC9EZXNrdG9wL09wZW5XUlQtQ2FtYnJpYS9vcGVud3J0L2luY2x1 ZGUvdG9wbGV2ZWwubWs6MTYwOiByZWNpcGUgZm9yIHRhcmdldCAncGFja2FnZS9uZXRtYXAvY29t cGlsZScgZmFpbGVkDQptYWtlOiAqKiogW3BhY2thZ2UvbmV0bWFwL2NvbXBpbGVdIEVycm9yIDIN CltvcGVud3J0QEFyY2hMaW51eDMyQml0cyBvcGVud3J0XSQgDQo= --047d7bf0d49843228104fdee493e-- From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 17:44:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1A73A1E for ; Fri, 11 Jul 2014 17:44:04 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F8E723CA for ; Fri, 11 Jul 2014 17:44:04 +0000 (UTC) Received: by mail-ig0-f174.google.com with SMTP id c1so54766igq.7 for ; Fri, 11 Jul 2014 10:44:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=E6K/aRWkLM132p8V0h1RxTgSyiIZ1rMmzhbaeKaircU=; b=ep1y+HaGzc45S9LCIQzRnSGA+oAWbA3KrkO9L3bARpdLRHoMIeozEEMMsfGv9+1WYV U7JMaxOiw3uDAn/dMYueXPcQMdGHXdK85dE1ePnI8ljLltuT2XiyBl9tNchvDMrlrxA8 b4CfYeGoNzIxyp9tQiKavCBwvTplKjkADUC96Des4QTumt2M//QuuxIvaCMR2TlPTGri ZuyzBcAbKoM/3paubkKKomWchqCoVfhUIKnieF28/BOMwCfv/ZqjjCkiDdc14ZRp1d2t xH3oSMfERpZGjzcekMyB51XueOodE0ygTt1YQueLofs0+xX9LUfC5nPMY53ayiXCITLp R4Dw== X-Received: by 10.50.51.3 with SMTP id g3mr6536086igo.34.1405100643927; Fri, 11 Jul 2014 10:44:03 -0700 (PDT) Received: from [10.1.68.187] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id u6sm7797153igs.12.2014.07.11.10.43.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jul 2014 10:44:00 -0700 (PDT) Message-ID: <53C0225D.7060608@gmail.com> Date: Fri, 11 Jul 2014 13:43:57 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: FreeBSD Net Subject: re: Network Intel X520-SR2 stopping X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 17:44:04 -0000 Marcelo; I recently had a case where an Intel card was flapping, but using LR transceivers. Turns out, the cable ends needed to be re-polished, as not enough light was making it through to register transmit power. You and the networking people may want to spend a few moments exploring that path. -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 18:04:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B54362E5 for ; Fri, 11 Jul 2014 18:04:13 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 686B925BF for ; Fri, 11 Jul 2014 18:04:12 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 2C2E825D37C7; Fri, 11 Jul 2014 18:04:10 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 51908C22C26; Fri, 11 Jul 2014 18:04:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id tGZOqRsmF2f9; Fri, 11 Jul 2014 18:04:07 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:80b3:3e87:431:f220] (unknown [IPv6:fde9:577b:c1a9:4410:80b3:3e87:431:f220]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 8CA68C22B9F; Fri, 11 Jul 2014 18:04:05 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: tuning routing using cxgbe and T580-CR cards? From: "Bjoern A. Zeeb" In-Reply-To: <53C01EB5.6090701@gmail.com> Date: Fri, 11 Jul 2014 18:03:45 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> References: <53C01EB5.6090701@gmail.com> To: John Jasem X-Mailer: Apple Mail (2.1878.6) Cc: FreeBSD Net , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 18:04:13 -0000 On 11 Jul 2014, at 17:28 , John Jasem wrote: > c) the defaults for the cxgbe driver appear to be 8 rx queues, and N = tx > queues, with N being the number of CPUs detected. For a system running > multiple cards, routing or firewalling, does this make sense, or would > balancing tx and rx be more ideal? And would reducing queues per card > based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? > =85 > g) Are there other settings I should be looking at, that may squeeze = out > a few more packets? If you are primarily forwarding packets (you say =93routing=94 multiple = times) the first thing you should do is turn off LRO and TSO on all = ports. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 18:07:57 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B32653A for ; Fri, 11 Jul 2014 18:07:57 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DD2025FA for ; Fri, 11 Jul 2014 18:07:57 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so1868512pab.36 for ; Fri, 11 Jul 2014 11:07:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=M7ovO7L4zdfh2DsNJmhmuqmjJTHWhkLjokDtQzfm3wQ=; b=BoJDUqKYZke0q/HbpjDlcx7wEpvWOhNTZpNvca1P8CnZjWxABOyGRlnIcg7KFdRSMn LxjYQFRIRwVLD67shp6Ly2i5V+7BD2ViF++Oj4FHh+8cPj1Dad3LM3WadFnBfoZgFNH7 D19b0ufxOBYQ7sMcZklzbIEOl8x4E0cAAph+CKlUtiYv5FIxgVEwfVXr7qrZ+lEi3pWD 5g9ozvDgIlR/xOD5TBtwa7REdbFkTqw97AtSMFKJ2CT/7kbmYCRAMLE6uaXXzBSA5dAY gDKIP0+Wj1gTROTA3BT4TFtljOIOH3C2/kFzkKCzBoDn9EGJqFhtJeR1Wy0+6B44YqHM fsqg== X-Received: by 10.68.171.193 with SMTP id aw1mr271843pbc.117.1405102076811; Fri, 11 Jul 2014 11:07:56 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id xc1sm11840748pab.39.2014.07.11.11.07.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jul 2014 11:07:56 -0700 (PDT) Message-ID: <53C027FB.7060807@gmail.com> Date: Fri, 11 Jul 2014 11:07:55 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Bjoern A. Zeeb" , John Jasem Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> In-Reply-To: <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 18:07:57 -0000 On 07/11/14 11:03, Bjoern A. Zeeb wrote: > On 11 Jul 2014, at 17:28 , John Jasem wrote: >=20 >> c) the defaults for the cxgbe driver appear to be 8 rx queues, and >> N tx queues, with N being the number of CPUs detected. For a system >> running multiple cards, routing or firewalling, does this make >> sense, or would balancing tx and rx be more ideal? And would >> reducing queues per card based on NUMBER-CPUS and NUM-CHELSIO-PORTS >> make sense at all? =85 g) Are there other settings I should be >> looking at, that may squeeze out a few more packets? >=20 > If you are primarily forwarding packets (you say =93routing=94 multiple= > times) the first thing you should do is turn off LRO and TSO on all > ports. LRO, sure. But TSO shouldn't really matter unless the packets originate from a local TCP endpoint on the system. Navdeep >=20 > =97 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames,= > 1983 >=20 From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 19:32:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 242C4D8A for ; Fri, 11 Jul 2014 19:32:07 +0000 (UTC) Received: from mail-pd0-x22c.google.com (mail-pd0-x22c.google.com [IPv6:2607:f8b0:400e:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED6C12EB2 for ; Fri, 11 Jul 2014 19:32:06 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id w10so1883259pde.17 for ; Fri, 11 Jul 2014 12:32:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=wPewy67AoXmw0jDSxiXnFZpaMkIkolNDR8/c5NkbGFs=; b=X0OUwrpWC37VcicxaUsL8afOqmEO8HelmNtSustH1ZdPMrIOERjsbpIMMG0NxJ/xtC SeVgS48LstbRE0eqVm7uFtzjqC9ID7bDaOTBVT9Oi15N9h59aBNz7sWo+tryPgz9dn0p JZGBRHGyfBdVykczgEK47NDJCKAASeO9yZjp66IkAkvBt4Wz1w1MYLjsWi1UXaJw858a abZvCHzW5sBQaxcHzc9kRYx/LMtfe1t0oHKlV7obTWbJz/OJ8iuPwu7MD/3gcBMP/qGX MglazFUaedOhHUXHRiBU04BUPcOwbKERxCw+7O1pbO7jdAbvBYzxfHEjjVJ7lGNSKlzb NlIg== X-Received: by 10.66.254.166 with SMTP id aj6mr839363pad.11.1405107126522; Fri, 11 Jul 2014 12:32:06 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id z2sm4058909pdp.91.2014.07.11.12.32.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jul 2014 12:32:05 -0700 (PDT) Message-ID: <53C03BB4.2090203@gmail.com> Date: Fri, 11 Jul 2014 12:32:04 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Jasem , FreeBSD Net Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> In-Reply-To: <53C01EB5.6090701@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 19:32:07 -0000 On 07/11/14 10:28, John Jasem wrote: > In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, > I've been able to use a collection of clients to generate approximately > 1.5-1.6 million TCP packets per second sustained, and routinely hit > 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the > quick read, accepting the loss of granularity). When forwarding, the pps rate is often more interesting, and almost always the limiting factor, as compared to the total amount of data being passed around. 10GB at this pps probably means 9000 MTU. Try with 1500 too if possible. "netstat -d 1" and "vmstat 1" for a few seconds when your system is under maximum load would be useful. And what kind of CPU is in this system? > > While performance has so far been stellar, and I'm honestly speculating > I will need more CPU depth and horsepower to get much faster, I'm > curious if there is any gain to tweaking performance settings. I'm > seeing, under multiple streams, with N targets connecting to N servers, > interrupts on all CPUs peg at 99-100%, and I'm curious if tweaking > configs will help, or its a free clue to get more horsepower. > > So, far, except for temporarily turning off pflogd, and setting the > following sysctl variables, I've not done any performance tuning on the > system yet. > > /etc/sysctl.conf > net.inet.ip.fastforwarding=1 > kern.random.sys.harvest.ethernet=0 > kern.random.sys.harvest.point_to_point=0 > kern.random.sys.harvest.interrupt=0 > > a) One of the first things I did in prior testing was to turn > hyperthreading off. I presume this is still prudent, as HT doesn't help > with interrupt handling? It is always worthwhile to try your workload with and without hyperthreading. > b) I briefly experimented with using cpuset(1) to stick interrupts to > physical CPUs, but it offered no performance enhancements, and indeed, > appeared to decrease performance by 10-20%. Has anyone else tried this? > What were your results? > > c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx > queues, with N being the number of CPUs detected. For a system running > multiple cards, routing or firewalling, does this make sense, or would > balancing tx and rx be more ideal? And would reducing queues per card > based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? The defaults are nrxq = min(8, ncores) and ntxq = min(16, ncores). The man page mentions this. The reason for 8 vs. 16 is that tx queues are "cheaper" as they don't have to be backed by rx buffers. It only needs some memory for the tx descriptor ring and some hardware resources. It appears that your system has >= 16 cores. For forwarding it probably makes sense to have nrxq = ntxq. If you're left with 8 or fewer cores after disabling hyperthreading you'll automatically get 8 rx and tx queues. Otherwise you'll have to fiddle with the hw.cxgbe.nrxq10g and ntxq10g tunables (documented in the man page). > d) dev.cxl.$PORT.qsize_rxq: 1024 and dev.cxl.$PORT.qsize_txq: 1024. > These appear to not be writeable when if_cxgbe is loaded, so I speculate > they are not to be messed with, or are loader.conf variables? Is there > any benefit to messing with them? Can't change them after the port has been administratively brought up even once. This is mentioned in the man page. I don't really recommend changing them any way. > > e) dev.t5nex.$CARD.toe.sndbuf: 262144. These are writeable, but messing > with values did not yield an immediate benefit. Am I barking up the > wrong tree, trying? The TOE tunables won't make a difference unless you have enabled TOE, the TCP endpoints lie on the system, and the connections are being handled by the TOE on the chip. This is not the case on your systems. The driver does not enable TOE by default and the only way to use it is to switch it on explicitly. There is no possibility that you're using it without knowing that you are. > > f) based on prior experiments with other vendors, I tried tweaks to > net.isr.* settings, but did not see any benefits worth discussing. Am I > correct in this speculation, based on others experience? > > g) Are there other settings I should be looking at, that may squeeze out > a few more packets? The pps rates that you've observed are within the chip's hardware limits by at least an order of magnitude. Tuning the kernel rather than the driver may be the best bang for your buck. Regards, Navdeep > > Thanks in advance! > > -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Fri Jul 11 21:55:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 784B48D8 for ; Fri, 11 Jul 2014 21:55:20 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DEA42AF6 for ; Fri, 11 Jul 2014 21:55:20 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id w7so1577062qcr.35 for ; Fri, 11 Jul 2014 14:55:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=hlo7phhdwCYLifgsmOm8eq0U0Ca1bcyQgTbw9GLz7pg=; b=y321RmCRjPS19PxFhQ/LD4uuOLRRTJKM3yf+uE9niUzekgJFZrCImR56nwR1BuFh6F mE9xcNbBQpg3dOKW5CwsMkMT2nqr/aTHqZ9/XAE6TFJsvtOUA/3zp2h4uLM2//0ZOeUy KTAVYuOePsl3lHOTC490mao+jXn2tpW0UkhBa/XKj/40TQF7+BommD4ROgSteHHLxZL+ RwngzeQsLN9KXIlT6JhFMEEFgg4UHUcpuR2BNLwYQGPLwTbnEXm9O8SWAQV+vWeaUmlG jgvfhVwEAo2OiB0GyXKRD1va+9w03qDUdH1snBoWorbuGQZQ+l4itVdkcKqPyeiT+IwY x+rQ== X-Received: by 10.224.76.1 with SMTP id a1mr2056888qak.4.1405115719339; Fri, 11 Jul 2014 14:55:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Fri, 11 Jul 2014 14:54:39 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Fri, 11 Jul 2014 22:54:39 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 21:55:20 -0000 OK, ignore what I said in the last e-mail. My Makefile is nor working properly and I'm trying to figure out why. OpenWRT documentation for module Makefiles creation is scarce and confuse... On 11 July 2014 18:27, Carlos Ferreira wrote: > I'm building for OpenWRT (trunk) for the IXP4xx target. > > Attached goes the output for the compile attempt. Maybe I'm missing > something very basic... > > > On 11 July 2014 17:13, Luigi Rizzo wrote: > >> >> >> >> On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira >> wrote: >> >>> Luigi one question. Does netmap requires a processor with 64 bits? I'm >>> having some trouble in compiling netmap, using the same Makefile I used >>> previously, but for an Intel IXP435 CPU (Gateworks Cambria). >>> >> >> =E2=80=8Bit used to build and work on 32 bit archs but we have not tried= that >> there i a while. >> Hopefully it is just a matter of casts in printfs. >> >> which OS and netmap versions are you using ? >> can you send me an error log ? >> >> cheers >> luigi >> =E2=80=8B >> >>> >>> > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 00:58:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EED35E20 for ; Sat, 12 Jul 2014 00:58:27 +0000 (UTC) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA8332A62 for ; Sat, 12 Jul 2014 00:58:27 +0000 (UTC) Received: by mail-ig0-f176.google.com with SMTP id r10so45038igi.15 for ; Fri, 11 Jul 2014 17:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=vQ4nFFjxfmo/2ROBBnmvQNwkrkGxf3pK64OC8ll96tQ=; b=fYogbGntyAiyedjZOHO3nWP6HjmvtqbNwKsvxJpHuMp/4lO6RdhqRXpKLcMBUzfh1D W+bF1XN65Xj7oMuvY5sGiWAz+9oFZHrbK5EPjCqYzZcife1BIEGJQrO/1CtH3yn07UqE 3mzUogFB771Js363NqFe4tFNnNCuYh+vGrMEglqlqWT+zJneBUraa1yce7x7YwRYyEYZ lnaYTwCNTWMy3s5EShcTAfTxao1v5l6Q0IEpxhb/Bp7/s8ZINcquehh1Wcj9A+R/dkwx ck3N8GumdYeYPXsg05PZGqVtmIAenPIq9yue7EQP+WqscXe36efkaljT9uqs0dt8aQCw +ulA== X-Received: by 10.50.112.132 with SMTP id iq4mr8826387igb.45.1405126707002; Fri, 11 Jul 2014 17:58:27 -0700 (PDT) Received: from [10.0.0.215] ([96.234.167.12]) by mx.google.com with ESMTPSA id ro10sm661350igb.18.2014.07.11.17.58.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jul 2014 17:58:23 -0700 (PDT) Message-ID: <53C0882D.5070100@gmail.com> Date: Fri, 11 Jul 2014 20:58:21 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Navdeep Parhar , FreeBSD Net Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> <53C03BB4.2090203@gmail.com> In-Reply-To: <53C03BB4.2090203@gmail.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 00:58:28 -0000 On 07/11/2014 03:32 PM, Navdeep Parhar wrote: > On 07/11/14 10:28, John Jasem wrote: >> In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, >> I've been able to use a collection of clients to generate approximately >> 1.5-1.6 million TCP packets per second sustained, and routinely hit >> 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the >> quick read, accepting the loss of granularity). > When forwarding, the pps rate is often more interesting, and almost > always the limiting factor, as compared to the total amount of data > being passed around. 10GB at this pps probably means 9000 MTU. Try > with 1500 too if possible. Yes, I am generally more interested/concerned with the pps. Using 1500-sized packets, I've seen around 2 million pps. I'll have hard numbers for the list, with netstat and vmstat output Monday. >> a) One of the first things I did in prior testing was to turn >> hyperthreading off. I presume this is still prudent, as HT doesn't help >> with interrupt handling? > It is always worthwhile to try your workload with and without > hyperthreading. Testing Mellanox cards, HT was severely detrimental. However, in almost every case so far, Mellanox and Chelsio have resulted in opposite conclusions (cpufreq, net.isr.*). >> c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx >> queues, with N being the number of CPUs detected. For a system running >> multiple cards, routing or firewalling, does this make sense, or would >> balancing tx and rx be more ideal? And would reducing queues per card >> based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? > The defaults are nrxq = min(8, ncores) and ntxq = min(16, ncores). The > man page mentions this. The reason for 8 vs. 16 is that tx queues are > "cheaper" as they don't have to be backed by rx buffers. It only needs > some memory for the tx descriptor ring and some hardware resources. > > It appears that your system has >= 16 cores. For forwarding it probably > makes sense to have nrxq = ntxq. If you're left with 8 or fewer cores > after disabling hyperthreading you'll automatically get 8 rx and tx > queues. Otherwise you'll have to fiddle with the hw.cxgbe.nrxq10g and > ntxq10g tunables (documented in the man page). I promise I did look through the man page before posting. :) This is actually a 12 core box with HT turned off. Mining the cxl stat entries in sysctl, it appears that the queues per port are reasonably well balanced, so I may be concerned over nothing. >> g) Are there other settings I should be looking at, that may squeeze out >> a few more packets? > The pps rates that you've observed are within the chip's hardware limits > by at least an order of magnitude. Tuning the kernel rather than the > driver may be the best bang for your buck. If I am missing obvious configurations for kernel tuning in this regard, it would not the be the first time. Thanks again! -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 02:32:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE230E95 for ; Sat, 12 Jul 2014 02:32:37 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ACEB62203 for ; Sat, 12 Jul 2014 02:32:37 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 1634D139CA for ; Fri, 11 Jul 2014 23:32:34 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1405132353; x=1405996354; bh=+yjLHaQ36GqD+/Wg9eMCJNMJtxpxZX+uPvk sJVrEeqo=; b=UGS2x790AVayAhJ18vAtTnpdkPHhwYnrlYl5gIWT7FHwIKQ+pgQ K91L9CptVkCTwYSa8CIC7n1mOq2Y0xsDgWJEeOjpgyhA3GghZlbWbu4gFOnQV33m TyLBEDpsOXnDayWO8ENyAkOgFyDh9YESM2hPdx0VKX8j3InnKpBJUMVU= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x6yaGL1BXiaY for ; Fri, 11 Jul 2014 23:32:33 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id E2004139C9; Fri, 11 Jul 2014 23:32:32 -0300 (BRT) Message-ID: <53C09E38.1020909@bsdinfo.com.br> Date: Fri, 11 Jul 2014 23:32:24 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Jasem , FreeBSD Net Subject: Re: Network Intel X520-SR2 stopping References: <53C0225D.7060608@gmail.com> In-Reply-To: <53C0225D.7060608@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 02:32:37 -0000 Em 11/07/2014 14:43, John Jasem escreveu: > Marcelo; > > I recently had a case where an Intel card was flapping, but using LR > transceivers. Turns out, the cable ends needed to be re-polished, as not > enough light was making it through to register transmit power. > > You and the networking people may want to spend a few moments exploring > that path. > > -- John Jasen (jjasen@gmail.com) Hi John, In my case I had to create a script for ping test and if ping not respond then to do a down and up in interface. #!/bin/sh while true ; do if ! ping -n -c 1 186.xxx.48.2; then if ! ping -n -c 1 186.xxx.61.2; then if ! ping -n -c 1 186.xxx.54.2; then if ! ping -n -c 1 177.xxx.240.253; then /sbin/ifconfig ix0 down /sbin/ifconfig ix0 up echo "`date`" >> /root/ix.log echo "`netstat -in|grep ix0|grep 1500`" >> /root/ix.log fi fi fi fi sleep 10 done From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 06:05:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A2F3A0F; Sat, 12 Jul 2014 06:05:50 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 387E2229C; Sat, 12 Jul 2014 06:05:50 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id y10so2496985pdj.19 for ; Fri, 11 Jul 2014 23:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ZEeVvYtnJMf5nYAaIkTWIgRZ7PFzns+5QSKIZMGKdg0=; b=mWTbaNXam7Ldgw/hZjWhPucoQ1G31Qobin96apYj66rDRZ818mQvPv2JYMgd/hj2DE b15Pa69k3nCOEdJxOhvZAVLW2sYND3OoJswr243lYh4trZST6uNe0rF7x1zC3o5SPA0n GwD4kKx1vwRQruQ7/cjmNpWEflojQuIqIj0c3gypOaAoHdiSaJM56A4YbNwM/vnoFZhN 2mUbag/1silDsLomF0Cf0TXm0odWzZUlH1JySGpyHpBYyKoVPGn9XwnPgux0xwvnKtM4 uNc4z1uqbV6Z8fIbQR7l35SnCfE26RTHdAubxOwKTLc5rvzm3RWSZufdn9c9ZKrei9F0 rKkg== X-Received: by 10.70.0.48 with SMTP id 16mr3696096pdb.8.1405145149742; Fri, 11 Jul 2014 23:05:49 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by mx.google.com with ESMTPSA id qn13sm5502173pdb.69.2014.07.11.23.05.46 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 11 Jul 2014 23:05:49 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Sat, 12 Jul 2014 15:05:39 +0900 Date: Sat, 12 Jul 2014 15:05:38 +0900 To: John Baldwin Subject: Re: NFS client READ performance on -current Message-ID: <20140712060538.GA3649@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <1610703198.9975909.1405031503143.JavaMail.root@uoguelph.ca> <201407110954.23381.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201407110954.23381.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: "Russell L. Carter" , freebsd-net@freebsd.org, Rick Macklem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 06:05:50 -0000 On Fri, Jul 11, 2014 at 09:54:23AM -0400, John Baldwin wrote: > On Thursday, July 10, 2014 6:31:43 pm Rick Macklem wrote: > > John Baldwin wrote: > > > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > > > Russell L. Carter wrote: > > > > > > > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > > > > > Could you please post the dmesg stuff for the network > > > > > > interface, > > > > > > so I can tell what driver is being used? I'll take a look at > > > > > > it, > > > > > > in case it needs to be changed to use m_defrag(). > > > > > > > > > > em0: port > > > > > 0xd020-0xd03f > > > > > mem > > > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at device 0.0 > > > > > on > > > > > pci2 > > > > > em0: Using an MSI interrupt > > > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > > > 001.000007 [2323] netmap_attach success for em0 tx > > > > > 1/1024 > > > > > rx > > > > > 1/1024 queues/slots > > > > > > > > > > This is one of those dual nic cards, so there is em1 as well... > > > > > > > > > Well, I took a quick look at the driver and it does use m_defrag(), > > > > but > > > > I think that the "retry:" label it does a goto after doing so might > > > > be in > > > > the wrong place. > > > > > > > > The attached untested patch might fix this. > > > > > > > > Is it convenient to build a kernel with this patch applied and then > > > > try > > > > it with TSO enabled? > > > > > > > > rick > > > > ps: It does have the transmit segment limit set to 32. I have no > > > > idea if > > > > this is a hardware limitation. > > > > > > I think the retry is not in the wrong place, but the overhead of all > > > those > > > pullups is apparently quite severe. > > The m_defrag() call after the first failure will just barely squeeze > > the just under 64K TSO segment into 32 mbuf clusters. Then I think any > > m_pullup() done during the retry will allocate an mbuf > > (at a glance it seems to always do this when the old mbuf is a cluster) > > and prepend that to the list. > > --> Now the list is > 32 mbufs again and the bus_dmammap_load_mbuf_sg() > > will fail again on the retry, this time fatally, I think? > > > > I can't see any reason to re-do all the stuff using m_pullup() and Russell > > reported that moving the "retry:" fixed his problem, from what I understood. > > Ah, I had assumed (incorrectly) that the m_pullup()s would all be nops in this > case. It seems the NIC would really like to have all those things in a single > segment, but it is not required, so I agree that your patch is fine. > I recall em(4) controllers have various limitation in TSO. Driver has to update IP header to make TSO work so driver has to get a writable mbufs. bpf(4) consumers will see IP packet length is 0 after this change. I think tcpdump has a compile time option to guess correct IP packet length. The firmware of controller also should be able to access complete IP/TCP header in a single buffer. I don't remember more details in TSO limitation but I guess you may be able to get more details TSO limitation from publicly available Intel data sheet. From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 10:53:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9A44230 for ; Sat, 12 Jul 2014 10:53:36 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CCF827A8 for ; Sat, 12 Jul 2014 10:53:36 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id dc16so1706442qab.8 for ; Sat, 12 Jul 2014 03:53:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=b3dGU79d4FWtd3xsn5zy622D8aIC8snGZxOY1E5VF9E=; b=kw5BNVH1zPfTIQbvs/P/ZB2g3EiyC7fWUmwGi2ZIYsHd7gbl/Upv6070J2xkVIk3TG qm/iQq+EovszFp+bYQA3+Lbsh5pJYIvpJljsJ5enYTShSyXwp6Ce7aeOahp83mkHq2vi fg9nFFw5xjfCMAaJZbFS5Ze7if+YIxDsaNDJQ+rGXYtdLyfuxudi0efujx9yOgAClN95 kv9hPobZafIlgzNwMwyUl2QHBZSb8kT4eiDNyaZtfmWocEXiMDY5EAMIUOEk3uOEEMgB vn0Y/OGcN05EctMXJh5r7FPO5SRpHltisWz0kpa58L5zkzL/Z5CRQaTWOlG/QCZHujQH Ggig== X-Received: by 10.224.0.79 with SMTP id 15mr4101353qaa.56.1405162415390; Sat, 12 Jul 2014 03:53:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Sat, 12 Jul 2014 03:52:54 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Sat, 12 Jul 2014 11:52:54 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 10:53:36 -0000 Ok,I solved that problem that I was having but now I have another one. For what I was able to determine, SMP is not supported for IXP4xx processors on OpenWRT. I'm able to compile it for x86, but not for IXP4xx, the CPU's that the SBC Cambria from Gateworks use. I'm still investigating if this is really the problem and if it is, if it is possible to overcome. I will try to keep regular updates on this situation. Carlos On 11 July 2014 22:54, Carlos Ferreira wrote: > OK, ignore what I said in the last e-mail. My Makefile is nor working > properly and I'm trying to figure out why. OpenWRT documentation for modu= le > Makefiles creation is scarce and confuse... > > > On 11 July 2014 18:27, Carlos Ferreira wrote: > >> I'm building for OpenWRT (trunk) for the IXP4xx target. >> >> Attached goes the output for the compile attempt. Maybe I'm missing >> something very basic... >> >> >> On 11 July 2014 17:13, Luigi Rizzo wrote: >> >>> >>> >>> >>> On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira >>> wrote: >>> >>>> Luigi one question. Does netmap requires a processor with 64 bits? I'm >>>> having some trouble in compiling netmap, using the same Makefile I use= d >>>> previously, but for an Intel IXP435 CPU (Gateworks Cambria). >>>> >>> >>> =E2=80=8Bit used to build and work on 32 bit archs but we have not trie= d that >>> there i a while. >>> Hopefully it is just a matter of casts in printfs. >>> >>> which OS and netmap versions are you using ? >>> can you send me an error log ? >>> >>> cheers >>> luigi >>> =E2=80=8B >>> >>>> >>>> >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 12:17:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F4A226B for ; Sat, 12 Jul 2014 12:17:54 +0000 (UTC) Received: from mail-vc0-x236.google.com (mail-vc0-x236.google.com [IPv6:2607:f8b0:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE53E2D84 for ; Sat, 12 Jul 2014 12:17:53 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id hq11so4105469vcb.41 for ; Sat, 12 Jul 2014 05:17:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=JP/LHdf05dKFIpOQt7TTEHxurjC/kVk9fdNTia5EmFk=; b=FzwzhZeBDH3JaF2Oi8R9WH5u5VzfpikektpqqO/SXyjNRogC3n95xx2NVGqmkHKXfA jcw3fdnddO2qmK6CwlIY6xZpzWypLtHIHSlxBNFKgbS4DJXluKmxNHgnD28X8bi6g6Ww TLk8WnU8nKVJ/CaZOgpuwSTlWVTNPoEj+P6hGwjG0OPF84eC2vMnf1lBjCFxMCgN1L6F 2qyUYDc2gw1lRYGPiZmc3MW5xvFcjarya7LBITOsjTDA+YsgahMNzJwSAkbz1cTH0M2M rxpv/ZEr2uqVi/yQIHY+/VGKigBAe1QLpWK0VoFPM5oGUSesiwMHYpo2kH5wBgs1TLJs ObNg== X-Received: by 10.58.8.12 with SMTP id n12mr50824vea.28.1405167472840; Sat, 12 Jul 2014 05:17:52 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.228.38 with HTTP; Sat, 12 Jul 2014 05:17:32 -0700 (PDT) In-Reply-To: <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> References: <53C01EB5.6090701@gmail.com> <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Sat, 12 Jul 2014 14:17:32 +0200 X-Google-Sender-Auth: u3c-BfqqZZqcD-e9kGqPkeeczu8 Message-ID: Subject: Re: tuning routing using cxgbe and T580-CR cards? To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , Navdeep Parhar , John Jasem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 12:17:54 -0000 On Fri, Jul 11, 2014 at 8:03 PM, Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > On 11 Jul 2014, at 17:28 , John Jasem wrote: > > > c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx > > queues, with N being the number of CPUs detected. For a system running > > multiple cards, routing or firewalling, does this make sense, or would > > balancing tx and rx be more ideal? And would reducing queues per card > > based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? > > ... > > g) Are there other settings I should be looking at, that may squeeze out > > a few more packets? > > If you are primarily forwarding packets (you say "routing" multiple times) > the first thing you should do is turn off LRO and TSO on all ports. > Hi Bjoern, I was not aware of disabling LRO+TSO for forwarding packet. If I read correctly the wikipedia page of LRO[1]: Disabling LRO is not a concern of performance but only of not breaking the end-to-end principle, right ? But regarding TSO[2]: It should improve performance only between the TCP and IP layer. But paquet forwarded didn't have to cross TCP<->IP layer, then disabling TSO should not impact performance, right ? I've tried to benchs the differences on my lab: - Hardware: quad cores (Intel Xeon L5630 2.13GHz, hyper-threading disabled) with 2 ports Intel 10-Gigabit X540-AT2 - Multi-flows (different UDP ports) of small packet (60B) at about 10Mpps (pkt-gen -f tx -i ix0 -n 1000000000 -l 60 -d 9.3.3.1:2000-9.3.3.1:4000 -D a0:36:9f:1e:28:14 -s 8.3.3.1 -w 4) - Result collected on the receiver side in Paquet-Per-Second unit. ministat -w 74 tso.lro.enabled tso.lro.disabled x tso.lro.enabled + tso.lro.disabled +--------------------------------------------------------------------------+ | + + x+ * x+ x x| ||____________M_|_A________________|________A_M_________________________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 1724046 1860817 1798145 1793343 61865.164 + 5 1702496 1798998 1725396 1734863.2 38178.905 No difference proven at 95.0% confidence => There is not difference: Then I can disable LRO for respecting the end-to-end principle. But why disabling TSO ? Regards, Olivier [1] http://en.wikipedia.org/wiki/Large_receive_offload [2] http://en.wikipedia.org/wiki/Large_segment_offload From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 12:33:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02693413 for ; Sat, 12 Jul 2014 12:33:51 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AACED2ECD for ; Sat, 12 Jul 2014 12:33:50 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 7C47525D3A85; Sat, 12 Jul 2014 12:33:47 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 83A5DC22BFD; Sat, 12 Jul 2014 12:33:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id REo1yqjB-j1T; Sat, 12 Jul 2014 12:33:44 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:80b3:3e87:431:f220] (unknown [IPv6:fde9:577b:c1a9:4410:80b3:3e87:431:f220]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id CB157C22BA9; Sat, 12 Jul 2014 12:33:43 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: tuning routing using cxgbe and T580-CR cards? From: "Bjoern A. Zeeb" In-Reply-To: Date: Sat, 12 Jul 2014 12:33:23 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <68918930-4FEC-413B-AB5E-B544C936D54F@lists.zabbadoz.net> References: <53C01EB5.6090701@gmail.com> <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> To: =?windows-1252?Q?Olivier_Cochard-Labb=E9?= X-Mailer: Apple Mail (2.1878.6) Cc: FreeBSD Net , John Jasem , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 12:33:51 -0000 On 12 Jul 2014, at 12:17 , Olivier Cochard-Labb=E9 = wrote: > On Fri, Jul 11, 2014 at 8:03 PM, Bjoern A. Zeeb < > bzeeb-lists@lists.zabbadoz.net> wrote: >=20 >> If you are primarily forwarding packets (you say "routing" multiple = times) >> the first thing you should do is turn off LRO and TSO on all ports. >>=20 >=20 > Hi Bjoern, >=20 > I was not aware of disabling LRO+TSO for forwarding packet. > If I read correctly the wikipedia page of LRO[1]: Disabling LRO is not = a > concern of performance but only of not breaking the end-to-end = principle, > right ? > But regarding TSO[2]: It should improve performance only between the = TCP > and IP layer. But paquet forwarded didn't have to cross TCP<->IP = layer, > then disabling TSO should not impact performance, right ? For forwarding it means that you are re-assembling a packet on receive, = buffering multiple, etc, then hand them up the stack, only to find that = you are sending it out again, and thus you break them into multiple = packets again. In other words: you do a lot more work and add latency = than you need/want. I seem to remember that we added the knob to automatically disable our = soft-LRO when forwarding is turned on (but I haven=92t checked if I = really did). If we did, at least for soft-LRO you won=92t notice a = difference indeed. > - Multi-flows (different UDP ports) of small packet (60B) at about = 10Mpps > =85 > No difference proven at 95.0% confidence >=20 > =3D> There is not difference: Then I can disable LRO for respecting = the > end-to-end principle. But why disabling TSO ? Try TCP flows. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 14:37:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9DBF7C1 for ; Sat, 12 Jul 2014 14:37:32 +0000 (UTC) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C57D27B2 for ; Sat, 12 Jul 2014 14:37:32 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id x3so75361qcv.37 for ; Sat, 12 Jul 2014 07:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Se7wM9dAn9Z25Yw5vc0zo7117P1mukqyWuHd1edOfuw=; b=OPTlkqnvkwcOv5xlymQWk4L60EkuOP/yqWT83dGdIPJr5ZRZriIDbOpUtMijQfaLYA zHsKp/TcEPwW96/3HUAwu4I0kYQk+shaFh9z80TCuX0m02yzw2/s3t64WHbwSSqseKZZ PUu7hslbJBiTg9B2XbhNV13HO22jDXvtN8aAM5yhsEnpSFZI91wZ2GBbdkP9ramSXUUb spZvypTKe9WtJgtsAWBKhVLxlIjLzTpeJq+XSins9Nx3/HIEAgGFV6NLpk0xUkRNvzQS N65Cx7PynpmL08vcyaQxpuGnc5BRhpqaVuaRwag5WTcv28dfnwA1ZNjCvjaTIYsOpK0P Dnfg== X-Received: by 10.140.85.101 with SMTP id m92mr8481220qgd.26.1405175851788; Sat, 12 Jul 2014 07:37:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Sat, 12 Jul 2014 07:36:51 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Sat, 12 Jul 2014 15:36:51 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 14:37:33 -0000 Ok it seems that Symmetric MultiProcessing is broken for the IXP4xx arch when compiling the OpenWRT linux kernel. Since netmap requires SMP to be active. Netmap uses the real_num_rx_queues field from the netdev_rx_queue which requires CONFIG_SYSFS, which is active when CONFIG_SMP is active. I just posted at the OpenWRT development mailing list, requesting info and also, contacted Ryan Erbstoesser at Gateworks to also request info. On 12 July 2014 11:52, Carlos Ferreira wrote: > Ok,I solved that problem that I was having but now I have another one. Fo= r > what I was able to determine, SMP is not supported for IXP4xx processors = on > OpenWRT. > I'm able to compile it for x86, but not for IXP4xx, the CPU's that the SB= C > Cambria from Gateworks use. > I'm still investigating if this is really the problem and if it is, if it > is possible to overcome. > > I will try to keep regular updates on this situation. > Carlos > > > On 11 July 2014 22:54, Carlos Ferreira wrote: > >> OK, ignore what I said in the last e-mail. My Makefile is nor working >> properly and I'm trying to figure out why. OpenWRT documentation for mod= ule >> Makefiles creation is scarce and confuse... >> >> >> On 11 July 2014 18:27, Carlos Ferreira wrote: >> >>> I'm building for OpenWRT (trunk) for the IXP4xx target. >>> >>> Attached goes the output for the compile attempt. Maybe I'm missing >>> something very basic... >>> >>> >>> On 11 July 2014 17:13, Luigi Rizzo wrote: >>> >>>> >>>> >>>> >>>> On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira >>> > wrote: >>>> >>>>> Luigi one question. Does netmap requires a processor with 64 bits? I'= m >>>>> having some trouble in compiling netmap, using the same Makefile I us= ed >>>>> previously, but for an Intel IXP435 CPU (Gateworks Cambria). >>>>> >>>> >>>> =E2=80=8Bit used to build and work on 32 bit archs but we have not tri= ed that >>>> there i a while. >>>> Hopefully it is just a matter of casts in printfs. >>>> >>>> which OS and netmap versions are you using ? >>>> can you send me an error log ? >>>> >>>> cheers >>>> luigi >>>> =E2=80=8B >>>> >>>>> >>>>> >>> >>> >>> -- >>> >>> Carlos Miguel Ferreira >>> Researcher at Telecommunications Institute >>> Aveiro - Portugal >>> Work E-mail - cmf@av.it.pt >>> Skype & GTalk -> carlosmf.pt@gmail.com >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>> >> >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 14:44:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D30E7994 for ; Sat, 12 Jul 2014 14:44:03 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 840012855 for ; Sat, 12 Jul 2014 14:44:03 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id q108so1945086qgd.20 for ; Sat, 12 Jul 2014 07:44:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=o/KExPvUWoHivgsGpKPX9yaBtScPaJ3kNegMACGPX+Y=; b=qQlYPJxWpALYkigGCoeCBffkma6XC78BogagdRxJ30niF8Niz6O3kvqlw0bChU4ITN ANDaJXF69NUdl/H+O/mdOyupJyShiafhaZS6NNFxps3Lf6NpZ0/Tth/ZuWh8UxblX6Cn ZwW3ydrnL784bVtqMycHlZnpc/K/Ux+Tzw8i2Ld5ppIjFBUEaVEEYwAz2xHCrWVlSR5E EXyJLw+O8PtNBnm8sO3FVL5pJG/xxmkE6f2981jUBLDQD8U3HMZKiJbsDvXmC13vr2Ux ceY5lzljd3doFeOb1NrrEjajPlo3mmeWkKe2KcYnu140de0opFOR55qGQJwKzEJ6vV5L puSg== X-Received: by 10.140.87.68 with SMTP id q62mr8060010qgd.21.1405176242658; Sat, 12 Jul 2014 07:44:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.74.69 with HTTP; Sat, 12 Jul 2014 07:43:22 -0700 (PDT) In-Reply-To: References: <20140614101523.GD22187@onelab2.iet.unipi.it> From: Carlos Ferreira Date: Sat, 12 Jul 2014 15:43:22 +0100 Message-ID: Subject: Re: netmap To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , Prashant Upadhyaya X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 14:44:03 -0000 I'm sorry, I made a mistake. The real_num_rx_queues field, with the 3.10 kernel which is currently being used with the OpenWRT trunk branch, is only defined when CONFIG_RPS flag is active. CONFIG_RPS is only active when CONFIG_SMP is active. This as changed (beyond 3.13 version) and the real_num_rx_queues field depends now on CONFIG_SYSFS field. On 12 July 2014 15:36, Carlos Ferreira wrote: > Ok it seems that Symmetric MultiProcessing is broken for the IXP4xx arch > when compiling the OpenWRT linux kernel. Since netmap requires SMP to be > active. Netmap uses the real_num_rx_queues field from the netdev_rx_queue > which requires CONFIG_SYSFS, which is active when CONFIG_SMP is active. > > I just posted at the OpenWRT development mailing list, requesting info an= d > also, contacted Ryan Erbstoesser at Gateworks to also request info. > > > On 12 July 2014 11:52, Carlos Ferreira wrote: > >> Ok,I solved that problem that I was having but now I have another one. >> For what I was able to determine, SMP is not supported for IXP4xx >> processors on OpenWRT. >> I'm able to compile it for x86, but not for IXP4xx, the CPU's that the >> SBC Cambria from Gateworks use. >> I'm still investigating if this is really the problem and if it is, if i= t >> is possible to overcome. >> >> I will try to keep regular updates on this situation. >> Carlos >> >> >> On 11 July 2014 22:54, Carlos Ferreira wrote: >> >>> OK, ignore what I said in the last e-mail. My Makefile is nor working >>> properly and I'm trying to figure out why. OpenWRT documentation for mo= dule >>> Makefiles creation is scarce and confuse... >>> >>> >>> On 11 July 2014 18:27, Carlos Ferreira wrote: >>> >>>> I'm building for OpenWRT (trunk) for the IXP4xx target. >>>> >>>> Attached goes the output for the compile attempt. Maybe I'm missing >>>> something very basic... >>>> >>>> >>>> On 11 July 2014 17:13, Luigi Rizzo wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Fri, Jul 11, 2014 at 6:07 PM, Carlos Ferreira < >>>>> carlosmf.pt@gmail.com> wrote: >>>>> >>>>>> Luigi one question. Does netmap requires a processor with 64 bits? >>>>>> I'm having some trouble in compiling netmap, using the same Makefile= I used >>>>>> previously, but for an Intel IXP435 CPU (Gateworks Cambria). >>>>>> >>>>> >>>>> =E2=80=8Bit used to build and work on 32 bit archs but we have not tr= ied that >>>>> there i a while. >>>>> Hopefully it is just a matter of casts in printfs. >>>>> >>>>> which OS and netmap versions are you using ? >>>>> can you send me an error log ? >>>>> >>>>> cheers >>>>> luigi >>>>> =E2=80=8B >>>>> >>>>>> >>>>>> >>>> >>>> >>>> -- >>>> >>>> Carlos Miguel Ferreira >>>> Researcher at Telecommunications Institute >>>> Aveiro - Portugal >>>> Work E-mail - cmf@av.it.pt >>>> Skype & GTalk -> carlosmf.pt@gmail.com >>>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>>> >>> >>> >>> >>> -- >>> >>> Carlos Miguel Ferreira >>> Researcher at Telecommunications Institute >>> Aveiro - Portugal >>> Work E-mail - cmf@av.it.pt >>> Skype & GTalk -> carlosmf.pt@gmail.com >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >>> >> >> >> >> -- >> >> Carlos Miguel Ferreira >> Researcher at Telecommunications Institute >> Aveiro - Portugal >> Work E-mail - cmf@av.it.pt >> Skype & GTalk -> carlosmf.pt@gmail.com >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira >> > > > > -- > > Carlos Miguel Ferreira > Researcher at Telecommunications Institute > Aveiro - Portugal > Work E-mail - cmf@av.it.pt > Skype & GTalk -> carlosmf.pt@gmail.com > LinkedIn -> http://www.linkedin.com/in/carlosmferreira > --=20 Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 21:14:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0BC2025A; Sat, 12 Jul 2014 21:14:08 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id B56DF24A1; Sat, 12 Jul 2014 21:14:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAL+jwVODaFve/2dsb2JhbABZg2BagnG+QwqGcFMBgSd1hAQBAQQBAQEgBCcgCwUWGAICDRkCKQEJJg4HBAEcBIghDa5+mAsXgSyNSAYBARs0B4J3gUwFmCKENpJTg2AhNX0IFyI X-IronPort-AV: E=Sophos;i="5.01,650,1400040000"; d="scan'208";a="139847356" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 12 Jul 2014 17:14:00 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 32840B40F3; Sat, 12 Jul 2014 17:14:00 -0400 (EDT) Date: Sat, 12 Jul 2014 17:14:00 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com Message-ID: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> In-Reply-To: <20140712060538.GA3649@michelle.fasterthan.com> Subject: Re: NFS client READ performance on -current MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: "Russell L. Carter" , freebsd-net@freebsd.org, John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 21:14:08 -0000 Yonghyeon Pyun wrote: > On Fri, Jul 11, 2014 at 09:54:23AM -0400, John Baldwin wrote: > > On Thursday, July 10, 2014 6:31:43 pm Rick Macklem wrote: > > > John Baldwin wrote: > > > > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > > > > Russell L. Carter wrote: > > > > > > > > > > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > > > > > > > Could you please post the dmesg stuff for the network > > > > > > > interface, > > > > > > > so I can tell what driver is being used? I'll take a look > > > > > > > at > > > > > > > it, > > > > > > > in case it needs to be changed to use m_defrag(). > > > > > > > > > > > > em0: port > > > > > > 0xd020-0xd03f > > > > > > mem > > > > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at > > > > > > device 0.0 > > > > > > on > > > > > > pci2 > > > > > > em0: Using an MSI interrupt > > > > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > > > > 001.000007 [2323] netmap_attach success for em0 > > > > > > tx > > > > > > 1/1024 > > > > > > rx > > > > > > 1/1024 queues/slots > > > > > > > > > > > > This is one of those dual nic cards, so there is em1 as > > > > > > well... > > > > > > > > > > > Well, I took a quick look at the driver and it does use > > > > > m_defrag(), > > > > > but > > > > > I think that the "retry:" label it does a goto after doing so > > > > > might > > > > > be in > > > > > the wrong place. > > > > > > > > > > The attached untested patch might fix this. > > > > > > > > > > Is it convenient to build a kernel with this patch applied > > > > > and then > > > > > try > > > > > it with TSO enabled? > > > > > > > > > > rick > > > > > ps: It does have the transmit segment limit set to 32. I have > > > > > no > > > > > idea if > > > > > this is a hardware limitation. > > > > > > > > I think the retry is not in the wrong place, but the overhead > > > > of all > > > > those > > > > pullups is apparently quite severe. > > > The m_defrag() call after the first failure will just barely > > > squeeze > > > the just under 64K TSO segment into 32 mbuf clusters. Then I > > > think any > > > m_pullup() done during the retry will allocate an mbuf > > > (at a glance it seems to always do this when the old mbuf is a > > > cluster) > > > and prepend that to the list. > > > --> Now the list is > 32 mbufs again and the > > > bus_dmammap_load_mbuf_sg() > > > will fail again on the retry, this time fatally, I think? > > > > > > I can't see any reason to re-do all the stuff using m_pullup() > > > and Russell > > > reported that moving the "retry:" fixed his problem, from what I > > > understood. > > > > Ah, I had assumed (incorrectly) that the m_pullup()s would all be > > nops in this > > case. It seems the NIC would really like to have all those things > > in a single > > segment, but it is not required, so I agree that your patch is > > fine. > > > > I recall em(4) controllers have various limitation in TSO. Driver > has to update IP header to make TSO work so driver has to get a > writable mbufs. bpf(4) consumers will see IP packet length is 0 > after this change. I think tcpdump has a compile time option to > guess correct IP packet length. The firmware of controller also > should be able to access complete IP/TCP header in a single buffer. > I don't remember more details in TSO limitation but I guess you may > be able to get more details TSO limitation from publicly available > Intel data sheet. I think that the patch should handle this ok. All of the m_pullup() stuff gets done the first time. Then, if the result is more than 32 mbufs in the list, m_defrag() is called to copy the chain. This should result in all the header stuff in the first mbuf cluster and the map call is done again with this list of clusters. (Without the patch, m_pullup() would allocate another prepended mbuf and make the chain more than 32mbufs again.) Russell seemed to confirm that the patch fixed the problem for him, but since I don't have em(4) hardware, it would be nice to have someone with commit privilege and access to em(4) hardware test and commit it. rick > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Jul 12 22:48:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93BC982D; Sat, 12 Jul 2014 22:48:56 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41FCA2B30; Sat, 12 Jul 2014 22:48:56 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id id10so4791146vcb.30 for ; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ubFymp2Oh0irJm3Wno878g2gTg8Gj80Q0lAXNkXG3HU=; b=mZQHP9Ie33Ux5RxjPPKGF671+rRVV1vME1exIvRvQYiUHm16O0ptuEsDsFqJlEAqj4 9jXyoHcghaXCwqQXZBjMeaUV1+1F7KZ7FehRTQH+bBRoGDg79l30gEpq1yCVgdN9Ry6N x2tXLPpJYKumoZx8pnm8yO4WS9sxTjaf0AsCZHieMSqmxIMHQU5qnTxdhqkC88fwV7EG Tn+b8X54K/gTGxv+KGhEca37+/1HsxJGLN+MWN/k/uZTEY+0QQ/cG913JqYmUrakz0CV tBOfgvCnxH3nZqmanB4QArxuLRSf+gmeoxNb0HsqnpgatesADPKMPAVvYWwW75reax9A bSHA== MIME-Version: 1.0 X-Received: by 10.220.251.134 with SMTP id ms6mr7561772vcb.10.1405205335150; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) Received: by 10.220.249.132 with HTTP; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) Date: Sat, 12 Jul 2014 18:48:55 -0400 Message-ID: Subject: ngX connected hosts not receiving replies from non-kernel IP services. From: Zaphod Beeblebrox To: FreeBSD Hackers , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 22:48:56 -0000 I have a network of computers at home. The gateway/firewall is FreeBSD 9.2 running mpd5. The host requesting the service is FreeBSD 9.2. The misbehaving host is FreeBSD 10.0p6 running mpd5. So the details: ssh is listening (output of netstat -an) tcp4 0 0 *.22 *.* LISTEN tcp6 0 0 *.22 *.* LISTEN named is listening (installed from bind99 port) tcp4 0 0 xx.yy.30.99.53 *.* LISTEN udp4 0 0 xx.yy.30.99.53 *.* mpd 5 on the server is up: [2:35:335]root@owl:~> ifconfig ng29 ng29: flags=88d1 metric 0 mtu 1436 inet xx.yy.31.6 --> xx.yy.16.50 netmask 0xffffffff inet6 fe80::219:b9ff:fef9:b9e7%ng29 prefixlen 64 scopeid 0x23 nd6 options=21 ping works: [1:71:137]root@virtual:/vr2/backup/nozfs/ox/local-etc> ping xx.yy.16.3 PING xx.yy.16.3 (xx.yy.16.3): 56 data bytes 64 bytes from xx.yy.16.3: icmp_seq=0 ttl=63 time=7.439 ms 64 bytes from xx.yy.16.3: icmp_seq=1 ttl=63 time=6.756 ms now tcpdumping from the FreeBSD 10.0p6 server host while I ssh: [2:29:329]root@owl:~> tcpdump -nvi ng29 host xx.yy.16.3 tcpdump: listening on ng29, link-type NULL (BSD loopback), capture size 65535 bytes capability mode sandbox enabled 18:14:36.276578 IP (tos 0x0, ttl 63, id 3249, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x4aa1 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435369805 ecr 0], length 0 18:14:39.290104 IP (tos 0x0, ttl 63, id 4999, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x3ee9 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435372805 ecr 0], length 0 18:14:42.502893 IP (tos 0x0, ttl 63, id 6832, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x3269 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435376005 ecr 0], length 0 Similarly tcpdumping from the server while running "dig google.ca @xx.yy.30.99" [2:37:337]root@owl:~> tcpdump -nvi ng29 host xx.yy.30.99 tcpdump: listening on ng29, link-type NULL (BSD loopback), capture size 65535 bytes capability mode sandbox enabled 18:36:02.841942 IP (tos 0x0, ttl 63, id 30407, offset 0, flags [none], proto UDP (17), length 66) xx.yy.20.52.27400 > xx.yy.30.99.53: 40608+ [1au] A? google.ca. (38) 18:36:07.838721 IP (tos 0x0, ttl 63, id 33612, offset 0, flags [none], proto UDP (17), length 66) xx.yy.20.52.27400 > xx.yy.30.99.53: 40608+ [1au] A? google.ca. (38) Frustratingly, ssh and bind work just fine from hosts that are on the lan with the server. It's like some portion of the packet routing machinery is broken with ngX. Before y'all ask, too, ip.forwarding is 1. The ng-connected hosts can use the rest of the internet ... just not non-kernel services on the host that breaks up their l2tp. From owner-freebsd-net@FreeBSD.ORG Sun Jul 13 02:13:06 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06E27B54; Sun, 13 Jul 2014 02:13:06 +0000 (UTC) Received: from mail.mahoroba.org (ent.mahoroba.org [IPv6:2001:2f0:104:8010::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "asuka.mahoroba.org", Issuer "ca.mahoroba.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 802F6286A; Sun, 13 Jul 2014 02:13:04 +0000 (UTC) Received: from yuga.mahoroba.org (ume@yuga.mahoroba.org [IPv6:2001:2f0:104:8010:7258:12ff:fe22:d94b]) (user=ume mech=DIGEST-MD5 bits=0) by mail.mahoroba.org (8.14.9/8.14.9) with ESMTP/inet6 id s6D2Cnt6092111 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 13 Jul 2014 11:12:55 +0900 (JST) (envelope-from ume@mahoroba.org) Date: Sun, 13 Jul 2014 11:15:53 +0900 Message-ID: From: Hajimu UMEMOTO To: Jeremie Le Hen Subject: Re: Description change proposal for the no_prefer_iface flag In-Reply-To: <20140710220115.GA91832@caravan.chchile.org> References: <20140710220115.GA91832@caravan.chchile.org> User-Agent: xcite1.60> Wanderlust/2.15.9 (Almost Unreal) Emacs/24.3 Mule/6.0 (HANACHIRUSATO) X-Operating-System: FreeBSD 9.3-PRERELEASE X-PGP-Key: http://www.mahoroba.org/~ume/publickey.asc X-PGP-Fingerprint: 1F00 0B9E 2164 70FC 6DC5 BF5F 04E9 F086 BF90 71FE MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.mahoroba.org [IPv6:2001:2f0:104:8010::1]); Sun, 13 Jul 2014 11:12:55 +0900 (JST) X-Virus-Scanned: clamav-milter 0.98.4 at asuka.mahoroba.org X-Virus-Status: Clean X-Spam-Status: No, score=-3.6 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RP_MATCHES_RCVD autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on asuka.mahoroba.org Cc: freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 02:13:06 -0000 Hi, >>>>> On Fri, 11 Jul 2014 00:01:16 +0200 >>>>> Jeremie Le Hen said: jlh> Hi Hajimu-san, net@, jlh> I plan to commit the following change to the no_prefer_iface flag jlh> description in the ifconfig(8) manpage. I find it easier to understand jlh> and more explicit than the current one. jlh> Comments? It seems good to me. Sincerely, jlh> Index: sbin/ifconfig/ifconfig.8 jlh> =================================================================== jlh> --- sbin/ifconfig/ifconfig.8 (revision 268508) jlh> +++ sbin/ifconfig/ifconfig.8 (working copy) jlh> @@ -679,9 +679,11 @@ jlh> Clear a flag jlh> .Cm nud . jlh> .It Cm no_prefer_iface jlh> -Set a flag to not prefer address on the interface as candidates of the jlh> -source address for outgoing packets, even when the interface is jlh> -outgoing interface. jlh> +Set a flag to not honor rule 5 of source address selection in RFC 3484. jlh> +In practice this means the address on the outgoing interface which will jlh> +be used will not be preferred, effectively yielding the decision to the jlh> +address selection policy table, configurable with jlh> +.Xr ip6addrctl 8 . jlh> .It Cm -no_prefer_iface jlh> Clear a flag jlh> .Cm no_prefer_iface . jlh> -- jlh> Jeremie Le Hen jlh> Scientists say the world is made up of Protons, Neutrons and Electrons. jlh> They forgot to mention Morons. jlh> _______________________________________________ jlh> freebsd-net@freebsd.org mailing list jlh> http://lists.freebsd.org/mailman/listinfo/freebsd-net jlh> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Hajimu UMEMOTO ume@mahoroba.org ume@{,jp.}FreeBSD.org http://www.mahoroba.org/~ume/ From owner-freebsd-net@FreeBSD.ORG Sun Jul 13 20:39:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 598DD28D for ; Sun, 13 Jul 2014 20:39:15 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 294F226ED for ; Sun, 13 Jul 2014 20:39:14 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1X6QF0-0005ZI-Ug for freebsd-net@freebsd.org; Sun, 13 Jul 2014 13:19:22 -0700 Date: Sun, 13 Jul 2014 13:19:22 -0700 (PDT) From: Beeblebrox To: freebsd-net@freebsd.org Message-ID: <20140713231903.0e1cf3f6@rsbsd.rsb> In-Reply-To: <20140702114110.GP45513@funkthat.com> References: <1403964739522-5924518.post@n5.nabble.com> <1186767237.5285620.1403988777812.JavaMail.root@uoguelph.ca> <1404041699791-5924681.post@n5.nabble.com> <1404071407420-5924767.post@n5.nabble.com> <1270654408.5439540.1404077062258.JavaMail.root@uoguelph.ca> <20140702091413.7e27e798@rsbsd.rsb> <338810855.6193094.1404297876056.JavaMail.root@uoguelph.ca> <20140702114110.GP45513@funkthat.com> Subject: PXE boot using Grub bootloader fails at mountroot; no PXE devs. MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 20:39:15 -0000 Sorry for the late response, but there were too many mixed results and I had to get all variables before commenting. I can't make any sense of it, Bu I can say that all required modules are loaded by both Grub & BTX boot settings. B: BTX, D: Debug Kernel, G: GRUB, NGM: No Grub Menu (not BSD-related) platform/if ] B-386D ] B-x64D ] B-x64 |] G-x64 ] G-x64D ] G-386D ] x64_Athl/re0] ok ] <3> ] <3> |] <3> ] <3> ] ok ] x64_P4/sis0 ] ok ] ok ] ok |] NGM ] NGM ] NGM ] x64_i3/msk0 ] <1> ] <4> ] ok |]reboots] <5> ] ok ] i386_P3/e100] <2> ] NA ] NA |] NA ] NA ] <2> ] <1>: Gets to "Booting..." message, then hangs /w nothing further. <2>: Drops to KDB debug & shows below. This is an old Tecra_8200 laptop with an almost ancient video card (trident bladeXP) Fatal trap 12: page fault while in vm86 mode fault virtual address 0x9dc82 fault code= user read, page not present <3>: Immediate black screen (like video err) unresponsive keyboard (ctrl+alt+del). This mobo has no internal video card, so I have placed a very old (i386-based) card. Might explain why the 386D kernel gets booted on the platform. <4>: Boots, gets to mount NFS root and loops with: newnfs server :/path not responding \ msk0 watchdog timeout \ msk0 down \ msk0 up <5>: Kernel boot panics with message: bootpc_init: no eligible interfaces (this #5 is the initial problem I had reported) ----- FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/PXE-boot-using-Grub-bootloader-fails-at-mountroot-no-PXE-devs-tp5924518p5928652.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Sun Jul 13 20:43:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33632338; Sun, 13 Jul 2014 20:43:07 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC787277C; Sun, 13 Jul 2014 20:43:06 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id j15so2570013qaq.39 for ; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=0KgQU9WCtoBM4fd6PjGffuyMmzI+1CCabI8fVQQ/7oE=; b=x6GQu3r5Jj9xq14djuGNjGb8KvB4cnOQl63OrTXv5Nr5a+uvtlPLn7p53P9qWP1HHs 8cCg7sRtNbLs0kRC1tD1QEjunAGrn4pTucn+X9w+jI9rozINo8ccB/1pKkevUgKyF9iA j+5agngwmYZVByPxY6XM+tGAY9aFVmskpqv5cP5cVuVCmcOBqv7+jbFtZU0aQ008AEIZ uBq/svn8FNzGzZQiSPlf0wnNWvI88BQGS6DgLSNPK3cr+2Bw27Wmju++ocBwzkAFhsjy NfCeoLf73SKDIhrjCvYcjzhyUIjXpuySV2RDwFanlbsf9gW0Hsxroy293yN4Efx9Bcri ZpPg== MIME-Version: 1.0 X-Received: by 10.140.88.243 with SMTP id t106mr3152478qgd.12.1405284185528; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) Date: Sun, 13 Jul 2014 13:43:05 -0700 X-Google-Sender-Auth: 5cDLK9T7hZnZRdDgGZpnptpU62M Message-ID: Subject: RSS update From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 20:43:07 -0000 Hi! for those who are interested, here's an update on my RSS hackery. I found a few hours to get through a few more things: * IPv4 TCP RSS is there; * IPv6 TCP RSS is now there; * Since it's IP options, there's a seperate set of IPv4 and IPv6 options; * I've verified that on igb and ixgbe that the incoming connections get redirected to the right RSS aware listen socket. Now, UDP is a pain - mostly because a lot of UDP traffic is going to be > 1 MTU and the ixgbe/igb/cxgbe NICs will hash IP fragments with the 2-tuple IP address. So things end up in the wrong spot. I however have UDPv4 RSS awareness working locally. I don't know when I'll get some more time to work on a way to handle IP fragments and requeuing the resulting frame to the correct netisr context. Thanks! -a From owner-freebsd-net@FreeBSD.ORG Sun Jul 13 23:11:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 887DE450 for ; Sun, 13 Jul 2014 23:11:46 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DB8E2214 for ; Sun, 13 Jul 2014 23:11:46 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id g10so1977211pdj.15 for ; Sun, 13 Jul 2014 16:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=Izdt+WdIDQ/wVK+Yxm8nd2C35XgQDVyevwp4eC38qQU=; b=uAMNKHeh1VNYVjDyDW8HfA4iIRZxhiti8tKMk8O3qAJuxFvyoMqMb5G7eToB/ANPxt MI1LUVXKd7Jmhzj0StGRGicoYBYEqf8if/IoU5g/aosGGgUoia8ZTn1h1+NWQKIbzVt9 vw1b4DOJg9qaJBHiqy56CNeDAQHdSncosgWsQF2EAeYLnQPhk8a2f02CUL3jY8YnFyQx JCkcAp133CFDAnMp3GirdSllVLg/Cu+OAsoen3xXmdSU7Y4eEBO+29u6vP8UJgypD+aL 5DTNPT/8mgEAMVDOktWAdIn2oOSv2+obPJsAgersaAm9teIls9vK9FyOCWMCyi9Zsr0k /cxw== X-Received: by 10.66.254.166 with SMTP id aj6mr13354572pad.11.1405293105964; Sun, 13 Jul 2014 16:11:45 -0700 (PDT) Received: from ox ([24.6.44.228]) by mx.google.com with ESMTPSA id wn7sm38102127pab.18.2014.07.13.16.11.44 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 13 Jul 2014 16:11:45 -0700 (PDT) Date: Sun, 13 Jul 2014 16:11:40 -0700 From: Navdeep Parhar To: John Jasem Subject: Re: tuning routing using cxgbe and T580-CR cards? Message-ID: <20140713231140.GA8690@ox> Mail-Followup-To: John Jasem , FreeBSD Net References: <53C01EB5.6090701@gmail.com> <53C03BB4.2090203@gmail.com> <53C0882D.5070100@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53C0882D.5070100@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 23:11:46 -0000 On Fri, Jul 11, 2014 at 08:58:21PM -0400, John Jasem wrote: > > On 07/11/2014 03:32 PM, Navdeep Parhar wrote: > > On 07/11/14 10:28, John Jasem wrote: > >> In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, > >> I've been able to use a collection of clients to generate approximately > >> 1.5-1.6 million TCP packets per second sustained, and routinely hit > >> 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the > >> quick read, accepting the loss of granularity). > > When forwarding, the pps rate is often more interesting, and almost > > always the limiting factor, as compared to the total amount of data > > being passed around. 10GB at this pps probably means 9000 MTU. Try > > with 1500 too if possible. > > Yes, I am generally more interested/concerned with the pps. Using > 1500-sized packets, I've seen around 2 million pps. I'll have hard > numbers for the list, with netstat and vmstat output Monday. Thanks! If possible, please try with even lower packet sizes (128B, 512B, whatever your clients are good at). You may have to disable Nagle (TCP_NODELAY option) on your clients to get small TCP packets out of them. Or you could just switch to UDP for pps testing. If all your incoming traffic is received on a single port then try setting hw.cxgbe.nrxq10g=12 in /boot/loader.conf. (You mentioned elsewhere this is a system with 12 real cores). Regards, Navdeep > > > > >> a) One of the first things I did in prior testing was to turn > >> hyperthreading off. I presume this is still prudent, as HT doesn't help > >> with interrupt handling? > > It is always worthwhile to try your workload with and without > > hyperthreading. > > Testing Mellanox cards, HT was severely detrimental. However, in almost > every case so far, Mellanox and Chelsio have resulted in opposite > conclusions (cpufreq, net.isr.*). > > >> c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx > >> queues, with N being the number of CPUs detected. For a system running > >> multiple cards, routing or firewalling, does this make sense, or would > >> balancing tx and rx be more ideal? And would reducing queues per card > >> based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? > > The defaults are nrxq = min(8, ncores) and ntxq = min(16, ncores). The > > man page mentions this. The reason for 8 vs. 16 is that tx queues are > > "cheaper" as they don't have to be backed by rx buffers. It only needs > > some memory for the tx descriptor ring and some hardware resources. > > > > It appears that your system has >= 16 cores. For forwarding it probably > > makes sense to have nrxq = ntxq. If you're left with 8 or fewer cores > > after disabling hyperthreading you'll automatically get 8 rx and tx > > queues. Otherwise you'll have to fiddle with the hw.cxgbe.nrxq10g and > > ntxq10g tunables (documented in the man page). > > I promise I did look through the man page before posting. :) This is > actually a 12 core box with HT turned off. > > Mining the cxl stat entries in sysctl, it appears that the queues per > port are reasonably well balanced, so I may be concerned over nothing. > > > > >> g) Are there other settings I should be looking at, that may squeeze out > >> a few more packets? > > The pps rates that you've observed are within the chip's hardware limits > > by at least an order of magnitude. Tuning the kernel rather than the > > driver may be the best bang for your buck. > > If I am missing obvious configurations for kernel tuning in this regard, > it would not the be the first time. > > Thanks again! > > -- John Jasen (jjasen@gmail.com) > From owner-freebsd-net@FreeBSD.ORG Sun Jul 13 23:50:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 863D9CD8; Sun, 13 Jul 2014 23:50:43 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 57C8924F2; Sun, 13 Jul 2014 23:50:43 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-250-191.lns20.per2.internode.on.net [121.45.250.191]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s6DNodKR070525 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 13 Jul 2014 16:50:41 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53C31B49.4060201@freebsd.org> Date: Mon, 14 Jul 2014 07:50:33 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Cristiano Deana Subject: Re: ng_netflow References: <53BEA125.6010400@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD net , FreeBSD Stable Mailing List X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 23:50:43 -0000 On 7/10/14, 10:39 PM, Cristiano Deana wrote: > On Thu, Jul 10, 2014 at 4:20 PM, Julian Elischer wrote: > > Hi, Julian > >> Is it possible that you are working with an interface that has TSO on? > it's a vlan interface, the device is a em0. > net.inet.tcp.tso: 1 > >> if so then netgraph will be seeing huge "aggregate" packets rather than the >> normal packets. >> so teh number of packets may be out by more than a factor of 30. > My bigger problem is with traffic counter, but maybe it's just my error. > >> netgraph nodes are relatively simple.. > Not really :) read this: http://people.freebsd.org/~julian/netgraph.html > > I have problems to find "easy" documentation about it. It's hard to > understand what lower, upper, right, etc meaning. I think I could hav > setup something wrong, maybe counting only incoming packet (or > outgoing). > > I followed, not fully undestand, the example in ng_netflow man page: > > /usr/sbin/ngctl -f- <<-SEQ > mkpeer fxp0: netflow lower iface0 > name fxp0:lower netflow > connect fxp0: netflow: upper out0 > mkpeer netflow: ksocket export inet/dgram/udp > msg netflow:export connect inet/10.0.0.1:4444 > SEQ > > Thank you > From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 02:35:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C72B89C2 for ; Mon, 14 Jul 2014 02:35:51 +0000 (UTC) Received: from gpo3.cc.swin.edu.au (gpo3.cc.swin.edu.au [136.186.1.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F04F2078 for ; Mon, 14 Jul 2014 02:35:50 +0000 (UTC) Received: from [136.186.229.154] (nwilliams-laptop.caia.swin.edu.au [136.186.229.154]) by gpo3.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s6E2ZiNv007769; Mon, 14 Jul 2014 12:35:46 +1000 Message-ID: <53C341FC.4060307@swin.edu.au> Date: Mon, 14 Jul 2014 12:35:40 +1000 From: Nigel Williams User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-net@freebsd.org Subject: Re: Multipath TCP for FreeBSD v0.4 References: <513CB9AF.3090409@swin.edu.au> <53BF8945.3000802@swin.edu.au> <20140711102535.7613DBE5@hub.freebsd.org> In-Reply-To: <20140711102535.7613DBE5@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 02:35:51 -0000 Hi Nils, On 11/07/14 20:24, Nils Beyer wrote: > Hi Nigel, > > Nigel Williams wrote: >> A new v0.4 patch is available at [1]. [...] > > Thanks a lot for publishing the latest patch. Already tried it on two phyiscal > machines with directly connected NICs. > Great, thanks for testing it out. > "iperf" looks nice: > =============================================================================== > #iperf -c 10.255.255.11 -i 1 > ------------------------------------------------------------ > Client connecting to 10.255.255.11, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 10.255.255.10 port 40171 connected with 10.255.255.11 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0- 1.0 sec 167 MBytes 1.40 Gbits/sec > [ 3] 1.0- 2.0 sec 171 MBytes 1.43 Gbits/sec > [ 3] 2.0- 3.0 sec 171 MBytes 1.44 Gbits/sec > [ 3] 3.0- 4.0 sec 171 MBytes 1.44 Gbits/sec > [ 3] 4.0- 5.0 sec 171 MBytes 1.44 Gbits/sec > [ 3] 5.0- 6.0 sec 169 MBytes 1.41 Gbits/sec > [ 3] 6.0- 7.0 sec 168 MBytes 1.41 Gbits/sec > [ 3] 7.0- 8.0 sec 169 MBytes 1.41 Gbits/sec > [ 3] 8.0- 9.0 sec 168 MBytes 1.41 Gbits/sec > [ 3] 9.0-10.0 sec 171 MBytes 1.43 Gbits/sec > [ 3] 0.0-10.0 sec 1.66 GBytes 1.42 Gbits/sec > =============================================================================== > > TCP networking is rather unstable after some "iperf" executions. So that new > SSH connections aren't possible anymore. > > Everything more complex than "iperf" - like NFS and FTP usage - leads to a > kernel panic (page fault). > > Do you want any crash dumps? If yes, where do you want them to be uploaded? > Yes, that would be helpful (I'll send you a link to a drop box). If you were able to email me the core text files that might also help. > > FWIW: I haven't set up any special routings or PF rules at all: > =============================================================================== > MPTCP1 > ------ > ifconfig_em1="10.255.255.10/8 -tso" > ifconfig_em0="192.168.1.1/24 -tso" > ifconfig_em2="192.168.2.1/24 -tso" > > > MPTCP2 > ------ > ifconfig_em1="10.255.255.11/8 -tso" > ifconfig_em0="192.168.1.2/24 -tso" > ifconfig_em2="192.168.2.2/24 -tso" > =============================================================================== > > Okay, I'll set up something similar. Did you configure any other TCP sysctls? cheers, nigel > > Thanks for all your work and regards, > Nils > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 06:30:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D18F9869 for ; Mon, 14 Jul 2014 06:30:15 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id 930DE229B for ; Mon, 14 Jul 2014 06:30:14 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id F2A911414818 for ; Mon, 14 Jul 2014 08:29:52 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 7914F1A0765 for ; Mon, 14 Jul 2014 08:29:57 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Mon, 14 Jul 2014 08:29:57 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Multipath TCP for FreeBSD v0.4 To: freebsd-net@freebsd.org References: <513CB9AF.3090409@swin.edu.au> <53BF8945.3000802@swin.edu.au> <20140711102535.7613DBE5@hub.freebsd.org> <53C341FC.4060307@swin.edu.au> Lines: 61 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 06:30:16 -0000 Hi Nigel, Nigel Williams wrote: >> Do you want any crash dumps? If yes, where do you want them to be >> uploaded? > > Yes, that would be helpful (I'll send you a link to a drop box). If you > were able to email me the core text files that might also help. Thanks for the link. I've sent you two crash dumps including the core text files. > Okay, I'll set up something similar. Did you configure any other TCP > sysctls? Yes: =============================================================================== net.inet.ip.fw.default_to_accept=1 net.inet.tcp.sack.enable=0 net.inet.tcp.tso=0 net.inet.tcp.mptcp.mp_addresses="192.168.1.1" net.inet.tcp.mptcp.linux_compat=0 =============================================================================== and I've built the whole system on both nodes without WITNESS and other debug- ging functionalities: =============================================================================== Index: /usr/src/sys/amd64/conf/GENERIC =================================================================== --- /usr/src/sys/amd64/conf/GENERIC (revision 265307) +++ /usr/src/sys/amd64/conf/GENERIC (working copy) @@ -76,14 +76,14 @@ options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. # For full debugger support use (turn off in stable branch): -options DDB # Support DDB. -options GDB # Support remote GDB. -options DEADLKRES # Enable the deadlock resolver -options INVARIANTS # Enable calls of extra sanity checking -options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -options WITNESS # Enable checks to detect deadlocks and cycles -options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed -options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones +#options DDB # Support DDB. +#options GDB # Support remote GDB. +#options DEADLKRES # Enable the deadlock resolver +#options INVARIANTS # Enable calls of extra sanity checking +#options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +#options WITNESS # Enable checks to detect deadlocks and cycles +#options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +#options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel =============================================================================== Regards, Nils From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 08:00:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2398DCF for ; Mon, 14 Jul 2014 08:00:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 772632A59 for ; Mon, 14 Jul 2014 08:00:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6E80D04043360 for ; Mon, 14 Jul 2014 08:00:13 GMT (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201407140800.s6E80D04043360@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 14 Jul 2014 08:00:13 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 08:00:13 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 14:57:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F883E3 for ; Mon, 14 Jul 2014 14:57:39 +0000 (UTC) Received: from mail-ig0-x229.google.com (mail-ig0-x229.google.com [IPv6:2607:f8b0:4001:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AB2C205F for ; Mon, 14 Jul 2014 14:57:39 +0000 (UTC) Received: by mail-ig0-f169.google.com with SMTP id r2so1702682igi.0 for ; Mon, 14 Jul 2014 07:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=jH+2vD3PS5KSFcRs3Fki13PMfu/+GuIYYCOL8cghYuE=; b=mu+mID5EyDCWmMcrD7lUMpwurlTSRouUZxP71VdM9ZNciD867iSiUOcVfpN9w6zrPP HbyYtJMlGzh1khAS3F1Q/D0+ZlNTh1SKAy2sk/f/QRG/XeyXhy4L2rhnCo7YzZ5VoOOV j7vw9pCAtEEeaZyXJ4avN0Rv3xkaDPuFxZ84kXtX/KfJjEY78+ngjRpJX2qw6zTxN7UU 5+g3eD6W8OBkRwXSvQOAGyCyJFUKzkJDEEf9u0e1IMmVXsPpO4iZoLHJSlQafvGeYN3A FapnGehYAJ6q79VQevqLsdlCzDaosK5awXBoZatctc1ALq2m1YCgevSFF1F81UelpC+S Ecig== X-Received: by 10.50.147.70 with SMTP id ti6mr5278852igb.45.1405349858754; Mon, 14 Jul 2014 07:57:38 -0700 (PDT) Received: from [10.1.68.187] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id dx6sm11134289igb.15.2014.07.14.07.57.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Jul 2014 07:57:34 -0700 (PDT) Message-ID: <53C3EFDC.2030100@gmail.com> Date: Mon, 14 Jul 2014 10:57:32 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Navdeep Parhar , FreeBSD Net Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> <53C03BB4.2090203@gmail.com> In-Reply-To: <53C03BB4.2090203@gmail.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 14:57:39 -0000 The two physical CPUs are: Intel(R) Xeon(R) CPU E5-4610 0 @ 2.40GHz (2400.05-MHz K8-class CPU) Hyperthreading, at least from initial appearances, seems to offer no benefits or drawbacks. I tested iperf3, using a packet generator on each subnet, each sending 4 streams to a server on another subnet. maximum segment size of 128 and 1460 used, with little variance. (iperf3 -M). A snapshot of netstat -d -b -w1 -W -h included. Midway through, the numbers dropped. This coincides with launching this was when I launched 16 more streams, 4 new clients, 4 new servers on different nets, 4 streams each. input (Total) output packets errs idrops bytes packets errs bytes colls drops 1.6M 0 514 254M 1.6M 0 252M 0 5 1.6M 0 294 244M 1.6M 0 246M 0 6 1.6M 0 95 255M 1.5M 0 236M 0 6 1.4M 0 0 216M 1.5M 0 224M 0 3 1.5M 0 0 225M 1.4M 0 219M 0 4 1.4M 0 389 214M 1.4M 0 216M 0 1 1.4M 0 270 207M 1.4M 0 207M 0 1 1.4M 0 279 210M 1.4M 0 209M 0 2 1.4M 0 12 207M 1.3M 0 204M 0 1 1.4M 0 303 206M 1.4M 0 214M 0 2 1.3M 0 2.3K 190M 1.4M 0 212M 0 1 1.1M 0 1.1K 175M 1.1M 0 176M 0 1 1.1M 0 1.6K 176M 1.1M 0 175M 0 1 1.1M 0 830 176M 1.1M 0 174M 0 0 1.2M 0 1.5K 187M 1.2M 0 187M 0 0 1.2M 0 1.1K 183M 1.2M 0 184M 0 1 1.2M 0 1.5K 197M 1.2M 0 196M 0 2 1.3M 0 2.2K 199M 1.2M 0 196M 0 0 1.3M 0 2.8K 200M 1.3M 0 202M 0 4 1.3M 0 1.5K 199M 1.2M 0 198M 0 1 vmstat also included. You see similar drops in faults. procs memory page disks faults cpu r b w avm fre flt re pi po fr sr mf0 cd0 in sy cs us sy id 0 0 0 574M 15G 82 0 0 0 0 6 0 0 188799 224 387419 0 74 26 0 0 0 574M 15G 2 0 0 0 0 6 0 0 207447 150 425576 0 72 28 0 0 0 574M 15G 82 0 0 0 0 6 0 0 205638 202 421659 0 75 25 0 0 0 574M 15G 2 0 0 0 0 6 0 0 200292 150 411257 0 74 26 0 0 0 574M 15G 82 0 0 0 0 6 0 0 200338 197 411537 0 77 23 0 0 0 574M 15G 2 0 0 0 0 6 0 0 199289 156 409092 0 75 25 0 0 0 574M 15G 82 0 0 0 0 6 0 0 200504 200 411992 0 76 24 0 0 0 574M 15G 2 0 0 0 0 6 0 0 165042 152 341207 0 78 22 0 0 0 574M 15G 82 0 0 0 0 6 0 0 171360 200 353776 0 78 22 0 0 0 574M 15G 2 0 0 0 0 6 0 0 197557 150 405937 0 74 26 0 0 0 574M 15G 82 0 0 0 0 6 0 0 170696 204 353197 0 78 22 0 0 0 574M 15G 2 0 0 0 0 6 0 0 174927 150 361171 0 77 23 0 0 0 574M 15G 82 0 0 0 0 6 0 0 153836 200 319227 0 79 21 0 0 0 574M 15G 2 0 0 0 0 6 0 0 159056 150 329517 0 78 22 0 0 0 574M 15G 82 0 0 0 0 6 0 0 155240 200 321819 0 78 22 0 0 0 574M 15G 2 0 0 0 0 6 0 0 166422 156 344184 0 78 22 0 0 0 574M 15G 82 0 0 0 0 6 0 0 162065 200 335215 0 79 21 0 0 0 574M 15G 2 0 0 0 0 6 0 0 172857 150 356852 0 78 22 0 0 0 574M 15G 82 0 0 0 0 6 0 0 81267 197 176539 0 92 8 0 0 0 574M 15G 2 0 0 0 0 6 0 0 82151 150 177434 0 91 9 0 0 0 574M 15G 82 0 0 0 0 6 0 0 73904 204 160887 0 91 9 0 0 0 574M 15G 2 0 0 0 8 6 0 0 73820 150 161201 0 91 9 0 0 0 574M 15G 82 0 0 0 0 6 0 0 73926 196 161850 0 92 8 0 0 0 574M 15G 2 0 0 0 0 6 0 0 77215 150 166886 0 91 9 0 0 0 574M 15G 82 0 0 0 0 6 0 0 77509 198 169650 0 91 9 0 0 0 574M 15G 2 0 0 0 0 6 0 0 69993 156 154783 0 90 10 0 0 0 574M 15G 82 0 0 0 0 6 0 0 69722 199 153525 0 91 9 0 0 0 574M 15G 2 0 0 0 0 6 0 0 66353 150 147027 0 91 9 0 0 0 550M 15G 102 0 0 0 101 6 0 0 67906 259 149365 0 90 10 0 0 0 550M 15G 0 0 0 0 0 6 0 0 71837 125 157253 0 92 8 0 0 0 550M 15G 80 0 0 0 0 6 0 0 73508 179 161498 0 92 8 0 0 0 550M 15G 0 0 0 0 0 6 0 0 72673 125 159449 0 92 8 0 0 0 550M 15G 80 0 0 0 0 6 0 0 75630 175 164614 0 91 9 On 07/11/2014 03:32 PM, Navdeep Parhar wrote: > On 07/11/14 10:28, John Jasem wrote: >> In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, >> I've been able to use a collection of clients to generate approximately >> 1.5-1.6 million TCP packets per second sustained, and routinely hit >> 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the >> quick read, accepting the loss of granularity). > When forwarding, the pps rate is often more interesting, and almost > always the limiting factor, as compared to the total amount of data > being passed around. 10GB at this pps probably means 9000 MTU. Try > with 1500 too if possible. > > "netstat -d 1" and "vmstat 1" for a few seconds when your system is > under maximum load would be useful. And what kind of CPU is in this system? > >> While performance has so far been stellar, and I'm honestly speculating >> I will need more CPU depth and horsepower to get much faster, I'm >> curious if there is any gain to tweaking performance settings. I'm >> seeing, under multiple streams, with N targets connecting to N servers, >> interrupts on all CPUs peg at 99-100%, and I'm curious if tweaking >> configs will help, or its a free clue to get more horsepower. >> >> So, far, except for temporarily turning off pflogd, and setting the >> following sysctl variables, I've not done any performance tuning on the >> system yet. >> >> /etc/sysctl.conf >> net.inet.ip.fastforwarding=1 >> kern.random.sys.harvest.ethernet=0 >> kern.random.sys.harvest.point_to_point=0 >> kern.random.sys.harvest.interrupt=0 >> >> a) One of the first things I did in prior testing was to turn >> hyperthreading off. I presume this is still prudent, as HT doesn't help >> with interrupt handling? > It is always worthwhile to try your workload with and without > hyperthreading. > >> b) I briefly experimented with using cpuset(1) to stick interrupts to >> physical CPUs, but it offered no performance enhancements, and indeed, >> appeared to decrease performance by 10-20%. Has anyone else tried this? >> What were your results? >> >> c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx >> queues, with N being the number of CPUs detected. For a system running >> multiple cards, routing or firewalling, does this make sense, or would >> balancing tx and rx be more ideal? And would reducing queues per card >> based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? > The defaults are nrxq = min(8, ncores) and ntxq = min(16, ncores). The > man page mentions this. The reason for 8 vs. 16 is that tx queues are > "cheaper" as they don't have to be backed by rx buffers. It only needs > some memory for the tx descriptor ring and some hardware resources. > > It appears that your system has >= 16 cores. For forwarding it probably > makes sense to have nrxq = ntxq. If you're left with 8 or fewer cores > after disabling hyperthreading you'll automatically get 8 rx and tx > queues. Otherwise you'll have to fiddle with the hw.cxgbe.nrxq10g and > ntxq10g tunables (documented in the man page). > > >> d) dev.cxl.$PORT.qsize_rxq: 1024 and dev.cxl.$PORT.qsize_txq: 1024. >> These appear to not be writeable when if_cxgbe is loaded, so I speculate >> they are not to be messed with, or are loader.conf variables? Is there >> any benefit to messing with them? > Can't change them after the port has been administratively brought up > even once. This is mentioned in the man page. I don't really recommend > changing them any way. > >> e) dev.t5nex.$CARD.toe.sndbuf: 262144. These are writeable, but messing >> with values did not yield an immediate benefit. Am I barking up the >> wrong tree, trying? > The TOE tunables won't make a difference unless you have enabled TOE, > the TCP endpoints lie on the system, and the connections are being > handled by the TOE on the chip. This is not the case on your systems. > The driver does not enable TOE by default and the only way to use it is > to switch it on explicitly. There is no possibility that you're using > it without knowing that you are. > >> f) based on prior experiments with other vendors, I tried tweaks to >> net.isr.* settings, but did not see any benefits worth discussing. Am I >> correct in this speculation, based on others experience? >> >> g) Are there other settings I should be looking at, that may squeeze out >> a few more packets? > The pps rates that you've observed are within the chip's hardware limits > by at least an order of magnitude. Tuning the kernel rather than the > driver may be the best bang for your buck. > > Regards, > Navdeep > >> Thanks in advance! >> >> -- John Jasen (jjasen@gmail.com) From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 15:34:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE066AB8 for ; Mon, 14 Jul 2014 15:34:58 +0000 (UTC) Received: from mail-ie0-x22c.google.com (mail-ie0-x22c.google.com [IPv6:2607:f8b0:4001:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7225023EF for ; Mon, 14 Jul 2014 15:34:58 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id lx4so1334154iec.3 for ; Mon, 14 Jul 2014 08:34:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=W6AWiEEe9ru0sxNMKHJMPMKjGyih7j+bJ+0EKJt1fCA=; b=QAFQfYu7RtX6vFk+CTQ5z3ijeWnux0TBwLvDsmmA+tU3dmLHjrET1a06GiJAb86Hdv BzsulP4KirVoYwuFW6ZqXQteBzw8DuZyMvLIShDUGaSptd57ou4BzOeqK3hslA40W8SB fBNaos1qQ4MBGbKD1UpdQ06rcIh/iDGK8SY3KIrCtnw3jeireMtbVLc/IUbpDJNzIirz 4H36SAZa2b0cgVhbNQnzo4RCo3F/eVZAwtb6czwDaxe+NFD90HfRqkF9WD+TIRWs0mgC xYADRtb4ZGk2IMTj+t9EkrJO6DINr075V0Y6mcrQuyLVOBflS/53Vc8+uAqHeEjPr5Ik WYMA== X-Received: by 10.50.36.106 with SMTP id p10mr26863746igj.9.1405352097815; Mon, 14 Jul 2014 08:34:57 -0700 (PDT) Received: from [10.1.68.187] (gs-sv-1-49-ac1.gsfc.nasa.gov. [198.119.56.43]) by mx.google.com with ESMTPSA id n1sm7130038igp.1.2014.07.14.08.34.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Jul 2014 08:34:56 -0700 (PDT) Message-ID: <53C3F89D.1020503@gmail.com> Date: Mon, 14 Jul 2014 11:34:53 -0400 From: John Jasem User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Bjoern A. Zeeb" , =?windows-1252?Q?Olivier_Cochard-Labb=E9?= Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> <01AABF44-4801-45B5-9509-1CA7BAA3CB30@lists.zabbadoz.net> <68918930-4FEC-413B-AB5E-B544C936D54F@lists.zabbadoz.net> In-Reply-To: <68918930-4FEC-413B-AB5E-B544C936D54F@lists.zabbadoz.net> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: FreeBSD Net , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 15:34:58 -0000 Dropping lro on the interfaces decreased interrupt usage on the CPUs, as measured by top -CHIPSu, by 15-20%, at least from eyeballing it. It did not otherwise have an effect on packet rates. Thanks! On 07/12/2014 08:33 AM, Bjoern A. Zeeb wrote: > On 12 Jul 2014, at 12:17 , Olivier Cochard-Labbé wrote: > >> On Fri, Jul 11, 2014 at 8:03 PM, Bjoern A. Zeeb < >> bzeeb-lists@lists.zabbadoz.net> wrote: >> >>> If you are primarily forwarding packets (you say "routing" multiple times) >>> the first thing you should do is turn off LRO and TSO on all ports. >>> >> Hi Bjoern, >> >> I was not aware of disabling LRO+TSO for forwarding packet. >> If I read correctly the wikipedia page of LRO[1]: Disabling LRO is not a >> concern of performance but only of not breaking the end-to-end principle, >> right ? >> But regarding TSO[2]: It should improve performance only between the TCP >> and IP layer. But paquet forwarded didn't have to cross TCP<->IP layer, >> then disabling TSO should not impact performance, right ? > For forwarding it means that you are re-assembling a packet on receive, buffering multiple, etc, then hand them up the stack, only to find that you are sending it out again, and thus you break them into multiple packets again. In other words: you do a lot more work and add latency than you need/want. > > I seem to remember that we added the knob to automatically disable our soft-LRO when forwarding is turned on (but I haven’t checked if I really did). If we did, at least for soft-LRO you won’t notice a difference indeed. > > >> - Multi-flows (different UDP ports) of small packet (60B) at about 10Mpps >> … >> No difference proven at 95.0% confidence >> >> => There is not difference: Then I can disable LRO for respecting the >> end-to-end principle. But why disabling TSO ? > Try TCP flows. > > — > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 > From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 19:03:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFB09963 for ; Mon, 14 Jul 2014 19:03:56 +0000 (UTC) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 83DE4281A for ; Mon, 14 Jul 2014 19:03:56 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id rd3so5871436pab.32 for ; Mon, 14 Jul 2014 12:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=aHuW7NKwAvaCTRbhoavr3Lf2FY0GpV2ItFlnVcev+ok=; b=N0pzthpwKbbksce7+z9aahYu2iGk9yfTrHvJoh/8n+9vZibPxx3pU4+Kh+yLNiiFwX aqu7LdRB5g8wwF2JFKb30x7G6tfT3jzeIpZcMZnMbxRD6TCcYHGhC7J1GnfVHo5yWoCf kYFELjos+B3AM4juyOD5t6O+EhHYmaWAfyeLeikEp4OUN/E+08aZhlu9WMoLg3JFoZkO s3toWE94UshA0lA8eamQUzpvgcE2KSBL3kxFXiLh5uguShPvRBxKTybfG2AY2o9OV7Kg goP/kL6VLQ53pevsbljbehb42ePqk9+3IlE0986+rMfbxyh9S3YoocN7wCvEn3mTUMWN gpzQ== X-Received: by 10.68.94.225 with SMTP id df1mr18324098pbb.86.1405364636154; Mon, 14 Jul 2014 12:03:56 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id mj9sm15498198pdb.44.2014.07.14.12.03.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Jul 2014 12:03:55 -0700 (PDT) Message-ID: <53C4299A.3000900@gmail.com> Date: Mon, 14 Jul 2014 12:03:54 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Jasem , FreeBSD Net Subject: Re: tuning routing using cxgbe and T580-CR cards? References: <53C01EB5.6090701@gmail.com> <53C03BB4.2090203@gmail.com> <53C3EFDC.2030100@gmail.com> In-Reply-To: <53C3EFDC.2030100@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 19:03:56 -0000 Use UDP if you want more control over your experiments. - It's easier to directly control the frame size on the wire. No TSO, LRO, segmentation to worry about. - UDP has no flow control so the transmitters will not let up even if a frame goes missing. TCP will go into recovery. Lack of protocol level flow control also means the transmitters cannot be influenced by the receivers in any way. - frames go only in the direction you want them to. With TCP you have the receiver transmitting all the time too (ACKs). Regards, Navdeep On 07/14/14 07:57, John Jasem wrote: > The two physical CPUs are: > Intel(R) Xeon(R) CPU E5-4610 0 @ 2.40GHz (2400.05-MHz K8-class CPU) > > Hyperthreading, at least from initial appearances, seems to offer no > benefits or drawbacks. > > I tested iperf3, using a packet generator on each subnet, each sending 4 > streams to a server on another subnet. > > maximum segment size of 128 and 1460 used, with little variance. (iperf3 > -M). > > A snapshot of netstat -d -b -w1 -W -h included. Midway through, the > numbers dropped. This coincides with launching this was when I launched > 16 more streams, 4 new clients, 4 new servers on different nets, 4 > streams each. > > input (Total) output > packets errs idrops bytes packets errs bytes colls drops > 1.6M 0 514 254M 1.6M 0 252M 0 5 > 1.6M 0 294 244M 1.6M 0 246M 0 6 > 1.6M 0 95 255M 1.5M 0 236M 0 6 > 1.4M 0 0 216M 1.5M 0 224M 0 3 > 1.5M 0 0 225M 1.4M 0 219M 0 4 > 1.4M 0 389 214M 1.4M 0 216M 0 1 > 1.4M 0 270 207M 1.4M 0 207M 0 1 > 1.4M 0 279 210M 1.4M 0 209M 0 2 > 1.4M 0 12 207M 1.3M 0 204M 0 1 > 1.4M 0 303 206M 1.4M 0 214M 0 2 > 1.3M 0 2.3K 190M 1.4M 0 212M 0 1 > 1.1M 0 1.1K 175M 1.1M 0 176M 0 1 > 1.1M 0 1.6K 176M 1.1M 0 175M 0 1 > 1.1M 0 830 176M 1.1M 0 174M 0 0 > 1.2M 0 1.5K 187M 1.2M 0 187M 0 0 > 1.2M 0 1.1K 183M 1.2M 0 184M 0 1 > 1.2M 0 1.5K 197M 1.2M 0 196M 0 2 > 1.3M 0 2.2K 199M 1.2M 0 196M 0 0 > 1.3M 0 2.8K 200M 1.3M 0 202M 0 4 > 1.3M 0 1.5K 199M 1.2M 0 198M 0 1 > > > vmstat also included. You see similar drops in faults. > > > procs memory page disks faults cpu > r b w avm fre flt re pi po fr sr mf0 cd0 in sy cs > us sy id > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 188799 224 > 387419 0 74 26 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 207447 150 > 425576 0 72 28 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 205638 202 > 421659 0 75 25 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 200292 150 > 411257 0 74 26 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 200338 197 > 411537 0 77 23 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 199289 156 > 409092 0 75 25 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 200504 200 > 411992 0 76 24 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 165042 152 > 341207 0 78 22 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 171360 200 > 353776 0 78 22 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 197557 150 > 405937 0 74 26 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 170696 204 > 353197 0 78 22 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 174927 150 > 361171 0 77 23 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 153836 200 > 319227 0 79 21 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 159056 150 > 329517 0 78 22 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 155240 200 > 321819 0 78 22 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 166422 156 > 344184 0 78 22 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 162065 200 > 335215 0 79 21 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 172857 150 > 356852 0 78 22 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 81267 197 > 176539 0 92 8 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 82151 150 > 177434 0 91 9 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 73904 204 > 160887 0 91 9 > 0 0 0 574M 15G 2 0 0 0 8 6 0 0 73820 150 > 161201 0 91 9 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 73926 196 > 161850 0 92 8 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 77215 150 > 166886 0 91 9 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 77509 198 > 169650 0 91 9 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 69993 156 > 154783 0 90 10 > 0 0 0 574M 15G 82 0 0 0 0 6 0 0 69722 199 > 153525 0 91 9 > 0 0 0 574M 15G 2 0 0 0 0 6 0 0 66353 150 > 147027 0 91 9 > 0 0 0 550M 15G 102 0 0 0 101 6 0 0 67906 259 > 149365 0 90 10 > 0 0 0 550M 15G 0 0 0 0 0 6 0 0 71837 125 > 157253 0 92 8 > 0 0 0 550M 15G 80 0 0 0 0 6 0 0 73508 179 > 161498 0 92 8 > 0 0 0 550M 15G 0 0 0 0 0 6 0 0 72673 125 > 159449 0 92 8 > 0 0 0 550M 15G 80 0 0 0 0 6 0 0 75630 175 > 164614 0 91 9 > > > > > On 07/11/2014 03:32 PM, Navdeep Parhar wrote: >> On 07/11/14 10:28, John Jasem wrote: >>> In testing two Chelsio T580-CR dual port cards with FreeBSD 10-STABLE, >>> I've been able to use a collection of clients to generate approximately >>> 1.5-1.6 million TCP packets per second sustained, and routinely hit >>> 10GB/s, both measured by netstat -d -b -w1 -W (I usually use -h for the >>> quick read, accepting the loss of granularity). >> When forwarding, the pps rate is often more interesting, and almost >> always the limiting factor, as compared to the total amount of data >> being passed around. 10GB at this pps probably means 9000 MTU. Try >> with 1500 too if possible. >> >> "netstat -d 1" and "vmstat 1" for a few seconds when your system is >> under maximum load would be useful. And what kind of CPU is in this system? >> >>> While performance has so far been stellar, and I'm honestly speculating >>> I will need more CPU depth and horsepower to get much faster, I'm >>> curious if there is any gain to tweaking performance settings. I'm >>> seeing, under multiple streams, with N targets connecting to N servers, >>> interrupts on all CPUs peg at 99-100%, and I'm curious if tweaking >>> configs will help, or its a free clue to get more horsepower. >>> >>> So, far, except for temporarily turning off pflogd, and setting the >>> following sysctl variables, I've not done any performance tuning on the >>> system yet. >>> >>> /etc/sysctl.conf >>> net.inet.ip.fastforwarding=1 >>> kern.random.sys.harvest.ethernet=0 >>> kern.random.sys.harvest.point_to_point=0 >>> kern.random.sys.harvest.interrupt=0 >>> >>> a) One of the first things I did in prior testing was to turn >>> hyperthreading off. I presume this is still prudent, as HT doesn't help >>> with interrupt handling? >> It is always worthwhile to try your workload with and without >> hyperthreading. >> >>> b) I briefly experimented with using cpuset(1) to stick interrupts to >>> physical CPUs, but it offered no performance enhancements, and indeed, >>> appeared to decrease performance by 10-20%. Has anyone else tried this? >>> What were your results? >>> >>> c) the defaults for the cxgbe driver appear to be 8 rx queues, and N tx >>> queues, with N being the number of CPUs detected. For a system running >>> multiple cards, routing or firewalling, does this make sense, or would >>> balancing tx and rx be more ideal? And would reducing queues per card >>> based on NUMBER-CPUS and NUM-CHELSIO-PORTS make sense at all? >> The defaults are nrxq = min(8, ncores) and ntxq = min(16, ncores). The >> man page mentions this. The reason for 8 vs. 16 is that tx queues are >> "cheaper" as they don't have to be backed by rx buffers. It only needs >> some memory for the tx descriptor ring and some hardware resources. >> >> It appears that your system has >= 16 cores. For forwarding it probably >> makes sense to have nrxq = ntxq. If you're left with 8 or fewer cores >> after disabling hyperthreading you'll automatically get 8 rx and tx >> queues. Otherwise you'll have to fiddle with the hw.cxgbe.nrxq10g and >> ntxq10g tunables (documented in the man page). >> >> >>> d) dev.cxl.$PORT.qsize_rxq: 1024 and dev.cxl.$PORT.qsize_txq: 1024. >>> These appear to not be writeable when if_cxgbe is loaded, so I speculate >>> they are not to be messed with, or are loader.conf variables? Is there >>> any benefit to messing with them? >> Can't change them after the port has been administratively brought up >> even once. This is mentioned in the man page. I don't really recommend >> changing them any way. >> >>> e) dev.t5nex.$CARD.toe.sndbuf: 262144. These are writeable, but messing >>> with values did not yield an immediate benefit. Am I barking up the >>> wrong tree, trying? >> The TOE tunables won't make a difference unless you have enabled TOE, >> the TCP endpoints lie on the system, and the connections are being >> handled by the TOE on the chip. This is not the case on your systems. >> The driver does not enable TOE by default and the only way to use it is >> to switch it on explicitly. There is no possibility that you're using >> it without knowing that you are. >> >>> f) based on prior experiments with other vendors, I tried tweaks to >>> net.isr.* settings, but did not see any benefits worth discussing. Am I >>> correct in this speculation, based on others experience? >>> >>> g) Are there other settings I should be looking at, that may squeeze out >>> a few more packets? >> The pps rates that you've observed are within the chip's hardware limits >> by at least an order of magnitude. Tuning the kernel rather than the >> driver may be the best bang for your buck. >> >> Regards, >> Navdeep >> >>> Thanks in advance! >>> >>> -- John Jasen (jjasen@gmail.com) > From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 19:57:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD826B4C for ; Mon, 14 Jul 2014 19:57:27 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E8AB2CC6 for ; Mon, 14 Jul 2014 19:57:27 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id hq11so7088414vcb.17 for ; Mon, 14 Jul 2014 12:57:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=aCB0wxLcOfcDBdp58uw7Oww4r2z2tMwQfTD1SG/hpd4=; b=sQFS6cGKttCIe2u7ulTrmddaeFBpCcSEoxZifOIHOYxGqhWvixz/Ru9onJwPUjk1b7 owewlMp5+lVjCRdUuEi02IFChI+NVkhF8/ms+dY538NmLfovkKctmlD4G9YG5m8cFkiG 5DmDKZTuXIezJJPNytsaWlIQVL4pu+F8prefFkdVekW+TopZid4IM4rXEdLLpApS3QFk cGKZziuW3YMk+d7UxHFPAxZMy6jqEGoVWZZ+AoGGV6K68zgOWR+Rm7T2Tl0rr1SpgqJC ZxEkbLkW6Es7PZCHxtu4CQG1xQnVeX8VbSZ9y6SQfGAw5qGT2pP6IjqIfSLk7Ov2RO+x /woA== MIME-Version: 1.0 X-Received: by 10.220.44.141 with SMTP id a13mr131869vcf.71.1405367846576; Mon, 14 Jul 2014 12:57:26 -0700 (PDT) Received: by 10.220.249.132 with HTTP; Mon, 14 Jul 2014 12:57:26 -0700 (PDT) Date: Mon, 14 Jul 2014 15:57:26 -0400 Message-ID: Subject: ng_iface regression from 9.2 to 10.0 From: Zaphod Beeblebrox To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 19:57:27 -0000 I'm going to post again with some new information. I have a 10.0p6 machine running mpd5 terminating a bunch of l2tp tunnels from subscribers (not encrypted). The specific regression between 9.2 and 10.0 is that hosts on the tunnels cannot communicate with local services. They can ping local IPs, and the server can ping them, but no userland connections can be had. IE: [2:15:315]root@owl:~> ifconfig ng29 ng29: flags=88d1 metric 0 mtu 1436 inet xx.yy.31.6 --> xx.yy.16.50 netmask 0xffffffff inet6 fe80::219:b9ff:fef9:b9e7%ng29 prefixlen 64 scopeid 0x23 nd6 options=21 [2:16:316]root@owl:~> ping xx.yy.16.50 PING xx.yy.16.50 (xx.yy.16.50): 56 data bytes 64 bytes from xx.yy.16.50: icmp_seq=0 ttl=64 time=11.580 ms 64 bytes from xx.yy.16.50: icmp_seq=1 ttl=64 time=16.515 ms 64 bytes from xx.yy.16.50: icmp_seq=2 ttl=64 time=6.253 ms ^C --- xx.yy.16.50 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 6.253/11.449/16.515/4.190 ms [2:17:317]root@owl:~> ssh xx.yy.16.50 ssh: connect to host xx.yy.16.50 port 22: Operation timed out It's worth noting, too, that all tunnel-connected hosts have full internet connectivity as does the tunnel server. Connections from one hop away (ie: not involving the tunnel server to run the process) work as usual. It's also worth noting that localhost and local-ip communication on the server are fine (ie: mpd5 communicates with radiusd running on the same machine). For interest's sake, xx.yy.16.50 is running mpd5 on 9.2. From owner-freebsd-net@FreeBSD.ORG Mon Jul 14 22:53:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41D4E117 for ; Mon, 14 Jul 2014 22:53:05 +0000 (UTC) Received: from quine.pinyon.org (quine.pinyon.org [65.101.5.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D7252CCA for ; Mon, 14 Jul 2014 22:53:04 +0000 (UTC) Received: by quine.pinyon.org (Postfix, from userid 122) id 3AB1D160338; Mon, 14 Jul 2014 15:52:58 -0700 (MST) Received: from feyerabend.n1.pinyon.org (feyerabend.n1.pinyon.org [10.0.10.6]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by quine.pinyon.org (Postfix) with ESMTPSA id 2F5931600CB for ; Mon, 14 Jul 2014 15:52:56 -0700 (MST) Message-ID: <53C45F48.6090607@pinyon.org> Date: Mon, 14 Jul 2014 15:52:56 -0700 From: "Russell L. Carter" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: NFS client READ performance on -current References: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> In-Reply-To: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 22:53:05 -0000 On 07/12/14 14:14, Rick Macklem wrote: > Russell seemed to confirm that the patch fixed the problem for him, > but since I don't have em(4) hardware, it would be nice to have someone > with commit privilege and access to em(4) hardware test and commit it. With Rick's quite minimal patch, NFSv4 client read performance has been solid for me ever since. Russell > > rick > >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org" >> From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 00:44:16 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C48AF297; Tue, 15 Jul 2014 00:44:16 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78A1B256B; Tue, 15 Jul 2014 00:44:16 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so1848060qaj.7 for ; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=FvN3gfIZgZCEaegkqLkRZsZs+kB/LvfwkJn2UNHjVFI=; b=pCmSgKvUj/I0TZPmRsKZpoc0cKyOX/T0SUG960r++eYdKWaNdckyJ4nIGKt8AiaQqs O9J9mkK7XFXAiaoZeWUQMNfkkGYk2G3dghCf9Ws1Q1Htg4w0UNbSQxLI4NtkA0E4hn65 JkbvEAXXVDExhi5YFWFwVplp5Z0Zu13QkH2/JTy0x5fQKkKVELR0vWR/vdbCW2xEr3xO AOHXuoTETzhUeK0Zpb6S3kMGjbQunisd/qml9AwNbwrKw5eUuOIyLqGAHU7406v0VrM0 IQSpXvWvKAguja+Hw0KXVBNC3XrCwCIfiyj+cFofUQrw3UDFjUTSrLCMg6Oqwaeo7wVh QHpA== MIME-Version: 1.0 X-Received: by 10.224.171.197 with SMTP id i5mr28743881qaz.55.1405385055606; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) Date: Mon, 14 Jul 2014 17:44:15 -0700 X-Google-Sender-Auth: drKNq-70BpKBM9JhjBBupuyS5po Message-ID: Subject: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 00:44:16 -0000 Hi, Whilst digging into UDP receive side scaling on the intel ixgbe(4) NIC, I stumbled across how it hashes traffic between IP fragmented traffic and non IP-fragmented traffic. Here's how it surfaced: * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and TCP/UDP (4-tuple); * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple and comes into queue A; * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple and comes into queue B. So if there's a mix of small and large datagrams, we'll end up with some packets coming in via queue A and some by queue B. In normal operation that'll result in out of order packets. For the RSS stuff I'm working on it means that some packets will match the PCBGROUP setup and some won't. By default UDP configures a 2-tuple hash so it expects packets to come in hashed appropriately. But that only matches for large frames. For small frames it'll be hashed via the 4-tuple and it won't match. The ip reassembly code doesn't recalculate the flowid/flowtype once it's finished. It'd be nice to do that before further processing so it can be placed in the right netisr. So there's a couple of semi-overlapping issues: * Right now we could get TCP and UDP frames out of order. I'd like to at least have ixgbe(4) hash on the 2-tuple for UDP rather than the 4-tuple. That fixes that silly corner case. It's not likely going to show up except for things like forwarding workloads. Maybe people doing memcached work, I'm not sure. * Whether or not to calculate the flowid/flowtype in ip_reass() (or maybe in the netisr input path, in case there's no flowid assigned) so work is better distributed; * .. then if we do that, we could do 4-tuple UDP hashing again and we'd just recalculate for any large frames. Here's what happened with Linux and ixgbe in 2010 on this topic: http://comments.gmane.org/gmane.linux.network/166687 What do people think? -a From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 01:08:24 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76A3E7FC for ; Tue, 15 Jul 2014 01:08:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E9C62733 for ; Tue, 15 Jul 2014 01:08:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6F18O84009848 for ; Tue, 15 Jul 2014 01:08:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 176671] [epair] MAC address for epair device not unique Date: Tue, 15 Jul 2014 01:08:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: dev.shutingrz@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 01:08:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671 Shuto.Imai changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dev.shutingrz@gmail.com --- Comment #3 from Shuto.Imai --- About this problem, is there any progress? Thankyou -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 02:04:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14DAED18 for ; Tue, 15 Jul 2014 02:04:01 +0000 (UTC) Received: from mail-oa0-x233.google.com (mail-oa0-x233.google.com [IPv6:2607:f8b0:4003:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D664D2B2C for ; Tue, 15 Jul 2014 02:04:00 +0000 (UTC) Received: by mail-oa0-f51.google.com with SMTP id o6so3244967oag.38 for ; Mon, 14 Jul 2014 19:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=A3QYFdBgHrZ4kgEi1oWCi1/DtuRpp4pMB/ndTKd5M94=; b=nyv3vYC/NiIVkowqVzKA9w+0oWxc60JZ51YlhnTeNNlV/ZOfYsdxf8rk7pay7aaPa5 MsaJFy4qYhbXl0gJh/x2r4gnMaQJ0ixloGmywB7Hj1LQ49yzAI1m3V3vE9bErKJD1kQ9 ZWrFwevWnv+8YeJtpd4OoXkmCJCBmFhLARe/lzZaUlSh+pfQrI+i0pEHEsZNj0QlHNvG ij+jgrwZY2o7OvGRztvMjZkocxJNJFD9LodzVk4T/Ej5zbUMiM90mULBnxgGwjDZDMkx lnBDHNBDjVZL+JsSejX/LnllowlWDYfoMOZpD9sJo/2GoQh5pKdWobfAKYQ2bh+Zes0I +lVw== MIME-Version: 1.0 X-Received: by 10.60.124.162 with SMTP id mj2mr21980806oeb.22.1405389840135; Mon, 14 Jul 2014 19:04:00 -0700 (PDT) Received: by 10.76.27.164 with HTTP; Mon, 14 Jul 2014 19:04:00 -0700 (PDT) Date: Tue, 15 Jul 2014 10:04:00 +0800 Message-ID: Subject: netmap: how to manage SW & NIC ring at same em0? From: upyzl To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 02:04:01 -0000 Hi all, I'm a newbie on netmap development using FreeBSD 10 release x64("device netmap" added and recompiled) I'm study on develop openflow-based datapath(software switch function/module) by netmap-bridge my understanding: ./bridge -i em0 -i em0 --- would be using SW ring for em0, and if there's a packet send to em0, it should forward to host stack; ./bridge -i em0 -i em1 --- normal bridge using netmap framework (NIC ring) (above in /usr/src/tools/tools/netmap) but in openflow, there's an required action "Forward: local", when a packet matched(I implemented matching flow by modified bridge.c->process_rings() ), the packeted should send to host stack, as SW ring does it is a possible scene: a ICMP packet arrived in em0, forward to em1; a TCP packet arrived in em0, forward to host stack; I want to know whether is a method to implement it? or could give me any advice... or in fact I'm wrong in understanding? Thanks in advance From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 06:21:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D247DEBC; Tue, 15 Jul 2014 06:21:13 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6CA222F90; Tue, 15 Jul 2014 06:21:13 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id hu12so4101117vcb.6 for ; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DMDHrfojD3PK1R3qhR+G//uG+VuerRoZxYFEL11Q4JA=; b=w5iwvuAnCQVUqBQh6pu81hf1OAccd/kXErxe7XonRnbh6xJm+yO7AdFyJoSxLVD/R4 QRvir/fLw7gLQkNqzfmReZLUNTCvOsEUYxcimQM3Wrxt7tlg2aU/sq8Vc8MdLtaR/pDR he0eo8i3/xmpz3SQXqx46cNUIh7GLhI97gJcVWFjNRPjM/vyOZja4Ku09RyeEfdpsNWW 1EyymacyqagvJ1ffuHpPRI/qfeuQFi+aSY5XlJTeKqoEMgAHeo15LRpu/7cGd+K1MBxq CxatMme9d8VXsqwCHLltsnC87pHs1aL0bPG9iOiCTTq/KfnJHrVkULxlfkwcL2fLBik6 YIsA== MIME-Version: 1.0 X-Received: by 10.220.2.136 with SMTP id 8mr20340949vcj.17.1405405272376; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) Received: by 10.221.53.199 with HTTP; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) In-Reply-To: References: Date: Mon, 14 Jul 2014 23:21:12 -0700 Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 06:21:13 -0000 I had missed the fact that Alex turned this off in the Linux driver, sounds to me like its the right thing to do for FreeBSD also. Jack On Mon, Jul 14, 2014 at 5:44 PM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 07:24:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54B67F83; Tue, 15 Jul 2014 07:24:59 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 220EF2588; Tue, 15 Jul 2014 07:24:59 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id et14so1339518pad.23 for ; Tue, 15 Jul 2014 00:24:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=PcczKJlQXd2Nid+bZBFSQ4jg9PHTO4QQC7uyR+d+dzs=; b=n0G/sFcfLwGXg+iHiKx62RdtTf0C9UVz6GgFRgkWNHVxAG1GGzvZCVqTVch/+XRm8D 3qe9pJav28Fpk/lYZSoBGixb1k7S6SMCstfGiG3G87XU+CzSE8o733EqF0V6OTNUWJt2 sQKYtNw2bSlwg/SliO4XRDMyFxHUXamciXnCkT323CJwKXkP/L/0Cwn95w8ZDiTxANvy ELhO3gSKdxWVIb5TnXmJdFbiasm+zI7O+PJwl6tbYuxSv2aosSZKCO4RgyQWyOI898Uh 4tODqNEKu0X37JepmcBl44HjBWLXdaT5eglFKee/9BP2UwTUM2Y2da/ZxmXtlvuqig83 waJg== X-Received: by 10.66.65.204 with SMTP id z12mr21094161pas.60.1405409098609; Tue, 15 Jul 2014 00:24:58 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by mx.google.com with ESMTPSA id ei4sm12967689pbb.42.2014.07.15.00.24.55 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 15 Jul 2014 00:24:57 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 15 Jul 2014 16:24:49 +0900 Date: Tue, 15 Jul 2014 16:24:49 +0900 To: Rick Macklem Subject: Re: NFS client READ performance on -current Message-ID: <20140715072449.GA1488@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <20140712060538.GA3649@michelle.fasterthan.com> <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.4.2.3i Cc: "Russell L. Carter" , freebsd-net@freebsd.org, John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 07:24:59 -0000 On Sat, Jul 12, 2014 at 05:14:00PM -0400, Rick Macklem wrote: > Yonghyeon Pyun wrote: > > On Fri, Jul 11, 2014 at 09:54:23AM -0400, John Baldwin wrote: > > > On Thursday, July 10, 2014 6:31:43 pm Rick Macklem wrote: > > > > John Baldwin wrote: > > > > > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > > > > > Russell L. Carter wrote: > > > > > > > > > > > > > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > > > > > > > > > Could you please post the dmesg stuff for the network > > > > > > > > interface, > > > > > > > > so I can tell what driver is being used? I'll take a look > > > > > > > > at > > > > > > > > it, > > > > > > > > in case it needs to be changed to use m_defrag(). > > > > > > > > > > > > > > em0: port > > > > > > > 0xd020-0xd03f > > > > > > > mem > > > > > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at > > > > > > > device 0.0 > > > > > > > on > > > > > > > pci2 > > > > > > > em0: Using an MSI interrupt > > > > > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > > > > > 001.000007 [2323] netmap_attach success for em0 > > > > > > > tx > > > > > > > 1/1024 > > > > > > > rx > > > > > > > 1/1024 queues/slots > > > > > > > > > > > > > > This is one of those dual nic cards, so there is em1 as > > > > > > > well... > > > > > > > > > > > > > Well, I took a quick look at the driver and it does use > > > > > > m_defrag(), > > > > > > but > > > > > > I think that the "retry:" label it does a goto after doing so > > > > > > might > > > > > > be in > > > > > > the wrong place. > > > > > > > > > > > > The attached untested patch might fix this. > > > > > > > > > > > > Is it convenient to build a kernel with this patch applied > > > > > > and then > > > > > > try > > > > > > it with TSO enabled? > > > > > > > > > > > > rick > > > > > > ps: It does have the transmit segment limit set to 32. I have > > > > > > no > > > > > > idea if > > > > > > this is a hardware limitation. > > > > > > > > > > I think the retry is not in the wrong place, but the overhead > > > > > of all > > > > > those > > > > > pullups is apparently quite severe. > > > > The m_defrag() call after the first failure will just barely > > > > squeeze > > > > the just under 64K TSO segment into 32 mbuf clusters. Then I > > > > think any > > > > m_pullup() done during the retry will allocate an mbuf > > > > (at a glance it seems to always do this when the old mbuf is a > > > > cluster) > > > > and prepend that to the list. > > > > --> Now the list is > 32 mbufs again and the > > > > bus_dmammap_load_mbuf_sg() > > > > will fail again on the retry, this time fatally, I think? > > > > > > > > I can't see any reason to re-do all the stuff using m_pullup() > > > > and Russell > > > > reported that moving the "retry:" fixed his problem, from what I > > > > understood. > > > > > > Ah, I had assumed (incorrectly) that the m_pullup()s would all be > > > nops in this > > > case. It seems the NIC would really like to have all those things > > > in a single > > > segment, but it is not required, so I agree that your patch is > > > fine. > > > > > > > I recall em(4) controllers have various limitation in TSO. Driver > > has to update IP header to make TSO work so driver has to get a > > writable mbufs. bpf(4) consumers will see IP packet length is 0 > > after this change. I think tcpdump has a compile time option to > > guess correct IP packet length. The firmware of controller also > > should be able to access complete IP/TCP header in a single buffer. > > I don't remember more details in TSO limitation but I guess you may > > be able to get more details TSO limitation from publicly available > > Intel data sheet. > I think that the patch should handle this ok. All of the m_pullup() > stuff gets done the first time. Then, if the result is more than 32 > mbufs in the list, m_defrag() is called to copy the chain. This should > result in all the header stuff in the first mbuf cluster and the map > call is done again with this list of clusters. (Without the patch, > m_pullup() would allocate another prepended mbuf and make the chain > more than 32mbufs again.) > Yes, your patch looks right. > Russell seemed to confirm that the patch fixed the problem for him, > but since I don't have em(4) hardware, it would be nice to have someone > with commit privilege and access to em(4) hardware test and commit it. > Due to breakage of power supply on a box with em(4) controller, I can't test the patch. But I guess it's ok to commit it and Russel already tested it. Thanks for your patch. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 08:22:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89462F85; Tue, 15 Jul 2014 08:22:43 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 327B92ABE; Tue, 15 Jul 2014 08:22:43 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id hu12so4312468vcb.6 for ; Tue, 15 Jul 2014 01:22:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wEE+H+8ny99ncj8SA3C575ussOmyDM35l3W+eVhkwYA=; b=yAzD+6oP0rjZ9JFMZflvp8R1CO56NUxdzTnetGAEmnqHtuZofzwDXXShntqtAO53Al eFtOYYUdUEg4uRjAAgG3QgbVRuzKx7HqTzvNhYLM703KTRYgA7O449rI6Nz2K6Nskp9d Xp71OrgzEHwe8ljq2xpLlp9BFT4cQeH04d7nog+DF62wm6fp/zUIpiwzVRgAvHLdIVxU oRmWDLareerE+nkYMFng1oUaX9UXT9lMYIX8IBW7BPxY5hiUjJdhAm8R/aLWuLlu8KKJ S2SN6309EMvdTtZcTlUrkVt38Owkcd8kZ+9BSDpg/Rbt0yc0TcSBq/f+M0RnFLTVigd7 w3Ug== MIME-Version: 1.0 X-Received: by 10.58.190.105 with SMTP id gp9mr12356841vec.17.1405412562121; Tue, 15 Jul 2014 01:22:42 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 01:22:42 -0700 (PDT) In-Reply-To: <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Tue, 15 Jul 2014 10:22:42 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 08:22:43 -0000 Hi, I found other problems in the "oce" driver during some experiments with netmap in emulation mode. In details: - missing locking: - in some functions there are write accesses on the wq struct (tx queue descriptor) without acquire LOCK on the queue, particularly in oce_wq_handler() that is invoked in the interrupt routine. For this reason there may be race conditions. - tx cleanup: - in oce_if_deactivate() the wq queues are drained but some still pending mbufs are not freed. For this reason, I added the oce_tx_clean() that releases any pending mbufs. I also tried experimenting with iperf3 using the same Borja environment and I don't have panic. Can you try this patch? Do you still have the panic? Cheers, Stefano Garzarella diff --git a/sys/dev/oce/oce_if.c b/sys/dev/oce/oce_if.c index af57491..33b35b4 100644 --- a/sys/dev/oce/oce_if.c +++ b/sys/dev/oce/oce_if.c @@ -142,6 +142,7 @@ static int oce_tx(POCE_SOFTC sc, struct mbuf **mpp, int wq_index); static void oce_tx_restart(POCE_SOFTC sc, struct oce_wq *wq); static void oce_tx_complete(struct oce_wq *wq, uint32_t wqe_idx, uint32_t status); +static void oce_tx_clean(POCE_SOFTC sc); static int oce_multiq_transmit(struct ifnet *ifp, struct mbuf *m, struct oce_wq *wq); @@ -585,8 +586,10 @@ oce_multiq_flush(struct ifnet *ifp) int i = 0; for (i = 0; i < sc->nwqs; i++) { + LOCK(&sc->wq[i]->tx_lock); while ((m = buf_ring_dequeue_sc(sc->wq[i]->br)) != NULL) m_freem(m); + UNLOCK(&sc->wq[i]->tx_lock); } if_qflush(ifp); } @@ -1052,6 +1055,19 @@ oce_tx_complete(struct oce_wq *wq, uint32_t wqe_idx, uint32_t status) } } +static void +oce_tx_clean(POCE_SOFTC sc) { + int i = 0; + struct oce_wq *wq; + + for_all_wq_queues(sc, wq, i) { + LOCK(&wq->tx_lock); + while (wq->pkt_desc_tail != wq->pkt_desc_head) { + oce_tx_complete(wq, 0, 0); + } + UNLOCK(&wq->tx_lock); + } +} static void oce_tx_restart(POCE_SOFTC sc, struct oce_wq *wq) @@ -1213,6 +1229,8 @@ oce_wq_handler(void *arg) struct oce_nic_tx_cqe *cqe; int num_cqes = 0; + LOCK(&wq->tx_lock); + bus_dmamap_sync(cq->ring->dma.tag, cq->ring->dma.map, BUS_DMASYNC_POSTWRITE); cqe = RING_GET_CONSUMER_ITEM_VA(cq->ring, struct oce_nic_tx_cqe); @@ -1237,6 +1255,8 @@ oce_wq_handler(void *arg) if (num_cqes) oce_arm_cq(sc, cq->cq_id, num_cqes, FALSE); + UNLOCK(&wq->tx_lock); + return 0; } @@ -2087,6 +2107,9 @@ oce_if_deactivate(POCE_SOFTC sc) /* Delete RX queue in card with flush param */ oce_stop_rx(sc); + /* Flush the mbufs that are still in TX queues */ + oce_tx_clean(sc); + /* Invalidate any pending cq and eq entries*/ for_all_evnt_queues(sc, eq, i) oce_drain_eq(eq); diff --git a/sys/dev/oce/oce_queue.c b/sys/dev/oce/oce_queue.c index 308c16d..161011b 100644 --- a/sys/dev/oce/oce_queue.c +++ b/sys/dev/oce/oce_queue.c @@ -969,7 +969,9 @@ oce_start_rq(struct oce_rq *rq) int oce_start_wq(struct oce_wq *wq) { + LOCK(&wq->tx_lock); /* XXX: maybe not necessary */ oce_arm_cq(wq->parent, wq->cq->cq_id, 0, TRUE); + UNLOCK(&wq->tx_lock); return 0; } @@ -1076,6 +1078,8 @@ oce_drain_wq_cq(struct oce_wq *wq) struct oce_nic_tx_cqe *cqe; int num_cqes = 0; + LOCK(&wq->tx_lock); /* XXX: maybe not necessary */ + bus_dmamap_sync(cq->ring->dma.tag, cq->ring->dma.map, BUS_DMASYNC_POSTWRITE); @@ -1093,6 +1097,7 @@ oce_drain_wq_cq(struct oce_wq *wq) oce_arm_cq(sc, cq->cq_id, num_cqes, FALSE); + UNLOCK(&wq->tx_lock); } 2014-07-07 13:57 GMT+02:00 Borja Marcos : > > On Jul 7, 2014, at 1:23 PM, Luigi Rizzo wrote: > > > On Mon, Jul 7, 2014 at 1:03 PM, Borja Marcos wrote: > > we'll try to investigate, can you tell us more about the environment you > use ? > > (FreeBSD version, card model (PCI id perhaps), iperf3 invocation line, > > interface configuration etc.) > > > > The main differences between 10.0.747.0 and the code in head (after > > our fix) is the use > > of drbr_enqueue/dequeue versus the peek/putback in the transmit routine. > > > > > > Both drivers still have issues when the link flaps because the > > transmit queue is not cleaned > > up properly (unlike what happens in the linux driver and all FreeBSD > > drivers for different > > hardware), so it might well be that you are seeing some side effect of > > that or other > > problem which manifests itself differently depending on the environment. > > > > 'instant panic' by itself does not tell us anything about what could > > be the problem you experience (and we do not see it with either driver). > > The environment details are here: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 > > The way I produce an instant panic is: > > 1) Connect to another machine (cross connect cable) > > 2) iperf3 -s on the other machine > (The other machine is different, it has an "ix" card) > > 3) iperf3 -t 30 -P 4 -c 10.0.0.1 -N > > In less than 30 seconds, panic. > > > > mierda dumped core - see /var/crash/vmcore.0 > > Mon Jul 7 13:06:44 CEST 2014 > > FreeBSD mierda 10.0-STABLE FreeBSD 10.0-STABLE #2: Mon Jul 7 11:41:45 > CEST 2014 root@mierda:/usr/obj/usr/src/sys/GENERIC amd64 > > panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 > clashing > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > panic: sbsndptr: sockbuf 0xfffff800a70489b0 and mbuf 0xfffff801a3326e00 > clashing > cpuid = 12 > KDB: stack backtrace: > #0 0xffffffff8092a470 at kdb_backtrace+0x60 > #1 0xffffffff808ef9c5 at panic+0x155 > #2 0xffffffff80962710 at sbdroprecord_locked+0 > #3 0xffffffff80a8ba8c at tcp_output+0xdbc > #4 0xffffffff80a8987f at tcp_do_segment+0x30ff > #5 0xffffffff80a85b34 at tcp_input+0xd04 > #6 0xffffffff80a1af57 at ip_input+0x97 > #7 0xffffffff809ba512 at netisr_dispatch_src+0x62 > #8 0xffffffff809b1ae6 at ether_demux+0x126 > #9 0xffffffff809b278e at ether_nh_input+0x35e > #10 0xffffffff809ba512 at netisr_dispatch_src+0x62 > #11 0xffffffff81c19ab9 at oce_rx+0x3c9 > #12 0xffffffff81c19536 at oce_rq_handler+0xb6 > #13 0xffffffff81c1bb1c at oce_intr+0xdc > #14 0xffffffff80938b35 at taskqueue_run_locked+0xe5 > #15 0xffffffff809395c8 at taskqueue_thread_loop+0xa8 > #16 0xffffffff808c057a at fork_exit+0x9a > #17 0xffffffff80ccb51e at fork_trampoline+0xe > Uptime: 51m20s > > > > > > > > > > > > > > Borja. > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Stefano Garzarella From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 08:39:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDC05690; Tue, 15 Jul 2014 08:39:21 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 85D382BE9; Tue, 15 Jul 2014 08:39:21 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 4569B9DC5BB; Tue, 15 Jul 2014 10:29:01 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 10:28:55 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 08:39:21 -0000 On Jul 15, 2014, at 10:22 AM, Stefano Garzarella wrote: > Hi, > I found other problems in the "oce" driver during some experiments = with > netmap in emulation mode. What about driver version 10.0.747.0? At least in my configuration it = works perfectly, no crashes despite keeping it running for several days = at full bandwidth. I have a server about to go into production. Should this patch work on = 10-STABLE? Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 08:43:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27219A1C; Tue, 15 Jul 2014 08:43:03 +0000 (UTC) Received: from mail-vc0-x233.google.com (mail-vc0-x233.google.com [IPv6:2607:f8b0:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C42C82C8B; Tue, 15 Jul 2014 08:43:02 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id id10so9426257vcb.38 for ; Tue, 15 Jul 2014 01:43:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=skgA9jSlAnEkimEGSYlRjj+9OoTI8li8XYSB7xBlNbA=; b=dFyzRAlpalYRmsWBOFt+XiS7ag80T9/JYZCEhPjPc6G67vZlgD06FBPGmZl+jpouzV t2/bFOZ0RnRJ0ywAJueRjJcXuscY6X1BfqYV8OD9If3hxb85kpw/OVQoT9XN2wwp1Ys6 1FyP/NkZIbRAKJ6eWFa6is9SUrzb50QLGo2vSErCJm+F8LEt2owQa4Wv8ANitBlAKOf0 IxCF+wi8PHLO0ZisV3NsouuVmfd+srvMpVv9cl402tEnW9qBwVKmrOBP7vuAbHE07acq c3yVZ6TnyxS1HrGDeYpuAygTGLuEo7FNccecJptzizWrXQ1hc+Tyh1b3k2W3iHrmykZU XSxA== MIME-Version: 1.0 X-Received: by 10.58.15.129 with SMTP id x1mr926582vec.2.1405413781836; Tue, 15 Jul 2014 01:43:01 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 01:43:01 -0700 (PDT) In-Reply-To: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Tue, 15 Jul 2014 10:43:01 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 08:43:03 -0000 I used the "oce" driver in CURRENT. I think that this patch in combination with the previous one should work in 10-STABLE. I have only tested if it works with CURRENT, but now I try if it works with 10-STABLE and I'll send you some feedback. Cheers, Stefano 2014-07-15 10:28 GMT+02:00 Borja Marcos : > > On Jul 15, 2014, at 10:22 AM, Stefano Garzarella wrote: > > > Hi, > > I found other problems in the "oce" driver during some experiments with > > netmap in emulation mode. > > What about driver version 10.0.747.0? At least in my configuration it > works perfectly, no crashes despite keeping it running for several days at > full bandwidth. > > I have a server about to go into production. Should this patch work on > 10-STABLE? > > > > > > > Borja. > > > -- Stefano Garzarella From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:02:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7293184; Tue, 15 Jul 2014 09:02:49 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 163962E50; Tue, 15 Jul 2014 09:02:48 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id e16so2191931lan.39 for ; Tue, 15 Jul 2014 02:02:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=n/4LTRxM71tw09Lb29UdD1Z/Zt09C9FQuAsVj1pSy8M=; b=rQKbDcAOrqMyjPb38vnOTiwSEzbGk97D9z+Og412gx+R7VnwWT9AvKStNL/7fAvfZ+ eDG3g9pC/tg2EtEXUHx0MawNpOqRu+jVrIzNjAUUruwIDtBqK+i18wJd5Y7CekqDUZH5 tYR1OxU/NpdtKVqZ1VTid4qWPtE9zrnLKouF9JIXPzMk6z5XYSlHSkWx/JBMFlHt9uUW AK0ntAamS0a0P2J14XTx+3/2TDVfOCdt0B2PHcUpebwEcgr1NCyhav7foHqyqiGtfc3y F3IxXxeHoFnDtb8HW4z41AXlTV8w37qzoBXUJjVvpPqJ9/touZfg+ztrQC/jsOwijyDP aD9g== X-Received: by 10.152.205.99 with SMTP id lf3mr1460665lac.63.1405414966866; Tue, 15 Jul 2014 02:02:46 -0700 (PDT) Received: from [192.168.2.30] ([2.176.190.226]) by mx.google.com with ESMTPSA id qx6sm19344336lbb.23.2014.07.15.02.01.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jul 2014 02:02:46 -0700 (PDT) Message-ID: <53C4EE00.5090705@gmail.com> Date: Tue, 15 Jul 2014 13:31:52 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:02:50 -0000 On 7/15/2014 5:14 AM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" Doesn't the problem applies to TCP too? TCP may be fragmented too but is less likely because of MSS. -- Best regards. Hooman Fazaeli From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:03:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1642E2C5; Tue, 15 Jul 2014 09:03:53 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C11C92E62; Tue, 15 Jul 2014 09:03:52 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 19C3D9DC899; Tue, 15 Jul 2014 11:03:50 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 11:03:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <8FD26C21-17BD-404C-ADA2-5DD42859B751@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:03:53 -0000 On Jul 15, 2014, at 10:43 AM, Stefano Garzarella wrote: > I used the "oce" driver in CURRENT.=20 > I think that this patch in combination with the previous one should = work in 10-STABLE. >=20 > I have only tested if it works with CURRENT, but now I try if it works = with 10-STABLE and I'll send you some feedback. I can still try. Will get back to you soon. Cheers, Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:12:13 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7A90646; Tue, 15 Jul 2014 09:12:13 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E3182F41; Tue, 15 Jul 2014 09:12:12 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 1553E9DCD6B; Tue, 15 Jul 2014 11:12:11 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 11:12:09 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:12:13 -0000 On Jul 15, 2014, at 10:43 AM, Stefano Garzarella wrote: > I used the "oce" driver in CURRENT. > I think that this patch in combination with the previous one should = work in > 10-STABLE. >=20 > I have only tested if it works with CURRENT, but now I try if it works = with > 10-STABLE and I'll send you some feedback. Hmmm. The patch seems to be broken. I have tried to apply it renaming = the a/usr/src... to oce_if.c.old and oce_if.c, etc, and patch complains: Patching file oce_if.c using Plan A... patch: **** malformed patch at line 6: int wq_index); Was it broken by the email client formatting? Or am I being especially = clumsy today? ;) Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:19:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 065CF869; Tue, 15 Jul 2014 09:19:40 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EBC72F9D; Tue, 15 Jul 2014 09:19:39 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id im17so9738684vcb.11 for ; Tue, 15 Jul 2014 02:19:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bd3qciFuFsBfGhQtZ6nC7Rnl+fHWg4d1m6+QZvPbLQI=; b=TE1dTI7KbY/AG6VkFgk5R9dwgxKQ2B7WZSJS0WUKi/trTllHdqRmVS+ZeGvEyLeyc7 FjgPLGU9lx0cRAWwytBjaytSA1A81Xxx9iqasgYf51filwthWPWVz40C507xP5MtERvm 8zDm3uIqGjFBMaWL5Wn6hZlrHOrYBv4RTyB/cHVMPY0wdgkAW/Lwv077lL7/8qw1eDbu hVLU414tt6BpEMnCjNvt2Ch27kH2i/5YVnukz2PpOfVsUhL48l+yCb9/gIf2FoyerIJ5 pXJFE+qJTVuCNRWpWH903HO1niYlU/zGoYumOGlYuEilfBEXpJSvrEhaKZNt9Dc4gXNu Idgw== MIME-Version: 1.0 X-Received: by 10.58.19.10 with SMTP id a10mr21608261vee.1.1405415978739; Tue, 15 Jul 2014 02:19:38 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 02:19:38 -0700 (PDT) In-Reply-To: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Tue, 15 Jul 2014 11:19:38 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: multipart/mixed; boundary=e89a8f8393fd2cd48c04fe37e814 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:19:40 -0000 --e89a8f8393fd2cd48c04fe37e814 Content-Type: text/plain; charset=UTF-8 I think there is some problem with the email formatting. I send you a file with both patches. Cheers, Stefano 2014-07-15 11:12 GMT+02:00 Borja Marcos : > > On Jul 15, 2014, at 10:43 AM, Stefano Garzarella wrote: > > > I used the "oce" driver in CURRENT. > > I think that this patch in combination with the previous one should work > in > > 10-STABLE. > > > > I have only tested if it works with CURRENT, but now I try if it works > with > > 10-STABLE and I'll send you some feedback. > > Hmmm. The patch seems to be broken. I have tried to apply it renaming the > a/usr/src... to oce_if.c.old and oce_if.c, etc, and patch complains: > > Patching file oce_if.c using Plan A... > patch: **** malformed patch at line 6: int wq_index); > > > Was it broken by the email client formatting? Or am I being especially > clumsy today? ;) > > > > > Borja. > > -- Stefano Garzarella --e89a8f8393fd2cd48c04fe37e814 Content-Type: application/octet-stream; name="oce_fix_STABLE10.patch" Content-Disposition: attachment; filename="oce_fix_STABLE10.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hxn0clrk0 ZGlmZiAtLWdpdCBhL3N5cy9kZXYvb2NlL29jZV9pZi5jIGIvc3lzL2Rldi9vY2Uvb2NlX2lmLmMK aW5kZXggNDhmODVlNi4uYzI5NWI4MCAxMDA2NDQKLS0tIGEvc3lzL2Rldi9vY2Uvb2NlX2lmLmMK KysrIGIvc3lzL2Rldi9vY2Uvb2NlX2lmLmMKQEAgLTE0Miw2ICsxNDIsNyBAQCBzdGF0aWMgaW50 ICBvY2VfdHgoUE9DRV9TT0ZUQyBzYywgc3RydWN0IG1idWYgKiptcHAsIGludCB3cV9pbmRleCk7 CiBzdGF0aWMgdm9pZCBvY2VfdHhfcmVzdGFydChQT0NFX1NPRlRDIHNjLCBzdHJ1Y3Qgb2NlX3dx ICp3cSk7CiBzdGF0aWMgdm9pZCBvY2VfdHhfY29tcGxldGUoc3RydWN0IG9jZV93cSAqd3EsIHVp bnQzMl90IHdxZV9pZHgsCiAJCQkJCXVpbnQzMl90IHN0YXR1cyk7CitzdGF0aWMgdm9pZCBvY2Vf dHhfY2xlYW4oUE9DRV9TT0ZUQyBzYyk7CiBzdGF0aWMgaW50ICBvY2VfbXVsdGlxX3RyYW5zbWl0 KHN0cnVjdCBpZm5ldCAqaWZwLCBzdHJ1Y3QgbWJ1ZiAqbSwKIAkJCQkgc3RydWN0IG9jZV93cSAq d3EpOwogCkBAIC01NjMsOSArNTY0LDYgQEAgb2NlX211bHRpcV9zdGFydChzdHJ1Y3QgaWZuZXQg KmlmcCwgc3RydWN0IG1idWYgKm0pCiAJaW50IHF1ZXVlX2luZGV4ID0gMDsKIAlpbnQgc3RhdHVz ID0gMDsKIAotCWlmICghc2MtPmxpbmtfc3RhdHVzKQotCQlyZXR1cm4gRU5YSU87Ci0KIAlpZiAo KG0tPm1fZmxhZ3MgJiBNX0ZMT1dJRCkgIT0gMCkKIAkJcXVldWVfaW5kZXggPSBtLT5tX3BrdGhk ci5mbG93aWQgJSBzYy0+bndxczsKIApAQCAtNTg4LDggKzU4NiwxMCBAQCBvY2VfbXVsdGlxX2Zs dXNoKHN0cnVjdCBpZm5ldCAqaWZwKQogCWludCBpID0gMDsKIAogCWZvciAoaSA9IDA7IGkgPCBz Yy0+bndxczsgaSsrKSB7CisJCUxPQ0soJnNjLT53cVtpXS0+dHhfbG9jayk7CiAJCXdoaWxlICgo bSA9IGJ1Zl9yaW5nX2RlcXVldWVfc2Moc2MtPndxW2ldLT5icikpICE9IE5VTEwpCiAJCQltX2Zy ZWVtKG0pOworCQlVTkxPQ0soJnNjLT53cVtpXS0+dHhfbG9jayk7CiAJfQogCWlmX3FmbHVzaChp ZnApOwogfQpAQCAtMTA1NSw2ICsxMDU1LDE5IEBAIG9jZV90eF9jb21wbGV0ZShzdHJ1Y3Qgb2Nl X3dxICp3cSwgdWludDMyX3Qgd3FlX2lkeCwgdWludDMyX3Qgc3RhdHVzKQogCX0KIH0KIAorc3Rh dGljIHZvaWQgCitvY2VfdHhfY2xlYW4oUE9DRV9TT0ZUQyBzYykgeworCWludCBpID0gMDsKKwlz dHJ1Y3Qgb2NlX3dxICp3cTsKKwkKKwlmb3JfYWxsX3dxX3F1ZXVlcyhzYywgd3EsIGkpIHsKKwkJ TE9DSygmd3EtPnR4X2xvY2spOworCQl3aGlsZSAod3EtPnBrdF9kZXNjX3RhaWwgIT0gd3EtPnBr dF9kZXNjX2hlYWQpIHsKKwkJCW9jZV90eF9jb21wbGV0ZSh3cSwgMCwgMCk7CisJCX0KKwkJVU5M T0NLKCZ3cS0+dHhfbG9jayk7CisJfQorfQogCiBzdGF0aWMgdm9pZAogb2NlX3R4X3Jlc3RhcnQo UE9DRV9TT0ZUQyBzYywgc3RydWN0IG9jZV93cSAqd3EpCkBAIC0xMjE2LDYgKzEyMjksOCBAQCBv Y2Vfd3FfaGFuZGxlcih2b2lkICphcmcpCiAJc3RydWN0IG9jZV9uaWNfdHhfY3FlICpjcWU7CiAJ aW50IG51bV9jcWVzID0gMDsKIAorCUxPQ0soJndxLT50eF9sb2NrKTsKKwogCWJ1c19kbWFtYXBf c3luYyhjcS0+cmluZy0+ZG1hLnRhZywKIAkJCWNxLT5yaW5nLT5kbWEubWFwLCBCVVNfRE1BU1lO Q19QT1NUV1JJVEUpOwogCWNxZSA9IFJJTkdfR0VUX0NPTlNVTUVSX0lURU1fVkEoY3EtPnJpbmcs IHN0cnVjdCBvY2VfbmljX3R4X2NxZSk7CkBAIC0xMjQwLDYgKzEyNTUsOCBAQCBvY2Vfd3FfaGFu ZGxlcih2b2lkICphcmcpCiAJaWYgKG51bV9jcWVzKQogCQlvY2VfYXJtX2NxKHNjLCBjcS0+Y3Ff aWQsIG51bV9jcWVzLCBGQUxTRSk7CiAKKwlVTkxPQ0soJndxLT50eF9sb2NrKTsKKwogCXJldHVy biAwOwogfQogCkBAIC0xMjc0LDcgKzEyOTEsNiBAQCBvY2VfbXVsdGlxX3RyYW5zbWl0KHN0cnVj dCBpZm5ldCAqaWZwLCBzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IG9jZV93cSAqd3EpCiAJCQkJZHJi cl9wdXRiYWNrKGlmcCwgYnIsIG5leHQpOwogCQkJCXdxLT50eF9zdGF0cy50eF9zdG9wcyArKzsK IAkJCQlpZnAtPmlmX2Rydl9mbGFncyB8PSBJRkZfRFJWX09BQ1RJVkU7Ci0JCQkJc3RhdHVzID0g ZHJicl9lbnF1ZXVlKGlmcCwgYnIsIG5leHQpOwogCQkJfSAgCiAJCQlicmVhazsKIAkJfQpAQCAt MTI4NSw3ICsxMzAxLDcgQEAgb2NlX211bHRpcV90cmFuc21pdChzdHJ1Y3QgaWZuZXQgKmlmcCwg c3RydWN0IG1idWYgKm0sIHN0cnVjdCBvY2Vfd3EgKndxKQogCQlFVEhFUl9CUEZfTVRBUChpZnAs IG5leHQpOwogCX0KIAotCXJldHVybiBzdGF0dXM7CisJcmV0dXJuIDA7CiB9CiAKIApAQCAtMjA5 MSw2ICsyMTA3LDkgQEAgb2NlX2lmX2RlYWN0aXZhdGUoUE9DRV9TT0ZUQyBzYykKIAkvKiBEZWxl dGUgUlggcXVldWUgaW4gY2FyZCB3aXRoIGZsdXNoIHBhcmFtICovCiAJb2NlX3N0b3Bfcngoc2Mp OwogCisJLyogRmx1c2ggdGhlIG1idWZzIHRoYXQgYXJlIHN0aWxsIGluIFRYIHF1ZXVlcyAqLwor CW9jZV90eF9jbGVhbihzYyk7CisKIAkvKiBJbnZhbGlkYXRlIGFueSBwZW5kaW5nIGNxIGFuZCBl cSBlbnRyaWVzKi8JCiAJZm9yX2FsbF9ldm50X3F1ZXVlcyhzYywgZXEsIGkpCQogCQlvY2VfZHJh aW5fZXEoZXEpOwpkaWZmIC0tZ2l0IGEvc3lzL2Rldi9vY2Uvb2NlX3F1ZXVlLmMgYi9zeXMvZGV2 L29jZS9vY2VfcXVldWUuYwppbmRleCAzMDhjMTZkLi4xNjEwMTFiIDEwMDY0NAotLS0gYS9zeXMv ZGV2L29jZS9vY2VfcXVldWUuYworKysgYi9zeXMvZGV2L29jZS9vY2VfcXVldWUuYwpAQCAtOTY5 LDcgKzk2OSw5IEBAIG9jZV9zdGFydF9ycShzdHJ1Y3Qgb2NlX3JxICpycSkKIGludAogb2NlX3N0 YXJ0X3dxKHN0cnVjdCBvY2Vfd3EgKndxKQogeworCUxPQ0soJndxLT50eF9sb2NrKTsgLyogWFhY OiBtYXliZSBub3QgbmVjZXNzYXJ5ICovCiAJb2NlX2FybV9jcSh3cS0+cGFyZW50LCB3cS0+Y3Et PmNxX2lkLCAwLCBUUlVFKTsKKwlVTkxPQ0soJndxLT50eF9sb2NrKTsKIAlyZXR1cm4gMDsKIH0K IApAQCAtMTA3Niw2ICsxMDc4LDggQEAgb2NlX2RyYWluX3dxX2NxKHN0cnVjdCBvY2Vfd3EgKndx KQogICAgICAgICBzdHJ1Y3Qgb2NlX25pY190eF9jcWUgKmNxZTsKICAgICAgICAgaW50IG51bV9j cWVzID0gMDsKIAorCUxPQ0soJndxLT50eF9sb2NrKTsgLyogWFhYOiBtYXliZSBub3QgbmVjZXNz YXJ5ICovCisKIAlidXNfZG1hbWFwX3N5bmMoY3EtPnJpbmctPmRtYS50YWcsIGNxLT5yaW5nLT5k bWEubWFwLAogCQkJCSBCVVNfRE1BU1lOQ19QT1NUV1JJVEUpOwogCkBAIC0xMDkzLDYgKzEwOTcs NyBAQCBvY2VfZHJhaW5fd3FfY3Eoc3RydWN0IG9jZV93cSAqd3EpCiAKIAlvY2VfYXJtX2NxKHNj LCBjcS0+Y3FfaWQsIG51bV9jcWVzLCBGQUxTRSk7CiAKKwlVTkxPQ0soJndxLT50eF9sb2NrKTsK IH0KIAogCg== --e89a8f8393fd2cd48c04fe37e814-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:33:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DFC4127 for ; Tue, 15 Jul 2014 09:33:05 +0000 (UTC) Received: from mail-wg0-f48.google.com (mail-wg0-f48.google.com [74.125.82.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A82FF216A for ; Tue, 15 Jul 2014 09:33:04 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id x13so5348097wgg.7 for ; Tue, 15 Jul 2014 02:32:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:user-agent:mime-version :content-type:content-transfer-encoding:message-id; bh=ZesQ3XPVMOxtzdEtcYL/y+oh03L97mCgXg75WiKglqM=; b=Sh1DZbMeNQBeuHb4X/9MqSVCERDj2RLiZvyHhX+9ZwdHh+A0faJ26le6RdESnqR/QB a8H73ueSygHxbYDklDNmibvyGL1xJtoBDUuA+2OyIGhUOren2cYMAIQDneGqyRxpcU8H YXv2HvkYDMTghrzuV2joZuTMwYsS4/UjqTyKWbw+n/hmnRXTxr58rf3lXdRuK6CSPWxN 87Gu0+gmLUxVAuW+RKfrpt3WRmO8LK/UxFTzPz+tdJQYxy88DCYIAaE+B+moDabsVNJA IPM4fJrb5kLWMwtgVPcfpKf/J3F40BI+o08WUtQoJPsncOg4MDOBhyMrS22opMMjI/gh /FHQ== X-Gm-Message-State: ALoCoQngHoK31X7y3SWCdhIDr/5mtET2BFiMmCmtlH5n5Kq4kSZXrS2fRbWwlmRDG+UxMo0i5zZ6 X-Received: by 10.180.14.33 with SMTP id m1mr4274232wic.50.1405416775770; Tue, 15 Jul 2014 02:32:55 -0700 (PDT) Received: from zvezda.localnet ([2a00:1f78:fffb:320:f2de:f1ff:fef6:bc5]) by mx.google.com with ESMTPSA id v4sm14046872wiz.16.2014.07.15.02.32.54 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jul 2014 02:32:54 -0700 (PDT) From: Kajetan Staszkiewicz To: freebsd-net@freebsd.org Subject: Why is r250764 not in 9.3? Date: Tue, 15 Jul 2014 11:32:46 +0200 User-Agent: KMail/1.13.7 (Linux/3.10.1; KDE/4.8.4; x86_64; ; ) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2560940.iJON7QEusJ"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201407151132.53587.vegeta@tuxpowered.net> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:33:05 -0000 --nextPart2560940.iJON7QEusJ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The time has come to upgrade my routers to FreeBSD 9.3. While going through list of patches I had on 9.1, I've noticed that r248070= got=20 into 9.3 but r250764 did not. Why is that? =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart2560940.iJON7QEusJ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEABECAAYFAlPE9T8ACgkQ47RQr217OhTaEwCeNpQQkyZqRmH8nJmXozQKz1Oo 2G0An0sO3qEYWlmSZRBJXnqVwGHX9ubb =hwBR -----END PGP SIGNATURE----- --nextPart2560940.iJON7QEusJ-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:33:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 419691BA; Tue, 15 Jul 2014 09:33:44 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 001992178; Tue, 15 Jul 2014 09:33:43 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id DFBEC1FE027; Tue, 15 Jul 2014 11:33:42 +0200 (CEST) Message-ID: <53C4F580.6020304@selasky.org> Date: Tue, 15 Jul 2014 11:33:52 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, FreeBSD Current Subject: Re: [RFC] Add support for changing the flow ID of TCP connections References: <53BC2E73.6090700@selasky.org> <53BC43AE.3040409@FreeBSD.org> <53BD5385.4090208@selasky.org> <20140709163146.GA21731@ox> In-Reply-To: <20140709163146.GA21731@ox> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:33:44 -0000 On 07/09/14 18:31, Navdeep Parhar wrote: > On Wed, Jul 09, 2014 at 04:36:53PM +0200, Hans Petter Selasky wrote: >> On 07/08/14 21:17, Navdeep Parhar wrote: > ... >>> >>> I think we need to design this to be as generic as possible. I have >>> quite a bit of code that does this stuff but I haven't pushed it >>> upstream or even offered it for review (yet). >>> >> >> Hi, >> >> When will the non hardware related patches be available for review? >> I understand there are multiple ways to reach the same goal, and I >> think it would be great if we could agree on a common API for >> applications. > > Here is the kernel side of the patch: > http://people.freebsd.org/~np/flow_pacing_kern.diff > > The registration parameters and the throttling parameters are probably > cxgbe-centric, because that's what it was written for. We'll need to > tidy up those structs certainly. And I'd like to add pps constraints to > the throttling parameters (all it does is bandwidth right now). Hi Navdeep, After reviewing your patch, we've concluded that we can't use your flow-ID API's AS-IS for the mlxen hardware. We are working on a new patch proposal after the feedback received here, and will probably post something in August, hence now it is vacation time and not all people are available. Also we are worried that the m_tags cause un-needed overhead, that it invokes malloc for every pkt header duplication on transmit. As to give you some clues: One of FreeBSD's targets is firewalls and routers. We think that a flow ID / hardware queue feature should also be usable by the firewall, and not only limited to TCP/UDP. That means you can create queues for multiple connections which then get rate limited as a firewall rule. Right now our main target is not the firewall, but we see that by minor modifications of the APIs, this feature becomes very easy to implement. --HPS From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:45:19 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE43D543; Tue, 15 Jul 2014 09:45:19 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64BB02281; Tue, 15 Jul 2014 09:45:19 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id id10so9827985vcb.30 for ; Tue, 15 Jul 2014 02:45:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Og7D/6PUF8ooiT73dkZEs8QaZclmVcuCXvHMw/5Ea4g=; b=VXIkLKri42Z+YrbqhMi9OBvj8G126Hfnw3ubpfW5YmKqQCHy9Pn57aWAa9aCduPM3V vbD0V/LMcKA6syi32tEMN85QIXgScsiufkUClGx9eN/xsWIlsBr+GTmSQAcFvdTGRlFg /dul+3gda10jqaoJkFigGhm79+Dyj+yt5l2hfOQ3AtwEgz0V4jFtD8nfFDNX4XsJ2+tT VuExodDoMMDzgqQulE5mvHcswNkiC1FPdHse3tb5+710iadZbMnvb5ENKedDlqJMuG2X /5eaBNm+uo9jFXqQHfm8Rk7h4JCiounGLPlk7G5uzasGZNcz8oiTLEN2KOSFV71rDBKw Hceg== MIME-Version: 1.0 X-Received: by 10.52.0.177 with SMTP id 17mr17868091vdf.12.1405417517826; Tue, 15 Jul 2014 02:45:17 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 02:45:17 -0700 (PDT) In-Reply-To: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Tue, 15 Jul 2014 11:45:17 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:45:19 -0000 I just tried to run iperf3 with this patch and STABLE-10 and it seems to work. Do you have a panic? Cheers, Stefano 2014-07-15 11:19 GMT+02:00 Stefano Garzarella : > I think there is some problem with the email formatting. > I send you a file with both patches. > > Cheers, > Stefano > > > 2014-07-15 11:12 GMT+02:00 Borja Marcos : > > >> On Jul 15, 2014, at 10:43 AM, Stefano Garzarella wrote: >> >> > I used the "oce" driver in CURRENT. >> > I think that this patch in combination with the previous one should >> work in >> > 10-STABLE. >> > >> > I have only tested if it works with CURRENT, but now I try if it works >> with >> > 10-STABLE and I'll send you some feedback. >> >> Hmmm. The patch seems to be broken. I have tried to apply it renaming the >> a/usr/src... to oce_if.c.old and oce_if.c, etc, and patch complains: >> >> Patching file oce_if.c using Plan A... >> patch: **** malformed patch at line 6: int wq_index); >> >> >> Was it broken by the email client formatting? Or am I being especially >> clumsy today? ;) >> >> >> >> >> Borja. >> >> > > > -- > Stefano Garzarella > -- Stefano Garzarella From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:46:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7BA26E9; Tue, 15 Jul 2014 09:46:15 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9BF17229E; Tue, 15 Jul 2014 09:46:15 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 7192E9DC62A; Tue, 15 Jul 2014 11:46:11 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 11:46:10 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <42DD9481-C121-4A53-8D1D-E272689593FD@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:46:16 -0000 On Jul 15, 2014, at 11:45 AM, Stefano Garzarella wrote: > I just tried to run iperf3 with this patch and STABLE-10 and it seems = to work. > Do you have a panic? Still compiling :) Anyway, you didn't suffer panics before, right? Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 09:56:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91A3CA04; Tue, 15 Jul 2014 09:56:52 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36D7F238A; Tue, 15 Jul 2014 09:56:52 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id hq11so8493732vcb.17 for ; Tue, 15 Jul 2014 02:56:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=oDXUwasXv16ML+SksKSAmXT6eJo9y2DDUg1h4/3rF5k=; b=AjoVPkD3ypFl7ktYmG6odg+9W8dWYoHlF1qCuTRigMLqGmNpNQXEM13Fz//OdnAfQ4 9Xtx8vduIhz9cJof1LG8d9x9FeTErwh/xiuDL1xeUFG+wRJYQrxVNz49tMjsVMM+ya+z GRRKAiTJMWqUnzjp9Cdvk+cexecavaw6qn/K775PkOixUExV5q7WuOZ6gVVuYd0fot6R Cj4tUGQXkkQ4LhaZz7uwwnDm/kCdxq9BtQo+k5jwVHeGAxTqqRom0Udq4wOoZoYtYARB 2TP67iFJhtPJUkNI+4CtDmMdNzdJzFEZ/mIx3I80Wm+1OAXN3RWCik5oMh8e/F5rhhMs CmKg== MIME-Version: 1.0 X-Received: by 10.58.182.105 with SMTP id ed9mr21346327vec.16.1405418211379; Tue, 15 Jul 2014 02:56:51 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 02:56:51 -0700 (PDT) In-Reply-To: <42DD9481-C121-4A53-8D1D-E272689593FD@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> <42DD9481-C121-4A53-8D1D-E272689593FD@sarenet.es> Date: Tue, 15 Jul 2014 11:56:51 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:56:52 -0000 2014-07-15 11:46 GMT+02:00 Borja Marcos : > > On Jul 15, 2014, at 11:45 AM, Stefano Garzarella wrote: > > > I just tried to run iperf3 with this patch and STABLE-10 and it seems to > work. > > Do you have a panic? > > Still compiling :) Anyway, you didn't suffer panics before, right? Right, I didn't suffer panics with iperf3, but with netmap in emulation mode I had a lot of panics before this patch. Stefano > > > > Borja. > > -- Stefano Garzarella From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 10:00:54 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35DF0CD5; Tue, 15 Jul 2014 10:00:54 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE10423BE; Tue, 15 Jul 2014 10:00:53 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 2AFC59DCE0A; Tue, 15 Jul 2014 12:00:51 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 12:00:49 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 10:00:54 -0000 On Jul 15, 2014, at 11:45 AM, Stefano Garzarella wrote: > I just tried to run iperf3 with this patch and STABLE-10 and it seems = to > work. > Do you have a panic? So far, so good. I've ran a couple of iperf3 tests (60 seconds, trying = both directions) and it doesn't crash. Without the fixes I obtained a panic quite reliably, in less than 30 = seconds. Still trying. But the bugs you mentioned (lack of locking and = deallocating, etc) seem to be consistent with the kind of failures I saw = and their apparent randomness. So, asking for spiritual counsel now. Would you use this driver in a = production environment instead of the 747 version downloaded from = Emulex? I think the latter is giving slightly better performance but, = anyway, I disable LRO and TSO because I see a horrible impact on NFS = performance. Cheers, Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 10:36:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15E4BA08 for ; Tue, 15 Jul 2014 10:36:40 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDECF26DF for ; Tue, 15 Jul 2014 10:36:39 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id kq14so4132678pab.26 for ; Tue, 15 Jul 2014 03:36:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:mime-version:content-type:content-transfer-encoding :date:message-id; bh=rMCQXcfD43gSYCP4ChNn3UqSbS4IieYlNgpy1oCnmPA=; b=qY112kgRZ1Hpkaw2AXDmbd0hdLkhD0O4OSHASqEjA4ETQc0w6l/WESYY19Ycbd9QLl 1XeU2VQz5qzMHAIFcXoujI+bC32cu0xP8kIJL4x8v803A0XoId5xTBC2Hkcxpd+vnWLx 8Jio6aCPUjW6ThEhR0EXFj4h4G3eb4FkjPhzDEQH44XAuB1qXTg8YblKB7Y/m4AWjxNI okcWxBLmtSRz6zAzK/NhEyGh0vi1ldIEYwPf4fnYQmDOgOcE9MC9NoX7o26aU70w0AC/ FxXrqPAqrc0OBKIzhf7dddP8LE7zMAziN9FWC1vHR/x9kCToiL0stri9tvLa7JgJW05w GUJA== X-Received: by 10.70.64.201 with SMTP id q9mr2955415pds.130.1405420599464; Tue, 15 Jul 2014 03:36:39 -0700 (PDT) Received: from smtpbg308.qq.com ([120.196.211.86]) by mx.google.com with ESMTPSA id z3sm56588795pas.15.2014.07.15.03.36.37 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 15 Jul 2014 03:36:38 -0700 (PDT) X-QQ-FEAT: z1X8ZlCQvrlllRFyDMEmgOIqymi1Ivcs9BiX+61N46VB0fD4JcyiYkkRpUMZK O7xDzcS/IoGTyj+8HCtm8fzF3gMD3yqNuIZUXVuZXLXTd471z1jMdE25CmROd8nm6b6OfIz DiZZ/IYVPGHyNs3qWcc0aYzqfCPFoLgEZLTD9N4= X-QQ-SSF: 000000000000000000000000000000S X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 14.154.224.222 X-QQ-STYLE: X-QQ-mid: webmail809t1405420594t992681 From: "=?utf-8?B?Y2hr?=" To: "=?utf-8?B?ZnJlZWJzZC1uZXQ=?=" Subject: tincd and mpd5 make kernel panic Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Date: Tue, 15 Jul 2014 18:36:34 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-SENDSIZE: 520 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 10:36:40 -0000 SGksICBldmVyeW9uZSwNCkhlbHAuLi4uDQpJIGhhdmUgYSB0aW5jZCB2cG4gcnVubmluZyBp biBmcmVlYnNkIGJveCAgRnJlZUJTRCAxMC4wLVJFTEVBU0UtcDIgIzAgcjI2NTMxOE0uDQpi ZWxvdyBpcyBpZmNvbmZpZyBvdXR1dDoNCltjaGtATlVDIH5dJCBpZmNvbmZpZw0KZW0wOiBm bGFncz04ODQzPFVQLEJST0FEQ0FTVCxSVU5OSU5HLFNJTVBMRVgsTVVMVElDQVNUPiBtZXRy aWMgMCBtdHUgMTUwMA0KICAgICAgICBvcHRpb25zPTQyMTliPFJYQ1NVTSxUWENTVU0sVkxB Tl9NVFUsVkxBTl9IV1RBR0dJTkcsVkxBTl9IV0NTVU0sVFNPNCxXT0xfTUFHSUMsVkxBTl9I V1RTTz4NCiAgICAgICAgZXRoZXIgZWM6YTg6NmI6ZjM6NzY6NmENCiAgICAgICAgaW5ldCAx OTIuMTY4LjIuMjAyIG5ldG1hc2sgMHhmZmZmZmYwMCBicm9hZGNhc3QgMjU1LjI1NS4yNTUu MjU1DQogICAgICAgIG5kNiBvcHRpb25zPTI5PFBFUkZPUk1OVUQsSUZESVNBQkxFRCxBVVRP X0xJTktMT0NBTD4NCiAgICAgICAgbWVkaWE6IEV0aGVybmV0IGF1dG9zZWxlY3QgKDEwMGJh c2VUWCA8ZnVsbC1kdXBsZXg+KQ0KICAgICAgICBzdGF0dXM6IGFjdGl2ZQ0KbG8wOiBmbGFn cz04MDQ5PFVQLExPT1BCQUNLLFJVTk5JTkcsTVVMVElDQVNUPiBtZXRyaWMgMCBtdHUgMTYz ODQNCiAgICAgICAgb3B0aW9ucz02MDAwMDM8UlhDU1VNLFRYQ1NVTSxSWENTVU1fSVBWNixU WENTVU1fSVBWNj4NCiAgICAgICAgaW5ldDYgOjoxIHByZWZpeGxlbiAxMjgNCiAgICAgICAg aW5ldDYgZmU4MDo6MSVsbzAgcHJlZml4bGVuIDY0IHNjb3BlaWQgMHgyDQogICAgICAgIGlu ZXQgMTI3LjAuMC4xIG5ldG1hc2sgMHhmZjAwMDAwMA0KICAgICAgICBuZDYgb3B0aW9ucz0y MTxQRVJGT1JNTlVELEFVVE9fTElOS0xPQ0FMPg0KcnVuMDogZmxhZ3M9ODg0MzxVUCxCUk9B RENBU1QsUlVOTklORyxTSU1QTEVYLE1VTFRJQ0FTVD4gbWV0cmljIDAgbXR1IDIyOTANCiAg ICAgICAgZXRoZXIgYzg6M2E6MzU6YzA6Yjg6MmYNCiAgICAgICAgbmQ2IG9wdGlvbnM9Mjk8 UEVSRk9STU5VRCxJRkRJU0FCTEVELEFVVE9fTElOS0xPQ0FMPg0KICAgICAgICBtZWRpYTog SUVFRSA4MDIuMTEgV2lyZWxlc3MgRXRoZXJuZXQgYXV0b3NlbGVjdCBtb2RlIDExZw0KICAg ICAgICBzdGF0dXM6IGFzc29jaWF0ZWQNCmVtMC4zOiBmbGFncz04ODQzPFVQLEJST0FEQ0FT VCxSVU5OSU5HLFNJTVBMRVgsTVVMVElDQVNUPiBtZXRyaWMgMCBtdHUgMTUwMA0KICAgICAg ICBvcHRpb25zPTEwMzxSWENTVU0sVFhDU1VNLFRTTzQ+DQogICAgICAgIGV0aGVyIGVjOmE4 OjZiOmYzOjc2OjZhDQogICAgICAgIGluZXQgMTkyLjE2OC4zLjEgbmV0bWFzayAweGZmZmZm ZjAwIGJyb2FkY2FzdCAyNTUuMjU1LjI1NS4wDQogICAgICAgIGluZXQ2IGZlODA6OmVlYTg6 NmJmZjpmZWYzOjc2NmElZW0wLjMgcHJlZml4bGVuIDY0IHNjb3BlaWQgMHg0DQogICAgICAg IG5kNiBvcHRpb25zPTI5PFBFUkZPUk1OVUQsSUZESVNBQkxFRCxBVVRPX0xJTktMT0NBTD4N CiAgICAgICAgbWVkaWE6IEV0aGVybmV0IGF1dG9zZWxlY3QgKDEwMGJhc2VUWCA8ZnVsbC1k dXBsZXg+KQ0KICAgICAgICBzdGF0dXM6IGFjdGl2ZQ0KICAgICAgICB2bGFuOiAzIHBhcmVu dCBpbnRlcmZhY2U6IGVtMA0Kd2xhbjA6IGZsYWdzPTg4NDM8VVAsQlJPQURDQVNULFJVTk5J TkcsU0lNUExFWCxNVUxUSUNBU1Q+IG1ldHJpYyAwIG10dSAxNTAwDQogICAgICAgIGV0aGVy IGM4OjNhOjM1OmMwOmI4OjJmDQogICAgICAgIGluZXQgMTkyLjE2OC4zMC4yMjIgbmV0bWFz ayAweGZmZmZmZjAwIGJyb2FkY2FzdCAyNTUuMjU1LjI1NS4wDQogICAgICAgIGluZXQ2IGZl ODA6OmNhM2E6MzVmZjpmZWMwOmI4MmYld2xhbjAgcHJlZml4bGVuIDY0IHNjb3BlaWQgMHg1 DQogICAgICAgIG5kNiBvcHRpb25zPTI5PFBFUkZPUk1OVUQsSUZESVNBQkxFRCxBVVRPX0xJ TktMT0NBTD4NCiAgICAgICAgbWVkaWE6IElFRUUgODAyLjExIFdpcmVsZXNzIEV0aGVybmV0 IGF1dG9zZWxlY3QgKGF1dG9zZWxlY3QpDQogICAgICAgIHN0YXR1czogbm8gY2Fycmllcg0K ICAgICAgICBzc2lkICIiIGNoYW5uZWwgMTAgKDI0NTcgTUh6IDExZykNCiAgICAgICAgY291 bnRyeSBVUyBhdXRobW9kZSBXUEExK1dQQTIvODAyLjExaSBwcml2YWN5IE1JWEVEIGRlZnR4 a2V5IFVOREVGDQogICAgICAgIHR4cG93ZXIgMCBibWlzcyA3IHNjYW52YWxpZCA2MCBwcm90 bW9kZSBDVFMgd21lIHJvYW1pbmcgTUFOVUFMDQp0dW4wOiBmbGFncz04MDQzPFVQLEJST0FE Q0FTVCxSVU5OSU5HLE1VTFRJQ0FTVD4gbWV0cmljIDAgbXR1IDE1MDANCiAgICAgICAgb3B0 aW9ucz04MDAwMDxMSU5LU1RBVEU+DQogICAgICAgIGluZXQgMTkyLjE2OC4zMC4yNTQgbmV0 bWFzayAweGZmZmZmZjAwIGJyb2FkY2FzdCAxOTIuMTY4LjMwLjI1NQ0KICAgICAgICBpbmV0 NiBmZTgwOjplZWE4OjZiZmY6ZmVmMzo3NjZhJXR1bjAgcHJlZml4bGVuIDY0IHNjb3BlaWQg MHg2DQogICAgICAgIG5kNiBvcHRpb25zPTIxPFBFUkZPUk1OVUQsQVVUT19MSU5LTE9DQUw+ DQogICAgICAgIE9wZW5lZCBieSBQSUQgMTAxNQ0KPT09PT09PT09PT09PT09PT09PT09PT09 PQ0KQ29udmVuaWVudCBmb3IgY29ubmVjdCB0byB2cG4sIEkgYWRkIGEgcHBwb2Vfc2VydmVy IHRvIG1wZDUsIGJ1dCB3aGVuIGNsaWVudCBkaWFsaW5nIHVwLCBrZXJuZWwgcGFuaWMuDQoN CmhlcmUgaXMgaW5mb3JtYXRpb24gb2YgY29yZSBkdW1wOg0KW3Jvb3RATlVDIC92YXIvbG9n XSMga2dkYiAtYyAuLi9jcmFzaC92bWNvcmUuMSAgL2Jvb3Qva2VybmVsL2tlcm5lbA0KR05V IGdkYiA2LjEuMSBbRnJlZUJTRF0NCkNvcHlyaWdodCAyMDA0IEZyZWUgU29mdHdhcmUgRm91 bmRhdGlvbiwgSW5jLg0KR0RCIGlzIGZyZWUgc29mdHdhcmUsIGNvdmVyZWQgYnkgdGhlIEdO VSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlLCBhbmQgeW91IGFyZQ0Kd2VsY29tZSB0byBjaGFu Z2UgaXQgYW5kL29yIGRpc3RyaWJ1dGUgY29waWVzIG9mIGl0IHVuZGVyIGNlcnRhaW4gY29u ZGl0aW9ucy4NClR5cGUgInNob3cgY29weWluZyIgdG8gc2VlIHRoZSBjb25kaXRpb25zLg0K VGhlcmUgaXMgYWJzb2x1dGVseSBubyB3YXJyYW50eSBmb3IgR0RCLiAgVHlwZSAic2hvdyB3 YXJyYW50eSIgZm9yIGRldGFpbHMuDQpUaGlzIEdEQiB3YXMgY29uZmlndXJlZCBhcyAiYW1k NjQtbWFyY2VsLWZyZWVic2QiLi4uDQoNClVucmVhZCBwb3J0aW9uIG9mIHRoZSBrZXJuZWwg bWVzc2FnZSBidWZmZXI6DQoNCg0KRmF0YWwgdHJhcCAxMjogcGFnZSBmYXVsdCB3aGlsZSBp biBrZXJuZWwgbW9kZQ0KY3B1aWQgPSAwOyBhcGljIGlkID0gMDANCmZhdWx0IHZpcnR1YWwg YWRkcmVzcyAgID0gMHgwDQpmYXVsdCBjb2RlICAgICAgICAgICAgICA9IHN1cGVydmlzb3Ig cmVhZCBkYXRhLCBwYWdlIG5vdCBwcmVzZW50DQppbnN0cnVjdGlvbiBwb2ludGVyICAgICA9 IDB4MjA6MHhmZmZmZmZmZjgwOThhZTA5DQpzdGFjayBwb2ludGVyICAgICAgICAgICA9IDB4 Mjg6MHhmZmZmZmUwMjM0NTg0NjYwDQpmcmFtZSBwb2ludGVyICAgICAgICAgICA9IDB4Mjg6 MHhmZmZmZmUwMjM0NTg0NmYwDQpjb2RlIHNlZ21lbnQgICAgICAgICAgICA9IGJhc2UgMHgw LCBsaW1pdCAweGZmZmZmLCB0eXBlIDB4MWINCiAgICAgICAgICAgICAgICAgICAgICAgID0g RFBMIDAsIHByZXMgMSwgbG9uZyAxLCBkZWYzMiAwLCBncmFuIDENCnByb2Nlc3NvciBlZmxh Z3MgICAgICAgID0gaW50ZXJydXB0IGVuYWJsZWQsIHJlc3VtZSwgSU9QTCA9IDANCmN1cnJl bnQgcHJvY2VzcyAgICAgICAgID0gNzgwICh3cGFfc3VwcGxpY2FudCkNCnRyYXAgbnVtYmVy ICAgICAgICAgICAgID0gMTINCnBhbmljOiBwYWdlIGZhdWx0DQpjcHVpZCA9IDANCktEQjog c3RhY2sgYmFja3RyYWNlOg0KIzAgMHhmZmZmZmZmZjgwOGY1OTEwIGF0IGtkYl9iYWNrdHJh Y2UrMHg2MA0KIzEgMHhmZmZmZmZmZjgwOGJkM2Y1IGF0IHBhbmljKzB4MTU1DQojMiAweGZm ZmZmZmZmODBjOWMxZTIgYXQgdHJhcF9mYXRhbCsweDNhMg0KIzMgMHhmZmZmZmZmZjgwYzlj NGI5IGF0IHRyYXBfcGZhdWx0KzB4MmM5DQojNCAweGZmZmZmZmZmODBjOWJjNDYgYXQgdHJh cCsweDVlNg0KIzUgMHhmZmZmZmZmZjgwYzgyZWUyIGF0IGNhbGx0cmFwKzB4OA0KIzYgMHhm ZmZmZmZmZjgwOTg1MmYwIGF0IHJuX3dhbGt0cmVlKzB4NzANCiM3IDB4ZmZmZmZmZmY4MDk4 YTQ3MCBhdCBzeXNjdGxfcnRzb2NrKzB4MWEwDQojOCAweGZmZmZmZmZmODA4Yzg5NGYgYXQg c3lzY3RsX3Jvb3QrMHgyNGYNCiM5IDB4ZmZmZmZmZmY4MDhjOGYwOCBhdCB1c2VybGFuZF9z eXNjdGwrMHgxZDgNCiMxMCAweGZmZmZmZmZmODA4YzhjZjQgYXQgc3lzX19fc3lzY3RsKzB4 NzQNCiMxMSAweGZmZmZmZmZmODBjOWNhZDcgYXQgYW1kNjRfc3lzY2FsbCsweDM1Nw0KIzEy IDB4ZmZmZmZmZmY4MGM4MzFjYiBhdCBYZmFzdF9zeXNjYWxsKzB4ZmINClVwdGltZTogMTBt MjRzDQooYWRhMDphaGNpY2gwOjA6MDowKTogU1RBTkRCWV9JTU1FRElBVEUuIEFDQjogZTAg MDAgMDAgMDAgMDAgNDAgMDAgMDAgMDAgMDAgMDAgMDANCihhZGEwOmFoY2ljaDA6MDowOjAp OiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCBpcyBpbiBwcm9ncmVzcw0KKGFkYTA6YWhjaWNo MDowOjA6MCk6IEVycm9yIDUsIFJldHJpZXMgZXhoYXVzdGVkDQooYWRhMDphaGNpY2gwOjA6 MDowKTogU3Bpbi1kb3duIGRpc2sgZmFpbGVkDQpEdW1waW5nIDQzOSBvdXQgb2YgODA2NyBN QjouLjQlLi4xMSUuLjIyJS4uMzMlLi40MSUuLjUxJS4uNjIlLi43MyUuLjgxJS4uOTIlDQoo a2dkYikgZiA3DQojNyAgMHhmZmZmZmZmZjgwOThhZTA5IGluIHN5c2N0bF9kdW1wZW50cnkg KHJuPTB4ZmZmZmY4MDAxMTBlYWUxMCwgdnc9MHhmZmZmZmUwMjM0NTg0NzQ4KQ0KICAgIGF0 IC91c3Ivc3JjL3N5cy9uZXQvcnRzb2NrLmM6MTU5Mg0KMTU5MiAgICAgICAgICAgICAgICAg ICAgaW5mby5ydGlfaW5mb1tSVEFYX0lGUF0gPSBydC0+cnRfaWZwLT5pZl9hZGRyLT5pZmFf YWRkcjsNCkN1cnJlbnQgbGFuZ3VhZ2U6ICBhdXRvOyBjdXJyZW50bHkgbWluaW1hbA0KKGtn ZGIpIGwNCjE1ODcgICAgICAgICAgICBpbmZvLnJ0aV9pbmZvW1JUQVhfRFNUXSA9IHJ0X2tl eShydCk7DQoxNTg4ICAgICAgICAgICAgaW5mby5ydGlfaW5mb1tSVEFYX0dBVEVXQVldID0g cnQtPnJ0X2dhdGV3YXk7DQoxNTg5ICAgICAgICAgICAgaW5mby5ydGlfaW5mb1tSVEFYX05F VE1BU0tdID0gcnRfbWFzayhydCk7DQoxNTkwICAgICAgICAgICAgaW5mby5ydGlfaW5mb1tS VEFYX0dFTk1BU0tdID0gMDsNCjE1OTEgICAgICAgICAgICBpZiAocnQtPnJ0X2lmcCkgew0K MTU5MiAgICAgICAgICAgICAgICAgICAgaW5mby5ydGlfaW5mb1tSVEFYX0lGUF0gPSBydC0+ cnRfaWZwLT5pZl9hZGRyLT5pZmFfYWRkcjsNCjE1OTMgICAgICAgICAgICAgICAgICAgIGlu Zm8ucnRpX2luZm9bUlRBWF9JRkFdID0gcnQtPnJ0X2lmYS0+aWZhX2FkZHI7DQoxNTk0ICAg ICAgICAgICAgICAgICAgICBpZiAocnQtPnJ0X2lmcC0+aWZfZmxhZ3MgJiBJRkZfUE9JTlRP UE9JTlQpDQoxNTk1ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGluZm8ucnRpX2luZm9b UlRBWF9CUkRdID0gcnQtPnJ0X2lmYS0+aWZhX2RzdGFkZHI7DQoxNTk2ICAgICAgICAgICAg fQ0KKGtnZGIpIGkgbG9jDQppbmZvID0ge3J0aV9hZGRycyA9IDAsIHJ0aV9pbmZvID0gezB4 ZmZmZmY4MDAwNWRhY2UwMCwgMHhmZmZmZjgwMDA1ZGFjZTEwLCAweDAsIDB4MCwgMHgwLCAw eDAsDQogICAgMHgwLCAweDB9LCBydGlfZmxhZ3MgPSAwLCBydGlfaWZhID0gMHgwLCBydGlf aWZwID0gMHgwfQ0KZXJyb3IgPSBDYW5ub3QgYWNjZXNzIG1lbW9yeSBhdCBhZGRyZXNzIDB4 MA0KKGtnZGIpIHAgKnJ0DQpObyBzeW1ib2wgInJ0IiBpbiBjdXJyZW50IGNvbnRleHQuDQoo a2dkYikgcCBydA0KTm8gc3ltYm9sICJydCIgaW4gY3VycmVudCBjb250ZXh0Lg0KKGtnZGIp IHAgaW5mbw0KJDEgPSB7cnRpX2FkZHJzID0gMCwgcnRpX2luZm8gPSB7MHhmZmZmZjgwMDA1 ZGFjZTAwLCAweGZmZmZmODAwMDVkYWNlMTAsIDB4MCwgMHgwLCAweDAsIDB4MCwgMHgwLA0K ICAgIDB4MH0sIHJ0aV9mbGFncyA9IDAsIHJ0aV9pZmEgPSAweDAsIHJ0aV9pZnAgPSAweDB9 From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 11:36:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C0ED872; Tue, 15 Jul 2014 11:36:06 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40C962BF5; Tue, 15 Jul 2014 11:36:06 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id ik5so9847270vcb.35 for ; Tue, 15 Jul 2014 04:36:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mS7JtPmz61C524kAn8UsUYs3HgqWHfsN4nj2EoxsM2M=; b=X50RsnR/ULkgxg/xdQCaTffJQi20acrcQJnQN/EPVUV3tS6ZO6rZtls2KAAgQPb1Di cpXrkbmkGP35ag5ZEdqVztxVSNbr4ZXeXYGBWp6+IY3nQcxm2bH/TRlfWwSbuLt6N/ls 5UucGZzNaNm+V7ib6St/Kp6jbusn4E58XNDMua6w2kqz70Y2IEMs8U6f7cc/KbdRKaOB OQJ//J8oAEEuesnedcel4MsKbGCl6Lbl/gQcPtBYgfZwCdHXNAailiNu6zMhO9vPPJyP YhKBa5u57UPXHDXYRSVkT3Zscza+9qb6KjUN5Wi3rkSkFQoqFwULosWXaGDEO7ELiIHY FdLQ== MIME-Version: 1.0 X-Received: by 10.221.59.194 with SMTP id wp2mr229982vcb.59.1405424165306; Tue, 15 Jul 2014 04:36:05 -0700 (PDT) Received: by 10.58.161.102 with HTTP; Tue, 15 Jul 2014 04:36:05 -0700 (PDT) In-Reply-To: References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> Date: Tue, 15 Jul 2014 13:36:05 +0200 Message-ID: Subject: Re: Fix Emulex "oce" driver in CURRENT From: Stefano Garzarella To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 11:36:06 -0000 2014-07-15 12:00 GMT+02:00 Borja Marcos : > > On Jul 15, 2014, at 11:45 AM, Stefano Garzarella wrote: > > > I just tried to run iperf3 with this patch and STABLE-10 and it seems to > > work. > > Do you have a panic? > > So far, so good. I've ran a couple of iperf3 tests (60 seconds, trying > both directions) and it doesn't crash. > > Without the fixes I obtained a panic quite reliably, in less than 30 > seconds. > Still trying. But the bugs you mentioned (lack of locking and > deallocating, etc) seem to be consistent with the kind of failures I saw > and their apparent randomness. > Well. > > So, asking for spiritual counsel now. Would you use this driver in a > production environment instead of the 747 version downloaded from Emulex? I > think the latter is giving slightly better performance but, anyway, I > disable LRO and TSO because I see a horrible impact on NFS performance. > > I made a diff between the two versions (CURRENT and 747) and I saw that the main difference is in the management of buf_ring through drbr API. In the CURRENT driver they use a new function drbr_peek() instead of drbr_dequeue() and I think this is better. However, even in the 747 version seems to have the problem of the lack of locking. Cheers, Stefano Cheers, > > > > > > Borja. > > -- Stefano Garzarella From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 11:39:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD5FDC47; Tue, 15 Jul 2014 11:39:31 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8BBA62C61; Tue, 15 Jul 2014 11:39:31 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 3DAA59DCA22; Tue, 15 Jul 2014 13:39:21 +0200 (CEST) Subject: Re: Fix Emulex "oce" driver in CURRENT Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Tue, 15 Jul 2014 13:39:19 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3B641ED6-58E7-4C7F-A98F-A56FEA59F79D@sarenet.es> References: <453BA9EC-BB63-4258-8141-847F41315E1E@sarenet.es> <6C8CF68D-68E2-4168-AA0A-6A629D363371@sarenet.es> To: Stefano Garzarella X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , freebsd-current , Luigi Rizzo , Xin LI X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 11:39:31 -0000 On Jul 15, 2014, at 1:36 PM, Stefano Garzarella wrote: > So, asking for spiritual counsel now. Would you use this driver in a = production environment instead of the 747 version downloaded from = Emulex? I think the latter is giving slightly better performance but, = anyway, I disable LRO and TSO because I see a horrible impact on NFS = performance. >=20 >=20 > I made a diff between the two versions (CURRENT and 747) and I saw = that the main difference is in the management of buf_ring through drbr = API. > In the CURRENT driver they use a new function drbr_peek() instead of = drbr_dequeue() and I think this is better. > However, even in the 747 version seems to have the problem of the lack = of locking. Well, definitely you saved my cake! So it was still a tickling time = bomb. Thank you very much! Borja. From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 12:42:56 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08D3AF91 for ; Tue, 15 Jul 2014 12:42:56 +0000 (UTC) Received: from forward-corp1e.mail.yandex.net (forward-corp1e.mail.yandex.net [IPv6:2a02:6b8:0:202::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A726E225F for ; Tue, 15 Jul 2014 12:42:55 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1e.mail.yandex.net (Yandex) with ESMTP id 0FC9A64051E; Tue, 15 Jul 2014 16:42:34 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id C16022C0A15; Tue, 15 Jul 2014 16:42:34 +0400 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id uDSVOWecV1-gY2WJTnB; Tue, 15 Jul 2014 16:42:34 +0400 (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) X-Yandex-Uniq: acb248e4-8119-45fd-bf68-6811aa8345f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1405428154; bh=WW8hQ23xblOrOA9C1Pbplm1uXHeI2VVmxaTsfmqxjJ4=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=X5LRvJ4vh8uf7PLQSQsEIKFgBTcJMucXL4hnqRvp9408H4etyfG+RaubpOiwwmUv6 SFMKJK6UOrs2LlutfhE1SNVsImIEb+hiwm76J4FlLwGLEygj/3J2Ci6LB9Kb+QcGD6 Ep4N4+ZEnG/G63bIOzqp4lWirQeKb4hXyOEm4DLg= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <53C521B1.5050605@yandex-team.ru> Date: Tue, 15 Jul 2014 16:42:25 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: chk , freebsd-net Subject: Re: tincd and mpd5 make kernel panic References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 12:42:56 -0000 On 15.07.2014 14:36, chk wrote: > Hi, everyone, > Help.... > I have a tincd vpn running in freebsd box FreeBSD 10.0-RELEASE-p2 #0 r265318M. > below is ifconfig outut: > [chk@NUC ~]$ ifconfig > em0: flags=8843 metric 0 mtu 1500 > options=4219b > ether ec:a8:6b:f3:76:6a > inet 192.168.2.202 netmask 0xffffff00 broadcast 255.255.255.255 > nd6 options=29 > media: Ethernet autoselect (100baseTX ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > run0: flags=8843 metric 0 mtu 2290 > ether c8:3a:35:c0:b8:2f > nd6 options=29 > media: IEEE 802.11 Wireless Ethernet autoselect mode 11g > status: associated > em0.3: flags=8843 metric 0 mtu 1500 > options=103 > ether ec:a8:6b:f3:76:6a > inet 192.168.3.1 netmask 0xffffff00 broadcast 255.255.255.0 > inet6 fe80::eea8:6bff:fef3:766a%em0.3 prefixlen 64 scopeid 0x4 > nd6 options=29 > media: Ethernet autoselect (100baseTX ) > status: active > vlan: 3 parent interface: em0 > wlan0: flags=8843 metric 0 mtu 1500 > ether c8:3a:35:c0:b8:2f > inet 192.168.30.222 netmask 0xffffff00 broadcast 255.255.255.0 > inet6 fe80::ca3a:35ff:fec0:b82f%wlan0 prefixlen 64 scopeid 0x5 > nd6 options=29 > media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) > status: no carrier > ssid "" channel 10 (2457 MHz 11g) > country US authmode WPA1+WPA2/802.11i privacy MIXED deftxkey UNDEF > txpower 0 bmiss 7 scanvalid 60 protmode CTS wme roaming MANUAL > tun0: flags=8043 metric 0 mtu 1500 > options=80000 > inet 192.168.30.254 netmask 0xffffff00 broadcast 192.168.30.255 > inet6 fe80::eea8:6bff:fef3:766a%tun0 prefixlen 64 scopeid 0x6 > nd6 options=21 > Opened by PID 1015 > ========================= > Convenient for connect to vpn, I add a pppoe_server to mpd5, but when client dialing up, kernel panic. Is it reproducible? Can you issue "route -n monitor" and share its output before the panic? > > here is information of core dump: > [root@NUC /var/log]# kgdb -c ../crash/vmcore.1 /boot/kernel/kernel > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x0 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8098ae09 > stack pointer = 0x28:0xfffffe0234584660 > frame pointer = 0x28:0xfffffe02345846f0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 780 (wpa_supplicant) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xffffffff808f5910 at kdb_backtrace+0x60 > #1 0xffffffff808bd3f5 at panic+0x155 > #2 0xffffffff80c9c1e2 at trap_fatal+0x3a2 > #3 0xffffffff80c9c4b9 at trap_pfault+0x2c9 > #4 0xffffffff80c9bc46 at trap+0x5e6 > #5 0xffffffff80c82ee2 at calltrap+0x8 > #6 0xffffffff809852f0 at rn_walktree+0x70 > #7 0xffffffff8098a470 at sysctl_rtsock+0x1a0 > #8 0xffffffff808c894f at sysctl_root+0x24f > #9 0xffffffff808c8f08 at userland_sysctl+0x1d8 > #10 0xffffffff808c8cf4 at sys___sysctl+0x74 > #11 0xffffffff80c9cad7 at amd64_syscall+0x357 > #12 0xffffffff80c831cb at Xfast_syscall+0xfb > Uptime: 10m24s > (ada0:ahcich0:0:0:0): STANDBY_IMMEDIATE. ACB: e0 00 00 00 00 40 00 00 00 00 00 00 > (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress > (ada0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): Spin-down disk failed > Dumping 439 out of 8067 MB:..4%..11%..22%..33%..41%..51%..62%..73%..81%..92% > (kgdb) f 7 > #7 0xffffffff8098ae09 in sysctl_dumpentry (rn=0xfffff800110eae10, vw=0xfffffe0234584748) > at /usr/src/sys/net/rtsock.c:1592 > 1592 info.rti_info[RTAX_IFP] = rt->rt_ifp->if_addr->ifa_addr; > Current language: auto; currently minimal > (kgdb) l > 1587 info.rti_info[RTAX_DST] = rt_key(rt); > 1588 info.rti_info[RTAX_GATEWAY] = rt->rt_gateway; > 1589 info.rti_info[RTAX_NETMASK] = rt_mask(rt); > 1590 info.rti_info[RTAX_GENMASK] = 0; > 1591 if (rt->rt_ifp) { > 1592 info.rti_info[RTAX_IFP] = rt->rt_ifp->if_addr->ifa_addr; This one looks strange. There is a check on added routes that rt_ifp is not NULL. > 1593 info.rti_info[RTAX_IFA] = rt->rt_ifa->ifa_addr; > 1594 if (rt->rt_ifp->if_flags & IFF_POINTOPOINT) > 1595 info.rti_info[RTAX_BRD] = rt->rt_ifa->ifa_dstaddr; > 1596 } > (kgdb) i loc > info = {rti_addrs = 0, rti_info = {0xfffff80005dace00, 0xfffff80005dace10, 0x0, 0x0, 0x0, 0x0, > 0x0, 0x0}, rti_flags = 0, rti_ifa = 0x0, rti_ifp = 0x0} > error = Cannot access memory at address 0x0 > (kgdb) p *rt > No symbol "rt" in current context. > (kgdb) p rt > No symbol "rt" in current context. > (kgdb) p info > $1 = {rti_addrs = 0, rti_info = {0xfffff80005dace00, 0xfffff80005dace10, 0x0, 0x0, 0x0, 0x0, 0x0, > 0x0}, rti_flags = 0, rti_ifa = 0x0, rti_ifp = 0x0} Can you decode which prefix it is? e.g. p (struct sockaddr_in *)info.rti_info[RTAX_DST] p (struct sockaddr_in *)info.rti_info[RTAX_NETMASK] and what is ifp (and others): p (struct rtentry *)0xfffff800110eae10 p *$1 p $1->rt_ip->if_addrs > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 15:38:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D5AAD6C for ; Tue, 15 Jul 2014 15:38:12 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 22B9222E7 for ; Tue, 15 Jul 2014 15:38:10 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CFE7AB97E; Tue, 15 Jul 2014 11:38:09 -0400 (EDT) From: John Baldwin To: Rick Macklem Subject: Re: NFS client READ performance on -current Date: Tue, 15 Jul 2014 10:34:54 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> In-Reply-To: <2136988575.13956627.1405199640153.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201407151034.54681.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 15 Jul 2014 11:38:09 -0400 (EDT) Cc: pyunyh@gmail.com, "Russell L. Carter" , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 15:38:12 -0000 On Saturday, July 12, 2014 5:14:00 pm Rick Macklem wrote: > Yonghyeon Pyun wrote: > > On Fri, Jul 11, 2014 at 09:54:23AM -0400, John Baldwin wrote: > > > On Thursday, July 10, 2014 6:31:43 pm Rick Macklem wrote: > > > > John Baldwin wrote: > > > > > On Thursday, July 03, 2014 8:51:01 pm Rick Macklem wrote: > > > > > > Russell L. Carter wrote: > > > > > > > > > > > > > > > > > > > > > On 07/02/14 19:09, Rick Macklem wrote: > > > > > > > > > > > > > > > Could you please post the dmesg stuff for the network > > > > > > > > interface, > > > > > > > > so I can tell what driver is being used? I'll take a look > > > > > > > > at > > > > > > > > it, > > > > > > > > in case it needs to be changed to use m_defrag(). > > > > > > > > > > > > > > em0: port > > > > > > > 0xd020-0xd03f > > > > > > > mem > > > > > > > 0xfe4a0000-0xfe4bffff,0xfe480000-0xfe49ffff irq 44 at > > > > > > > device 0.0 > > > > > > > on > > > > > > > pci2 > > > > > > > em0: Using an MSI interrupt > > > > > > > em0: Ethernet address: 00:15:17:bc:29:ba > > > > > > > 001.000007 [2323] netmap_attach success for em0 > > > > > > > tx > > > > > > > 1/1024 > > > > > > > rx > > > > > > > 1/1024 queues/slots > > > > > > > > > > > > > > This is one of those dual nic cards, so there is em1 as > > > > > > > well... > > > > > > > > > > > > > Well, I took a quick look at the driver and it does use > > > > > > m_defrag(), > > > > > > but > > > > > > I think that the "retry:" label it does a goto after doing so > > > > > > might > > > > > > be in > > > > > > the wrong place. > > > > > > > > > > > > The attached untested patch might fix this. > > > > > > > > > > > > Is it convenient to build a kernel with this patch applied > > > > > > and then > > > > > > try > > > > > > it with TSO enabled? > > > > > > > > > > > > rick > > > > > > ps: It does have the transmit segment limit set to 32. I have > > > > > > no > > > > > > idea if > > > > > > this is a hardware limitation. > > > > > > > > > > I think the retry is not in the wrong place, but the overhead > > > > > of all > > > > > those > > > > > pullups is apparently quite severe. > > > > The m_defrag() call after the first failure will just barely > > > > squeeze > > > > the just under 64K TSO segment into 32 mbuf clusters. Then I > > > > think any > > > > m_pullup() done during the retry will allocate an mbuf > > > > (at a glance it seems to always do this when the old mbuf is a > > > > cluster) > > > > and prepend that to the list. > > > > --> Now the list is > 32 mbufs again and the > > > > bus_dmammap_load_mbuf_sg() > > > > will fail again on the retry, this time fatally, I think? > > > > > > > > I can't see any reason to re-do all the stuff using m_pullup() > > > > and Russell > > > > reported that moving the "retry:" fixed his problem, from what I > > > > understood. > > > > > > Ah, I had assumed (incorrectly) that the m_pullup()s would all be > > > nops in this > > > case. It seems the NIC would really like to have all those things > > > in a single > > > segment, but it is not required, so I agree that your patch is > > > fine. > > > > > > > I recall em(4) controllers have various limitation in TSO. Driver > > has to update IP header to make TSO work so driver has to get a > > writable mbufs. bpf(4) consumers will see IP packet length is 0 > > after this change. I think tcpdump has a compile time option to > > guess correct IP packet length. The firmware of controller also > > should be able to access complete IP/TCP header in a single buffer. > > I don't remember more details in TSO limitation but I guess you may > > be able to get more details TSO limitation from publicly available > > Intel data sheet. > I think that the patch should handle this ok. All of the m_pullup() > stuff gets done the first time. Then, if the result is more than 32 > mbufs in the list, m_defrag() is called to copy the chain. This should > result in all the header stuff in the first mbuf cluster and the map > call is done again with this list of clusters. (Without the patch, > m_pullup() would allocate another prepended mbuf and make the chain > more than 32mbufs again.) Hmm, I am surprised by the m_pullup() behavior that it doesn't just notice that the first mbuf with a cluster has the desired data already and returns without doing anything. That is, I'm surprised the first statement in m_pullup() isn't just: if (n->m_len >= len) return (n); -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 17:03:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E292B887; Tue, 15 Jul 2014 17:03:33 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94A162C38; Tue, 15 Jul 2014 17:03:33 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id i17so2686800qcy.35 for ; Tue, 15 Jul 2014 10:03:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iMBVrtjCfSKPfG17B8jSMStCpaZPSlHA4ZjhYx9eCZE=; b=I2KfISOM1kDot+iS/KZRfqrJRL5fe8lm3lzJGpk6Gmom9B2HS9TMPICuTkhGiq57yv Ws2VuF9Rw9TO5gGrVlUhZ1ykjA+5KxbERhNAJWxj1YzZ9fG6azhqIrutNGf89GCIq4t2 nBHer2Cg5niSsDfeCmUSMsyb92YIvkUx/D0+/W1ov2YslcchfwfGxfHoAplRdcsbJc7f u6Di7MhVUx26AgYaMWiF4PcI0VI+YLJs3FugXVfxPwt++W64Yl7LeXbrMVh3kcQR2le3 QOU63VEsGj3lu9Tp91hNLVN1LJGx7j543NFyEd0BxRsXsHNup+6gt5w0exhASOrdMDOc ohMw== MIME-Version: 1.0 X-Received: by 10.224.80.67 with SMTP id s3mr34522754qak.92.1405443812708; Tue, 15 Jul 2014 10:03:32 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Tue, 15 Jul 2014 10:03:32 -0700 (PDT) In-Reply-To: <201407151132.53587.vegeta@tuxpowered.net> References: <201407151132.53587.vegeta@tuxpowered.net> Date: Tue, 15 Jul 2014 10:03:32 -0700 Message-ID: Subject: Re: Why is r250764 not in 9.3? From: hiren panchasara To: Kajetan Staszkiewicz , melifaro@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 17:03:34 -0000 + Alexander On Tue, Jul 15, 2014 at 2:32 AM, Kajetan Staszkiewicz wrote: > The time has come to upgrade my routers to FreeBSD 9.3. > > While going through list of patches I had on 9.1, I've noticed that r248070 got > into 9.3 but r250764 did not. Why is that? Probably just missed it. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 17:40:46 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 570AF222 for ; Tue, 15 Jul 2014 17:40:46 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 186252061 for ; Tue, 15 Jul 2014 17:40:45 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id l6so5102779qcy.11 for ; Tue, 15 Jul 2014 10:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Z+H4UJaLBOZ15gIFqAnk/4pwbS3HFcyODMG/xMxA9eE=; b=NY8A/QcpUC7I5OI0oTP5FNawTzkE3b5LBWEJy7jtCcLKqc9kp4iQkBEfSOZzT+4UmF iPAMjttMlvRkpCTV/1kDhWdxkxTfAuDE64q2w0DK4PcfX+floV8FZMYYE88L4cG8bpvn 43jVwWs5WrdQJOG+nKPYf0kcUGDnw2SdaaUpZDuwbs9Yp2ODuhDgkmqkjCNT+N4atpaB ldk9cXrHC2fGG//qpNI3JRAqsbnHtaujq7v+Dkrp3rJqDcj1s9o1Xr519gITXDkJQ/B2 E+IObJgyFUOo3N0ccohrCJUl2mikbUCZ98jl8GfG0yL8mcakRdfIwveXS3Q5xPJ/IvcK IAag== MIME-Version: 1.0 X-Received: by 10.224.57.148 with SMTP id c20mr1390846qah.31.1405446045185; Tue, 15 Jul 2014 10:40:45 -0700 (PDT) Received: by 10.140.105.66 with HTTP; Tue, 15 Jul 2014 10:40:45 -0700 (PDT) In-Reply-To: <53C521B1.5050605@yandex-team.ru> References: <53C521B1.5050605@yandex-team.ru> Date: Wed, 16 Jul 2014 01:40:45 +0800 Message-ID: Subject: Re: tincd and mpd5 make kernel panic From: chenkun To: "Alexander V. Chernikov" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 17:40:46 -0000 On Tue, Jul 15, 2014 at 8:42 PM, Alexander V. Chernikov wrote: > > On 15.07.2014 14:36, chk wrote: >> >> Hi, everyone, >> Help.... >> I have a tincd vpn running in freebsd box FreeBSD 10.0-RELEASE-p2 #0 r2= 65318M. >> below is ifconfig outut: >> [chk@NUC ~]$ ifconfig >> em0: flags=3D8843 metric 0 mtu 1= 500 >> options=3D4219b >> ether ec:a8:6b:f3:76:6a >> inet 192.168.2.202 netmask 0xffffff00 broadcast 255.255.255.255 >> nd6 options=3D29 >> media: Ethernet autoselect (100baseTX ) >> status: active >> lo0: flags=3D8049 metric 0 mtu 16384 >> options=3D600003 >> inet6 ::1 prefixlen 128 >> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >> inet 127.0.0.1 netmask 0xff000000 >> nd6 options=3D21 >> run0: flags=3D8843 metric 0 mtu = 2290 >> ether c8:3a:35:c0:b8:2f >> nd6 options=3D29 >> media: IEEE 802.11 Wireless Ethernet autoselect mode 11g >> status: associated >> em0.3: flags=3D8843 metric 0 mtu= 1500 >> options=3D103 >> ether ec:a8:6b:f3:76:6a >> inet 192.168.3.1 netmask 0xffffff00 broadcast 255.255.255.0 >> inet6 fe80::eea8:6bff:fef3:766a%em0.3 prefixlen 64 scopeid 0x4 >> nd6 options=3D29 >> media: Ethernet autoselect (100baseTX ) >> status: active >> vlan: 3 parent interface: em0 >> wlan0: flags=3D8843 metric 0 mtu= 1500 >> ether c8:3a:35:c0:b8:2f >> inet 192.168.30.222 netmask 0xffffff00 broadcast 255.255.255.0 >> inet6 fe80::ca3a:35ff:fec0:b82f%wlan0 prefixlen 64 scopeid 0x5 >> nd6 options=3D29 >> media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) >> status: no carrier >> ssid "" channel 10 (2457 MHz 11g) >> country US authmode WPA1+WPA2/802.11i privacy MIXED deftxkey UN= DEF >> txpower 0 bmiss 7 scanvalid 60 protmode CTS wme roaming MANUAL >> tun0: flags=3D8043 metric 0 mtu 1500 >> options=3D80000 >> inet 192.168.30.254 netmask 0xffffff00 broadcast 192.168.30.255 >> inet6 fe80::eea8:6bff:fef3:766a%tun0 prefixlen 64 scopeid 0x6 >> nd6 options=3D21 >> Opened by PID 1015 >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D >> Convenient for connect to vpn, I add a pppoe_server to mpd5, but when cl= ient dialing up, kernel panic. > > Is it reproducible? > Can you issue "route -n monitor" and share its output before the panic? Yes, after several times of re dail up, kernel panic route -n monitor print as below: [root@NUC /usr/home/chk]# route -n monitor got message of size 24 on Wed Jul 16 01:15:28 2014 RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq 6881280, errno 0, flags: locks: inits: got message of size 24 on Wed Jul 16 01:15:34 2014 RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq 6881280, errno 0, flags: locks: inits: got message of size 24 on Wed Jul 16 01:15:41 2014 RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq 6881280, errno 0, flags: locks: inits: got message of size 24 on Wed Jul 16 01:15:46 2014 RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival got message of size 168 on Wed Jul 16 01:15:46 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 184 on Wed Jul 16 01:15:46 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 100 on Wed Jul 16 01:15:46 2014 RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: sockaddrs: 0.0.0.0 ng0 (0) (0) got message of size 184 on Wed Jul 16 01:15:46 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 108 on Wed Jul 16 01:15:46 2014 RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: sockaddrs: 255.255.255.255 ng0 (0) (0) got message of size 116 on Wed Jul 16 01:15:46 2014 RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: sockaddrs: 255.255.255.255 ng0 192.168.40.1 192.168.41.50 got message of size 224 on Wed Jul 16 01:15:46 2014 RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 192.168.41.50 link#7 got message of size 148 on Wed Jul 16 01:15:46 2014 RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: sockaddrs: ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) got message of size 272 on Wed Jul 16 01:15:46 2014 RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: got message of size 104 on Wed Jul 16 01:15:46 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1:fff3:766a%ng0 got message of size 104 on Wed Jul 16 01:15:46 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1%ng0 got message of size 104 on Wed Jul 16 01:15:46 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:ffd7:6760%ng0 got message of size 104 on Wed Jul 16 01:15:46 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:d767:6055%ng0 got message of size 104 on Wed Jul 16 01:15:46 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff01::1%ng0 got message of size 344 on Wed Jul 16 01:15:46 2014 RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 fe80::eea8:6bff:fef3:766a%ng0 got message of size 168 on Wed Jul 16 01:15:46 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 168 on Wed Jul 16 01:15:46 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 272 on Wed Jul 16 01:15:46 2014 RTM_DELETE: Delete Route: len 272, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::eea8:6bff:fef3:766a%7 link#0 ffff:ffff:ffff:ffff:: got message of size 148 on Wed Jul 16 01:15:46 2014 RTM_DELADDR: address being removed from iface: len 148, metric 0, flags: sockaddrs: ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%7 (0) got message of size 344 on Wed Jul 16 01:15:46 2014 RTM_DELETE: Delete Route: len 344, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::%7 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 fe80::eea8:6bff:fef3:766a%7 got message of size 24 on Wed Jul 16 01:15:46 2014 RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: departure got message of size 24 on Wed Jul 16 01:15:48 2014 RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq 6881280, errno 0, flags: locks: inits: got message of size 24 on Wed Jul 16 01:15:54 2014 RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq 6881280, errno 0, flags: locks: inits: got message of size 24 on Wed Jul 16 01:15:55 2014 RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival got message of size 168 on Wed Jul 16 01:15:55 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 184 on Wed Jul 16 01:15:55 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 100 on Wed Jul 16 01:15:55 2014 RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: sockaddrs: 0.0.0.0 ng0 (0) (0) got message of size 184 on Wed Jul 16 01:15:55 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 108 on Wed Jul 16 01:15:55 2014 RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: sockaddrs: 255.255.255.255 ng0 (0) (0) got message of size 116 on Wed Jul 16 01:15:55 2014 RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: sockaddrs: 255.255.255.255 ng0 192.168.40.1 192.168.41.50 got message of size 224 on Wed Jul 16 01:15:55 2014 RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 192.168.41.50 link#7 got message of size 148 on Wed Jul 16 01:15:55 2014 RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: sockaddrs: ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) got message of size 272 on Wed Jul 16 01:15:55 2014 RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: got message of size 104 on Wed Jul 16 01:15:55 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1:fff3:766a%ng0 got message of size 104 on Wed Jul 16 01:15:55 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1%ng0 got message of size 104 on Wed Jul 16 01:15:55 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:ffd7:6760%ng0 got message of size 104 on Wed Jul 16 01:15:55 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:d767:6055%ng0 got message of size 104 on Wed Jul 16 01:15:55 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff01::1%ng0 got message of size 344 on Wed Jul 16 01:15:55 2014 RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 fe80::eea8:6bff:fef3:766a%ng0 got message of size 168 on Wed Jul 16 01:15:55 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 168 on Wed Jul 16 01:15:55 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 272 on Wed Jul 16 01:15:55 2014 RTM_DELETE: Delete Route: len 272, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::eea8:6bff:fef3:766a%7 link#0 ffff:ffff:ffff:ffff:: got message of size 148 on Wed Jul 16 01:15:55 2014 RTM_DELADDR: address being removed from iface: len 148, metric 0, flags: sockaddrs: ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%7 (0) got message of size 344 on Wed Jul 16 01:15:55 2014 RTM_DELETE: Delete Route: len 344, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::%7 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 fe80::eea8:6bff:fef3:766a%7 got message of size 24 on Wed Jul 16 01:15:55 2014 RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: departure got message of size 24 on Wed Jul 16 01:15:59 2014 RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival got message of size 168 on Wed Jul 16 01:15:59 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: got message of size 184 on Wed Jul 16 01:15:59 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 100 on Wed Jul 16 01:15:59 2014 RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: sockaddrs: 0.0.0.0 ng0 (0) (0) got message of size 184 on Wed Jul 16 01:15:59 2014 RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 0.0.0.0 (0) (0) got message of size 108 on Wed Jul 16 01:15:59 2014 RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: sockaddrs: 255.255.255.255 ng0 (0) (0) got message of size 116 on Wed Jul 16 01:15:59 2014 RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: sockaddrs: 255.255.255.255 ng0 192.168.40.1 192.168.41.50 got message of size 224 on Wed Jul 16 01:15:59 2014 RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: 192.168.41.50 link#7 got message of size 148 on Wed Jul 16 01:15:59 2014 RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: sockaddrs: ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) got message of size 272 on Wed Jul 16 01:15:59 2014 RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: got message of size 104 on Wed Jul 16 01:15:59 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1:fff3:766a%ng0 got message of size 104 on Wed Jul 16 01:15:59 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::1%ng0 got message of size 104 on Wed Jul 16 01:15:59 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:ffd7:6760%ng0 got message of size 104 on Wed Jul 16 01:15:59 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff02::2:d767:6055%ng0 got message of size 104 on Wed Jul 16 01:15:59 2014 RTM_NEWMADDR: new multicast group membership on iface: len 104, sockaddrs: ng0 ff01::1%ng0 got message of size 344 on Wed Jul 16 01:15:59 2014 RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 fe80::eea8:6bff:fef3:766a%ng0 got message of size 168 on Wed Jul 16 01:15:59 2014 RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, flags: > > > >> >> here is information of core dump: >> [root@NUC /var/log]# kgdb -c ../crash/vmcore.1 /boot/kernel/kernel >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you= are >> welcome to change it and/or distribute copies of it under certain condit= ions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for detai= ls. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid =3D 0; apic id =3D 00 >> fault virtual address =3D 0x0 >> fault code =3D supervisor read data, page not present >> instruction pointer =3D 0x20:0xffffffff8098ae09 >> stack pointer =3D 0x28:0xfffffe0234584660 >> frame pointer =3D 0x28:0xfffffe02345846f0 >> code segment =3D base 0x0, limit 0xfffff, type 0x1b >> =3D DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags =3D interrupt enabled, resume, IOPL =3D 0 >> current process =3D 780 (wpa_supplicant) >> trap number =3D 12 >> panic: page fault >> cpuid =3D 0 >> KDB: stack backtrace: >> #0 0xffffffff808f5910 at kdb_backtrace+0x60 >> #1 0xffffffff808bd3f5 at panic+0x155 >> #2 0xffffffff80c9c1e2 at trap_fatal+0x3a2 >> #3 0xffffffff80c9c4b9 at trap_pfault+0x2c9 >> #4 0xffffffff80c9bc46 at trap+0x5e6 >> #5 0xffffffff80c82ee2 at calltrap+0x8 >> #6 0xffffffff809852f0 at rn_walktree+0x70 >> #7 0xffffffff8098a470 at sysctl_rtsock+0x1a0 >> #8 0xffffffff808c894f at sysctl_root+0x24f >> #9 0xffffffff808c8f08 at userland_sysctl+0x1d8 >> #10 0xffffffff808c8cf4 at sys___sysctl+0x74 >> #11 0xffffffff80c9cad7 at amd64_syscall+0x357 >> #12 0xffffffff80c831cb at Xfast_syscall+0xfb >> Uptime: 10m24s >> (ada0:ahcich0:0:0:0): STANDBY_IMMEDIATE. ACB: e0 00 00 00 00 40 00 00 00= 00 00 00 >> (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress >> (ada0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): Spin-down disk failed >> Dumping 439 out of 8067 MB:..4%..11%..22%..33%..41%..51%..62%..73%..81%.= .92% >> (kgdb) f 7 >> #7 0xffffffff8098ae09 in sysctl_dumpentry (rn=3D0xfffff800110eae10, vw= =3D0xfffffe0234584748) >> at /usr/src/sys/net/rtsock.c:1592 >> 1592 info.rti_info[RTAX_IFP] =3D rt->rt_ifp->if_addr-= >ifa_addr; >> Current language: auto; currently minimal >> (kgdb) l >> 1587 info.rti_info[RTAX_DST] =3D rt_key(rt); >> 1588 info.rti_info[RTAX_GATEWAY] =3D rt->rt_gateway; >> 1589 info.rti_info[RTAX_NETMASK] =3D rt_mask(rt); >> 1590 info.rti_info[RTAX_GENMASK] =3D 0; >> 1591 if (rt->rt_ifp) { >> 1592 info.rti_info[RTAX_IFP] =3D rt->rt_ifp->if_addr-= >ifa_addr; > > This one looks strange. There is a check on added routes that rt_ifp is n= ot NULL. > >> 1593 info.rti_info[RTAX_IFA] =3D rt->rt_ifa->ifa_addr= ; >> 1594 if (rt->rt_ifp->if_flags & IFF_POINTOPOINT) >> 1595 info.rti_info[RTAX_BRD] =3D rt->rt_ifa->= ifa_dstaddr; >> 1596 } >> (kgdb) i loc >> info =3D {rti_addrs =3D 0, rti_info =3D {0xfffff80005dace00, 0xfffff8000= 5dace10, 0x0, 0x0, 0x0, 0x0, >> 0x0, 0x0}, rti_flags =3D 0, rti_ifa =3D 0x0, rti_ifp =3D 0x0} >> error =3D Cannot access memory at address 0x0 >> (kgdb) p *rt >> No symbol "rt" in current context. >> (kgdb) p rt >> No symbol "rt" in current context. >> (kgdb) p info >> $1 =3D {rti_addrs =3D 0, rti_info =3D {0xfffff80005dace00, 0xfffff80005d= ace10, 0x0, 0x0, 0x0, 0x0, 0x0, >> 0x0}, rti_flags =3D 0, rti_ifa =3D 0x0, rti_ifp =3D 0x0} > > Can you decode which prefix it is? > e.g. > p (struct sockaddr_in *)info.rti_info[RTAX_DST] (kgdb) p *(struct sockaddr_in *)info.rti_info[0] $2 =3D {sin_len =3D 16 '\020', sin_family =3D 2 '\002', sin_port =3D 0, sin_addr =3D {s_addr =3D 841590976}, sin_zero =3D "\000\000\000\000\000\000\000"} > p (struct sockaddr_in *)info.rti_info[RTAX_NETMASK] info.rti_info[RTAX_NETMASK] is NULL. > > and what is ifp (and others): > > p (struct rtentry *)0xfffff800110eae10 > p *$1 > p $1->rt_ip->if_addrs (kgdb) p (struct rtentry *)0xfffff800110eae10 $3 =3D (struct rtentry *) 0xfffff800110eae10 (kgdb) p *$3 Cannot access memory at address 0xfffff800110eae10 ####0xfffff800110eae10 is address of rn? It's different in the last core du= mp. (kgdb) p (struct rtentry *)0xfffff800122f0258 $4 =3D (struct rtentry *) 0xfffff800122f0258 (kgdb) p *$4 $5 =3D {rt_nodes =3D {{rn_mklist =3D 0x0, rn_parent =3D 0xfffff800122f0288, rn_bit =3D -1, rn_bmask =3D 0 '\0', rn_flags =3D 4 '\004', rn_u =3D {rn_leaf =3D {rn_Key =3D 0xfffff80005851c00 "\020\002", rn_Mask =3D 0x0, rn_Dupedkey =3D 0x0}, rn_node =3D {rn_Off =3D 92609536, rn_L =3D 0x0, rn_R =3D 0x0}}}, {rn_mklist =3D 0x0, rn_parent =3D 0xfffff800122f08c8, rn_bit =3D 55, rn_bmask =3D 1 '\001', rn_flags =3D 4 '\004', rn_u =3D { rn_leaf =3D {rn_Key =3D 0x6
, rn_Mask = =3D 0xfffff8001208abe8 "=C3=80\234\003\022", rn_Dupedkey =3D 0xfffff800122f0258}, rn_node =3D {rn_Off =3D 6, rn_L =3D 0xfffff8001208abe8, rn_R =3D 0xfffff800122f0258}}}}, rt_gateway =3D 0xfffff80005851c10, rt_flags =3D 1048581, rt_refcnt =3D 0, rt_ifp =3D 0xfffff800131fd000, rt_ifa =3D 0xfffff80005ff7800, rt_rmx =3D {rmx_mtu =3D 1480, rmx_expire =3D 0, rmx_pksent =3D 0, rmx_weight =3D 1}, rt_fibnum =3D 0, rt_mtx =3D {lock_object =3D { lo_name =3D 0xffffffff80eff57f "rtentry", lo_flags =3D 21168128, lo_data =3D 0, lo_witness =3D 0x0}, mtx_lock =3D 4}} (kgdb) p $4->rt_ip->if_addrs There is no member named rt_ip. (kgdb) p $5->rt_ip->if_addrs There is no member named rt_ip. (kgdb) l 1587 info.rti_info[RTAX_DST] =3D rt_key(rt); 1588 info.rti_info[RTAX_GATEWAY] =3D rt->rt_gateway; 1589 info.rti_info[RTAX_NETMASK] =3D rt_mask(rt); 1590 info.rti_info[RTAX_GENMASK] =3D 0; 1591 if (rt->rt_ifp) { 1592 info.rti_info[RTAX_IFP] =3D rt->rt_ifp->if_addr->if= a_addr; 1593 info.rti_info[RTAX_IFA] =3D rt->rt_ifa->ifa_addr; 1594 if (rt->rt_ifp->if_flags & IFF_POINTOPOINT) 1595 info.rti_info[RTAX_BRD] =3D rt->rt_ifa->ifa_dstaddr; 1596 } (kgdb) p $5->rt_ifp->if_addrs There is no member named if_addrs. (kgdb) p $5->rt_ifp->if_addr $6 =3D (struct ifaddr *) 0x0 Is that mean trap occur because of access a NULL point? > >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 17:59:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1020914 for ; Tue, 15 Jul 2014 17:59:07 +0000 (UTC) Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.229]) by mx1.freebsd.org (Postfix) with ESMTP id B2A1A221B for ; Tue, 15 Jul 2014 17:59:06 +0000 (UTC) Received: from [75.80.59.51] ([75.80.59.51:53974] helo=spandex.luckie.org.nz) by cdptpa-oedge01 (envelope-from ) (ecelerity 3.5.0.35861 r(Momo-dev:tip)) with ESMTP id 00/10-32691-9EB65C35; Tue, 15 Jul 2014 17:59:06 +0000 Received: from mjl by spandex.luckie.org.nz with local (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X770L-000PIG-72 for freebsd-net@freebsd.org; Tue, 15 Jul 2014 10:59:05 -0700 Date: Tue, 15 Jul 2014 10:59:05 -0700 From: Matthew Luckie To: freebsd-net@freebsd.org Subject: two freebsd 10 wlan0 oddities Message-ID: <20140715175905.GA97207@spandex.luckie.org.nz> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SLDf9lqlvOQaIe6s" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-RR-Connecting-IP: 107.14.168.118:25 X-Cloudmark-Score: 0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 17:59:08 -0000 --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, Two oddities with wlan0: =46rom dmesg: ath0: mem 0xfeaf0000-0xfeafffff irq 18 at device 0.0 on pci3 [ath] AR9285 Main LNA config: LNA2 [ath] AR9285 Alt LNA config: LNA1 [ath] LNA diversity enabled, Diversity enabled [ath] Enabling diversity for Kite ath0: [HT] enabling HT modes ath0: [HT] 1 stream STBC receive enabled ath0: [HT] 1 RX streams; 1 TX streams ath0: AR9285 mac 192.2 RF5133 phy 14.0 ath0: 2GHz radio: 0x0000; 5GHz radio: 0x00c0 $ ifconfig wlan0 wlan0: flags=3D8943 metric = 0 mtu 1500 ether 00:24:2c:36:90:55 inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255=20 inet6 fe80::224:2cff:fe36:9055%wlan0 prefixlen 64 scopeid 0x7=20 inet6 2001:470:d:4de::1 prefixlen 64=20 nd6 options=3D21 media: IEEE 802.11 Wireless Ethernet autoselect mode 11g status: running ssid mjl channel 11 (2462 MHz 11g) bssid 00:24:2c:36:90:55 regdomain 106 indoor ecm authmode WPA privacy MIXED deftxkey 3 TKIP 2:128-bit TKIP 3:128-bit txpower 20 scanvalid 60 protmode CTS = wme burst dtimperiod 1 -dfs First, since I updated to FreeBSD 10, the system does not seem to be counting ifOutOctets, but does count ifInOctets. Looking at it with netstat: $ netstat -I wlan0 -b Name Mtu Network Address Ipkts Ierrs Idrop Ibytes= Opkts Oerrs Obytes Coll wlan0 1500 00:24:2c:36:90:55 2715328 56 0 930414649= 3549300 2965 0 0 wlan0 1500 192.168.2.0 spandex 266442 - - 183610706= 186618 - 77852137 - wlan0 1500 fe80::224:2cf fe80::224:2cff:fe 1 - - 72= 17 - 1304 - wlan0 1500 2001:470:d:4d 2001:470:d:4de::1 0 - - 0= 1 - 136 - Is this a bug, or am I polling the wrong interface for Obytes? When I look at ath0, it does not seem to be counting bytes out either. Second, for whatever reason rtadvd does not seem to be advertising on wlan0. My /etc/rtadvd.conf: wlan0:\ :addrs#1:addr=3D"2001:470:d:4d3::":prefixlen#64:tc=3Dether: If I /etc/rc.d/rtadvd restart, then route advertisements begin to flow. $ sudo rtadvctl show Password: wlan0: flags=3D status=3D mtu 1500 DefaultLifetime: 30m MinAdvInterval/MaxAdvInterval: 3m20s/10m AdvLinkMTU: , Flags: , Preference: medium ReachableTime: 0s, RetransTimer: 0s, CurHopLimit: 64 AdvIfPrefixes: yes Next RA send: Tue Jul 15 10:52:29 2014 Last RA send: Tue Jul 8 16:00:38 2014 Thoughts? Matthew --SLDf9lqlvOQaIe6s Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlPFa+kACgkQKyuDKSEQAGAgyACgoxfsy/MsutPxbHUj+BmCSvRZ RzwAoIW2FTN2MSisCOJ7D7Gl/aqbzjxl =FAVf -----END PGP SIGNATURE----- --SLDf9lqlvOQaIe6s-- From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 18:21:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B787BB for ; Tue, 15 Jul 2014 18:21:26 +0000 (UTC) Received: from forward-corp1f.mail.yandex.net (forward-corp1f.mail.yandex.net [IPv6:2a02:6b8:0:801::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C387E24AE for ; Tue, 15 Jul 2014 18:21:25 +0000 (UTC) Received: from smtpcorp4.mail.yandex.net (smtpcorp4.mail.yandex.net [95.108.252.2]) by forward-corp1f.mail.yandex.net (Yandex) with ESMTP id 7E6E22420068; Tue, 15 Jul 2014 22:21:13 +0400 (MSK) Received: from smtpcorp4.mail.yandex.net (localhost [127.0.0.1]) by smtpcorp4.mail.yandex.net (Yandex) with ESMTP id 434812C0A54; Tue, 15 Jul 2014 22:21:13 +0400 (MSK) Received: from unknown (unknown [2a02:6b8:0:401:222:4dff:fe50:cd2f]) by smtpcorp4.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id vOFLs0ra0Q-LD28OLLP; Tue, 15 Jul 2014 22:21:13 +0400 (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) X-Yandex-Uniq: f2218227-b348-45c9-b2c3-2c842100aa91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1405448473; bh=dA5LMpjdRmO9frlmmQSbyPon+utQx/groGxTsUewRzQ=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=pBPxV9iQk19a82dG7vYR1BaM82itRu7PF79MTJY73Juo4UV7ThVepxFz9hhQ1IphM X14+V9Xbj7WZtO9hNF2NVzbFJWZxLs4fTn/lu1nFGHkkL5R1D0IMLpxoo4BGE0yd27 kfEpNajIhNDScDBJguVY5bBu/2/N67SzHYnuzLk0= Authentication-Results: smtpcorp4.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Message-ID: <53C5710F.3010701@yandex-team.ru> Date: Tue, 15 Jul 2014 22:21:03 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: chenkun Subject: Re: tincd and mpd5 make kernel panic References: <53C521B1.5050605@yandex-team.ru> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 18:21:26 -0000 On 15.07.2014 21:40, chenkun wrote: > On Tue, Jul 15, 2014 at 8:42 PM, Alexander V. Chernikov > wrote: >> On 15.07.2014 14:36, chk wrote: >>> Hi, everyone, >>> Help.... >>> I have a tincd vpn running in freebsd box FreeBSD 10.0-RELEASE-p2 #0 r265318M. >>> below is ifconfig outut: >>> [chk@NUC ~]$ ifconfig >>> em0: flags=8843 metric 0 mtu 1500 >>> options=4219b >>> ether ec:a8:6b:f3:76:6a >>> inet 192.168.2.202 netmask 0xffffff00 broadcast 255.255.255.255 >>> nd6 options=29 >>> media: Ethernet autoselect (100baseTX ) >>> status: active >>> lo0: flags=8049 metric 0 mtu 16384 >>> options=600003 >>> inet6 ::1 prefixlen 128 >>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >>> inet 127.0.0.1 netmask 0xff000000 >>> nd6 options=21 >>> run0: flags=8843 metric 0 mtu 2290 >>> ether c8:3a:35:c0:b8:2f >>> nd6 options=29 >>> media: IEEE 802.11 Wireless Ethernet autoselect mode 11g >>> status: associated >>> em0.3: flags=8843 metric 0 mtu 1500 >>> options=103 >>> ether ec:a8:6b:f3:76:6a >>> inet 192.168.3.1 netmask 0xffffff00 broadcast 255.255.255.0 >>> inet6 fe80::eea8:6bff:fef3:766a%em0.3 prefixlen 64 scopeid 0x4 >>> nd6 options=29 >>> media: Ethernet autoselect (100baseTX ) >>> status: active >>> vlan: 3 parent interface: em0 >>> wlan0: flags=8843 metric 0 mtu 1500 >>> ether c8:3a:35:c0:b8:2f >>> inet 192.168.30.222 netmask 0xffffff00 broadcast 255.255.255.0 >>> inet6 fe80::ca3a:35ff:fec0:b82f%wlan0 prefixlen 64 scopeid 0x5 >>> nd6 options=29 >>> media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) >>> status: no carrier >>> ssid "" channel 10 (2457 MHz 11g) >>> country US authmode WPA1+WPA2/802.11i privacy MIXED deftxkey UNDEF >>> txpower 0 bmiss 7 scanvalid 60 protmode CTS wme roaming MANUAL >>> tun0: flags=8043 metric 0 mtu 1500 >>> options=80000 >>> inet 192.168.30.254 netmask 0xffffff00 broadcast 192.168.30.255 >>> inet6 fe80::eea8:6bff:fef3:766a%tun0 prefixlen 64 scopeid 0x6 >>> nd6 options=21 >>> Opened by PID 1015 >>> ========================= >>> Convenient for connect to vpn, I add a pppoe_server to mpd5, but when client dialing up, kernel panic. >> Is it reproducible? >> Can you issue "route -n monitor" and share its output before the panic? > Yes, after several times of re dail up, kernel panic > > route -n monitor print as below: > > [root@NUC /usr/home/chk]# route -n monitor > > got message of size 24 on Wed Jul 16 01:15:28 2014 > RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq > 6881280, errno 0, > flags: > locks: inits: > got message of size 24 on Wed Jul 16 01:15:34 2014 > RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq > 6881280, errno 0, > flags: > locks: inits: > got message of size 24 on Wed Jul 16 01:15:41 2014 > RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq > 6881280, errno 0, > flags: > locks: inits: > got message of size 24 on Wed Jul 16 01:15:46 2014 > RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival > > got message of size 168 on Wed Jul 16 01:15:46 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 184 on Wed Jul 16 01:15:46 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 100 on Wed Jul 16 01:15:46 2014 > RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: > sockaddrs: > 0.0.0.0 ng0 (0) (0) > > got message of size 184 on Wed Jul 16 01:15:46 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 108 on Wed Jul 16 01:15:46 2014 > RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 (0) (0) > > got message of size 116 on Wed Jul 16 01:15:46 2014 > RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 192.168.40.1 192.168.41.50 > > got message of size 224 on Wed Jul 16 01:15:46 2014 > RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > 192.168.41.50 link#7 > > got message of size 148 on Wed Jul 16 01:15:46 2014 > RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: > sockaddrs: > ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) > > got message of size 272 on Wed Jul 16 01:15:46 2014 > RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: > > got message of size 104 on Wed Jul 16 01:15:46 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1:fff3:766a%ng0 > > got message of size 104 on Wed Jul 16 01:15:46 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1%ng0 > > got message of size 104 on Wed Jul 16 01:15:46 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:ffd7:6760%ng0 > > got message of size 104 on Wed Jul 16 01:15:46 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:d767:6055%ng0 > > got message of size 104 on Wed Jul 16 01:15:46 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff01::1%ng0 > > got message of size 344 on Wed Jul 16 01:15:46 2014 > RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 > fe80::eea8:6bff:fef3:766a%ng0 > > got message of size 168 on Wed Jul 16 01:15:46 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 168 on Wed Jul 16 01:15:46 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 272 on Wed Jul 16 01:15:46 2014 > RTM_DELETE: Delete Route: len 272, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::eea8:6bff:fef3:766a%7 link#0 ffff:ffff:ffff:ffff:: > > got message of size 148 on Wed Jul 16 01:15:46 2014 > RTM_DELADDR: address being removed from iface: len 148, metric 0, flags: > sockaddrs: > ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%7 (0) > > got message of size 344 on Wed Jul 16 01:15:46 2014 > RTM_DELETE: Delete Route: len 344, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::%7 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 > fe80::eea8:6bff:fef3:766a%7 > > got message of size 24 on Wed Jul 16 01:15:46 2014 > RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: departure > > got message of size 24 on Wed Jul 16 01:15:48 2014 > RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq > 6881280, errno 0, > flags: > locks: inits: > got message of size 24 on Wed Jul 16 01:15:54 2014 > RTM_IEEE80211: IEEE 802.11 wireless event: len 24, pid: 0, seq > 6881280, errno 0, > flags: > locks: inits: > got message of size 24 on Wed Jul 16 01:15:55 2014 > RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival > > got message of size 168 on Wed Jul 16 01:15:55 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 184 on Wed Jul 16 01:15:55 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 100 on Wed Jul 16 01:15:55 2014 > RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: > sockaddrs: > 0.0.0.0 ng0 (0) (0) > > got message of size 184 on Wed Jul 16 01:15:55 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 108 on Wed Jul 16 01:15:55 2014 > RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 (0) (0) > > got message of size 116 on Wed Jul 16 01:15:55 2014 > RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 192.168.40.1 192.168.41.50 > > got message of size 224 on Wed Jul 16 01:15:55 2014 > RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > 192.168.41.50 link#7 > > got message of size 148 on Wed Jul 16 01:15:55 2014 > RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: > sockaddrs: > ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) > > got message of size 272 on Wed Jul 16 01:15:55 2014 > RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: > > got message of size 104 on Wed Jul 16 01:15:55 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1:fff3:766a%ng0 > > got message of size 104 on Wed Jul 16 01:15:55 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1%ng0 > > got message of size 104 on Wed Jul 16 01:15:55 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:ffd7:6760%ng0 > > got message of size 104 on Wed Jul 16 01:15:55 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:d767:6055%ng0 > > got message of size 104 on Wed Jul 16 01:15:55 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff01::1%ng0 > > got message of size 344 on Wed Jul 16 01:15:55 2014 > RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 > fe80::eea8:6bff:fef3:766a%ng0 > > got message of size 168 on Wed Jul 16 01:15:55 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 168 on Wed Jul 16 01:15:55 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 272 on Wed Jul 16 01:15:55 2014 > RTM_DELETE: Delete Route: len 272, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::eea8:6bff:fef3:766a%7 link#0 ffff:ffff:ffff:ffff:: > > got message of size 148 on Wed Jul 16 01:15:55 2014 > RTM_DELADDR: address being removed from iface: len 148, metric 0, flags: > sockaddrs: > ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%7 (0) > > got message of size 344 on Wed Jul 16 01:15:55 2014 > RTM_DELETE: Delete Route: len 344, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::%7 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 > fe80::eea8:6bff:fef3:766a%7 > > got message of size 24 on Wed Jul 16 01:15:55 2014 > RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: departure > > got message of size 24 on Wed Jul 16 01:15:59 2014 > RTM_IFANNOUNCE: interface arrival/departure: len 24, if# 7, what: arrival > > got message of size 168 on Wed Jul 16 01:15:59 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > got message of size 184 on Wed Jul 16 01:15:59 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 100 on Wed Jul 16 01:15:59 2014 > RTM_DELADDR: address being removed from iface: len 100, metric 0, flags: > sockaddrs: > 0.0.0.0 ng0 (0) (0) > > got message of size 184 on Wed Jul 16 01:15:59 2014 > RTM_DELETE: Delete Route: len 184, pid: 0, seq 0, errno 0, > flags: > locks: inits: > sockaddrs: > 0.0.0.0 (0) (0) > > got message of size 108 on Wed Jul 16 01:15:59 2014 > RTM_DELADDR: address being removed from iface: len 108, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 (0) (0) > > got message of size 116 on Wed Jul 16 01:15:59 2014 > RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: > sockaddrs: > 255.255.255.255 ng0 192.168.40.1 192.168.41.50 > > got message of size 224 on Wed Jul 16 01:15:59 2014 > RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > 192.168.41.50 link#7 > > got message of size 148 on Wed Jul 16 01:15:59 2014 > RTM_NEWADDR: address being added to iface: len 148, metric 0, flags: > sockaddrs: > ffff:ffff:ffff:ffff:: ng0 fe80::eea8:6bff:fef3:766a%ng0 (0) > > got message of size 272 on Wed Jul 16 01:15:59 2014 > RTM_ADD: Add Route: len 272, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::eea8:6bff:fef3:766a%ng0 0.0.0.0.0.0 ffff:ffff:ffff:ffff:: > > got message of size 104 on Wed Jul 16 01:15:59 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1:fff3:766a%ng0 > > got message of size 104 on Wed Jul 16 01:15:59 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::1%ng0 > > got message of size 104 on Wed Jul 16 01:15:59 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:ffd7:6760%ng0 > > got message of size 104 on Wed Jul 16 01:15:59 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff02::2:d767:6055%ng0 > > got message of size 104 on Wed Jul 16 01:15:59 2014 > RTM_NEWMADDR: new multicast group membership on iface: len 104, > sockaddrs: > ng0 ff01::1%ng0 > > got message of size 344 on Wed Jul 16 01:15:59 2014 > RTM_ADD: Add Route: len 344, pid: 0, seq 0, errno 0, flags: > locks: inits: > sockaddrs: > fe80::%ng0 link#7 (255) ffff ffff ffff ffff ffff ffff ffff ng0 > fe80::eea8:6bff:fef3:766a%ng0 > > got message of size 168 on Wed Jul 16 01:15:59 2014 > RTM_IFINFO: iface status change: len 168, if# 7, link: unknown, > flags: > > >> >> >>> here is information of core dump: >>> [root@NUC /var/log]# kgdb -c ../crash/vmcore.1 /boot/kernel/kernel >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and you are >>> welcome to change it and/or distribute copies of it under certain conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for details. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> >>> Unread portion of the kernel message buffer: >>> >>> >>> Fatal trap 12: page fault while in kernel mode >>> cpuid = 0; apic id = 00 >>> fault virtual address = 0x0 >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff8098ae09 >>> stack pointer = 0x28:0xfffffe0234584660 >>> frame pointer = 0x28:0xfffffe02345846f0 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = interrupt enabled, resume, IOPL = 0 >>> current process = 780 (wpa_supplicant) >>> trap number = 12 >>> panic: page fault >>> cpuid = 0 >>> KDB: stack backtrace: >>> #0 0xffffffff808f5910 at kdb_backtrace+0x60 >>> #1 0xffffffff808bd3f5 at panic+0x155 >>> #2 0xffffffff80c9c1e2 at trap_fatal+0x3a2 >>> #3 0xffffffff80c9c4b9 at trap_pfault+0x2c9 >>> #4 0xffffffff80c9bc46 at trap+0x5e6 >>> #5 0xffffffff80c82ee2 at calltrap+0x8 >>> #6 0xffffffff809852f0 at rn_walktree+0x70 >>> #7 0xffffffff8098a470 at sysctl_rtsock+0x1a0 >>> #8 0xffffffff808c894f at sysctl_root+0x24f >>> #9 0xffffffff808c8f08 at userland_sysctl+0x1d8 >>> #10 0xffffffff808c8cf4 at sys___sysctl+0x74 >>> #11 0xffffffff80c9cad7 at amd64_syscall+0x357 >>> #12 0xffffffff80c831cb at Xfast_syscall+0xfb >>> Uptime: 10m24s >>> (ada0:ahcich0:0:0:0): STANDBY_IMMEDIATE. ACB: e0 00 00 00 00 40 00 00 00 00 00 00 >>> (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress >>> (ada0:ahcich0:0:0:0): Error 5, Retries exhausted >>> (ada0:ahcich0:0:0:0): Spin-down disk failed >>> Dumping 439 out of 8067 MB:..4%..11%..22%..33%..41%..51%..62%..73%..81%..92% >>> (kgdb) f 7 >>> #7 0xffffffff8098ae09 in sysctl_dumpentry (rn=0xfffff800110eae10, vw=0xfffffe0234584748) >>> at /usr/src/sys/net/rtsock.c:1592 >>> 1592 info.rti_info[RTAX_IFP] = rt->rt_ifp->if_addr->ifa_addr; >>> Current language: auto; currently minimal >>> (kgdb) l >>> 1587 info.rti_info[RTAX_DST] = rt_key(rt); >>> 1588 info.rti_info[RTAX_GATEWAY] = rt->rt_gateway; >>> 1589 info.rti_info[RTAX_NETMASK] = rt_mask(rt); >>> 1590 info.rti_info[RTAX_GENMASK] = 0; >>> 1591 if (rt->rt_ifp) { >>> 1592 info.rti_info[RTAX_IFP] = rt->rt_ifp->if_addr->ifa_addr; >> This one looks strange. There is a check on added routes that rt_ifp is not NULL. >> >>> 1593 info.rti_info[RTAX_IFA] = rt->rt_ifa->ifa_addr; >>> 1594 if (rt->rt_ifp->if_flags & IFF_POINTOPOINT) >>> 1595 info.rti_info[RTAX_BRD] = rt->rt_ifa->ifa_dstaddr; >>> 1596 } >>> (kgdb) i loc >>> info = {rti_addrs = 0, rti_info = {0xfffff80005dace00, 0xfffff80005dace10, 0x0, 0x0, 0x0, 0x0, >>> 0x0, 0x0}, rti_flags = 0, rti_ifa = 0x0, rti_ifp = 0x0} >>> error = Cannot access memory at address 0x0 >>> (kgdb) p *rt >>> No symbol "rt" in current context. >>> (kgdb) p rt >>> No symbol "rt" in current context. >>> (kgdb) p info >>> $1 = {rti_addrs = 0, rti_info = {0xfffff80005dace00, 0xfffff80005dace10, 0x0, 0x0, 0x0, 0x0, 0x0, >>> 0x0}, rti_flags = 0, rti_ifa = 0x0, rti_ifp = 0x0} >> Can you decode which prefix it is? >> e.g. >> p (struct sockaddr_in *)info.rti_info[RTAX_DST] > (kgdb) p *(struct sockaddr_in *)info.rti_info[0] > $2 = {sin_len = 16 '\020', sin_family = 2 '\002', sin_port = 0, > sin_addr = {s_addr = 841590976}, > sin_zero = "\000\000\000\000\000\000\000"} So this looks like route to the other end of ng0 got message of size 116 on Wed Jul 16 01:15:46 2014 RTM_NEWADDR: address being added to iface: len 116, metric 0, flags: sockaddrs: 255.255.255.255 ng0 192.168.40.1 192.168.41.50 got message of size 224 on Wed Jul 16 01:15:46 2014 RTM_ADD: Add Route: len 224, pid: 0, seq 0, errno 0, flags: locks: inits: sockaddrs: ---------- ^^^^^^^^^^ 192.168.41.50 link#7 No ifp passed here. There are, however, some messy logic to determine ifp/ifa based on GW inside rtsock code.. > > >> p (struct sockaddr_in *)info.rti_info[RTAX_NETMASK] > info.rti_info[RTAX_NETMASK] is NULL. > >> and what is ifp (and others): >> >> p (struct rtentry *)0xfffff800110eae10 >> p *$1 >> p $1->rt_ip->if_addrs > (kgdb) p (struct rtentry *)0xfffff800110eae10 > $3 = (struct rtentry *) 0xfffff800110eae10 > (kgdb) p *$3 > Cannot access memory at address 0xfffff800110eae10 > ####0xfffff800110eae10 is address of rn? It's different in the last core dump. Yes. > (kgdb) p (struct rtentry *)0xfffff800122f0258 > $4 = (struct rtentry *) 0xfffff800122f0258 > (kgdb) p *$4 > $5 = {rt_nodes = {{rn_mklist = 0x0, rn_parent = 0xfffff800122f0288, > rn_bit = -1, rn_bmask = 0 '\0', > rn_flags = 4 '\004', rn_u = {rn_leaf = {rn_Key = > 0xfffff80005851c00 "\020\002", rn_Mask = 0x0, > rn_Dupedkey = 0x0}, rn_node = {rn_Off = 92609536, rn_L = > 0x0, rn_R = 0x0}}}, {rn_mklist = 0x0, > rn_parent = 0xfffff800122f08c8, rn_bit = 55, rn_bmask = 1 > '\001', rn_flags = 4 '\004', rn_u = { > rn_leaf = {rn_Key = 0x6
, rn_Mask = > 0xfffff8001208abe8 "À\234\003\022", > rn_Dupedkey = 0xfffff800122f0258}, rn_node = {rn_Off = 6, > rn_L = 0xfffff8001208abe8, > rn_R = 0xfffff800122f0258}}}}, rt_gateway = > 0xfffff80005851c10, rt_flags = 1048581, > rt_refcnt = 0, rt_ifp = 0xfffff800131fd000, rt_ifa = > 0xfffff80005ff7800, rt_rmx = {rmx_mtu = 1480, > rmx_expire = 0, rmx_pksent = 0, rmx_weight = 1}, rt_fibnum = 0, > rt_mtx = {lock_object = { > lo_name = 0xffffffff80eff57f "rtentry", lo_flags = 21168128, > lo_data = 0, lo_witness = 0x0}, > mtx_lock = 4}} > (kgdb) p $4->rt_ip->if_addrs > There is no member named rt_ip. > (kgdb) p $5->rt_ip->if_addrs > There is no member named rt_ip. Hm. ok. p (struct ifnet *)0xfffff800131fd000 p *$1 p *$1->if_addr p (struct ifaddr *)0xfffff80005ff7800 p *$2 > (kgdb) l > 1587 info.rti_info[RTAX_DST] = rt_key(rt); > 1588 info.rti_info[RTAX_GATEWAY] = rt->rt_gateway; > 1589 info.rti_info[RTAX_NETMASK] = rt_mask(rt); > 1590 info.rti_info[RTAX_GENMASK] = 0; > 1591 if (rt->rt_ifp) { > 1592 info.rti_info[RTAX_IFP] = rt->rt_ifp->if_addr->ifa_addr; > 1593 info.rti_info[RTAX_IFA] = rt->rt_ifa->ifa_addr; > 1594 if (rt->rt_ifp->if_flags & IFF_POINTOPOINT) > 1595 info.rti_info[RTAX_BRD] = > rt->rt_ifa->ifa_dstaddr; > 1596 } > (kgdb) p $5->rt_ifp->if_addrs > There is no member named if_addrs. > (kgdb) p $5->rt_ifp->if_addr > $6 = (struct ifaddr *) 0x0 > > Is that mean trap occur because of access a NULL point? Yes. > >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> From owner-freebsd-net@FreeBSD.ORG Tue Jul 15 22:52:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A29FB2E6; Tue, 15 Jul 2014 22:52:43 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5715D2E6C; Tue, 15 Jul 2014 22:52:43 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id z60so67409qgd.19 for ; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=aSOMGrTJZXB82yk4BJ8BorMlH2iLLEVWb0th88MjbZc=; b=uo7fuXEt4CTnvJ2c+4bDMPz/z5DjpiP87AtUE9dveQETy/Vz+zqKNTpx9DA/awuMvE x1iuznOwqB9YEwzkmsx0brYmjAJy+i59OJGVqYtvJkSRt/B6BaZcT8cOmXsGL8Ow6gdn KrMpuKe/rA/eYl4wgK53HOM/gj60TNGvMfK0yRRTdHU6U9pxCuYTKUwNP7Kjk94r+naV tWxlXFx7rvpTK0iDLAmL5Cjics6F0l70Y1HJ3AVY7tqciYJE84YXB4H3VjtVazxxDAXE yn7azr7nWfACj1RJtGvLK819c0XYOfduRloH2lJhGfmtEeMS3EOhfSJKxLSaT5iP8fiq 3SiQ== MIME-Version: 1.0 X-Received: by 10.140.89.18 with SMTP id u18mr38478104qgd.87.1405464762428; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) Date: Tue, 15 Jul 2014 15:52:42 -0700 X-Google-Sender-Auth: SIstEZ94FVCCWHZOkZJ-40V12_A Message-ID: Subject: ixgbe and igb - how many queues? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 22:52:43 -0000 hiya, How many queues does the ixgbe and igb hardware support? The documentation reflects many more transmit and receive queues than the 8 currently auto-configured. For RSS configuration I'd like to expand it out to the 16 that I _think_ the hardware supports, but I'd like some confirmation that the documentation mostly reflects the reality out there and I wouldn't be breaking things on other peoples obscure/onboard/laptop/etc intel platforms. Thanks! -a From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 00:04:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 296FA925; Wed, 16 Jul 2014 00:04:06 +0000 (UTC) Received: from mail-oa0-x22b.google.com (mail-oa0-x22b.google.com [IPv6:2607:f8b0:4003:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D108B24A7; Wed, 16 Jul 2014 00:04:05 +0000 (UTC) Received: by mail-oa0-f43.google.com with SMTP id i7so172844oag.2 for ; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UMZo+wW62WvzdTs7XM8Vrr4J0JMp/xwQ9aCZcj6B86M=; b=CcTY6RM1Xh0HZ7m30B44JlL1lamRGlcfmqnhgec8poHWoRfBiT3cXTV6E7lbFdUQBa L8yi0iFvroPKyGs5CbtFv22rt00lOxjxg4P/xr5Gj/8DnukQLKQ1JjezJ955OeEQpP4b zRlabGDrIQX7oS2H1vkSErWtfAUlF+evbN6/t5QylrVz7zJDzWTyNnasNDXqIqnmoD7P Uy0ht/qP3/jitv3q0w7+x/I65BTgsA01Z5JBcg0bPRFxMQmvgtX7kxaBKsqRpIh8iWgg ocJEW0LfwjK7Yy2VIp27s1hOMdS58lq8JT9dwL7t+fzYT4LTB8K0ctxEaQ2L1ILtnknb kHDw== MIME-Version: 1.0 X-Received: by 10.182.241.130 with SMTP id wi2mr30381342obc.27.1405469045214; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) Received: by 10.76.213.7 with HTTP; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) In-Reply-To: References: Date: Tue, 15 Jul 2014 20:04:05 -0400 Message-ID: Subject: Re: ixgbe and igb - how many queues? From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:04:06 -0000 The oldest hardware supported by the ixgbe driver is the 82598, which supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). I believe that the 82599 and X520 are more capable. I have no idea what the situation is on igb, as there seems to be a lot more hardware supported by that driver and I don't work with the 1GB devices very much. From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 00:49:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7A0559C; Wed, 16 Jul 2014 00:49:00 +0000 (UTC) Received: from mga03.intel.com (mga03.intel.com [143.182.124.21]) by mx1.freebsd.org (Postfix) with ESMTP id 7D3F82818; Wed, 16 Jul 2014 00:49:00 +0000 (UTC) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga101.ch.intel.com with ESMTP; 15 Jul 2014 17:47:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,668,1400050800"; d="scan'208";a="457453656" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by azsmga001.ch.intel.com with ESMTP; 15 Jul 2014 17:47:51 -0700 Received: from orsmsx152.amr.corp.intel.com (10.22.226.39) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 15 Jul 2014 17:47:50 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.5]) by ORSMSX152.amr.corp.intel.com ([169.254.8.128]) with mapi id 14.03.0123.003; Tue, 15 Jul 2014 17:47:50 -0700 From: "Pieper, Jeffrey E" To: Ryan Stone , Adrian Chadd Subject: RE: ixgbe and igb - how many queues? Thread-Topic: ixgbe and igb - how many queues? Thread-Index: AQHPoImBUZxqX6tP/0aByQuTV1j2BJuh2KXA Date: Wed, 16 Jul 2014 00:47:50 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:49:00 -0000 82598, 82599, X520, and X540 can have up to 64 Tx/Rx queues (1 per CPU), bu= t are limited to 16 RSS queues.=20 For igb it varies, depending on HW. I can put together a list. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Ryan Stone Sent: Tuesday, July 15, 2014 5:04 PM To: Adrian Chadd Cc: FreeBSD Net; freebsd-arch@freebsd.org Subject: Re: ixgbe and igb - how many queues? The oldest hardware supported by the ixgbe driver is the 82598, which supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). I believe that the 82599 and X520 are more capable. I have no idea what the situation is on igb, as there seems to be a lot more hardware supported by that driver and I don't work with the 1GB devices very much. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 00:49:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E81286CF; Wed, 16 Jul 2014 00:49:53 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 973E6282D; Wed, 16 Jul 2014 00:49:53 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id r5so151123qcx.13 for ; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=RJl4Z7zxfRGoKJEp77aVtTR8FOpfhxfNaJjL8cHrifY=; b=asztDofMTuA8FYTRMNjZEoUwuGrrUg3wMnQLMHhWbJYZdGEwy3D+q3aA0b3AjCBG29 e0rWUBkA4jT9l7eJxxV99yyMszEp+VjF8lv/AzTIGCnVN5tygBd+cI9osVH976Qh0lmF J+moL4aipCmpIxSfHw04cFNeV/tqXZaSDzhdGTWdCILlvmAIcQRAwEYULstlc/j1Ei4W lfyT0A9nNEwRyeyINRIP3gCOlsl+i7ELxn3wk16UpdQUnhVo+9WCtYnSWXTb1j7pNEpO got7WguhLWpkjibj4lQEej1UxjuGVjaZMcyobR54iMQMeyXDbpUUBHAJ/2klMcyuz5jj yG3Q== MIME-Version: 1.0 X-Received: by 10.224.130.201 with SMTP id u9mr37977348qas.98.1405471792714; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> References: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> Date: Tue, 15 Jul 2014 17:49:52 -0700 X-Google-Sender-Auth: -MmkgzV3fMZQnmcQeSoRMjvwO-Q Message-ID: Subject: Re: ixgbe and igb - how many queues? From: Adrian Chadd To: "Pieper, Jeffrey E" Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:49:54 -0000 On 15 July 2014 17:47, Pieper, Jeffrey E wrote: > 82598, 82599, X520, and X540 can have up to 64 Tx/Rx queues (1 per CPU), but are limited to 16 RSS queues. > For igb it varies, depending on HW. I can put together a list. That'd be great, thanks! -a From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 01:01:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 300359A3; Wed, 16 Jul 2014 01:01:42 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx1.freebsd.org (Postfix) with ESMTP id 0AEFD2988; Wed, 16 Jul 2014 01:01:41 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 15 Jul 2014 18:01:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,668,1400050800"; d="scan'208";a="562368945" Received: from orsmsx109.amr.corp.intel.com ([10.22.240.7]) by fmsmga001.fm.intel.com with ESMTP; 15 Jul 2014 18:01:34 -0700 Received: from orsmsx153.amr.corp.intel.com (10.22.226.247) by ORSMSX109.amr.corp.intel.com (10.22.240.7) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 15 Jul 2014 18:01:34 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.5]) by ORSMSX153.amr.corp.intel.com ([169.254.12.126]) with mapi id 14.03.0123.003; Tue, 15 Jul 2014 18:01:34 -0700 From: "Pieper, Jeffrey E" To: Hooman Fazaeli , Adrian Chadd Subject: RE: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Topic: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Index: AQHPn8XtuhYKXvJgqkGKs4ElIQPwu5uhTKMAgACVKiA= Date: Wed, 16 Jul 2014 01:01:33 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> References: <53C4EE00.5090705@gmail.com> In-Reply-To: <53C4EE00.5090705@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:01:42 -0000 I believe the Linux driver DOES however support UDP RSS port hash. It is us= ed for environments where you can ensure there is no fragmentation. In such= use cases, this allows for a significant performance benefit. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Hooman Fazaeli Sent: Tuesday, July 15, 2014 2:02 AM To: Adrian Chadd Cc: FreeBSD Net; freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with ha= rdware hashing On 7/15/2014 5:14 AM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >Doesn't the problem applies to TCP too? >TCP may be fragmented too but is less likely because of MSS. > >--=20 > >Best regards. >Hooman Fazaeli _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 01:24:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53ED5E7D for ; Wed, 16 Jul 2014 01:24:14 +0000 (UTC) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 180852B3F for ; Wed, 16 Jul 2014 01:24:13 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so232899oag.41 for ; Tue, 15 Jul 2014 18:24:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=Zd6oBsPkKN0eAXNPakUamXFBkz4ZPZ10fNGtT2UFKLM=; b=gAv5izwE8fQY/NrpXu7QxHL9KXKEdGZmiGpxQJ19yLBIpDMYoXBNi6KC2yS0mJoODR nHHYgF6s9+tgL+2NhYX98HjilRnFEHh49kP8IcbdrFXjsUUils3VRB2mL3cOl12lJhhR W8vkJE02uWNnRm6MoOZx4Z+kQ4fOgjgHZIQh1xirqQIj3xBnATK/r1BdRoNPQ/s+nY1v UcWTXALengmX6BoDyII0lXEOhYw4fz2VCBDkDX5Dp9VrbMSCGBCxFc3G2isToA4iEGGX yfDNUOVvvubRk+wiZ59Odexq9Q33+wXN+sEnI6r+kHViHB4Wr0k8Q+cGzrSt6BVz/Fz2 H3yw== X-Gm-Message-State: ALoCoQlhcU11CQnOaJvCDoSOD1GA1l2u9zrp7uhe92MpBhejOOWQp3b+9H1FkxLWNTMXSACIMJGS X-Received: by 10.182.142.42 with SMTP id rt10mr15504669obb.80.1405473852309; Tue, 15 Jul 2014 18:24:12 -0700 (PDT) Received: from [172.21.0.13] (65-36-83-120.static.grandenetworks.net. [65.36.83.120]) by mx.google.com with ESMTPSA id v8sm32900930obo.7.2014.07.15.18.24.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jul 2014 18:24:11 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <09C05822-716B-4D78-A47E-7ADE4226546D@netgate.com> X-Mailer: iPhone Mail (11D257) From: Jim Thompson Subject: Re: ixgbe and igb - how many queues? Date: Tue, 15 Jul 2014 20:24:11 -0500 To: Ryan Stone Cc: FreeBSD Net , Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:24:14 -0000 But only 8 per VF. -- Jim > On Jul 15, 2014, at 19:04, Ryan Stone wrote: > > The oldest hardware supported by the ixgbe driver is the 82598, which > supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). > I believe that the 82599 and X520 are more capable. > > I have no idea what the situation is on igb, as there seems to be a > lot more hardware supported by that driver and I don't work with the > 1GB devices very much. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 01:41:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD6961AB; Wed, 16 Jul 2014 01:41:35 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE4F2C28; Wed, 16 Jul 2014 01:41:35 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id l6so184643qcy.31 for ; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=qVdL5AYYcrH6sI1Xlk8UYZfFh/iXccnGKUmLW64pG+M=; b=Aw0zb2pE8IN+wuCSGYcCZZFztuIUW1XAz+qPs2KybgCVMkRoYuDHepUwprmENRaE1j DXDHaqW4TlmlDi7yCGwknpFJ36gPfVLcVwXvp36y4x22q8zTFH0DCPavgQMTkXWazEZn SaToyq3eh9zk1GcmqnY10O1MpmKjlAqswVFdzhxgsX4DHZ0Mw4eIt0xElzb1Xb+sPV5W bkAnMlMIEhU/JaWFuMnLoHYKX/swnwt/5Wj8U2mgL/yW95CRZ1pTW54ZcLSWW26NwQWE ialSTViF299gRp0A7qquTJi5x2yE2ApIeLcJH3KjHTr6k4Sv1nQd3K4X/fuGmlMryoJ7 vgpw== MIME-Version: 1.0 X-Received: by 10.140.88.243 with SMTP id t106mr24164894qgd.12.1405474894742; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> References: <53C4EE00.5090705@gmail.com> <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> Date: Tue, 15 Jul 2014 18:41:34 -0700 X-Google-Sender-Auth: BA0wfbTNtPsFAkkl6s4E4QBbpkk Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: "Pieper, Jeffrey E" Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Hooman Fazaeli , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:41:36 -0000 On 15 July 2014 18:01, Pieper, Jeffrey E wrote: > I believe the Linux driver DOES however support UDP RSS port hash. It is used for environments where you can ensure there is no fragmentation. In such use cases, this allows for a significant performance benefit. > Right. For my RSS hacking, I've added a new in_rss.c function that returns the currently supported hashes, so it'll be possible for an administrator to define whether to support UDP hashing on 4-tuple or 2-tuple. I plan on adding a software hash calculation function to optionally calculate the 2-tuple or 4-tuple RSS hash as appropriately required so it can be run after IP reassembly if it's required. So, if the admin wants a 4-tuple UDP hash and it receives fragmented UDP, then it will re-assemble, calculate 4-tuple and kick it up. If they want 2-tuple, then it'll only calculate a software hash if it wasn't given one by the driver. Argh, so many weird corner cases to handle and I haven't even started hacking on the memcached/varnish stuff :( Thanks! -a From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 07:01:12 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4DFF4AF for ; Wed, 16 Jul 2014 07:01:12 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C76225DF for ; Wed, 16 Jul 2014 07:01:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6G71CNq088607 for ; Wed, 16 Jul 2014 07:01:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 16 Jul 2014 07:01:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-BETA3 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 07:01:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #17 from borjam@sarenet.es --- Created attachment 144718 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=144718&action=edit 10-STABLE patch for the oce driver by Stefano Garzarella -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 07:03:37 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 646AD613 for ; Wed, 16 Jul 2014 07:03:37 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C7BE2621 for ; Wed, 16 Jul 2014 07:03:37 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6G73bPQ013381 for ; Wed, 16 Jul 2014 07:03:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 183391] [oce] 10GbE networking problems with Emulex OCE 11102 CNA Date: Wed, 16 Jul 2014 07:03:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-BETA3 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: borjam@sarenet.es X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 07:03:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183391 --- Comment #18 from borjam@sarenet.es --- Luigi Rizzo and Stefano Garzarella pointed out several bugs. Notably, no locking operations in critical sections. Stefano Garzarella sent a test patch against -STABLE. With the patch I haven't been able to reproduce the panics. Stefano noted that the latest version I was trying, 747, has some missing locking statements. Relevant thread on freebsd-net: http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039148.html -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 08:58:22 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F403598 for ; Wed, 16 Jul 2014 08:58:22 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23A822F64 for ; Wed, 16 Jul 2014 08:58:22 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id c9so490470qcz.29 for ; Wed, 16 Jul 2014 01:58:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5Xox9ml0rNziRzR23OLLs6LTWnvR0G2ez05DQ8ZW+Fw=; b=ZnmB6eWZkp+lDCTeysS6jK4VJbyCUYn8VjMOHFlwby+aqbb2/VrwxE3DT5ccFapNmG 1ThDVkmYYwu4A11Sfm/g+M4bkiJsKO4igjenbkeSh+RFwad5CFQaqondclbidyenwKJk yTOIEdNyRTwqLmO3Q1PgNnNdE+UAtn3lrEPGQ06MGTodEaeg/wZTgN+GptmyE2OcdfoS /7gDJuvC6j5LU5BSsrN7gJKpM50l5BqWQEGs1dwP7/F/NG4f3Po+i8VRwa6/Da8IKUPX w46RXMqhQLMyRn7cSzmPpjpE1qR3tgT/JPs3qD9by58jVckM8bRF1fNmy9c7k2Env4xi xVOg== MIME-Version: 1.0 X-Received: by 10.140.95.101 with SMTP id h92mr43120557qge.35.1405501101227; Wed, 16 Jul 2014 01:58:21 -0700 (PDT) Received: by 10.96.73.39 with HTTP; Wed, 16 Jul 2014 01:58:21 -0700 (PDT) Date: Wed, 16 Jul 2014 01:58:21 -0700 Message-ID: Subject: UDP sendto() returning ENOBUFS - "No buffer space available" From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 08:58:22 -0000 Return values in sendto() manpage says: [ENOBUFS] The system was unable to allocate an internal buffer. The operation may succeed when buffers become avail- able. [ENOBUFS] The output queue for a network interface was full. This generally indicates that the interface has stopped sending, but may be caused by transient con- gestion. If I hit the first condition, it should reflect as failures in "netstat -m". Is that a correct assumption? I want to understand what happens when/if we hit the second condition. And how to prevent that from happening. Is it just application's job to rate-limit data it sends to the n/w interface card so that it doesn't saturate? Does kernel do any sort of queuing in the case of ENOBUFS? OR does the message just gets dropped? For an application sending a lot of UDP data and returning ENOBUFS, what all udp and other tunables I should tweak? I can only think of: - number of tx ring descriptors - increasing this will get us more txds. - kern.ipc.maxsockbuf: Increasing this will increase buffer size allocated for sockets. what else? Any comments/suggestions/corrections? cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 09:22:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFA22F3F for ; Wed, 16 Jul 2014 09:22:43 +0000 (UTC) Received: from smtp10.hushmail.com (smtp10.hushmail.com [65.39.178.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.hushmail.com", Issuer "Self-signed" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A4E802228 for ; Wed, 16 Jul 2014 09:22:42 +0000 (UTC) Received: from smtp10.hushmail.com (localhost [127.0.0.1]) by smtp10.hushmail.com (Postfix) with SMTP id 9BDB1C0144 for ; Wed, 16 Jul 2014 09:22:35 +0000 (UTC) X-hush-tls-connected: 1 Received: from smtp.hushmail.com (w7.hushmail.com [65.39.178.32]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp10.hushmail.com (Postfix) with ESMTPS for ; Wed, 16 Jul 2014 09:22:35 +0000 (UTC) Message-ID: <954bff7d9b34af15cff55470670ac70c@smtp.hushmail.com> Date: Wed, 16 Jul 2014 11:22:32 +0200 From: Jean Paul Galea User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Icedove/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: problems with ifconfig alias via rc.conf Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 09:22:43 -0000 Hello, Today I ran into a little problem with ifconfig and rc scripts. I have tried googling this issue, although all I could find, are people pointing at syntax errors. Perhaps someone here has ran into this before and could provide some pointers. Here is the relevant /etc/rc.conf settings; defaultrouter="94.247.171.193" ifconfig_igb0="up" ifconfig_igb1="up" ifconfig_igb2="up" ifconfig_igb3="up" cloned_interfaces="lagg0 lagg1" ifconfig_lagg0="laggproto failover laggport igb0 laggport igb1 94.247.171.197/32 netmask 255.255.255.240 broadcast 94.247.171.207" #ifconfig_lagg0_alias0="inet 94.247.171.195 netmask 255.255.255.255" ifconfig_lagg1="laggproto failover laggport igb2 laggport igb3 10.0.0.53/32 netmask 255.255.255.0" #ifconfig_lagg1_alias0="inet 10.0.0.150 netmask 255.255.255.255" static_routes="vpn" route_vpn="-net 10.0.8.0/22 10.0.0.1" In short, - we have four "real" igb network cards. - they are paired into lagg0, lagg1 fail-over mode. - each lagg interface has an alias which by default is disabled. - static route for vpn traffic. Today we were doing some maintenance and wanted to enable the aliases. So after removing the commented out lines from rc.conf, we ran; ~# service netif restart ~# service routing restart ~# service pf restart Output from `ifconfig` showed the aliases where listening on their respective interfaces. However, ping/telnet on 94.247.171.195 failed. At some point we decided to manually cycle the alias; ~# ifconfig lagg0 -alias 94.247.171.195 netmask 255.255.255.255 ~# ifconfig lagg0 alias 94.247.171.195 netmask 255.255.255.255 Which then worked. Output from `ifconfig` was just like before. Some info; - we are running FreeBSD 9.2-RELEASE-p10. - in dmesg we had "ifa_del_loopback_route: deletion failed". - interestingly enough, the other 10.0.0.150 alias worked just fine. Perhaps something is mis-configured in /etc/rc.conf? Some argument is missing in ifconfig_* variables ? Thanks in advance, Jean Paul From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 10:59:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06C59B79 for ; Wed, 16 Jul 2014 10:59:37 +0000 (UTC) Received: from man.dat.pl (dat.pl [80.51.155.34]) by mx1.freebsd.org (Postfix) with ESMTP id BEAB82A1C for ; Wed, 16 Jul 2014 10:59:36 +0000 (UTC) Received: from man.dat.pl (localhost [127.0.0.1]) by man.dat.pl (Postfix) with ESMTP id 133E0CEF678; Wed, 16 Jul 2014 12:52:02 +0200 (CEST) X-Virus-Scanned: amavisd-new at dat.pl Received: from man.dat.pl ([127.0.0.1]) by man.dat.pl (man.dat.pl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id mSkidYV6v61o; Wed, 16 Jul 2014 12:52:01 +0200 (CEST) Message-ID: <53C65951.3070000@dat.pl> Date: Wed, 16 Jul 2014 12:52:01 +0200 From: Maciej Milewski MIME-Version: 1.0 To: Jean Paul Galea , freebsd-net@freebsd.org Subject: Re: problems with ifconfig alias via rc.conf References: <954bff7d9b34af15cff55470670ac70c@smtp.hushmail.com> In-Reply-To: <954bff7d9b34af15cff55470670ac70c@smtp.hushmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 10:59:37 -0000 On 16.07.2014 11:22, Jean Paul Galea wrote: > ifconfig_lagg0="laggproto failover laggport igb0 laggport igb1 > 94.247.171.197/32 netmask 255.255.255.240 broadcast 94.247.171.207" > #ifconfig_lagg0_alias0="inet 94.247.171.195 netmask 255.255.255.255" Double mask definition? You are trying to define 94.247.171.197 with mask 32 and then with mask 28. Remove the /32 from this line and use simple 94.247.171.197 netmask 255.255.255.240 broadcast 94.247.171.207 Alias is defined correctly. > ifconfig_lagg1="laggproto failover laggport igb2 laggport igb3 > 10.0.0.53/32 netmask 255.255.255.0" The same as above. Better use 10.0.0.53/24 or 10.0.0.53 netmask 255.255.255.0 -- Pozdrawiam, Maciej Milewski From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 11:49:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F541374 for ; Wed, 16 Jul 2014 11:49:28 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCCE62E59 for ; Wed, 16 Jul 2014 11:49:27 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id hz20so554078lab.36 for ; Wed, 16 Jul 2014 04:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=acXpvFRd8QAY/UTT9KZcL+l0ecVgG5Nr8ToUT656Eac=; b=bp4ozRVbz7HEP8RlAlLi0kYvPyjQQSbltgIsqzLNSC0puQjWhjmCyLAR/2EyqlVC4d YKFWwRtERU2EIpolyuDLS0cMOGlXzKCamXCr/1b+8Bi8FrYASJadoxskRU6FlM/CK6Qm mirSdsEq2OV2ZzgTLiZ8Qck0KEoqxMIwng3XLwgL42tpSytcZN5RVg3tbJ8Bd1awHreM ePomOV6E3qDNqNWhCwPLvePw/HfAY4yIcKuJLGp/u7jfMilANmrrJ6CfKZKeaqGon+MA w6VmXU+O8A3E+3EI1DFxAubWhF20Vse0vgQ+QRFx3mIb8KdIz02/88sXnQDEBd/z8NvZ NBzg== X-Received: by 10.152.206.105 with SMTP id ln9mr25352924lac.45.1405511365791; Wed, 16 Jul 2014 04:49:25 -0700 (PDT) Received: from 95.108.173.189-red.dhcp.yndx.net (95.108.173.189-red.dhcp.yndx.net. [95.108.173.189]) by mx.google.com with ESMTPSA id as6sm24354985lbc.31.2014.07.16.04.49.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Jul 2014 04:49:25 -0700 (PDT) From: Dmitry Sivachenko Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: does setsockopt(SO_RCVTIMEO) work? Message-Id: Date: Wed, 16 Jul 2014 15:49:23 +0400 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 11:49:28 -0000 Hello! I am having trouble using {g,s}etsockopt(SO_RCVTIMEO). Consider the = following small test program. I expect to retrieve the value of 1 second via getsockopt call, I expect = the following output: tv_sec=3D1, tv_usec=3D0 But I actually get tv_sec=3D0, tv_usec=3D0 What am I missing?=20 Thanks! PS: on Linux it works as I expect. #include #include #include #include #include int main() { struct timeval tv; int fd; tv.tv_sec=3D1; tv.tv_usec=3D0; fd =3D socket(PF_UNIX, SOCK_STREAM, 0); if (fd < 0) err(1, "socket"); socklen_t len =3D sizeof(struct timeval); if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, len)) err(1, "setsockopt"); if (getsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, &len)) err(1, "getsockopt"); printf("tv_sec=3D%ld, tv_usec=3D%ld\n", tv.tv_sec, tv.tv_usec); } From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 12:51:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C190CB0 for ; Wed, 16 Jul 2014 12:51:35 +0000 (UTC) Received: from smtp5.hushmail.com (smtp5.hushmail.com [65.39.178.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.hushmail.com", Issuer "Self-signed" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D961124B5 for ; Wed, 16 Jul 2014 12:51:34 +0000 (UTC) Received: from smtp5.hushmail.com (localhost [127.0.0.1]) by smtp5.hushmail.com (Postfix) with SMTP id 3EF0B60297 for ; Wed, 16 Jul 2014 12:20:52 +0000 (UTC) X-hush-tls-connected: 1 Received: from smtp.hushmail.com (w9.hushmail.com [65.39.178.29]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp5.hushmail.com (Postfix) with ESMTPS; Wed, 16 Jul 2014 12:20:51 +0000 (UTC) Message-ID: <8bb2a8b5ea3cc781bd154c0650fdbf6e@smtp.hushmail.com> Date: Wed, 16 Jul 2014 14:20:48 +0200 From: Jean Paul Galea User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Icedove/24.6.0 MIME-Version: 1.0 To: Maciej Milewski , freebsd-net@freebsd.org Subject: Re: problems with ifconfig alias via rc.conf References: <954bff7d9b34af15cff55470670ac70c@smtp.hushmail.com> <53C65951.3070000@dat.pl> In-Reply-To: <53C65951.3070000@dat.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 12:51:35 -0000 On 07/16/2014 12:52 PM, Maciej Milewski wrote: > Double mask definition? > You are trying to define 94.247.171.197 with mask 32 and then with mask 28. > Remove the /32 from this line and use simple 94.247.171.197 netmask > 255.255.255.240 broadcast 94.247.171.207 > Alias is defined correctly. > The same as above. > Better use 10.0.0.53/24 or 10.0.0.53 netmask 255.255.255.0 I can't believe it was such a stupid mistake from my end :-) I guess you could say I had a syntax error ;-) Sorry for the trouble, and thank you for your time! Jean Paul From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 13:24:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 761E1742 for ; Wed, 16 Jul 2014 13:24:30 +0000 (UTC) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 20330281B for ; Wed, 16 Jul 2014 13:24:29 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id 3E5DC1A2A81; Wed, 16 Jul 2014 23:24:21 +1000 (EST) Date: Wed, 16 Jul 2014 23:24:20 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Dmitry Sivachenko Subject: Re: does setsockopt(SO_RCVTIMEO) work? In-Reply-To: Message-ID: <20140716223814.O2597@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=B4eAjodM c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=4dXZpFGpyrYA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=MpIYFU-AUWHGpaUdnW4A:9 a=CjuIK1q_8ugA:10 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 13:24:30 -0000 On Wed, 16 Jul 2014, Dmitry Sivachenko wrote: > I am having trouble using {g,s}etsockopt(SO_RCVTIMEO). Consider the following small test program. > I expect to retrieve the value of 1 second via getsockopt call, I expect the following output: > tv_sec=1, tv_usec=0 > But I actually get > tv_sec=0, tv_usec=0 > > What am I missing? > > Thanks! > > PS: on Linux it works as I expect. On old versions of FreeBSD it works as expected. This was broken last year. I pointed out the bug and many others in my review, but it is still there. From the review: @ On Sun, 1 Sep 2013, Davide Italiano wrote: @ @ > Log: @ > Fix socket buffer timeouts precision using the new sbintime_t KPI instead @ > of relying on the tvtohz() workaround. The latter has been introduced @ > lately by jhb@ (r254699) in order to have a fix that can be backported @ > to STABLE. @ > @ > Reported by: Vitja Makarov @ > Reviewed by: jhb (earlier version) @ @ This reintroduces overflow bugs that were fixed by using tvtohz(). tvtohz() @ clamps the value to INT_MAX instead of overflowing, but tvtosbt() just @ overflows. These bugs are small (they are are only for timevals larger than INT_MAX seconds, which can only occur on systems with bloated 64-bit time_t's), but they were fixed long ago. @ @ This gives much larger overflow bugs in getsockopt(). This bug is still there. So in your test program, the timeout is set correctly, but it is returned as 0 after overflow of a large value gives 0. All values larger than 1 second are truncated to less than 1 second. I expected more garbage in the values for timevals between 0.5 seconds (inclusive) and 1.0 seconds (not inclusive). I think they would happen for the corresonding bug in setsockopt(), but in getsockopt() there is only a small (but annoying) rounding error. I tried a timeval of 0 seconds, 500000 microseconds. This was returned as 0 seconds, 499999 microseconds. This rounding error happens because almost no values in the power-of-10 time scale for timevals can be represented in the power-of-2 time scale for sbintimes. The rounding is always down, so conversion back and forth drops 1 ULP in almost all cases. bintime gives similar errors. sys/time.h has a large comment defending this bug. @ @ > ... @ > Modified: head/sys/kern/uipc_socket.c @ > ============================================================================== @ > --- head/sys/kern/uipc_socket.c Sun Sep 1 23:06:28 2013 (r255137) @ > +++ head/sys/kern/uipc_socket.c Sun Sep 1 23:34:53 2013 (r255138) @ > @@ -2541,7 +2541,7 @@ sosetopt(struct socket *so, struct socko @ > int error, optval; @ > struct linger l; @ > struct timeval tv; @ > - u_long val; @ > + sbintime_t val; @ > uint32_t val32; @ > #ifdef MAC @ > struct mac extmac; @ @ This fixes a style bug (type error) that should have been fixed in the @ previous commit. 'int optval' is used for most options, but the old @ code needed `u_long val' for its home made conversion. u_long became @ unnecessary and the extra variable became bogus when tvtohz() was used, @ since optval can hold tvtohz(). @ @ > @@ -2703,7 +2703,7 @@ sosetopt(struct socket *so, struct socko @ > error = EDOM; @ > goto bad; @ > } @ @ There used to be more range checking above here. Some was moved into @ tvtohz(). Changing to tvtosbt() moves it to /dev/null. @ @ > - val = tvtohz(&tv); @ > + val = tvtosbt(tv); @ > @ > switch (sopt->sopt_name) { @ > case SO_SNDTIMEO: @ @ optval can't hold tvtosbt(), so an extra variable is not a large style @ bug now. But it can be avoided here by doing 2 assignments of tvtosbt() @ to so_snd/rcv.sb_timeout. 'val' _can_ hold tvtosbt() (it was enlarged to do this), so there is only a far-off overflow bug here (when tvtosbt() overflows internally)). @ @ > @@ -2857,8 +2857,7 @@ integer: @ > optval = (sopt->sopt_name == SO_SNDTIMEO ? @ > so->so_snd.sb_timeo : so->so_rcv.sb_timeo); @ @ This is now very broken. optval is only int, so it can't hold the 64-bit @ sbintime_t. I think it can hold times less than 0.5 seconds. Overflow @ starts at 0.5 seconds by giving sign extension bugs on 2's complement machines. @ @ Style bug: non-KNF indentation. This still has all its bugs and style bugs. @ @ > @ @ Style bug: extra blank line separates related code. @ @ > - tv.tv_sec = optval / hz; @ > - tv.tv_usec = (optval % hz) * tick; @ > + tv = sbttotv(optval); @ @ This together with the style can be fixed by 2 assignments also (1 @ assignment in the conditional operater also fixes verbosenes): @ @ tv = sbttotv(sopt->sopt_name == SO_SNDTIMEO ? @ so->so_snd.sb_timeo : so->so_rcv.sb_timeo); Fix for the main bug some style bugs. [... 200 more lines with further duscussion of style and overflow bugs] Bruce From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 15:32:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D488A2EE for ; Wed, 16 Jul 2014 15:32:37 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98A13249A for ; Wed, 16 Jul 2014 15:32:37 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id ij19so1949357vcb.25 for ; Wed, 16 Jul 2014 08:32:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=vmhzBhnB2jaYXX5OaL64Y2Hc9r9g+Gpv7oEa6a+qM8U=; b=JQ/k6P9mcbiX6pA8TNN9RQoWfbiyVgF7TsihGXNXeDiyHNI9SEjX7BetDe5pgXyFuy kfTkhf1Xwd9E5hc4OidIb7HmDIar98P23DCOKpz9IfQd/ys1DLlDEYdaZB0uHH883HuL /SLwN4x7KawN1tk/SKgZ81ucLLwSlA69Q/C71iHjigJnzJmDplRs3lC747HXt01ZEf/0 m1N0rkrklUelqyBlz/5FF6UDXJFtLnLyatISdnWoj8yepQXDZumqzpDhsTjljHQpJzdl m2OZ9yQNzc0AkqJseEIfipul2aeFVVutI8d26sm3GE2xTPgEy3H/FK7Wm6n8P6iHA1AL Mdbg== MIME-Version: 1.0 X-Received: by 10.220.5.138 with SMTP id 10mr3203908vcv.67.1405524756565; Wed, 16 Jul 2014 08:32:36 -0700 (PDT) Received: by 10.221.0.147 with HTTP; Wed, 16 Jul 2014 08:32:36 -0700 (PDT) Date: Wed, 16 Jul 2014 17:32:36 +0200 Message-ID: Subject: nic performance: bce vs em From: Cristiano Deana To: FreeBSD net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 15:32:37 -0000 Hi all, I'm gonna setup a Dell 1955 to use as router. Dual xeon dual core, with these network cards: # pciconf -lv | grep -B2 network vendor = 'Broadcom Corporation' device = 'NetXtreme II BCM5708S Gigabit Ethernet' class = network -- vendor = 'Intel Corporation' device = '82546GB Gigabit Ethernet Controller' class = network -- vendor = 'Intel Corporation' device = '82546GB Gigabit Ethernet Controller' class = network -- vendor = 'Broadcom Corporation' device = 'NetXtreme II BCM5708S Gigabit Ethernet' class = network Now I have a similar machine, with only one CPU and using vlan based interfaces. A week ago I started collect flow data from interfaces with ng_flow and I have a high CPU usage: CPU 0: 0.0% user, 0.0% nice, 0.0% system, 21.7% interrupt, 78.3% idle CPU 1: 0.0% user, 0.0% nice, 1.6% system, 23.6% interrupt, 74.8% idle This is right now, with low traffic, so I'm gonna upgrade with new system Any experience about differences between bce and em? Load, irq... In peek hours we have 80k pps and 500Mbit per interface. Thank you -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/ From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 16:48:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8237469 for ; Wed, 16 Jul 2014 16:48:45 +0000 (UTC) Received: from frv199.fwdcdn.com (frv199.fwdcdn.com [212.42.77.199]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63D2A2B1D for ; Wed, 16 Jul 2014 16:48:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=2QKZqjbv5cWo1EgFrKKC0i3onHDlXPR7VqDJLQ3KNVs=; b=m+0CIhkFUZe5Dp7WutUhTgDIIOy9U4czd8kz434RO5IxhCWFUYdToHj/qHPjDidAkCT8wuVB09WYS1AFDUAPNhg8YT8hNzzzsbBrR8T8SkGSNsu3hzaV3wf0qECAXj2kp5g4RQa7E1XG2rynQ7gFykvklouqB2WqPljSDfuC5j4=; Received: from [10.10.10.35] (helo=frv35.fwdcdn.com) by frv199.fwdcdn.com with smtp ID 1X7SNd-000HhT-Re for freebsd-net@freebsd.org; Wed, 16 Jul 2014 19:48:33 +0300 Date: Wed, 16 Jul 2014 19:48:33 +0300 From: Vladislav Prodan Subject: Re: nic performance: bce vs em To: Cristiano Deana X-Mailer: mail.ukr.net 5.0 Message-Id: <1405529290.276499188.5r7c4x15@frv35.fwdcdn.com> In-Reply-To: References: MIME-Version: 1.0 Received: from universite@ukr.net by frv35.fwdcdn.com; Wed, 16 Jul 2014 19:48:33 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: FreeBSD net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 16:48:45 -0000 --- Original message --- From: "Cristiano Deana" Date: 16 July 2014, 18:32:44 > Hi all, > > I'm gonna setup a Dell 1955 to use as router. Dual xeon dual core, > with these network cards: > > # pciconf -lv | grep -B2 network > vendor = 'Broadcom Corporation' > device = 'NetXtreme II BCM5708S Gigabit Ethernet' > class = network > -- > vendor = 'Intel Corporation' > device = '82546GB Gigabit Ethernet Controller' > class = network > -- > vendor = 'Intel Corporation' > device = '82546GB Gigabit Ethernet Controller' > class = network > -- > vendor = 'Broadcom Corporation' > device = 'NetXtreme II BCM5708S Gigabit Ethernet' > class = network > > Now I have a similar machine, with only one CPU and using vlan based interfaces. > A week ago I started collect flow data from interfaces with ng_flow > and I have a high CPU usage: > CPU 0: 0.0% user, 0.0% nice, 0.0% system, 21.7% interrupt, 78.3% idle > CPU 1: 0.0% user, 0.0% nice, 1.6% system, 23.6% interrupt, 74.8% idle > This is right now, with low traffic, so I'm gonna upgrade with new system > Try to do these steps: 1) Go to the site DELL and see update firmware for embedded NICs. 2) Get a output sysctl kern.eventtimer.choice, and if there is LAPIC, try to change eventtimer 3) Disable options RXCSUM, TXCSUM, TSO4 from network cards Broadcom: ifconfig bce0 -rxcsum -txcsum -tso .... ifconfig bce1 -rxcsum -txcsum -tso .... -- Vladislav V. Prodan System & Network Administrator support.od.ua From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 17:49:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DC709E7 for ; Wed, 16 Jul 2014 17:49:15 +0000 (UTC) Received: from a0i308.smtpcorp.com (a0i308.smtpcorp.com [216.22.15.140]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46E9020E5 for ; Wed, 16 Jul 2014 17:49:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpcorp.com; s=a0_1; h=Content-Type:MIME-Version:Message-ID:Date:Subject:To:From; bh=KBje91U65J0aG7v2c7k8uZWFxq4bq0k8bWYVQQkrEOg=; b=HmIT+kA+XRsKXKT7Q+tBu97sGWUjDM8Id+50qJiUhM2VwYkb0zkLxHCc1ef/7Sqhxlnx9EMzztcB0EdbulgAHHTTGl3XrLh7wN1BDiaIrHvNNvalWtOKQUv9E3bReHuGXEbNegNejE2g2midku372EeZMMt49hlZFGfqnVUhxbo=; From: Daniel Corbe To: freebsd-net@freebsd.org Subject: netmap, selective processing. Date: Wed, 16 Jul 2014 13:48:58 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain X-Smtpcorp-Track: 1b7TKF4gfGnzGx.xrsSmHQbq X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 17:49:15 -0000 I hope this it the right place to ask questions about netmap. I'm toying with the idea of writing a netmap-based OSPF implementation because bird's OSPF implementation isn't as good as its BGP implementation, quagga doesn't scale well and openospfd doesn't compile on 10-RELEASE or CURRENT. But I'm only interested in selectively processing packets on the netmap-enabled interface. Is there a way to do this? Or alternatively if I throw the IF into netmap mode, can I process what I'm interested in processing and then somehow throw the rest of the traffic back up to the host's IP stack? From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 18:00:50 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90AA3D03 for ; Wed, 16 Jul 2014 18:00:50 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 526DF21E7 for ; Wed, 16 Jul 2014 18:00:50 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id w7so1092124qcr.40 for ; Wed, 16 Jul 2014 11:00:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=xZCG6GRKAfStMP/0X6FOunvhiQYfMrrKyOwvYA8Ad38=; b=zCDEVosU0Affo/+pzuJP8elRND6uyznsNxBEjwiZ0WTfMnvoVxLPb3RTNj6MaNiTFO po5Zo61GFA8cbmMcQ63HITOE+UD45rtw20s8iV0ugeDD8tO/PxRf9jwT1Vi10rxu8fcd x3E6lyB1qLhR2GL0GgC5M8/VnG9D0VniGQHP7gaGASOIoenY6fPHIImpVF08BRBZrTCh BG8YTa3PoLLGbDeASGPD04yOJcGwclSCu5tX77IFP56zzL8+n3nlzOvMrvrDkaj04OTy 2jpvR+eZB98VKb4DEUWWleH7abDtL59pP2Kw7zureFAUq3v25+tQ+wWMGEwgk65kpe5K Z52w== MIME-Version: 1.0 X-Received: by 10.224.171.197 with SMTP id i5mr48707672qaz.55.1405533648737; Wed, 16 Jul 2014 11:00:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Wed, 16 Jul 2014 11:00:48 -0700 (PDT) In-Reply-To: References: Date: Wed, 16 Jul 2014 11:00:48 -0700 X-Google-Sender-Auth: 3MSHRCDzJ0oArz-IxtNMsPsMGpo Message-ID: Subject: Re: UDP sendto() returning ENOBUFS - "No buffer space available" From: Adrian Chadd To: hiren panchasara Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 18:00:50 -0000 Hi! So the UDP transmit path is udp_usrreqs->pru_send() == udp_send() -> udp_output() -> ip_output() udp_output() does do a M_PREPEND() which can return ENOBUFS. ip_output can also return ENOBUFS. it doesn't look like the socket code (eg sosend_dgram()) is doing any buffering - it's just copying the frame and stuffing it up to the driver. No queuing involved before the NIC. So a _well behaved_ driver will return ENOBUFS _and_ not queue the frame. However, it's entirely plausible that the driver isn't well behaved - the intel drivers screwed up here and there with transmit queue and failure to queue vs failure to transmit. So yeah, try tweaking the tx ring descriptor for the driver your'e using and see how big a bufring it's allocating. -a On 16 July 2014 01:58, hiren panchasara wrote: > Return values in sendto() manpage says: > > [ENOBUFS] The system was unable to allocate an internal buffer. > The operation may succeed when buffers become avail- > able. > > [ENOBUFS] The output queue for a network interface was full. > This generally indicates that the interface has > stopped sending, but may be caused by transient con- > gestion. > > If I hit the first condition, it should reflect as failures in > "netstat -m". Is that a correct assumption? > > I want to understand what happens when/if we hit the second condition. > And how to prevent that from happening. > Is it just application's job to rate-limit data it sends to the n/w > interface card so that it doesn't saturate? > Does kernel do any sort of queuing in the case of ENOBUFS? OR does the > message just gets dropped? > > For an application sending a lot of UDP data and returning ENOBUFS, > what all udp and other tunables I should tweak? I can only think of: > - number of tx ring descriptors - increasing this will get us more txds. > - kern.ipc.maxsockbuf: Increasing this will increase buffer size > allocated for sockets. > > what else? > > Any comments/suggestions/corrections? > > cheers, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 18:02:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32223E99 for ; Wed, 16 Jul 2014 18:02:45 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E714A2278 for ; Wed, 16 Jul 2014 18:02:44 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id i50so1058637qgf.6 for ; Wed, 16 Jul 2014 11:02:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9ZDt8s0yjbvacPJWvZRgTfUF+59NoBVEleC/GH18NAU=; b=tehMPb9p9QefcSaQz/ORdv8esShZQGe3HkKTYNC2Js3RZ6gGeIA7a/uzKrNX7RxGyy FedTJDO625IQUhzBhVERIvxkR/EI+3kECI7a3nIRZLGCFUEbVYGhQ8BSRmx0T0y+EOa2 qf9dwHgX1RDV/fHFfS6hPx0GeadmA+FEiqorMUbYaIK1yfsuYFUBMDREQ+JR4whOszOY udtUUCdjl8DrThi96HmxQK5VTwYf5ZyO7t5TuglF5RqHuzYXa+TP6oQBEzzGGSjM6UHX ZFUggfT3KJewj11klmT1P61KoXRWzKDsDVbIzJksZpWZUgvqVDQaYFO4swSAmWZdnrzm pSHQ== MIME-Version: 1.0 X-Received: by 10.140.47.80 with SMTP id l74mr24579835qga.24.1405533763742; Wed, 16 Jul 2014 11:02:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Wed, 16 Jul 2014 11:02:43 -0700 (PDT) In-Reply-To: <20140716223814.O2597@besplex.bde.org> References: <20140716223814.O2597@besplex.bde.org> Date: Wed, 16 Jul 2014 11:02:43 -0700 X-Google-Sender-Auth: 8rJuvwkkdW8bZ05aQRZB4BOYhxk Message-ID: Subject: Re: does setsockopt(SO_RCVTIMEO) work? From: Adrian Chadd To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Dmitry Sivachenko X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 18:02:45 -0000 I'm about to bump into this - would you be able to put together a patch to address these issues? That way I can also test it out with the UDP stuff I'm working on and get it into the tree. Thanks, -a On 16 July 2014 06:24, Bruce Evans wrote: > On Wed, 16 Jul 2014, Dmitry Sivachenko wrote: > >> I am having trouble using {g,s}etsockopt(SO_RCVTIMEO). Consider the >> following small test program. >> I expect to retrieve the value of 1 second via getsockopt call, I expect >> the following output: >> tv_sec=1, tv_usec=0 >> But I actually get >> tv_sec=0, tv_usec=0 >> >> What am I missing? >> >> Thanks! >> >> PS: on Linux it works as I expect. > > > On old versions of FreeBSD it works as expected. > > This was broken last year. I pointed out the bug and many others in my > review, but it is still there. From the review: > > @ On Sun, 1 Sep 2013, Davide Italiano wrote: > @ @ > Log: > @ > Fix socket buffer timeouts precision using the new sbintime_t KPI > instead > @ > of relying on the tvtohz() workaround. The latter has been introduced > @ > lately by jhb@ (r254699) in order to have a fix that can be backported > @ > to STABLE. > @ > > @ > Reported by: Vitja Makarov > @ > Reviewed by: jhb (earlier version) > @ @ This reintroduces overflow bugs that were fixed by using tvtohz(). > tvtohz() > @ clamps the value to INT_MAX instead of overflowing, but tvtosbt() just > @ overflows. > > These bugs are small (they are are only for timevals larger than INT_MAX > seconds, which can only occur on systems with bloated 64-bit time_t's), > but they were fixed long ago. > > @ @ This gives much larger overflow bugs in getsockopt(). > > This bug is still there. So in your test program, the timeout is set > correctly, but it is returned as 0 after overflow of a large value > gives 0. All values larger than 1 second are truncated to less than > 1 second. > > I expected more garbage in the values for timevals between 0.5 seconds > (inclusive) and 1.0 seconds (not inclusive). I think they would happen > for the corresonding bug in setsockopt(), but in getsockopt() there is > only a small (but annoying) rounding error. I tried a timeval of > 0 seconds, 500000 microseconds. This was returned as 0 seconds, > 499999 microseconds. This rounding error happens because almost no values > in the power-of-10 time scale for timevals can be represented in the > power-of-2 time scale for sbintimes. The rounding is always down, so > conversion back and forth drops 1 ULP in almost all cases. bintime > gives similar errors. sys/time.h has a large comment defending this > bug. > > @ @ > ... > @ > Modified: head/sys/kern/uipc_socket.c > @ > > ============================================================================== > @ > --- head/sys/kern/uipc_socket.c Sun Sep 1 23:06:28 2013 > (r255137) > @ > +++ head/sys/kern/uipc_socket.c Sun Sep 1 23:34:53 2013 > (r255138) > @ > @@ -2541,7 +2541,7 @@ sosetopt(struct socket *so, struct socko > @ > int error, optval; > @ > struct linger l; > @ > struct timeval tv; > @ > - u_long val; > @ > + sbintime_t val; > @ > uint32_t val32; > @ > #ifdef MAC > @ > struct mac extmac; > @ @ This fixes a style bug (type error) that should have been fixed in the > @ previous commit. 'int optval' is used for most options, but the old > @ code needed `u_long val' for its home made conversion. u_long became > @ unnecessary and the extra variable became bogus when tvtohz() was used, > @ since optval can hold tvtohz(). > @ @ > @@ -2703,7 +2703,7 @@ sosetopt(struct socket *so, struct socko > @ > error = EDOM; > @ > goto bad; > @ > } > @ @ There used to be more range checking above here. Some was moved into > @ tvtohz(). Changing to tvtosbt() moves it to /dev/null. > @ @ > - val = tvtohz(&tv); > @ > + val = tvtosbt(tv); > @ > > @ > switch (sopt->sopt_name) { > @ > case SO_SNDTIMEO: > @ @ optval can't hold tvtosbt(), so an extra variable is not a large style > @ bug now. But it can be avoided here by doing 2 assignments of tvtosbt() > @ to so_snd/rcv.sb_timeout. > > 'val' _can_ hold tvtosbt() (it was enlarged to do this), so there is only > a far-off overflow bug here (when tvtosbt() overflows internally)). > > @ @ > @@ -2857,8 +2857,7 @@ integer: > @ > optval = (sopt->sopt_name == SO_SNDTIMEO ? > @ > so->so_snd.sb_timeo : > so->so_rcv.sb_timeo); > @ @ This is now very broken. optval is only int, so it can't hold the > 64-bit > @ sbintime_t. I think it can hold times less than 0.5 seconds. Overflow > @ starts at 0.5 seconds by giving sign extension bugs on 2's complement > machines. > @ @ Style bug: non-KNF indentation. > > This still has all its bugs and style bugs. > > @ @ > > @ @ Style bug: extra blank line separates related code. > @ @ > - tv.tv_sec = optval / hz; > @ > - tv.tv_usec = (optval % hz) * tick; > @ > + tv = sbttotv(optval); > @ @ This together with the style can be fixed by 2 assignments also (1 > @ assignment in the conditional operater also fixes verbosenes): > @ @ tv = sbttotv(sopt->sopt_name == SO_SNDTIMEO ? > @ so->so_snd.sb_timeo : so->so_rcv.sb_timeo); > > Fix for the main bug some style bugs. > > [... 200 more lines with further duscussion of style and overflow bugs] > > Bruce > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 18:05:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB56CF84 for ; Wed, 16 Jul 2014 18:05:27 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E4B522A2 for ; Wed, 16 Jul 2014 18:05:27 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id z60so1070690qgd.33 for ; Wed, 16 Jul 2014 11:05:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EAHIacivCQLv74YBqw3plWO/KM89lYqGA7DS54VT1Q0=; b=cZLadjOeg+3hhdyC2sen2LSsdNO1r2qhJNkRXJuWzzjJybDfZIeNNfXm2a6ld6NL1J fhIpCVyXJjji+czDVOanvQx4Hdj2m/krljG+W8nHTMy3OM0wzDokLDSxKL4JDHrS39dx iUTaOacvC2PzxR89tiDDznVdv+B+dlXfvNWxU9z4cDdOKXTtKJ4cwQD4QoXgct0xTOJ0 /d5JsHdya75C1bK7uOwWDjC76dRa0DbczrjUASZ1frHzGCMvCUEemJSB4X26bCkDCUVc 3i2LvkESYhBedhiUCqeFgdGKh/sqQtfuIr+KYEQP9KZk0erWfk28iz6oGYwyD4xBeMqh 4ngg== MIME-Version: 1.0 X-Received: by 10.224.171.197 with SMTP id i5mr48754892qaz.55.1405533926666; Wed, 16 Jul 2014 11:05:26 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Wed, 16 Jul 2014 11:05:26 -0700 (PDT) In-Reply-To: References: Date: Wed, 16 Jul 2014 11:05:26 -0700 X-Google-Sender-Auth: Bt3KSUkdsz-P9Fp76bQROambb0s Message-ID: Subject: Re: netmap, selective processing. From: Adrian Chadd To: Daniel Corbe Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 18:05:27 -0000 Hi, Yes. You can write some matching code to match on what you care about and reinject the rest back to the netmap "host interface" that you can create. There's a way to grab both say, em0 for netmap and the host side of em0 so you can reinject packets back up to the host stack and get them from the host stack to process. Luigi would know the details. I just stumbled across it when pecking around. :) -a On 16 July 2014 10:48, Daniel Corbe wrote: > > I hope this it the right place to ask questions about netmap. I'm > toying with the idea of writing a netmap-based OSPF implementation > because bird's OSPF implementation isn't as good as its BGP > implementation, quagga doesn't scale well and openospfd doesn't compile > on 10-RELEASE or CURRENT. > > But I'm only interested in selectively processing packets on the > netmap-enabled interface. Is there a way to do this? Or alternatively > if I throw the IF into netmap mode, can I process what I'm interested in > processing and then somehow throw the rest of the traffic back up to the > host's IP stack? > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 21:01:52 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03D58C43; Wed, 16 Jul 2014 21:01:52 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx1.freebsd.org (Postfix) with ESMTP id C5953233F; Wed, 16 Jul 2014 21:01:51 +0000 (UTC) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 16 Jul 2014 13:55:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,673,1400050800"; d="scan'208";a="544419551" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga001.jf.intel.com with ESMTP; 16 Jul 2014 14:00:40 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.231]) by ORSMSX106.amr.corp.intel.com ([169.254.5.72]) with mapi id 14.03.0123.003; Wed, 16 Jul 2014 14:00:39 -0700 From: "Pieper, Jeffrey E" To: Adrian Chadd Subject: RE: ixgbe and igb - how many queues? Thread-Topic: ixgbe and igb - how many queues? Thread-Index: AQHPoImBUZxqX6tP/0aByQuTV1j2BJuh2KXAgAB7VQCAANquwA== Date: Wed, 16 Jul 2014 21:00:39 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E909341@ORSMSX111.amr.corp.intel.com> References: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: FreeBSD Net , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 21:01:52 -0000 DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBhZHJpYW4uY2hhZGRAZ21haWwu Y29tIFttYWlsdG86YWRyaWFuLmNoYWRkQGdtYWlsLmNvbV0gT24gQmVoYWxmIE9mIEFkcmlhbiBD aGFkZA0KU2VudDogVHVlc2RheSwgSnVseSAxNSwgMjAxNCA1OjUwIFBNDQpUbzogUGllcGVyLCBK ZWZmcmV5IEUNCkNjOiBSeWFuIFN0b25lOyBGcmVlQlNEIE5ldDsgZnJlZWJzZC1hcmNoQGZyZWVi c2Qub3JnDQpTdWJqZWN0OiBSZTogaXhnYmUgYW5kIGlnYiAtIGhvdyBtYW55IHF1ZXVlcz8NCg0K T24gMTUgSnVseSAyMDE0IDE3OjQ3LCBQaWVwZXIsIEplZmZyZXkgRSA8amVmZnJleS5lLnBpZXBl ckBpbnRlbC5jb20+IHdyb3RlOg0KPiA4MjU5OCwgODI1OTksIFg1MjAsIGFuZCBYNTQwIGNhbiBo YXZlIHVwIHRvIDY0IFR4L1J4IHF1ZXVlcyAoMSBwZXIgQ1BVKSwgYnV0IGFyZSBsaW1pdGVkIHRv IDE2IFJTUyBxdWV1ZXMuDQo+IEZvciBpZ2IgaXQgdmFyaWVzLCBkZXBlbmRpbmcgb24gSFcuIEkg Y2FuIHB1dCB0b2dldGhlciBhIGxpc3QuDQo+DQo+VGhhdCdkIGJlIGdyZWF0LCB0aGFua3MhDQo+ DQo+DQo+LWENCg0KSSB3YXMgaW5jb3JyZWN0IGFib3V0IHRoZSBUeC9SeCBxdWV1ZXMgZm9yIGl4 Z2JlOg0KDQotaWdiLQ0KODI1NzUgLSAgIDQgVHgvUngsICAgNCBSU1MNCjgyNTc2IC0gMTYgVHgv UngsIDE2IFJTUw0KODI1ODAgLSAgIDggVHgvUngsICA4IFJTUw0KaTM1MCAgLSAgICAgIDggVHgv UngsICA4IFJTUw0KaTIxMCAgLSAgICAgIDQgVHgvUngsICA0IFJTUw0KaTIxMSAgLSAgICAgIDIg VHgvUngsICAyIFJTUw0KDQotaXhnYmUtDQo4MjU5OCAtICAgNjQgUngvMzIgVHgsIDE2IFJTUw0K ODI1OTkgLSAxMjggUngvVHgsICAgICAgMTYgUlNTDQpYNTQwIC0gICAxMjggUngvVHgsICAgICAg MTYgUlNTDQoNCkhvcGUgdGhpcyBoZWxwcywNCg0KSmVmZg0KDQoNCg==